Chapter 10: Joran

This chapter is outdated and needs to be re-written to account for the massive changes occurring in 1.3

Joran stands for a cold north-west wind which, every now and then, blows forcefully on Lake Geneva. Located right in the middle of Western-Europe, the surface of Lake Geneva is smaller than many other European lakes. However, with its average depth of 153 meters, it is unusually deep, and happens to be, by volume, the largest sweet water reserve in Western-Europe.

As apparent in previous chapters, logback relies on Joran, a mature, flexible and powerful configuration framework. Many of the capabilities offered by logback modules are only possible on account of Joran. This chapter focuses on Joran, its basic design and its salient features.

Joran is actually a generic configuration system which can be used independently of logging. To emphasize this point, we should mention that the logback-core module does not have a notion of loggers. In that spirit, most of the examples in this chapter have nothing to do with loggers, appenders or layouts.

The examples presented in this chapter can be found under LOGBACK_HOME/logback-examples/​src/main/java/chapters/​onJoran/ folder.

To install Joran, simply download logback and add logback-core-1.5.12.jar to your classpath.

Historical perspective

Reflection is a powerful feature of the Java language, making it possible to configure software systems declaratively. For example, many important properties of an EJB are configured with the ejb.xml file. While EJBs are written in Java, many of their properties are specified within the ejb.xml file. Similarly, logback settings can be specified in a configuration file, expressed in XML format. Annotations available in JDK 1.5 and heavily used in EJB 3.0 replace many directives previously found in XML files. Joran also makes use of annotations but at a much smaller extent. Due to the dynamic nature of logback configuration data (compared to EJBs) Joran's use of annotations is rather limited.

In log4j, logback's predecessor, the DOMConfigurator class, which is part of log4j version 1.2.x and later, could also parse configuration files written in XML. DOMConfigurator was written in a way that forced us, the developers, to tweak the code each time the structure of the configuration file changed. The modified code had to be recompiled and redeployed. Just as importantly, the code of the DOMConfigurator consisted of loops dealing with child elements containing many interspersed if/else statements. One could not help but notice that this particular code reeked of redundancy and duplication. The commons-digester project had shown us that it was possible to parse XML files using pattern matching rules. At parse time, digester would apply rules that matched designated patterns. Rule classes were usually quite small and specialized. Consequently, they were relatively easy to understand and maintain.

Armed with the DOMConfigurator experience, we began developing Joran, a powerful configuration framework to be used in logback. Joran was largely inspired by the commons-digester project. Nevertheless, it uses a slightly different terminology. In commons-digester, a rule can be seen as consisting of a pattern and a rule, as shown by the Digester.addRule(String pattern, Rule rule) method. We find it unnecessarily confusing to have a rule to consist of itself, not recursively but with a different meaning. In Joran, a rule consists of a pattern and an action. An action is invoked when a match occurs for the corresponding pattern. This relation between patterns and actions lies at the core of Joran. Quite remarkably, one can deal with quite complex requirements by using simple patterns, or more precisely with exact matches and wildcard matches.

SAX or DOM?

Due to the event-based architecture of the SAX API, a tool based on SAX cannot easily deal with forward references, that is, references to elements which are defined later than the current element being processed. Elements with cyclical references are equally problematic. More generally, the DOM API allows the user to perform searches on all the elements and make forward jumps.

This extra flexibility initially led us to choose the DOM API as the underlying parsing API for Joran. After some experimentation, it quickly became clear that dealing with jumps to distant elements while parsing the DOM tree did not make sense when the interpretation rules were expressed in the form of patterns and actions. Joran only needs to be given the elements in the XML document in a sequential, depth-first order.

Moreover, the SAX API offers element location information which allows Joran to display the exact line and column number where an error occurred. Location information comes in very handy in the identification of parsing problems.

Non goals

Given its highly dynamic nature, the Joran API is not intended to be used to parse very large XML documents with many thousands of elements.

Pattern

A Joran pattern is essentially a string. There are two kind of patterns, exact and wildcard. The pattern "a/b" can be used to match a <b> element nested within a top-level <a> element. The "a/b" pattern will not match any other element, hence the exact match designation.

Wildcards can be used to match suffixes or prefixes. For example, the "*/a" pattern can be used to match any suffix ending with "a", that is any <a> element within an XML document but not any elements nested within <a>. The "a/*" pattern will match any element prefixed by <a>, that is any element nested within an <a> element.

Actions

As mentioned above, Joran parsing rules consists of the association of patterns. Actions extend the Action class, consisting of the following abstract methods. Other methods have been omitted for brevity.

package ch.qos.logback.core.joran.action;

import org.xml.sax.Attributes;
import org.xml.sax.Locator;
import ch.qos.logback.core.​joran.spi.InterpretationContext;

public abstract class Action extends ContextAwareBase {
  /**
   * Called when the parser encounters an element matching a
   * {@link ch.qos.logback.​core.joran.spi.Pattern Pattern}.
   */
  public abstract void begin(InterpretationContext ic, String name,
      Attributes attributes) throws ActionException;

  /**
   * Called to pass the body (as text) contained within an element.
   */
  public void body(InterpretationContext ic, String body)
      throws ActionException {
    // NOP
  }

  /*
   * Called when the parser encounters an endElement event matching a
   * {@link ch.qos.logback.​core.joran.spi.Pattern Pattern}.
   */
  public abstract void end(InterpretationContext ic, String name)
      throws ActionException;
}

Thus, every action must implement the begin() and end() methods. The implementation of the body() method is optional on account of the empty/nop implementation provided by Action.

RuleStore

As mentioned previously, the invocation of actions according to matching patterns is a central concept in Joran. A rule is an association of a pattern and an action. Rules are stored in a RuleStore.

As mentioned above, Joran is built on top of the SAX API. As an XML document is parsed, each element generates events corresponding to the start, body and end of each element. When a Joran configurator receives these events, it will attempt to find in its rule store an action corresponding to the current pattern. For example, the current pattern for the start, body or end event of element B nested within a top-level A element is "A/B". The current pattern is a data structure maintained automatically by Joran as it receives and processes SAX events.

When several rules match the current pattern, then exact matches override suffix matches, and suffix matches override prefix matches. For exact details of the implementation, please see the SimpleRuleStore class.

Interpretation context

To allow various actions to collaborate, the invocation of begin and end methods include an interpretation context as the first parameter. The interpretation context includes an object stack, an object map, an error list and a reference to the Joran interpreter invoking the action. Please see the InterpretationContext class for the exact list of fields contained in the interpretation context.

Actions can collaborate together by fetching, pushing or popping objects from the common object stack, or by putting and fetching keyed objects on the common object map. Actions can report any error conditions by adding error items on the interpretation context's StatusManager.

Hello world

The first example in this chapter illustrates the minimal plumbing required for using Joran. The example consists of a trivial action called HelloWorldAction which prints "Hello World" on the console when its begin() method is invoked. The parsing of XML files is done by a configurator. For the purposes of this chapter, we have developed a very simple configurator called SimpleConfigurator. The HelloWorld application brings all these pieces together:

• It creates a map of rules and a Context
• It creates a parsing rule by associating the hello-world pattern with a corresponding HelloWorldAction instance
• It creates a SimpleConfigurator, passing it the aforementioned rules map
• It then invokes the doConfigure method of the configurator, passing the designated XML file as parameter
• As a last step, the accumulated Status message in the context, if any, are printed

The hello.xml file contains one <hello-world> element, without any other nested elements. See the logback-examples/​src/main/java/chapters/​onJoran/helloWorld/ folder for exact contents.

Running the HelloWorld application with hello.xml file will print "Hello World" on the console.

java chapters.onJoran.​helloWorld.HelloWorld src/main/java/chapters/​onJoran/helloWorld/​hello.xml

You are highly encouraged to poke about in this example, by adding new rules on the rule store, modifying the XML document (hello.xml) and adding new actions.

Collaborating actions

The logback-examples/​src/main/java/joran/​calculator/ directory includes several actions which collaborate together through the common object stack in order to accomplish simple computations.

The calculator1.xml file contains a computation element, with a nested literal element. Here are its contents.

Example 10.: First calculator example (logback-examples/​src/main/​java/chapters/​onJoran/calculator/​calculator1.xml)

<computation name="total">
  <literal value="3"/>
</computation>

In the Calculator1 application, we declare various parsing rules (patterns and actions) collaborating together to compute a result based on the contents of an XML document.

Running Calculator1 application with calculator1.xml

java chapters.onJoran.​calculator.Calculator1 src/main/java/chapters/​onJoran/calculator/​calculator1.xml

will print:

The computation named [total] resulted in the value 3

Parsing the calculator1.xml document (listed above) involves the following steps:

• The start event corresponding to the <computation> element translates into the current pattern "/computation". Since in the Calculator1 application we associated the pattern "/computation" with a ComputationAction1 instance, the begin() method of that ComputationAction1 instance is invoked.

• The start event corresponding to the <literal> element translates into the current pattern "/computation/literal". Given the association of the "/computation/literal" pattern with a LiteralAction instance, the begin() method of that LiteralAction instance is called.

• By the same token, the end event corresponding to the <literal> element triggers the invocation of the end() method of the same LiteralAction instance.

• Similarly, the event corresponding to the end of <computation> element triggers the invocation the end() method of the ComputationAction1 same instance.

What is interesting here is the way actions collaborate. The LiteralAction reads a literal value and pushes it in the object stack maintained by the InterpretationContext. Once done, any other action can pop the value to read or modify it. Here, the end() method of the ComputationAction1 class pops the value from the stack and prints it.

The next example, calculator2.xml file is a bit more complex, but also more interesting.

Example 10.: Calculator configuration file (logback-examples/​src/main/java/chapters/​onJoran/calculator/​calculator2.xml)

<computation name="toto">
  <literal value="7"/>
  <literal value="3"/>
  <add/>
  <literal value="3"/>
  <multiply/>
</computation>

As in the previous example, in response to the <literal> element,the appropriate LiteralAction instance will push an integer, corresponding to the value attribute, at the top of the interpretation context's object stack. In this example, that is calculator2.xml, the values are 7 and 3. In response to the <add> element, the appropriate AddAction will pop two previously pushed integers, compute their sum and push the result, i.e. 10 (=7+3), at the top of the interpretation context's stack. The next literal element will cause LiteralAction to push an integer with value 3 at the top of the stack. In response to the <multiply> element, the appropriate MultiplyAction will pop two previously pushed integers, i.e. 10 and 3, and compute their product. It will push the result, i.e. 30, at the top of the stack. At the very end, in response to the end event corresponding to the </computation> tag, the ComputationAction1 will print the object at the top of the stack. Thus, running:

java chapters.onJoran.​calculator.Calculator1 src/main/java/chapters/​onJoran/calculator/​calculator2.xml 

will yield

The computation named [toto] resulted in the value 30

Implicit actions

The rules defined thus far are called explicit actions because a pattern/action association could be found in the rule store for the current element. However, in highly extensible systems, the number and type of components can be so large so as to make it very tedious to associate an explicit action for all patterns.

At the same time, even in highly extensible systems one can observe recurrent rules linking various parts together. Assuming we could identify such rules, we could process components composed of subcomponents unknown at compilation time (of logback). For example, Apache Ant is capable of handling tasks which contain tags unknown at compile time, simply by inspecting the component for methods whose names start with add, as in addFile, or addClassPath. When Ant encounters an embedded tag within a task, it simply instantiates an object that matches the signature of the task class' add method and attaches the resulting object to the parent.

Joran supports a similar capability in the form of implicit actions. Joran keeps a list of implicit actions which are applied if no explicit pattern could match the current pattern. However, applying an implicit action may not be always appropriate. Before executing the implicit action, Joran asks a given implicit action whether it is appropriate in the current situation. Only if the action replies in the affirmative does the Joran configurator invoke the (implicit) action. Note that this extra step makes it possible to support multiple implicit actions or possibly none, if no implicit action is appropriate for a given situation.

You can create and register a custom implicit action as illustrated in the next example contained within the logback-examples/​src/main/java/chapters/​onJoran/implicit folder.

The PrintMe application associates an NOPAction instance with the pattern "*/foo", that is any element named as "foo". As its name indicates, the begin() and end() methods of NOPAction are empty. The PrintMe application also registers an instance of PrintMeImplicitAction in its list of implicit actions. The PrintMeImplicitAction is applicable for any element which has a printme attribute set to true. See the isApplicable() method in PrintMeImplicitAction. The begin()() method of PrintMeImplicitAction prints the name of the current element on the console.

The XML document implicit1.xml is designed to illustrate how implicit actions come into play.

Example 10.: Usage of implicit rules (logback-examples/​src/main/java/chapters/​onJoran/implicit/​implicit1.xml)

<foo>
  <xyz printme="true">
    <abc printme="true"/>
  </xyz>

  <xyz/>

  <foo printme="true"/>

</foo>

Running

java chapters.onJoran.implicit.PrintMe src/main/java/chapters/​onJoran/implicit/​implicit1.xml

yields:

Element [xyz] asked to be printed. Element [abc] asked to be printed. 20:33:43,750 |-ERROR in c.q.l.c.joran.spi.Interpreter@10:9 - no applicable action for [xyz], current pattern is [[foo][xyz]]

Given that NOPAction instance is explicitly associated with the "*/foo" pattern, NOPAction's begin() and end() methods are invoked on <foo> elements. PrintMeImplicitAction is never triggered for any of the <foo> elements. For other elements, since there are no matching explicit actions, the isApplicable() method of PrintMeImplicitAction is invoked. It will return true only for elements having a printme attribute set to true, namely the first <xyz> element (but not the second) and the <abc> element. The second <xyz> element on line 10, there are no applicable actions, an internal error message is generated. This message is printed by the StatusPrinter.print invocation, the last statement in the PrintMe application. This explains the output shown above (see previous paragraph).

Implicit actions in practice

The respective Joran configurators of logback-classic and logback-access include just two implicit actions, namely NestedBasicPropertyIA and NestedComplexPropertyIA.

NestedBasicPropertyIA is applicable for any property whose type is a primitive type (or equivalent object type in the java.lang package), an enumeration type, or any type adhering to the "valueOf" convention. Such properties are said to be basic or simple. A class is said to adhere to the "valueOf" convention if it contains a static method named valueOf() taking a java.lang.String as parameter and returning an instance of the type in question. At present, the Level, Duration and FileSize classes follow this convention.

NestedComplexPropertyIA action is applicable, in the remaining cases where NestedBasicPropertyIA is not applicable and if the object at the top of the object stack has a setter or adder method for a property name equal to the current element name. Note that such properties can in turn contain other components. Thus, such properties are said to be complex. In presence of a complex property, NestedComplexPropertyIA will instantiate the appropriate class for the nested component and attach it to the parent component (at the top of the object stack) by using the setter/adder method of the parent component and the nested element's name. The corresponding class is specified by the class attribute of the (nested) current element. However, if the class attribute is missing, the class name can be deduced implicitly, if any of the following is true:

1. there is an internal rule associating the parent object's property with a designated class
2. the setter method contains a @DefaultClass attribute designating a given class
3. the parameter type of the setter method is a concrete class possessing a public constructor

Default class mapping

In logback-classic, there are a handful of internal rules mapping parent class/property name pairs to a default class. These are listed in the table below.

This list may change in future releases. Please see logback-classic JoranConfigurator's addDefaultNested​ComponentRegistryRules method for the latest rules.

In logback-access, the rules are very similar. In the default class for the nested component, the ch.qos.logback.classic package is replaced by ch.qos.logback.access. See logback-access JoranConfigurator's addDefault​NestedComponent​RegistryRules method for the latest rules.

Collection of properties

Note that in addition to single simple properties or single complex properties, logback's implicit actions support collections of properties, be they simple or complex. Instead of a setter method, the property is specified by an "adder" method.

New rules on the fly

Joran includes an action which allows the Joran interpreter to learn new rules on the fly, that is while interpreting an XML document. See the logback-examples/​src/main/java/chapters/​onJoran/newRule/ directory for sample code. In this package, the NewRuleCalculator application sets up just two rules, one rule to process the top-most element, and a second rule to learn new rules. Here is the relevant code from NewRuleCalculator.

ruleMap.put(new Pattern("*/computation"), new ComputationAction1());
ruleStore.addRule(new Pattern("/computation/newRule"), new NewRuleAction());

NewRuleAction, part of logback-core, works pretty much like the other actions. It has a begin() and end() method, and is called each time the parser finds a newRule element. When invoked, the begin() method looks for pattern and actionClass attributes. It then instantiates the corresponding action class and adds the pattern/action association as a new rule in Joran's rule store.

Here is how new rules can be declared in an XML file:

<newRule pattern="*/computation/literal"
          actionClass="chapters.​onJoran.calculator.​LiteralAction"/>

Using such newRule declarations, we can transform NewRuleCalculator to behave like the Calculator1 application we saw earlier. involving the calculation, could be expressed this way:

Example 10..: Configuration file using new rules on the fly (logback-examples/​src/main/java/chapters/​onJoran/newrule/​newRule.xml)

<computation name="toto">
  <newRule pattern="*/computation/literal" 
            actionClass="chapters.​onJoran.calculator.​LiteralAction"/>
  <newRule pattern="*/computation/add" 
            actionClass="chapters.​onJoran.calculator.​AddAction"/>
  <newRule pattern="*/computation/multiply" 
            actionClass="chapters.​onJoran.calculator.​MultiplyAction"/​>

  <computation>
    <literal value="7"/>
    <literal value="3"/>
    <add/>
  </computation>   
 
  <literal value="3"/>
  <multiply/>
</computation>

java java chapters.onJoran.​newRule.NewRuleCalculator src/main/java/chapters/​onJoran/newRule/​newRule.xml

yields

The computation named [toto] resulted in the value 30

which is identical to the output of the original calculator example.