Java Tip 135: Layer and compare property files

Property files offer an elegant solution for storing program settings loaded at runtime. The elegance comes from the hierarchical naming scheme, the ability to override settings on the command line, and the idea of default properties. Earlier generations of settings files like .rc (read command) or .ini (initialize) files lack these features. Default properties are underutilized, however, probably because good examples are rare and no tools support comparing and manipulation.

The problem with one big property file is that developers inevitably need conflicting settings, and the settings might significantly differ for coding, testing, and deployment. The most common way to express these conflicts, or shall I say, alternatives, is by including commented-out settings. But this strategy does not work well for automated builds, and writing comments about what part should be uncommented and why requires effort. Some try to solve this problem by creating many property files that have overlapping definitions, but this leads to the developer either editing many files to keep them consistent or enduring the inconsistencies as they crop up—both of which waste time. Another common way to accommodate differing properties is to set them on the command line, but this quickly grows messy, and who wants to commit messy start-up scripts to source code control to share usage examples?

With the PropDiff utility described here, not only can variants of large property files be compared with ease, but also the layering of property files can be used as an organizing principle that makes the early stages of implementation much smoother.

Layers

Property files can be loaded by instantiating a Properties object and then using the load(InputStream inStream) method. Then the getProperty(String key) method is used to search for the value associated with the given key. When a Properties object is instantiated with the Properties(Properties defaults) constructor, the given Properties defaults object is searched only when a particular property is not found. The defaults parameter allows multilayered property files of an arbitrary depth to chain together like a linked list.

For example, two layers can load like this:

     Properties defaultProps = new Properties();
    defaultProps.load(new FileInputStream("default.props"));
    Properties props = new Properties(defaultProps);
    props.load(new FileInputStream("installation.props"));
    String value = props.get("my.setting");

Notice that default properties load first because they are needed in the next layer's constructor. Strictly speaking, empty Properties objects could be chained together and then loaded in any order, but you should pass to a constructor as much state as possible so that objects are fully formed right from the start. For clarity, imagine the property layers stacking up so that the defaults are created earlier and thus deeper than other Properties that use the defaults.

Layers in WebLogic

BEA WebLogic introduced me to the concept of using property file layers to separate settings into three levels (listed in loading order, so deepest layer is first):

• Global settings used by all application servers in a cluster
• Settings specific to a host server
• Application server instance-specific settings

The organizing principle here is reminiscent of how inheritance is used to specialize classes in object-oriented programming. Any particular WebLogic instance has its own process that listens for connections on one port and, on startup, loads one property file for each of these layers. Each layer can override any previous one for any particular setting because deeper layers provide less specific information, just like specialization in a subclass overrides the superclass. An ordered-by-precedence scheme like this is needed to separate settings into three groups: cluster, host, and instance.

While using WebLogic for development and deployment, I encountered situations where it became necessary to determine the differences among WebLogic instances to correct a misbehaving server or to bring a server up to date without wiping out particular settings it needed. The first time this situation arose, I tried to compare by hand, but eyeballing the needle in the haystack proved painfully difficult because the difference turned out to be a typo in a property name; a G in a property name should have been a C, and the particular fixed-width terminal font I was using minimized the difference between these two characters to one or two pixels.

The next time I needed to compare property files, I decided to automate the process. First I tried using the Unix diff tool on property files, but this did not successfully work because, while the order of settings within a property file is irrelevant, they show up as differences. Second, I tried concatenating and sorting the files first and then doing the diff. However, that revealed another difficulty: the layering of property files is completely lost in the results, making it hard to trace the differences to the original files. (Alternatively, comparing files layer by layer does not reveal the combined settings.) Finally, it became obvious that a small Java utility would solve this problem once and for all.

Organize properties by layer for development

We can generalize from the idea of using property file layers in a cluster to organizing properties over time and among people during development, testing, and deployment. For example, in one multiperson project I found the following layers to be useful:

1. System defaults that are infrequently changed and valid for most configurations
2. Purpose-delineated settings (coding, testing, deployment)
3. Personal developer settings for each developer, like installation location, debug/log level, and experimental development

In projects organized like this, there typically would be one property file for Layer 1, three in Layer 2, and one or two for each developer in Layer 3. At runtime, the application loads one property file from each layer. The particular property files from Layers 2 and 3 are specified on the command line as an argument or -Dproperty.

The benefits of layering

Classifying properties in this way is analogous to a stylized inheritance hierarchy. The benefits are:

• Multiple developers can maximize sharing of property files
• Layering reduces the need to comment individual property settings because classification by layer and by particular file within a layer effectively makes implied comments about the expected variability
• Compared to one big property file, layering reduces the number of source code control commits needed for any particular property file and also reduces the likelihood of merging

PropDiff utility

One problem with using multiple layers of property files is that understanding which properties are set or where they were overridden can become less than obvious. I wrote the PropDiff utility to easily solve these issues.

PropDiff is a command-line utility that can quickly find the union, intersection, and difference between property files. For instance, given two property files p1 and p2, from layer n and (n+1), and no other arguments, PropDiff creates six property files containing:

1. Property settings common to both p1 and p2, where p2 takes precedence (-c)
2. Union of p1 and p2, where p2 has higher precedence (-u)
3. Property settings only in p1 (-1)
4. Property settings only in p2 (-2)
5. Intersection of properties in p1 and p2 that have different values (-d)
6. Intersection of properties in p1 and p2 that have equal values (-e)

Using the specific command-line flags indicated in parentheses, the outputs can be selectively emitted. Each of these created property files is appropriately named and has a header comment indicating its contents along with the real filenames corresponding to p1 and p2. Alternatively, results can dump to the console (System.out) by typing -f - on the command line without creating any new files.

PropDiff eases the examination of two property layers with a precedence relationship. Just run the utility with the filenames and no extra command-line arguments, then view the files produced. Use -f - to see the output without creating new files.

In the WebLogic example, two application server instances on one host differ only among the instance-specific properties. You can compare these properties with any of the flags except -u or -c, which only make sense when a precedence relationship exists.

When comparing two or more layers between server instances, first use the -u flag to make, for each instance, a union of two property files that respects the precedence relationship. Then compare the union property files with any or all of the following flags: -1, -2, -d, -e.

Naming tips

This a good place to discuss best practices for selecting property names. The first example is from WebLogic: always put the units in the name of a property. For instance:

• myapp.timeoutMsecs is better than myapp.timeout
• myapp.heapsizeKBytes is better than myapp.heapsize

Self-documenting names like these makes configuration changes faster, easier, and less error prone.

Similarly, I recommend myapp.foo_path_BASEDIR_rel over myapp.foo_path to indicate that the path to foo could be relative to the path from property myapp.BASEDIR. You should define a property setting for a project's base directory as early as possible to avoid multiple, partially redundant paths right from the start.

PropDiff makes a difference

Large, monolithic property files are unwieldy because they prove difficult to compare and often need excessive merging during early development. By defining layers of property files, we eliminate the conflicts and receive the benefit of implied comments. The PropDiff utility can help deal with both large and layered property files.