Manual · Appenders · Appenders: Rolling File

Appenders: Rolling File

RollingFileAppender

RollingFileAppender extends FileAppender with the capability to rollover log files. For example, RollingFileAppender can log to a file named log.txt file and, once a certain condition is met, change its logging target to another file.

There are two important subcomponents that interact with RollingFileAppender. The first RollingFileAppender sub-component, namely RollingPolicy, (see below) is responsible for undertaking the actions required for a rollover. A second subcomponent of RollingFileAppender, namely TriggeringPolicy, (see below) will determine if and exactly when rollover occurs. Thus, RollingPolicy is responsible for the what and TriggeringPolicy is responsible for the when.

To be of any use, a RollingFileAppender must have both a RollingPolicy and a TriggeringPolicy set up. However, if its RollingPolicy also implements the TriggeringPolicy interface, then only the former needs to be specified explicitly.

Here are the available properties for RollingFileAppender:

Overview of rolling policies

RollingPolicy is responsible for the rollover procedure which involves file moving and renaming.

The RollingPolicy interface is presented below:

package ch.qos.logback.core.rolling;

import ch.qos.logback.core.FileAppender;
import ch.qos.logback.core.spi.LifeCycle;

public interface RollingPolicy extends LifeCycle {

  public void rollover() throws RolloverFailure;
  public String getActiveFileName();
  public CompressionMode getCompressionMode();
  public void setParent(FileAppender appender);
}

The rollover method accomplishes the work involved in archiving the current log file. The getActiveFileName() method is called to compute the file name of the current log file (where live logs are written to). As indicated by getCompressionMode method a RollingPolicy is also responsible for determining the compression mode. Lastly, a RollingPolicy is given a reference to its parent via the setParent method.

TimeBasedRollingPolicy

TimeBasedRollingPolicy is possibly the most popular rolling policy. It defines a rollover policy based on time, for example by day or by month. TimeBasedRollingPolicy assumes the responsibility for rollover as well as for the triggering of said rollover. Indeed, TimeBasedTriggeringPolicy implements both RollingPolicy and TriggeringPolicy interfaces.

TimeBasedRollingPolicy's configuration takes one mandatory fileNamePattern property and several optional properties.

/var/log/%d{yyyy/MM, aux}/myapplication.%d{yyyy-MM-dd}.log

aFolder/test.%d{yyyy-MM-dd-HH, UTC}.log

Here are a few fileNamePattern values with an explanation of their effects.

Any forward or backward slash characters are interpreted as folder (directory) separators. Any required folder will be created as necessary. You can thus easily place your log files in separate folders.

Automatic file compression

TimeBasedRollingPolicy supports automatic file compression. This feature is enabled if the value of the fileNamePattern option ends with .gz .zip or .xz. Note that xz compression requires Tukaani project's XZ library for Java. In case XZ compression is requested but the XZ library is missing, then logback will substitute GZ compression as a fallback.

The fileNamePattern serves a dual purpose. First, by studying the pattern, logback computes the requested rollover periodicity. Second, it computes each archived file's name. Note that it is possible for two different patterns to specify the same periodicity. The patterns yyyy-MM and yyyy@MM both specify monthly rollover, although the resulting archive files will carry different names.

By setting the file property you can decouple the location of the active log file and the location of the archived log files. The logging output will be targeted into the file specified by the file property. It follows that the name of the active log file will not change over time. However, if you choose to omit the file property, then the active file will be computed anew for each period based on the value of fileNamePattern. By leaving the file option unset you can avoid file renaming errors which occur while there exist external file handles referencing log files during roll over.

The maxHistory property controls the maximum number of archive files to keep, deleting older files. For example, if you specify monthly rollover, and set maxHistory to 6, then 6 months worth of archives files will be kept with files older than 6 months deleted. Note as old archived log files are removed, any folders which were created for the purpose of log file archiving will be removed as appropriate.

For various technical reasons, rollovers are not clock-driven but depend on the arrival of logging events. For example, on 8th of March 2002, assuming the fileNamePattern is set to yyyy-MM-dd (daily rollover), the arrival of the first event after midnight will trigger a rollover. If there are no logging events during, say 23 minutes and 47 seconds after midnight, then rollover will actually occur at 00:23'47 AM on March 9th and not at 0:00 AM. Thus, depending on the arrival rate of events, rollovers might be triggered with some latency. However, regardless of the delay, the rollover algorithm is known to be correct, in the sense that all logging events generated during a certain period will be output in the correct file delimiting that period.

Here is a sample configuration for RollingFileAppender in conjunction with a TimeBasedRollingPolicy.

Example: Sample configuration of a RollingFileAppender using a TimeBasedRollingPolicy (logback-examples/​src/main/resources/​chapters/appenders/​conf/logback-RollingTimeBased.​xml)

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>logFile.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <!-- daily rollover -->
      <fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>

      <!-- keep 30 days' worth of history capped at 3GB total size -->
      <maxHistory>30</maxHistory>
      <totalSizeCap>3GB</totalSizeCap>

    </rollingPolicy>

    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} -%kvp- %msg%n</pattern>
    </encoder>
  </appender> 

  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

The next configuration sample illustrates the use of RollingFileAppender associated with TimeBasedRollingPolicy in prudent mode.

Example: Sample configuration of a RollingFileAppender using a TimeBasedRollingPolicy (logback-examples/​src/main/resources/​chapters/appenders/​conf/logback-PrudentTimeBasedRolling.​xml)

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <!-- Support multiple-JVM writing to the same log file -->
    <prudent>true</prudent>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>
      <maxHistory>30</maxHistory> 
      <totalSizeCap>3GB</totalSizeCap>
    </rollingPolicy>

    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} -%kvp -%msg%n</pattern>
    </encoder>
  </appender> 

  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

Size and time based rolling policy

Sometimes you may wish to archive files essentially by date but at the same time limit the size of each log file, in particular if post-processing tools impose size limits on the log files. In order to address this requirement, logback ships with SizeAndTimeBasedRollingPolicy.

Note that TimeBasedRollingPolicy already allows limiting the combined size of archived log files. If you only wish to limit the combined size of log archives, then TimeBasedRollingPolicy described above and setting the totalSizeCap property should be amply sufficient. Moreover, given that file renaming is a relatively slow process and is frought with problems, we discourage the use of SizeAndTimeBasedRollingPolicy unless you have a real-world use case.

The roll over based on size relies on the "%i" conversion token in addition to "%d". Both the %i and %d tokens are mandatory. Each time the current log file reaches maxFileSize before the current time period ends, it will be archived with an increasing index, starting at 0.

The table below lists the properties applicable for SizeAndTimeBasedRollingPolicy. Note these properties complement those applicable for TimeBasedRollingPolicy.

Here is a sample configuration file demonstrating time and size based log file archiving.

Example: Sample configuration for SizeAndTimeBasedRollingPolicy (logback-examples/​src/main/resources/​chapters/appenders/​conf/logback-sizeAndTime.​xml)

<configuration>
  <appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>mylog.txt</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
      <!-- rollover daily -->
      <fileNamePattern>mylog-%d{yyyy-MM-dd}.%i.txt</fileNamePattern>
       <!-- each file should be at most 100MB, keep 60 days worth of history, but at most 20GB -->
       <maxFileSize>100MB</maxFileSize>    
       <maxHistory>60</maxHistory>
       <totalSizeCap>20GB</totalSizeCap>
    </rollingPolicy>
    <encoder>
      <pattern>%msg%n</pattern>
    </encoder>
  </appender>


  <root level="DEBUG">
    <appender-ref ref="ROLLING" />
  </root>

</configuration>

Note the "%i" conversion token in addition to "%d". Both the %i and %d tokens are mandatory. Each time the current log file reaches maxFileSize before the current time period ends, it will be archived with an increasing index, starting at 0.

Size and time based archiving supports deletion of old archive files. You need to specify the number of periods to preserve with the maxHistory property. When your application is stopped and restarted, logging will continue at the correct location, i.e. at the largest index number for the current period.

In versions prior to 1.1.7, this document mentioned a component called SizeAndTimeBasedFNATP. However, given that SizeAndTimeBasedRollingPolicy offers a simpler configuration structure, we no longer document SizeAndTimeBasedFNATP. Moreover, in logback version 1.5.8, SizeAndTimeBasedFNATP was renamed as SizeAndTimeBasedFileNamingAndTriggeringPolicy. Thus, earlier configuration files using SizeAndTimeBasedFNATP will no longer work.

FixedWindowRollingPolicy

Given that file renaming is a relatively slow process and is frought with problems, we consider FixedWindowRollingPolicy as a deprecated policy and do not recommend its use.

When rolling over, FixedWindowRollingPolicy renames files according to a fixed window algorithm as described below.

The fileNamePattern option represents the file name pattern for the archived (rolled over) log files. This option is required and must include an integer token %i somewhere within the pattern.

Here are the available properties for FixedWindowRollingPolicy

Given that the fixed window rolling policy requires as many file renaming operations as the window size, large window sizes are strongly discouraged. When large values are specified by the user, the current implementation will automatically reduce the window size to 20.

Let us go over a more concrete example of the fixed window rollover policy. Suppose that minIndex is set to 1, maxIndex set to 3, fileNamePattern property set to foo%i.log, and that file property is set to foo.log.

The configuration file below gives an example of configuring RollingFileAppender and FixedWindowRollingPolicy. Note that the File option is mandatory even if it contains some of the same information as conveyed with the fileNamePattern option.

Example: Sample configuration of a RollingFileAppender using a FixedWindowRollingPolicy (logback-examples/​src/main/resources/​chapters/appenders/​conf/logback-RollingFixedWindow.​xml)

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>test.log</file>

    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>tests.%i.log.zip</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>3</maxIndex>
    </rollingPolicy>

    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} -%kvp -%msg%n</pattern>
    </encoder>
  </appender>
	
  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

Overview of triggering policies

TriggeringPolicy implementations are responsible for instructing the RollingFileAppender when to rollover.

The TriggeringPolicy interface contains only one method.

package ch.qos.logback.core.rolling;

import java.io.File;
import ch.qos.logback.core.spi.LifeCycle;

public interface TriggeringPolicy<E> extends LifeCycle {

  public boolean isTriggeringEvent(final File activeFile, final <E> event);
}

The isTriggeringEvent() method takes as parameters the active file and the logging event currently being processed. The concrete implementation determines whether the rollover should occur or not, based on these parameters.

The most widely-used triggering policy, namely TimeBasedRollingPolicy which also doubles as a rolling policy, was already discussed earlier along with other rolling policies.

SizeBasedTriggeringPolicy

SizeBasedTriggeringPolicy looks at the size of the currently active file. If it grows larger than the specified size, it will signal the owning RollingFileAppender to trigger the rollover of the existing active file.

SizeBasedTriggeringPolicy accepts only one parameter, namely maxFileSize, with a default value of 10 MB.

The maxFileSize option can be specified in bytes, kilobytes, megabytes or gigabytes by suffixing a numeric value with KB, MB and respectively GB. For example, 5000000, 5000KB, 5MB and 2GB are all valid values, with the first three being equivalent.

Here is a sample configuration with a RollingFileAppender in conjunction with SizeBasedTriggeringPolicy triggering rollover when the log file reaches 5MB in size.

Example: Sample configuration of a RollingFileAppender using a SizeBasedTriggeringPolicy (logback-examples/​src/main/resources/​chapters/appenders/​conf/logback-RollingSizeBased.​xml)

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>test.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>test.%i.log.zip</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>3</maxIndex>
    </rollingPolicy>

    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} -%kvp -%msg%n</pattern>
    </encoder>
  </appender>
	
  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>