Spring Boot Log4j 2 advanced configuration #1 – saving logs to files

Apart from having logs displayed directly in the console, we can configure our Spring Boot project to write log entries into files. It not only allows us to easily store and share logs for debugging, but also can be the first step towards browsing them in Kibana.

What we are going to build

Spring Boot provides a ready to use logging mechanism within the spring-boot-starter-logging dependency. First, we will disable it so our logs won’t be handled by Logback. Then, we will include the dependency for Log4j 2 and configure it using the YAML format. We are working on a basic Spring Boot 2.2.6 app with the Web dependency.

You can find the example project in the spring-boot-log4j-2-scaffolding repository.

Add Log4j 2 dependency to the project

In this part you will find details about switching from the default logging mechanism to Log4j 2.

Disable the default logging

According to the documentation, thanks to the Web dependency, logs are handled with Logback by default. The spring-boot-starter-web depends transitively on the logging starter that provides Logback. Therefore, we have to exclude it manually in our pom.xml file:

Add the Log4j 2 Spring Boot starter

Now we can update the pom.xml file by adding the starter for Log4j2:

The documentation contains information about handling logging dependencies without the starters:

If you do not use the starters, you need to provide (at least) spring-jcl in addition to Log4j 2.

https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-configure-log4j-for-logging

Enable YAML configuration

To use an alternative to the XML format, we have to append additional dependencies. For applying a config in the YAML format, include jackson-databind and jackson-dataformat-yaml in your pom.xml file:

You can run the project. The logs will be printed in the console in line with the Log4j 2 Automatic Configuration. The work done in this section is contained in the f57af50f8338787633a88badc127ce0b4fd7864f commit.

Configure Log4j 2 to handle printing logs in the console as well as saving them in a file

The main concepts used in the library are listed in the Architecture section of the Log4j 2 documentation. Spring Boot also gives some insight about custom log configuration. You can find the official YAML configuration example in the documentation.

Let’s assume that we want to print logs to the console and, at the same time, save them in files. The log level for our loggers is the same – INFO.

Below you will find description for all options from the example file alongside with the links to the official documentation.

properties

To keep the config more readable I used the Property Substitution mechanism. There are many places the values can originate from. To provide an attribute value from the outside of the config file we use the ${prefix:name} syntax. prefix points to the context in which the property value should be taken from. If there is no value set in the given context, the one provided within the config file will be used by default. In the ${sys:LOG_EXCEPTION_CONVERSION_WORD} example the value from the system context, if present, will be used.

Caveat: Another possible source for property values are environment variables. You can read about differences between an env and a system property in the Java system properties and environment variables question on Stack Overflow.

Appenders

To specify the output for logs I need to configure Appenders. It is extremely useful to get yourself acquainted with the role that appenders play in the Log4j 2 architecture. You can also browse available appenders to find those that are the most suitable for your use case.

Console appender

After adding the custom log4j2-spring.yaml configuration file I no longer enjoy benefits of having logs printed in my console by default. Therefore, I had to configure the Console appender. I used it as an output for the root logger. It means that all logs are going to be sent to this appender. I had to give it a unique name so I can later provide it to the associated logger. I also specified the pattern for my log entries.

File appender

I want to send all logs that are printed in the console to a file. For that reason, I added a File appender that is associated with the root logger and will store log entries in the all.log file. For the sake of the simplicity, the log/ directory is kept directly in the project’s root folder.

PatternLayout

We can configure our appenders to send formatted output to their destinations by adding the PatternLayout. The specified layout will be responsible for the arrangement of log entries. You can configure the format to keep the default Spring Boot pattern for a console and a file output.

We are going to build our layouts using available conversion patterns according to the following structure: the % sign followed by a format modifier with a conversion character at the end. The formatting used in the example is based on the log4j2.xml file from the spring-boot project.

• Our example console log pattern:

• The file log pattern:

Conversion characters

Let’s start with the elements that determine the type of data. Below you can see the documentation for characters used in the example config file:

• d – the date of the logging event;
• p – the level of the logging event;
• pid – the process ID;
• t – the name of the thread that generated the logging event;
• c{1.} – the name of the logger that published the logging event (adding the {1.} precision specifier in our example will cause a logger name to be abbreviated from org.apache.commons.Foo to o.a.c.Foo);
• m – the message associated with the logging event;
• n – the platform dependent line separator;
• xwEx – the throwable conversion word. The ExtendedWhitespaceThrowablePatternConverter adds additional whitespace around the stack trace (Meaning of xwEx for log4j2? question on Stack Overflow).

Below you will find the code that generates an example exception (logged with Lombok):

On the following screenshot you can see the console output for that code:

As you can see the exception is visually separated from the rest of the log entries (the %xwEx pattern).

Format modifiers

An optional format modifier is placed between the % sign and a conversion character. The format modifiers control such things as field width, padding, left and right justification. The details are enclosed in the documentation. For example:

• -40.40c – this pattern starts with the left justification flag (-), then we specified the minimum field width (40, filled with whitespace if shorter) and, separated by the period (.), the maximum width (40) for a logger name.
• 15.15t – a thread name is right-aligned, the minimum (filled with whitespace if shorter) and maximum width for this field is 15 characters.

Highlight and style

• I overridden the colour for the TRACE level with the following code:

• The colour of the process ID is set to magenta:

• The logger name is printed in cyan:

Therefore, my console output looks like on the image below:

If you want to change colours of log levels, read about the highlight conversion in the docs:

File pattern

The entries collected in files look like on the image below:

fileName

The file that the logs will be saved to. If the file or its parent directories don’t exist, they will be automatically created.

Loggers

We determined the destination of logs in the Appenders section. Now it’s time to actually capture logs and send them to the right appender. We are required to name all loggers except the root one. If we don’t indicate a level attribute it will default to ERROR. Because I didn’t set the additivity attribute it will default to false and log entries from all loggers will be printed in the appenders associated with the root logger.

AppenderRef

You have to provide targets for entries gathered by each logger. Use the names specified in the appenders section. For the root logger I send entries to the all.log file and the console with the following lines:

Additivity

We can send entries extracted by one logger to multiple appenders. By default, the logger that is higher in the hierarchy will print in its appenders entries collected by all “children” loggers.

In our example all appenders associated with the root logger would contain entries from the in.keepgrowing.springbootlog4j2scaffolding logger. Let’s change the level to DEBUG for this particular logger, and see, how during the application startup we have the following entries printed in the console, the all.log and the app.log files:

If we want to send logs from one logger explicitly to its own appenders we need to set the additivity flag to false:

With those setting we can see that the entries form this logger are missing in the console and other appenders associated with the parent logger – the root logger:

This logger sends entries solely to the app.log file:

Root logger

The root logger is required in every configuration. It doesn’t need a name. By default it will be assigned the ERROR level and the console as its appender. This logger doesn’t have a parent so we don’t specify the additivity attribute for it. In the example project it collects all entries that are at least at the INFO level and sends them to the console and the all.log file.

Other loggers

You can add other, more granular loggers. Name them after the package from which they are supposed to draw entries and provide one or more destination (appender) for those logs. In the example we have the in.keepgrowing.springbootlog4j2scaffolding logger. Remember to adjust the configuration of your loggers so the logs are send to appenders according to your preferences.

The work done in this section is contained in the 75c3667a79a7342a82f2d0df5f6e286e5a5b299f commit.

Deeper understanding

The example provided in this post represents a very simplistic approach and can be only used as a starting point for configuring Log4j 2 in a project. Please follow this advice from the documentation when working with this library:

An understanding of how loggers work in Log4j is critical before trying to configure them. Please reference the Log4j architecture if more information is required. Trying to configure Log4j without understanding those concepts will lead to frustration.

https://logging.apache.org/log4j/2.x/manual/configuration.html

Photo by Matt Glm on StockSnap