# Apache Log4j

## Summary


Apache Log4j is a logging framework for Java. A Java application can use Log4j to log important events during the course of its lifetime. The logs are later used for tracing application behaviour or performing audits. Unlike debugging an application using debuggers, log generation requires no user intervention apart from the initial calls to the logger API. Logs generated from production deployments give lot of useful context. Logs are a permanent record of what happened unlike debugging sessions that are transient. 

Log4j is an open-source project managed by the Apache Software Foundation. Log4j 1.x is incompatible with Log4j 2. The latter has better design and richer features. 

GUI-based tools are available to view and analyze Log4j logs. Log4j has inspired logging solutions in other languages.

## Discussion

### What's the architecture of Log4j?

`Logger` class is the starting point. Applications use the `LogManager` to return a logger object with a specific `LoggingContext`. If such a logger doesn't exist, it's created. 

Every logger has a `LoggerConfig`. These config objects are initialized from configuration files. LoggerConfig has a class hierarchy, that is, one config class can inherit from another. 

A LoggerConfig object is associated with one or more `Appender` objects that actually deliver log events. For example, a logger can log to both the console and to a file. Among the appender types are ConsoleAppender, FileAppender, RollingFileAppender, AsyncAppender, SocketAppender, SMTPAppender, and many more.  Because config objects have a hierarchy, events will be sent to appenders of parent classes as well. 

Each appender can be configured to log messages in a specific format. Such formatting is the job of the `Layout` class. 

A LoggerConfig object is associated with a log level. Levels enable automatic filtering of events. In addition, `Filter` objects can be applied to LoggerConfig objects and Appender objects. Filters give more flexibility at runtime. Only those events that pass the filters are send to appenders. 


### What are the essential features of Log4j?

Log4j separates its API from implementations. Hence, applications can swap implementations while the code consistently calls Log4j API. 

An application typically has many packages, classes, and methods. Log4j allows us to configure logging for each component separately. However, to save work on managing many configurations, configurations can be inherited and hence reused. Thus, `X.Y.Z` inherits and overrides some configuration from `X.Y`. 

Log4j has log levels to control the scope of logging. Developers can also define custom levels. Likewise, custom Message types, Layouts, Filters and Lookups can be created. Because of a plugin architecture, Log4j is easily extensible. 

Log4j has built-in support for JMX. Many parts of the Log4j system can be locally or remotely monitored and controlled. 

For better performance, Log4j supports asynchronous logging. It does garbage-free or low garbage logging so that pressure on garbage collector is relieved. For better concurrency, it locks resources at the lowest possible level. Log4j won't lose events when appenders are reconfigured. Hence, it can be also be used for audit logging. 


### What log levels are available in Log4j?

A log level is assigned to a `LoggerConfig` object. If not, the object inherits the level from one of its ancestors along the class hierarchy. Log4j defines the following built-in levels: ALL (Integer.MAX\_VALUE), TRACE (600), DEBUG (500), INFO (400), WARN (300), ERROR (200), FATAL (100), and OFF (0). Applications can also define custom levels. Levels OFF and ALL are not generally used in API calls. 

Every event is logged with an indication of its level. For example, if the application sees an unexpected scenario it may log it as WARN; log it as ERROR if the transaction can't be completed; log it as FATAL if application can't proceed further without a reboot or reinitialization.

We can control how much to log without changing the code. For example, with log level ERROR only errors or more series events will be logged. So the call `logger.error()` will be logged but not `logger.info()`. During debugging, log level can be changed to DEBUG to log more messages. 

Log level can be dynamically changed at runtime: `Configurator.setLevel("com.example.Foo", Level.DEBUG);` 


### How can I format Log4j messages?

Log4j supports a number of logging formats, what are formally called as **Layouts**: CSV, GELF, JSON (pretty or compact), JSON Template, Pattern, RFC5424 (enhanced Syslog), Serialized (deprecated), Syslog (BSD Syslog and used in Log4j 1.2), XML (pretty or compact), and YAML. 

If location information is logged (filename, line number) or location-specific patterns are used (`%C` or `%class`, `%F` or `%file`, `%l` or `%location`, `%L` or `%line`, `%M` or `%method`), there's a performance impact, especially for asynchronous loggers. Default configuration excludes location information. 

For the pattern layout, message formatting relies on the *conversion pattern*, which is similar to `printf` formatting in C. For example, the conversion pattern `"[%-5p] %d{yyyy/MM/dd HH:mm:ss,SSS} %t %c: %m%n"` logs right padded five-character width logging level (`%-5p`), datetime in a specific format (`%d{pattern}`), thread name (`%t`), logger name (`%c`), log message (`%m`) and newline (`%n`). The "-5" in "%-5p" is a format modifier. Modifier "20.-30" implies left pad to 20 characters or truncate from the end to 30 characters. 


### What are Log4j API essentials that a developer should know?

The main package defining the Log4j API is `org.apache.logging.log4j`. Public message types are in package `org.apache.logging.log4j.message`. An implementation is available in package `org.apache.logging.log4j.simple`. 

Essentially, if you're developing a library you should use the API, not the concrete implementation. The application that uses your library will determine whether to use `org.apache.logging.log4j.simple` or another implementation of the Log4j API. 

Log4j API has four main interfaces: 

    + `LogBuilder`: To construct log events before logging them.
    + `Logger`: Main interface of the `log4j` package. Class `org.apache.logging.log4j.simple.SimpleLogger` implements this interface.
    + `Marker`: Adds filterable information to log messages. Markers are hierarchical. For example, "Error" marker can have children "SystemError" and "ApplicationError".
    + `ThreadContext.ContextStack`: ThreadContext Stack interface.Among the classes are `EventLogger`, `Level`, `LogManager`, `MarkerManager`, `ThreadContext`, and more. `LogManager` is the anchor point for the Log4j system. Methods of this class include `getContext()`, `getLogger()`, `getRootLogger()`, and more. 

`LoggingException` is thrown when logging error occurs. 

For detailed information, consult the complete Log4j API documentation.


### What's SLF4J and is it better than Log4j 2?

SLF4J is a "simple logging facade", which is really a logging interface that applications can call. SLF4J forwards API calls to a concrete implementation used by the application. Such an implementation could be `java.util.logging`, Log4j, Logback, etc. A logging implementation can conform to SLF4J API and thereby enable developers to easily switch implementations. We may say that SLF4J does for logging what ORMs have done for database interfacing.

Both Log4j 2 and SLF4J separate the API from the implementation. This gives developers flexibility. To use Log4j, application code can call SLF4J API that are then routed to Log4j's specific implementation. An alternative is to make Log4j calls but use the *log4j-to-slf4j* adapter to route calls to any SLF4J-compliant library. Hence, calling Log4j API directly in code doesn't tie you down to only Log4j. 

In fact, Log4j has some advantages over SLF4J. SLF4J can log only strings but Log4j can also log objects (implementations do the necessary conversions). Log4j supports Java 8 lambda expressions. Log4j has better support for garbage-free logging. 


### What are some best practices when using Log4j?

Logging gives us insights into application use and helps in troubleshooting. Too much logging can affect application's performance. Too little logging can be ineffective. Logs should include descriptive messages along with sufficient context to make them useful. 

Don't log sensitive information such as passwords, credit card numbers or access tokens. 

Use suitable log levels for your messages and use them consistently. 

When an exception in logged, include the exception object for more context, such as `try {...} catch (IOException ioe) { LOGGER.error("Error while executing main thread", ioe); }`. This will include the stack trace, which can be very useful. 

Logging in JSON can be better for storing logs in a log management system. JSON is also better for multiline messages such as stack traces. An appender can be configured for logging in JSON. 

Where performance is critical, consider logging asynchronously. Logging directly over a network may not be ideal since network errors can cause some log entries to be lost. Inside containers, log files are not permanent. Hence, log to the standard output or over a network to a centralized system. 


### What other projects are inspired from or associated with Log4j?

Apache Log4j comes under an umbrella project named *The Apache Logging Services Project*. Apart from Log4j, this includes Apache Log4j Kotlin and Apache Log4j Scala. Both these are APIs in their respective languages for Log4j. There's also Apache log4cxx and Apache Log4Net that are ports of Log4j for C++ and .NET respectively. Apache Log4j Audit is meant for audit logging. 

Other ports include log4c (C); log4js, JSNLog (JavaScript); log4perl (Perl); Apache log4php (PHP); and log4r (Ruby). Log4j can also be used in languages Clojure and Groovy. 

Apache Chainsaw is a GUI-based log viewer. It simplifies the analysis of logs via colour coding, tooltips, filtering, search function, navigation, etc. In fact, there are many other log viewers out there: OtrosLogViewer, Lilith, Eclipse LogViewer Version, Splunk, LogSaw, LogMX, and more. 

Complete log management is possible with SolarWinds Log Analyzer, Sematext Logs, Datadog, Fluentd, LogDNA, Splunk, Elastic Stack, and more.  


### As a beginner, how do I get started with Log4j?

The official Apache Log4j 2 project site is an ideal place to get started. It has links to the user's guide, articles, tutorials and FAQ. It also hosts the project's Javadoc API documentation.

Installing and configuring Log4j 2 is also described. You need to update project dependencies in file `pom.xml` used by Maven builder. Log4j 2 configuration can be in XML, JSON, YAML or properties formats. This file would include the appenders, their pattern layouts, the loggers, and the mapping between loggers and appenders. 

Documentation informs how to configure Log4j for other builders including Maven, Ivy, Gradle, and SBT. Project dependencies are also listed.  

Migration guides are available for those who wish to migrate from Log4j 1.x to 2.x.  

## Milestones

1991

James Gosling and others start work on a new language for interactive television. They name it *Oak*. In 1995, this is renamed to *Java*. The first public release of the language happens in 1996. 

1996

The E.U. SEMPER project creates its own tracing API for Java, which later evolves to *Log4j*. In these early years of Java, there's no universal logging framework or library. Every application includes its own logging or tracing API. 

Jan  
1999

The Apache Software Foundation adopts Log4j under its *The Apache Logging Services Project*. This project "creates and maintains open-source software related to the logging of application behavior." First versions of Log4j appears in October.  

Jan  
2001

**Apache Log4j v1.0** is released. Package hierarchy starts at `org.apache.log4j`. In subsequent years, Log4j proves to be the first of its kind to gain traction in the Java world for logging. It's also credited for introducing hierarchical logging, a concept that other loggers go on to adopt. 

Feb  
2002

Sun Microsystems releases Java 1.4 that includes `java.util.logging` (available since mid-2001 in its beta release). Its API appears to be similar to Log4j. Open source implementations of this API for Java 1.2 and 1.3 become available. 

May  
2002

**Apache Log4j v1.2** is released. This release is maintained till May 2012 when v1.2.17 is released. In August 2015, Apache announces **end of life for Log4j 1.x**.  

Jul  
2012

Work starts on **Apache Log4j 2**. It's a complete rewrite of Log4j and it's incompatible with Log4j 1.x. It's inspired by Log4j 1.x and `java.util.logging`. Log4j v2.0 released in July 2014. Package hierarchy changes in Log4j 2 to `org.apache.logging.log4j`. 

Sep  
2017

Java 9 is released. This **breaks compatibility for Log4j 1.x** due to a feature called Mapped Diagnostic Context (MDC). MDC's equivalent in Log4j 2 is Thread Context Map. Applications using Log4j 1.x but without MDC may still work. But given Log4j 2's many features, developers will find it worthwhile to migrate to it. 

Nov  
2021

A member of the Alibaba Cloud Security Team finds a vulnerability in Log4j 2. The team informs Apache. Called **Log4Shell**, the vulnerability is made public in December. Given the widespread use of Java and hence Log4j 2, this is considered a series vulnerability affecting cloud servers, enterprise applications, IoT devices, routers, firewalls, printers, etc. Apache fixes the issue in releases 2.15.0, 2.16.0 and 2.17.0 (Dec 2021). Meanwhile, websites and businesses list affected software and mitigation strategies.