How to Read Log File and Split in Java
Yous tin can employ this guide to observe, understand and utilise the right Java logging library for your applications, similar Log4j2, Logback, or java.util.logging.
Logging 'seems' like a very elementary topic, merely can be rather tricky in practice and is not covered anywhere in sufficient detail. Read this guide to fully make sense of the Java logging landscape.
Introduction
Sooner or later, every Java awarding needs logging.
It could be that you simply desire to log your system'southward land or user actions to a file, so your operations staff has a way of knowing what'due south happening.
logger . info ( "Application successfully started on port 8080" ); It could be that y'all need to tape error messages whenever an exception happens and and so ship an e-post or text bulletin to a homo for urgent intervention.
logger . error ( "Database connection is downwardly" , exception ); Or i of your batch jobs might want to record and send warnings to a central, GUI-based log server, whenever it fails to import some records of a csv file.
logger . warn ( "Invalid bank account number for record=[{}]" , 53 ); No matter what you want to do, yous'll need to make certain to have an appropriate logging library and so configure and use it correctly.
Unfortunately, the Java universe has a plethora of logging libraries available (come across this video for a prissy overview of Java logging hell) and a programmer should have a rough agreement of why there are that many options and when to use which option.
Let'south find out.
Legacy Logging Libraries
To understand contempo developments with Java'south logging libraries, it makes sense to run into & understand the dinosaurs - Coffee'south oldest logging libraries, that you will still observe in some product environments today.
java.util.logging (JUL)
Since Coffee one.4 (2002), the JDK comes arranged with its own logging 'framework', called java.util.logging, often abbreviated to JUL. Here'southward an case of how yous would log things with JUL:
// coffee.util.logging coffee . util . logging . Logger logger = coffee . util . logging . Logger . getLogger ( this . getClass (). getName ()); logger . info ( "This is an info message" ); logger . severe ( "This is an error message" ); // == ERROR logger . fine ( "Here is a debug message" ); // == DEBUG As is the case for every logging library, first y'all get a Logger for a specific class or package, and so y'all can log statements. You might think that the log levels 'severe' and 'fine' await weird, but they basically correspond to the 'error' and 'debug' levels of all modernistic Java logging libraries.
And of course, you tin configure your loggers with so called handlers, like a FileHandler (which writes log statements to files) or a ConsoleHandler (which writes to System.err).
FileHandler fileHandler = new FileHandler ( "status.log" ); logger . addHandler ( fileHandler ); So, you might be asking, if there's JUL, why would anyone need another logging framework?
While JUL does the job, there have been discussions about its shortcomings in the past, spanning its inconsistent API, tedious performance, lack of (complex) configuration options, documentation etc - which ultimately lead to people developing and using other logging frameworks.
Log4j (v1)
For a long time, the most popular choice for logging in Java state was Log4j (version 1), which was initially released in 2001 and maintained until 2015. In fact, you lot'll still find it existence used in a fair number of corporate projects, in 2018.
With Log4j, the logging instance from higher up would look like this:
// Log4j V1 org . apache . log4j . Logger logger = org . apache . log4j . Logger . getLogger ( MyClass . getClass (). getName ()); logger . info ( "This is an info message" ); logger . error ( "This is an error message" ); logger . debug ( "Here is a debug bulletin" ); It'southward not simply that Log4j has sane log level names, similar 'error' and 'debug'. Information technology likewise comes with a ton of unlike and clever appenders, similar SMTPAppender (to e-post log events), SyslogAppenders (to send events to a remote syslog daemon), JdbcAppenders (to send them to the database) and many more than.
Also information technology gives you a fair amount of control over what exactly a log message should expect like, with the help of PatternLayouts. So, the same log issue in your Coffee application, could be printed out in a log file like and then, depending on the layout:
# contents of status.log [INFO] 2012-11-02 21:57:53,662 MyLoggingClass - Application succesfully started on port 8080 # or 2010.03.23-mainThread --INFO -MyLoggingClass:Awarding succesfully started on port 8080 # or other endless possibilities Log4j does the job quite well, but got superseded by Log4j2 in the past years, which is not fully compatible with Log4j1, hence we volition talk about it in the next section.
Apache Eatables Logging (JCL)
Around the same time, in 2002, another library popped up, chosen JCL, which goes by ii names. Jakarta Commons Logging or Apache Commons Logging.
The interesting affair about JCL is, that it is not a logging framework implementation itself. Rather, it is an interface for other logging implementations. What does that mean?
As you might have guessed, the logging code itself stays fairly unproblematic, only instead of referencing JUL or Log4j classes, yous at present reference the JCL classes - provided you lot have the JCL library on your classpath.
// Apache Commons Logging org . apache . commons . logging . Log log = org . apache . commons . logging . LogFactory . getLog ( MyApp . course ); log . info ( "This is an info message" ); log . error ( "This is an mistake bulletin" ); log . debug ( "Here is a debug message" ); So, your lawmaking only uses JCL specific classes. But then the actual logging is done by some other logging framework, and it doesn't thing if it is Log4j, JUL or (the long defunct) Apache Avalon.
That means you demand another library, similar Log4j, on your classpath and configure these 2 to work together.
Why would anyone want to do this?
Why wouldn't you desire to apply Log4j direct, instead? There's one major use-case, which is writing libraries:
When writing libraries, you only practise not know which logging framework a user of your library wants to use in his own application. It therefore makes sense to write your library to use a logging interface - and and so the user tin plug in any logging implementation that he or she wants, whenever he deploys his ain application.
Where's the grab?
The trouble with JCL is, that information technology relies on classloader hacks to notice out which logging implementation it should use - at runtime. And that tin can lead to a lot of pain. In add-on, you'll find that the API is somewhat inflexible, that information technology comes with a off-white amount of cruft, and at that place'south simply better alternatives out there, nowadays.
Modernistic Logging Libraries
SLF4J & Logback
At some point, the original creator of Log4j, Ceki Gülcü, decided to split from the Log4j project and create a successor, which was not called Log4j2, but Logback. You tin can read about what he tried to make ameliorate with Logback here.
Suffice to say, that Logback is a mature and solid logging library, with a ton of features, of which machine-reloading of configuration files in production is one that developers seem to think the nearly.
Somewhat in parallel, he as well started writing the Simple Logging Facade for Java, also known as SLF4J, which is very like to the Apache Commons Logging 'bridging' library above, but with a meliorate implementation. Let's see what that means:
To get started with SLF4J, you only need one library on the classpath, the slf4j-api dependency (see screencast for the following department). If you are using a dependency direction tool like Maven, then you would add together the post-obit dependency to your dependencies department:
<dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.7.30</version> </dependency> Having the API on your classpath will allow y'all to write log statements like the following:
// SLF4J org . slf4j . Logger logger = org . slf4j . LoggerFactory . getLogger ( MyClass . class ); logger . info ( "This is an info message" ); logger . error ( "This is an error message" ); logger . debug ( "Here is a debug message" ); // y'all practise not need 'logger.isDebugEnabled' checks anymore. SLF4J will handle that for yous). Just similar JCL, SLF4J cannot exercise logging itself. Information technology needs a logging library to practice the actual logging, similar Log4j, JUL, Logback etc. Then, say you want to apply Log4j v1, you would then need the slf4j-log4j12 binding library in your classpath:
<dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>ane.7.xxx</version> </dependency> That dependency will transitively pull in Log4j (v1) for you and likewise brand sure that SLF4J logs 'through' Log4j. If you are interested in how this works, read the section about bindings in the SLF4J manual.
Other libraries, like Logback, don't demand a binding library, because they implement SLF4J natively, so you tin merely use the slf4j-api dependency, driblet in the logback-archetype jar as well, and you can log through Logback.
<dependency> <groupId>ch.qos.logback</groupId> <artifactId>logback-classic</artifactId> <version>1.2.3</version> </dependency> The beauty of this arroyo is, that your code but knows SLF4J. There are no references to Log4j, Logback or Jul. And if you are writing a library, that's even better. Because if your library uses SLF4J, and so the end-user of your library can make up one's mind to log either with Log4j, or Logback, or whatever library he wants. Considering that choice can simply be made by adding or removing a couple of jars to or from the classpath.
Concur on, aren't we missing something?
Things get interesting, whenever you are using 3rd political party libraries, that are hardcoded to use a specific logging library. Imagine you lot are using a PDF generator library, which is hardcoded to use Log4j. You are also using an e-mail sending library, which uses JUL. Your ain application uses SLF4J, but yous cannot just go and change the source code of these libraries to utilise SLF4J as well.
What to do at present?
Thankfully, the SLF4J creators thought about that apply-case also (see screencast here). Let's meet what that looks like, by looking at the Maven dependencies get-go:
Whenever you pull in a 3rd party-library that uses Log4j, information technology volition, patently, pull in the Log4j dependency. The Log4j dependency looks like this:
<dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.two.17</version> </dependency> You then need to make certain to exclude that dependency from your project and apply the post-obit drop-in replacement instead:
<dependency> <groupId>org.slf4j</groupId> <artifactId>log4j-over-slf4j</artifactId> <version>ane.7.xxx</version> </dependency> Hither's the trick: Inside that log4j-over-slf4j.jar, y'all'll find classes like org.apache.log4j.Logger, but they have nothing to do with Log4j! Instead, these are SLF4J specific classes, i.e. your code 'thinks' it calls Log4j, but instead everything gets routed to SLF4J. (The aforementioned is true for the other 'over-slf4j' libraries, except for the JUL library, which you can read near hither).
Which in turn ways, that y'all, as the cease-user of a library, can utilize any logging library you want, fifty-fifty though the original library creator wanted y'all to use Log4j specifically.
Existent-Life
Then, depending on what you are building and the third-party libraries you are using, yous might terminate-up with the following libraries in your classpath:
-
The SLF4J API
-
Your SLF4J implementation, similar Logback, or Log4j, etc.
-
One or multiple bridging libraries, like log4j-over-slf4j, jul-to-slf4j, jcl-over-slf4j etc.
Main Takeaway
With SLF4J, y'all lawmaking to the API, and can choose the implementation (Log4j, Logback etc.) later (compile time). In addition, you can use bridging libraries to brand legacy 3rd party libraries 'speak' SLF4J.
While all of this might look scary for beginners, it all makes sense with a bit of experience.
Log4j2
One might think that SLF4J, with all the surrounding logging libraries, would cover almost all logging needs. Seems to be not so. In 2014, a successor to the Log4j (v1) library was released, called Log4j2 - existence a complete rewrite and of form heavily inspired by all other existing logging libraries.
Additionally, just similar SLF4J, or JCL, or Eatables Logging, Log4j2 can be used equally a bridge, equally it comes with two dependencies:
An API dependency:
<dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>2.14.0</version> </dependency> And the actual logging implementation:
<dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-cadre</artifactId> <version>2.14.0</version> </dependency> The API dependency works with a diversity of other logging frameworks, just similar SLF4J or JCL do. Yous tin can drib in Log4j2'south own logging implementation, use an SLF4J implementation, or use i of the bridging/adapter libraries to setup your logging in whatever mode you wish. Your code, all the same, will only reference the Log4j2 classes like so:
// Log4j (version 2) org . apache . logging . log4j . Logger logger = org . apache . logging . log4j . LogManager . getLogger ( MyApp . form ); logger . info ( "This is an info bulletin" ); logger . error ( "This is an error message" ); logger . debug ( "Here is a debug bulletin" ); If you take read the previous sections, you might conclude that SLF4J and Log4j2 accept a lot in mutual and it is unclear, why one would want to use Log4j2 instead of just sticking with SLF4J.
The creators of Log4j2 have tried to answer this question themselves here, with the main difference seemingly being the performance (AsyncLogger, Garbage Collections] and the slightly better API (Abilities to log objects, not just strings, Lambda Support etc).
Though information technology should be said that, while these reasons might make a difference in complex loftier-load applications, developers working on 'normal' applications likely won't notice a divergence.
Jboss Logging
Talking most logging libraries wouldn't be complete, without mentioning JBoss-Logging. It's yet another logging bridge, very similar to SLF4J or JCL, so you'll have to use it with some other logging implementation, or even SLF4J itself.
It's main claim to fame, compared to these other logging bridges, seems to exist its internationalization features. Other than that at that place seems to be very piffling reason to base your projection entirely on jboss-logging, though you will notice projects like Hibernate employ it, because both libraries are developed under the RedHat umbrella.
How to log
In one case you decided on your favourite logging framework, it's time to actually apply your logger as well. Which brings united states of america to the question: How should y'all log?
A small-scale, technical nit-pick
If yous take a look across unlike Java projects in your organization, or fifty-fifty just wait at one projection, you'll probable see a variety of ways that people try to get a Logger instance: The class that enables them to log in the starting time-identify.
This can take the looks of:
// in class 1 private Logger LOG = LoggerFactory . getLogger (...); // in class two private static last Logger LOGGER = ....; // in class 3 private static Logger log = Logger . getLogger (...); // in class four private Logger LOG_INSTANCE = ...; // etc. etc. Now what should this look like instead? To which in that location is an easy answer. If the class y'all are creating, and the method you lot are calling to create that class, both have the word "logger" inside, then telephone call the variable 'logger'.
Don't fret too much virtually static or non-static, final or non-final, just brand sure to exist homogeneous in your option, throughout your entire project.
And finally, at that place is really no need to UPPER_CASE your logger simply for the sake of it, certainly non equally the only exception in your codebase.
Log Levels & Files
An insanely interesting topic is: To which log level should yous actually log to? You take the choice of TRACE, DEBUG, INFO, WARN, Error, FATAL and a fair amount of developers experience unsure of when to utilise which one.
Hither'due south a general arroyo that I've seen used successfully in a few places, just annotation that this is non written in stone (meet screencast here). Make changes to these guidelines where appropriate, but make sure you have a solid use case and reasoning for it. And near importantly, make sure that your developers and operations guys are on the aforementioned folio.
Now let'south first have a expect at the "error group" log levels individually, and what you might use them for.
FATAL
Anything at this level means your Java process cannot proceed and will now end.
The least interesting log-level, as yous are very unlikely to utilise information technology in your application and API's such equally SLF4J don't even support it directly.
ERROR
A request was aborted and the underlying reason requires man intervention ASAP.
WARN
A asking was not serviced satisfactorily, intervention is required soon, but not necessarily immediately.
What does that mean in applied terms?
For assessing entries confronting Fault & WARN you can enquire the question "what is the action required" and if it doesn't audio like an "OMG! Get that action covered now!" type of event, it gets demoted to a lesser level for non meeting criteria.
Recall nearly you lot pushing a new characteristic of your shiny fintech (depository financial institution) application to production, which unfortunately triggers the infamous Hibernate LazyLoadingException whenever a user tries to display the recent transactions for his bank account. That sounds like a pretty strong OMG situation, and y'all'll want these errors to be logged equally "errors" - and trigger appropriate reactive measures.
2018-09-eleven 08:48:36.480 Mistake 10512 --- [ost-startStop-1] com.marcobehler.UserService : Retrieving transaction list for user[id={}] failed org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: User.transactionDetails, could not initialize proxy - no Session at org.hide.collection.internal.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:582) at org.hibernate.drove.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:201) at org.hide.collection.internal.AbstractPersistentCollection.initialize(AbstractPersistentCollection.coffee:561) at org.hide.collection.internal.AbstractPersistentCollection.read(AbstractPersistentCollection.java:132) at org.hibernate.collection.internal.PersistentBag.iterator(PersistentBag.java:277) at coffee.lang.Iterable.forEach(Iterable.java:74) at LibraryTest.spring_test(LibraryTest.java:78) ... Then think about a batch job, which imports transactions on a daily or weekly basis. As is the case quite frequently, some records might be malformed and thus cannot exist imported into the system. Someone, a person, needs to take a look at these records manually and gear up them. Only probable this isn't as time-sensitive and urgent as the fault example, so y'all'll cull to log these items with the WARN level.
2018-09-11 00:00:36.480 WARN 10512 --- [ost-startStop-ane] com.marcobehler.BatchJob : Could not import record[id=25] from csv file[name=transactions.csv] because of malformed[firstName,lastName] The main reasoning behind keeping ERROR and WARN tags clean is that it makes monitoring and hence reacting to those events much simpler.
Or in unproblematic terms: Make sure to wake-up your operations guy at 3am in the morning for the right (kind of) error.
INFO
Info is the log level developers probably feel near 'comfy' using and in practise yous'll observe that developers print out a ton of statements with the INFO level, from client activities (webapps), progress information (batch jobs) to quite intricate, internal process flow details.
Once more, it tin can exist a blurry line to determine what should be INFO and what should be DEBUG, only in general, procedure flow details should rather exist logged with the debug level, instead of completely replicating a user's journey through your application in an info.log.
Historically, the main reason for logging out nigh everything as INFO was, that it has been difficult to change log levels on the fly for applications, without having to restart (bounciness) said application. Sometimes, organizational silos between developers and operations staff are also also large, to effortlessly and swiftly change log levels. Hence, developers chose to exist on the safe-side and impress out 'more', than less, to the panel to be able to trace unabridged calls through the organization.
Plenty introduction. Let's have a look at some examples.
Apparently, you lot can use the INFO level to log out awarding status, like the following:
2018-09-11 08:46:26.547 INFO 8844 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path '' Only another, intriguing way to recollect about the INFO level is every bit an additional (weak) mistake case: A request was not serviced satisfactorily simply resolution details take been passed to the requestor and no proactive back up is required.
An example info would be "user login failed, wrong username or password".
2018-09-eleven 08:46:26.547 INFO 8844 --- [ master] com.marcobehler.UserService : User with id [=45] tried to login with incorrect username/password combination Yous might want to log these because the user (through layers of support) might come to ops asking questions well-nigh why they cannot apply the awarding. The ops staff will exist able to see the reason in the logfile (even although the user has already been given this info via the app front end).
Finally, there'southward two more than log levels, 'debug' and 'trace'. There have been quite a few heated discussions on the need for a trace level online, and SLF4J introduced the trace log level only in i of its later on (newer) versions - after many community requests.
Once more than, the line between these two can be blurry, just let's have a quick expect at them:
DEBUG
Advanced level detail of internal process flows. This is only turned on during investigation of specific issues and turned off again later. Depending on the logging library used, it might not exist possible to exercise this without bouncing (restarting) the application, which might not be acceptable.
2018-08-01 05:05:00,031 DEBUG - Checking uploaded XML files for valid construction [...] 2018-08-01 05:06:00,031 DEBUG - Checking uploaded XML files for valid content [...] 2018-08-01 05:07:00,031 DEBUG - Masking inputs for XML file[id=5] [...] 2018-08-01 05:08:00,031 DEBUG - Replacing [...] XML sections for file[id=v] with [...] ... 2018-08-01 05:09:00,142 DEBUG - Forwarding XML file to archiving service TRACE - More details than debug or reserved for use in specific environments
You could see the trace level equally even more detailed as the debug level, or you lot could decide to couple the trace level with certain environments, i.e. the DEV or Test environment, where developers can go crazy and feel complimentary to log out whatever they want and know that 'trace' will ever be disabled on production. (Although this could also easily be accomplished with different log configurations/profiles)
Nevertheless, if you desire to have a wait at a framework that uses the TRACE logging framework diligently, so just have a wait at the Spring Framework. When using Spring's transaction management, you will merely be able to see the real database transaction boundaries, when you enable the TRACE logging level:
2018-08-01 05:05:00,031 TRACE - Getting transaction for [com.marcobehler.BitcoinApp.mine] ... your own log statements./.. 2018-08-01 05:05:00,142 TRACE - Completing transaction for [com.marcobehler.BitcoinApp.mine] Log Files
When talking about log files, a common approach is to have split up files for carve up use cases. That ways an application volition normally log to more than ane logfile.
You could take an error.log (with a filename pattern of <appname>.<instance-proper name>.YYYYMMDD.ZZZ.mistake.log), which is consumed by monitoring and alerting organization and past operations staff. Obviously, yous'd only want entries in that log file you desire to alert on, i.e. your ERROR or WARN statements.
You could have another log file called info.log or status.log (with a filename pattern of <appname>.<instance-proper noun>.YYYYMMDD.ZZZ.status.log), which contains the aforementioned information nearly application progress or user action, as well as e.g. a trace.log file whenever you want to go crazy with logging.
When logging to separate files it makes sense to take a command line utility (like log-merger, or simply a plain old fustigate script) to merge these separate logfiles on the fly for a specific timestamp.
Imagine y'all have 2 files:
fault.log
2015-08-29 15:49:46,641 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-iv) MSC000001: Failed to start service jboss.undertow.listener.default: org.jboss.msc.service.StartException in service jboss.undertow.listener.default: Could not starting time http listener at org.wildfly.extension.undertow.ListenerService.start(ListenerService.java:150) at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.coffee:1948) at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.coffee:1881) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.coffee:617) at java.lang.Thread.run(Thread.coffee:745) Caused by: java.cyberspace.BindException: Dice Adresse wird bereits verwendet at sun.nio.ch.Internet.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:436) at sunday.nio.ch.Net.bind(Net.java:428) at sun.nio.ch.ServerSocketChannelImpl.demark(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at sunday.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.coffee:67) at org.xnio.nio.NioXnioWorker.createTcpConnectionServer(NioXnioWorker.java:182) at org.xnio.XnioWorker.createStreamConnectionServer(XnioWorker.java:243) at org.wildfly.extension.undertow.HttpListenerService.startListening(HttpListenerService.coffee:115) at org.wildfly.extension.undertow.ListenerService.start(ListenerService.java:147) ... 5 more and
status.log
2015-08-29 15:49:46,033 INFO [org.xnio] (MSC service thread 1-3) XNIO version 3.3.1.Final After running the log-merger utility, they can exist viewed, on-the-wing, equally follows:
[1] 2015-08-29 15:49:46,033 INFO [org.xnio] (MSC service thread ane-3) XNIO version 3.iii.1.Terminal [0] 2015-08-29 15:49:46,641 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-4) MSC000001: Failed to commencement service jboss.undertow.listener.default: org.jboss.msc.service.StartException in service jboss.undertow.listener.default: Could not offset http listener [0] at org.wildfly.extension.undertow.ListenerService.first(ListenerService.coffee:150) [0] at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.coffee:1948) [0] at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.coffee:1142) [0] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [0] at java.lang.Thread.run(Thread.java:745) [0] Caused by: coffee.cyberspace.BindException: Dice Adresse wird bereits verwendet [0] at sun.nio.ch.Cyberspace.bind0(Native Method) [0] at sun.nio.ch.Net.demark(Net.java:436) [0] at sun.nio.ch.Net.bind(Net.java:428) [0] at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) [0] at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.coffee:74) [0] at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67) [0] at org.xnio.nio.NioXnioWorker.createTcpConnectionServer(NioXnioWorker.coffee:182) [0] at org.xnio.XnioWorker.createStreamConnectionServer(XnioWorker.java:243) [0] at org.wildfly.extension.undertow.HttpListenerService.startListening(HttpListenerService.coffee:115) [0] at org.wildfly.extension.undertow.ListenerService.offset(ListenerService.coffee:147) [0] ... 5 more You tin can of grade choose also to log everything into ane file from the showtime.
At that place's ane caveat notwithstanding: Experience shows that developers frequently mistakenly presume that, just because log statements have a time/location correlation - which might run counter-intuitive, peculiarly as nosotros have merely talked virtually merging log files.
Here's an instance: Imagine y'all have an application which uses Hibernate. Information technology boots up to a certain betoken and and then it hangs, yous see no more log messages. The awarding simply doesn't start up.
The last log bulletin you see is the following:
2018-09-11 09:35:xix.166 INFO 14620 --- [ost-startStop-i] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit of measurement 'default' It's easy to assume that something must exist broken with JPA or Hide, merely considering that's your final log message. In fact, it could be Hide, just your awarding could as well hang trying to boot up another part/3rd-party framework, that has not gotten effectually to log stuff out, notwithstanding.
So exist wary, when y'all come up to quick conclusions, which happens frequently in high-pressure situations: When there's a astringent bug on production. Correlation in the log file through location/timestamp doesn't automatically mean it IS correlated, only that it CAN be.
MDC
To make correlation of log statements potent, there's some other important concept to know, especially if you are working with multiple distributed processes (microservices): The Mapped Diagnostic Context (MDC) or Thread Context.
Imagine you take a user-request coming in, that is routed to multiple, different microservices. When something goes wrong, a request fails, how exercise you know which log lines from which microservices correspond to that very request. Easy: You need a generated request-id that you lot want to log out with every log message.
And because you lot are lazy, you don't desire to have to manually log out that id, it should only work automatically. That's where the MDC comes in.
Somewhere in your code, in an HTTP servlet-filter, you'll have something similar this:
MDC . put ( "requestId" , "lknwelqk-12093alks-123nlkasn-5t234-lnakmwen" ); That's enough. Just ane call to a static method.
Later, in your application code, you will go along logging as usual:
logger . info ( "Hi, my name is: Slim Shady!" ); Y'all also need to configure your log library to log out the MDC variable with every log statement (see here). Which will give y'all log letters that look similar this:
[lknwelqk-12093alks-123nlkasn-5t234-lnakmwen] - Hi, my name is: Slim Shady! So it's very easy to correlate all respective log messages, you lot simply have to specify or search for the aforementioned request-id across all your log files or in your centralized logging server.
Sensitive Data
It goes without saying that you lot should avoid (read: must not) logging out sensitive data: User credentials (i.due east. passwords) or fiscal information (similar credit menu numbers etc.) or similar sensitive user details.
Depending on the complexity of your system, you might however not desire to worry nigh fixing up every individual log argument in your system (though you lot might be forced to, by ways of an inspect), but accept more of a generic solution that makes sure that certain information is masked - either partially or completely, depending on the security standards you need to attach to.
In the instance of Log4j2 for instance this means writing a custom LogEventPatternConverter, that masks log events co-ordinate to your regulations.
Apparently, a full masking solution is out of the scope of this guide, merely you can become some pointers hither and here.
Proactive Help
Another topic that isn't really covered in detail anywhere, is WHAT exactly to write inside your log statements. And that brings us to the concept of proactive help.
A corking example is the Bound Boot framework. When you are edifice a web application with Leap Kick and first it up for testing, the application will run under port 8080, so you can admission it in your browser from http://localhost:8080.
Sometimes it happens, that you have some other Spring Boot application or an older version of the aforementioned awarding already running on port 8080. Which means you cannot get-go up your application as that will fail. In older Leap Boot versions, they but logged out the "raw" exception, which looks like this:
2018-09-eleven 09:35:57.062 Error 15516 --- [ main] o.apache.catalina.cadre.StandardService : Failed to get-go connector [Connector[HTTP/1.1-8080]] org.apache.catalina.LifecycleException: Failed to commencement component [Connector[HTTP/one.1-8080]] at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:167) at org.apache.catalina.core.StandardService.addConnector(StandardService.coffee:225) at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.addPreviouslyRemovedConnectors(TomcatWebServer.coffee:256) at org.springframework.kicking.spider web.embedded.tomcat.TomcatWebServer.first(TomcatWebServer.java:198) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.startWebServer(ServletWebServerApplicationContext.java:300) at org.springframework.kick.spider web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:162) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:553) at org.springframework.boot.spider web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759) at org.springframework.kick.SpringApplication.refreshContext(SpringApplication.java:395) at org.springframework.boot.SpringApplication.run(SpringApplication.coffee:327) at org.springframework.boot.SpringApplication.run(SpringApplication.coffee:1255) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1243) at com.marcobehler.MarcobehlerRootApplication.main(MarcobehlerRootApplication.coffee:26) Caused by: org.apache.catalina.LifecycleException: Protocol handler start failed at org.apache.catalina.connector.Connector.startInternal(Connector.java:1020) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) ... xiii common frames omitted ... Developers can of course make the mental translation needed to get from "Failed to start connector" → "Another Leap Boot instance is running", but a meliorate style would be what Spring Boot offers in newer versions:
2018-09-11 17:44:49.179 Fault 24745 --- [ chief] o.due south.b.d.LoggingFailureAnalysisReporter : *************************** APPLICATION FAILED TO Commencement *************************** Clarification: Embedded servlet container failed to start. Port 8080 was already in use. Action: Identify and stop the procedure that's listening on port 8080 or configure this application to heed on another port. It could be even more than improved past adding that it could be a random process or just another (IDE) instance of your application that needs to be shutdown manually.
The primary takeaway here is nevertheless, that it makes sense to not only log out an error message wherever you can, but also hint at possible fixes/calls for action, if you can. That will save fourth dimension after, when analysing bugs and the corresponding log files.
(If yous want to meet more than examples of proactive aid, you can also have a look at the Apache Wicket framework, which comes with a 'suggested fix' for a fair amount of errors. Have a look at its source code for more details.)
Migrating Legacy Applications
To get from a normal situation to the setup discussed in this guide, i.e. unified logging practices, log libraries etc.) takes some time. Especially if you are talking multiple, existing legacy applications that you all want to migrate to a somewhat homogeneous logging strategy. Fifty-fifty disregarding all the issues of management buy-in etc, yous'll demand to exercise the migration slowly for all the apps over several releases.
One arroyo could be to put everything into the same logfile to brainstorm with and equally code and logging configuration is revisited, information technology would have all its logging revised to encounter the new criteria. Unfortunately, this is just a rough pointer, as a fully detailed migration guide is outside the scope of this guide.
Centralized Logging
As presently as yous accept more one instance of an awarding running, or even multiple instances of multiple applications (think bigger organization or microservices), the question of "how to manage those logs" gets interesting. A very simple solution could be that OPs has some sort of script, that copies all the log files from all your instances to some shared network binder, and then that developers tin access it.
A more advanced solution are centralized logging servers or stacks, like Graylog or Splunk or Elk.
Let's have a look at Graylog. There are Graylog Extended Log Format - GELF appenders for all major Coffee logging frameworks, which means you can configure due east.g. Log4j2 to directly send its log events to Graylog. Graylog will consume the log result from as many instances and applications equally needed and display them in a nice, niggling graphical UI - albeit the syntax for searching specific log events/date ranges takes some time to become used to.
Hither's a glance of what a Graylog dashboard looks like: (linked from the graylog homepage). For further information on what Graylog looks like, consult its documentation.
Takeaway: If you are big enough to need a centralized log server, do a quick comparison of all available options and then make sure of the post-obit, independent of your concluding choice:
That your developers and applications can not only write to your log server. Only that ops and dev staff actually also knows how to properly handle the UI of the log server and issue proper search queries - particularly if multiple instances of multiple applications are involved.
Deciding On The Correct (™) Way To Log
It has been quite a ride then far. We discussed a ton of different logging libraries and different ways how to log. Allow's sum everything up and - as always - think that in that location is no 'one and only right style'.
-
If you are developing a new project, start out with SLF4J + Logback or Log4j2. You volition exist fine either fashion.
-
If yous are working on a legacy project, try to migrate everything to a log facade (SLF4J) and introduce a homogeneous and unified approach to logging across all of your teams
-
Practicing the right log levels, message contents, monitoring measures etc. will take time, so don't get frustrated. Information technology's an iterative process.
-
If yous are big enough (multiple applications, multiple instances), you might want to look into a centralized log server
-
Most of all, enjoy logging!
That's it for today. If y'all have any questions or if y'all plant some errors (spelling, logical, whatever) just mail them to the comment section or e-mail service me.
Thanks for reading
Acknowledgements
3amopsguy, who was a real mentor when it came to logging, back in 2012.
There's more where that came from
I'll ship you an update when I publish new guides. Admittedly no spam, ever. Unsubscribe anytime.
Comments
Source: https://www.marcobehler.com/guides/java-logging
0 Response to "How to Read Log File and Split in Java"
Post a Comment