Vivek Agarwal’s Portal/Java Blog

An IBM Gold Consultant’s weblog about IBM, Lotus, WebSphere, J2EE, IT Processes, and other IT technologies

Missing log4j log files with DailyRollingFileAppender when they should roll over?

Posted by Vivek Agarwal on February 9, 2008


Yes, I am back with a blog entry after quite a gap here! I have been knocked out sick for 5 weeks now and I am sick of being sick! Better now and hoping to be back at a 100% soon. Enough excuses – onto the subject of this blog entry. We use Log4j’s DailyRollingFileAppender in a WebSphere Portal environment to generate daily log files for various custom portlets that we have deployed. This worked just fine on some WPE v6 test servers with multiple portlets logging to the same log file, and the daily rollover occurring every midnight. However, on some other WPE v6 test servers running the exact same applications, while logging did work the daily rollover did not. Instead of us having multiple log files with date timestamps in their names, we would only have today’s log file – the previous log files would simply disappear! This issue results from the fact that the DailyRollingFileAppender (DRFA) tries to rename the log file during rollover. However, when the rename fails, it ends up overwriting the log file, hence losing information. On checking the application server’s SystemErr.log, we found an error each midnight similar to -

log4j:ERROR Failed to rename [logfilename] to logfilename.2008.01.11].

This appears to be a fairly common issue with log4j 1.2.x in Java application servers running on Windows. While there is no good resolution for this issue there is a workaround that works pretty well.

We ended up getting the DatedFileAppender which is a log4j extension that is publicly available at http://minaret.biz/tips/datedFileAppender.html. DatedFileAppender avoids the need for renaming the log file by simply creating the log file initially with the date in the file name – this is a simple workaround for the problem that we were experiencing. Switching from the DailyRollingFileAppender to DatedFileAppender was fairly seamless.  In addition to installing the jar in WebSphere Portal, we had to switch a few lines of our log4j configuration. We changed the following lines -

log4j.appender.logFileAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.logFileAppender.DatePattern=’.’yyyy-MM-dd
log4j.appender.logFileAppender.File=MyApps.log

to

log4j.appender.logFileAppender=biz.minaret.log4j.DatedFileAppender
log4j.appender.logFileAppender.Directory=.
log4j.appender.logFileAppender.Prefix=MyApps_
log4j.appender.logFileAppender.Suffix=.log
log4j.appender.sametimeSyncAppender.Period=month # this line is only needed if the period is not a day

About these ads

18 Responses to “Missing log4j log files with DailyRollingFileAppender when they should roll over?”

  1. jtcunningham said

    Interesting – thanks…

    But i also have a more basic question about getting log4j to be used across all portlets.
    I’ve put log4.jar in the PortalServer/shared/app & the log4j.properties all over the place, but it doesn’t seem to log anything?
    Any ideas??

    many thanks

    joe

  2. Vivek Agarwal said

    Hmm! You should need to only put log4j.jar and log4j.properties in ${PortalServer}\shared\app. Possibly you could be encountering either one of these issues –

    1) The logs are being generated but in a different folder than what you expect if you DO NOT have absolute paths for the log files in the log4j.properties.
    2) There is a log4j.properties in ${PortalServer}\installedApps\WPSPortlet_arketWatch_PA_dlwyxxi.ear\MarketWatch.war\WEB-INF\classes\etc. You may need to delete that file.

    Finally you might look in SystemErr.log for any log4j errors.

  3. Jayanth said

    Hi,

    We have a problem when using the Log4j: It seems that not
    all log statements are writing to the file and It is Multithreaded environment.

    In Detailed:
    1 All the messages are writing at the Database
    2. By using Sys Prints – we could able see all the messages.
    3. But not able to see all the messages in the log file.
    4. It is Multithreaded env and Springs and Hybernate configuration

    Example:
    We are dealing with around 11,000 Messages and all the 11,000 messages are getting updated at the Database,Could able to see 11,000 SysOutPut messages but 100 messages are missing if we trying through the log4j.

    Any help would be highley appreciated.

    Please revert for any questions/clarifications.

    Regards,
    Jayanth

  4. Julpikar Jul-akmad said

    Hi,

    In my case sometimes it was able to create daily rollover sometimes not but the data still present in current logfile (data not lost). We are using resin 1.2.

    Regards,
    Ficks

  5. Kumarapandian said

    Hi Vivek

    Can you let us know what is the root cause for the old files being deleted / missing when a rolled over is happening. Is there any other way to solve the problem without using the DatedFileAppender.

    Thanking You
    Kumarapandian

  6. Shivan Koendan Panday said

    Hello Kumarapandian,

    see http://marc.info/?l=log4j-dev&m=122787435708955&w=2 vor more info.

    Shivan

  7. Islam A Khan said

    Hi,

    If log4j is thread safe then what is the actual reason for locking of the log file so that why it couldn’t roll over.

    In my application midnight is the time when no one uses that application and log4j releases lock before renaming for rolling. But nothing is helping me out for getting the reason.

    Thanks

  8. Rohini Kumar Chowdary Podile said

    Hi,
    We are working on a multi threaded J2EE application, In my case new log file getting generated each day by renaming old one wiht the prvious date. But some log statements going to old (renamed one) file and some going to new file. Can any one explaing how is it happening? Thank You!!!!

  9. ajin said

    I need my log file to rollover yearly i used yyyy in datepattern but it gives me invalid periodicity error can u help!!

  10. The issue has to do with windows file locks actually… make sure only one process controls a given log configuration process and the regular log4j file appenders will work just fine.

  11. Venkat said

    It’s good solution. I have the same issue and it worked.

  12. Mary said

    Hi,

    Can you please let me know how to achieve the following in log4j.xml rather than log4j.properties file. How should “sametimeSyncAppender” be declared?

    log4j.appender.sametimeSyncAppender.Period=month # this line is only needed if the period is not a day

  13. Fateh said

    Dear All,

    is it possible with DatedFileAppender to log avery hour?, like DailyRollingFileAppender with DatePattern=’.’yyyy-MM-dd_HH

    I am getting this warning : log4j:WARN No such property [period] in biz.minaret.log4j.DatedFileAppender.

    Thank you

  14. akjnk12 said

    new version 1.0.3 is here:
    http://sourceforge.net/projects/log4j-dfa/files/

    new property:

    # The schedule type: MONTHLY, WEEKLY, DAILY (default), HOURLY, CUSTOM
    log4j.appender.T.ScheduleType=DAILY

  15. Fateh said

    Thank you for the update,

    I have tested it and it’s opening a new file every minute within the same hour.

    Any suggestion ?

    Thank you

  16. Patrick said

    my problem is that this works on one server and not the next, with both being at the same java version.

  17. Thomas said

    Hi,
    very good solution,

    Is there any plans to support a MaxBackupIndex property or if not, any suggested work arounds?
    thanks

Sorry, the comment form is closed at this time.

 
Follow

Get every new post delivered to your Inbox.

Join 43 other followers

%d bloggers like this: