Monday, March 16, 2015

spark 1.3 log settings

I do not like all the log info being dumped to the console when I start the spark shell or other programs. I've changed the log settings in conf/log4j.properties to the below values:

log4j.rootCategory=INFO,app
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
log4j.appender.app=org.apache.log4j.FileAppender
log4j.appender.app.file=app.log
log4j.appender.app.append=true
log4j.appender.app.layout=org.apache.log4j.PatternLayout
log4j.appender.app.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n


Each time I run, a file app.log is created that captures the log output. This setting appends to the previous log file but you could change that by setting append=false.

No comments:

Post a Comment