register
other register

Saturday, January 26, 2008

Ant, Log4J, JUnit, Eclipse


1. Technical Specification


The target application environment Java compiler's version is Java 1.5.
Create a project in Eclipse.
Need to connect a database (e.g. Oracle).
Should log error message using Log4J.
Delivery the project as a jar.


2. Procedures


2.1 Create User Library for Log4J and JDBC driver (oracle-jdbc)

2.2 Add these two user libraries to the project build path

2.3 Establish The Application Environment in Eclipse

2.3.1 Create file.properties file as a property file for the application to read

2.3.2 Create a log4j.properties file for log4j to read

2.4 Follow the instructions to set up JUnit available to Ant

2.5 Create build.xml file



2.1 Create User Libraries


1. In Eclipse, Window > Preferences > Java > Build Path > User Libraries.
2. Click New button, and enter the 'user library name'.
3. Click Add JARs... button, and select the jar files to be added to this user library.
4. Click OK button.

Do the above steps for Log4J, and oracle-jdbc


2.2 Add User Libraries to the project


1. Right click the project, and select properties (or Alt+Enter).
2. Select Java Build Path > Libraries, and click Add Library... button
3. Select User Library, and click next> button
4. Select lo4j, and oracle-jdbc (the two we just created in the previous step).
5. Click OK to add these user libraries to the project environment.

Doing this can make sure these libraries are in the project classpath.



2.3 Establish Project Structure

1. Create build directory, and its sub directories (classes and jar)
md build/classes
md build/jar
2. Create lib directory, and copy log4j and oracle-jdbc jars in.

2.3.1 Create file.properties file as a property file for the application to read
name1 = value1
name2 = value2

2.3.2 In the src directory, create a log4j.properties file for log4j to read. Below is a sample

For more info about log4j, please look at: How to use log4j logging API and Log4J Manual

************************************************
# Log levels
# Uncomment the following line to enable full loggin for every class
#log4j.rootLogger=trace, stdout, R
log4j.logger.pma=error, stdout, R

# Console appender configuration
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n

# Rolling File Appender
log4j.appender.R=org.apache.log4j.RollingFileAppender
# Path and file name to store the log file.
log4j.appender.R.File=./logs/error.log
log4j.appender.R.MaxFileSize=500KB
# Keep one backup file
log4j.appender.R.MaxBackupIndex=1
# Rolling File Appender layout
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%d - %c - %p - %m%n

****************************************************


2.4 Follow the instructions to set up JUnit available to Ant

2.5 Create build.xml file

For more information about Ant and its build.xml file, please have a look at Tutorial: Hello World with Ant

Below is a sample build.xml file:

********************************************************

<?xml version="1.0"?>

<project name="pma" basedir="." default="main">
<property name="src.dir" value="src">
</property>
<property name="build.dir" value="build">
</property>
<property name="classes.dir" value="${build.dir}/classes">
</property>
<property name="jar.dir" value="${build.dir}/jar">
</property>
<property name="main-class" value="pma.PMADAO_Split">
</property>
<property name="lib.dir" value="lib">
</property>
<property name="report.dir" value="${build.dir}/junitreport">
</property>

<path id="classpath">
<fileset dir="${lib.dir}" includes="**/*.jar">
</fileset>
</path>

<target name="clean">
<delete dir="${build.dir}">
</delete>
</target>

<target name="compile">
<mkdir dir="${classes.dir}"/>
<javac srcdir="${src.dir}" destdir="${classes.dir}"
classpathref="classpath"
source="1.5"
target="1.5"
compiler="javac1.5"/>
<copy todir="${classes.dir}">
<fileset dir="${src.dir}" excludes="**/*.java">
</fileset>
</copy>
</target>

<target name="jar" depends="compile">
<mkdir dir="${jar.dir}"/>
<jar destfile="${jar.dir}/${ant.project.name}.jar" basedir="${classes.dir}" update="true">

<zipfileset src="${lib.dir}/ojdbc14.jar"/>
<zipfileset src="${lib.dir}/log4j-1.2.15.jar"/>

<manifest>
<attribute name="Main-Class" value="${main-class}"/>
</manifest>
</jar>
</target>

<target name="run" depends="jar">
<java fork="true" classname="${main-class}">
<classpath>
<path refid="classpath">
</path>
<path id="application" location="${jar.dir}/${ant.project.name}.jar">
</path>
</classpath>
</java>
</target>

<target name="clean-build" depends="clean,jar">
</target>
<target name="main" depends="clean,run">
</target>

<target name="junit" depends="jar">
<mkdir dir="${report.dir}" />
<junit printsummary="yes">
<classpath>
<path refid="classpath" />
<path refid="application" />
</classpath>
<formatter type="xml" />
<batchtest fork="yes" todir="${report.dir}">
<fileset dir="${src.dir}" includes="*Test.java" />
</batchtest>
</junit>
</target>

<target name="junitreport">
<junitreport todir="${report.dir}">
<fileset dir="${report.dir}" includes="TEST-*.xml"/>
<report todir="${report.dir}"/>
</junitreport>
</target>

</project>

**********************************************




In the task "jar", the following code can make sure the created jar (pma.jar) file contains the following libraries (log4j, and oracle-jdbc jars) in an extracted format. If the libraries (log4j, and oracle-jdbc jars) in the created jar (pma.jar) are in its jar format, then the application won't find the libraries in its classpath when running.

<zipfileset src="${lib.dir}/ojdbc14.jar"/>
<zipfileset src="${lib.dir}/log4j-1.2.15.jar"/>


In the task "compile", we have to add:

source="1.5" target="1.5" compiler="javac1.5" to make the application java 1.5 compatible on the target environment, otherwise it will complains about "class version something" when running.








Tuesday, January 15, 2008

crontab

Weblog Analysis with Webalizer

Webalizer is a web log analysis tool which runs very fast. You can download Webalizer Here.

By default

>./webalizer logfile

will process only one log file. If all the logs are in one file, then it is ok. But for logs which rotate on a daily basis, you have to configure webalizer to do so.

Here is my webalizer.conf, which should be at the webalizer command directory.

**********************************************
...
LogType clf
OutputDir /outputdirectory
HistoryName webalizer.hist
Incremental yes
IncrementalName webalizer.current
...
**********************************************


Create a script foo.sh like below will do the business:

**********************************************
#!/bin/sh

for file in /logfiles/*
do
/pathtowebalizer/webalizer $file
done
exit 0
**********************************************

Friday, January 11, 2008

Java Logging using java.util.logging

Java's logging facility (see Sun's overview and API ) has two parts : a configuration file, and an API for using logging services. It is a good tool, and is perfectly fine for simple and moderate logging needs. (For complex logging needs, you might consider the log4j tool as an alternative.)

Log entries can be sent to these destinations, as either simple text or as XML :

  • the console
  • a file
  • a stream
  • memory
  • a TCP socket on a remote host
The Level class defines seven levels of logging enlightenment :
  • FINEST, FINER, FINE, CONFIG, INFO, WARNING, SEVERE
  • ALL and OFF are defined values as well
Here is one style of using these Levels in code, which may be modified as desired :
  • upon startup, use CONFIG to log configuration parameters
  • during normal operation, use INFO to log high-level "heartbeat" information
  • when bugs or critical conditions occur, use SEVERE
  • debugging information might default to FINE, with FINER and FINEST used occasionally, according to taste.
There is flexibility in how logging levels can be changed at runtime, without the need for a restart :
  • simply change the configuration file and call LogManager.readConfiguration.
  • or, change the level in the body of your code, using the logging API ; for example, one might automatically increase the logging level in response to unexpected events
Levels are attached to these items :
  • an originating logging request (from a single line of code)
  • a Logger (usually attached to the package containing the above line of code)
  • a Handler (attached to an application)
The flow of execution for a particular logging request usually proceeds as follows :

logging request of some level is made to logger attached to current package
if the request level is too low for that package's logger {
discard it
}
otherwise {
cycle through all handlers {
if the request level is too low for that handler {
discard it
}
otherwise {
log the request
}
}
}

Here is an example of a logging configuration file :

# Properties file which configures the operation of the JDK
# logging facility.

# The system will look for this config file, first using
# a System property specified at startup:
#
# >java -Djava.util.logging.config.file=myLoggingConfigFilePath
#
# If this property is not specified, then the config file is
# retrieved from its default location at:
#
# JDK_HOME/jre/lib/logging.properties

# Global logging properties.
# ------------------------------------------
# The set of handlers to be loaded upon startup.
# Comma-separated list of class names.
# (? LogManager docs say no comma here, but JDK example has comma.)
handlers=java.util.logging.FileHandler, java.util.logging.ConsoleHandler

# Default global logging level.
# Loggers and Handlers may override this level
.level=INFO

# Loggers
# ------------------------------------------
# Loggers are usually attached to packages.
# Here, the level for each package is specified.
# The global level is used by default, so levels
# specified here simply act as an override.
myapp.ui.level=ALL
myapp.business.level=CONFIG
myapp.data.level=SEVERE

# Handlers
# -----------------------------------------

# --- ConsoleHandler ---
# Override of global logging level
java.util.logging.ConsoleHandler.level=SEVERE
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter

# --- FileHandler ---
# Override of global logging level
java.util.logging.FileHandler.level=ALL

# Naming style for the output file:
# (The output file is placed in the directory
# defined by the "user.home" System property.)
java.util.logging.FileHandler.pattern=%h/java%u.log

# Limiting size of output file in bytes:
java.util.logging.FileHandler.limit=50000

# Number of output files to cycle through, by appending an
# integer to the base file name:
java.util.logging.FileHandler.count=1

# Style of output (Simple or XML):
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter

An application should likely centralize the naming policy for Loggers, since switching naming styles becomes a simple edit in one method, instead of a large number of edits spread throughout the application. Reasons to alter the naming policy might include :

  • in a shared environment, adding an application name to the Logger name can help distinguish one application's entries from others
  • adding application version information might be beneficial for debugging
There is a common complaint with the JDK logging config file : a Handler cannot be restricted to a specific Logger. That is, Handlers defined in the config file are global, in the sense that they will be attached to all Loggers. This is particularly annoying in a server environment, since different applications will, by default, output logging statements into the same file. This behavior can be easily changed, but only by using the JDK logging API. Many would prefer to do this directly in the config file.

Here is an example of using the logging API :

package myapp.business;

import java.util.logging.*;

/**
* Demonstrate Java's logging facilities, in conjunction
* with a logging config file.
*/

public final class SimpleLogger {

public static void main(String argv[]) {
SimpleLogger thing = new SimpleLogger();
thing.doSomething();
}

public void doSomething() {
//Log messages, one for each level
//The actual logging output depends on the configured
//level for this package. Calls to "inapplicable"
//messages are inexpensive.
fLogger.finest("this is finest");
fLogger.finer("this is finer");
fLogger.fine("this is fine");
fLogger.config("this is config");
fLogger.info("this is info");
fLogger.warning("this is a warning");
fLogger.severe("this is severe");

//In the above style, the name of the class and
//method which has generated a message is placed
//in the output on a best-efforts basis only.
//To ensure that this information is always
//included, use the following "precise log"
//style instead :
fLogger.logp(Level.INFO, this.getClass().toString(), "doSomething", "blah");

//For the very common task of logging exceptions, there is a
//method which takes a Throwable :
Throwable ex = new IllegalArgumentException("Some exception text");
fLogger.log(Level.SEVERE, "Some message", ex);

//There are convenience methods for exiting and
//entering a method, which are at Level.FINER :
fLogger.exiting(this.getClass().toString(), "doSomething");

//Display user.home directory, if desired.
//(This is the directory where the log files are generated.)
//System.out.println("user.home dir: " + System.getProperty("user.home") );
}

// PRIVATE //

//This logger will inherit the config of its parent, and add
//any further config as an override. A simple style is to use
//all config of the parent except, perhaps, for logging level.

//This style uses a hard-coded literal and should likely be avoided:
//private static final Logger fLogger = Logger.getLogger("myapp.business");

//This style has no hard-coded literals, but forces the logger
//to be non-static.
//private final Logger fLogger=Logger.getLogger(this.getClass().getPackage().getName());

//This style uses a static field, but hard-codes a class literal.
//This is probably acceptable.
private static final Logger fLogger =
Logger.getLogger(SimpleLogger.class.getPackage().getName());
}


Example Run 1 , using the config file defined above :

>java -Djava.util.logging.config.file=C:\Temp\logging.properties
-cp . myapp.business.SimpleLogger

ConsoleHandler is configured to show only SEVERE messages :

Jan 8, 2003 10:43:47 AM myapp.business.SimpleLogger doSomething
SEVERE: this is severe
Jan 8, 2003 10:43:47 AM myapp.business.SimpleLogger doSomething
SEVERE: Some message
java.lang.IllegalArgumentException: Some exception text
at myapp.business.SimpleLogger.doSomething(SimpleLogger.java:39)
at myapp.business.SimpleLogger.main(SimpleLogger.java:13)

While the FileHandler shows ALL that is sent to it :

Jan 8, 2003 10:43:46 AM myapp.business.SimpleLogger doSomething
CONFIG: this is config
Jan 8, 2003 10:43:47 AM myapp.business.SimpleLogger doSomething
INFO: this is info
Jan 8, 2003 10:43:47 AM myapp.business.SimpleLogger doSomething
WARNING: this is a warning
Jan 8, 2003 10:43:47 AM myapp.business.SimpleLogger doSomething
SEVERE: this is severe
Jan 8, 2003 10:43:47 AM class myapp.business.SimpleLogger doSomething
INFO: blah
Jan 8, 2003 10:43:47 AM myapp.business.SimpleLogger doSomething
SEVERE: Some message
java.lang.IllegalArgumentException: Some exception text
at myapp.business.SimpleLogger.doSomething(SimpleLogger.java:39)
at myapp.business.SimpleLogger.main(SimpleLogger.java:13)

Example Run 2, showing the ConsoleHandler output, using the logging.properties file which ships with the JDK, and which logs only INFO level and above :

>java -cp . myapp.business.SimpleLogger

Jan 8, 2003 10:52:31 AM myapp.business.SimpleLogger doSomething
INFO: this is info
Jan 8, 2003 10:52:31 AM myapp.business.SimpleLogger doSomething
WARNING: this is a warning
Jan 8, 2003 10:52:31 AM myapp.business.SimpleLogger doSomething
SEVERE: this is severe
Jan 8, 2003 10:52:31 AM class myapp.business.SimpleLogger doSomething
INFO: blah
Jan 8, 2003 10:52:31 AM myapp.business.SimpleLogger doSomething
SEVERE: Some message
java.lang.IllegalArgumentException: Some exception text
at myapp.business.SimpleLogger.doSomething(SimpleLogger.java:39)
at myapp.business.SimpleLogger.main(SimpleLogger.java:13)