We assume that you have downloaded a binary package of CloverETL framework and Clover Examples or you have built one from sources. Extract the examples to the engine directory. Thus, if you have your engine in the pathtoyourengine/cloverETL/ directory, after extracting the examples to this directory, it will contain the following subdirectories: bin/, docs/, examples/, lib/ and plugins/. The examples can be found in the examples/ subdirectory.

In the directory with older examples, there is a file examples/workspace.prm with WORKSPACE parameter definition. Thus, older examples represent one project. The workspace.prm file is located in the examples/ directory along with some subdirectories (for example, data/, output/, metadata/, etc.). You need to change this parameter to proper value before running graphs.

In the directory with newest examples, such workspace.prm file is not located in this examples/ directory. Now the examples/ directory contains six subdirectories (AdvancedExamples/, ExtExamples/, JavaExamples/, and SimpleExamples/, and another two newest subdirectories: CompanyTransactionsTutorial/ and CTLFunctionsTutorial/), each of them represents an individual project. Thus, each of these six projects contains such workspace.prm file of its own along with some subdirectories of the project (for example, data-in/, data-out/, meta/, etc.). However, each of these workspace.prm files contains a definition of PROJECT parameter (instead of the older WORKSPACE parameter) along with the other parameters that are necessary to run the graphs of the project. By default, the PROJECT value is set to a dot, allowing this way to run graphs without any need to specify the location of the project.

To run graphs of both older and newer examples, you must be switched in the directory where such a workspace.prm file is located. In older examples, you need to be in the examples/ directory, in newer examples, in one of the following subdirectories: CTLFunctionsTutorial/, CompanyTransactionsTutorial/, AdvancedExamples/, ExtExamples/, JavaExamples/, or SimpleExamples/.

Note: It is a good idea to read CloverETL general documentation first.

To run graphs of older examples, you could use the run.bat or run.sh scripts included in the cloverETL.zip distribution in scripts directory. They assumes that environment variable JAVA_HOME is set to java home directory.

CloverETL requires Java (JVM) version 1.5 or newer!

Older example graph performing:

../bin/run.sh graphSimpleCopy.grf
../bin/run.bat graphSimpleCopy.grf

Or you could use old fashion way (for Linux):

  JARSPATH="../cloveretl.engine.jar:../lib/commons-logging.jar:../lib/poi-2.5.1.jar"
  JARSPATH="$JARSPATH:../lib/commons-cli-1.0.jar:../lib/jms.jar:../lib/jxl.jar"
  JARSPATH="$JARSPATH:../lib/log4j-1.2.12.jar:../lib/javolution.jar:javaExamples"
  java -cp "$JARSPATH" org.jetel.main.runGraph -plugins ../plugins graphSimpleCopy.grf

To run graphs of newer examples, you can use the clover.bat or clover.sh scripts included in the cloverETL.zip distribution in the bin/ subdirectory of the engine or execute similar commands in a similar way like in older examples.

For newer examples you can use similar commands, however, from within corresponding project subdirectory (from within the directory where one of the four workspace.prm files is located). On Linux, you can execute commands according to the following pattern (shown how it can be done from within simpleExamples/ directory, for example):

  JARSPATH="../../cloveretl.engine.jar:../../lib/commons-logging.jar:../../lib/poi-2.5.1.jar"
  JARSPATH="$JARSPATH:../../lib/commons-cli-1.0.jar:../../lib/jms.jar:../lib/jxl.jar"
  JARSPATH="$JARSPATH:../../lib/log4j-1.2.12.jar:../../lib/javolution.jar:trans"
  java -cp "$JARSPATH" org.jetel.main.runGraph -plugins ../../plugins graph/graphSimpleCopy.grf

For more detailed information of how newer examples can be executed, expand the Examples category on the left side of this page and consult one of the items. You can use the examples index as well.

If you want to use some command line arguments, you can do it in the following way (the arguments should be typed before the graph name):

  JARSPATH="../../cloveretl.engine.jar:../../lib/commons-logging.jar:../../lib/poi-2.5.1.jar"
  JARSPATH="$JARSPATH:../../lib/commons-cli-1.0.jar:../../lib/jms.jar:../lib/jxl.jar"
  JARSPATH="$JARSPATH:../../lib/log4j-1.2.12.jar:../../lib/javolution.jar:trans"
  java -cp "$JARSPATH" org.jetel.main.runGraph -plugins ../../plugins -loglevel INFO graph/graphSimpleCopy.grf

Clover engine offers runGraph utility – located in org.jetel.main package, runGraph can be executed from command line you see above. If you type runGraph without parameters you can see help:

Usage: runGraph [-(v|cfg|P:|tracking|info|plugins|pass|stdin|loghost|checkconfig)] <graph definition file>
Options:
-v                      be verbose - print even graph layout
-P:<key>=<value>        add definition of property/parameter to global graph's property list
-cfg <filename>         load definitions of properties/parameters from specified file
-tracking <seconds>     how frequently output the graph processing status
-info                   print info about Clover library version
-plugins                directory where to look for plugins/components
-pass                   password for decrypting of hidden connections passwords
-stdin                  load graph definition from STDIN instead from file
-loghost                define host and port number for socket appender of log4j 
                        (log4j library is required); i.e. localhost:4445
-checkconfig            only check graph configuration
-skipcheckconfig        skip checking of graph configuration
-noJMX                  turn-off sending of tracking info through JMX bean (may help performance)
-loglevel <loglevel>    ALL | TRACE | DEBUG | INFO | WARN | ERROR | FATAL | OFF
-logcfg <filename>	load log4j properties from specified file; if not specified, "log4j.properties" should be in classpath
-config <filename>	load default engine properties from specified file
-classpath <paths|jars> Paths and/or jars where clover can find transformation/generator/JMS processor classes. Use ";" as separator. It's not necessary to specify this clover parameter if all classes are in java classpath. (since 2.8)
-nodebug                Turn off all runtime debugging (for example edge debugging)
-debugdirectory <foldername> directory dedicated to store temporary debug data

Note: <graph definition file> can be either local filename or URL of local/remote file

CloverETL uses Apache Commons Logging as its logging layer. Commons Logging is a thin wrapper around one of three logging subsystems:

• Java's logging package (present from JVM 1.4)
• Apache Log4J (http://logging.apache.org/log4j/)
• Very simple logging subsystem which is part of Commons Logging

It is recommended to use Log4J package together with CloverETL as it gives greatest flexibility. Log4J is activated simply by adding its .jar into Clover's CLASSPATH when executed. CloverETL already contains configuration file for Log4J which ensures proper display of messages CloverETL from version 2.0 contains Log4J bundled with it.

Log4J properties:

  # An example log4j configuration file that outputs to System.out.  The
  # output information consists of relative time, log level, thread
  # name, logger name, nested diagnostic context and the message in that
  # order.
 
  # For the general syntax of property based configuration files see the
  # documenation of org.apache.log4j.PropertyConfigurator.
 
 
  #this options sets the finest level of messages being output
  #you may set it to INFO in order to eliminate debugging messages
  #log level option are TRACE, DEBUG, INFO, WARN, ERROR and FATAL
  log4j.rootLogger=DEBUG, A1
 
  # A1 is set to be a ConsoleAppender which outputs to System.out. 
  log4j.appender.A1=org.apache.log4j.ConsoleAppender
  #log4j.appender.A1=org.apache.log4j.net.SocketAppender
  #log4j.appender.A1=org.apache.log4j.FileAppender
 
  # A1 uses PatternLayout.
  log4j.appender.A1.layout=org.apache.log4j.PatternLayout
  #log4j.appender.A1.RemoteHost=localhost
  #log4j.appender.A1.Port=4560
 
  # The conversion pattern uses format specifiers. You might want to
  # change the pattern an watch the output format change.
  #log4j.appender.A1.layout.ConversionPattern=%-4r %-5p [%t] %37c %3x - %m%n
  log4j.appender.A1.layout.ConversionPattern=%-5p [%t] - %m%n

  #this options sets the finest level of messages being output
  #you may set it to INFO in order to eliminate debugging messages
  log4j.rootLogger=DEBUG, A1 , A2
 
  # A1 is set to be a ConsoleAppender which outputs to System.out. 
  log4j.appender.A1=org.apache.log4j.ConsoleAppender
 
  # A1 uses PatternLayout.
  log4j.appender.A1.layout=org.apache.log4j.PatternLayout
  log4j.appender.A1.layout.ConversionPattern=%-5p [%t] - %m%n
 
  # A2 is set to be a FileAppender which outputs to network socket. 
  log4j.appender.A2=org.apache.log4j.FileAppender
 
  # A2 sends traffic to following host & port
  log4j.appender.A2.File=log.out
  log4j.appender.A2.Append=true
 
  # A2 uses PatternLayout.
  log4j.appender.A2.layout=org.apache.log4j.PatternLayout
  log4j.appender.A2.layout.ConversionPattern=%-5p [%t] - %m%n

Clover engine is running on JVM (Java Virtual Machine) version 1.5. Java has a lot of parameters and settings you can see on java pages or if you run java program without parameters. Following list shows very interesting settings you can use.

• except for very small data sets, JVM should be run in server mode – command line option “-server”
• maximum heap size (-Xmx option) should be set to appropriately high number (but still leaving some memory for operating system)
• on multi-CPU machines, the parallel garbage collector should be enabled “-XX:+UseParallelGC”

Clover engine internal settings/defaults are stored in defaultProperties file located in engine's .jar, parameters are widely used by Clover when instantiating components and have impact on

• engine's performance
• engine's memory footprint !!

Any change to this file affects engine when next executed (through runGraph utility), more informations are on default constants page. If you need to modify these settings, you may just create a local file with new definitions and use the “-config” option when executing Clover engine from command line to load them.