== Where is the test code? ==

   1. The code is on cp3wks05 in the directory /Users/madgraph/TestMG 
   1. He is also available via CVS: ''cvs co !TestMG'' . He is suppose to be place on same directory as MG_ME

== How do you execute the test? ==

you can run two different types of tests: "fast" and "full" (see next section for details).
The main script that controls the various tests is a Python script called test.py.
Here are some examples of commands you can type to perform the tests:

   *    $>./test.py fast all         (to do all available checks in the "fast" mode)
   *    $>./test.py fast MW      (to check only MadWeight in the "fast" mode)
   *    $>./test.py full all          (to do all available checks in the "full" mode)    
   *    $>./test.py fast MW      (to check MadWeight MadWeight in the "full" mode)
   *    $>./test.py MW/NAME  (to launch only the fast test  NAME in MW) 


== What is the difference between the "fast" mode and the "full" mode? ==

These are two different levels of test, for each module in MadGraph: 
   * the "FAST" mode is intended to be rapid. It should not last much more than 10 minutes or so. You can use it to check whether 
     some modifications in a specific module (e.g. MadOnia) do not interfere with the good-working of other modules (e.g. MadDipol). 
   * the "FULL" mode is intended to check that ''all the results''  you can produce with a specific updated  module is in agreement with the results from the previous version of this module.

Essentially, if you update a specific module ''mymod'', you can run the test script in "fast" mode to test all the modules, 

 $>./test.py fast all

and in "full" mode to test specifically ''mymod''

 $>./test.py full mymod

NOTE: for the moment only madonia has is own full test.

== Structure of the code ==

All required information to test a specific module is contained in one given
directory located in   /Users/madgraph/TEST. This directory should be named
as '''test_????'''. For a specific test in this module, the relevant files are recorded 
in a sub-directory '''fast_???''' or '''full_???'''. Each of these test directories contains

   1. all input Cards or files required to perform the test,
   1. the file '''move_files_level1''' 
       which specifies which input files should be copied at level 1 (i.e. before the execution of any script) 
       and in which directory they must be copied in, 
   1. the file '''routine_level1''' specifies which scripts should be executed first (usually ./bin/newprocess) 
   1. the file '''move_files_level2''' 
       which specifies which input files should be copied at level 2 (i.e. after the execution of the scripts mentioned in the file routine_level1) 
       and in which directory they must be copied in, 
   1. the file '''routine_level2''' specifies which scripts should be executed at level 2 (such as ./bin/generate_events) 
   1. the file '''dir_test.py''' which specifies how to extract the results to be compared with the expected one (see section [http://cp3wks05.fynu.ucl.ac.be/twiki/bin/view/Software/TestProgram#How_to_write_dir_test_py how define dir_test.py])
   1. the file '''expected_results.txt''' which contains the reference results. 

Note: the above files can also be placed in the directory '''test_????'''. They are then used in case there is no such file in the '''fast_????''' (or '''full_????''') directory.

== How does it work? ==
   1. The script test.py copies the directory "Template" into "Auto_test_???" 
   1. It copies the input files according to the prescription given in '''move_files_level1'''
   1. It launches executables (such as new_process) according to the prescription given in '''routine_level1'''.
       The paths are expected to be given from the main directory of the Template copy.
   1. It copies the input files according to the prescription given in '''move_files_level2'''
   1. It launches executables (such as generate_events) according to the prescription given in '''routine_level2'''.
       The paths are expected to be given from the main directory of the Template copy. 
   1. A python code collects the results.
   1. These results are compared to the expected one written in '''expected_results.txt'''


== How to implement a new test?  ==


   1. First create a new directory in TEST for your module: $>mkdir test_MYMOD
   1. $>cd test_MYMOD
   1. Create a directory for a specific test $> mkdir fast_NAME or $>mkdir full_NAME
   1. $> cd fast_NAME 
   1. Define the six files  '''move_files_level1''',  '''move_files_level2''', '''routine_level1''' ,  '''routine_level2''',  
 '''dir_test.py''', '''expected_results.txt''' and import a copy of  all input cards (or files) in this directory.


=== A practical way to develop all those things... ===

This is only a proposition of the different step used to devellop a new step
   1. Create your new test directory
   1. The defines the four files  '''move_files_level1''',  '''move_files_level2''', '''routine_level1''' ,  '''routine_level2''' (this should be easy)
   1. Copy the default files /Users/madgraph/TEST/Python_lib/dir_test_default.py in your directory '''fast_????''' (respectively '''full_????''')
   1. Change his name in dir_test.py
   1. Launch the test: (in TEST)$> ./test.py test_????/fast_????  (respectively for full)
   1. Edit the value of actif and put this value to zero (avoid the overwriting of the Template copy and the execution of the different test script)
   1. Implement your python (or external) routine and test this one with  (in TEST)$> ./test.py test_MYMOD/fast_MYTEST 
   1. When the last step  works correctly, remove the line actif=0 of the file dir_test.py
   1. Copy the file test_result.dat in 'expected_result.dat': (in fast_???) $> cp test_results.dat expected_results.dat
   1. add the third column (precision requirment) in the file expected_results.dat
   1. launch completly your test to be sure that everithing is fine. (in TEST)$> ./test.py test_MYMOD/fast_MYTEST
   1. store your module on the cvs

=== How to write input files? ===
==== How to write  "move_files_level1" and  "move_files_level2" ? ====

 This files contains two columns.
   1. The first one indicates the name of the files to copy (path starting from directory '''fast_????''' (or '''full_????''')
   1. The second indicates where to copy the files (path starting from the main directory of the Template copy)
All lines begining with a # not taking into account

==== How to write "routine_level1" and "routine_level2"? ====

Those files are simply batch files. They will be executed from the main directory of the Template copy.
A good way to code those files will be to redirect all the output.

==== How to write "dir_test.py" ? ====

   1. First copy the file /Users/madgraph/TEST/Python_lib/dir_test_default.py in your fast_???? (or full_????) directory
   1. Change his name in dir_test.py
   1. Then follows the instruction inside (All the line starting with '#>>' indicates an action to perform). 

This files indicates how to use an external programs and what you have to do if you choose to implement your python code. 

==== How to write "expected_results.txt" ?  ====
This files must contains three columns
   1. the first one is of string symbolizing the result (this string must be unique)
   1. The second one is the expected value 
   1. The third one is the difference pourcentage authorize without warning. 
      a. if you want to compare string put 's'
      a. if you want to have exact value put '0'

==== How to write "test_results.txt" ?  ====
If you choose to develop your own code to collect data, you have to write explicitly the file 'test_result.txt' (in the directory '''fast_????''' or '''full_????''')
This files must contains two columns
   1. The first one is of string symbolizing the result (this string must be unique)
   1. The second one is the related value


== Discussion/Idea?  ==




-- Main.OlivierMattelaer - 18 Nov 2008