This Pages concerns The Test Suite for MG4

This page is therefore not usefull for MG5 at all where the test suite is much more evolve than the one described in this page.

Where is the test code?

ask by email.

How do you execute the test?

you can run two different types of tests: "fast" and "full" (see next section for details). The main script that controls the various tests is a Python script called test.py. Here are some examples of commands you can type to perform the tests:

• $>./test.py fast all (to do all available checks in the "fast" mode)
• $>./test.py fast MW (to check only MadWeight in the "fast" mode)
• $>./test.py full all (to do all available checks in the "full" mode)
• $>./test.py fast MW (to check MadWeight MadWeight in the "full" mode)
• $>./test.py MW/NAME (to launch only the fast test NAME in MW)

What is the difference between the "fast" mode and the "full" mode?

These are two different levels of test, for each module in MadGraph:

• the "FAST" mode is intended to be rapid. It should not last much more than 10 minutes or so. You can use it to check whether some modifications in a specific module (e.g. MadOnia) do not interfere with the good-working of other modules (e.g. MadDipol).
• the "FULL" mode is intended to check that all the results you can produce with a specific updated module is in agreement with the results from the previous version of this module.

Essentially, if you update a specific module mymod, you can run the test script in "fast" mode to test all the modules,

$>./test.py fast all

and in "full" mode to test specifically mymod

$>./test.py full mymod

NOTE: for the moment only madonia has is own full test.

Structure of the code

All required information to test a specific module is contained in one given directory located in /Users/madgraph/TEST. This directory should be named as test_????. For a specific test in this module, the relevant files are recorded in a sub-directory fast_??? or full_???. Each of these test directories contains

1. all input Cards or files required to perform the test,
2. the file move_files_level1

which specifies which input files should be copied at level 1 (i.e. before the execution of any script) and in which directory they must be copied in,

1. the file routine_level1 specifies which scripts should be executed first (usually ./bin/newprocess)
2. the file move_files_level2

which specifies which input files should be copied at level 2 (i.e. after the execution of the scripts mentioned in the file routine_level1) and in which directory they must be copied in,

1. the file routine_level2 specifies which scripts should be executed at level 2 (such as ./bin/generate_events)
2. the file dir_test.py which specifies how to extract the results to be compared with the expected one
3. the file expected_results.txt which contains the reference results.

Note: the above files can also be placed in the directory test_????. They are then used in case there is no such file in the fast_???? (or full_????) directory.

How does it work?

1. The script test.py copies the directory "Template" into "Auto_test_???"
2. It copies the input files according to the prescription given in move_files_level1
3. It launches executables (such as new_process) according to the prescription given in routine_level1.

The paths are expected to be given from the main directory of the Template copy.

1. It copies the input files according to the prescription given in move_files_level2
2. It launches executables (such as generate_events) according to the prescription given in routine_level2.

The paths are expected to be given from the main directory of the Template copy.

1. A python code collects the results.
2. These results are compared to the expected one written in expected_results.txt

How to implement a new test?

1. First create a new directory in TEST for your module: $>mkdir test_MYMOD
2. $>cd test_MYMOD
3. Create a directory for a specific test $> mkdir fast_NAME or $>mkdir full_NAME
4. $> cd fast_NAME
5. Define the six files move_files_level1, move_files_level2, routine_level1 , routine_level2,

dir_test.py, expected_results.txt and import a copy of all input cards (or files) in this directory.

A practical way to develop all those things…

This is only a proposition of the different step used to devellop a new step

1. Create your new test directory
2. The defines the four files move_files_level1, move_files_level2, routine_level1 , routine_level2 (this should be easy)
3. Copy the default files /Users/madgraph/TEST/Python_lib/dir_test_default.py in your directory fast_???? (respectively full_????)
4. Change his name in dir_test.py
5. Launch the test: (in TEST)$> ./test.py test_????/fast_???? (respectively for full)
6. Edit the value of actif and put this value to zero (avoid the overwriting of the Template copy and the execution of the different test script)
7. Implement your python (or external) routine and test this one with (in TEST)$> ./test.py test_MYMOD/fast_MYTEST
8. When the last step works correctly, remove the line actif=0 of the file dir_test.py
9. Copy the file test_result.dat in 'expected_result.dat': (in fast_???) $> cp test_results.dat expected_results.dat
10. add the third column (precision requirment) in the file expected_results.dat
11. launch completly your test to be sure that everithing is fine. (in TEST)$> ./test.py test_MYMOD/fast_MYTEST
12. store your module on the cvs

How to write input files?

How to write "move_files_level1" and "move_files_level2" ?

This files contains two columns.

1. The first one indicates the name of the files to copy (path starting from directory fast_???? (or full_????)
2. The second indicates where to copy the files (path starting from the main directory of the Template copy)

All lines begining with a # not taking into account

How to write "routine_level1" and "routine_level2"?

Those files are simply batch files. They will be executed from the main directory of the Template copy. A good way to code those files will be to redirect all the output.

How to write "dir_test.py" ?

1. First copy the file /Users/madgraph/TEST/Python_lib/dir_test_default.py in your fast_???? (or full_????) directory
2. Change his name in dir_test.py
3. Then follows the instruction inside (All the line starting with '#>>' indicates an action to perform).

This files indicates how to use an external programs and what you have to do if you choose to implement your python code.

How to write "expected_results.txt" ?

This files must contains three columns

1. the first one is of string symbolizing the result (this string must be unique)
2. The second one is the expected value
3. The third one is the difference pourcentage authorize without warning.
   1. if you want to compare string put 's'
   2. if you want to have exact value put '0'

How to write "test_results.txt" ?

If you choose to develop your own code to collect data, you have to write explicitly the file 'test_result.txt' (in the directory fast_???? or full_????) This files must contains two columns

1. The first one is of string symbolizing the result (this string must be unique)
2. The second one is the related value

Discussion/Idea?

-- Main.OlivierMattelaer - 18 Nov 2008