Structure

The idea of the plugin idea is to allow some modification of the code behavior without the need to modify the internal code. Allowing an easy support of many modification in the future release of MG5aMC. So far three type of modification are supported by this module:

1. New output type (for LO processes)
2. New cluster type
3. Modification of the interface (new command/modification of the command)

The plugin is a directory to placed inside the PLUGIN directory of MG5aMC. It should at least have one file named "__init__.py" which should contains some minimal information (see below). This file should link to other python file if needed.

The final directory structure should be

M5aMC/
  PLUGIN/
    __init__.py (empty file --do not touch this one--)
    MY_PLUGIN1_DIR/
      __init__.py (file with mandatory information see below)
      my_python_file.py (need to be load from __init__.py)
      other_data and directory
    MY_PLUGIN2_DIR/
      __init__.py (file with mandatory information see below)
      my_python_file.py (need to be load from __init__.py)
      other_data and directory 

__init__.py file:

The minimal information that need to be present in that file is:

## import the required files
# example: import maddm_interface as maddm_interface # local file
#          import madgraph.various.cluster as cluster #MG5 distribution file
# Three types of functionality are allowed in a plugin
#   1. new output mode
#   2. new cluster support
#   3. new interface

# 1. Define new output mode
#    example: new_output = {'myformat': MYCLASS}
#    madgraph will then allow the command "output myformat PATH"
#    MYCLASS should inherated of the class madgraph.iolibs.export_v4.VirtualExporter 
new_output = {}

# 2. Define new way to handle the cluster.
#    example new_cluster = {'mycluster': MYCLUSTERCLASS}
#    allow "set cluster_type mycluster" in madgraph
#    MYCLUSTERCLASS should inherated from madgraph.various.cluster.Cluster
new_cluster = {}


# 3. Define a new interface (allow to add/modify MG5 command)
#    This can be activated via ./bin/mg5_aMC --mode=PLUGINNAME
## Put None if no dedicated command are required
new_interface = None
 
 
########################## CONTROL VARIABLE ####################################
__author__ = ''
__email__ = ''
__version__ = (1,0,0)
minimal_mg5amcnlo_version = (2,3,4) 
maximal_mg5amcnlo_version = (1000,1000,1000)
latest_validated_version = (2,4,0)

create new output mode

Base class which can be use to create a new exporter from scratch:

At the beginning of madgraph/various/iolibs/export_v4.py. A virtual class "VirtualExporter" is defined all exporter class should derive from that class. That virtual class list all the functions called by the interface that you can define (that class in itself does nothing) and list all the common attribute that need to be defined with the explanation of their impact.

In Most of the case, it is more convenient to subclass one of the class creating a real output like ProcessExporterFortran.

List of function that need to be defined in the exporter For Fortran/CPP output:

This list correspond to the call to the exporter performed outside the class itself. Please check the VirtualExporter class (begin of madgraph/various/iolibs/export_v4.py) to check that the following list is up-to-date with your version of MG5aMC.

1. init(self, dir_path = "", opt=None)
2. copy_template(self, modelname)
3. generate_subprocess_directory(self, subproc_group, helicity_model, me=None) [for grouped]
4. generate_subprocess_directory(self, matrix_element, helicity_model, me_number) [for ungrouped]
5. convert_model(model, wanted_lorentz=[], wanted_coupling=[])
6. finalize(self,matrix_element, cmdhistory, MG5options, outputflag)
7. modify_grouping(self, matrix_element)
8. export_model_files(self, model_v4_path) [only if you want to support old model format]
9. export_helas(self, HELAS_PATH) [only if you want to support old model format]

List of class variable that need to be defined in the exporter:

This list correspond to the attribute of the exporter called outside the exporter class. Please check the VirtualExporter class (begin of madgraph/various/iolibs/export_v4.py) to check that the following list is up-to-date with your version of MG5aMC.

1. grouped_mode This variable changes the type of object called within 'generate_subprocess_directory' functions.
   1. False to avoid grouping (only identical matrix element are merged)
   2. 'madevent' group the massless quark and massless lepton
   3. 'madweight' group the gluon with the massless quark
2. sa_symmetry If no grouped_mode=False, uu~ and u~u will be called independently. Putting sa_symmetry generates only one of the two matrix-element. (might not work for cpp output.
3. check Ask madgraph to check if the directory already exists and propose to the user to remove it first if this is the case
4. output [Template, None, dir]
   1. 'Template', madgraph will call copy_template
   2. 'dir', madgraph will just create an empty directory for initialisation
   3. None, madgraph do nothing for initialisation
5. exporter ['v4'/'cpp'] language of the output 'v4' for Fortran output, 'cpp' for C++ output

Example

List of currently existing plugin

Example on how to implement a plugin

The LSF cluster at SLAC is very similar to what is supported by the MG5aMC native cluster's module. However, there are a couple of differences, for instance:

1. After being submitted, jobs at SLAC typically take about 5 to 10 seconds to even register and show up on the 'bjobs' list; MG5aMC must wait for this registration otherwise missing job entries in 'bjobs' are problematic.
2. Wall time specification is mandatory.
3. Quite often, jobs at SLAC can be superseeded, in which case they are listed as 'SSUSP'. This status code is unknown to MG5aMC by default.

This cluster plugin example addresses these issues: plugin for LSF at SLAC

Example 1: hello world (adding command to the interface)

__init__.py file

## import the required files
# example: import maddm_interface as maddm_interface # local file
#          import madgraph.various.cluster as cluster #MG5 distribution file

import helloworld as helloworld # local file

# Three types of functionality are allowed in a plugin
#   1. new output mode
#   2. new cluster support
#   3. new interface

# 1. Define new output mode
#    example: new_output = {'myformat': MYCLASS}
#    madgraph will then allow the command "output myformat PATH"
#    MYCLASS should inherated of the class madgraph.iolibs.export_v4.VirtualExporter 
new_output = {}

# 2. Define new way to handle the cluster.
#    example new_cluster = {'mycluster': MYCLUSTERCLASS}
#    allow "set cluster_type mycluster" in madgraph
#    MYCLUSTERCLASS should inherated from madgraph.various.cluster.Cluster
new_cluster = {}


# 3. Define a new interface (allow to add/modify MG5 command)
#    This can be activated via ./bin/mg5_aMC --mode=PLUGINNAME
## Put None if no dedicated command are required
new_interface = helloworld.NewInterface
 
 
########################## CONTROL VARIABLE ####################################
__author__ = 'Mattelaer Olivier'
__email__ = 'o.p.c.mattelaer@durham.ac.uk'
__version__ = (1,0,0)
minimal_mg5amcnlo_version = (2,5,0) 
maximal_mg5amcnlo_version = (1000,1000,1000)
latest_validated_version = (2,5,0)

helloworld.py file

import madgraph.interface.master_interface as master_interface

class NewInterface(master_interface.MasterCmd):

    def do_helloworld(self, line):
        """print hello world"""
        print "hello world " + line
    
    def help_helloworld(self):
        """routine providing help for helloworld command"""
        print "routine to print hello world and copy the rest of the line"

    def complete_helloworld(self, text, line, begidx, endidx):
        """routine defining the auto-completion
           auto-complete to get  Olivier Mattelaer|Valentin Hirschi| Valentin Alwall | Oliver Mattelaer"""
         
        args = self.split_arg(line[0:begidx])
        if len(args)== 1:
            return  self.list_completion(text,['Olivier', 'Valentin', 'Oliver'])
        if len(args)==2:
            if args[1].startswith('Oliv'):
               return  self.list_completion(text,['Mattelaer'])
            elif args[1] == 'Valentin':
               return  self.list_completion(text,['Alwall', 'Hirschi'])

    #command to change the prompt 
    def preloop(self, *args, **opts):
        """only change the prompt after calling  the mother preloop command"""
        super(NewInterface, self).preloop(*args,**opts)
        self.prompt = 'Hello? >'
        

Now you can run the standard executable with

./bin/mg5_aMC --mode=helloworld

Note that "helloworld" is the name of the DIRECTORY where the above file have been placed.

The above command create you a shell where you have access to one additional command "helloworld"

Hello? >helloworld
hello world

For a real physics code using that functionality, you can check maddm.

Example 2: New Exporter

__init__.py file

## import the required files
# example: import maddm_interface as maddm_interface # local file
#          import madgraph.various.cluster as cluster #MG5 distribution file

import output as output #local file

# Three types of functionality are allowed in a plugin
#   1. new output mode
#   2. new cluster support
#   3. new interface

# 1. Define new output mode
#    example: new_output = {'myformat': MYCLASS}
#    madgraph will then allow the command "output myformat PATH"
#    MYCLASS should inherated of the class madgraph.iolibs.export_v4.VirtualExporter 
new_output = {'madweight2': output.My_MW_Exporter,
              'standalone_lib': output.MY_CPP_Standalone}

# 2. Define new way to handle the cluster.
#    example new_cluster = {'mycluster': MYCLUSTERCLASS}
#    allow "set cluster_type mycluster" in madgraph
#    MYCLUSTERCLASS should inherated from madgraph.various.cluster.Cluster
new_cluster = {}


# 3. Define a new interface (allow to add/modify MG5 command)
#    This can be activated via ./bin/mg5_aMC --mode=PLUGINNAME
## Put None if no dedicated command are required
new_interface = None
 
 
########################## CONTROL VARIABLE ####################################
__author__ = 'Mattelaer Olivier'
__email__ = 'o.p.c.mattelaer@durham.ac.uk'
__version__ = (1,0,0)
minimal_mg5amcnlo_version = (2,5,0) 
maximal_mg5amcnlo_version = (1000,1000,1000)
latest_validated_version = (2,5,0)

output.py file

import madgraph.iolibs.export_cpp as export_cpp
import madgraph.iolibs.export_v4 as export_v4
import madgraph.various.misc as misc

class My_MW_Exporter(export_v4.ProcessExporterFortranMWGroup):

    check = True
    # check status of the directory. Remove it if already exists
    # Language type: 'v4' for f77/ 'cpp' for C++ output
    exporter = 'v4'
    # Output type:
    #[Template/dir/None] copy the Template, just create dir  or do nothing 
    output = 'Template' 
    # Decide which type of merging to used [madevent/madweight]
    grouped_mode = 'madweight'    
    # if no grouping on can decide to merge uu~ and u~u anyway:
    sa_symmetry = True 

    nb_done = 0
    
    def __init__(self, *args, **opts):
        misc.sprint("Initialise the exporter")
        return super(My_MW_Exporter, self).__init__(*args, **opts)

    def copy_template(self, *args, **opts):
        misc.sprint("copy the associate template")
        return super(My_MW_Exporter, self).copy_template(*args, **opts)

    def generate_subprocess_directory(self, subproc_group,
                                         fortran_model, me=None):
        
        misc.sprint(me)
        if me is None:
            MyExporter.nb_done +=1
            current_generated = MyExporter.nb_done
        else:
            current_generated = me
        return super(My_MW_Exporter, self).generate_subprocess_directory(subproc_group, fortran_model, current_generated)

class MY_CPP_Standalone(export_cpp.ProcessExporterPythia8):

    # check status of the directory. Remove it if already exists
    check = True 
    # Language type: 'v4' for f77/ 'cpp' for C++ output
    exporter = 'cpp'
    # Output type:
    #[Template/dir/None] copy the Template, just create dir  or do nothing 
    output = 'Template'
    # Decide which type of merging if used [madevent/madweight]
    grouped_mode = False
    # if no grouping on can decide to merge uu~ and u~u anyway:
    sa_symmetry = True

    def __init__(self, *args, **opts):
        misc.sprint("Initialise the exporter")
        return super(MY_CPP_Standalone, self).__init__(*args, **opts)

    def copy_template(self, model):

        misc.sprint("initialise the directory")
        return super(MY_CPP_Standalone, self).copy_template(model)


    def generate_subprocess_directory(self, subproc_group,
                                         fortran_model, me=None):
        
        misc.sprint('create the directory')
        return super(MY_CPP_Standalone, self).generate_subprocess_directory(subproc_group, fortran_model, me)


    def convert_model(self, model, wanted_lorentz=[], wanted_coupling=[]):
        misc.sprint('create the model')
        return super(MY_CPP_Standalone, self).convert_model(model, wanted_lorentz, wanted_coupling)

    def finalize(self, matrix_element, cmdhistory, MG5options, outputflag):
        """typically creating jpeg/HTML output/ compilation/...
           cmdhistory is the list of command used so far.
           MG5options are all the options of the main interface
           outputflags is a list of options provided when doing the output command"""
        misc.sprint("pass here")
        return super(MY_CPP_Standalone, self).finalize(matrix_element, cmdhistory, MG5options, outputflag)

    def modify_grouping(self, matrix_element):
        """allow to modify the grouping (if grouping is in place)
            return two value:
            - True/False if the matrix_element was modified
            - the new(or old) matrix element"""
        #irrelevant here since group_mode=False so this function is never called
        return False, matrix_element

Example 3: New Cluster

A condor cluster not submitting more than a given number of job simultaneously:

__init__.py

## import the required files
# example: import maddm_interface as maddm_interface # local file
#          import madgraph.various.cluster as cluster #MG5 distribution file

import madgraph.various.cluster as cluster
import time

#local file

class MYcluster(cluster.MultiCore):

    maximum_submited_jobs = 100
    restart_submission = 50

    def submit(self,  prog, argument=[], cwd=None, stdout=None, stderr=None, log=None,
               required_output=[], nb_submit=0):
        """pause the submission of jobs if more than """

        me_dir = self.get_jobs_identifier(cwd, prog)
        if len(self.submitted_ids) > self.maximum_submited_jobs:
            fct = lambda idle, run, finish: logger.info('Waiting for free slot: %s %s %s' % (idle, run, finish))
            self.wait(me_dir, fct, self.restart_submission)


        # call the normal submission scheme
        super(MYcluster, self).submit( prog, argument, cwd, stdout, stderr, log, required_output, nb_submit)


# Three types of functionality are allowed in a plugin
#   1. new output mode
#   2. new cluster support
#   3. new interface

# 1. Define new output mode
#    example: new_output = {'myformat': MYCLASS}
#    madgraph will then allow the command "output myformat PATH"
#    MYCLASS should inherated of the class madgraph.iolibs.export_v4.VirtualExporter 
new_output = {}

# 2. Define new way to handle the cluster.
#    example new_cluster = {'mycluster': MYCLUSTERCLASS}
#    allow "set cluster_type mycluster" in madgraph
#    MYCLUSTERCLASS should inherated from madgraph.various.cluster.Cluster
new_cluster = {'new_condor': MYcluster}


# 3. Define a new interface (allow to add/modify MG5 command)
#    This can be activated via ./bin/mg5_aMC --mode=PLUGINNAME
## Put None if no dedicated command are required
new_interface = None
 

 
########################## CONTROL VARIABLE ####################################
__author__ = 'Mattelaer Olivier'
__email__ = 'o.p.c.mattelaer@durham.ac.uk'
__version__ = (1,0,0)
minimal_mg5amcnlo_version = (2,5,0) 
maximal_mg5amcnlo_version = (1000,1000,1000)
latest_validated_version = (2,5,0)

to use this plugin, you can either edit your file "input/mg5_configuration.txt" or type in the MG5aMC shell "set cluster_type new_condor".

Example 4: Change hidden parameter of the run_card via the cluster setting

This cluster request madgraph to reduce the number of submitted jobs (and therefore submitting longer jobs). By setting the associated parameter in the run_card if the user did not specify those. (implemeted with/for Roberto Franscini)

init.py

## import the required files
# example: import maddm_interface as maddm_interface # local file
#          import madgraph.various.cluster as cluster #MG5 distribution file

import madxgraph.various.cluster as cluster
import time

#local file

class MYcluster(cluster.PBSCluster):

    def modify_interface(self, run_interface):
        """
        routine which allow to modify the run_card/mg5cmd object to change the
        default behavior of the runs.
        This is called at the time of the compilation of the run_card.
        Note that this function can be called multiple times by run.
        """
        run_card = run_interface.run_card
        run_card.set('survey_nchannel_per_job',10, changeifuserset=False)
        run_card.set('refine_evt_by_job', 25000, changeifuserset=False)
        return super(MYcluster, self).modify_interface(run_interface)


# Three types of functionality are allowed in a plugin
#   1. new output mode
#   2. new cluster support
#   3. new interface

# 1. Define new output mode
#    example: new_output = {'myformat': MYCLASS}
#    madgraph will then allow the command "output myformat PATH"
#    MYCLASS should inherated of the class madgraph.iolibs.export_v4.VirtualExporter
new_output = {}

# 2. Define new way to handle the cluster.
#    example new_cluster = {'mycluster': MYCLUSTERCLASS}
#    allow "set cluster_type mycluster" in madgraph
#    MYCLUSTERCLASS should inherated from madgraph.various.cluster.Cluster
new_cluster = {'new_pbs': MYcluster}


# 3. Define a new interface (allow to add/modify MG5 command)
#    This can be activated via ./bin/mg5_aMC --mode=PLUGINNAME
## Put None if no dedicated command are required
new_interface = None


########################## CONTROL VARIABLE ####################################
__author__ = 'Mattelaer Olivier'
__email__ = 'o.p.c.mattelaer@durham.ac.uk'
__version__ = (1,0,0)
minimal_mg5amcnlo_version = (2,5,0)
maximal_mg5amcnlo_version = (1000,1000,1000)
latest_validated_version = (2,6,0)