Automatic Validation Package from the Web

• Description: A web site is being developed that will allow a user to upload their model files (including restriction files and parameter files). The web site will then run FR, load the models, run the interfaces and attempt to compile the resulting files. If that is successful, the user will be allowed to generate 2->2 processes (and others in the future) and run comparisons. In the first version, I envision it being useful to us (FR developers). In a later version (next year), I envision it being ready for the public.
• Names: Neil, Claude, Benj, Christian.
• Status: 50% of essential developer required code done.
  • Full FR model can be uploaded.
  • Site creates dir for model, stores model files there in tar.gz format.
  • Site creates entries in database for model.
  • Site starts condor jobs to:
    • Load model files
    • Load restriction files.
    • Run CH, FA, MG, SH, WO interfaces.
    • Run CH & MG on resulting code.
    • Each of these condor jobs relies on previous ones. If a necessary previous step fails, then the next job is not run. This is controlled by a "DAG" script.
  • If 1 or more MEG passes, the user is allowed to run 2->2 cs processes.
    • They can choose the MEG they want to test (among the ones that pass the previous tests).
    • User can choose "restrictions" on 2->2 processes.
      • If no restrictions are chosen, then _all_ 2->2 processes are generated.
      • User can restrict to certain kinds of fields (fermions, vectors,...)
      • User can restrict on indices and charges.
  • After processes are generated, user can run them.
    • The processes are run on Condor.
      • Currently 4 parallel nodes. Seems pretty fast.
      • Adds 10 processes to the queue at a time. This allows multiple users and validations to be run at the same time and make progress at the same time.
      • User can refresh their browser to see the progress. Each process and MEG that is finished will show up as they are finished.
    • User can rerun processes.
  • User can create multiple validations.
  • Multiple validations can be run concurrently. Condor handles this.
  • Multiple users can run at the same time. Condor handles this.
  • Users can delete models. (But only when no condor jobs are running for this model.)
  • Users can delete validations. (But only when this validation is not being run.)
• To-do:
  • Get FR-SH and FR-WO working. (Christian Speckner is helping with WO). This shouldn't be difficult. The scaffolding is there.
  • Test whether a Mathematica license is available. If not, wait and try again.
  • Use parameter files in tests.
  • Allow upload and comparison with "stock" versions of models.
  • Improve security:
    • Move all MEG to condor nodes and unlink home dir.
    • Test that nothing can be written outside of condor job dir.
    • Test that condor job cannot do any network connections _except_ certain database updates.
    • Control time, memory and disk space available to condor jobs.
  • This ends the critical to-do's for this release (for developers).
  • Add links moving around model database.
  • Add history (if model changes, then keep the old model with versioning. Do the same for validations, etc..)
  • Finish other pages of model database web site. Login, etc...
  • Allow the user to modify part of their model (currently have to delete model and start over).
    • Intelligently only run pieces that are modified.
  • Allow user to stop running Condor jobs.