The change from the "old" fermilab batch system
to condor was pretty straightforward, so I have
about 4 years experience now!

The basic idea is that almost everything has to
be done remotely, but your beautiful code is
set up to do this nicely.

You package everything up you need for each run --
which is just a few files.  Here is a typical condor
job description file:

universe = vanilla
Notification = Error
executable =
/farm/run2mc_stage01/condor/W2/SubProcesses/Pug_e+vedg/ajob8.sh
environment = F77=g77
transfer_output = true
transfer_error = true
transfer_executable = true
should_transfer_files = YES
transfer_input_files =
ajob8,input_app.txt,madevent,input-card.dat,cteq6l1.tbl,randinit
WhenToTransferOutput = ON_EXIT_OR_EVICT
input = /dev/null
output = $(Cluster).$(Process).out
error =  $(Cluster).$(Process).err
log =    $(Cluster).$(Process).log
queue

After each step (survey, refine, refine), a bunch of files come
back into the appropriate directories.  THEN, I run a
shell script locally:

#!/usr/local/bin/bash

./bin/march
cd SubProcesses
../bin/sumall
../bin/combine_events
cd ..


"march" is a script I wrote that "marches" through all of the
directories, untars alot of tarballs, and basically prepares the
output for combine_events.

Also, I found that it was more efficient to deal with
"gzipped" events.dat files.  There is actually a library
out there that lets you READ gzipped files in fortran.

As far as I can tell, there is no GOOD way to avoid the
local processing step between survey, refine, refine.

People won't like that, but that is because they haven't
really thought about the problem.

