CLAS Simulation Stages and Objectives

Objectives for CLAS simulation:
Provide simulated data for acceptance calculations and for understanding and estimating the effects of cuts on the data
Provide enough statistics so that the statistical error from simulation does not contribute the dominant error to the physics analysis overall
Provide low-statistics accurate simulations and/or fast simulations early in the data analysis projects, and high-statistics accurate simulations for the later phases of data analysis
Accurately assess the needs of the collaboration so that general-purpose simulations address as many analyses as possible
Problems:
existing simulation is slow, and can probably never be much faster
the data volume is large, which puts constraints on simulation requirements for statistical accuracy
many aspects of the code are not well-tested
many parts of the code are incomplete (arguably not the important ones)
the number of available CPU's is far too small
the analysis methods which use the results are varied and not all well-defined: difficult to assess simulation needs
existing event generators all have significant limitations

An Optimistic Schedule

Short-term (2-3 months)
begin generating acceptances for first publications (determine which publications will be first!)
use existing distributed farm (equivalent to ~0.4*39 Linux 300 MHz boxes, <~16 full-time CPU's)
find/fix problems with the code and event generators
try further strategies for speeding code up
implement and use bookkeeping system as soon as possible; if bugs are found as time goes on, exclude bad files from the analysis via date/cvs version
look into options for increasing total CPU power at least 5-fold; even more would be beneficial
develop a faster, simpler simulation program from existing code
get experience with obtaining efficient use of distributed farms
try to define the simulation needs of the collaboration
Intermediate-term (6-8 months)
build up a much larger compute farm (distributed + local)
finish acceptance calculations for first publications
transition into full-time, large-scale continuous computing
continue to define the simulation needs of the collaboration
continue to improve the simulation accuracy
continue to improve the event generator(s)

Distributed Computing Farms - Plans and Issues

Quality control
Bookkeeping system

Convenient in any case, but mandatory when multiple institutions, multiple code versions, and continuous code changes are involved.

Each program involved (CELEG, aao, GSIM, perhaps TSIM, etc.) encapsulates its input parameters, time of compilation, time of execution, cvs version, etc, into BOS banks which travel with the file (one bank per file).

Files from various institutions are gathered to a common area (e.g., the /work/clas/gsim area), and are archived. The archiving script first gets the parameter information out of the bookkeeping BOS banks, transfers the parameter information into the offline database, then archives the file.

Information on each GSIM run is available from web pages which are dynamically generated from the database.

Automated monitoring of results via simple event statistics
Rapid reconstruction turn-around to catch deeper problems
Data transfer
Presently using ftp; no complaints so far. UVA rate is 0.8 Mbytes/sec, maximum. Present production rate is 0.6 k/sec-cpu, ->
Could use DLT's, but requires some investment and cooperation by remote institutions. Smaller institutions may not want to buy DLT drive. Probably could live with a mixture of ftp transfers and DLT tapes.
Reconstruction of events
Total rate (0.25 Gevents in 0.75 months) is 128 events/second. At 50 ms/event, need 7 cpu's @ 100% to do reconstruction of simulated events. Can

A Good Solution

Jlab purchases 60 CPU's for an on-site farm (~100k$ plus one CC person). Dedicate the farm to CLAS simulations, expect to run continuously for years.
Expand existing collaborations with external institutions to include 25-50 more Linux-equivalent cpu's at 40% utilization.
Continue to try to speed up the code where possible

CLAS Simulation Stages and Objectives

An Optimistic Schedule

Distributed Computing Farms - Plans and Issues

A Good Solution

Attain desired objectives for simulation

Other possible solutions