# Presentation: 748531

## Reconstruction production 

- Y. Fisyak, BNL

## What is needed to run athena ?

- To define script to install a working directory and JobOptionFile.txt
- To define a bunch of environment variables depending on ATLAS release:
- PATH and LD_LIBRARY_PATH to find executable(s) and shared libraries
- ~650 “package”ROOT variables to handle constructions
**#include "$ATHENACOMMONROOT/share/Atlas_ZebraTDR.UnixStandardJob.txt“**
- in JobOptionFile.txt and
- constructions like
**cp -f $ATHENACOMMONROOT/share/slug.car** .
- in shell script during installation of working directory
- To define input and output files

## How is it defined in production?

- In current production (HLT with 6.0.4 release) this is done via athena_job perl script 				(see** **_[http://www.usatlas.bnl.gov/~fisyak/atlas/Production/](http://www.usatlas.bnl.gov/~fisyak/atlas/Production/)__)_
**A typical job looks like:**
**perl athena_job **
**	****	****–****i dc1.002001.lumi10.05259.hlt.pythia_jet_25.zebra **
**	****	****-o ../dc1.002001.lumi10.recon.010._05259.hlt.pythia_jet_25.hbook **
**	****	****-w /usatlas/data04/dc1/work/6.0.4 **
**	****	****-s  recon.gen.v7.bnl **
**	****	****-j trigger.lumi10.noise.604.job**
**It defines** **input ****(-i) and ****output**** (-o) files, ****working directory**** (-w),**** script ****to install the working directory (-s), and**** jobOptionFile**** (-j)**

## How is it defined (cont.)

**The perl script uses a set of default environment variables:**
- setenv ATLASDATA /afs/usatlas.bnl.gov/offline/data
- setenv ATLAS_BASE_RELEASE 6.0.4
- setenv ATLAS_ROOT /afs/usatlas.bnl.gov
- setenv CERNSW cernsw
- setenv CMTCONFIG i686-rh73-gcc295
- setenv CMTSITE BNL
- setenv CMTVERS v1r12p20020606
- setenv DIST /afs/usatlas.bnl.gov/software/dist
- setenv TEST_RELEASE_DIR /afs/usatlas.bnl.gov/users/fisyak/public/prod/6.0.4
- setenv CMTROOT /afs/usatlas.bnl.gov/cernsw/contrib/CMT/v1r12p20020606
- setenv CMTPATH /afs/usatlas.bnl.gov/users/fisyak/public/prod/6.0.4:
- setenv DISTREL /afs/usatlas.bnl.gov/software/dist/6.0.4
- setenv ATLASBIN /afs/usatlas.bnl.gov/software/dist/6.0.4/InstallArea/i686-rh73-gcc295/bin
**Any of these environment variables can be redefined **
**	****athena_job ... **ATLAS_BASE_RELEASE=6.3.0 ...

## What athena_job does ?

- Create working directory
- Create in this directory requirements file and run a particular version of cmt with particular version of AtlasSettings (the versions of cmt and AtlasSettings are depended on ATLAS_BASE_RELEASE)
- Create soft links and copy data files in the working directory
- Find input file using magda (magda_getfile) on disks or HPSS
- Run athena with the **JobOptionFile.txt**
- Save output file (**ntuple.hbook**)
- Remove the working directory
- Release input file (**magda_releasefile**)

## Bookkeeping

- Bookkeeping is done with one MySQL table (Production.jobdata) in which it is tracked transformation through the steps:
- DEFINED (registration of the output file),
- lsf submit status (PEND, RUN, SSUSP),
- ABORTED (job fails in lsf),
- UNDECIDED (job finished successfully, output file has to be validated),
- FAILED     (job finished successfully but output file is corrupted),
- VALIDATED .

## Mini productions (PWG/user oriented).

- Two types of production outside of DC chain:
- Physics Working Group (PWG) production with significant consumption of resources
- It should be managed via USATLAS computing,
- It can be done using the above scheme with tracking all modification of environment with respect to “official’’ DC production in “Virtual Data Catalog”
- Personal (user) production with resources which are not collided with DC and PWG productions.
-  **This production requires adjustments of above scheme to user environment**
**It is not clear what level of bookkeeping this production is necessary (MySQL versus flat files)**