Project: datamanagement
Issue DM-588: Get one HTCondor ClassAds scenario running
We have a worker script that takes three arguments: a CCD number, a cache limit, and a length of time. The script starts out looking at a slot-specific filesystem for a calibration file /local/slot#/calib/CCD#. If that file doesn't exist, it checks to see if there are more than the cache limit of files in /local/slot#/calib. If there are, it removes one at random (later we could try other algorithms). It then immediately transfers /gpfs/calib/CCD# (maybe a 1 GB file) to /local/slot#/calib/CCD#. The script then waits for the given length of time and exits. We make sure we can measure how many file copies from /gpfs are performed and when so that we can determine peak demand rates and average rates. Meanwhile, the HTCondor hook checks for the same /local/slot#/calib/CCD# files and sets the ClassAds based on which CCD#s are present. Scenario 1) 4 slots, 4 CCDs, cache limit 1, jobs take 1 min, total of 5 jobs per CCD, 1 job per CCD submitted every 2 min (1234 pause 1234 pause 1234 pause 1234 pause 1234)
Story points: 14