Project: datamanagement
Issue DM-654: Run "single slow worker" HTCondor ClassAds Scenario
We run and study a "single slow worker" HTCondor ClassAds Scenario. The scenario is a perturbation of the baseline HTCondor ClassAds Scenario. In the baseline, Jobs for a given CCD are consistently pinned to a slot/node that advertises the presence of associated data files/calibration files for that CCD. A baseline run may proceed, for example, with jobs for 4 CCDs repeating executing within same HTCondor slot on a node (even when spare processing slots are readily available.). The baseline is observed to be quite stable, as the pool is empty each time a wave of jobs is submitted. In the "single slow worker" scenario, we cause one of the jobs for a chosen CCD to stall (mocking up a slow file transfer, lengthy computation in an algorithm, etc), such that the pool is not empty at the the submission time for a wave of jobs. We seek to observe how Rank places jobs in this scenario, and work to assign Rank (especially for spare slots) in an optimal way so as to minimize file transfers.
Story points: 4