CellProfiler/CellProfiler — Issue #239: JIRA [IMG-76]Improve batch profiler handling of suspended jobs.
Author: thouis
Created: 2011-11-23 20:50:47+00:00
State: open

Batch profiler should monitor jobs and kill and rerun jobs that are suspended beyond some configurable timeout. Individual nodes can get overwhelmed and can suspend jobs for that reason - these complete in a timely manner if run on another node.

Original reporter: leek
Jira link: https://jira.broadinstitute.org/browse/IMG-76
Assignee: leek

**Comment from anne on Fri, 16 Jul 2010 10:19:12 -0400:**
Check with Gordon Saksena at Broad. He has written some code that might be helpful; it monitorsthe cluster and re-sends jobs that take too long with a new name. Needs to catch issues like jobs just taking a long time or running out of memory so it doesn't send tons of jobs tons of times.
Lee thinks this is massive and requires whole re-write of BatchProfiler.