CellProfiler/CellProfiler — Issue #181: JIRA [IMG-837] Recording time spent for error detection and for speed optimization
Author: thouis
Created: 2011-11-23 20:50:32+00:00
State: open

(A) We want to pay special attention to speed improvements for CellProfiler for the grant.
(B) We also want sudden changes in the run time (e.g., during nightly builds via CruiseControl) to trigger a warning of a potential error.
For (A), whole pipelines are good (even if the data is noisy) so we can see trends in speed over time for realistic use cases. Probably Mark's Super-Pipeline as well as a set of real pipelines (if it's not too painful, we may as well record times all of our example pipelines). It's probably also worthwhile to compare CP1 and CP2, especially IdentifyPrimaryObjects, MeasureAreaSizeShape with the Zernikes, MeasureTexture and TrackObjects since these are modules that take a long time to run.
For (B), Lee says,"Most useful would be per-module tests, with the most critical ones being done first. Per-pipeline is not quite as useful, more likely to fail because of some file/network glitch than a test that is completely self-contained." (though Anne adds,"perhaps if a pipeline'fails' to run at normal speeds, it can be automatically run again before trying to track down a problem?"). Lee also says,"Keeping track of speed is a perfect thing for the nightly build tests. You'd fail the test if the pipeline took more than a certain percentage of what it took the last time it was run. This could be done for the more expensive modules too which would alert us to problems caused by changes in algorithm. Maybe I can write a test for MeasureAreaSizeShape that can serve as an example for others who will write tests for the Identify and other modules." Perhaps Mark can take it from there.
Adam can do the parts of this Jira that relate to Cruise Control.
Note also that any attempts to improve CellProfiler's speed should be entered In Jira and contain the word"Speed" so they are easy to find later. Ideally, you would include before and after statistics for the time required for the particular module/functionality.

Original reporter: anne
Jira link: https://jira.broadinstitute.org/browse/IMG-837
Assignee: leek