CellProfiler/CellProfiler — Issue #196: JIRA [IMG-696] Add tests for changes in settings text
Author: thouis
Created: 2011-11-23 20:50:35+00:00
State: open

Make tests, according to the following email exchange:
Hi Anne + Peggy,
Just a quick note: Be cautious about altering constants or definitions which Lee typically defines in all caps. While these may not look like they are part of the settings choices, oftentimes they are used to define the string that lists a settings option (as opposed to the setting text/help). By rewording them for readability, effectively it's altering the setting itself and a previous pipeline which doesn't have the option with the same exact wording will break. Fixing this involves a addition to upgrade_settings. This occurred in the Resize module and I didn't catch it until I loaded the AllModules pipeline. AFAIK, this is the only module that had this type of edit, which is now fixed.
-Mark
Can you think of an automated test that can check whether any options' text ever changes? Other than a pipeline where EVERY settings option is exercised? Because I suppose if I changed options text for an option that didn't happen to be in your pipeline I am not sure we would know...
-a
The unit tests should catch that situation hypothetically. At least, that's how Lee has caught them when I've made the same mistake. Since the unit test pipeline is force-fed the settings consistent with the time that the test was created, if a setting changes later, the test fails outright. The tests can be run as a group, but I have yet to do so b/c my python dies when I try it from Wing (presumably it's something specific to my machine again).
-Mark
Thanks but... I'd rather not guarantee that I have a test where every option of every module of every version is tested. -Lee
Do you think it's worthwhile making such a test? (low-priority, of course)
-Anne
Ideally, it really should be done. Going forward, it's easier to test the new .cp files than it was to test the .mat files because you can edit them by hand so you can quickly create a pipeline with all of the necessary combinations (and when we upgrade formats now and include CP files instead of .mat ones, I do add tests for all of the options). I'd rather not do this for the .mat files because it involves making quite a few of them - at worst, the bug would be that the choice would appear as an error and the user would see the old text. They could probably guess what the correct choice would be.
-Lee

Original reporter: anne
Jira link: https://jira.broadinstitute.org/browse/IMG-696
Assignee: leek