Factor out enum into its own method, as it is generally useful
Report all on-demand configurations, even if they pass
retest is not just things that have already been tested