If we are supporting the scenario where staff will generate classifiers off the current course submissions:
1) Question opens
2) 10,000 students submit answers
3) Staff grades 100+ and trains models

It sounds like the submissions will be queued, fail, then manually re-entered in the queue when the classifiers are trained? Two thoughts
1) We may end up with a spike in submissions in the queue when the classifiers are trained, not a big deal, but something worth anticipating.
2) It may be something that occurs automatically, as part of training the classifiers. 
3) I may be misunderstanding this.