cms-dev/cms — pull request #356 by artikz (2014-09-17 18:12:33+00:00)
Title: Evaluating testcases separately

This implements #342

Shortly:

- each "evaluate submission X on dataset Y" job is split to "evaluate submission X on testcase Z from dataset Y" by putting information about only one testcase into JobGroup for Worker
- for simplicity, the order of testcase jobs is now not very smart - they are just ordered lexicographically
- queue_status() method of ES is optimized to output only one evaluation job per submission per dataset
- check() method of ESOperation is optimized for bulk checks