droolsjbpm/drools-planner — pull request #4 by triceo (2012-04-21 12:17:33+00:00)
Title: JBRULES-3405: Evaluate benchmark results per average ranking, not per average score

This pull request introduces another way of comparing benchmarks that may prove beneficial when there are more than 1 inputSolutionFiles. What's done:

- Introducing new API called BenchmarkRanker.
- The current method of comparing benchmarks is implemented as SimpleBenchmarkRanker.
- New method of comparing benchmarks is introduced as RelativePositionBenchmarkRanker. It doesn't compare benchmark's total score - instead, it compares each problem score separately with every other benchmark's problem score and rank as the best the benchmark with the most "wins."
- solutionBenchmarkRankerClass configuration option added to the benchmark, SimpleBenchmarkRouter is the default choice.

The default behavior of Planner is unchanged.