Tidying up perf infra

Cleaning up the core infra now that we have rationalized our current test suite:
* Removing threaded test infra (removes a bunch of special casing in runners etc.
* Changing `MetricCollector` to a more general `TestHarness` to allow more functionality in future
* Adding a `NullTestHarness` for warmup iterations
* Cleaning up other dead code etc.
+1 for getting moving default values to somewhere more appropriate