Hack benchmark runner so that the num threads can be capped