Change default for RunBenchmarks config to run only once #4289