Only check for the benchmark engine executables before actually running.