Add a "dumb" mode which doesn't create environments or fetch any repo data

This allows for a quick turnaround (around 3s., instead of 45s. here) when developing benchmarks.