Make the run configuration singleton

Since it can't be run twice on the same caches anyway