Persist config, attempt to rerun tests