Speed up setup and test script, by reducing config file churn