Remove remnants of old config