Clean up old configs.