Remove old configs as well