Remove template file and keep configuration file correctly