Remove file configured by chef