Writing configs and removing as needed in Rakefile