Removes file with not so secret keys anymore