Deletes files after Chef notifications have run