Apparently there is such thing as a cleanup task, and lo and behold chef_client implements it.