Clean up and delete no longer needed files after rewrite