Clean up empty date branch directories better.