Do our best to cleanup every created directories