Only delete cache and config directories, not entire data directory