Only remove the directory on Berkshelf 3