Cleaning gems properly only removes them from the local dir