Remove files that do not exist in gem files