Remove .ruby-gemset too