(Maint) Update Gemfile to allow excluding extra gems

We normally have a concept of "basic" and "extra features", but the
Gemfile didn't contain this difference. This updates it to work with
that distinction and also cleans up the handling of the specific git
branches for our (bad) mixture of having a gemspec and not.