Gemspec: Work on gem dependencies

Two goals here:

2. Speed up bundle install. Jenkins has timed out a couple times during bundle install.
1. Be able to bundle in isolation.