Remove more gem groups from travis build
Update travis-ci config to match convention in other gems