[CI] Run specs and tests

While we move away from rspec and towards minitest, now seems like a good time to start running both to make sure builds don't break.

cc @jch @schaary 

:+1: to `allow_failures`. Be good to add `ruby-head` in there as well to foresee future incompatibilities