[CI] Run specs and tests

While we move away from rspec and towards minitest, now seems like a good time to start running both to make sure builds don't break.

cc @jch @schaary 

Was feeling the same while testing this out. I think we can keep the builds, but put them under `allow_failures` so we can keep an eye on it.