Remove OSX from our build matrix

The Travis OSX builders are incredibly inconsistent for how long they take to run our test suite. It is currently ranging anywhere from ~15 minutes to well beyond 50 minutes, at which point it times out.