Don't build benchmarks on travis-ci

Trying to install the benchmark dependencies causes a dependency
cycle.