Remove performance test from Travis

It just takes too long to be worth it. I should just make it a point to
do performance tests as we go along and take records.