Remove clang from travis.yml for now.

Can't build with the same CFLAGS, so remove it for now.