Stop testing 32-bit on travis

This has been broken for quite awhile now on 32-bit linux due to what looks like
libcurl going awry. This can be reactivated later, but consistently green travis
builds are more important right now.

Additionally, we already have coverage on the buildbots for 32-bit flavors of
architectures.