Disable more of the Travis spec build for PR validation

So as to avoid:

	$ ssh-add -D
	All identities removed.
	$ ssh-add spec/id_dsa_travis
	Enter passphrase for spec/id_dsa_travis:
	No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.

Review by @adriaanm