travis: minimize the dependencies available for each build config

This is a more realistic representation of a build environment. Helps to ensure
configure logic and that headers aren't accidentally assumed to be present.