Need to use the entire env matrix

In order to allow failures in travis it looks like we have to use the
entire set of environmental variables.