(PUP-1842) Make running the spec tests on 3.2.x work the same as master

3.2.x is the branch we're shipping in PE right now. We should be
testing it more or less the same way we test master.