Fix some active test suite running issues

This is a cherry-pick of a commit from #2034.  Since it's a bona fide bug in master, I think it makes sense to get this in before #2034 is fully resolved.  Assuming Travis passes, I'll go ahead and merge.