Revert "Use Travis to handle the Matrix"

This reverts commit 01894aee7be04fb0756f4e73a462ccf5ae2c886f.
Merge pull request #1550 from dstufft/revert-dynamic-toxenv

Revert "Use Travis to handle the Matrix"
Cleanup after each test case