Revert "Make it easy to install test deps given in tox"

Tox didnt like

This reverts commit 0946416e645db4f75f8a468fbf020129d756aee6.