[MRG] Test common refactor

This actually makes the tests faster and hopefully it will make the actual testing file way better to maintain.
I am not fan of this.