[WIP] Make conftest remove tests instead of skipping.

Fixes #976.

Let's see if this workaround actually works. 
Based on looking at pytest's code, does this handle classes that are decorated with these correctly?