Test the collection of tests

This is to workaround pytest issues #149 / #180 where distributed
testing doesn't error out if it fails to collect tests.