Get rid of test_utils (bug 1090184)

Fixes [bug 1090184](https://bugzilla.mozilla.org/show_bug.cgi?id=1090184)

This is heavily inspired by https://github.com/mozilla/zamboni/pull/2681/files
I wonder if we should do that in the `tearDownClass` below instead?