This is more or less an adapted copy paste straight from python's unittest module. Couldn't find any way of reusing the original (nose does the same thing).