Test contents were somehow duplicated, resulting in any testcase that fails automatically failing twice.  Removing the duplicates.