Bug 847953 - Don't reset the testsuite for every new metric we add