cbeust/testng — pull request #229 by fpavageau (2012-06-08 16:20:03+00:00)
Title: More performance improvements

Following your comment in https://github.com/cbeust/testng/pull/226, and since the testng-failed.xml generation still took more than 7 minutes (way better than the 41 minutes it took with 6.5.2, but still...), I looked deeper into the problem.

With my project, I get a bit more than 23K calls to MethodGroupHelper.findMethodsThatBelongToGroup(), but simply caching the Patterns has a negligible effect. However, caching the matches has much more impact, as does pre-computing the method signatures, as there are lots of calls to BaseTestMethod.toString() during the topological sort.

I now have the following results:
- 6.5.3-SNAPSHOT unmodified:
Tests run: 7900, Failures: 415, Errors: 0, Skipped: 7271, Time elapsed: 393.218 sec <<< FAILURE!
- 6.5.3-SNAPSHOT with match caches:
Tests run: 7900, Failures: 415, Errors: 0, Skipped: 7271, Time elapsed: 232.967 sec <<< FAILURE!

With Yourkit, I also have the following results regarding garbage generation:
- 6.5.3-SNAPSHOT unmodified: 3.81 GB of garbage generated by FailedReporter.generateReport()
- 6.5.3-SNAPSHOT with all the modifications: 360 MB of garbage

Most of the garbage was the method signatures and the matchers, and it's a lot more than the Pairs that are now created for the cache lookups.

Of course, the generation still takes almost 4 minutes, but now I think it's essentially due to the topological sort algorithm and the number of dependencies (lots of tests with Spring injection, so a lot of before/after methods).