Ovid/test-class-moose — pull request #29 by autarch (2014-03-28 16:20:59+00:00)
Title: Multiple test class instance first pass

This is for review only. I don't think you should merge this right now.

There are a couple ways in which this branch sucks:

The parameterized test instance role is almost certainly broken when combined with the parallel role. This would require an even deeper refactoring to fix. I think the right way to do this is to move this functionality into `Test::Class::Moose` and default to a single job. Anything else is going to be a mess, I suspect.

Alternatively, we could probably just implement a simple `schedule` method in TCM and let the role override that.

There are a number of hacks here to get around the fact that the code assumes that there is only a single test instance per test class. For example, there are various places that the class name is used as a test name. I've added a `_tcm_instance_name` that defaults to the test class name, but I'm not sure this is really sufficient. It might require deeper changes in the reporting classes to really make sense. At the very least, `Test::Class::Moose::Report::Class` is very confusing right now, since if a class produces multiple instances all the counts, method lists, etc. go in a single report. It might make more sense to instead have a `Test::Class::Moose::Report::Instance` class and create one per instance.

That all said, modulo the parallel incompability, this does seem to pretty much "just work" and the resulting test report is pretty readable. Run `prove -lv t/t/parameterized.t` to see what I mean. You'll note that you get an extra level of subtest, one per test class instance, and each of those subtests has a unique name.

On a side note, the design of Test::Class::Moose is quite confusing. The fact that the test runner is a bare `Test::Class::Moose` object while the classes being tested are subclasses of `Test::Class::Moose` is my issue. It would make a lot more sense to me if the test running and the "being a test class" functionality were separated into two entirely separate classes with a well-defined interface between them. While working on this patch I kept getting confused about what changes were needed for the sake of the runner code and what were needed for the sake of classes being tested. I think you could maintain some backwards compatibility by moving running to a new class and making `Test::Class::Moose->runtests` delegate to that runner (with a deprecation warning at some point).