riptano/cassandra-dtest — pull request #122 by knifewine (2014-11-27 02:25:39+00:00)
Title: add support for "scenarios", add more testing for paging with limit

I set out to add some missing test coverage for paging and ended up implementing a new dtest feature to support the new tests. I think it's a Good Idea but scrutiny would be much appreciated.

Basically I was looking at adding a bunch of tests with similar logic and a bunch of inscrutible method names like test_paging_with_limit_when_limit_is_less_than_fetch_and_fetch_is_less_than_data_size ... well, you get the idea. I realized I just wanted a repeatable test template to be run with multiple configurations (regrettably we can't use nose's test generators because our tests classes are subclasses of unittest.TestCase). But the problem with a template is if you plan to run it for 10 configurations, but it dies on the 3rd configuration the failure could be masking other failures in the remaining configurations. Anyway, I think I came up with a good solution to this.

The proposal here is to add a concept of "scenarios" where the same chunk of code can run several times with different inputs. Its meant to mostly be used from single tests where we want coverage of a bunch of different permutations, but these permutations would be unwieldy or heavy on duplication if each had to be in its own test method. This lowers duplication, and prevents overhead from bringing up new ccm clusters and the like. It also allows deferring exceptions so we can run a bunch of scenarios and then raise an exception when they are all complete.

This doesn't make any fundamental changes to the core of dtest functionality so we could try it out for a while and remove it if it's not beneficial.

I've got the new paging tests set to use it, and you can try it out by running:
nosetests -vs paging_test.py:TestPagingWithModifiers.test_with_limit

One scenario is actually triggering what appears to be a bug with 'limit' so you can see what the deferred exceptions look like by trying it out.