nose-devs/nose — Issue #110: Feature request: Attribute to determine order of test execution for a test class
Author: jpellerin
Created: 2011-12-14 15:59:36+00:00
State: open

GOAL:

I would like Nose to be able to have more control over the order of test execution for test classes that does NOT rely on alphabetical discovery of the method names. I don't want discovery via the method names per se, I would like the order to be determined via an attribute of the class that is a list of the method names where the ordering of the items in the list determines the ordering of the test execution. For example, let's say if we define an attribute for any class like this:

self.__testing_order__ = ['test_foo', 'test_bar']

the result would be that 'test_foo' would run first despite coming after 'test_bar' alphabetically.

USE CASE:

1. I'm using Nose to test a database (since there are very few database testing tools available that are effective). Most database testing tools are crude, not cross-platform, not friendly with other testing tools, DBMS specific, expensive, or all of these.

2. Because databases are stateful between test runs, database testing isn't effective unless you have some way of ensuring you reset the state when you need to, such as by dropping and recreating the database. But other tests don't necessarily need a full "clean slate".

3. Recreating a database is a time-consuming operation. One of the reasons why I'm using Nose rather than unittest is that unittest forces you to use the fixtures per each test, whereas test classes with Nose let you run the fixtures per class. This allows me to recreate the database once, create the connections, run the tests in the desired order, then disconnect. It's simply not feasible to have the database be recreated for every single test, nor is it necessary for the semantics of the tests (which expect data to be cumulative from previous commits, but this test session as a *whole* is a "clean slate" due to recreating the database automatically).

4. I tried using the alphabetical way via naming the methods in the format of test_0001_* , but if you ever want to change the order of tests for even a single test, every following test must be renamed. Obviously if I end up with thousands of tests, I don't want to have to rename every one of them for the sake of moving a single test somewhere else in the execution order. With a list, I simply cut and paste the test name somewhere else in the list.

P.S. Sorry if this was posted to the wrong place, there didn't seem to be a "feature request" option when reporting a "new issue".

Google Code Info:
Issue #: 344
Author: robwalke...@gmail.com
Created On: 2010-06-17T16:13:06.000Z
Closed On: 2011-06-01T17:41:25.000Z