gabrielfalcao/lettuce — pull request #449 by muhammad-ammar (2014-06-07 21:50:55+00:00)
Title: Re-run flaky tests

Implemented flaky tag which will automatically re-run flaky tests. It is inspired from [flaky](https://github.com/box/flaky)

There are two ways flaky tag can be used.
1. @flaky
2. @flaky(max_runs, min_passes)

> max_runs: Total number of times a Scenario can run.
> min_passes: Total number of times a Scenario should pass.

@flaky without any parameters have```max_runs = 2``` and ```min_passes = 1```

##### Rules:
1. min_passes can't be greater than max_runs
2. zero and negative values will not be entertained.

In presence of either of above two conditions ```max_runs``` and ```min_passes``` will both set to ```1```
If there is no @flaky tag then max_runs and min_passes will both set to 1

##### Examples:

To mark a Scenario as flaky, place the @flaky tag above the ```Scenario```
```python
@flaky
Scenario: This scenario pass randomly
Given this test step pass
```
The above is a non-parametrized flaky tag

```python
@flaky(4, 3)
Scenario: This scenario pass randomly
Given this test step pass
```
The above is a parametrized flaky tag

@flaky tag can be applied to a ```Feature```

```python
@flaky
Feature: Test a feature with flaky tag

Scenario: This scenario always Fail
Given this test step fail

@flaky(3,2)
Scenario: This scenario always Pass
Given this test step pass
```

In the above example ```default``` values for flaky tag will be applied to ```Scenario: This scenario always Fail``` but ```Scenario: This scenario always Pass``` have explicitly mentioned the flaky values so these will override the default values.