sebastianbergmann/phpunit — Issue #498: The test methods won't be run if a dataProvider throws Exception
Author: elzenar
Created: 2012-02-13 21:18:48+00:00
State: open

Hello!

I accidentally found the following strange behaviour.

For example we have a file 'SomeTestClass.php' with the following content:

```php

class SomeTestClass extends PHPUnit_Framework_TestCase
{

/**
* @test
* @dataProvider shouldBeTrueDataProvider
* @group falseOnly
*/
public function shouldBeTrue($testData)
{
$this->assertTrue(true);
}

/**
* @test
* @dataProvider shouldBeFalseDataProvider
* @group trueOnly
*/
public function shouldBeFalse($testData)
{
$this->assertFalse(false);
}

public function shouldBeTrueDataProvider()
{

//throw new Exception("Can't create the data");
return array(
array(true),
array(false)
);
}

public function shouldBeFalseDataProvider()
{

throw new Exception("Can't create the data");
return array(
array(true),
array(false)
);
}
}

```

If run the command:

```bash

phpunit --group trueOnly .

```

the following result will be shown:

```

PHPUnit 3.6.4 by Sebastian Bergmann.

Time: 0 seconds, Memory: 3.00Mb

OK (0 tests, 0 assertions)

```

but if run

```bash

phpunit --group falseOnly .

```

the result will be:

```

PHPUnit 3.6.4 by Sebastian Bergmann.

..

Time: 0 seconds, Memory: 3.75Mb

OK (2 tests, 2 assertions)

```

**In the first case the test method 'shouldBeTrue()' wasn't run never once.**

In addition if run the command:

```

phpunit .

```

the following result will be shown:

```

PHPUnit 3.6.4 by Sebastian Bergmann.

..F

Time: 0 seconds, Memory: 3.75Mb

There was 1 failure:

1) Warning
The data provider specified for SomeTestClass::shouldBeFalse is invalid.
Can't create the data

FAILURES!
Tests: 3, Assertions: 2, Failures: 1.

```

I made a research and I think the reason is in the following code

```

243 /**
244 * Adds a test to the suite.
245 *
246 * @param PHPUnit_Framework_Test $test
247 * @param array $groups
248 */
249 public function addTest(PHPUnit_Framework_Test $test, $groups = array())
250 {
251 $class = new ReflectionClass($test);
252
253 if (!$class->isAbstract()) {
254 $this->tests[] = $test;
255 $this->numTests = -1;
256
257 if ($test instanceof PHPUnit_Framework_TestSuite &&
258 empty($groups)) {
259 $groups = $test->getGroups();
260 }
261
262 if (empty($groups)) {
263 $groups = array('__nogroup__');
264 }
265
266 foreach ($groups as $group) {
267 if (!isset($this->groups[$group])) {
268 $this->groups[$group] = array($test);
269 } else {
270 $this->groups[$group][] = $test;
271 }
272 }
273 }
274 }

```

If in dataProvider a exception be thrown the argument $test will instance of PHPUnit_Framework_Warning. In this case the code at line 259 won't be run.

I hope this information help us to make PHPUnit better.