schwern/test-more — Issue #250: Return value of subtest()
Author: guillaumeaubert
Created: 2012-01-21 23:55:29+00:00
State: open

I just noticed that subtest() in 0.94 returns a boolean indicating whether the subtests where successful, but 1.005000_002 doesn't. This feature is very useful for test modules to aggregate the results of a block of tests (I use it in Test::Dist::VersionSync for example).

Simple script to demonstrate the difference:

```
#!perl

use strict;
use warnings;

use Data::Dumper;
use Test::More tests => 2;

my $subtest1 = subtest(
'A subtest that succeeds.',
sub
{
ok( 1, 'Fine' );
ok( 1, 'Fine' );
}
);
print "Return value for successful subtests is: " . Dumper( $subtest1 ) . "\n";

my $subtest2 = subtest(
'A subtest that fails.',
sub
{
ok( 1, 'Fine' );
ok( 0, 'Not fine' );
}
);
print "Return value for failing subtests is: " . Dumper( $subtest2 ) . "\n";
```

With 0.94, this returns:

guillaume@avalon:~/Documents$ perl subtest.pl
1..2
ok 1 - Fine
ok 2 - Fine
1..2
ok 1 - A subtest that succeeds.
Return value for successful subtests is: $VAR1 = 1;

ok 1 - Fine
not ok 2 - Not fine
# Failed test 'Not fine'
# at subtest.pl line 24.
1..2
# Looks like you failed 1 test of 2.
not ok 2 - A subtest that fails.
# Failed test 'A subtest that fails.'
# at subtest.pl line 26.
Return value for failing subtests is: $VAR1 = 0;

# Looks like you failed 1 test of 2.

With 1.005000_002, this returns:

guillaume@avalon:~/Documents$ perl -Iuserlib/Test-Simple-1.005000_002/lib/ subtest.pl
TAP version 13
1..2
TAP version 13
ok 1 - Fine
ok 2 - Fine
1..2
ok 1 - A subtest that succeeds.
Return value for successful subtests is: $VAR1 = undef;

TAP version 13
ok 1 - Fine
not ok 2 - Not fine
# Failed test 'Not fine'
# at subtest.pl line 24.
1..2
# 1 test of 2 failed.
not ok 2 - A subtest that fails.
# Failed test 'A subtest that fails.'
# at subtest.pl line 26.
Return value for failing subtests is: $VAR1 = undef;

# 1 test of 2 failed.

If this feature was not intentionally removed, I'm willing to take a look at https://github.com/schwern/test-more/blob/Test-Builder1.5/lib/Test/Builder.pm and see what could be returned at line 188 to make it work again. Let me know!