schwern/test-more — pull request #291 by ribasushi (2012-04-25 07:10:31+00:00)
Title: Test builder1.5

Minimal fixes, more to come.

Currently outstanding issues are 5.15.recent failures, < 5.8.9 failures, and an intermittent failure (timing issues) with t/threads_shared.t

Disabling the 5.8 thread tests is not advised, because while it will "pass" tests for T::B itself, anyone using threads in *their* tests will see the same segfaults, and start searching for a non-existent flaw in their own dist.

Current "smoke" status:

```
rabbit@Thesaurus:~/devel/test-more$ AUTHOR_TESTING=1 HARNESS_OPTIONS=j10 perlbrew exec bash -c '(perl Makefile.PL && make realclean && perl Makefile.PL && make)&>/dev/null && make test 2>&1 | tail -n 10'

5.10.0_2
==========
t/useing.t ................................. ok
t/undef.t .................................. ok
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/threads.t ................................ ok
t/exit.t ................................... ok
t/threads_shared.t ......................... ok
All tests successful.
Files=212, Tests=3592, 12 wallclock secs ( 1.76 usr 0.44 sys + 46.12 cusr 2.26 csys = 50.58 CPU)
Result: PASS

5.10.0_dbg1
==========
t/undef.t .................................. ok
t/use_ok.t ................................. ok
t/versions.t ............................... ok
t/subtest/todo.t ........................... ok
t/utf8.t ................................... ok
t/useing.t ................................. ok
t/exit.t ................................... ok
All tests successful.
Files=212, Tests=2134, 8 wallclock secs ( 1.44 usr 0.46 sys + 44.76 cusr 2.06 csys = 48.72 CPU)
Result: PASS

5.10.1
==========
t/use_ok.t ................................. ok
t/versions.t ............................... ok
t/use_ok_leaking.t ......................... ok
t/utf8.t ................................... ok
t/threads.t ................................ ok
t/exit.t ................................... ok
t/threads_shared.t ......................... ok
All tests successful.
Files=212, Tests=3662, 13 wallclock secs ( 1.78 usr 0.38 sys + 46.36 cusr 2.36 csys = 50.88 CPU)
Result: PASS

5.10.1_dbg1
==========
t/use_ok_leaking.t ......................... ok
t/useing.t ................................. ok
t/use_ok.t ................................. ok
t/subtest/todo.t ........................... ok
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/exit.t ................................... ok
All tests successful.
Files=212, Tests=2066, 7 wallclock secs ( 1.40 usr 0.36 sys + 41.46 cusr 2.06 csys = 45.28 CPU)
Result: PASS

5.10.1_no_thr
==========
t/versions.t ............................... ok
t/use_ok_leaking.t ......................... ok
t/undef.t .................................. ok
t/useing.t ................................. ok
t/use_ok.t ................................. ok
t/utf8.t ................................... ok
t/exit.t ................................... ok
All tests successful.
Files=212, Tests=2134, 7 wallclock secs ( 1.20 usr 0.34 sys + 38.10 cusr 2.02 csys = 41.66 CPU)
Result: PASS

5.12.0_dbg1
==========
t/use_ok_leaking.t ......................... ok
t/useing.t ................................. ok
t/use_ok.t ................................. ok
t/undef.t .................................. ok
t/utf8.t ................................... ok
t/versions.t ............................... ok
t/exit.t ................................... ok
All tests successful.
Files=212, Tests=2066, 8 wallclock secs ( 1.38 usr 0.42 sys + 44.38 cusr 2.20 csys = 48.38 CPU)
Result: PASS

5.12.1
==========
Failed 1/3 subtests

Test Summary Report
-------------------
t/threads_shared.t (Wstat: 0 Tests: 3 Failed: 1)
Failed test: 1
Files=212, Tests=3662, 12 wallclock secs ( 1.66 usr 0.48 sys + 47.78 cusr 2.64 csys = 52.56 CPU)
Result: FAIL
Failed 1/212 test programs. 1/3662 subtests failed.
make: *** [test_dynamic] Error 255

5.12.2
==========
t/use_ok_leaking.t ......................... ok
t/use_ok.t ................................. ok
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/useing.t ................................. ok
t/undef.t .................................. ok
t/exit.t ................................... ok
All tests successful.
Files=212, Tests=2134, 7 wallclock secs ( 1.24 usr 0.36 sys + 40.86 cusr 2.26 csys = 44.72 CPU)
Result: PASS

5.12.3
==========
t/use_ok.t ................................. ok
t/useing.t ................................. ok
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/threads.t ................................ ok
t/exit.t ................................... ok
t/threads_shared.t ......................... ok
All tests successful.
Files=212, Tests=3660, 12 wallclock secs ( 1.74 usr 0.58 sys + 49.84 cusr 2.50 csys = 54.66 CPU)
Result: PASS

5.12.4_dbg1
==========
t/use_ok.t ................................. ok
t/undef.t .................................. ok
t/use_ok_leaking.t ......................... ok
t/useing.t ................................. ok
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/exit.t ................................... ok
All tests successful.
Files=212, Tests=2066, 8 wallclock secs ( 1.40 usr 0.36 sys + 43.96 cusr 2.24 csys = 47.96 CPU)
Result: PASS

5.14.1_thr
==========
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/use_ok_leaking.t ......................... ok
t/useing.t ................................. ok
t/threads.t ................................ ok
t/exit.t ................................... ok
t/threads_shared.t ......................... ok
All tests successful.
Files=212, Tests=3660, 9 wallclock secs ( 1.66 usr 0.24 sys + 47.18 cusr 2.72 csys = 51.80 CPU)
Result: PASS

5.14.1_thr_mb_dbg
==========
t/useing.t ................................. ok
t/use_ok_leaking.t ......................... ok
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/threads.t ................................ ok
t/exit.t ................................... ok
t/threads_shared.t ......................... ok
All tests successful.
Files=212, Tests=3592, 14 wallclock secs ( 2.08 usr 0.24 sys + 56.90 cusr 2.80 csys = 62.02 CPU)
Result: PASS

5.14.2_dbg_thr_mb
==========
Failed 1/3 subtests

Test Summary Report
-------------------
t/threads_shared.t (Wstat: 0 Tests: 3 Failed: 1)
Failed test: 1
Files=212, Tests=3592, 10 wallclock secs ( 1.98 usr 0.30 sys + 55.88 cusr 2.78 csys = 60.94 CPU)
Result: FAIL
Failed 1/212 test programs. 1/3592 subtests failed.
make: *** [test_dynamic] Error 255

5.14.2_thr_cln
==========
t/versions.t ............................... ok
t/useing.t ................................. ok
t/utf8.t ................................... ok
t/use_ok.t ................................. ok
t/threads.t ................................ ok
t/exit.t ................................... ok
t/threads_shared.t ......................... ok
All tests successful.
Files=212, Tests=3662, 12 wallclock secs ( 1.60 usr 0.26 sys + 47.28 cusr 2.42 csys = 51.56 CPU)
Result: PASS

5.15.8
==========
Tests out of sequence. Found (5) but expected (7)
Tests out of sequence. Found (5) but expected (8)
Tests out of sequence. Found (5) but expected (9)
Tests out of sequence. Found (5) but expected (10)
Displayed the first 5 of 6 TAP syntax errors.
Re-run prove with the -p option to see them all.
Files=212, Tests=3665, 12 wallclock secs ( 1.70 usr 0.30 sys + 61.18 cusr 2.82 csys = 66.00 CPU)
Result: FAIL
Failed 1/212 test programs. 5/3665 subtests failed.
make: *** [test_dynamic] Error 255

5.15.9
==========
Tests out of sequence. Found (5) but expected (7)
Tests out of sequence. Found (5) but expected (8)
Tests out of sequence. Found (5) but expected (9)
Tests out of sequence. Found (5) but expected (10)
Displayed the first 5 of 6 TAP syntax errors.
Re-run prove with the -p option to see them all.
Files=212, Tests=3665, 13 wallclock secs ( 1.70 usr 0.30 sys + 61.26 cusr 2.80 csys = 66.06 CPU)
Result: FAIL
Failed 1/212 test programs. 5/3665 subtests failed.
make: *** [test_dynamic] Error 255

5.6.2
==========

5.8.1
==========
t/subtest/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=2128, 14 wallclock secs ( 1.38 usr 0.52 sys + 45.88 cusr 2.64 csys = 50.42 CPU)
Result: FAIL
Failed 6/212 test programs. 0/2128 subtests failed.
make: *** [test_dynamic] Error 255

5.8.1_dbg_mb
==========
t/subtest/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=2128, 12 wallclock secs ( 1.30 usr 0.52 sys + 44.76 cusr 2.68 csys = 49.26 CPU)
Result: FAIL
Failed 6/212 test programs. 0/2128 subtests failed.
make: *** [test_dynamic] Error 255

5.8.2
==========
t/subtest/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=2060, 14 wallclock secs ( 1.32 usr 0.56 sys + 44.36 cusr 2.86 csys = 49.10 CPU)
Result: FAIL
Failed 6/212 test programs. 0/2060 subtests failed.
make: *** [test_dynamic] Error 255

5.8.3
==========
t/subtest/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=2128, 9 wallclock secs ( 1.34 usr 0.50 sys + 45.90 cusr 2.70 csys = 50.44 CPU)
Result: FAIL
Failed 6/212 test programs. 0/2128 subtests failed.
make: *** [test_dynamic] Error 255

5.8.3_cln
==========
t/Tester/import.t (Wstat: 65280 Tests: 0 Failed: 0)
Non-zero exit status: 255
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=1942, 12 wallclock secs ( 1.26 usr 0.54 sys + 43.94 cusr 2.74 csys = 48.48 CPU)
Result: FAIL
Failed 11/212 test programs. 0/1942 subtests failed.
make: *** [test_dynamic] Error 255

5.8.3_dbg_thr
==========
t/subtest/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=2060, 12 wallclock secs ( 2.22 usr 0.52 sys + 75.38 cusr 2.98 csys = 81.10 CPU)
Result: FAIL
Failed 6/212 test programs. 0/2060 subtests failed.
make: *** [test_dynamic] Error 255

5.8.4
==========
t/subtest/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=2128, 8 wallclock secs ( 1.40 usr 0.50 sys + 45.66 cusr 2.84 csys = 50.40 CPU)
Result: FAIL
Failed 6/212 test programs. 0/2128 subtests failed.
make: *** [test_dynamic] Error 255

5.8.5
==========
t/subtest/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
t/threads.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=2128, 10 wallclock secs ( 1.44 usr 0.50 sys + 46.56 cusr 2.70 csys = 51.20 CPU)
Result: FAIL
Failed 6/212 test programs. 0/2128 subtests failed.
make: *** [test_dynamic] Error 255

5.8.6
==========

Test Summary Report
-------------------
t/Streamer/ThreadSafeFilehandleAccessor.t (Wstat: 11 Tests: 4 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=3659, 8 wallclock secs ( 1.84 usr 0.50 sys + 49.36 cusr 2.96 csys = 54.66 CPU)
Result: FAIL
Failed 1/212 test programs. 0/3659 subtests failed.
make: *** [test_dynamic] Error 255

5.8.7
==========

Test Summary Report
-------------------
t/Streamer/ThreadSafeFilehandleAccessor.t (Wstat: 11 Tests: 4 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=3659, 10 wallclock secs ( 1.78 usr 0.58 sys + 48.94 cusr 2.82 csys = 54.12 CPU)
Result: FAIL
Failed 1/212 test programs. 0/3659 subtests failed.
make: *** [test_dynamic] Error 255

5.8.8
==========

Test Summary Report
-------------------
t/Streamer/ThreadSafeFilehandleAccessor.t (Wstat: 11 Tests: 4 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=3661, 13 wallclock secs ( 1.84 usr 0.54 sys + 49.36 cusr 2.90 csys = 54.64 CPU)
Result: FAIL
Failed 1/212 test programs. 0/3661 subtests failed.
make: *** [test_dynamic] Error 255

5.8.8_dbg
==========

Test Summary Report
-------------------
t/Streamer/ThreadSafeFilehandleAccessor.t (Wstat: 11 Tests: 4 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=212, Tests=3591, 13 wallclock secs ( 1.98 usr 0.48 sys + 55.84 cusr 3.08 csys = 61.38 CPU)
Result: FAIL
Failed 1/212 test programs. 0/3591 subtests failed.
make: *** [test_dynamic] Error 255

5.8.9
==========
t/use_ok_leaking.t ......................... ok
t/useing.t ................................. ok
t/versions.t ............................... ok
t/utf8.t ................................... ok
t/threads.t ................................ ok
t/exit.t ................................... ok
t/threads_shared.t ......................... ok
All tests successful.
Files=212, Tests=3662, 10 wallclock secs ( 1.84 usr 0.52 sys + 50.62 cusr 2.68 csys = 55.66 CPU)
Result: PASS

5.8.9_dbg1
==========
t/todo.t ................................... ok
t/use_ok_leaking.t ......................... ok
t/Types/stringify.t ........................ ok
t/utf8.t ................................... ok
t/useing.t ................................. ok
t/versions.t ............................... ok
t/exit.t ................................... ok
All tests successful.
Files=212, Tests=2066, 7 wallclock secs ( 1.32 usr 0.52 sys + 41.76 cusr 2.38 csys = 45.98 CPU)
Result: PASS
```