cbeust/testng — Issue #188: suite parallel="methods" does not work when there are multiple <test> tags
Author: sthadeshwar
Created: 2012-03-08 23:33:43+00:00
State: open

suite parallel="methods" does not work when there are multiple tags and multiple testng xml files

Below is the example:

testng.xml:

Below is a sample class A.java and all other test classes mentioned in above xml have similar dummy tests which just prints the test name and the thread name:

@Test
public void testA1() {
System.out.println("Executing testA1 : " + Thread.currentThread().getName());
System.out.println("Waiting for 10 secs inside testA1 : " + System.currentTimeMillis());
CommonUtils.sleep(10);
System.out.println("Done waiting for 10 secs inside testA1 : " + System.currentTimeMillis());
}

@Test
public void testA2() {
System.out.println("Executing testA2 : " + Thread.currentThread().getName());
}

@Test
public void testA3() {
System.out.println("Executing testA3 : " + Thread.currentThread().getName());
}
....
....

Below is the output:

run:
[testng] [TestNGAntTask] TESTNG PASSED @/var/folders/Xr/XrWzi723gI0qWejNi3tO+JPvw5I/-Tmp-/testng5975172026000129049 WHICH CONTAINS:
[testng] -log
[testng] 2
[testng] -suitename
[testng] Ant suite
[testng] -testname
[testng] Ant test
[testng] /path-to-my-testng-file/run-testng.xml
[testng] [TestNG] Running:
[testng] /path-to-my-testng-file/run-testng.xml
[testng]
[testng] [TestRunner] Starting executor for test AB with time out:2147483647 milliseconds.
[testng] Executing testB5 : pool-2-thread-10
[testng] Executing testB4 : pool-2-thread-9
[testng] Executing testB2 : pool-2-thread-7
[testng] Executing testA1 : pool-2-thread-1
[testng] Executing testA3 : pool-2-thread-3
[testng] Executing testB1 : pool-2-thread-6
[testng] Executing testA5 : pool-2-thread-5
[testng] Executing testB3 : pool-2-thread-8
[testng] Executing testA4 : pool-2-thread-4
[testng] Executing testA2 : pool-2-thread-2
[testng] Waiting for 10 secs inside testA1 : 1331249133317
[testng] Done waiting for 10 secs inside testA1 : 1331249143320
[testng] PASSED: testB4
[testng] PASSED: testB5
[testng] PASSED: testB2
[testng] PASSED: testA3
[testng] PASSED: testB1
[testng] PASSED: testA5
[testng] PASSED: testB3
[testng] PASSED: testA4
[testng] PASSED: testA2
[testng] PASSED: testA1
[testng]
[testng] ===============================================
[testng] AB
[testng] Tests run: 10, Failures: 0, Skips: 0
[testng] ===============================================
[testng]
[testng] Executing testC1 : pool-3-thread-1
[testng] Executing testC3 : pool-3-thread-3
[testng] Executing testC2 : pool-3-thread-2
[testng] Executing testC4 : pool-3-thread-4
[testng] Executing testC5 : pool-3-thread-5
[testng] Executing testD1 : pool-3-thread-6
[testng] Executing testD2 : pool-3-thread-7
[testng] [TestRunner] Starting executor for test CD with time out:2147483647 milliseconds.
[testng] Executing testD3 : pool-3-thread-8
[testng] Executing testD4 : pool-3-thread-9
[testng] Executing testD5 : pool-3-thread-10
[testng] PASSED: testC1
[testng] PASSED: testC3
[testng] PASSED: testC2
[testng] PASSED: testC4
[testng] PASSED: testC5
[testng] PASSED: testD1
[testng] PASSED: testD2
[testng] PASSED: testD3
[testng] PASSED: testD4
[testng] PASSED: testD5
[testng]
[testng] ===============================================
[testng] CD
[testng] Tests run: 10, Failures: 0, Skips: 0
[testng] ===============================================
[testng]
[testng] Executing testE1 : pool-4-thread-1
[testng] Executing testE2 : pool-4-thread-2
[testng] Executing testE3 : pool-4-thread-3
[testng] Executing testE4 : pool-4-thread-4
[testng] Executing testE5 : pool-4-thread-5
[testng] Fxecuting testF1 : pool-4-thread-6
[testng] Fxecuting testF2 : pool-4-thread-7
[testng] [TestRunner] Starting executor for test EF with time out:2147483647 milliseconds.
[testng] Fxecuting testF3 : pool-4-thread-8
[testng] Fxecuting testF4 : pool-4-thread-9
[testng] Fxecuting testF5 : pool-4-thread-10
[testng] PASSED: testE2
[testng] PASSED: testE3
[testng] PASSED: testE4
[testng] PASSED: testE5
[testng] PASSED: testF1
[testng] PASSED: testF2
[testng] PASSED: testF3
[testng] PASSED: testF4
[testng] PASSED: testF5
[testng] PASSED: testE1
[testng]
[testng] ===============================================
[testng] EF
[testng] Tests run: 10, Failures: 0, Skips: 0
[testng] ===============================================
[testng]
[testng]
[testng] ===============================================
[testng] p,q suite(s)
[testng] Total tests run: 30, Failures: 0, Skips: 0
[testng] ===============================================
[testng]
[testng] [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@2a9df354: 25 ms
[testng] [TestNG] Time taken by [TestListenerAdapter] Passed:0 Failed:0 Skipped:0]: 0 ms
[testng] [TestNG] Time taken by org.testng.reporters.EmailableReporter@51b48197: 6 ms
[testng] [TestNG] Time taken by org.testng.reporters.JUnitReportReporter@58ecb281: 6 ms
[testng] [TestNG] Time taken by org.testng.reporters.XMLReporter@3a6ac461: 13 ms

BUILD SUCCESSFUL
Total time: 13 seconds

From the output you can see that the execution gets blocked at testA1's sleep() for 10 seconds. Also, testng creates 3 thread-pools since there are 3 tags and executes the test tags sequentially.

************************************************************************************************************************
Now if I update my testng.xml to something like this:

It works just as expected: It will run ALL the methods in parallel in its own thread:

run:
[testng] [TestNGAntTask] TESTNG PASSED @/var/folders/Xr/XrWzi723gI0qWejNi3tO+JPvw5I/-Tmp-/testng3337858474629479499 WHICH CONTAINS:
[testng] -log
[testng] 2
[testng] -suitename
[testng] Ant suite
[testng] -testname
[testng] Ant test
[testng] /path-to-my-testng-file/run-testng.xml
[testng] [TestNG] Running:
[testng] /path-to-my-testng-file/run-testng.xml
[testng]
[testng] [TestRunner] Starting executor for test ABCDEF with time out:2147483647 milliseconds.
[testng] Executing testC1 : pool-2-thread-11
[testng] Executing testA5 : pool-2-thread-5
[testng] Executing testB1 : pool-2-thread-6
[testng] Executing testE1 : pool-2-thread-21
[testng] Executing testA1 : pool-2-thread-1
[testng] Executing testD4 : pool-2-thread-19
[testng] Executing testA4 : pool-2-thread-4
[testng] Fxecuting testF4 : pool-2-thread-29
[testng] Executing testA3 : pool-2-thread-3
[testng] Executing testB4 : pool-2-thread-9
[testng] Executing testE3 : pool-2-thread-23
[testng] Executing testA2 : pool-2-thread-2
[testng] Executing testB5 : pool-2-thread-10
[testng] Executing testE5 : pool-2-thread-25
[testng] Executing testD3 : pool-2-thread-18
[testng] Executing testE2 : pool-2-thread-22
[testng] Executing testD2 : pool-2-thread-17
[testng] Executing testE4 : pool-2-thread-24
[testng] Executing testD1 : pool-2-thread-16
[testng] Executing testB2 : pool-2-thread-7
[testng] Waiting for 10 secs inside testA1 : 1331249533970
[testng] Fxecuting testF3 : pool-2-thread-28
[testng] Executing testC2 : pool-2-thread-12
[testng] Executing testB3 : pool-2-thread-8
[testng] Executing testC4 : pool-2-thread-14
[testng] Fxecuting testF5 : pool-2-thread-30
[testng] Executing testD5 : pool-2-thread-20
[testng] Executing testC3 : pool-2-thread-13
[testng] Fxecuting testF1 : pool-2-thread-26
[testng] Executing testC5 : pool-2-thread-15
[testng] Fxecuting testF2 : pool-2-thread-27
[testng] Done waiting for 10 secs inside testA1 : 1331249543974
[testng] PASSED: testA5
[testng] PASSED: testB1
[testng] PASSED: testC1
[testng] PASSED: testD4
[testng] PASSED: testA4
[testng] PASSED: testF4
[testng] PASSED: testA3
[testng] PASSED: testB4
[testng] PASSED: testE3
[testng] PASSED: testA2
[testng] PASSED: testB5
[testng] PASSED: testE5
[testng] PASSED: testD3
[testng] PASSED: testE2
[testng] PASSED: testD2
[testng] PASSED: testE4
[testng] PASSED: testD1
[testng] PASSED: testB2
[testng] PASSED: testF3
[testng] PASSED: testC2
[testng] PASSED: testB3
[testng] PASSED: testC4
[testng] PASSED: testF5
[testng] PASSED: testD5
[testng] PASSED: testC3
[testng] PASSED: testF1
[testng] PASSED: testC5
[testng] PASSED: testF2
[testng] PASSED: testE1
[testng] PASSED: testA1
[testng]
[testng] ===============================================
[testng] ABCDEF
[testng] Tests run: 30, Failures: 0, Skips: 0
[testng] ===============================================
[testng]
[testng]
[testng] ===============================================
[testng] p,q suite(s)
[testng] Total tests run: 30, Failures: 0, Skips: 0
[testng] ===============================================
[testng]
[testng] [TestNG] Time taken by [TestListenerAdapter] Passed:0 Failed:0 Skipped:0]: 0 ms
[testng] [TestNG] Time taken by org.testng.reporters.JUnitReportReporter@2acdb06e: 10 ms
[testng] [TestNG] Time taken by org.testng.reporters.XMLReporter@6fc5f743: 12 ms
[testng] [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@15a62c31: 17 ms
[testng] [TestNG] Time taken by org.testng.reporters.EmailableReporter@1f4384c2: 8 ms

BUILD SUCCESSFUL
Total time: 13 seconds