Repository: ansible/ansible

Issue:
junit callback plugins output results in confusing xml output
##### ISSUE TYPE
 - Bug Report


##### COMPONENT NAME
junit

##### ANSIBLE VERSION
```
ansible 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
Enable the junit callback plugin:
```
callback_whitelist = junit
```

##### SUMMARY
The resulting xml from the callback plugin has a confusing class in each package which makes reading the output much harder than it should be.

##### STEPS TO REPRODUCE
Enable the junit plugin and run any playbook. View the junit results, or use a parser like the jenkins (or similar CI) junit plugins.

##### EXPECTED RESULTS
A package (or class) with all the tests when parsing the output such as:
```xml
<testcase classname="/srv/ansible/ansible_role_jenkins/tasks/add_signature_whitelist" name="[localhost] test-ci-jenkins: ansible_role_jenkins : check if signature whitelisted password={{ jenkins_admin_pass }}, user={{ jenkins_admin_user }}, args={u'signature': u'{{ item }}'}, script={{ jenkins_groovy_check_signature_approval }}" time="0.326584">
			<system-out>{
&quot;changed&quot;: false, 
&quot;output&quot;: &quot;Result: false\n&quot;
}</system-out>
		</testcase>
		<testcase classname="/srv/ansible/ansible_role_jenkins/tasks/add_signature_whitelist" name="[localhost] test-ci-jenkins: ansible_role_jenkins : add signature to whitelist password={{ jenkins_admin_pass }}, user={{ jenkins_admin_user }}, args={u'signature': u'{{ item }}'}, script={{ jenkins_groovy_add_signature_approval }}" time="0.305978">
			<system-out>{
&quot;changed&quot;: true, 
&quot;output&quot;: &quot;&quot;
}</system-out>
		</testcase>
```

##### ACTUAL RESULTS

The XML output results in a package of ""/srv/ansible/ansible_role_jenkins/tasks/add_signature_whitelist" and then two classes within it of yml:1 and yml:9 with the different tasks within that. This makes trying to look at results or durations of multiple plays very painful.

```xml
<testcase classname="/srv/ansible/ansible_role_jenkins/tasks/add_signature_whitelist.yml:1" name="[localhost] test-ci-jenkins: ansible_role_jenkins : check if signature whitelisted password={{ jenkins_admin_pass }}, user={{ jenkins_admin_user }}, args={u'signature': u'{{ item }}'}, script={{ jenkins_groovy_check_signature_approval }}" time="0.326584">
			<system-out>{
&quot;changed&quot;: false, 
&quot;output&quot;: &quot;Result: false\n&quot;
}</system-out>
		</testcase>
		<testcase classname="/srv/ansible/ansible_role_jenkins/tasks/add_signature_whitelist.yml:9" name="[localhost] test-ci-jenkins: ansible_role_jenkins : add signature to whitelist password={{ jenkins_admin_pass }}, user={{ jenkins_admin_user }}, args={u'signature': u'{{ item }}'}, script={{ jenkins_groovy_add_signature_approval }}" time="0.305978">
			<system-out>{
&quot;changed&quot;: true, 
&quot;output&quot;: &quot;&quot;
}</system-out>
		</testcase>

```

I have worked around this in my jenkins instances by doing a `sed -E 's/\.yml:[0-9]+//' results/*.xml`  before recording the result output and have confirmed that all the tests then appear under a package of "root" and the class matches the actual task being included, with tests per task within that so merely chomping the .yml:[0-9]+ from the classname generated makes a huge difference.




Discussion:

Resolving patch:
diff --git a/lib/ansible/plugins/callback/junit.py b/lib/ansible/plugins/callback/junit.py
--- a/lib/ansible/plugins/callback/junit.py
+++ b/lib/ansible/plugins/callback/junit.py
@@ -20,6 +20,7 @@
 
 import os
 import time
+import re
 
 from ansible.module_utils._text import to_bytes, to_text
 from ansible.plugins.callback import CallbackBase
@@ -55,6 +56,8 @@ class CallbackModule(CallbackBase):
     This plugin makes use of the following environment variables:
         JUNIT_OUTPUT_DIR (optional): Directory to write XML files to.
                                      Default: ~/.ansible.log
+        JUNIT_TASK_CLASS (optional): Configure the output to be one class per yaml file
+                                     Default: False
 
     Requires:
         junit_xml
@@ -70,6 +73,7 @@ def __init__(self):
         super(CallbackModule, self).__init__()
 
         self._output_dir = os.getenv('JUNIT_OUTPUT_DIR', os.path.expanduser('~/.ansible.log'))
+        self._task_class = os.getenv('JUNIT_TASK_CLASS', 'False').lower()
         self._playbook_path = None
         self._playbook_name = None
         self._play_name = None
@@ -136,8 +140,13 @@ def _build_test_case(self, task_data, host_data):
         name = '[%s] %s: %s' % (host_data.name, task_data.play, task_data.name)
         duration = host_data.finish - task_data.start
 
+        if self._task_class == 'true':
+            junit_classname = re.sub('\.yml:[0-9]+$', '', task_data.path)
+        else:
+            junit_classname = task_data.path
+
         if host_data.status == 'included':
-            return TestCase(name, task_data.path, duration, host_data.result)
+            return TestCase(name, junit_classname, duration, host_data.result)
 
         res = host_data.result._result
         rc = res.get('rc', 0)
@@ -145,9 +154,9 @@ def _build_test_case(self, task_data, host_data):
         dump = self._cleanse_string(dump)
 
         if host_data.status == 'ok':
-            return TestCase(name, task_data.path, duration, dump)
+            return TestCase(name, junit_classname, duration, dump)
 
-        test_case = TestCase(name, task_data.path, duration)
+        test_case = TestCase(name, junit_classname, duration)
 
         if host_data.status == 'failed':
             if 'exception' in res:


Test patch: