Repository: ansible/ansible

Issue:
Failed task exposes warning (Removed unexpected internal key)
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
Ansible core

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
v2.3

##### SUMMARY
<!--- Explain the problem briefly -->
Since some time, a task failure (caused by an exception) always exposes this (new) warning:
```
 [WARNING]: Removed unexpected internal key in module return: _ansible_parsed = False
```
To me this offers absolutely no additional information to the failure and only adds to the confusion. (As it's the first thing the user sees before seeing the error, and the pink color stands out more than dark red.

I traced this warning to:
https://github.com/ansible/ansible/blame/dd8d699981249038b6d9d8dda05ea57239a7d2d4/lib/ansible/plugins/action/__init__.py#L767

I would prefer to get rid of this warning altogether.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
Make it fail, you will see :-)


Discussion:
cc: @bcoca @jctanner 

Resolving patch:
diff --git a/lib/ansible/plugins/action/__init__.py b/lib/ansible/plugins/action/__init__.py
--- a/lib/ansible/plugins/action/__init__.py
+++ b/lib/ansible/plugins/action/__init__.py
@@ -742,7 +742,6 @@ def _execute_module(self, module_name=None, module_args=None, tmp=None, task_var
 
         # remove internal keys
         self._remove_internal_keys(data)
-        data['_ansible_parsed'] = True
 
         # cleanup tmp?
         if (self._play_context.become and self._play_context.become_user != 'root') and not persist_files and delete_remote_tmp or tmpdir_delete:
@@ -767,7 +766,7 @@ def _execute_module(self, module_name=None, module_args=None, tmp=None, task_var
 
     def _remove_internal_keys(self, data):
         for key in list(data.keys()):
-            if key.startswith('_ansible_') or key in C.INTERNAL_RESULT_KEYS:
+            if key.startswith('_ansible_') and key != '_ansible_parsed' or key in C.INTERNAL_RESULT_KEYS:
                 display.warning("Removed unexpected internal key in module return: %s = %s" % (key, data[key]))
                 del data[key]
 
@@ -824,6 +823,7 @@ def _parse_returned_data(self, res):
             if 'ansible_facts' in data and isinstance(data['ansible_facts'], dict):
                 self._clean_returned_data(data['ansible_facts'])
                 data['ansible_facts'] = wrap_var(data['ansible_facts'])
+            data['_ansible_parsed'] = True
         except ValueError:
             # not valid json, lets try to capture error
             data = dict(failed=True, _ansible_parsed=False)


Test patch:
diff --git a/test/units/plugins/action/test_action.py b/test/units/plugins/action/test_action.py
--- a/test/units/plugins/action/test_action.py
+++ b/test/units/plugins/action/test_action.py
@@ -591,6 +591,7 @@ def test_json_empty(self):
                          'stdout_lines': stdout.splitlines(),
                          'stderr': err}
         res = action_base._parse_returned_data(returned_data)
+        del res['_ansible_parsed'] # we always have _ansible_parsed
         self.assertEqual(len(res), 0)
         self.assertFalse(res)