Repository: ansible/ansible

Issue:
Double extended error message in AnsibleParserError (triple in 2.4)
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
-->

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
AnsibleError, AnsibleParserError, Task#preprocess_data, ModuleArgsParser#parse

##### ANSIBLE VERSION
```
2.4, 2.5, devel
```

##### CONFIGURATION
No special configuration, local connection.

ansible-playbook -c local -i localhost, playbook.yml

playbook.yml:
```
---
- hosts: all
  tasks:
  - apk: msgfd
```


##### OS / ENVIRONMENT
Ubuntu 16.04, Python 3.5.2 (default, Nov 23 2017, 16:37:01) in virtualenv

##### SUMMARY

`ModuleArgsParser#parse` raises `AnsibleParserError` if something is wrong with tasks or their arguments. 
`AnsibleParserError#__init__` append extended error to the message.
Then `Task#preprocess_data` wraps `AnsibleParserError` with new instance of `AnsibleParserError` and thus extended error is doubled in message.

Extended error is printed 3 times in version 2.4. `devel` branch has merged PR https://github.com/ansible/ansible/pull/24468 and extended error is printed 2 times.

May be something wrong with this wrapping?
https://github.com/ansible/ansible/blob/ebf971f931290a113f738f658d7a6e095994e120/lib/ansible/playbook/task.py#L183-L187

##### STEPS TO REPRODUCE

Simple test case that raises error about extra params from https://github.com/ansible/ansible/blob/ebf971f931290a113f738f658d7a6e095994e120/lib/ansible/parsing/mod_args.py#L303

```yaml
---
- hosts: all
  tasks:
  - apk: msgfd
```

Create file and run it localy for verification:
```
$ ansible-playbook -c local -i localhost, playbook.yml
```

##### EXPECTED RESULTS

Clean error message with offending lines:

```
ERROR! this task 'apk' has extra params, which is only allowed in the following modules: command, win_command, shell, win_shell, script, include, include_vars, include_tasks, include_role, import_tasks, import_role, add_host, group_by, set_fact, raw, meta

The error appears to have been in '/home/diafour/Projects/dapp/ansible-test/debug-playbook.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - apk: msgfd
      ^ here
```

##### ACTUAL RESULTS

###### devel branch:

Double output

```
ERROR! this task 'apk' has extra params, which is only allowed in the following modules: command, win_command, shell, win_shell, script, include, include_vars, include_tasks, include_role, import_tasks, import_role, add_host, group_by, set_fact, raw, meta

The error appears to have been in '/home/diafour/Projects/dapp/ansible-test/debug-playbook.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - apk: msgfd
      ^ here


The error appears to have been in '/home/diafour/Projects/dapp/ansible-test/debug-playbook.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - apk: msgfd
      ^ here

```

###### version 2.4 (git checkout stable-2.4)

Triple output

```
ERROR! this task 'apk' has extra params, which is only allowed in the following modules: command, win_command, shell, win_shell, script, include, include_vars, include_tasks, include_role, import_tasks, import_role, add_host, group_by, set_fact, raw, meta

The error appears to have been in '/home/diafour/Projects/dapp/ansible-test/debug-playbook.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - apk: msgfd
      ^ here


The error appears to have been in '/home/diafour/Projects/dapp/ansible-test/debug-playbook.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - apk: msgfd
      ^ here

exception type: <class 'ansible.errors.AnsibleParserError'>
exception: this task 'apk' has extra params, which is only allowed in the following modules: command, win_command, shell, win_shell, script, include, include_vars, include_tasks, include_role, import_tasks, import_role, add_host, group_by, set_fact, raw, meta

The error appears to have been in '/home/diafour/Projects/dapp/ansible-test/debug-playbook.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - apk: msgfd
      ^ here

```



Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
!component +lib/ansible/errors/__init__.py
!component +lib/ansible/parsing/mod_args.py
!component +lib/ansible/playbook/task.py
Files identified in the description:
* [lib/ansible/errors](https://github.com/ansible/ansible/blob/devel/lib/ansible/errors)
* [lib/ansible/parsing/mod_args](https://github.com/ansible/ansible/blob/devel/lib/ansible/parsing/mod_args)
* [lib/ansible/playbook/task](https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/task)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Files identified in the description:
* [lib/ansible/errors/__init__.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/errors/__init__.py)
* [lib/ansible/parsing/mod_args.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/parsing/mod_args.py)
* [lib/ansible/playbook/task.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/task.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
The mod_args.py and task.py cases need some tweaking, since currently they catch AnsibleParserError and then raises a new AnsibleParserError() with additional args, so the final displayed exception's orig_exc has two or more instances of the same exception type with the same message.  And when displayed as shown, that doesn't make a ton of sense.

For cases like:

https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/task.py#L187

... guess there is no particular reason to re raise a new AnsibleParserError in those cases.
 As far as I can tell,  we could either just do a plain (re) `raise`, or if we need to set/update the AnsibleParserError._obj we can do it the original Exception instance and (re) `raise`

``` python

try:
    whatever()
except AnsibleParserError as e:
    # if we don't need to add/change the AnsibleError._obj, just re raise
    raise

try:
    whatever(some_thing_related_to_some_yaml_object)
except AnsibleParserError as e:
    # if whatever() doesn't specify obj=some_yaml_obj when it raises AnsibleParserError()
    # we can just set/update it here and re raise
    e._obj = some_yaml_obj
    raise

```
   

Resolving patch:
diff --git a/lib/ansible/playbook/task.py b/lib/ansible/playbook/task.py
--- a/lib/ansible/playbook/task.py
+++ b/lib/ansible/playbook/task.py
@@ -184,6 +184,11 @@ def preprocess_data(self, ds):
         try:
             (action, args, delegate_to) = args_parser.parse()
         except AnsibleParserError as e:
+            # if the raises exception was created with obj=ds args, then it includes the detail
+            # so we dont need to add it so we can just re raise.
+            if e._obj:
+                raise
+            # But if it wasn't, we can add the yaml object now to get more detail
             raise AnsibleParserError(to_native(e), obj=ds, orig_exc=e)
 
         # the command/shell/script modules used to support the `cmd` arg,


Test patch:
diff --git a/test/units/playbook/test_task.py b/test/units/playbook/test_task.py
--- a/test/units/playbook/test_task.py
+++ b/test/units/playbook/test_task.py
@@ -20,7 +20,10 @@
 __metaclass__ = type
 
 from ansible.compat.tests import unittest
+from ansible.compat.tests.mock import patch
 from ansible.playbook.task import Task
+from ansible.parsing.yaml import objects
+from ansible import errors
 
 
 basic_command_task = dict(
@@ -32,6 +35,10 @@
     action='command echo hi'
 )
 
+# See #36848
+kv_bad_args_str = '- apk: sdfs sf sdf 37'
+kv_bad_args_ds = {'apk': 'sdfs sf sdf 37'}
+
 
 class TestTask(unittest.TestCase):
 
@@ -42,7 +49,7 @@ def tearDown(self):
         pass
 
     def test_construct_empty_task(self):
-        t = Task()
+        Task()
 
     def test_construct_task_with_role(self):
         pass
@@ -65,9 +72,27 @@ def test_load_task_kv_form(self):
         self.assertEqual(t.action, 'command')
         self.assertEqual(t.args, dict(_raw_params='echo hi'))
 
+    @patch.object(errors.AnsibleError, '_get_error_lines_from_file')
+    def test_load_task_kv_form_error_36848(self, mock_get_err_lines):
+        ds = objects.AnsibleMapping(kv_bad_args_ds)
+        ds.ansible_pos = ('test_task_faux_playbook.yml', 1, 1)
+        mock_get_err_lines.return_value = (kv_bad_args_str, '')
+
+        with self.assertRaises(errors.AnsibleParserError) as cm:
+            Task.load(ds)
+
+        self.assertIsInstance(cm.exception, errors.AnsibleParserError)
+        self.assertEqual(cm.exception._obj, ds)
+        self.assertEqual(cm.exception._obj, kv_bad_args_ds)
+        self.assertIn("The error appears to have been in 'test_task_faux_playbook.yml", cm.exception.message)
+        self.assertIn(kv_bad_args_str, cm.exception.message)
+        self.assertIn('apk', cm.exception.message)
+        self.assertEquals(cm.exception.message.count('The offending line'), 1)
+        self.assertEquals(cm.exception.message.count('The error appears to have been in'), 1)
+
     def test_task_auto_name(self):
         assert 'name' not in kv_command_task
-        t = Task.load(kv_command_task)
+        Task.load(kv_command_task)
         # self.assertEqual(t.name, 'shell echo hi')
 
     def test_task_auto_name_with_role(self):