Repository: ansible/ansible

Issue:
Invalid YAML group_vars file not showing errors on failed import
##### ISSUE TYPE
 - Bug Report


##### COMPONENT NAME
hostvars

##### ANSIBLE VERSION
```
ansible 2.3.2.0
  config file = /Users/Yves/Infrastructure/ansible-provisioning/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.10 (default, Feb  7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
N/A

##### SUMMARY
There are a few hosts I'm able to connect to, to execute tasks successfully - but even in the same play the hostvars for this host are empty.

##### STEPS TO REPRODUCE

```yaml
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************************************
ok: [host1.example.com]

TASK [return motd to registered var] ******************************************************************************************************************************************************************************************************************************************
changed: [host1.example.com]

TASK [debug test - motd] ******************************************************************************************************************************************************************************************************************************************************
ok: [host1.example.com] => {
    "motd": {
        "changed": true, 
        "cmd": [
            "cat", 
            "/etc/motd"
        ], 
        "delta": "0:00:00.002175", 
        "end": "2017-08-10 23:55:52.192378", 
        "rc": 0, 
        "start": "2017-08-10 23:55:52.190203", 
        "stderr": "", 
        "stderr_lines": [], 
        "stdout": "", 
        "stdout_lines": []
    }
}

TASK [debug test - hostvars] **************************************************************************************************************************************************************************************************************************************************
ok: [host1.example.com] => {
    "hostvars['host1.example.com']": "VARIABLE IS NOT DEFINED!"
}
```



##### EXPECTED RESULTS

```yaml
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************************************
ok: [workinghost.example.com]

TASK [return motd to registered var] ******************************************************************************************************************************************************************************************************************************************
changed: [workinghost.example.com]

TASK [debug test - motd] ******************************************************************************************************************************************************************************************************************************************************
ok: [workinghost.example.com] => {
    "motd": {
        "changed": true, 
        "cmd": [
            "cat", 
            "/etc/motd"
        ], 
        "delta": "0:00:00.003442", 
        "end": "2017-08-10 23:59:45.907492", 
        "rc": 0, 
        "start": "2017-08-10 23:59:45.904050", 
        "stderr": "", 
        "stderr_lines": [], 
        "stdout": "", 
        "stdout_lines": []
    }
}

TASK [debug test - hostvars] **************************************************************************************************************************************************************************************************************************************************
ok: [workinghost.example.com] => {
    "hostvars['workinghost.example.com']": {
        "ansible_all_ipv4_addresses": [
            "10.255.0.36"
        ], 
        "ansible_all_ipv6_addresses": [
            "fe80::a25:6ccb:ef7:eacd"
        ], 
       (…)
    }
}
```

##### ACTUAL RESULTS
```yaml
ok: [host1.example.com] => {
    "hostvars['host1.example.com']": "VARIABLE IS NOT DEFINED!"
}
```
Even as host facts have been gathered in the same run.


I've discovered that this is only true for versions > 2.3.0.0.

With version 2.3.0.0 installed the result above is:

```yaml
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************************************
ok: [host1.example.com]

TASK [return motd to registered var] ******************************************************************************************************************************************************************************************************************************************
changed: [host1.example.com]

TASK [debug test - motd] ******************************************************************************************************************************************************************************************************************************************************
ok: [host1.example.com] => {
    "changed": false, 
    "motd": {
        "changed": true, 
        "cmd": [
            "cat", 
            "/etc/motd"
        ], 
        "delta": "0:00:00.002076", 
        "end": "2017-08-11 00:05:43.564600", 
        "rc": 0, 
        "start": "2017-08-11 00:05:43.562524", 
        "stderr": "", 
        "stderr_lines": [], 
        "stdout": "", 
        "stdout_lines": []
    }
}

TASK [debug test - hostvars] **************************************************************************************************************************************************************************************************************************************************
ok: [host1.example.com] => {
    "changed": false, 
    "hostvars['host1.example.com']": "{u'ohai_chef_packages': {u'chef': {u'chef_root': u'/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib', u'version': u'12.5.1'}, u'ohai': {u'version': u'8.7.0', u'ohai_root': u'/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/ohai-8.7.0/lib/ohai'}}, u'module_setup': True, u'ohai_block_device': {u'ram14': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram15': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram12': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram13': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram10': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram11': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'md2': {u'rotational': u'1', u'removable': u'0', u'size': u'1459899616'}, u'md0': {u'rotational': u'1', u'removable': u'0', u'size': u'4194280'}, u'md1': {u'rotational': u'1', u'removable': u'0', u'size': u'1048552'}, u'ram8': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram9': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram4': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram5': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram6': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram7': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram0': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram1': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram2': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'ram3': {u'rotational': u'1', u'removable': u'0', u'size': u'32768'}, u'sda': {u'rotational': u'1', u'vendor': u'ATA', u'removable': u'0', u'rev': u'1H15', u'state': u'running', u'timeout': u'30', u'model': u'WDC WD7500AALX-0', u'size': u'1465149168'}
    (…)
}


```



Discussion:
Alright… when I'm using more than just test data I can reproduce this error even with 2.3.0.0.
But it seems that this happens when an inline vault string is used.
When using 2.3.0.0,  the error is "Odd-length string". For version > 2.3.0.0 it seems that the hostvars are fully discarded.

Here are my hostvars:

```yaml

---
  
rsyncd:
  shares:
    - name: root
      path: /
      auth_users: backup
      use_chroot: no
      read_only: no
      hosts_allow: xx.x.x.x
      
  users:
    - name: backup    
      password: !vault |
        $ANSIBLE_VAULT;1.1;AES256
        65313732383033353939373464343335656666643335386266353663373962303165316139613866
        6237313234343230643738396361386634393834316339380a333363636266663039303432666535
        37343864333466373335393763616230376636373538643436383934373232386338663134363935
        6633323466656437380a356163313562643266383863373339663239653532353630643061323730
        3032 

```

When not using an inline vault and a plain string instead everything works as expected.

OK… this happens when the inline vault is not indented correctly.

```yaml
rsyncd:
  shares:
    - name: root
      path: /
      auth_users: backup
      use_chroot: no
      read_only: no
      hosts_allow: x.x.x.x
      
  users:
    - name: backup    
      password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          63626664366661373462633938376532663233353732376333386532663731353738616464643164
          6164383065343731336566346238303661633564656331370a623563623261346234326164373666
          36303435343434363234366432613936383432633138303438633962643331336230623139323739
          3938313230346535310a343239386165356233613564333432636661393637353933653265343165
          3034

```


Conclusion: Using invalid inline vault leads to unavailability of hostvars.
Not really, other hostvars should exist, but making the YAML file invalid is probably preventing any of the other vars in same file from being defined. 

Since you don't supply a reproducing playbook (just the output), i cannot conform nor deny that this affects anything else.

The current bug would be that you get no warning on invalid YAML file (but i do get one in my tests). In either case I need more info to reproduce the error.


Please don't try to make the YAML invalid. Try to make just the inline vault string itself invalid.
im not making it invalid, i suspect that is what is currently happening, so the other vars are not getting declared. I'm planning on issuing an error when this happens.
Hi Brian, how can I help you? Should I create some files which reproduce this behaviour?
yes, that is what i asked for above, since i dont seem to be able to, tried to reconstruct same with info given but I either get everything working or an error on badly formatted yaml file.
Alright, I created a repo.

https://github.com/yves-vogl/ansible-inline-vault-issue

Here are the steps to reproduce:

Run 

```$ ansible-playbook -i inventory --vault-password-file=vault-secret tasks.yml ```


Output will be:

```yaml

PLAY [all] ***************************************************************************************************************************************************************

TASK [Display hostvars] **************************************************************************************************************************************************
fatal: [broken.example.com]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'ansible.vars.hostvars.HostVars object' has no attribute u'broken.example.com'\n\nThe error appears to have been in '/Users/Yves/Desktop/#28038/tasks.yml': line 7, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n    - name: Display hostvars\n      ^ here\n"}
ok: [working.example.com] => {
    "msg": "working.example.com hostvars: {'inventory_hostname': u'working.example.com', 'inventory_file': u'/Users/Yves/Desktop/#28038/inventory', 'ansible_playbook_python': '/usr/bin/python', 'inventory_hostname_short': u'working', 'playbook_dir': u'/Users/Yves/Desktop/#28038', 'omit': '__omit_place_holder__a1db8ff276dbc6f01e212386f63938d9f50adc3e', 'inventory_dir': u'/Users/Yves/Desktop/#28038', 'group_names': [u'test-servers'], 'groups': {'ungrouped': [], 'all': [u'broken.example.com', u'working.example.com'], u'test-servers': [u'broken.example.com', u'working.example.com']}, 'ansible_check_mode': False, u'foo': {u'bar': u'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.'}, 'ansible_version': {'major': 2, 'full': '2.3.1.0', 'string': '2.3.1.0', 'minor': 3, 'revision': 1}}"
}
	to retry, use: --limit @/Users/Yves/Desktop/#28038/tasks.retry

PLAY RECAP ***************************************************************************************************************************************************************
broken.example.com         : ok=0    changed=0    unreachable=0    failed=1   
working.example.com        : ok=1    changed=0    unreachable=0    failed=0   
```

The happens because of the content in `host_vars/broken.example.com/vars`:

```yaml
---
  
rsyncd:
  shares:
    - name: root
      path: /
      auth_users: backup
      use_chroot: no
      read_only: no
      hosts_allow: 1.2.3.4
      
  users:
    - name: backup    
      password: !vault |
         $ANSIBLE_VAULT;1.1;AES256
         64323332393930623633306662363165386332376638653035356132646165663632616263653366
         6233383362313531623238613461323861376137656265380a366464663835633065616361636231
         ifuckedupthissecret
         3366323866663763660a323766383531396433663861656532373663373134376263383263316261
         3137
```

Here you can see that I obviously messed around with the encrypted string.


When you fix the secret (see comment in tasks.yml) you get the complete output:

```yaml

$ ansible-playbook -i inventory --vault-password-file=vault-secret tasks.yml 

PLAY [all] ***************************************************************************************************************************************************************

TASK [Display hostvars] **************************************************************************************************************************************************
ok: [working.example.com] => {
    "msg": "working.example.com hostvars: {'inventory_hostname': u'working.example.com', 'inventory_file': u'/Users/Yves/Desktop/#28038/inventory', 'ansible_playbook_python': '/usr/bin/python', 'inventory_hostname_short': u'working', 'playbook_dir': u'/Users/Yves/Desktop/#28038', 'omit': '__omit_place_holder__f5ca75f8ba369d9173101da152d5bcc2e68f4f76', 'inventory_dir': u'/Users/Yves/Desktop/#28038', 'group_names': [u'test-servers'], 'groups': {'ungrouped': [], 'all': [u'broken.example.com', u'working.example.com'], u'test-servers': [u'broken.example.com', u'working.example.com']}, 'ansible_check_mode': False, u'foo': {u'bar': u'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.'}, 'ansible_version': {'major': 2, 'full': '2.3.1.0', 'string': '2.3.1.0', 'minor': 3, 'revision': 1}}"
}
ok: [broken.example.com] => {
    "msg": "broken.example.com hostvars: {'inventory_hostname': u'broken.example.com', 'inventory_file': u'/Users/Yves/Desktop/#28038/inventory', 'ansible_playbook_python': '/usr/bin/python', 'inventory_hostname_short': u'broken', 'playbook_dir': u'/Users/Yves/Desktop/#28038', 'group_names': [u'test-servers'], 'inventory_dir': u'/Users/Yves/Desktop/#28038', u'rsyncd': {u'users': [{u'password': u'test', u'name': u'backup'}], u'shares': [{u'read_only': False, u'use_chroot': False, u'name': u'root', u'auth_users': u'backup', u'path': u'/', u'hosts_allow': u'1.2.3.4'}]}, 'groups': {'ungrouped': [], 'all': [u'broken.example.com', u'working.example.com'], u'test-servers': [u'broken.example.com', u'working.example.com']}, 'ansible_check_mode': False, 'omit': '__omit_place_holder__f5ca75f8ba369d9173101da152d5bcc2e68f4f76', 'ansible_version': {'major': 2, 'full': '2.3.1.0', 'string': '2.3.1.0', 'minor': 3, 'revision': 1}}"
}

PLAY RECAP ***************************************************************************************************************************************************************
broken.example.com         : ok=1    changed=0    unreachable=0    failed=0   
working.example.com        : ok=1    changed=0    unreachable=0    failed=0   

```

To summarize: I'd have expected to just the attribute with the broken secret instead of skipping the whole hostvars. Also I'd have expected a warning/error message which tells me that the encrypted string could not be decrypted.

But "decryption failed" is only logged if the decryption key is wrong - not if the encrypted string itself could not be decrypted.


I hope that helps.








Hi @bcoca,

can I help you any further to figure out this?
I can reproduce this with devel/

```
[newswoop:F25:ansible-inline-vault-issue (master u=)]$ ansible-playbook -i inventory --vault-password-file=vault-secret tasks.yml

PLAY [all] *********************************************************************************************************************************************************************************************************

TASK [Display hostvars] ********************************************************************************************************************************************************************************************
fatal: [broken.example.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.vars.hostvars.HostVars object' has no attribute u'broken.example.com'\n\nThe error appears to have been in '/home/adrian/src/ansible-inline-vault-issue/tasks.yml': line 7, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n    - name: Display hostvars\n      ^ here\n"}
ok: [working.example.com] => {
    "msg": "working.example.com hostvars: {'inventory_hostname': u'working.example.com', 'inventory_file': u'/home/adrian/src/ansible-inline-vault-issue/inventory', 'ansible_playbook_python': '/usr/bin/python', u'foo': {u'bar': u'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.'}, 'inventory_hostname_short': u'working', 'ansible_forks': 5, 'playbook_dir': u'/home/adrian/src/ansible-inline-vault-issue', 'omit': '__omit_place_holder__b373396dd3c0b54904351af8be04d0ba72883ef5', 'ansible_diff_mode': False, 'inventory_dir': u'/home/adrian/src/ansible-inline-vault-issue', 'group_names': [u'test-servers'], 'ansible_inventory_sources': [u'/home/adrian/src/ansible-inline-vault-issue/inventory'], 'groups': {'ungrouped': [], 'all': [u'broken.example.com', u'working.example.com'], u'test-servers': [u'broken.example.com', u'working.example.com']}, 'ansible_check_mode': False, 'ansible_run_tags': [u'all'], 'ansible_skip_tags': [], 'ansible_version': {'major': 2, 'full': '2.5.0', 'string': '2.5.0', 'minor': 5, 'revision': 0}}"
}
	to retry, use: --limit @/home/adrian/src/ansible-inline-vault-issue/tasks.retry

PLAY RECAP *********************************************************************************************************************************************************************************************************
broken.example.com         : ok=0    changed=0    unreachable=0    failed=1   
working.example.com        : ok=1    changed=0    unreachable=0    failed=0   

[newswoop:F25:ansible-inline-vault-issue (master u=)]$ ansible --version
ansible 2.5.0 (devel 83674af284) last updated 2017/11/01 13:11:12 (GMT -400)
  config file = None
  configured module search path = [u'/home/adrian/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/adrian/src/ansible/lib/ansible
  executable location = /home/adrian/src/ansible/bin/ansible
  python version = 2.7.13 (default, May 10 2017, 20:04:28) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)]
```
after replacing some wrapped exceptions with direct raises and using py3, I get to the following nested exceptions:

```
Traceback (most recent call last):
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 96, in load
    new_data = json.loads(data)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 108, in load
    new_data = self._safe_load(in_data, file_name=file_name)
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 170, in _safe_load
    return loader.get_single_data()
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/constructor.py", line 35, in get_single_data
    node = self.get_single_node()
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/composer.py", line 39, in get_single_node
    if not self.check_event(StreamEndEvent):
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/parser.py", line 174, in parse_document_start
    self.peek_token().start_mark)
yaml.parser.ParserError: expected '<document start>', but found '<scalar>'
  in "<unicode string>", line 2, column 1:
    broken.example.com
    ^

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adrian/src/ansible/lib/ansible/plugins/inventory/yaml.py", line 86, in parse
    data = self.loader.load_from_file(path)
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 133, in load_from_file
    parsed_data = self.load(data=file_data, file_name=file_name, show_content=show_content)
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 110, in load
    self._handle_error(yaml_exc, file_name, show_content)
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 230, in _handle_error
    raise AnsibleParserError(YAML_SYNTAX_ERROR % to_native(err_msg), obj=err_obj, show_content=show_content, orig_exc=yaml_exc)
ansible.errors.AnsibleParserError: Syntax Error while loading YAML.
  expected '<document start>', but found '<scalar>'

The error appears to have been in '/home/adrian/src/ansible-inline-vault-issue/inventory': line 2, column 1, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

[test-servers]
broken.example.com
^ here


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adrian/src/ansible/bin/ansible-playbook", line 109, in <module>
    exit_code = cli.run()
  File "/home/adrian/src/ansible/lib/ansible/cli/playbook.py", line 104, in run
    loader, inventory, variable_manager = self._play_prereqs(self.options)
  File "/home/adrian/src/ansible/lib/ansible/cli/__init__.py", line 798, in _play_prereqs
    inventory = InventoryManager(loader=loader, sources=options.inventory)
  File "/home/adrian/src/ansible/lib/ansible/inventory/manager.py", line 145, in __init__
    self.parse_sources()
  File "/home/adrian/src/ansible/lib/ansible/inventory/manager.py", line 206, in parse_sources
    parse = self.parse_source(source, cache=cache)
  File "/home/adrian/src/ansible/lib/ansible/inventory/manager.py", line 261, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/adrian/src/ansible/lib/ansible/plugins/inventory/yaml.py", line 89, in parse
    raise AnsibleParserError(e)
ansible.errors.AnsibleParserError: Syntax Error while loading YAML.
  expected '<document start>', but found '<scalar>'

The error appears to have been in '/home/adrian/src/ansible-inline-vault-issue/inventory': line 2, column 1, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

[test-servers]
broken.example.com
^ here
```
After unwrapping another exception or two, an initial yaml error comes from:

```
Traceback (most recent call last):
  File "/home/adrian/src/ansible/lib/ansible/plugins/inventory/yaml.py", line 86, in parse
    data = self.loader.load_from_file(path)
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 134, in load_from_file
    parsed_data = self.load(data=file_data, file_name=file_name, show_content=show_content)
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 108, in load
    new_data = self._safe_load(in_data, file_name=file_name)
  File "/home/adrian/src/ansible/lib/ansible/parsing/dataloader.py", line 171, in _safe_load
    return loader.get_single_data()
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/constructor.py", line 35, in get_single_data
    node = self.get_single_node()
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/composer.py", line 39, in get_single_node
    if not self.check_event(StreamEndEvent):
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/home/adrian/.local/lib/python3.6/site-packages/yaml/parser.py", line 174, in parse_document_start
    self.peek_token().start_mark)
yaml.parser.ParserError: expected '<document start>', but found '<scalar>'
  in "<unicode string>", line 2, column 1:
    broken.example.com
    ^
```

The tracebacks more or less point to host_vars/broken.example.com/vars being invalid yaml. 

That is true in this case (the malformed !vault scalar), not sure how to work around that. 

But that is after modifying a couple of try/except blocks, so I need to verify it's the same issue as unmodified code. There seems to be at least something else going to cause the HostVars attribute error, so need to track that down as well.

If the root cause is the invalid yaml, I'm not sure how to work around that. One approach would be to catch the parse error, then modify the yaml string to remove the malformed !vault and try it again. Doesn't seem worth the complexity to handle that case to me and likely would obscure a lot of busted yaml.

The more likely improvement would be better errors on yaml parse failure. The original error message from the yaml exception gets lots as it's wrapper/transformed into an AnsibleParserError, so it may be possible to preserve that detail

Resolving patch:
diff --git a/lib/ansible/parsing/vault/__init__.py b/lib/ansible/parsing/vault/__init__.py
--- a/lib/ansible/parsing/vault/__init__.py
+++ b/lib/ansible/parsing/vault/__init__.py
@@ -29,6 +29,7 @@
 import warnings
 from binascii import hexlify
 from binascii import unhexlify
+from binascii import Error as BinasciiError
 from hashlib import md5
 from hashlib import sha256
 from io import BytesIO
@@ -105,6 +106,10 @@ class AnsibleVaultPasswordError(AnsibleVaultError):
     pass
 
 
+class AnsibleVaultFormatError(AnsibleError):
+    pass
+
+
 def is_encrypted(data):
     """ Test if this is vault encrypted data blob
 
@@ -148,20 +153,7 @@ def is_encrypted_file(file_obj, start_pos=0, count=-1):
         file_obj.seek(current_position)
 
 
-def parse_vaulttext_envelope(b_vaulttext_envelope, default_vault_id=None):
-    """Retrieve information about the Vault and clean the data
-
-    When data is saved, it has a header prepended and is formatted into 80
-    character lines.  This method extracts the information from the header
-    and then removes the header and the inserted newlines.  The string returned
-    is suitable for processing by the Cipher classes.
-
-    :arg b_vaulttext: byte str containing the data from a save file
-    :returns: a byte str suitable for passing to a Cipher class's
-        decrypt() function.
-    """
-    # used by decrypt
-    default_vault_id = default_vault_id or C.DEFAULT_VAULT_IDENTITY
+def _parse_vaulttext_envelope(b_vaulttext_envelope, default_vault_id=None):
 
     b_tmpdata = b_vaulttext_envelope.splitlines()
     b_tmpheader = b_tmpdata[0].strip().split(b';')
@@ -169,7 +161,6 @@ def parse_vaulttext_envelope(b_vaulttext_envelope, default_vault_id=None):
     b_version = b_tmpheader[1].strip()
     cipher_name = to_text(b_tmpheader[2].strip())
     vault_id = default_vault_id
-    # vault_id = None
 
     # Only attempt to find vault_id if the vault file is version 1.2 or newer
     # if self.b_version == b'1.2':
@@ -181,6 +172,37 @@ def parse_vaulttext_envelope(b_vaulttext_envelope, default_vault_id=None):
     return b_ciphertext, b_version, cipher_name, vault_id
 
 
+def parse_vaulttext_envelope(b_vaulttext_envelope, default_vault_id=None, filename=None):
+    """Parse the vaulttext envelope
+
+    When data is saved, it has a header prepended and is formatted into 80
+    character lines.  This method extracts the information from the header
+    and then removes the header and the inserted newlines.  The string returned
+    is suitable for processing by the Cipher classes.
+
+    :arg b_vaulttext: byte str containing the data from a save file
+    :kwarg default_vault_id: The vault_id name to use if the vaulttext does not provide one.
+    :kwarg filename: The filename that the data came from.  This is only
+        used to make better error messages in case the data cannot be
+        decrypted. This is optional.
+    :returns: A tuple of byte str of the vaulttext suitable to pass to parse_vaultext,
+        a byte str of the vault format version,
+        the name of the cipher used, and the vault_id.
+    :raises: AnsibleVaultFormatError: if the vaulttext_envelope format is invalid
+    """
+    # used by decrypt
+    default_vault_id = default_vault_id or C.DEFAULT_VAULT_IDENTITY
+
+    try:
+        return _parse_vaulttext_envelope(b_vaulttext_envelope, default_vault_id)
+    except Exception as exc:
+        msg = "Vault envelope format error"
+        if filename:
+            msg += ' in %s' % (filename)
+        msg += ': %s' % exc
+        raise AnsibleVaultFormatError(msg)
+
+
 def format_vaulttext_envelope(b_ciphertext, cipher_name, version=None, vault_id=None):
     """ Add header and format to 80 columns
 
@@ -222,6 +244,41 @@ def format_vaulttext_envelope(b_ciphertext, cipher_name, version=None, vault_id=
     return b_vaulttext
 
 
+def _unhexlify(b_data):
+    try:
+        return unhexlify(b_data)
+    except (BinasciiError, TypeError) as exc:
+        raise AnsibleVaultFormatError('Vault format unhexlify error: %s' % exc)
+
+
+def _parse_vaulttext(b_vaulttext):
+    b_vaulttext = _unhexlify(b_vaulttext)
+    b_salt, b_crypted_hmac, b_ciphertext = b_vaulttext.split(b"\n", 2)
+    b_salt = _unhexlify(b_salt)
+    b_ciphertext = _unhexlify(b_ciphertext)
+
+    return b_ciphertext, b_salt, b_crypted_hmac
+
+
+def parse_vaulttext(b_vaulttext):
+    """Parse the vaulttext
+
+    :arg b_vaulttext: byte str containing the vaulttext (ciphertext, salt, crypted_hmac)
+    :returns: A tuple of byte str of the ciphertext suitable for passing to a
+        Cipher class's decrypt() function, a byte str of the salt,
+        and a byte str of the crypted_hmac
+    :raises: AnsibleVaultFormatError: if the vaulttext format is invalid
+    """
+    # SPLIT SALT, DIGEST, AND DATA
+    try:
+        return _parse_vaulttext(b_vaulttext)
+    except AnsibleVaultFormatError:
+        raise
+    except Exception as exc:
+        msg = "Vault vaulttext format error: %s" % exc
+        raise AnsibleVaultFormatError(msg)
+
+
 def verify_secret_is_not_empty(secret, msg=None):
     '''Check the secret against minimal requirements.
 
@@ -609,7 +666,8 @@ def decrypt_and_get_vault_id(self, vaulttext, filename=None):
                 msg += "%s is not a vault encrypted file" % filename
             raise AnsibleError(msg)
 
-        b_vaulttext, dummy, cipher_name, vault_id = parse_vaulttext_envelope(b_vaulttext)
+        b_vaulttext, dummy, cipher_name, vault_id = parse_vaulttext_envelope(b_vaulttext,
+                                                                             filename=filename)
 
         # create the cipher object, note that the cipher used for decrypt can
         # be different than the cipher used for encrypt
@@ -665,6 +723,13 @@ def decrypt_and_get_vault_id(self, vaulttext, filename=None):
                     vault_id_used = vault_secret_id
                     display.vvvvv('decrypt succesful with secret=%s and vault_id=%s' % (vault_secret, vault_secret_id))
                     break
+            except AnsibleVaultFormatError as exc:
+                msg = "There was a vault format error"
+                if filename:
+                    msg += ' in %s' % (filename)
+                msg += ': %s' % exc
+                display.warning(msg)
+                raise
             except AnsibleError as e:
                 display.vvvv('Tried to use the vault secret (%s) to decrypt (%s) but it failed. Error: %s' %
                              (vault_secret_id, filename, e))
@@ -862,7 +927,8 @@ def edit_file(self, filename):
 
         # Figure out the vault id from the file, to select the right secret to re-encrypt it
         # (duplicates parts of decrypt, but alas...)
-        dummy, dummy, cipher_name, vault_id = parse_vaulttext_envelope(b_vaulttext)
+        dummy, dummy, cipher_name, vault_id = parse_vaulttext_envelope(b_vaulttext,
+                                                                       filename=filename)
 
         # vault id here may not be the vault id actually used for decrypting
         # as when the edited file has no vault-id but is decrypted by non-default id in secrets
@@ -1136,7 +1202,7 @@ def decrypt(cls, b_vaulttext, secret, key_length=32):
                            'switch to the newer VaultAES256 format', version='2.3')
         # http://stackoverflow.com/a/14989032
 
-        b_vaultdata = unhexlify(b_vaulttext)
+        b_vaultdata = _unhexlify(b_vaulttext)
         b_salt = b_vaultdata[len(b'Salted__'):16]
         b_ciphertext = b_vaultdata[16:]
 
@@ -1289,7 +1355,7 @@ def _decrypt_cryptography(cls, b_ciphertext, b_crypted_hmac, b_key1, b_key2, b_i
         hmac = HMAC(b_key2, hashes.SHA256(), CRYPTOGRAPHY_BACKEND)
         hmac.update(b_ciphertext)
         try:
-            hmac.verify(unhexlify(b_crypted_hmac))
+            hmac.verify(_unhexlify(b_crypted_hmac))
         except InvalidSignature as e:
             raise AnsibleVaultError('HMAC verification failed: %s' % e)
 
@@ -1351,11 +1417,8 @@ def _decrypt_pycrypto(cls, b_ciphertext, b_crypted_hmac, b_key1, b_key2, b_iv):
 
     @classmethod
     def decrypt(cls, b_vaulttext, secret):
-        # SPLIT SALT, DIGEST, AND DATA
-        b_vaulttext = unhexlify(b_vaulttext)
-        b_salt, b_crypted_hmac, b_ciphertext = b_vaulttext.split(b"\n", 2)
-        b_salt = unhexlify(b_salt)
-        b_ciphertext = unhexlify(b_ciphertext)
+
+        b_ciphertext, b_salt, b_crypted_hmac = parse_vaulttext(b_vaulttext)
 
         # TODO: would be nice if a VaultSecret could be passed directly to _decrypt_*
         #       (move _gen_key_initctr() to a AES256 VaultSecret or VaultContext impl?)


Test patch:
diff --git a/test/integration/targets/vault/invalid_format/README.md b/test/integration/targets/vault/invalid_format/README.md
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/README.md
@@ -0,0 +1 @@
+Based on https://github.com/yves-vogl/ansible-inline-vault-issue
diff --git a/test/integration/targets/vault/invalid_format/broken-group-vars-tasks.yml b/test/integration/targets/vault/invalid_format/broken-group-vars-tasks.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/broken-group-vars-tasks.yml
@@ -0,0 +1,23 @@
+---
+- hosts: broken-group-vars
+  gather_facts: false
+  tasks:
+    - name: EXPECTED FAILURE
+      debug:
+        msg: "some_var_that_fails: {{ some_var_that_fails }}"
+
+    - name: EXPECTED FAILURE Display hostvars
+      debug:
+        msg: "{{inventory_hostname}} hostvars: {{ hostvars[inventory_hostname] }}"
+
+
+# ansible-vault --vault-password-file=vault-secret encrypt_string test
+# !vault |
+#           $ANSIBLE_VAULT;1.1;AES256
+#           64323332393930623633306662363165386332376638653035356132646165663632616263653366
+#           6233383362313531623238613461323861376137656265380a366464663835633065616361636231
+#           39653230653538366165623664326661653135306132313730393232343432333635326536373935
+#           3366323866663763660a323766383531396433663861656532373663373134376263383263316261
+#           3137
+
+# $ ansible-playbook -i inventory --vault-password-file=vault-secret tasks.yml 
diff --git a/test/integration/targets/vault/invalid_format/broken-host-vars-tasks.yml b/test/integration/targets/vault/invalid_format/broken-host-vars-tasks.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/broken-host-vars-tasks.yml
@@ -0,0 +1,7 @@
+---
+- hosts: broken-host-vars
+  gather_facts: false
+  tasks:
+    - name: EXPECTED FAILURE Display hostvars
+      debug:
+        msg: "{{inventory_hostname}} hostvars: {{ hostvars[inventory_hostname] }}"
diff --git a/test/integration/targets/vault/invalid_format/group_vars/broken-group-vars.yml b/test/integration/targets/vault/invalid_format/group_vars/broken-group-vars.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/group_vars/broken-group-vars.yml
@@ -0,0 +1,8 @@
+$ANSIBLE_VAULT;1.1;AES256
+64306566356165343030353932383461376334336665626135343932356431383134306338353664
+6435326361306561633165633536333234306665346437330a366265346466626464396264393262
+34616366626565336637653032336465363165363334356535353833393332313239353736623237
+6434373738633039650a353435303366323139356234616433613663626334643939303361303764
+3636363333333333333333333
+36313937643431303637353931366363643661396238303530323262326334343432383637633439
+6365373237336535353661356430313965656538363436333836
diff --git a/test/integration/targets/vault/invalid_format/host_vars/broken-host-vars.example.com/vars b/test/integration/targets/vault/invalid_format/host_vars/broken-host-vars.example.com/vars
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/host_vars/broken-host-vars.example.com/vars
@@ -0,0 +1,11 @@
+---
+example_vars:
+  some_key:
+    another_key: some_value
+    bad_vault_dict_key: !vault |
+      $ANSIBLE_VAULT;1.1;AES256
+      64323332393930623633306662363165386332376638653035356132646165663632616263653366
+      623338xyz2313531623238613461323861376137656265380a366464663835633065616361636231
+      3366323866663763660a323766383531396433663861656532373663373134376263383263316261
+      3137
+
diff --git a/test/integration/targets/vault/invalid_format/inventory b/test/integration/targets/vault/invalid_format/inventory
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/inventory
@@ -0,0 +1,5 @@
+[broken-group-vars]
+broken.example.com
+
+[broken-host-vars]
+broken-host-vars.example.com
diff --git a/test/integration/targets/vault/invalid_format/original-broken-host-vars b/test/integration/targets/vault/invalid_format/original-broken-host-vars
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/original-broken-host-vars
@@ -0,0 +1,6 @@
+$ANSIBLE_VAULT;1.1;AES256
+64323332393930623633306662363165386332376638653035356132646165663632616263653366
+6233383362313531623238613461323861376137656265380a366464663835633065616361636231
+3366323866663763660a323766383531396433663861656532373663373134376263383263316261
+3137
+
diff --git a/test/integration/targets/vault/invalid_format/original-group-vars.yml b/test/integration/targets/vault/invalid_format/original-group-vars.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/original-group-vars.yml
@@ -0,0 +1,2 @@
+---
+some_var_that_fails: blippy
diff --git a/test/integration/targets/vault/invalid_format/some-vars b/test/integration/targets/vault/invalid_format/some-vars
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/some-vars
@@ -0,0 +1,6 @@
+$ANSIBLE_VAULT;1.1;AES256
+37303462633933386339386465613039363964643466663866356261313966663465646262636333
+3965643566363764356563363334363431656661636634380a333837343065326239336639373238
+64316236383836383434366662626339643561616630326137383262396331396538363136323063
+6236616130383264620a613863373631316234656236323332633166623738356664353531633239
+3533
diff --git a/test/integration/targets/vault/invalid_format/vault-secret b/test/integration/targets/vault/invalid_format/vault-secret
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/vault/invalid_format/vault-secret
@@ -0,0 +1 @@
+enemenemu
\ No newline at end of file
diff --git a/test/integration/targets/vault/runme.sh b/test/integration/targets/vault/runme.sh
--- a/test/integration/targets/vault/runme.sh
+++ b/test/integration/targets/vault/runme.sh
@@ -23,6 +23,7 @@ echo "This is a test file for edit2" > "${TEST_FILE_EDIT2}"
 FORMAT_1_1_HEADER="\$ANSIBLE_VAULT;1.1;AES256"
 FORMAT_1_2_HEADER="\$ANSIBLE_VAULT;1.2;AES256"
 
+
 VAULT_PASSWORD_FILE=vault-password
 # new format, view, using password client script
 ansible-vault view "$@" --vault-id vault-password@test-vault-client.py format_1_1_AES256.yml
@@ -367,3 +368,9 @@ WRONG_RC=$?
 echo "rc was $WRONG_RC (1 is expected)"
 [ $WRONG_RC -eq 1 ]
 
+# test invalid format ala https://github.com/ansible/ansible/issues/28038
+EXPECTED_ERROR='Vault format unhexlify error: Non-hexadecimal digit found'
+ansible-playbook "$@" -i invalid_format/inventory --vault-id invalid_format/vault-secret invalid_format/broken-host-vars-tasks.yml 2>&1 | grep "${EXPECTED_ERROR}"
+
+EXPECTED_ERROR='Vault format unhexlify error: Odd-length string'
+ansible-playbook "$@" -i invalid_format/inventory --vault-id invalid_format/vault-secret invalid_format/broken-group-vars-tasks.yml 2>&1 | grep "${EXPECTED_ERROR}"
diff --git a/test/units/parsing/vault/test_vault.py b/test/units/parsing/vault/test_vault.py
--- a/test/units/parsing/vault/test_vault.py
+++ b/test/units/parsing/vault/test_vault.py
@@ -41,6 +41,62 @@
 from units.mock.vault_helper import TextVaultSecret
 
 
+class TestUnhexlify(unittest.TestCase):
+    def test(self):
+        b_plain_data = b'some text to hexlify'
+        b_data = hexlify(b_plain_data)
+        res = vault._unhexlify(b_data)
+        self.assertEquals(res, b_plain_data)
+
+    def test_odd_length(self):
+        b_data = b'123456789abcdefghijklmnopqrstuvwxyz'
+
+        self.assertRaisesRegexp(vault.AnsibleVaultFormatError,
+                                '.*Vault format unhexlify error.*',
+                                vault._unhexlify,
+                                b_data)
+
+    def test_nonhex(self):
+        b_data = b'6z36316566653264333665333637623064303639353237620a636366633565663263336335656532'
+
+        self.assertRaisesRegexp(vault.AnsibleVaultFormatError,
+                                '.*Vault format unhexlify error.*Non-hexadecimal digit found',
+                                vault._unhexlify,
+                                b_data)
+
+
+class TestParseVaulttext(unittest.TestCase):
+    def test(self):
+        vaulttext_envelope = u'''$ANSIBLE_VAULT;1.1;AES256
+33363965326261303234626463623963633531343539616138316433353830356566396130353436
+3562643163366231316662386565383735653432386435610a306664636137376132643732393835
+63383038383730306639353234326630666539346233376330303938323639306661313032396437
+6233623062366136310a633866373936313238333730653739323461656662303864663666653563
+3138'''
+
+        b_vaulttext_envelope = to_bytes(vaulttext_envelope, errors='strict', encoding='utf-8')
+        b_vaulttext, b_version, cipher_name, vault_id = vault.parse_vaulttext_envelope(b_vaulttext_envelope)
+        res = vault.parse_vaulttext(b_vaulttext)
+        self.assertIsInstance(res[0], bytes)
+        self.assertIsInstance(res[1], bytes)
+        self.assertIsInstance(res[2], bytes)
+
+    def test_non_hex(self):
+        vaulttext_envelope = u'''$ANSIBLE_VAULT;1.1;AES256
+3336396J326261303234626463623963633531343539616138316433353830356566396130353436
+3562643163366231316662386565383735653432386435610a306664636137376132643732393835
+63383038383730306639353234326630666539346233376330303938323639306661313032396437
+6233623062366136310a633866373936313238333730653739323461656662303864663666653563
+3138'''
+
+        b_vaulttext_envelope = to_bytes(vaulttext_envelope, errors='strict', encoding='utf-8')
+        b_vaulttext, b_version, cipher_name, vault_id = vault.parse_vaulttext_envelope(b_vaulttext_envelope)
+        self.assertRaisesRegexp(vault.AnsibleVaultFormatError,
+                                '.*Vault format unhexlify error.*Non-hexadecimal digit found',
+                                vault.parse_vaulttext,
+                                b_vaulttext_envelope)
+
+
 class TestVaultSecret(unittest.TestCase):
     def test(self):
         secret = vault.VaultSecret()