Repository: ansible/ansible

Issue:
upgrade from 2.3 to 2.4 breaks vars loading when using 'path' as inventory_hostname
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
[chroot](https://docs.ansible.com/ansible/devel/plugins/connection/chroot.html)

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.1.0
  config file = /home/ksmets/my-ansible-playbooks/ansible.cfg
  configured module search path = [u'/home/ksmets/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ksmets/.virtualenvs/ansible/local/lib/python2.7/site-packages/ansible
  executable location = /home/ksmets/.virtualenvs/ansible/bin/ansible
  python version = 2.7.11+ (default, Apr 17 2016, 14:00:29) [GCC 5.3.1 20160413]
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"

Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).

-->
```
```
##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->

```
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
```

```
Linux midgard 4.4.0-98-generic #121-Ubuntu SMP Tue Oct 10 14:24:03 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
```

##### SUMMARY
<!--- Explain the problem briefly -->

After upgrading from a 2.3-release (i.e. `2.3.2.0`) to the latest 2.4-release (i.e. `2.4.1.0`) the chroot connection plugin no longer works.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
# file: build-rpi-sd-card.yml

- hosts: localhost
  vars:
    rasbian_version: 2017-09-07-raspbian-stretch-lite
    rasbian_mountpoint: /mnt/rpi
  pre_tasks:
  - name: install required packages
    apt:
      name: "{{ item }}"
      state: present
    with_items:
    - qemu
    - qemu-user-static
    - binfmt-support
  tasks:
  - name: download rasbian
    get_url:
      url: http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2017-09-08/{{ rasbian_version }}.zip
      dest: ~/Downloads/{{ rasbian_version }}.zip

  - name: extract rasbian image
    unarchive:
      src: ~/Downloads/{{ rasbian_version }}.zip
      dest: /dev/shm
      creates: /dev/shm/{{ rasbian_version }}.img

  - name: get the name of the first unused loop device
    command: losetup --find
    register: result
    changed_when: False

  - name: register the name of the first unused loop device
    set_fact:
      loopdev: "{{ result.stdout }}"

  - name: set up the loop device
    command: losetup --partscan {{ loopdev }} /dev/shm/{{ rasbian_version }}.img creates={{ loopdev }}p*

  - name: ensure the mount point exists
    file:
      path: "{{ rasbian_mountpoint }}"
      state: directory

  - name: mount the root partition
    command: mount {{ loopdev }}p2 {{ rasbian_mountpoint }} creates={{ rasbian_mountpoint }}/boot warn=no

  - name: mount the boot partition
    command: mount {{ loopdev }}p1 {{ rasbian_mountpoint }}/boot creates={{ rasbian_mountpoint }}/boot/kernel.img warn=no

  - name: copy qemu-arm-static
    copy:
      src: /usr/bin/qemu-arm-static
      dest: "{{ rasbian_mountpoint }}/usr/bin/qemu-arm-static"
      mode: 0755

  - name: comment line in /etc/ld.so.preload
    lineinfile:
      path: "{{ rasbian_mountpoint }}/etc/ld.so.preload"
      regexp: '/usr/lib/arm-linux-gnueabihf/libarmmem.so'
      line: '#/usr/lib/arm-linux-gnueabihf/libarmmem.so'

  - name: add {{ rasbian_mountpoint }} to chroots group
    add_host:
      name: "{{ rasbian_mountpoint }}"
      groups: chroots

- hosts: chroots
  connection: chroot
  tasks:
  - ping:

- hosts: localhost
  vars:
    rasbian_mountpoint: /mnt/rpi
  tasks:
  - name: uncomment line in /etc/ld.so.preload
    lineinfile:
      path: "{{ rasbian_mountpoint }}/etc/ld.so.preload"
      regexp: '#/usr/lib/arm-linux-gnueabihf/libarmmem.so'
      line: '/usr/lib/arm-linux-gnueabihf/libarmmem.so'

  - name: remove qemu-arm-static
    file:
      path: "{{ rasbian_mountpoint }}/usr/bin/qemu-arm-static"
      state: absent

  - name: unmount the boot partition
    command: umount {{ rasbian_mountpoint }}/boot removes={{ rasbian_mountpoint }}/boot/kernel.img

  - name: unmount the root partition
    command: umount {{ rasbian_mountpoint }} removes={{ rasbian_mountpoint }}/boot

  - name: ensure the mount point doesn't exist
    file:
      path: "{{ rasbian_mountpoint }}"
      state: absent

  - name: detach the loop device
    command: losetup --detach {{ loopdev }} removes={{ loopdev }}p*
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
```
sudo ansible -c chroot -i /mnt/rpi, -m ping all
/mnt/rpi | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
```

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
sudo ansible -c chroot -i /mnt/rpi, -m ping all -vvvv

ansible 2.4.1.0
  config file = /home/ksmets/my-ansible-playbooks/ansible.cfg
  configured module search path = [u'/home/ksmets/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ksmets/.virtualenvs/ansible/local/lib/python2.7/site-packages/ansible
  executable location = /home/ksmets/.virtualenvs/ansible/bin/ansible
  python version = 2.7.11+ (default, Apr 17 2016, 14:00:29) [GCC 5.3.1 20160413]
Using /home/ksmets/my-ansible-playbooks/ansible.cfg as config file
setting up inventory plugins
Unable to parse address from hostname, leaving unchanged: Not a valid network hostname: /mnt/rpi
Parsed /mnt/rpi, inventory source with host_list plugin
Loading callback plugin minimal of type stdout, v2.0 from /home/ksmets/.virtualenvs/ansible/local/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc
META: ran handlers
ERROR! Syntax Error while loading YAML.


The error appears to have been in '/mnt/rpi/etc/dhcp/debug': line 12, column 1, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

if [ "$RUN" = "yes" ]; then
	echo "$(date): entering ${1%/*}, dumping variables." \
^ here
There appears to be a tab character at the start of the line.

YAML does not use tabs for formatting. Tabs should be replaced with spaces.

For example:
    - name: update tooling
      vars:
        version: 1.2.3
#    ^--- there is a tab there.

Should be written as:
    - name: update tooling
      vars:
        version: 1.2.3
# ^--- all spaces here.

exception type: <class 'yaml.scanner.ScannerError'>
exception: while scanning for the next token
found character '\t' that cannot start any token
  in "<unicode string>", line 12, column 1:
    	echo "$(date): entering ${1%/*} ... 
    ^
```



Discussion:
its not the chroot plugin that breaks here, its 'host_vars loading' as the inventory hostname is valid path and the vars plugin finds a file but it fails to parse it.

This should  workaround  it for now:

`ansible -c chroot -i rpi_host, -m ping all -e 'ansible_host=/mnt/rpi`

Resolving patch:
diff --git a/lib/ansible/plugins/vars/host_group_vars.py b/lib/ansible/plugins/vars/host_group_vars.py
--- a/lib/ansible/plugins/vars/host_group_vars.py
+++ b/lib/ansible/plugins/vars/host_group_vars.py
@@ -74,31 +74,33 @@ def get_vars(self, loader, path, entities, cache=True):
             else:
                 raise AnsibleParserError("Supplied entity must be Host or Group, got %s instead" % (type(entity)))
 
-            try:
-                found_files = []
-                # load vars
-                opath = os.path.realpath(os.path.join(self._basedir, subdir))
-                key = '%s.%s' % (entity.name, opath)
-                if cache and key in FOUND:
-                    found_files = FOUND[key]
-                else:
-                    b_opath = to_bytes(opath)
-                    # no need to do much if path does not exist for basedir
-                    if os.path.exists(b_opath):
-                        if os.path.isdir(b_opath):
-                            self._display.debug("\tprocessing dir %s" % opath)
-                            found_files = self._find_vars_files(opath, entity.name)
-                            FOUND[key] = found_files
-                        else:
-                            self._display.warning("Found %s that is not a directory, skipping: %s" % (subdir, opath))
-
-                for found in found_files:
-                    new_data = loader.load_from_file(found, cache=True, unsafe=True)
-                    if new_data:  # ignore empty files
-                        data = combine_vars(data, new_data)
-
-            except Exception as e:
-                raise AnsibleParserError(to_native(e))
+            # avoid 'chroot' type inventory hostnames /path/to/chroot
+            if not entity.name.startswith(os.path.sep):
+                try:
+                    found_files = []
+                    # load vars
+                    opath = os.path.realpath(os.path.join(self._basedir, subdir))
+                    key = '%s.%s' % (entity.name, opath)
+                    if cache and key in FOUND:
+                        found_files = FOUND[key]
+                    else:
+                        b_opath = to_bytes(opath)
+                        # no need to do much if path does not exist for basedir
+                        if os.path.exists(b_opath):
+                            if os.path.isdir(b_opath):
+                                self._display.debug("\tprocessing dir %s" % opath)
+                                found_files = self._find_vars_files(opath, entity.name)
+                                FOUND[key] = found_files
+                            else:
+                                self._display.warning("Found %s that is not a directory, skipping: %s" % (subdir, opath))
+
+                    for found in found_files:
+                        new_data = loader.load_from_file(found, cache=True, unsafe=True)
+                        if new_data:  # ignore empty files
+                            data = combine_vars(data, new_data)
+
+                except Exception as e:
+                    raise AnsibleParserError(to_native(e))
         return data
 
     def _find_vars_files(self, path, name):


Test patch: