Repository: ansible/ansible

Issue:
Ansible incorrectly constructing paths when running a playbook in a subdirectory
- **Issue Type:** Bug Report
- **Ansible Version:** 2.0.0
- **Ansible Configuration:** Added an additional roles path: `../../ansible/roles`
- **Environment:** OS X 10.11
##### Summary:

```
$ tree project
project
|-- Vagrantfile
|-- ansible.cfg
|-- ansible
|   |-- group_vars
|           |-- all
|   |-- playbook.yml
```

Running `ansible-playbook ansible/playbook.yml` via Vagrant provisioner fails to locate the `provisioning/group_vars/all`. In pre-2.0.0, the `ansible/group_vars/all` was found and used correctly.

Also, it would be nice to be able to use `ansible/ansible.cfg` instead of `./ansible.cfg` just to consolidate my files. In this case `project` is my top-level project directory.
##### Steps To Reproduce:

```
$ vagrant provision
==> vagrant_build: Running provisioner: ansible...
ERROR! the file_name '/Users/luke/project/ansible/ansible/group_vars/builders' does not exist, or is not readable
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

$ cat .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory 
# Generated by Vagrant

vagrant_build ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222 ansible_ssh_private_key_file=/Users/luke/project/.vagrant/machines/vagrant_build/virtualbox/private_key

[builders]
vagrant_build
```

Notice the doubling of the directory `ansible/ansible/group_vars` instead of the correct single `ansible/group_vars`.
##### Relevant Bit of Vagrantfile

``` ruby
  config.vm.define "vagrant_build" do |vagrant_build|
    vagrant_build.vm.hostname = "vagrant-build"
    vagrant_build.vm.provision "ansible" do |ansible|
      ansible.limit = "vagrant_build" # Disable default limit to connect to all the machines
      ansible.playbook = "ansible/playbook.yml"
      ansible.groups = {
        "builders" => ["vagrant_build"]
      }
    end
  end
```



Discussion:
Updated. Thanks for the tip.

first this looks like a vagrant issue, second i believe the error is already telling you why:

```
/Users/luke/project/ansible/ansible/group_vars/builders
```

^ there seems to be an extra ansible in your path.

That's the error: an extra ansible is getting in the path that I didn't put there. The error started when updating ansible to 2.0. I haven't changed Vagrant.

using ansible directly I have no issues like this

I'll submit a Vagrant issue and see what comes of it.

Looks like this is not related to Vagrant. I was able to reproduce directly:

```
$ ansible-playbook ansible/playbook.yml -i .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory 
ERROR! the file_name '/Users/luke/project/ansible/ansible/group_vars/builders' does not exist, or is not readable
```

Note the inventory file is quoted in previous comments.

Yeah, here's a shorter (non-vagrant) way to reproduce:

```
$ find .                                                                        (17:00:57)
.
./ansible
./ansible/group_vars
./ansible/group_vars/all
./ansible/site.yml
./inven
$ cat inven                                                                     (17:01:34)
localhost
$ cat ansible/group_vars/all                                                    (17:01:47)
test_var: "make a test"
$ cat ansible/site.yml                                                          (17:02:12)
---
- hosts: all
  tasks:
      - debug: var=test_var
$ ansible-playbook ansible/site.yml -i inven                                    (17:02:30)

PLAY ****************************************************************************

TASK [setup] ********************************************************************
ok: [localhost]

TASK [debug var=test_var] *******************************************************
ok: [localhost] => {
    "changed": false, 
    "test_var": "VARIABLE IS NOT DEFINED!"
}

PLAY RECAP **********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0
```

(Note: I'd already started working on this in #11806 so that's why the variable isn't defined instead of an error being thrown.  If you do this with a fresh checkout, you'll get the reported error from running ansible-playbook)

If there's no variables defined in group_vars/all then the error isn't thrown.  Not sure why that is.

So far I've tracked this to DataLoader._basedir being the directory path up to the .yml file. (which seems proper) and vars/__init__.py:_load_inventory_file() getting a filename that seems to be the dirname(YAML_FILE_GIVEN_ON_CLI).  The latter seems to be the issue to me. 


Resolving patch:
diff --git a/lib/ansible/inventory/__init__.py b/lib/ansible/inventory/__init__.py
--- a/lib/ansible/inventory/__init__.py
+++ b/lib/ansible/inventory/__init__.py
@@ -640,6 +640,7 @@ def _get_hostgroup_vars(self, host=None, group=None, new_pb_basedir=False):
         else:
             basedirs = [self._playbook_basedir]
 
+        cur_loader_basedir = self._loader.get_basedir()
         for basedir in basedirs:
 
             # this can happen from particular API usages, particularly if not run
@@ -660,13 +661,16 @@ def _get_hostgroup_vars(self, host=None, group=None, new_pb_basedir=False):
             # FIXME: these should go to VariableManager
             if group and host is None:
                 # load vars in dir/group_vars/name_of_group
-                base_path = os.path.join(basedir, "group_vars/%s" % group.name)
+                self._loader.set_basedir(basedir)
+                base_path = os.path.join("group_vars", group.name)
                 results = self._variable_manager.add_group_vars_file(base_path, self._loader)
             elif host and group is None:
                 # same for hostvars in dir/host_vars/name_of_host
-                base_path = os.path.join(basedir, "host_vars/%s" % host.name)
+                self._loader.set_basedir(basedir)
+                base_path = os.path.join("host_vars", host.name)
                 results = self._variable_manager.add_host_vars_file(base_path, self._loader)
 
+        self._loader.set_basedir(cur_loader_basedir)
         # all done, results is a dictionary of variables for this particular host.
         return results
 
diff --git a/lib/ansible/parsing/__init__.py b/lib/ansible/parsing/__init__.py
--- a/lib/ansible/parsing/__init__.py
+++ b/lib/ansible/parsing/__init__.py
@@ -112,15 +112,19 @@ def load_from_file(self, file_name):
         return parsed_data
 
     def path_exists(self, path):
+        path = self.path_dwim(path)
         return os.path.exists(path)
 
     def is_file(self, path):
+        path = self.path_dwim(path)
         return os.path.isfile(path)
 
     def is_directory(self, path):
+        path = self.path_dwim(path)
         return os.path.isdir(path)
 
     def list_directory(self, path):
+        path = self.path_dwim(path)
         return os.listdir(path)
 
     def _safe_load(self, stream, file_name=None):


Test patch:
diff --git a/test/units/mock/loader.py b/test/units/mock/loader.py
--- a/test/units/mock/loader.py
+++ b/test/units/mock/loader.py
@@ -29,11 +29,11 @@ class DictDataLoader(DataLoader):
     def __init__(self, file_mapping=dict()):
         assert type(file_mapping) == dict
 
+        super(DictDataLoader, self).__init__()
+
         self._file_mapping = file_mapping
         self._build_known_directories()
 
-        super(DictDataLoader, self).__init__()
-
     def load_from_file(self, path):
         if path in self._file_mapping:
             return self.load(self._file_mapping[path], path)