Repository: ansible/ansible

Issue:
Meta: refresh_inventory fails if inventory is relative path
##### ISSUE TYPE
- Bug Report
##### ANSIBLE VERSION

```
ansible 2.1.0.0
  config file = /home/git/ansible.cfg
  configured module search path = ['/usr/share/ansible']
```
##### CONFIGURATION

<!---
[defaults]
inventory = inventory/
-->
##### SUMMARY

When using a directory as inventory, `meta: refresh_inventory` doesn't do anything and just prints  `[WARNING]: Host file not found: inventory`
##### STEPS TO REPRODUCE

```
cat << EOF > ansible.cfg
[defaults]
inventory = inventory/
EOF

echo -e 'host_a \n hostb' > inventory/hosts

cat << EOF > playbook.yml

---
- hosts: all
  tasks:
    - name: debug
      debug:
        msg: "Hello world!"
    - meta: refresh_inventory
EOF
```
##### EXPECTED RESULTS

A refreshed inventory
##### ACTUAL RESULTS

`[WARNING]: Host file not found: inventory`



Discussion:
@alvaroaleman thank you for the concise test case. However, I am unable to reproduce the described error. Every version of ansible that included -meta passes for me with your testcase ...

```
v1.9.4-1 ; 0
v1.9.5-0.1.rc1 ; 0
v1.9.5-1 ; 0
v1.9.6-0.1.rc1 ; 0
v1.9.6-1 ; 0
v2.0.0-0.6.rc1 ; 0
v2.0.0-0.7.rc2 ; 0
v2.0.0-0.8.rc3 ; 0
v2.0.0-0.9.rc4 ; 0
v2.0.0.0-1 ; 0
v2.0.0.1-1 ; 0
v2.0.0.2-1 ; 0
v2.0.1.0-0.1.rc1 ; 0
v2.0.1.0-0.2.rc2 ; 0
v2.0.1.0-1 ; 0
v2.0.2.0-0.1.rc1 ; 0
v2.0.2.0-0.2.rc2 ; 0
v2.0.2.0-0.3.rc3 ; 0
v2.0.2.0-0.4.rc4 ; 0
v2.0.2.0-1 ; 0
v2.1.0.0-0.1.rc1 ; 0
v2.1.0.0-0.2.rc2 ; 0
v2.1.0.0-0.3.rc3 ; 0
v2.1.0.0-0.4.rc4 ; 0
v2.1.0.0-1 ; 0
v2.1.1.0-0.1.rc1 ; 0
v2.1.1.0-0.2.rc2 ; 0
v2.1.1.0-0.3.rc3 ; 0
v2.1.1.0-0.4.rc4 ; 0
v2.1.1.0-0.5.rc5 ; 0
v2.1.1.0-1 ; 0
```

Can you give us the full ansible-playbook cli and -vvvv output?

Thanks for your efforts  and sorry I didn't properly test my testcase.

Apparently, the issue only occurs if the playbook is in a subdirectory and being called from the main directory, where the `ansible.cfg` and `inventory` folder are.

A sample testcase can be found here: https://github.com/alvaroaleman/ansible_inventory_testcase

Tested on 2.1.0.0

Hi, any known workaround for the issue which I could use in my playbooks in 2.1.0.0? Thanks!

I have the issue too. I have to manually run `./inventory/ec2.py --refresh-cache` to update my inventory after provisioning a new EC2 instance.

@linusgg, thanks for the tip... I was using:

```
  tasks:
    - name: Remove dynamic inventory cashe
      command: rm -fr ~/.ansible/tmp/* warn=no
      changed_when: False
```

`refresh_inventory` cannot handle custom caches in the dynamic scripts, if you have those enabled it won't do anything. Caching is not standardized for inventory scripts.

@alvaroaleman ping, this issue is still waiting on your feedback. We will close the issue if you do not respond.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_closure --->

I think the needs_info tag doesn't fit here anymore as I provided a testcase in [comment#3](https://github.com/ansible/ansible/issues/16857#issuecomment-236205253) @jctanner ?

I think I can shine some light on the cause. I have a "similar" setup.

My Dir layout looks like this

```
ansible_dir
  - inventory
  - roles
  - playbooks
```

When I call:
`
ansible-playbook -i inventory/updates_weekly playbooks/updates_weekly.yml --ask-become
`

The vars from ansible are wrong (I used debug var):

```
inventory_dir: /ansible_dir/inventory
inventory_file: inventory/updates_weekly
```

So the meta refresh fails:
`Host file not found: inventory/updates_weekly`

Because the path he builds is:
`/ansible_dir/inventory/inventory/updates_weekly`

He doubles the inventory directory :(

I rewrite my inventory file at the beginning of the play. That's why I called the refresh.

To add to @Nebelwolf's comments, the issue only appears to crop up when the playbook is in a different working directory than where the ansible-playbook command is issued.

I was able to replicate the issue with the following directory structure:
ansible_dir
- inventory
  - ec2.py
  - ec2.ini
- playbooks
  - test.yaml

With test.yaml containing:

```
---
- hosts: localhost
  become: no
  gather_facts: no
  tasks:
  - debug: var=groups
  - meta: refresh_inventory
  - debug: var=groups
```

ansible-playbook -i inventory/ec2.py playbooks/test.yaml results in the following output:

```
PLAY [localhost] ***************************************************************

TASK [debug] *******************************************************************
Tuesday 18 October 2016  15:56:12 -0400 (0:00:00.041)       0:00:00.041 ******* 
ok: [localhost] => {
    "groups": {
        "all": [
            "host1", 
            "host2", 
            "host3"
        ], 
        "ami-10251c7a": [
            "host1", 
            "host2", 
            "host3"
        ], 
        "ec2": [
            "host1", 
            "host2", 
            "host3"        ], 
        "i-305f9000": [
            "host1"
        ], 
        "i-4cb71adf": [
            "host2"
        ], 
        "i-f65ddce0": [
            "host3"
        ], 
        "key_default": [
            "host1", 
            "host2", 
            "host3"
        ], 
        "ungrouped": [
            "localhost"
        ], 
        "us-east-1": [
            "host1", 
            "host2", 
            "host3"
        ]
    }
}
 [WARNING]: Host file not found: inventory/ec2.py


TASK [debug] *******************************************************************
Tuesday 18 October 2016  15:56:12 -0400 (0:00:00.045)       0:00:00.087 ******* 
ok: [localhost] => {
    "groups": {
        "all": [], 
        "ungrouped": []
    }
}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0   
```

I ran into a similar issue, but to solve the issue, I had to give the full path of the directory. Relative paths failed

I have also encountered this problem.  I have several different "environments" which must be maintained separately.  Each is in it's own directory.  My M.O. was to be in that directory, and run playbooks from there:

[/opt/ansible/inv/TEST1] $ ansible-playbook ../playbooks/create_instances.yml

After creating the instances, I tried meta: refresh_inventory.  It told me that /opt/ansible/playbooks/ec2.py does not exist.  Of course not -- it's in /opt/ansible/inv/TEST1.  The file /opt/ansible/inv/TEST1/ansible.cfg is, apparently, completely ignored.  The ansible-playbook is looking in ITS current directory, which is the playbook directory, for the inventory.

My workaround was to set ANSIBLE_INVENTORY and ANSIBLE_CONFIG variables, and pass the private key on the command line.

The fix would be to get refresh_inventory() to change back to the directory from which ansible-playbook was invoked, then return to the playbook directory.  


Resolving patch:
diff --git a/lib/ansible/inventory/__init__.py b/lib/ansible/inventory/__init__.py
--- a/lib/ansible/inventory/__init__.py
+++ b/lib/ansible/inventory/__init__.py
@@ -38,6 +38,7 @@
 from ansible.parsing.utils.addresses import parse_address
 from ansible.plugins import vars_loader
 from ansible.utils.vars import combine_vars
+from ansible.utils.path import unfrackpath
 
 try:
     from __main__ import display
@@ -58,7 +59,7 @@ def __init__(self, loader, variable_manager, host_list=C.DEFAULT_HOST_LIST):
 
         # the host file file, or script path, or list of hosts
         # if a list, inventory data will NOT be loaded
-        self.host_list = host_list
+        self.host_list = unfrackpath(host_list, follow=False)
         self._loader = loader
         self._variable_manager = variable_manager
         self.localhost = None
diff --git a/lib/ansible/utils/path.py b/lib/ansible/utils/path.py
--- a/lib/ansible/utils/path.py
+++ b/lib/ansible/utils/path.py
@@ -26,12 +26,12 @@
 __all__ = ['unfrackpath', 'makedirs_safe']
 
 
-def unfrackpath(path):
+def unfrackpath(path, follow=True):
     '''
-    Returns a path that is free of symlinks, environment
-    variables, relative path traversals and symbols (~)
+    Returns a path that is free of symlinks (if follow=True), environment variables, relative path traversals and symbols (~)
 
     :arg path: A byte or text string representing a path to be canonicalized
+    :arg follow: A boolean to indicate of symlinks should be resolved or not
     :raises UnicodeDecodeError: If the canonicalized version of the path
         contains non-utf8 byte sequences.
     :rtype: A text string (unicode on pyyhon2, str on python3).
@@ -41,9 +41,13 @@ def unfrackpath(path):
     example::
         '$HOME/../../var/mail' becomes '/var/spool/mail'
     '''
-    canonical_path = os.path.normpath(os.path.realpath(os.path.expanduser(os.path.expandvars(to_bytes(path, errors='surrogate_or_strict')))))
-    return to_text(canonical_path, errors='surrogate_or_strict')
 
+    if follow:
+        final_path = os.path.normpath(os.path.realpath(os.path.expanduser(os.path.expandvars(to_bytes(path, errors='surrogate_or_strict')))))
+    else:
+        final_path = os.path.normpath(os.path.abspath(os.path.expanduser(os.path.expandvars(to_bytes(path, errors='surrogate_or_strict')))))
+
+    return to_text(final_path, errors='surrogate_or_strict')
 
 def makedirs_safe(path, mode=None):
     '''Safe way to create dirs in muliprocess/thread environments.


Test patch: