Repository: ansible/ansible

Issue:
is_dir always fails on path with a tilde (~)
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
Always add information AFTER of these html comments.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report

##### COMPONENT NAME
<!---
Insert below this comment the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path
-->
Path tests `is_dir` and `exists`

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.0
  config file = /home/user/.ansible.cfg
  configured module search path = [u'/home/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
```
CACHE_PLUGIN(/home/user/.ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/home/user/.ansible.cfg) = ~/.ansible/facts
CACHE_PLUGIN_TIMEOUT(/home/user/.ansible.cfg) = 86400
DEFAULT_GATHERING(/home/user/.ansible.cfg) = smart
DEFAULT_STDOUT_CALLBACK(/home/user/.ansible.cfg) = debug
HOST_KEY_CHECKING(/home/user/.ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
Red Hat Enterprise Linux Server release 7.4 (Maipo)

##### SUMMARY
<!--- Explain the problem briefly -->
When testing a path containing the tilde, e.g. `~/tmp`, for existence or being a directory by using `is_dir` or `exists` tests, the result always fails.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- hosts: localhost
  gather_facts: no
  vars:
    local_dir: ~/temp

  tasks:

    - name: Create dir
      file:
        path: "{{ local_dir }}"
        state: directory

    - name: Fail if dir doesn't exist
      fail: msg="The {{ local_dir }} directory doesn't exist!"
      when: not local_dir is is_dir
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
The second task (fail) should not run.

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
The second task always runs, regardless the directory existing or not, i.e. the `when` condition gets evaluated wrongly.

<!--- Paste verbatim command output between quotes below -->
```
ansible-playbook 2.5.0
  config file = /home/user/.ansible.cfg
  configured module search path = [u'/home/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
Using /home/user/.ansible.cfg as config file
setting up inventory plugins
Parsed /home/user/cloud/hosts.yaml inventory source with yaml plugin
Loading callback plugin debug of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/debug.pyc

PLAYBOOK: test.yaml ***************************************************************************************************************
1 plays in test.yaml

PLAY [localhost] ******************************************************************************************************************
META: ran handlers

TASK [Create dir] *****************************************************************************************************************
task path: /home/user/ansible/test.yaml:22
Using module file /usr/lib/python2.7/site-packages/ansible/modules/files/file.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: user
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp/ansible-tmp-1523367391.87-3417914523823 `" && e
cho ansible-tmp-1523367391.87-3417914523823="` echo /home/user/.ansible/tmp/ansible-tmp-1523367391.87-3417914523823 `" ) && slee
p 0'
<127.0.0.1> PUT /home/user/.ansible/tmp/ansible-local-15923XfiKZB/tmpaXRWoY TO /home/user/.ansible/tmp/ansible-tmp-1523367391.87
-3417914523823/file.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/user/.ansible/tmp/ansible-tmp-1523367391.87-3417914523823/ /home/user/.ansible/tmp/
ansible-tmp-1523367391.87-3417914523823/file.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/user/.ansible/tmp/ansible-tmp-1523367391.87-3417914523823/file.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1523367391.87-3417914523823/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "changed": false,
    "diff": {
        "after": {
            "path": "/home/user/temp"
        },
        "before": {
            "path": "/home/user/temp"
        }
    },
    "gid": 1002,
    "group": "user",
    "invocation": {
        "module_args": {
            "attributes": null,
            "backup": null,
            "content": null,
            "delimiter": null,
            "diff_peek": null,
            "directory_mode": null,
            "follow": false,
            "force": false,
            "group": null,
            "mode": null,
            "original_basename": null,
            "owner": null,
            "path": "/home/user/temp",
            "recurse": false,
            "regexp": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "src": null,
            "state": "directory",
            "unsafe_writes": null,
            "validate": null
        }
    },
    "mode": "0775",
    "owner": "user",
    "path": "/home/user/temp",
    "secontext": "unconfined_u:object_r:user_home_t:s0",
    "size": 6,
    "state": "directory",
    "uid": 1002
}

TASK [Fail if dir doesn't exist] **************************************************************************************************
task path: /home/user/ansible/test.yaml:27
fatal: [localhost]: FAILED! => {
    "changed": false
}

MSG:

The ~/temp directory doesn't exist!
        to retry, use: --limit @/home/user/ansible/test.retry

PLAY RECAP ************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1
```



Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
!component =lib/ansible/plugins/test/files.py
Files identified in the description:
* [lib/ansible/plugins/test/files.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/test/files.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
As per @bcoca, we probably need expandvars/expanduser to wrap all of the files tests.

Resolving patch:
diff --git a/lib/ansible/plugins/filter/core.py b/lib/ansible/plugins/filter/core.py
--- a/lib/ansible/plugins/filter/core.py
+++ b/lib/ansible/plugins/filter/core.py
@@ -530,6 +530,7 @@ def filters(self):
             'basename': partial(unicode_wrap, os.path.basename),
             'dirname': partial(unicode_wrap, os.path.dirname),
             'expanduser': partial(unicode_wrap, os.path.expanduser),
+            'expandvars': partial(unicode_wrap, os.path.expandvars),
             'realpath': partial(unicode_wrap, os.path.realpath),
             'relpath': partial(unicode_wrap, os.path.relpath),
             'splitext': partial(unicode_wrap, os.path.splitext),


Test patch: