Repository: ansible-collections/community.general

Issue:
git_config task expands tilde in its value and always reports as changed
**Summary**

If `value` option of `git_config` task starts with tilde and slash (~/), `git_config` expands it as `HOME` path.  Moreover, second execution of this task doesn't report `ok` but `changed`.


**Issue Type**

Bug Report

**Component Name**

git_config

**Ansible Version**

- ansible: 2.10.7
- ansible-base: 2.10.5

```console
$ ansible --version
ansible 2.10.5
  config file = None
  configured module search path = ['/home/kai2nenobu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible
  executable location = /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/bin/ansible
  python version = 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]
```


**Configuration**

```console
$ ansible-config dump --only-changed
```

(no output)


**OS / Environment**

```console
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
$ uname -a
Linux pollmann 4.19.104-microsoft-standard #1 SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
```


**Steps To Reproduce**

minimal playbook (`test.yml`)

```yaml
- hosts: localhost
  gather_facts: no
  tasks:
    - git_config:
        name: core.hooksPath
        value: '~/foo/bar'
        state: present
        scope: global
```

and apply this playbook as below.

```sh
ansible-playbook -i localhost, -c local test.yml -vvv
```



**Expected Results**

First play configures `core.hooksPath` to `~/foo/bar`, and second play changes nothing and reports `ok`.


**Actual Results**

First play successes, but `core.hooksPath` is configured to expanded value (`/home/kai2nenobu/foo/bar`).

```console
$ git config core.hooksPath
/home/kai2nenobu/foo/bar
```

<details>
<summary>Console output for first play</summary>

```console
$ ansible-playbook -i localhost, -c local test.yml -vvvv
ansible-playbook 2.10.5
  config file = None
  configured module search path = ['/home/kai2nenobu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible
  executable location = /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/bin/ansible-playbook
  python version = 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]
No config file found; using defaults
setting up inventory plugins
Set default localhost to localhost
Parsed localhost, inventory source with host_list plugin
redirecting (type: modules) ansible.builtin.git_config to community.general.git_config
Loading collection community.general from /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible_collections/community/general
Loading callback plugin default of type stdout, v2.0 from /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: test.yml *****************************************************************************************************
Positional arguments: test.yml
verbosity: 4
connection: local
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('localhost,',)
forks: 5
1 plays in test.yml

PLAY [localhost] *******************************************************************************************************
META: ran handlers

TASK [git_config] ******************************************************************************************************
task path: /home/kai2nenobu/tmp/git_config_bug/test.yml:4
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: kai2nenobu
<localhost> EXEC /bin/sh -c 'echo ~kai2nenobu && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/kai2nenobu/.ansible/tmp `"&& mkdir "` echo /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962340.654051-3866-132583835726410 `" && echo ansible-tmp-1612962340.654051-3866-132583835726410="` echo /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962340.654051-3866-132583835726410 `" ) && sleep 0'
redirecting (type: modules) ansible.builtin.git_config to community.general.git_config
<localhost> Attempting python interpreter discovery
<localhost> EXEC /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'python2.6'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
Using module file /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible_collections/community/general/plugins/modules/git_config.py
<localhost> PUT /home/kai2nenobu/.ansible/tmp/ansible-local-3860m6fwifes/tmp_ui8du8w TO /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962340.654051-3866-132583835726410/AnsiballZ_git_config.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962340.654051-3866-132583835726410/ /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962340.654051-3866-132583835726410/AnsiballZ_git_config.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962340.654051-3866-132583835726410/AnsiballZ_git_config.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962340.654051-3866-132583835726410/ > /dev/null 2>&1 && sleep 0'
[DEPRECATION WARNING]: Distribution ubuntu 20.04 on host localhost should use /usr/bin/python3, but is using
/usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using
the discovered platform python for this host. See
https://docs.ansible.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more information. This
feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
changed: [localhost] => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": true,
    "diff": {
        "after": "~/foo/bar\n",
        "after_header": "/usr/bin/git config --includes --global core.hooksPath",
        "before": "\n",
        "before_header": "/usr/bin/git config --includes --global core.hooksPath"
    },
    "invocation": {
        "module_args": {
            "list_all": false,
            "name": "core.hooksPath",
            "repo": null,
            "scope": "global",
            "state": "present",
            "value": "~/foo/bar"
        }
    },
    "msg": "setting changed"
}
META: ran handlers
META: ran handlers

PLAY RECAP *************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```

</details>


Second play reports `changed`, but `core.hooksPath` still remains `/home/kai2nenobu/foo/bar`.

<details>
<summary>Console output for second play</summary>


```console
$ ansible-playbook -i localhost, -c local test.yml -vvvv
ansible-playbook 2.10.5
  config file = None
  configured module search path = ['/home/kai2nenobu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible
  executable location = /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/bin/ansible-playbook
  python version = 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]
No config file found; using defaults
setting up inventory plugins
Set default localhost to localhost
Parsed localhost, inventory source with host_list plugin
redirecting (type: modules) ansible.builtin.git_config to community.general.git_config
Loading collection community.general from /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible_collections/community/general
Loading callback plugin default of type stdout, v2.0 from /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: test.yml *****************************************************************************************************
Positional arguments: test.yml
verbosity: 4
connection: local
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('localhost,',)
forks: 5
1 plays in test.yml

PLAY [localhost] *******************************************************************************************************
META: ran handlers

TASK [git_config] ******************************************************************************************************
task path: /home/kai2nenobu/tmp/git_config_bug/test.yml:4
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: kai2nenobu
<localhost> EXEC /bin/sh -c 'echo ~kai2nenobu && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/kai2nenobu/.ansible/tmp `"&& mkdir "` echo /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962414.718582-3914-117938253070940 `" && echo ansible-tmp-1612962414.718582-3914-117938253070940="` echo /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962414.718582-3914-117938253070940 `" ) && sleep 0'
redirecting (type: modules) ansible.builtin.git_config to community.general.git_config
<localhost> Attempting python interpreter discovery
<localhost> EXEC /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'python2.6'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
Using module file /home/kai2nenobu/.cache/pypoetry/virtualenvs/git-config-bug-0-3PCpOc-py3.8/lib/python3.8/site-packages/ansible_collections/community/general/plugins/modules/git_config.py
<localhost> PUT /home/kai2nenobu/.ansible/tmp/ansible-local-39087e1f6diq/tmpbbmxlymh TO /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962414.718582-3914-117938253070940/AnsiballZ_git_config.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962414.718582-3914-117938253070940/ /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962414.718582-3914-117938253070940/AnsiballZ_git_config.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962414.718582-3914-117938253070940/AnsiballZ_git_config.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/kai2nenobu/.ansible/tmp/ansible-tmp-1612962414.718582-3914-117938253070940/ > /dev/null 2>&1 && sleep 0'
[DEPRECATION WARNING]: Distribution ubuntu 20.04 on host localhost should use /usr/bin/python3, but is using
/usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using
the discovered platform python for this host. See
https://docs.ansible.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more information. This
feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
changed: [localhost] => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": true,
    "diff": {
        "after": "~/foo/bar\n",
        "after_header": "/usr/bin/git config --includes --global core.hooksPath",
        "before": "/home/kai2nenobu/foo/bar\n",
        "before_header": "/usr/bin/git config --includes --global core.hooksPath"
    },
    "invocation": {
        "module_args": {
            "list_all": false,
            "name": "core.hooksPath",
            "repo": null,
            "scope": "global",
            "state": "present",
            "value": "~/foo/bar"
        }
    },
    "msg": "setting changed"
}
META: ran handlers
META: ran handlers

PLAY RECAP *************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```

</details>






Discussion:
Files identified in the description:
* [`plugins/modules/source_control/git_config.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/source_control/git_config.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @djmattyg007 @mgedmin
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Huh!  Looking at the source I see this:
https://github.com/ansible-collections/community.general/blob/e9551df5ed8000f5b13ae14d22dfef24be829b5f/plugins/modules/source_control/git_config.py#L233

which scares me, and which wasn't part of [my code](https://gist.github.com/mgedmin/b38c74e2d25cb4f47908#file-gitconfig-py-L80).  AFICT the `' '.join(cmd)` part is not needed, this should be `module.run_command(cmd, cwd=dir)`.

Anyway, that's not the problem.

Next we have
https://github.com/ansible-collections/community.general/blob/e9551df5ed8000f5b13ae14d22dfef24be829b5f/plugins/modules/source_control/git_config.py#L262-L265

which looks wrong to me -- no quoting (`shlex_quote`) should be necessary when you pass a list to run_command(), since the command [doesn't go through shell in that case](https://github.com/ansible/ansible/blob/8b07d4616633ee7be190149eacb225d02fd2c810/lib/ansible/module_utils/basic.py#L2529)!

So, this explains to me why the module returns `changed` all the time -- it's setting the quoted value but expects an unquoted value.

I don't understand where the ~ is getting expanded.
@mgedmin apparently the `' '.join()` has been in there for a long time: https://github.com/ansible/ansible-modules-extras/pull/1945/files#diff-4c75f862bc4829e624aa2301973ad669ccf7684b1f1f9358b964619d4d7b5f6dR178

The `~` might get expanded by some shell magic, though I concur that I don't understand why that happens either...
Adding @kai2nenobu 's test case to the integration tests for `git_config`, working on the case.
Heyyyyy look: module.run_command() expands all the arguments by default!
https://github.com/ansible/ansible/blob/8b07d4616633ee7be190149eacb225d02fd2c810/lib/ansible/module_utils/basic.py#L2616-L2618

That's the source of the bug.  The fix would be (1) drop the shlex_quote, and (2) make sure to pass expand_user_and_vars=False to module.run_command().

(Wow what a collection of footguns module.run_command is!)
@mgedmin interesting find, I wasn't aware of that at all. I guess it has been there for ages and used to be needed for backwards compatibility, and now it's still there and nobody knows how many modules / playbooks rely on it in some way...
Bingo, the test is now working with that change, @mgedmin 

Resolving patch:
diff --git a/plugins/modules/source_control/git_config.py b/plugins/modules/source_control/git_config.py
--- a/plugins/modules/source_control/git_config.py
+++ b/plugins/modules/source_control/git_config.py
@@ -157,7 +157,6 @@
 import os
 
 from ansible.module_utils.basic import AnsibleModule
-from ansible.module_utils.six.moves import shlex_quote
 
 
 def main():
@@ -230,7 +229,7 @@ def main():
         # Run from root directory to avoid accidentally picking up any local config settings
         dir = "/"
 
-    (rc, out, err) = module.run_command(' '.join(args), cwd=dir)
+    (rc, out, err) = module.run_command(args, cwd=dir, expand_user_and_vars=False)
     if params['list_all'] and scope and rc == 128 and 'unable to read config file' in err:
         # This just means nothing has been set at the given scope
         module.exit_json(changed=False, msg='', config_values={})
@@ -259,15 +258,14 @@ def main():
             args.insert(len(args) - 1, "--" + unset)
             cmd = args
         else:
-            new_value_quoted = shlex_quote(new_value)
-            cmd = args + [new_value_quoted]
+            cmd = args + [new_value]
         try:  # try using extra parameter from ansible-base 2.10.4 onwards
-            (rc, out, err) = module.run_command(cmd, cwd=dir, ignore_invalid_cwd=False)
+            (rc, out, err) = module.run_command(cmd, cwd=dir, ignore_invalid_cwd=False, expand_user_and_vars=False)
         except TypeError:
             # @TODO remove try/except when community.general drop support for 2.10.x
             if not os.path.isdir(dir):
                 module.fail_json(msg="Cannot find directory '{0}'".format(dir))
-            (rc, out, err) = module.run_command(cmd, cwd=dir)
+            (rc, out, err) = module.run_command(cmd, cwd=dir, expand_user_and_vars=False)
         if err:
             module.fail_json(rc=rc, msg=err, cmd=cmd)
 


Test patch:
diff --git a/tests/integration/targets/git_config/tasks/main.yml b/tests/integration/targets/git_config/tasks/main.yml
--- a/tests/integration/targets/git_config/tasks/main.yml
+++ b/tests/integration/targets/git_config/tasks/main.yml
@@ -24,5 +24,7 @@
     - import_tasks: precedence_between_unset_and_value.yml
     # testing state=absent with check mode
     - import_tasks: unset_check_mode.yml
+    # testing for case in issue #1776
+    - import_tasks: set_value_with_tilde.yml
   when: git_installed is succeeded and git_version.stdout is version(git_version_supporting_includes, ">=")
 ...
diff --git a/tests/integration/targets/git_config/tasks/set_value_with_tilde.yml b/tests/integration/targets/git_config/tasks/set_value_with_tilde.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/git_config/tasks/set_value_with_tilde.yml
@@ -0,0 +1,33 @@
+---
+#- import_tasks: setup_no_value.yml
+
+- name: setting value
+  git_config:
+    name: core.hooksPath
+    value: '~/foo/bar'
+    state: present
+    scope: global
+  register: set_result
+
+- name: setting value again
+  git_config:
+    name: core.hooksPath
+    value: '~/foo/bar'
+    state: present
+    scope: global
+  register: set_result2
+
+- name: getting value
+  git_config:
+    name: core.hooksPath
+    scope: global
+  register: get_result
+
+- name: assert set changed and value is correct
+  assert:
+    that:
+      - set_result is changed
+      - set_result2 is not changed
+      - get_result is not changed
+      - get_result.config_value == '~/foo/bar'
+...