Repository: ansible/ansible

Issue:
git module fails due to overly simplified pre-connection hostkey check
##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report
##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
git

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.4.0 (devel e9e661ebbb) last updated 2017/05/20 18:38:46 (GMT +200)
  config file = 
  configured module search path = [u'/home/andreas/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/andreas/.local/ansible/lib/ansible
  executable location = /home/andreas/.local/ansible/bin/ansible
  python version = 2.7.13 (default, Jan 19 2017, 14:48:08) [GCC 6.3.0 20170118]
```

##### CONFIGURATION

##### OS / ENVIRONMENT
Ubuntu 17.04

##### SUMMARY
When cloning a git repository by way of ssh the git module expects to find a matching hostkey entry in either _/etc/ssh/ssh_known_hosts_ or in _~/.ssh/known_hosts_. That is an overly simplified check, causing unnecessary fails, given that there are other ways for ssh to verify a host identity. Some of them being as follows.

1. ssh is configured to check for the hostkey in another file, by way of `UserKnownHostsFile`.
2. ssh verifies the host identity by of a host certificate and a ssh host CA.
3. By SSHFP entries in DNS, secured by DNSSEC. Relies on `VerifyHostKeyDNS` config.

##### STEPS TO REPRODUCE

1. Configure ssh to trust a host without an explicit hostkey entry in mentioned files. In this case by way of an ssh CA in _/etc/ssh/ssh_known_hosts_.

```
andreas@corrino:~$ head -n1 /etc/ssh/ssh_known_hosts
@cert-authority *.arrakis.se,*.shv25.se,*.krelln.net ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM73bJBKZo15popFUa4441gAlp+YQOZTVaQbHa9GZflD ca2015@arrakis
andreas@corrino:~$ ssh -v halleck.arrakis.se
...
debug1: Server host certificate: ssh-ed25519-cert-v01@openssh.com SHA256:DeOMELQB5YCGdDHPrilE06H4PRsKEF+oWFk3yQz1+4c, serial 0 ID "halleck" CA ssh-ed25519 SHA256:LlFDzYG+B3ZGerh7shM/R4m1oXPn7Ref9wpybdG03ZU valid from 2017-05-20T02:04:00 to 2017-06-03T02:05:08
debug1: Host 'halleck.arrakis.se' is known and matches the ED25519-CERT host certificate.
debug1: Found CA key in /etc/ssh/ssh_known_hosts:1
...
andreas@corrino:~$
```

2. Verify that a regular git clone works.

```
andreas@corrino:~$ git clone git@halleck.arrakis.se:rasble $(mktemp -d)
Cloning into '/tmp/tmp.LUQEPH4Qf8'...
....
andreas@corrino:~$
```

3. Locally try the same clone using the Ansible git module.

```yaml
---
- hosts: all
  tasks:
    - name: git ssh clone
      git:
        repo: git@halleck.arrakis.se:rasble
        dest: /home/andreas/.cache/ag/rasble
```

```
andreas@corrino:~/workdir/ag$ ansible-playbook -vvv --inventory=localhost, --connection=local ./ag.yaml
```
Resulting in the following failure.

```json
fatal: [localhost]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "accept_hostkey": false, 
            "archive": null, 
            "bare": false, 
            "clone": true, 
            "depth": null, 
            "dest": "/home/andreas/.cache/ag/rasble", 
            "executable": null, 
            "force": false, 
            "key_file": null, 
            "recursive": true, 
            "reference": null, 
            "refspec": null, 
            "remote": "origin", 
            "repo": "git@halleck.arrakis.se:rasble", 
            "ssh_opts": null, 
            "track_submodules": false, 
            "umask": null, 
            "update": true, 
            "verify_commit": false, 
            "version": "HEAD"
        }
    }, 
    "msg": "halleck.arrakis.se has an unknown hostkey. Set accept_hostkey to True or manually add the hostkey prior to running the git module"
}
```

##### EXPECTED RESULTS
For Ansible to trust that ssh has been configured correctly, and let ssh itself do the failing.

##### ACTUAL RESULTS
Ansible second-guessed ssh, causing an unnecessary failure.

##### OBSERVATIONS
This appear to be tied to the `add_git_host_key` function, defined in _module_utils/known_hosts.py_.




Discussion:
cc @robinro
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Thanks for this report @andreaso 

This is related to https://github.com/ansible/ansible/issues/20653 but more general.
Afaik the reason to handle hostkeys in ansible, instead of letting git/ssh deal with them, is to be able to produce reasonable error messages.
Parsing git/ssh stdout is hard and should be avoided where possible.

@willthames @abadger Since you recently worked on this: What's your opinion?
@robinro: Are you sure that this is related to #20653? That issue seem more about the scenario where `accept_hostkey` is set to True? So is not the case here.

While I agree that controlled error messages are a good thing, are they really more important than a correct behavior? Given all the ways which ssh can verify a host identify, is it really reasonable to expect that they can be checked ahead of time? How about instead take the following approach?

1. Only call the `add_git_host_key` function when `accept_hostkey`  is set to True.
2. Otherwise call git with the ssh option `BatchMode=yes`, and simply pass along the ssh error to Ansible.
#3253 seems to be another example where the same code path in the Git module introduce an unnecessary failure.
#20653 is not exactly the same codepath, but it would not have happened if the hostkey checking would be up to git/ssh instead of done by ansible.

One could also skip `add_git_host_key` when `accept_hostkey==True` and rely on `StrictHostKeyChecking=no` to do the right thing.

Feel free to open a PR with your suggested changes.
I guess it'll take quite some work to make sure all current usecases are covered, especially since there is barely no integration testing of the known_hosts setup.
Good points!

I'll start working on an initial pull request. Always easier to reason about something a bit more concrete.

Resolving patch:
diff --git a/lib/ansible/module_utils/known_hosts.py b/lib/ansible/module_utils/known_hosts.py
--- a/lib/ansible/module_utils/known_hosts.py
+++ b/lib/ansible/module_utils/known_hosts.py
@@ -43,26 +43,6 @@
 HASHED_KEY_MAGIC = "|1|"
 
 
-def add_git_host_key(module, url, accept_hostkey=True, create_dir=True):
-
-    """ idempotently add a git url hostkey """
-
-    if is_ssh_url(url):
-
-        fqdn, port = get_fqdn_and_port(url)
-
-        if fqdn:
-            known_host = check_hostkey(module, fqdn)
-            if not known_host:
-                if accept_hostkey:
-                    rc, out, err = add_host_key(module, fqdn, port=port, create_dir=create_dir)
-                    if rc != 0:
-                        module.fail_json(msg="failed to add %s hostkey: %s" % (fqdn, out + err))
-                else:
-                    module.fail_json(msg="%s has an unknown hostkey. Set accept_hostkey to True "
-                                     "or manually add the hostkey prior to running the git module" % fqdn)
-
-
 def is_ssh_url(url):
 
     """ check if url is ssh """
diff --git a/lib/ansible/modules/source_control/git.py b/lib/ansible/modules/source_control/git.py
--- a/lib/ansible/modules/source_control/git.py
+++ b/lib/ansible/modules/source_control/git.py
@@ -60,9 +60,8 @@
         choices: [ "yes", "no" ]
         version_added: "1.5"
         description:
-            - if C(yes), adds the hostkey for the repo url if not already
-              added. If ssh_opts contains "-o StrictHostKeyChecking=no",
-              this parameter is ignored.
+            - if C(yes), ensure that "-o StrictHostKeyChecking=no" is
+              present as an ssh options.
     ssh_opts:
         required: false
         default: None
@@ -281,7 +280,6 @@
 
 from ansible.module_utils.basic import AnsibleModule, get_module_path
 from ansible.module_utils.basic import get_exception
-from ansible.module_utils.known_hosts import add_git_host_key
 from ansible.module_utils.six import b, string_types
 from ansible.module_utils._text import to_native
 
@@ -368,6 +366,9 @@ def write_ssh_wrapper():
     BASEOPTS=$GIT_SSH_OPTS
 fi
 
+# Let ssh fail rather than prompt
+BASEOPTS="$BASEOPTS -o BatchMode=yes"
+
 if [ -z "$GIT_KEY" ]; then
     ssh $BASEOPTS "$@"
 else
@@ -777,15 +778,6 @@ def submodules_fetch(git_path, module, remote, track_submodules, dest):
             if not os.path.exists(os.path.join(dest, path, '.git')):
                 changed = True
 
-        # add the submodule repo's hostkey
-        if line.strip().startswith('url'):
-            repo = line.split('=', 1)[1].strip()
-            if module.params['ssh_opts'] is not None:
-                if "-o StrictHostKeyChecking=no" not in module.params['ssh_opts']:
-                    add_git_host_key(module, repo, accept_hostkey=module.params['accept_hostkey'])
-            else:
-                add_git_host_key(module, repo, accept_hostkey=module.params['accept_hostkey'])
-
     # Check for updates to existing modules
     if not changed:
         # Fetch updates
@@ -1031,6 +1023,13 @@ def main():
 
     result = dict(changed=False, warnings=list())
 
+    if module.params['accept_hostkey']:
+        if ssh_opts is not None:
+            if "-o StrictHostKeyChecking=no" not in ssh_opts:
+                ssh_opts += " -o StrictHostKeyChecking=no"
+        else:
+            ssh_opts = "-o StrictHostKeyChecking=no"
+
     # evaluate and set the umask before doing anything else
     if umask is not None:
         if not isinstance(umask, string_types):
@@ -1064,18 +1063,10 @@ def main():
     # create a wrapper script and export
     # GIT_SSH=<path> as an environment variable
     # for git to use the wrapper script
-    ssh_wrapper = None
-    if key_file or ssh_opts:
-        ssh_wrapper = write_ssh_wrapper()
-        set_git_ssh(ssh_wrapper, key_file, ssh_opts)
-        module.add_cleanup_file(path=ssh_wrapper)
-
-    # add the git repo's hostkey
-    if module.params['ssh_opts'] is not None:
-        if "-o StrictHostKeyChecking=no" not in module.params['ssh_opts']:
-            add_git_host_key(module, repo, accept_hostkey=module.params['accept_hostkey'])
-    else:
-        add_git_host_key(module, repo, accept_hostkey=module.params['accept_hostkey'])
+    ssh_wrapper = write_ssh_wrapper()
+    set_git_ssh(ssh_wrapper, key_file, ssh_opts)
+    module.add_cleanup_file(path=ssh_wrapper)
+
     git_version_used = git_version(git_path, module)
 
     if depth is not None and git_version_used < LooseVersion('1.9.1'):


Test patch:
diff --git a/test/integration/targets/git/tasks/missing_hostkey.yml b/test/integration/targets/git/tasks/missing_hostkey.yml
--- a/test/integration/targets/git/tasks/missing_hostkey.yml
+++ b/test/integration/targets/git/tasks/missing_hostkey.yml
@@ -16,7 +16,6 @@
 - assert:
     that:
       - 'git_result.failed'
-      - 'git_result.msg == "github.com has an unknown hostkey. Set accept_hostkey to True or manually add the hostkey prior to running the git module"'
 
 - name: checkout git@github.com repo with accept_hostkey (expected pass)
   git: