Repository: ansible/ansible

Issue:
Digest authentication mechanism doesn't work for uri module
<!--- Verify first that your issue/request is not already reported in GitHub -->
##### ISSUE TYPE

<!--- Pick one below and delete the rest: -->
- Bug Report
##### ANSIBLE VERSION

<!--- Paste verbatim output from “ansible --version” between quotes below -->

```
ansible 2.1.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```
##### CONFIGURATION

<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

default
##### OS / ENVIRONMENT

<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->

ansible host:

```
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"
```

target host: 

```
dockerized - Debian GNU/Linux 8 (jessie)
```
##### SUMMARY

Digest authentication mechanism doesn't work for [uri module](https://github.com/ansible/ansible-modules-core/blob/devel/network/basics/uri.py)
##### STEPS TO REPRODUCE

<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

gerrit-user.json

```
    {
    "name": "Gerrit User",
    "email": "john.doe@example.com",
    "ssh_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD3KFWWQvwOJXgTsS9UOsjO4eT9I1mZ+KV0Fo3b8wLmO4SyCwpFSIB14Yn9BkvCZJbjJ6jwrd49rPw1jmAvNkRO38zrq64V+7Pjo5FYGsV1JlqLVIt2td27QHUGocqd/VxmSxDvMy29Y1XBummzl2KN+4LfClKdfISZeWHzRuLENxdP446bvIkZopwKfL+PGQEMGy96h9f2DH7UrwJ+r0XANlmSOpqrgwtU8H5zanveUQLYk4+WdlAmbRu44Lgwz3tSZL5ux0OCk9z+W7TuNkgvuT+nIRg/5bAi46nq9mnV8qAyHTpH0C2qeqtH0QjJk796I+AxOJGPIZx6jfmTRWMp gerrit@example.com",
    "http_password": "aaaaa",
    "groups": [
      "Non-Interactive Users"
    ]
  }
```

For now work around is to use `curl`:

```
curl -X PUT --digest --user admin:secret --data-binary @gerrit-user.json --header "Content-Type: application/json" http://localhost:8080/a/accounts/john
```

Use `uri` module instead of `curl`

<!--- Paste example playbooks or commands between quotes below -->

```
# ansible playbook snip:
- uri:
    url: http://localhost:8080/a/accounts/john
    method: PUT
    user: admin
    password: secret
    body: "{{ lookup('file','gerrit-user.json') }}"
    status_code: 201
    body_format: json
    validate_certs: no
    headers:
      Content-Type: application/json
```

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

Digest Auth works as expected
##### ACTUAL RESULTS

<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->

```
TASK [gerrit : uri] ************************************************************
task path: /home/sergk/projects/ansible/roles/gerrit/tasks/manage.yml:13
<ci-gerrit> ESTABLISH DOCKER CONNECTION FOR USER: root
<ci-gerrit> EXEC ['/usr/bin/docker', 'exec', '-u', u'root', '-i', u'ci-gerrit', u'/bin/sh', '-c', u'/bin/sh -c \'( umask 77 && mkdir -p "` echo /tmp/.ansible/ansible-tmp-1470233480.82-148065190795595 `" && echo ansible-tmp-1470233480.82-148065190795595="` echo /tmp/.ansible/ansible-tmp-1470233480.82-148065190795595 `" ) && sleep 0\'']
<ci-gerrit> PUT /tmp/tmpHR_lO5 TO /tmp/.ansible/ansible-tmp-1470233480.82-148065190795595/uri
<ci-gerrit> EXEC ['/usr/bin/docker', 'exec', '-u', u'root', '-i', u'ci-gerrit', u'/bin/sh', '-c', u'/bin/sh -c \'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/env python /tmp/.ansible/ansible-tmp-1470233480.82-148065190795595/uri; rm -rf "/tmp/.ansible/ansible-tmp-1470233480.82-148065190795595/" > /dev/null 2>&1 && sleep 0\'']
fatal: [ci-gerrit]: FAILED! => {"cache_control": "no-cache, no-store, max-age=0, must-revalidate", "changed": false, "connection": "close", "content": "Unauthorized", "content_length": "12", "content_type": "text/plain; charset=ISO-8859-1", "date": "Wed, 03 Aug 2016 14:11:21 GMT", "expires": "Fri, 01 Jan 1990 00:00:00 GMT", "failed": true, "invocation": {"module_args": {"backup": null, "body": {"email": "john.doe@example.com", "groups": ["Non-Interactive Users"], "http_password": "aaaaa", "name": "Gerrit User", "ssh_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQD3KFWWQvwOJXgTsS9UOsjO4eT9I1mZ+KV0Fo3b8wLmO4SyCwpFSIB14Yn9BkvCZJbjJ6jwrd49rPw1jmAvNkRO38zrq64V+7Pjo5FYGsV1JlqLVIt2td27QHUGocqd/VxmSxDvMy29Y1XBummzl2KN+4LfClKdfISZeWHzRuLENxdP446bvIkZopwKfL+PGQEMGy96h9f2DH7UrwJ+r0XANlmSOpqrgwtU8H5zanveUQLYk4+WdlAmbRu44Lgwz3tSZL5ux0OCk9z+W7TuNkgvuT+nIRg/5bAi46nq9mnV8qAyHTpH0C2qeqtH0QjJk796I+AxOJGPIZx6jfmTRWMp gerrit@example.com"}, "body_format": "json", "content": null, "creates": null, "delimiter": null, "dest": null, "directory_mode": null, "follow": false, "follow_redirects": "safe", "force": false, "force_basic_auth": false, "group": null, "headers": {"Content-Type": "application/json"}, "http_agent": "ansible-httpget", "method": "PUT", "mode": null, "owner": null, "password": "secret", "regexp": null, "remote_src": null, "removes": null, "return_content": false, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": null, "status_code": ["201"], "timeout": 30, "url": "http://localhost:8080/a/accounts/john", "url_password": "secret", "url_username": "admin", "use_proxy": true, "user": "admin", "validate_certs": false}, "module_name": "uri"}, "msg": "Status code was not [201]: HTTP Error 401: Unauthorized", "pragma": "no-cache", "redirected": false, "status": 401, "url": "http://localhost:8080/a/accounts/john", "www_authenticate": "Digest realm=\"Gerrit Code Review\", domain=\"http://localhost:8080/\", qop=\"auth\", nonce=\"SRKqsFiUYVh8DNURJ28iG5o49b17WDobRQV4DQ==$\""}

PLAY RECAP *********************************************************************
ci-gerrit              : ok=24   changed=15   unreachable=0    failed=1

```

It seems that we don't have Digest Auth support in [urls.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/urls.py)



Discussion:
+1 This issue is very annoying
Why is it marked as "feature_idea" and not a bug? The documentation at http://docs.ansible.com/ansible/uri_module.html states that uri module support Digest authentication, which is clearly not true.

Resolving patch:
diff --git a/lib/ansible/module_utils/urls.py b/lib/ansible/module_utils/urls.py
--- a/lib/ansible/module_utils/urls.py
+++ b/lib/ansible/module_utils/urls.py
@@ -819,9 +819,11 @@ def open_url(url, data=None, headers=None, method=None, use_proxy=True,
             # use this username/password combination for  urls
             # for which `theurl` is a super-url
             authhandler = urllib_request.HTTPBasicAuthHandler(passman)
+            digest_authhandler = urllib_request.HTTPDigestAuthHandler(passman)
 
             # create the AuthHandler
             handlers.append(authhandler)
+            handlers.append(digest_authhandler)
 
         elif username and force_basic_auth:
             headers["Authorization"] = basic_auth_header(username, password)


Test patch:
diff --git a/test/integration/targets/uri/tasks/main.yml b/test/integration/targets/uri/tasks/main.yml
--- a/test/integration/targets/uri/tasks/main.yml
+++ b/test/integration/targets/uri/tasks/main.yml
@@ -175,6 +175,13 @@
     user: user
     password: passwd
 
+- name: test digest auth
+  uri:
+    url: 'http://{{ httpbin_host }}/digest-auth/auth/user/passwd'
+    user: user
+    password: passwd
+    HEADER_Cookie: "fake=fake_value"
+
 - name: test PUT
   uri:
     url: 'http://{{ httpbin_host }}/put'