Repository: ansible/ansible

Issue:
URI module - pass JSON body without setting Content-Type header to 'application/json'
##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
uri module

##### ANSIBLE VERSION
```
$ ansible --version
ansible 2.3.0 (devel c1f5432cc0) last updated 2017/01/09 14:36:42 (GMT +100)
  config file = /Users/nbailey/automation/roles-linux/ambari_server/tests/ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
`hash_behaviour=merge` is the only even potentially relevant one.

##### OS / ENVIRONMENT
n/a

##### SUMMARY

I've hit an issue a couple of times where an API wants JSON formatted data but a Content-Type header set to 'application/x-www-form-urlencoded'.

I can't find a convenient way to pass a JSON body to the URI module, but avoid setting the header to `application/json`.

If I set `body_format` to `raw`, I can set the body to a string that contains valid JSON... But if I use a lookup against a Jinja template, the module gets handed a YAML hash instead, and I get an error ("An unknown error occurred: sendall() argument 1 must be string or buffer, not list").

##### STEPS TO REPRODUCE

I'm trying to interact with the Ambari Server API (https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md), doing the equivalent of this curl:

```
$ curl --user admin:admin -H 'X-Requested-By: ambari' -X POST -d '[{"Event": {"specs": [{"names": "<redacted>,<redacted>", "principal_type": "groups", "sync_type": "specific"}]}}]' http://192.168.2.20:8080/api/v1/ldap_sync_events
{
  "resources" : [
    {
      "href" : "http://192.168.2.20:8080/api/v1/ldap_sync_events/6",
      "Event" : {
        "id" : 6
      }
    }
  ]
```
Task using `body_format: json`:

```
- name: Synchronise LDAP groups and users.
  uri:
    url: "{{ ambari_server.endpoint | default(inventory_hostname) }}/api/v1/ldap_sync_events"
    headers:
      X-Requested-By: "ambari"
      Content-Type: "application/x-www-form-urlencoded"
    method: POST
    user: "{{ ambari_server.admin_user | default('admin') }}"
    password: "{{ ambari_server.admin_password | default('admin') }}"
    force_basic_auth: True
    body_format: raw
    body: "{{ lookup('template', 'ldap_sync.json.j2') }}"
    status_code: 201
    return_content: True
  when: ambari_server is defined and ambari_server.ldap is defined
```

This passes the JSON blob happily, but the server doesn't like it. 

Response: "Status code was not [201]: HTTP Error 500: java.lang.IllegalStateException: Expected a string but was BEGIN_ARRAY at line 1 column 2

Running the task with `body_format: raw` to avoid getting `application/json` set gives me an Ansible-side error: "Status code was not [201]: An unknown error occurred: sendall() argument 1 must be string or buffer, not list". 

I don't understand how Ansible formats parameters before handing them off to modules very well. I think what's happening here is that the module errors on getting a YAML hash rather than a string. The `body_format: json` parameter expects JSON and uses `json.dumps()` on that parameter before passing it on to whatever makes the HTTP request, but the module code that handles things with a `raw` format does not. 

What does work? I don't know if this is a good or widely useful solution, but I just set the uri module to only set a Content-Type header if there wasn't already one:

```
    # grabbed from https://github.com/ansible/ansible-modules-core/blob/devel/network/basics/uri.py#L398
    # commit d749ec91e513c40a50b539a7eaccf29c31dda4d4
    if body_format == 'json':
        # Encode the body unless its a string, then assume it is pre-formatted JSON
        if not isinstance(body, basestring):
            body = json.dumps(body)
        if 'Content-Type' not in dict_headers:
            dict_headers['Content-Type'] = 'application/json'
```

Is this is a useful default behaviour for the module? If so, I'm happy to put a PR in for it.

If not, is there another way I can achieve the same effect? 


Discussion:
cc @romeotheriault
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/network/basics/uri.py b/lib/ansible/modules/network/basics/uri.py
--- a/lib/ansible/modules/network/basics/uri.py
+++ b/lib/ansible/modules/network/basics/uri.py
@@ -65,6 +65,8 @@
     description:
       - The serialization format of the body. When set to json, encodes the
         body argument, if needed, and automatically sets the Content-Type header accordingly.
+        As of C(2.3) it is possible to override the `Content-Type` header, when
+        set to json via the I(headers) option.
     required: false
     choices: [ "raw", "json" ]
     default: raw
@@ -134,7 +136,9 @@
     default: null
   headers:
     description:
-        - Add custom HTTP headers to a request in the format of a YAML hash
+        - Add custom HTTP headers to a request in the format of a YAML hash. As
+          of C(2.3) supplying C(Content-Type) here will override the header
+          generated by supplying C(json) for I(body_format).
     required: false
     default: null
     version_added: '2.1'
@@ -399,7 +403,9 @@ def main():
         # Encode the body unless its a string, then assume it is pre-formatted JSON
         if not isinstance(body, basestring):
             body = json.dumps(body)
-        dict_headers['Content-Type'] = 'application/json'
+        lower_header_keys = [key.lower() for key in dict_headers]
+        if 'content-type' not in lower_header_keys:
+             dict_headers['Content-Type'] = 'application/json'
 
     # Grab all the http headers. Need this hack since passing multi-values is
     # currently a bit ugly. (e.g. headers='{"Content-Type":"application/json"}')


Test patch:
diff --git a/test/integration/targets/uri/tasks/main.yml b/test/integration/targets/uri/tasks/main.yml
--- a/test/integration/targets/uri/tasks/main.yml
+++ b/test/integration/targets/uri/tasks/main.yml
@@ -280,3 +280,16 @@
     status_code: 202
     method: POST
     body: foo
+
+- name: Validate body_format json does not override content-type in 2.3 or newer
+  uri:
+    url: "https://{{ httpbin_host }}/post"
+    method: POST
+    body:
+      foo: bar
+    body_format: json
+    headers:
+      'Content-Type': 'text/json'
+    return_content: true
+  register: result
+  failed_when: result.json.headers['Content-Type'] != 'text/json'