Repository: ansible/ansible

Issue:
Cannot set cache headers in s3 and aws_s3 module
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
s3 and aws_s3

##### ANSIBLE VERSION

```
ansible 2.4.0.0
  config file = None
  configured module search path = [u'/home/krl/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/krl/.virtualenvs/devops/local/lib/python2.7/site-packages/ansible
  executable location = /home/krl/.virtualenvs/devops/bin/ansible
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

```

##### CONFIGURATION
vanilla

##### OS / ENVIRONMENT
ubuntu 16.04

##### SUMMARY

Setting `Cache-Control` headers in  `metadata` will result in setting the metadata key: `aws_s3 x-amz-meta-cache-control`  on AWS. This makes it impossible to set the `Cache-Control` header on the s3 object. The `Content-Type` was set automatically before. Now it defaults to `binary/octet-stream`

##### STEPS TO REPRODUCE

Try to put a file to s3

```yaml
    - name: Upload images files
      s3:
        bucket: "bucket-name"
        object: "{{ item | basename }}"
        region: "{{ aws_region }}"
        src: "{{ item }}"
        mode: put
        permission: 'public-read'
        metadata: ''Cache-Control='max-age=3600'"
      with_fileglob:
        - "dist/*.png"
```

##### EXPECTED RESULTS
I expect the `Cache-Control` header to be set

##### ACTUAL RESULTS
 `aws_s3 x-amz-meta-cache-control` was set not `Cache-Control`


Cannot set cache headers in s3 and aws_s3 module
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
s3 and aws_s3

##### ANSIBLE VERSION

```
ansible 2.4.0.0
  config file = None
  configured module search path = [u'/home/krl/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/krl/.virtualenvs/devops/local/lib/python2.7/site-packages/ansible
  executable location = /home/krl/.virtualenvs/devops/bin/ansible
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

```

##### CONFIGURATION
vanilla

##### OS / ENVIRONMENT
ubuntu 16.04

##### SUMMARY

Setting `Cache-Control` headers in  `metadata` will result in setting the metadata key: `aws_s3 x-amz-meta-cache-control`  on AWS. This makes it impossible to set the `Cache-Control` header on the s3 object. The `Content-Type` was set automatically before. Now it defaults to `binary/octet-stream`

##### STEPS TO REPRODUCE

Try to put a file to s3

```yaml
    - name: Upload images files
      s3:
        bucket: "bucket-name"
        object: "{{ item | basename }}"
        region: "{{ aws_region }}"
        src: "{{ item }}"
        mode: put
        permission: 'public-read'
        metadata: ''Cache-Control='max-age=3600'"
      with_fileglob:
        - "dist/*.png"
```

##### EXPECTED RESULTS
I expect the `Cache-Control` header to be set

##### ACTUAL RESULTS
 `aws_s3 x-amz-meta-cache-control` was set not `Cache-Control`




Discussion:
cc @lwade
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
+1 Have the some bug here
cc @s-hertel
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
cc @lwade
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
+1 Have the some bug here
cc @s-hertel
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/aws_s3.py b/lib/ansible/modules/cloud/amazon/aws_s3.py
--- a/lib/ansible/modules/cloud/amazon/aws_s3.py
+++ b/lib/ansible/modules/cloud/amazon/aws_s3.py
@@ -237,6 +237,7 @@
 '''
 
 import os
+import mimetypes
 import traceback
 from ansible.module_utils.six.moves.urllib.parse import urlparse
 from ssl import SSLError
@@ -392,6 +393,21 @@ def path_check(path):
         return False
 
 
+def option_in_extra_args(option):
+    temp_option = option.replace('-', '').lower()
+
+    allowed_extra_args = {'acl': 'ACL', 'cachecontrol': 'CacheControl', 'contentdisposition': 'ContentDisposition',
+                          'contentencoding': 'ContentEncoding', 'contentlanguage': 'ContentLanguage',
+                          'contenttype': 'ContentType', 'expires': 'Expires', 'grantfullcontrol': 'GrantFullControl',
+                          'grantread': 'GrantRead', 'grantreadacp': 'GrantReadACP', 'grantwriteacp': 'GrantWriteACP',
+                          'metadata': 'Metadata', 'requestpayer': 'RequestPayer', 'serversideencryption': 'ServerSideEncryption',
+                          'storageclass': 'StorageClass', 'ssecustomeralgorithm': 'SSECustomerAlgorithm', 'ssecustomerkey': 'SSECustomerKey',
+                          'ssecustomerkeymd5': 'SSECustomerKeyMD5', 'ssekmskeyid': 'SSEKMSKeyId', 'websiteredirectlocation': 'WebsiteRedirectLocation'}
+
+    if temp_option in allowed_extra_args:
+        return allowed_extra_args[temp_option]
+
+
 def upload_s3file(module, s3, bucket, obj, src, expiry, metadata, encrypt, headers):
     if module.check_mode:
         module.exit_json(msg="PUT operation skipped - running in check mode", changed=True)
@@ -400,7 +416,23 @@ def upload_s3file(module, s3, bucket, obj, src, expiry, metadata, encrypt, heade
         if encrypt:
             extra['ServerSideEncryption'] = 'AES256'
         if metadata:
-            extra['Metadata'] = dict(metadata)
+            extra['Metadata'] = {}
+
+            # determine object metadata and extra arguments
+            for option in metadata:
+                extra_args_option = option_in_extra_args(option)
+                if extra_args_option is not None:
+                    extra[extra_args_option] = metadata[option]
+                else:
+                    extra['Metadata'][option] = metadata[option]
+
+        if 'ContentType' not in extra:
+            content_type = mimetypes.guess_type(src)[0]
+            if content_type is None:
+                # s3 default content type
+                content_type = 'binary/octet-stream'
+            extra['ContentType'] = content_type
+
         s3.upload_file(Filename=src, Bucket=bucket, Key=obj, ExtraArgs=extra)
         for acl in module.params.get('permission'):
             s3.put_object_acl(ACL=acl, Bucket=bucket, Key=obj)


Test patch: