Repository: ansible-collections/amazon.aws

Issue:
s3_object in copy mode cannot change metadata
### Summary

The [documentation](https://docs.ansible.com/ansible/latest/collections/amazon/aws/s3_object_module.html#parameter-metadata) suggests that metadata can be set when `mode=copy` but this doesn't appear to be the case. I cannot get metadata to be changed on the copy whatever I do.

This appears to be because [the module](https://github.com/ansible-collections/amazon.aws/blob/275b736c0050439f3bde40266f146c67cc15145c/plugins/modules/s3_object.py#L1317) doesn't override (or provide the option to override) the `MetadataDirective` parameter, which defaults to `COPY` (search for it in the [boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/copy_object.html)).

I first observed this trying to copy an object while modifying its metadata (not shown here, but the copy always has the same metadata as its source). I suspect the reason is that objects have system defined metadata called `Content-Type` (visible in the console), which in essence is copied due to the directive, but this is speculation on my part.

Additionally, please consider updating the examples which show `metadata` as a comma separated string while the documentation lists it as a dict (which works as shown below).

### Issue Type

Bug Report

### Component Name

s3_object

### Ansible Version

```console (paste below)
$ ansible --version
ansible [core 2.16.3]
  config file = /home/andrei/work/tools/andrei/playbooks/ansible.cfg
  configured module search path = ['/home/andrei/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /home/andrei/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.7 (main, Jan 29 2024, 16:03:57) [GCC 13.2.1 20230801] (/usr/bin/python)
  jinja version = 3.1.3
  libyaml = True
```


### Collection Versions

```console (paste below)
$ ansible-galaxy collection list

# /home/andrei/.ansible/collections/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               7.2.0  
ansible.netcommon                        4.1.0  
ansible.posix                            1.5.1  
ansible.utils                            3.0.0  
cloud.common                             2.1.3  
community.aws                            7.1.0  
community.crypto                         2.10.0 
community.docker                         3.4.0  
community.general                        7.4.0  
community.postgresql                     3.3.0  
community.routeros                       2.5.0  
kubernetes.core                          2.4.0  

# /usr/lib/python3.11/site-packages/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               7.2.0  
ansible.netcommon                        5.3.0  
ansible.posix                            1.5.4  
ansible.utils                            2.12.0 
ansible.windows                          2.2.0  
arista.eos                               6.2.2  
awx.awx                                  23.6.0 
azure.azcollection                       1.19.0 
check_point.mgmt                         5.2.2  
chocolatey.chocolatey                    1.5.1  
cisco.aci                                2.8.0  
cisco.asa                                4.0.3  
cisco.dnac                               6.10.2 
cisco.intersight                         2.0.7  
cisco.ios                                5.3.0  
cisco.iosxr                              6.1.1  
cisco.ise                                2.7.0  
cisco.meraki                             2.17.2 
cisco.mso                                2.5.0  
cisco.nxos                               5.3.0  
cisco.ucs                                1.10.0 
cloud.common                             2.1.4  
cloudscale_ch.cloud                      2.3.1  
community.aws                            7.1.0  
community.azure                          2.0.0  
community.ciscosmb                       1.0.7  
community.crypto                         2.17.1 
community.digitalocean                   1.26.0 
community.dns                            2.8.0  
community.docker                         3.7.0  
community.general                        8.3.0  
community.grafana                        1.7.0  
community.hashi_vault                    6.1.0  
community.hrobot                         1.9.0  
community.library_inventory_filtering_v1 1.0.0  
community.libvirt                        1.3.0  
community.mongodb                        1.6.3  
community.mysql                          3.8.0  
community.network                        5.0.2  
community.okd                            2.3.0  
community.postgresql                     3.3.0  
community.proxysql                       1.5.1  
community.rabbitmq                       1.2.3  
community.routeros                       2.12.0 
community.sap                            2.0.0  
community.sap_libs                       1.4.2  
community.sops                           1.6.7  
community.vmware                         4.1.0  
community.windows                        2.1.0  
community.zabbix                         2.3.1  
containers.podman                        1.11.0 
cyberark.conjur                          1.2.2  
cyberark.pas                             1.0.25 
dellemc.enterprise_sonic                 2.4.0  
dellemc.openmanage                       8.7.0  
dellemc.powerflex                        2.1.0  
dellemc.unity                            1.7.1  
f5networks.f5_modules                    1.27.1 
fortinet.fortimanager                    2.3.1  
fortinet.fortios                         2.3.4  
frr.frr                                  2.0.2  
gluster.gluster                          1.0.2  
google.cloud                             1.3.0  
grafana.grafana                          2.2.4  
hetzner.hcloud                           2.4.1  
hpe.nimble                               1.1.4  
ibm.qradar                               2.1.0  
ibm.spectrum_virtualize                  2.0.0  
ibm.storage_virtualize                   2.2.0  
infinidat.infinibox                      1.3.12 
infoblox.nios_modules                    1.6.1  
inspur.ispim                             2.2.0  
inspur.sm                                2.3.0  
junipernetworks.junos                    5.3.1  
kubernetes.core                          2.4.0  
lowlydba.sqlserver                       2.2.2  
microsoft.ad                             1.4.1  
netapp.aws                               21.7.1 
netapp.azure                             21.10.1
netapp.cloudmanager                      21.22.1
netapp.elementsw                         21.7.0 
netapp.ontap                             22.9.0 
netapp.storagegrid                       21.11.1
netapp.um_info                           21.8.1 
netapp_eseries.santricity                1.4.0  
netbox.netbox                            3.16.0 
ngine_io.cloudstack                      2.3.0  
ngine_io.exoscale                        1.1.0  
openstack.cloud                          2.2.0  
openvswitch.openvswitch                  2.1.1  
ovirt.ovirt                              3.2.0  
purestorage.flasharray                   1.26.0 
purestorage.flashblade                   1.15.0 
purestorage.fusion                       1.6.0  
sensu.sensu_go                           1.14.0 
splunk.es                                2.1.2  
t_systems_mms.icinga_director            2.0.1  
telekom_mms.icinga_director              1.35.0 
theforeman.foreman                       3.15.0 
vmware.vmware_rest                       2.3.1  
vultr.cloud                              1.12.1 
vyos.vyos                                4.1.0  
wti.remote                               1.0.5
```


### AWS SDK versions

```console (paste below)
$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.34.34
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.34.34
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
```


### Configuration

```
$ ansible-config dump --only-changed
CONFIG_FILE() = /home/andrei/work/tools/andrei/playbooks/ansible.cfg
DEFAULT_HOST_LIST(/home/andrei/work/tools/andrei/playbooks/ansible.cfg) = ...
DEFAULT_ROLES_PATH(/home/andrei/work/tools/andrei/playbooks/ansible.cfg) = ...
EDITOR(env: EDITOR) = vim
INVENTORY_ENABLED(/home/andrei/work/tools/andrei/playbooks/ansible.cfg) = ['ini', 'constructed', 'script']
PAGER(env: PAGER) = less
```


### OS / Environment

Arch Linux

### Steps to Reproduce

```yaml (paste below)
- name: Test playbook
  hosts: localhost
  gather_facts: false
  tasks:
    - name: Create test bucket
      amazon.aws.s3_bucket:
        name: ansiblemetatest
        state: present

    - name: Create test object
      amazon.aws.s3_object:
        bucket: ansiblemetatest
        object: nometa
        mode: put
        content: "some content"

    - name: Copy and add metadata
      amazon.aws.s3_object:
        bucket: ansiblemetatest
        object: "metacopy"
        mode: copy
        copy_src:
          bucket: ansiblemetatest
          object: nometa
        metadata:
          something: exists
          version: "1.0.2"

    - name: Create test object with metadata
      amazon.aws.s3_object:
        bucket: ansiblemetatest
        object: withmeta
        mode: put
        content: "some content"
        metadata:
          something: exists
          version: "1.0.2"

    - name: Get object info
      amazon.aws.s3_object_info:
        bucket_name: ansiblemetatest
        object_name: "{{ item }}"
      loop:
        - nometa
        - metacopy
        - withmeta
      register: obj_info

    - debug:
        var: obj_info.results
```


### Expected Results

`metacopy` should have its metadata set, like `withmeta` does.

### Actual Results

```console (paste below)
PLAY [Test playbook] **********************************************************************************************************

TASK [Create test bucket] *****************************************************************************************************
changed: [localhost]

TASK [Create test object] *****************************************************************************************************
changed: [localhost]

TASK [Copy and add metadata] **************************************************************************************************
[WARNING]: PutObjectAcl operation : The bucket does not allow ACLs.
changed: [localhost]

TASK [Create test object with metadata] ***************************************************************************************
changed: [localhost]

TASK [Get object info] ********************************************************************************************************
ok: [localhost] => (item=nometa)
ok: [localhost] => (item=metacopy)
ok: [localhost] => (item=withmeta)

TASK [debug] ******************************************************************************************************************
ok: [localhost] => {
    "obj_info.results": [
        {
            "ansible_loop_var": "item",
            "changed": false,
            "failed": false,
            "invocation": {
                "module_args": {
                    "access_key": null,
                    "aws_ca_bundle": null,
                    "aws_config": null,
                    "bucket_name": "ansiblemetatest",
                    "ceph": false,
                    "debug_botocore_endpoint_logs": false,
                    "dualstack": false,
                    "endpoint_url": null,
                    "object_details": null,
                    "object_name": "nometa",
                    "profile": "sandbox",
                    "region": null,
                    "secret_key": null,
                    "session_token": null,
                    "validate_certs": true
                }
            },
            "item": "nometa",
            "object_info": [
                {
                    "object_data": {
                        "accept_ranges": "bytes",
                        "content_length": 12,
                        "content_type": "binary/octet-stream",
                        "e_tag": "\"9893532233caff98cd083a116b013c0b\"",
                        "last_modified": "2024-02-23T07:51:24+00:00",
                        "metadata": {},
                        "server_side_encryption": "AES256"
                    }
                }
            ]
        },
        {
            "ansible_loop_var": "item",
            "changed": false,
            "failed": false,
            "invocation": {
                "module_args": {
                    "access_key": null,
                    "aws_ca_bundle": null,
                    "aws_config": null,
                    "bucket_name": "ansiblemetatest",
                    "ceph": false,
                    "debug_botocore_endpoint_logs": false,
                    "dualstack": false,
                    "endpoint_url": null,
                    "object_details": null,
                    "object_name": "metacopy",
                    "profile": "sandbox",
                    "region": null,
                    "secret_key": null,
                    "session_token": null,
                    "validate_certs": true
                }
            },
            "item": "metacopy",
            "object_info": [
                {
                    "object_data": {
                        "accept_ranges": "bytes",
                        "content_length": 12,
                        "content_type": "binary/octet-stream",
                        "e_tag": "\"9893532233caff98cd083a116b013c0b\"",
                        "last_modified": "2024-02-23T07:51:26+00:00",
                        "metadata": {},
                        "server_side_encryption": "AES256"
                    }
                }
            ]
        },
        {
            "ansible_loop_var": "item",
            "changed": false,
            "failed": false,
            "invocation": {
                "module_args": {
                    "access_key": null,
                    "aws_ca_bundle": null,
                    "aws_config": null,
                    "bucket_name": "ansiblemetatest",
                    "ceph": false,
                    "debug_botocore_endpoint_logs": false,
                    "dualstack": false,
                    "endpoint_url": null,
                    "object_details": null,
                    "object_name": "withmeta",
                    "profile": "sandbox",
                    "region": null,
                    "secret_key": null,
                    "session_token": null,
                    "validate_certs": true
                }
            },
            "item": "withmeta",
            "object_info": [
                {
                    "object_data": {
                        "accept_ranges": "bytes",
                        "content_length": 12,
                        "content_type": "binary/octet-stream",
                        "e_tag": "\"9893532233caff98cd083a116b013c0b\"",
                        "last_modified": "2024-02-23T07:51:28+00:00",
                        "metadata": {
                            "something": "exists",
                            "version": "1.0.2"
                        },
                        "server_side_encryption": "AES256"
                    }
                }
            ]
        }
    ]
}

PLAY RECAP ********************************************************************************************************************
localhost                  : ok=6    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:

Resolving patch:
diff --git a/plugins/modules/s3_object.py b/plugins/modules/s3_object.py
--- a/plugins/modules/s3_object.py
+++ b/plugins/modules/s3_object.py
@@ -315,7 +315,9 @@
     object: /my/desired/key.txt
     src: /usr/local/myfile.txt
     mode: put
-    metadata: 'Content-Encoding=gzip,Cache-Control=no-cache'
+    metadata:
+      Content-Encoding: gzip
+      Cache-Control: no-cache
 
 - name: PUT/upload with custom headers
   amazon.aws.s3_object:
@@ -1314,6 +1316,11 @@ def copy_object_to_bucket(module, s3, bucket, obj, encrypt, metadata, validate,
                     metadata,
                 )
             )
+            if metadata:
+                # 'MetadataDirective' Specifies whether the metadata is copied from the source object or replaced
+                # with metadata that's provided in the request. The default value is 'COPY', therefore when user
+                # specifies a metadata we should set it to 'REPLACE'
+                params.update({"MetadataDirective": "REPLACE"})
             s3.copy_object(aws_retry=True, **params)
             put_object_acl(module, s3, bucket, obj)
             # Tags


Test patch:
diff --git a/tests/integration/targets/s3_object/aliases b/tests/integration/targets/s3_object/aliases
--- a/tests/integration/targets/s3_object/aliases
+++ b/tests/integration/targets/s3_object/aliases
@@ -1,3 +1,4 @@
 cloud/aws
 aws_s3
 s3_object_info
+time=12m
diff --git a/tests/integration/targets/s3_object/tasks/copy_object.yml b/tests/integration/targets/s3_object/tasks/copy_object.yml
--- a/tests/integration/targets/s3_object/tasks/copy_object.yml
+++ b/tests/integration/targets/s3_object/tasks/copy_object.yml
@@ -1,5 +1,12 @@
 ---
-- block:
+- vars:
+    withmeta_data:
+      something: exists
+      version: "1.0.2"
+    metacopy_data:
+      name: metacopy
+      version: "1.0.3"
+  block:
     - name: define bucket name used for tests
       ansible.builtin.set_fact:
         copy_bucket:
@@ -142,6 +149,68 @@
           - result is not changed
           - result.msg == "Key this_key_does_not_exist.txt does not exist in bucket "+copy_bucket.src+"."
 
+    # Copy with metadata
+    - name: Set fact for bucket name
+      ansible.builtin.set_fact:
+        bucket_name: "{{ copy_bucket.dst }}"
+
+    - name: Create test bucket
+      amazon.aws.s3_bucket:
+        name: "{{ bucket_name }}"
+        state: present
+
+    - name: Create test object
+      amazon.aws.s3_object:
+        bucket: "{{ bucket_name }}"
+        object: nometa
+        mode: put
+        content: "some content"
+
+    - name: Copy and add metadata
+      amazon.aws.s3_object:
+        bucket: "{{ bucket_name }}"
+        object: metacopy
+        mode: copy
+        copy_src:
+          bucket: "{{ bucket_name }}"
+          object: nometa
+        metadata: "{{ metacopy_data }}"
+
+    - name: Create test object with metadata
+      amazon.aws.s3_object:
+        bucket: "{{ bucket_name }}"
+        object: withmeta
+        mode: put
+        content: "another content"
+        metadata: "{{ withmeta_data }}"
+
+    - name: Copy and preserve metadata
+      amazon.aws.s3_object:
+        bucket: "{{ bucket_name }}"
+        object: copywithmeta
+        mode: copy
+        copy_src:
+          bucket: "{{ bucket_name }}"
+          object: withmeta
+
+    - name: Get objects info
+      amazon.aws.s3_object_info:
+        bucket_name: "{{ bucket_name }}"
+        object_name: "{{ item }}"
+      loop:
+        - nometa
+        - metacopy
+        - withmeta
+        - copywithmeta
+      register: obj_info
+
+    - assert:
+        that:
+          - obj_info.results | selectattr('item', 'equalto', 'nometa') | map(attribute='object_info.0.object_data.metadata') | first == {}
+          - obj_info.results | selectattr('item', 'equalto', 'withmeta') | map(attribute='object_info.0.object_data.metadata') | first == withmeta_data
+          - obj_info.results | selectattr('item', 'equalto', 'metacopy') | map(attribute='object_info.0.object_data.metadata') | first == metacopy_data
+          - obj_info.results | selectattr('item', 'equalto', 'copywithmeta') | map(attribute='object_info.0.object_data.metadata') | first == withmeta_data
+
   always:
     - ansible.builtin.include_tasks: delete_bucket.yml
       with_items:
diff --git a/tests/integration/targets/s3_object/tasks/main.yml b/tests/integration/targets/s3_object/tasks/main.yml
--- a/tests/integration/targets/s3_object/tasks/main.yml
+++ b/tests/integration/targets/s3_object/tasks/main.yml
@@ -837,8 +837,6 @@
         that:
           - binary_files.results[0].stat.checksum == binary_files.results[1].stat.checksum
 
-    - ansible.builtin.include_tasks: copy_object.yml
-    - ansible.builtin.include_tasks: copy_object_acl_disabled_bucket.yml
     - name: Run tagging tests
       block:
         # ============================================================
@@ -1074,6 +1072,8 @@
               - (result.tags | length) == 0
 
         - ansible.builtin.include_tasks: copy_recursively.yml
+        - ansible.builtin.include_tasks: copy_object.yml
+        - ansible.builtin.include_tasks: copy_object_acl_disabled_bucket.yml
   always:
     - name: delete temporary files
       file: