Repository: ansible-collections/amazon.aws

Issue:
aws_s3 Folder Creation issue on S3 Buckets with ACLs disabled
### Summary

When I try to create a folder using aws_s3 on an S3 bucket with ACLs disabled, I get an error `UnboundLocalError: local variable 'object_acl' referenced before assignment. `

### Issue Type

Bug Report

### Component Name

aws_s3

### Ansible Version

```console (paste below)
$ ansible --version
Ansible Full Version:  6.4.0
Ansible-Core Version:  ansible [core 2.13.4]
  config file = /opt/ansible/ansible.cfg
  configured module search path = ['/home/ec2-user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ec2-user/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /opt/ansible/ansible_collections
  executable location = /home/ec2-user/.local/bin/ansible
  python version = 3.9.7 (default, Sep 13 2021, 08:18:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
  jinja version = 3.0.2
  libyaml = True
```


### Collection Versions

```console (paste below)
$ ansible-galaxy collection list
Collection                    Version
----------------------------- -------
amazon.aws                    3.4.0
ansible.netcommon             3.1.1
ansible.posix                 1.4.0
ansible.utils                 2.6.1
ansible.windows               1.11.1
arista.eos                    5.0.1
awx.awx                       21.5.0
azure.azcollection            1.13.0
check_point.mgmt              2.3.0
chocolatey.chocolatey         1.3.0
cisco.aci                     2.2.0
cisco.asa                     3.1.0
cisco.dnac                    6.6.0
cisco.intersight              1.0.19
cisco.ios                     3.3.1
cisco.iosxr                   3.3.1
cisco.ise                     2.5.3
cisco.meraki                  2.11.0
cisco.mso                     2.0.0
cisco.nso                     1.0.3
cisco.nxos                    3.1.1
cisco.ucs                     1.8.0
cloud.common                  2.1.2
cloudscale_ch.cloud           2.2.2
community.aws                 3.5.0
community.azure               1.1.0
community.ciscosmb            1.0.5
community.crypto              2.5.0
community.digitalocean        1.21.0
community.dns                 2.3.2
community.docker              2.7.1
community.fortios             1.0.0
community.general             5.6.0
community.google              1.0.0
community.grafana             1.5.2
community.hashi_vault         3.2.0
community.hrobot              1.5.2
community.kubernetes          *
community.libvirt             1.2.0
community.mongodb             1.4.2
community.mysql               3.5.1
community.network             4.0.1
community.okd                 2.2.0
community.postgresql          2.2.0
community.proxysql            1.4.0
community.rabbitmq            1.2.2
community.routeros            2.3.0
community.sap                 1.0.0
community.sap_libs            1.3.0
community.skydive             1.0.0
community.sops                1.4.0
community.vmware              2.9.1
community.windows             1.11.0
community.zabbix              1.8.0
containers.podman             1.9.4
cyberark.conjur               1.2.0
cyberark.pas                  1.0.14
dellemc.enterprise_sonic      1.1.2
dellemc.openmanage            5.5.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.19.0
fortinet.fortimanager         2.1.5
fortinet.fortios              2.1.7
frr.frr                       2.0.0
gluster.gluster               1.0.2
google.cloud                  1.0.2
hetzner.hcloud                1.8.2
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.9.0
infinidat.infinibox           1.3.3
infoblox.nios_modules         1.3.0
inspur.ispim                  1.0.1
inspur.sm                     2.0.0
junipernetworks.junos         3.1.0
kubernetes.core               2.3.2
mellanox.onyx                 1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.19.0
netapp.elementsw              21.7.0
netapp.ontap                  21.23.0
netapp.storagegrid            21.11.0
netapp.um_info                21.8.0
netapp_eseries.santricity     1.3.1
netbox.netbox                 3.7.1
ngine_io.cloudstack           2.2.4
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.2
openstack.cloud               1.9.1
openvswitch.openvswitch       2.1.0
ovirt.ovirt                   2.2.3
purestorage.flasharray        1.13.0
purestorage.flashblade        1.10.0
purestorage.fusion            1.1.0
sensu.sensu_go                1.13.1
servicenow.servicenow         1.0.6
splunk.es                     2.1.0
t_systems_mms.icinga_director 1.31.0
theforeman.foreman            3.6.0
vmware.vmware_rest            2.2.0
vultr.cloud                   1.1.0
vyos.vyos                     3.0.1
wti.remote                    1.0.4

# /opt/ansible/ansible_collections
Collection       Version
---------------- -------
amazon.aws       5.0.1
ansible.posix    1.4.0
community.aws    5.0.0
community.crypto 2.7.0
```


### AWS SDK versions

```console (paste below)
$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.24.80
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/ec2-user/.local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.27.80
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/ec2-user/.local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed
COLLECTIONS_PATHS(/opt/ansible/ansible.cfg) = ['/opt/ansible/ansible_collections']
DEFAULT_FORKS(/opt/ansible/ansible.cfg) = 50
DEFAULT_HOST_LIST(/opt/ansible/ansible.cfg) = ['/opt/ansible/inventory/dynamic']
DEFAULT_LOG_PATH(/opt/ansible/ansible.cfg) = /var/log/ansible.log
DEFAULT_PRIVATE_KEY_FILE(/opt/ansible/ansible.cfg) = /opt/ansible/keys/ansible.pem
DEFAULT_REMOTE_USER(/opt/ansible/ansible.cfg) = ansible
DEFAULT_ROLES_PATH(/opt/ansible/ansible.cfg) = ['/opt/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/opt/ansible/ansible.cfg) = yaml
HOST_KEY_CHECKING(/opt/ansible/ansible.cfg) = False
 ```


### OS / Environment

Red Hat Enterprise Linux release 8.6 (Ootpa)

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
-- Play:
# Create S3 Folders
- name: create s3 folders for 1st SFTP ID - /jrglynn2
  delegate_to: localhost
  environment:
    AWS_PROFILE: "ansible-devops"
  amazon.aws.aws_s3:
    bucket: bucket_acls_disabled
    object: "{{ item }}"
    mode: create
    permission: bucket-owner-full-control
    purge_tags: false
  loop:
    - "/jrglynn2"
```


### Expected Results

I expect the folder `jrglynn2` to be created on S3 Bucket `bucket_acls_disabled`

### Actual Results

```console (paste below)
-- Verbose Error (run with -vvv)
task path: /opt/ansible/roles/sftp_transfer_config/tasks/main.yml:6
redirecting (type: modules) amazon.aws.aws_s3 to amazon.aws.s3_object
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: ec2-user
<localhost> EXEC /bin/sh -c 'echo ~ec2-user && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ec2-user/.ansible/tmp `"&& mkdir "` echo /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921 `" && echo ansible-tmp-1665089009.6957664-129681-268809806524921="` echo /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921 `" ) && sleep 0'
redirecting (type: modules) amazon.aws.aws_s3 to amazon.aws.s3_object
Using module file /opt/ansible/ansible_collections/amazon/aws/plugins/modules/s3_object.py
<localhost> PUT /home/ec2-user/.ansible/tmp/ansible-local-129638dpd3owbt/tmpcwwanxip TO /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/ /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py && sleep 0'
<localhost> EXEC /bin/sh -c 'AWS_PROFILE=ansible-devops /usr/bin/python3 /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/ > /dev/null 2>&1 && sleep 0'
redirecting (type: modules) amazon.aws.aws_s3 to amazon.aws.s3_object
The full traceback is:
Traceback (most recent call last):
  File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 107, in <module>
    _ansiballz_main()
  File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
  File "/usr/lib64/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1276, in <module>
  File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1221, in main
UnboundLocalError: local variable 'object_acl' referenced before assignment
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
<localhost> EXEC /bin/sh -c 'echo ~ec2-user && sleep 0'
failed: [asd01-srv-jrglynn2 -> localhost] (item=/jrglynn2) => changed=false
  ansible_loop_var: item
  item: /jrglynn2
  module_stderr: |-
    Traceback (most recent call last):
      File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 107, in <module>
        _ansiballz_main()
      File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 47, in invoke_module
        runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
      File "/usr/lib64/python3.9/runpy.py", line 210, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1276, in <module>
      File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1221, in main
    UnboundLocalError: local variable 'object_acl' referenced before assignment
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1
 
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Resolving patch:
diff --git a/plugins/modules/s3_object.py b/plugins/modules/s3_object.py
--- a/plugins/modules/s3_object.py
+++ b/plugins/modules/s3_object.py
@@ -396,6 +396,7 @@
 import base64
 import time
 
+
 try:
     import botocore
 except ImportError:
@@ -607,6 +608,8 @@ def create_dirkey(module, s3, bucket, obj, encrypt, expiry):
             s3.put_object_acl(ACL=acl, Bucket=bucket, Key=obj)
     except is_boto3_error_code(IGNORE_S3_DROP_IN_EXCEPTIONS):
         module.warn("PutObjectAcl is not implemented by your storage provider. Set the permissions parameters to the empty list to avoid this warning")
+    except is_boto3_error_code('AccessControlListNotSupported'):
+        module.warn("PutObjectAcl operation : The bucket does not allow ACLs.")
     except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e:  # pylint: disable=duplicate-except
         module.fail_json_aws(e, msg="Failed while creating object %s." % obj)
 
@@ -696,6 +699,8 @@ def upload_s3file(module, s3, bucket, obj, expiry, metadata, encrypt, headers, s
                 s3.put_object_acl(ACL=acl, Bucket=bucket, Key=obj)
         except is_boto3_error_code(IGNORE_S3_DROP_IN_EXCEPTIONS):
             module.warn("PutObjectAcl is not implemented by your storage provider. Set the permission parameters to the empty list to avoid this warning")
+        except is_boto3_error_code('AccessControlListNotSupported'):
+            module.warn("PutObjectAcl operation : The bucket does not allow ACLs.")
         except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e:  # pylint: disable=duplicate-except
             module.fail_json_aws(e, msg="Unable to set object ACL")
 
@@ -831,6 +836,8 @@ def copy_object_to_bucket(module, s3, bucket, obj, encrypt, metadata, validate,
             module.exit_json(msg="Object copied from bucket %s to bucket %s." % (bucketsrc['Bucket'], bucket), tags=tags, changed=True)
     except is_boto3_error_code(IGNORE_S3_DROP_IN_EXCEPTIONS):
         module.warn("PutObjectAcl is not implemented by your storage provider. Set the permissions parameters to the empty list to avoid this warning")
+    except is_boto3_error_code('AccessControlListNotSupported'):
+        module.warn("PutObjectAcl operation : The bucket does not allow ACLs.")
     except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e:  # pylint: disable=duplicate-except
         module.fail_json_aws(e, msg="Failed while copying object %s from bucket %s." % (obj, module.params['copy_src'].get('Bucket')))
 
@@ -1081,6 +1088,7 @@ def main():
     # check if bucket exists, if yes, check if ACL is disabled
     acl_disabled = False
     exists = bucket_check(module, s3, bucket)
+
     if exists:
         try:
             ownership_controls = s3.get_bucket_ownership_controls(Bucket=bucket)['OwnershipControls']
@@ -1217,15 +1225,17 @@ def main():
                 if key_check(module, s3, bucket, dirobj):
                     module.exit_json(msg="Bucket %s and key %s already exists." % (bucket, obj), changed=False)
                 else:
-                    # setting valid object acls for the create_dirkey function
-                    module.params['permission'] = object_acl
+                    if not acl_disabled:
+                        # setting valid object acls for the create_dirkey function
+                        module.params['permission'] = object_acl
                     create_dirkey(module, s3, bucket, dirobj, encrypt, expiry)
             else:
                 # only use valid bucket acls for the create_bucket function
                 module.params['permission'] = bucket_acl
                 create_bucket(module, s3, bucket, location)
-                # only use valid object acls for the create_dirkey function
-                module.params['permission'] = object_acl
+                if not acl_disabled:
+                    # only use valid object acls for the create_dirkey function
+                    module.params['permission'] = object_acl
                 create_dirkey(module, s3, bucket, dirobj, encrypt, expiry)
 
     # Support for grabbing the time-expired URL for an object in S3/Walrus.
@@ -1265,8 +1275,9 @@ def main():
             # only use valid bucket acls for create_bucket function
             module.params['permission'] = bucket_acl
             create_bucket(module, s3, bucket, location)
-        # only use valid object acls for the copy operation
-        module.params['permission'] = object_acl
+        if not acl_disabled:
+            # only use valid object acls for the copy operation
+            module.params['permission'] = object_acl
         copy_object_to_bucket(module, s3, bucket, obj, encrypt, metadata, validate, d_etag)
 
     module.exit_json(failed=False)


Test patch:
diff --git a/tests/integration/targets/s3_object/tasks/copy_object_acl_disabled_bucket.yml b/tests/integration/targets/s3_object/tasks/copy_object_acl_disabled_bucket.yml
--- a/tests/integration/targets/s3_object/tasks/copy_object_acl_disabled_bucket.yml
+++ b/tests/integration/targets/s3_object/tasks/copy_object_acl_disabled_bucket.yml
@@ -85,17 +85,39 @@
           - upload_file_result.msg != "PUT operation complete"
           - '"s3:PutObject" not in upload_file_result.resource_actions'
 
+    - name: Create an object in the bucket with permissions (permission not set)
+      amazon.aws.s3_object:
+        bucket: "{{ bucket_name }}-acl-disabled"
+        object: "/test_directory"
+        permission: bucket-owner-full-control
+        mode: create
+      register: permission_result
+
+    - assert:
+        that:
+          - permission_result is changed
+          - upload_file_result is not failed
+          - '"PutObjectAcl operation : The bucket does not allow ACLs." in permission_result.warnings'
+          - '"Virtual directory test_directory/ created" in permission_result.msg'
+
   always:
 
     - name: Delete the file in the bucket
       amazon.aws.s3_object:
         bucket: "{{ bucket_name }}-acl-disabled"
-        src: "{{ remote_tmp_dir }}/acl_disabled_upload_test.txt"
-        object: "acl_disabled_upload_test.txt"
+        object: "{{ item }}"
         mode: delobj
       retries: 3
       delay: 3
       ignore_errors: true
+      loop:
+        - "acl_disabled_upload_test.txt"
+        - "test_directory/"
+
+    - name: List keys simple
+      amazon.aws.s3_object:
+        bucket: "{{ bucket_name }}-acl-disabled"
+        mode: list
 
     - name: Delete bucket created in this test
       s3_bucket: