Repository: ansible/ansible

Issue:
aws_s3 no longer supports async since ansible 2.5
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
aws_s3

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/jostle/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
ANSIBLE_PIPELINING(/etc/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/etc/ansible/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=60s
DEFAULT_GATHERING(/etc/ansible/ansible.cfg) = explicit
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False


##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
N/A

##### SUMMARY
<!--- Explain the problem briefly -->
All the plays that we have that use aws_s3 module and use the async function are failing since we upgraded to Ansible 2.5.2

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->
$ ansible-playbook hello.yml 

PLAY [localhost] **************************************************************************************************************************************************************************************************************************************************************

TASK [Test aws_s3] ************************************************************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "async is not supported for this task."}


<!--- Paste example playbooks or commands between quotes below -->
```
- hosts: localhost
  connection: local
  gather_facts: False

  tasks:
  - name: Test aws_s3
    aws_s3:
      bucket: example-bucket
      region: us-west-2
      object: example.file
      src: example.file
      mode: put
    async: 720
    poll: 5

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
$ ansible-playbook -v hello.yml 
Using /etc/ansible/ansible.cfg as config file

PLAY [localhost] **************************************************************************************************************************************************************************************************************************************************************

TASK [Test aws_s3] ************************************************************************************************************************************************************************************************************************************************************
changed: [localhost] => {"ansible_job_id": "515962900981.18057", "changed": true, "failed": false, "finished": 1, "msg": "PUT operation complete", "url": "xxxxxxxx"}

PLAY RECAP ********************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
TASK [Test aws_s3] ************************************************************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "async is not supported for this task."}
<!--- Paste verbatim command output between quotes below -->
```

```

aws_s3 no longer supports async since ansible 2.5
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
aws_s3

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/jostle/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
ANSIBLE_PIPELINING(/etc/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/etc/ansible/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=60s
DEFAULT_GATHERING(/etc/ansible/ansible.cfg) = explicit
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False


##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
N/A

##### SUMMARY
<!--- Explain the problem briefly -->
All the plays that we have that use aws_s3 module and use the async function are failing since we upgraded to Ansible 2.5.2

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->
$ ansible-playbook hello.yml 

PLAY [localhost] **************************************************************************************************************************************************************************************************************************************************************

TASK [Test aws_s3] ************************************************************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "async is not supported for this task."}


<!--- Paste example playbooks or commands between quotes below -->
```
- hosts: localhost
  connection: local
  gather_facts: False

  tasks:
  - name: Test aws_s3
    aws_s3:
      bucket: example-bucket
      region: us-west-2
      object: example.file
      src: example.file
      mode: put
    async: 720
    poll: 5

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
$ ansible-playbook -v hello.yml 
Using /etc/ansible/ansible.cfg as config file

PLAY [localhost] **************************************************************************************************************************************************************************************************************************************************************

TASK [Test aws_s3] ************************************************************************************************************************************************************************************************************************************************************
changed: [localhost] => {"ansible_job_id": "515962900981.18057", "changed": true, "failed": false, "finished": 1, "msg": "PUT operation complete", "url": "xxxxxxxx"}

PLAY RECAP ********************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
TASK [Test aws_s3] ************************************************************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "async is not supported for this task."}
<!--- Paste verbatim command output between quotes below -->
```

```



Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/aws_s3.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/aws_s3.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @lwade @s-hertel
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/aws_s3.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/aws_s3.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @lwade @s-hertel
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/plugins/action/aws_s3.py b/lib/ansible/plugins/action/aws_s3.py
--- a/lib/ansible/plugins/action/aws_s3.py
+++ b/lib/ansible/plugins/action/aws_s3.py
@@ -23,6 +23,7 @@
 from ansible.errors import AnsibleError, AnsibleAction, AnsibleActionFail, AnsibleFileNotFound
 from ansible.module_utils._text import to_text
 from ansible.plugins.action import ActionBase
+from ansible.utils.vars import merge_hash
 
 
 class ActionModule(ActionBase):
@@ -31,6 +32,8 @@ class ActionModule(ActionBase):
 
     def run(self, tmp=None, task_vars=None):
         ''' handler for aws_s3 operations '''
+        self._supports_async = True
+
         if task_vars is None:
             task_vars = dict()
 
@@ -55,8 +58,14 @@ def run(self, tmp=None, task_vars=None):
                     except AnsibleError as e:
                         raise AnsibleActionFail(to_text(e))
 
-            # execute the aws_s3 module now, with the updated args
-            result.update(self._execute_module(module_args=new_module_args, task_vars=task_vars))
+            wrap_async = self._task.async_val and not self._connection.has_native_async
+            # execute the aws_s3 module with the updated args
+            result = merge_hash(result, self._execute_module(module_args=new_module_args, task_vars=task_vars, wrap_async=wrap_async))
+
+            if not wrap_async:
+                # remove a temporary path we created
+                self._remove_tmp_path(self._connection._shell.tmpdir)
+
         except AnsibleAction as e:
             result.update(e.result)
         return result


Test patch:
diff --git a/test/integration/targets/aws_s3/tasks/main.yml b/test/integration/targets/aws_s3/tasks/main.yml
--- a/test/integration/targets/aws_s3/tasks/main.yml
+++ b/test/integration/targets/aws_s3/tasks/main.yml
@@ -79,6 +79,23 @@
           - result.changed == True
           - result.msg == "PUT operation complete"
 
+    - name: test using aws_s3 with async
+      aws_s3:
+        bucket: "{{ bucket_name }}"
+        mode: put
+        src: "{{ tmp1.path }}"
+        object: delete.txt
+        <<: *aws_connection_info
+      register: test_async
+      async: 30
+      poll: 0
+    - name: ensure it completed
+      async_status:
+        jid: "{{ test_async.ansible_job_id }}"
+      register: status
+      until: status.finished
+      retries: 10
+
     - name: check that roles file lookups work as expected
       aws_s3:
         bucket: "{{ bucket_name }}"