Repository: ansible/ansible

Issue:
S3 delete non-existent bucket fails
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report


##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
 - S3 module

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.3.1.0
  config file = 
  configured module search path = Default w/o overrides
  python version = 2.7.10 (default, Jul 14 2015, 19:46:27) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
Ansible config file:
```
[defaults]
host_key_checking = False
remote_user = centos
callback_whitelist = profile_tasks, human_log
callback_plugins = callback_plugins
lookup_plugins = lookup_plugins
log_path = log/ansible.log
retry_files_enabled = False
hostfile = inventory/ec2.py
timeout = 30

[ssh_connection]
ssh_args = -F ./ssh.cfg -o ForwardAgent=yes -o UserKnownHostsFile=/dev/null
control_path = none
pipelining = True
```

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->
Playbook launched on OS X 10.0.5, Darwin Kernel Version 14.5.0

Remote EC2 instance CentOS6 (not relevant for this bug)

##### SUMMARY
<!--- Explain the problem briefly -->

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
If an S3 bucket has already been deleted or does not exist, running the S3 module with the "delete" mode on it will cause the module to fail with a stack trace. This occurs even with the "ignore_nonexistent_bucket" parameter set. Earlier versions of the S3 module did not fail in this case.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- name: foo
  hosts: localhost
  connection: local
  tasks:
      - name: delete nonexistent bucket
        s3:
            bucket: "some-bucket-that-does-not-exist"
            mode: "delete"
            ignore_nonexistent_bucket: "True"
```

A possible fix is to extend the "ignore_nonexistent_bucket" parameter handling as follows around line 557:

```python
    if not ignore_nonexistent_bucket:
        validate = True
        if mode not in ('create', 'put') and not bucketrtn:
            module.fail_json(msg="Source bucket cannot be found.")
    else:
        if mode == 'delete' and not bucketrtn:
            module.exit_json(msg="Bucket not found for deletion", changed=False)
        validate = False
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
If "ignore_nonexistent_bucket" is set, I expect the S3 task to succeed on deletion even if the bucket does not exist. If the "ignore_nonexistent_bucket" is not set, I expect the S3 task to fail on deletion.

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
The S3 module fails with a stack trace.

<!--- Paste verbatim command output between quotes below -->
```
fatal: [localhost]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/mk/zyr99n9j3dg3tlqp__8xqp_0kbzv15/T/ansible_15ONhL/ansible_module_s3.py\", line 765, in <module>\n    main()\n  File \"/var/folders/mk/zyr99n9j3dg3tlqp__8xqp_0kbzv15/T/ansible_15ONhL/ansible_module_s3.py\", line 651, in main\n    deletertn = delete_bucket(module, s3, bucket)\n  File \"/var/folders/mk/zyr99n9j3dg3tlqp__8xqp_0kbzv15/T/ansible_15ONhL/ansible_module_s3.py\", line 340, in delete_bucket\n    bucket_contents = bucket.list()\nAttributeError: 'NoneType' object has no attribute 'list'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
```

Here's the stack trace in a readable form:
```
The full traceback is:
Traceback (most recent call last):
  File "/var/folders/mk/zyr99n9j3dg3tlqp__8xqp_0kbzv15/T/ansible_15ONhL/ansible_module_s3.py", line 765, in <module>
    main()
  File "/var/folders/mk/zyr99n9j3dg3tlqp__8xqp_0kbzv15/T/ansible_15ONhL/ansible_module_s3.py", line 651, in main
    deletertn = delete_bucket(module, s3, bucket)
  File "/var/folders/mk/zyr99n9j3dg3tlqp__8xqp_0kbzv15/T/ansible_15ONhL/ansible_module_s3.py", line 340, in delete_bucket
    bucket_contents = bucket.list()
AttributeError: 'NoneType' object has no attribute 'list'
```



Discussion:

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/s3.py b/lib/ansible/modules/cloud/amazon/s3.py
--- a/lib/ansible/modules/cloud/amazon/s3.py
+++ b/lib/ansible/modules/cloud/amazon/s3.py
@@ -350,13 +350,32 @@ def delete_bucket(module, s3, bucket):
     if module.check_mode:
         module.exit_json(msg="DELETE operation skipped - running in check mode", changed=True)
     try:
-        bucket = s3.lookup(bucket)
+        bucket = s3.lookup(bucket, validate=False)
         bucket_contents = bucket.list()
         bucket.delete_keys([key.name for key in bucket_contents])
+    except s3.provider.storage_response_error as e:
+        if e.status == 404:
+            # bucket doesn't appear to exist
+            return False
+        elif e.status == 403:
+            # bucket appears to exist but user doesn't have list bucket permission; may still be able to delete bucket
+            pass
+        else:
+            module.fail_json(msg=str(e), exception=traceback.format_exc())
+    try:
         bucket.delete()
         return True
     except s3.provider.storage_response_error as e:
-        module.fail_json(msg= str(e))
+        if e.status == 403:
+            module.exit_json(msg="Unable to complete DELETE operation. Check you have have s3:DeleteBucket "
+                             "permission. Error: {0}.".format(e.message),
+                             exception=traceback.format_exc())
+        elif e.status == 409:
+            module.exit_json(msg="Unable to complete DELETE operation. It appears there are contents in the "
+                             "bucket that you don't have permission to delete. Error: {0}.".format(e.message),
+                             exception=traceback.format_exc())
+        else:
+            module.fail_json(msg=str(e), exception=traceback.format_exc())
 
 def delete_key(module, s3, bucket, obj, validate=True):
     if module.check_mode:
@@ -676,8 +695,8 @@ def main():
     if mode == 'delete':
         if bucket:
             deletertn = delete_bucket(module, s3, bucket)
-            if deletertn is True:
-                module.exit_json(msg="Bucket %s and all keys have been deleted."%bucket, changed=True)
+            message = "Bucket {0} and all keys have been deleted.".format(bucket)
+            module.exit_json(msg=message, changed=deletertn)
         else:
             module.fail_json(msg="Bucket parameter is required.")
 


Test patch:
diff --git a/test/integration/targets/s3/aliases b/test/integration/targets/aws_s3/aliases
similarity index 100%
rename from test/integration/targets/s3/aliases
rename to test/integration/targets/aws_s3/aliases
diff --git a/test/integration/targets/s3/defaults/main.yml b/test/integration/targets/aws_s3/defaults/main.yml
similarity index 100%
rename from test/integration/targets/s3/defaults/main.yml
rename to test/integration/targets/aws_s3/defaults/main.yml
diff --git a/test/integration/targets/s3/meta/main.yml b/test/integration/targets/aws_s3/meta/main.yml
similarity index 100%
rename from test/integration/targets/s3/meta/main.yml
rename to test/integration/targets/aws_s3/meta/main.yml
diff --git a/test/integration/targets/s3/tasks/main.yml b/test/integration/targets/aws_s3/tasks/main.yml
similarity index 94%
rename from test/integration/targets/s3/tasks/main.yml
rename to test/integration/targets/aws_s3/tasks/main.yml
--- a/test/integration/targets/s3/tasks/main.yml
+++ b/test/integration/targets/aws_s3/tasks/main.yml
@@ -215,3 +215,16 @@
     that:
       - result.changed == True
 # ============================================================
+- name: test delete a nonexistent bucket
+  s3:
+    bucket: "{{ bucket.stdout + '.bucket' }}"
+    mode: delete
+    security_token: "{{security_token}}"
+    aws_access_key: "{{ ec2_access_key }}"
+    aws_secret_key: "{{ ec2_secret_key }}"
+  register: result
+- name: assert that changed is False
+  assert:
+    that:
+      - result.changed == False
+# ============================================================