Repository: ansible/ansible-modules-extras

Issue:
s3_bucket versioning parameter is required
##### ISSUE TYPE
- Bug Report
##### COMPONENT NAME

s3_bucket
##### ANSIBLE VERSION

<!--- Paste verbatim output from “ansible --version” between quotes below -->

```
ansible 2.1.0.0
  config file =
  configured module search path = Default w/o overrides
```
##### CONFIGURATION

<!---

-->

NONE
##### OS / ENVIRONMENT

<!---

-->

Mac OS X 10.11.6
##### SUMMARY

<!--- Explain the problem briefly -->

When running a play to perform a change other than changing the versioning of the bucket, Ansible will assume that versioning needs to be turned off (default is no).  In all documentation, this parameter is marked as not required.  But if I perform an iteration over many s3 buckets to add something as simple as a tag, some of which need to have versioning enabled (for Cross Region Replication), it will suspend versioning on the buckets that have versioning enabled.  
##### STEPS TO REPRODUCE

<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->

```

---

- hosts: localhost
  tasks:
  - name: Tag S3 buckets
    s3_bucket: {name: "{{ item }}", tags:{Application: "S3"}}
    with_items:
      - policy-test-rwetzelberger
```

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

<!--- What did you expect to happen when running the steps above? -->

I expect to see that the s3 bucket named "policy-test-rwetzelberger" has a tag verified to have or added called "S3".  This bucket had versioning enabled.  Versioning that is enabled should not be suspended.
##### ACTUAL RESULTS

<!--- What actually happened? If possible run with high verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->

```
ansible-playbook -vvvv s3-tag.yml
No config file found; using defaults
Loaded callback default of type stdout, v2.0

PLAYBOOK: s3-tag.yml ***********************************************************
1 plays in s3-tag.yml

PLAY [localhost] ***************************************************************

TASK [setup] *******************************************************************
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: <username>
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1471356881.14-189485368299932 `" && echo ansible-tmp-1471356881.14-189485368299932="` echo $HOME/.ansible/tmp/ansible-tmp-1471356881.14-189485368299932 `" ) && sleep 0'
<127.0.0.1> PUT /var/folders/m2/5vh7dqn90gz834mv6swzzdzmh8qf0x/T/tmpiE06ET TO /Users/<username>/.ansible/tmp/ansible-tmp-1471356881.14-189485368299932/setup
<127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /Users/<username>/.ansible/tmp/ansible-tmp-1471356881.14-189485368299932/setup; rm -rf "/Users/<username>/.ansible/tmp/ansible-tmp-1471356881.14-189485368299932/" > /dev/null 2>&1 && sleep 0'
ok: [localhost]

TASK [Tag S3 buckets] **********************************************************
task path: /Users/<username>/fork-ansible-playbooks/Gio/tasks/s3-tag.yml:5
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: <username>
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1471356885.59-45084579463130 `" && echo ansible-tmp-1471356885.59-45084579463130="` echo $HOME/.ansible/tmp/ansible-tmp-1471356885.59-45084579463130 `" ) && sleep 0'
<127.0.0.1> PUT /var/folders/m2/5vh7dqn90gz834mv6swzzdzmh8qf0x/T/tmpVvlVdL TO /Users/<username>/.ansible/tmp/ansible-tmp-1471356885.59-45084579463130/s3_bucket
<127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /Users/<username>/.ansible/tmp/ansible-tmp-1471356885.59-45084579463130/s3_bucket; rm -rf "/Users/<username>/.ansible/tmp/ansible-tmp-1471356885.59-45084579463130/" > /dev/null 2>&1 && sleep 0'
changed: [localhost] => (item=policy-test-rwetzelberger) => {"changed": true, "invocation": {"module_args": {"aws_access_key": null, "aws_secret_key": null, "ec2_url": null, "force": false, "name": "policy-test-rwetzelberger", "policy": null, "profile": null, "region": null, "requester_pays": false, "s3_url": null, "security_token": null, "state": "present", "tags": {"Application": "S3"}, "validate_certs": true, "versioning": false}, "module_name": "s3_bucket"}, "item": "policy-test-rwetzelberger", "name": "policy-test-rwetzelberger", "policy": null, "requester_pays": false, "tags": {"Application": "S3"}, "versioning": {"MfaDelete": "Disabled", "Versioning": "Suspended"}}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0
```



Discussion:
<!-- @ryanwetzelberger: Thanks for filing this bug report! -->

@wimnat ping, this issue is waiting for your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_bug_new --->

This also will wipe out any defined policy on an existing bucket.  This should not be modifying resources on an existing bucket if the parameter is undefined.  

@wimnat, ping. This issue is still waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_renotify_maintainer --->


Resolving patch:
diff --git a/cloud/amazon/s3_bucket.py b/cloud/amazon/s3_bucket.py
--- a/cloud/amazon/s3_bucket.py
+++ b/cloud/amazon/s3_bucket.py
@@ -68,7 +68,7 @@
     description:
       - Whether versioning is enabled or disabled (note that once versioning is enabled, it can only be suspended)
     required: false
-    default: no
+    default: null
     choices: [ 'yes', 'no' ]
 extends_documentation_fragment:
     - aws
@@ -122,6 +122,7 @@
 except ImportError:
     HAS_BOTO = False
 
+
 def get_request_payment_status(bucket):
 
     response = bucket.get_request_payment()
@@ -131,6 +132,7 @@ def get_request_payment_status(bucket):
 
     return (payer != "BucketOwner")
 
+
 def create_tags_container(tags):
 
     tag_set = TagSet()
@@ -141,6 +143,7 @@ def create_tags_container(tags):
     tags_obj.add_tag_set(tag_set)
     return tags_obj
 
+
 def _create_or_update_bucket(connection, module, location):
 
     policy = module.params.get("policy")
@@ -161,22 +164,22 @@ def _create_or_update_bucket(connection, module, location):
 
     # Versioning
     versioning_status = bucket.get_versioning_status()
-    if not versioning_status:
-        if versioning:
-            try:
-                bucket.configure_versioning(versioning)
-                changed = True
-                versioning_status = bucket.get_versioning_status()
-            except S3ResponseError as e:
-                module.fail_json(msg=e.message)
-    elif versioning_status['Versioning'] == "Enabled" and not versioning:
-        bucket.configure_versioning(versioning)
-        changed = True
-        versioning_status = bucket.get_versioning_status()
-    elif ( (versioning_status['Versioning'] == "Disabled" and versioning) or (versioning_status['Versioning'] == "Suspended" and versioning) ):
-        bucket.configure_versioning(versioning)
-        changed = True
-        versioning_status = bucket.get_versioning_status()
+    if versioning_status:
+        if versioning is not None:
+            if versioning and versioning_status['Versioning'] != "Enabled":
+                try:
+                    bucket.configure_versioning(versioning)
+                    changed = True
+                    versioning_status = bucket.get_versioning_status()
+                except S3ResponseError as e:
+                    module.fail_json(msg=e.message)
+            elif not versioning and versioning_status['Versioning'] != "Enabled":
+                try:
+                    bucket.configure_versioning(versioning)
+                    changed = True
+                    versioning_status = bucket.get_versioning_status()
+                except S3ResponseError as e:
+                    module.fail_json(msg=e.message)
 
     # Requester pays
     requester_pays_status = get_request_payment_status(bucket)
@@ -198,27 +201,26 @@ def _create_or_update_bucket(connection, module, location):
         else:
             module.fail_json(msg=e.message)
 
-    if policy is not None:
-        compare_policy = json.loads(policy)
-
-        if current_policy is None or json.loads(current_policy) != compare_policy:
+    if current_policy is not None:
+        if policy == {}:
             try:
-                bucket.set_policy(policy)
+                bucket.delete_policy()
                 changed = True
                 current_policy = bucket.get_policy()
             except S3ResponseError as e:
-                module.fail_json(msg=e.message)
-    elif current_policy is not None:
+                if e.error_code == "NoSuchBucketPolicy":
+                    current_policy = None
+                else:
+                    module.fail_json(msg=e.message)
+        if policy is not None:
+            if json.loads(current_policy) != json.loads(policy):
+                try:
+                    bucket.set_policy(policy)
+                    changed = True
+                    current_policy = bucket.get_policy()
+                except S3ResponseError as e:
+                    module.fail_json(msg=e.message)
 
-        try:
-            bucket.delete_policy()
-            changed = True
-            current_policy = bucket.get_policy()
-        except S3ResponseError as e:
-            if e.error_code == "NoSuchBucketPolicy":
-                current_policy = None
-            else:
-                module.fail_json(msg=e.message)
     # Tags
     try:
         current_tags = bucket.get_tags()
@@ -228,13 +230,12 @@ def _create_or_update_bucket(connection, module, location):
         else:
             module.fail_json(msg=e.message)
 
-    if current_tags is not None or tags is not None:
-
-        if current_tags is None:
-            current_tags_dict = {}
-        else:
-            current_tags_dict = dict((t.key, t.value) for t in current_tags[0])
+    if current_tags is None:
+        current_tags_dict = {}
+    else:
+        current_tags_dict = dict((t.key, t.value) for t in current_tags[0])
 
+    if tags is not None:
         if current_tags_dict != tags:
             try:
                 if tags:
@@ -248,6 +249,7 @@ def _create_or_update_bucket(connection, module, location):
 
     module.exit_json(changed=changed, name=bucket.name, versioning=versioning_status, requester_pays=requester_pays_status, policy=current_policy, tags=current_tags_dict)
 
+
 def _destroy_bucket(connection, module):
 
     force = module.params.get("force")
@@ -280,6 +282,7 @@ def _destroy_bucket(connection, module):
 
     module.exit_json(changed=changed)
 
+
 def _create_or_update_bucket_ceph(connection, module, location):
     #TODO: add update
 
@@ -301,22 +304,26 @@ def _create_or_update_bucket_ceph(connection, module, location):
     else:
         module.fail_json(msg='Unable to create bucket, no error from the API')
 
+
 def _destroy_bucket_ceph(connection, module):
 
     _destroy_bucket(connection, module)
 
+
 def create_or_update_bucket(connection, module, location, flavour='aws'):
     if flavour == 'ceph':
         _create_or_update_bucket_ceph(connection, module, location)
     else:
         _create_or_update_bucket(connection, module, location)
 
+
 def destroy_bucket(connection, module, flavour='aws'):
     if flavour == 'ceph':
         _destroy_bucket_ceph(connection, module)
     else:
         _destroy_bucket(connection, module)
 
+
 def is_fakes3(s3_url):
     """ Return True if s3_url has scheme fakes3:// """
     if s3_url is not None:
@@ -324,6 +331,7 @@ def is_fakes3(s3_url):
     else:
         return False
 
+
 def is_walrus(s3_url):
     """ Return True if it's Walrus endpoint, not S3
 
@@ -339,15 +347,15 @@ def main():
     argument_spec = ec2_argument_spec()
     argument_spec.update(
         dict(
-            force = dict(required=False, default='no', type='bool'),
-            policy = dict(required=False, type='json'),
-            name = dict(required=True, type='str'),
-            requester_pays = dict(default='no', type='bool'),
-            s3_url = dict(aliases=['S3_URL'], type='str'),
-            state = dict(default='present', type='str', choices=['present', 'absent']),
-            tags = dict(required=None, default={}, type='dict'),
-            versioning = dict(default='no', type='bool'),
-            ceph = dict(default='no', type='bool')
+            force=dict(required=False, default='no', type='bool'),
+            policy=dict(required=False, type='json'),
+            name=dict(required=True, type='str'),
+            requester_pays=dict(default='no', type='bool'),
+            s3_url=dict(aliases=['S3_URL'], type='str'),
+            state=dict(default='present', type='str', choices=['present', 'absent']),
+            tags=dict(required=False, default=None, type='dict'),
+            versioning=dict(default=None, type='bool'),
+            ceph=dict(default='no', type='bool')
         )
     )
 


Test patch: