Repository: ansible/ansible

Issue:
Python 2.6 format method error for s3_sync module
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report


##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
s3_sync

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.3.0.0 (stable-2.3 af018b75f0) last updated 2017/08/10 13:31:56 (GMT +1000)
  config file = /home/mulrddep/.ansible.cfg
  configured module search path = [u'/apps/ansible/library', u'/apps/mule-platform/ansible/library', u'/apps/ansible-playbooks/common/library']
  python version = 2.6.6 (r266:84292, Aug 18 2016, 15:13:37) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->more 
ansible.cfg:
[defaults]
host_key_checking=False
hash_behaviour=merge
roles_path=/apps/mule-platform/roles:/apps/mule-ansible-tree/roles

env vars:
ANSIBLE_HOME=/apps/ansible
ANSIBLE_HOSTS=/apps/mule-platform/mule-goserver/hosts.ini


##### OS / ENVIRONMENT
CentOS release 6.9 (Final)

##### SUMMARY
<!--- Explain the problem briefly -->
When using the s3_sync module on a server with Python 2.6, the playbook will fail due to a syntax error with the format method used by the module. The following value error is returned:
"ValueError: zero length field name in format"

Lines 277, 282, 432 and 433 of this module contain the string format method without positional argument specifiers, which isn't supported by python 2.6 (e.g.  entry['whytime'] = '{} / {}'.format .... )
This is fixed by using positional argument specifiers (e.g.  '{0} {1}' )
See here for ref: 
https://docs.python.org/2/library/string.html#format-string-syntax

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- hosts: localhost
  connection: local
    
  tasks:
    - name: "Upload to s3 bucket"
      s3_sync:
        aws_secret_key: "{{aws_secret_access_key}}"
        aws_access_key: "{{aws_access_key_id}}"
        security_token: "{{security_token}}"
        bucket: "{{ s3_transfer_bucket }}"
        key_prefix: "{{ s3_transfer_bucket_path }}"
        file_root: "{{ s3_transfer_local_path }}"
        file_change_strategy: force
        region: "{{ aws_region }}"

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
S3 upload should be successful. 
I've modified the s3_sync.py module locally to fix formatting to support Python 2.6 and this same playbook ran successfully. 


##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
Using /apps/mule-platform/mule/ansible.cfg as config file
Loading callback plugin default of type stdout, v2.0 from /apps/ansible/lib/ansible/plugins/callback/__init__.pyc

PLAYBOOK: s3.yml **************************************************************************************************
1 plays in s3.yml

PLAY [localhost] **************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************
Using module file /apps/ansible/lib/ansible/modules/system/setup.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mulrddep
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mulrddep/.ansible/tmp/ansible-tmp-1502754249.89-250223534758667 `" && echo ansible-tmp-1502754249.89-250223534758667="` echo /home/mulrddep/.ansible/tmp/ansible-tmp-1502754249.89-250223534758667 `" ) && sleep 0'
<localhost> PUT /tmp/tmpL1iEtj TO /home/mulrddep/.ansible/tmp/ansible-tmp-1502754249.89-250223534758667/setup.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mulrddep/.ansible/tmp/ansible-tmp-1502754249.89-250223534758667/ /home/mulrddep/.ansible/tmp/ansible-tmp-1502754249.89-250223534758667/setup.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/mulrddep/.ansible/tmp/ansible-tmp-1502754249.89-250223534758667/setup.py; rm -rf "/home/mulrddep/.ansible/tmp/ansible-tmp-1502754249.89-250223534758667/" > /dev/null 2>&1 && sleep 0'
ok: [localhost]
META: ran handlers


TASK [Upload to s3 bucket] ****************************************************************************************
task path: /apps/mule-platform/mule/s3.yml:36
Using module file /apps/ansible/lib/ansible/modules/cloud/amazon/s3_sync.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mulrddep
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mulrddep/.ansible/tmp/ansible-tmp-1502754251.47-153974990253487 `" && echo ansible-tmp-1502754251.47-153974990253487="` echo /home/mulrddep/.ansible/tmp/ansible-tmp-1502754251.47-153974990253487 `" ) && sleep 0'
<localhost> PUT /tmp/tmpL1iEtj TO /home/mulrddep/.ansible/tmp/ansible-tmp-1502754251.47-153974990253487/s3_sync.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mulrddep/.ansible/tmp/ansible-tmp-1502754251.47-153974990253487/ /home/mulrddep/.ansible/tmp/ansible-tmp-1502754251.47-153974990253487/s3_sync.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/mulrddep/.ansible/tmp/ansible-tmp-1502754251.47-153974990253487/s3_sync.py; rm -rf "/home/mulrddep/.ansible/tmp/ansible-tmp-1502754251.47-153974990253487/" > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "aws_access_key": "ASIAILNFILJ74DT75P3A",
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "bucket": "suncorp-mule-connector-updatesites",
            "ec2_url": null,
            "exclude": ".*",
            "file_change_strategy": "force",
            "file_root": "/tmp/connector-life400/",
            "include": "*",
            "key_prefix": "connector-life400",
            "mime_map": null,
            "mode": "push",
            "permission": null,
            "profile": null,
            "region": "ap-southeast-2",
            "retries": null,
            "security_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "validate_certs": true
        }
    },
    "msg": "zero length field name in format zero length field name in format - <type 'exceptions.ValueError'>",
    "traceback": [
        "Traceback (most recent call last):",
        "  File \"/tmp/ansible_ZjFzBU/ansible_module_s3_sync.py\", line 478, in main",
        "    result['filelist_local_etag'] = calculate_local_etag(result['filelist_s3'])",
        "  File \"/tmp/ansible_ZjFzBU/ansible_module_s3_sync.py\", line 330, in calculate_local_etag",
        "    retentry['local_etag'] = calculate_multipart_etag(fileentry['fullpath'])",
        "  File \"/tmp/ansible_ZjFzBU/ansible_module_s3_sync.py\", line 265, in calculate_multipart_etag",
        "    new_etag = '\"{}\"'.format(md5s[0].hexdigest())",
        "ValueError: zero length field name in format"
    ]
}
        to retry, use: --limit @/apps/mule-platform/mule/s3.retry

PLAY RECAP ********************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1

```



Discussion:
cc @tedder
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Yep, looks like a good fix. Put it in a PR?

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/s3_sync.py b/lib/ansible/modules/cloud/amazon/s3_sync.py
--- a/lib/ansible/modules/cloud/amazon/s3_sync.py
+++ b/lib/ansible/modules/cloud/amazon/s3_sync.py
@@ -274,12 +274,12 @@ def calculate_multipart_etag(source_path, chunk_size=DEFAULT_CHUNK_SIZE):
             md5s.append(hashlib.md5(data))
 
     if len(md5s) == 1:
-        new_etag = '"{}"'.format(md5s[0].hexdigest())
+        new_etag = '"{0}"'.format(md5s[0].hexdigest())
     else:  # > 1
         digests = b"".join(m.digest() for m in md5s)
 
         new_md5 = hashlib.md5(digests)
-        new_etag = '"{}-{}"'.format(new_md5.hexdigest(), len(md5s))
+        new_etag = '"{0}-{1}"'.format(new_md5.hexdigest(), len(md5s))
 
     return new_etag
 
@@ -429,8 +429,8 @@ def filter_list(s3, bucket, s3filelist, strategy):
 
                 remote_size = entry['s3_head']['ContentLength']
 
-                entry['whytime'] = '{} / {}'.format(local_modified_epoch, remote_modified_epoch)
-                entry['whysize'] = '{} / {}'.format(local_size, remote_size)
+                entry['whytime'] = '{0} / {1}'.format(local_modified_epoch, remote_modified_epoch)
+                entry['whysize'] = '{0} / {1}'.format(local_size, remote_size)
 
                 if local_modified_epoch <= remote_modified_epoch or local_size == remote_size:
                     entry['skip_flag'] = True


Test patch: