Repository: ansible/ansible

Issue:
aws_s3 module error with version id None does not exist
##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
`aws_s3`

##### ANSIBLE VERSION
```
ansible 2.4.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/arbab/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/arbab/.virtualenvs/ansible24/lib/python2.7/site-packages/ansible
  executable location = /Users/arbab/.virtualenvs/ansible24/bin/ansible
  python version = 2.7.10 (default, Feb  7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
```

##### CONFIGURATION
```
ANSIBLE_NOCOWS(/etc/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/etc/ansible/ansible.cfg) = -o ForwardAgent=yes
ANSIBLE_SSH_PIPELINING(/etc/ansible/ansible.cfg) = True
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 100
DEFAULT_MANAGED_STR(/etc/ansible/ansible.cfg) = Ansible managed: This file was modified on %Y-%m-%d %H:%M:%S by {uid}, don't make changes here - they will be overwritten.
DEFAULT_NO_TARGET_SYSLOG(/etc/ansible/ansible.cfg) = False
DEFAULT_SUDO_FLAGS(/etc/ansible/ansible.cfg) = -HE
DEFAULT_TIMEOUT(/etc/ansible/ansible.cfg) = 120
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False
```

##### OS / ENVIRONMENT
``` Mac OS X and Ubuntu```

##### SUMMARY

I have simple task to download file from s3 bucket which works fine with Ansible 2.3, when I have upgraded the ansible from 2.3 to 2.4, it complain that I need to use the aws_s3 module instead of s3 which I did but get an error:
`Key /node/qa/1.0.483/node.tar.gz with version id None does not exist`

##### STEPS TO REPRODUCE
Here is my task:
```yaml
- name: Download the latest version of from S3 bucket
  aws_s3:
    aws_access_key: "xxxxxxxxxxx"
    aws_secret_key: "yyyyyyyyyyyyyyyyyyyyyyy"
    bucket: "artifacts-bucket"
    object: "/node/qa/1.0.483/node.tar.gz"
    mode: get
    dest: "node.tar.gz"
```

##### EXPECTED RESULTS
Object should be download from the S3 bucket

##### ACTUAL RESULTS
```
ansible-playbook -i localhost, playbook.yml                                                                    4 ↵


PLAY [all] *******************************************************************************************************************


TASK [Download the latest version of from S3 bucket] *************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Key /node/qa/1.0.483/node.tar.gz with version id None does not exist."}


PLAY RECAP *******************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1
```

Error when run that task with extra verbosity:

```
fatal: [localhost]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "aws_access_key": "xxxxxxxxxxxxx",
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "bucket": "artifacts",
            "dest": "node.tar.gz",
            "ec2_url": null,
            "encrypt": true,
            "expiry": 600,
            "headers": null,
            "ignore_nonexistent_bucket": false,
            "marker": "",
            "max_keys": 1000,
            "metadata": null,
            "mode": "get",
            "object": "/node/qa/1.0.483/node.tar.gz",
            "overwrite": "always",
            "permission": [
                "private"
            ],
            "prefix": "",
            "profile": null,
            "region": null,
            "retries": 0,
            "rgw": false,
            "s3_url": null,
            "security_token": null,
            "src": null,
            "validate_certs": true,
            "version": null
        }
    },
    "msg": "Key /node/qa/1.0.483/node.tar.gz with version id None does not exist."
}
```
aws_s3 module error with version id None does not exist
##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
`aws_s3`

##### ANSIBLE VERSION
```
ansible 2.4.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/arbab/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/arbab/.virtualenvs/ansible24/lib/python2.7/site-packages/ansible
  executable location = /Users/arbab/.virtualenvs/ansible24/bin/ansible
  python version = 2.7.10 (default, Feb  7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
```

##### CONFIGURATION
```
ANSIBLE_NOCOWS(/etc/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/etc/ansible/ansible.cfg) = -o ForwardAgent=yes
ANSIBLE_SSH_PIPELINING(/etc/ansible/ansible.cfg) = True
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 100
DEFAULT_MANAGED_STR(/etc/ansible/ansible.cfg) = Ansible managed: This file was modified on %Y-%m-%d %H:%M:%S by {uid}, don't make changes here - they will be overwritten.
DEFAULT_NO_TARGET_SYSLOG(/etc/ansible/ansible.cfg) = False
DEFAULT_SUDO_FLAGS(/etc/ansible/ansible.cfg) = -HE
DEFAULT_TIMEOUT(/etc/ansible/ansible.cfg) = 120
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False
```

##### OS / ENVIRONMENT
``` Mac OS X and Ubuntu```

##### SUMMARY

I have simple task to download file from s3 bucket which works fine with Ansible 2.3, when I have upgraded the ansible from 2.3 to 2.4, it complain that I need to use the aws_s3 module instead of s3 which I did but get an error:
`Key /node/qa/1.0.483/node.tar.gz with version id None does not exist`

##### STEPS TO REPRODUCE
Here is my task:
```yaml
- name: Download the latest version of from S3 bucket
  aws_s3:
    aws_access_key: "xxxxxxxxxxx"
    aws_secret_key: "yyyyyyyyyyyyyyyyyyyyyyy"
    bucket: "artifacts-bucket"
    object: "/node/qa/1.0.483/node.tar.gz"
    mode: get
    dest: "node.tar.gz"
```

##### EXPECTED RESULTS
Object should be download from the S3 bucket

##### ACTUAL RESULTS
```
ansible-playbook -i localhost, playbook.yml                                                                    4 ↵


PLAY [all] *******************************************************************************************************************


TASK [Download the latest version of from S3 bucket] *************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Key /node/qa/1.0.483/node.tar.gz with version id None does not exist."}


PLAY RECAP *******************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1
```

Error when run that task with extra verbosity:

```
fatal: [localhost]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "aws_access_key": "xxxxxxxxxxxxx",
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "bucket": "artifacts",
            "dest": "node.tar.gz",
            "ec2_url": null,
            "encrypt": true,
            "expiry": 600,
            "headers": null,
            "ignore_nonexistent_bucket": false,
            "marker": "",
            "max_keys": 1000,
            "metadata": null,
            "mode": "get",
            "object": "/node/qa/1.0.483/node.tar.gz",
            "overwrite": "always",
            "permission": [
                "private"
            ],
            "prefix": "",
            "profile": null,
            "region": null,
            "retries": 0,
            "rgw": false,
            "s3_url": null,
            "security_token": null,
            "src": null,
            "validate_certs": true,
            "version": null
        }
    },
    "msg": "Key /node/qa/1.0.483/node.tar.gz with version id None does not exist."
}
```


Discussion:
cc @lwade @s-hertel
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Adding a note about this bug:
Prior to porting this module to boto3 object: "/path/to/object" refered to bucketname/path/to/object. In 2.4 this became bucketname//path/to/object.
documentation is incorrect here: http://docs.ansible.com/ansible/devel/aws_s3_module.html

The leading / in the object argument should be omit.

This example work for me:

```yaml
- name: Download the latest version of from S3 bucket
  aws_s3:
    aws_access_key: "xxxxxxxxxxx"
    aws_secret_key: "yyyyyyyyyyyyyyyyyyyyyyy"
    bucket: "artifacts-bucket"
    object: "node/qa/1.0.483/node.tar.gz"
    mode: get
    dest: "node.tar.gz"
```
Note the `object` argument doesn't have leading / in it.  Thanks @s-hertel for help
cc @lwade @s-hertel
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Adding a note about this bug:
Prior to porting this module to boto3 object: "/path/to/object" refered to bucketname/path/to/object. In 2.4 this became bucketname//path/to/object.
documentation is incorrect here: http://docs.ansible.com/ansible/devel/aws_s3_module.html

The leading / in the object argument should be omit.

This example work for me:

```yaml
- name: Download the latest version of from S3 bucket
  aws_s3:
    aws_access_key: "xxxxxxxxxxx"
    aws_secret_key: "yyyyyyyyyyyyyyyyyyyyyyy"
    bucket: "artifacts-bucket"
    object: "node/qa/1.0.483/node.tar.gz"
    mode: get
    dest: "node.tar.gz"
```
Note the `object` argument doesn't have leading / in it.  Thanks @s-hertel for help

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/aws_s3.py b/lib/ansible/modules/cloud/amazon/aws_s3.py
--- a/lib/ansible/modules/cloud/amazon/aws_s3.py
+++ b/lib/ansible/modules/cloud/amazon/aws_s3.py
@@ -585,6 +585,10 @@ def main():
 
     if module.params.get('object'):
         obj = module.params['object']
+        # If there is a top level object, do nothing - if the object starts with /
+        # remove the leading character to maintain compatibility with Ansible versions < 2.4
+        if obj.startswith('/'):
+            obj = obj[1:]
 
     # Bucket deletion does not require obj.  Prevents ambiguity with delobj.
     if obj and mode == "delete":


Test patch: