Repository: ansible/ansible

Issue:
Cloudtrail module not loading profile information
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report


##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
Cloudtrail

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible --version
ansible 2.3.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Jul 19 2017, 08:25:13) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
retry_files_enabled = False
##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->

##### SUMMARY
<!--- Explain the problem briefly -->
When running my play with roles, every task is sourcing it's profile information properly like it should. However when crafting another task with the cloudtrail module, it is loading my default ~/.aws/credentials. {{ profile }} isn't getting loaded.
##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
create a play with the standard role layout. I am loading from a vars file. Within the vars file is a list I source for my cloudtrail task.

```
trails:
      - { profile: "sbx", trailname: "cloudtrail-sbx", s3: "cloudtrail-sbx", region: "us-east-1", stream_state: "enabled" }
```
Here is my task
```
---

# roles/cloudtrails/tasks/main.yml


- name:              Create cloudtrail
  with_items:         "{{ trails }}"
  cloudtrail:
    profile:          "{{ item.profile }}"
    name:             "{{ item.trailname }}"
    region:           "{{ item.region }}"
    state:            "{{ item.stream_state }}"
    s3_bucket_name:   "{{ item.s3 }}"
    s3_key_prefix:    ""
```
I have verified my boto and aws credentials are proper. It continues to load default from ~/.aws/credentials.

Swapping out default creds loads the play in the proper location. But this is not how the module is documented to work. I have hardcoded the profile, recycled my keys and the outcome is the same.

<!--- Paste example playbooks or commands between quotes below -->
```yaml

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
That the cloudtrail trail would be loaded in the proper profile that I specify
##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
It runs the cloudtrail in the default account. See verbosity.

What's even more disturbing is the logic breaks when creating the trail with the default profile. It creates the trail with configuration items that don't even exist. AKA Bucket doesn't exist, but ansible allows the trail to be configured anyway.

<!--- Paste verbatim command output between quotes below -->
```
-> % ansible-playbook vpc-playbook.yml -i inventory -e @sbx.yml -vvvv
Using /etc/ansible/ansible.cfg as config file
Loading callback plugin default of type stdout, v2.0 from /usr/local/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc

PLAYBOOK: vpc-playbook.yml ******************************************************************************************************************************************************************************
1 plays in vpc-playbook.yml

PLAY [local] ********************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/system/setup.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: <user>
<localhost> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
ok: [localhost]
META: ran handlers

TASK [cloudtrails : Create cloudtrail] ******************************************************************************************************************************************************************
task path: /Users/<user>/Documents/kraken/git/ansible-aws-vpc/roles/cloudtrails/tasks/main.yml:6
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/cloud/amazon/cloudtrail.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: <user>
<localhost> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
changed: [localhost] => (item={u'profile': u'sbx', u's3': u'cloudtrail-sbx', u'region': u'us-east-1', u'trailname': u'cloudtrail-sbx', u'stream_state': u'enabled'}) => {
    "changed": true,
    "enable": {
        "IncludeGlobalServiceEvents": true,
        "IsMultiRegionTrail": false,
        "LogFileValidationEnabled": false,
        "Name": "cloudtrail-sbx",
        "S3BucketName": "cloudtrail-sbx",
        "S3KeyPrefix": "",
        "TrailARN": "arn:aws:cloudtrail:us-east-1:XXXXXXXXX:trail/cloudtrail-sbx"
    },
    "exists": false,
    "invocation": {
        "module_args": {
            "aws_access_key": null,
            "aws_secret_key": null,
            "ec2_url": null,
            "include_global_events": true,
            "name": "cloudtrail-sbx",
            "profile": "sbx",
            "region": "us-east-1",
            "s3_bucket_name": "cloudtrail-sbx",
            "s3_key_prefix": "",
            "security_token": null,
            "state": "enabled",
            "validate_certs": true
        }
    },
    "item": {
        "profile": "sbx",
        "region": "us-east-1",
        "s3": "cloudtrail-sbx",
        "stream_state": "enabled",
        "trailname": "cloudtrail-sbx"
    },
    "logging_enabled": true,
    "view_status": {
        "IsLogging": false,
        "LatestDeliveryAttemptSucceeded": "",
        "LatestDeliveryAttemptTime": "",
        "LatestNotificationAttemptSucceeded": "",
        "LatestNotificationAttemptTime": "",
        "TimeLoggingStarted": "",
        "TimeLoggingStopped": ""
    },
    "was_logging_enabled": false
}
META: ran handlers
META: ran handlers

PLAY RECAP **********************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0
```

Cloudtrail module not loading profile information
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report


##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
Cloudtrail

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible --version
ansible 2.3.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Jul 19 2017, 08:25:13) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
retry_files_enabled = False
##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->

##### SUMMARY
<!--- Explain the problem briefly -->
When running my play with roles, every task is sourcing it's profile information properly like it should. However when crafting another task with the cloudtrail module, it is loading my default ~/.aws/credentials. {{ profile }} isn't getting loaded.
##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
create a play with the standard role layout. I am loading from a vars file. Within the vars file is a list I source for my cloudtrail task.

```
trails:
      - { profile: "sbx", trailname: "cloudtrail-sbx", s3: "cloudtrail-sbx", region: "us-east-1", stream_state: "enabled" }
```
Here is my task
```
---

# roles/cloudtrails/tasks/main.yml


- name:              Create cloudtrail
  with_items:         "{{ trails }}"
  cloudtrail:
    profile:          "{{ item.profile }}"
    name:             "{{ item.trailname }}"
    region:           "{{ item.region }}"
    state:            "{{ item.stream_state }}"
    s3_bucket_name:   "{{ item.s3 }}"
    s3_key_prefix:    ""
```
I have verified my boto and aws credentials are proper. It continues to load default from ~/.aws/credentials.

Swapping out default creds loads the play in the proper location. But this is not how the module is documented to work. I have hardcoded the profile, recycled my keys and the outcome is the same.

<!--- Paste example playbooks or commands between quotes below -->
```yaml

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
That the cloudtrail trail would be loaded in the proper profile that I specify
##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
It runs the cloudtrail in the default account. See verbosity.

What's even more disturbing is the logic breaks when creating the trail with the default profile. It creates the trail with configuration items that don't even exist. AKA Bucket doesn't exist, but ansible allows the trail to be configured anyway.

<!--- Paste verbatim command output between quotes below -->
```
-> % ansible-playbook vpc-playbook.yml -i inventory -e @sbx.yml -vvvv
Using /etc/ansible/ansible.cfg as config file
Loading callback plugin default of type stdout, v2.0 from /usr/local/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc

PLAYBOOK: vpc-playbook.yml ******************************************************************************************************************************************************************************
1 plays in vpc-playbook.yml

PLAY [local] ********************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/system/setup.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: <user>
<localhost> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
ok: [localhost]
META: ran handlers

TASK [cloudtrails : Create cloudtrail] ******************************************************************************************************************************************************************
task path: /Users/<user>/Documents/kraken/git/ansible-aws-vpc/roles/cloudtrails/tasks/main.yml:6
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/cloud/amazon/cloudtrail.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: <user>
<localhost> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
changed: [localhost] => (item={u'profile': u'sbx', u's3': u'cloudtrail-sbx', u'region': u'us-east-1', u'trailname': u'cloudtrail-sbx', u'stream_state': u'enabled'}) => {
    "changed": true,
    "enable": {
        "IncludeGlobalServiceEvents": true,
        "IsMultiRegionTrail": false,
        "LogFileValidationEnabled": false,
        "Name": "cloudtrail-sbx",
        "S3BucketName": "cloudtrail-sbx",
        "S3KeyPrefix": "",
        "TrailARN": "arn:aws:cloudtrail:us-east-1:XXXXXXXXX:trail/cloudtrail-sbx"
    },
    "exists": false,
    "invocation": {
        "module_args": {
            "aws_access_key": null,
            "aws_secret_key": null,
            "ec2_url": null,
            "include_global_events": true,
            "name": "cloudtrail-sbx",
            "profile": "sbx",
            "region": "us-east-1",
            "s3_bucket_name": "cloudtrail-sbx",
            "s3_key_prefix": "",
            "security_token": null,
            "state": "enabled",
            "validate_certs": true
        }
    },
    "item": {
        "profile": "sbx",
        "region": "us-east-1",
        "s3": "cloudtrail-sbx",
        "stream_state": "enabled",
        "trailname": "cloudtrail-sbx"
    },
    "logging_enabled": true,
    "view_status": {
        "IsLogging": false,
        "LatestDeliveryAttemptSucceeded": "",
        "LatestDeliveryAttemptTime": "",
        "LatestNotificationAttemptSucceeded": "",
        "LatestNotificationAttemptTime": "",
        "TimeLoggingStarted": "",
        "TimeLoggingStopped": ""
    },
    "was_logging_enabled": false
}
META: ran handlers
META: ran handlers

PLAY RECAP **********************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0
```



Discussion:
cc @shepdelacreme @tedder
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@shdobxr Thanks for opening the issue. Trying to understand what is happening here...it appears the profile sbx has been passed in successfully with with_items and is used by the module. Is the profile sbx by appearance only and the default actually utilized anyway?
I tried reproducing this by passing in two different profiles, neither of which has Cloudtrail permissions. The traceback displays the profile that got the AccessDeniedException, and the module appears to be correctly using profiles for me (and never used the default). Can you share the format of ~/.aws/credentials? Are you using source profiles?
I tested simply with something like:
```
[default]
aws_access_key_id = #######
aws_secret_access_key = #######

[testprofile]
aws_access_key_id = #######
aws_secret_access_key = #######

[shertel]
aws_access_key_id = #######
aws_secret_access_key = #######
```

needs_info
@shdobxr Also wanting to note that this module has been ported to boto3 for 2.4, which should get rid of issues with boto3-style credentials (which I think could be the problem here). The nonexistent bucket issue has also been fixed for 2.4.
Hey Shertel,

My layout of ~/.aws/credentials is much like yours obv without the
[testprofile] and [shertel] profiles.

My role for cloudtrails looks precisely like:

   ---
   
   # roles/cloudtrails/tasks/main.yml
   
   
   - name:              Create cloudtrail
     with_items:         "{{ trails }}"
     cloudtrail:
       profile:          "{{ item.profile }}"
       name:             "{{ item.trailname }}"
       region:           "{{ item.region }}"
       state:            "{{ item.stream_state }}"
       s3_bucket_name:   "{{ item.s3 }}"
       s3_key_prefix:    ""

And as you can see I source my vars from a list called trails

   trails:
         - { profile: "sbx", trailname: "cloudtrail-sbx", s3:"cloudtrail-sbx", region: "us-east-1", stream_state: "enabled" }

With verbosity pumped up I do see the same as you, the successful loading
of the profile. But it still creates the trail in my default account. I can
only direct the cloudtrail module to produce the trail in the proper
account by modifying my default keys within ~/.aws/credentials

When is 2.4 supposed to be released? I'd prefer to stay with prod and not run test versions of ansible.


@shdobxr This is an unfortunate bug. I just checked in the AWS console and I have reproduced the issue. Thanks for the quick reply.
@s-hertel No problem, thanks for taking this on. Any idea when 2.4 will be release for pub consumption?
@shdobxr Within a couple weeks I think. https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/roadmap/ROADMAP_2_4.rst
I guess I could pull 2.4 and test. Were you able to get the same bug in 2.4?

@s-hertel if this is fixed in devel can we link to the PR fixing this?

2.3.2.0 same issue. Pulling 2.4.0.0 to see if this works as designed. Thanks for the comments and direction folks (tips hat). I installed a dev branch awhile back but my brain isn't working this morning. How would I go about installing 2.4? I see that pip shows the latest 2.3.2.0

nvmind, coffee wins

`git clone https://github.com/ansible/ansible.git`
`cd ansible`
`pip install -r requirements.txt`
`python setup.py install`

Ahh I get an error 

`    "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/kr/pdz2r4pd43q4f4bwfsx6d_20b2sqwb/T/ansible_NZkGyd/ansible_module_cloudtrail.py\", line 248, in <module>\n    from botocore.exceptions import ClientError\nImportError: No module named botocore.exceptions\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0`

`~/.env/bin/ansible-playbook --version
ansible-playbook 2.4.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/username/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/username/.env/lib/python2.7/site-packages/ansible-2.4.0-py2.7.egg/ansible
  executable location = /Users/username/.env/bin/ansible-playbook
  python version = 2.7.10 (default, Feb  7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]`
@shdobxr The refactored module in 2.4 uses boto3 and requires the boto3 and botocore pip packages. Even so it probably should be giving a nice error instead of that traceback like that.

@willthames This is the PR for the boto3 refactor. #22884 

@willthames Porting this to boto3 seems to have fixed the issue, but I'm not sure exactly what changed between the way boto2 reads the credentials and boto3. Need to look through the source code. There are enough modules still using boto2 though that I'm concerned the underlying problem is still a bug needing to be fixed.
@shepdelacreme I was thinking it was most definitely a boto3 dependency. Working to get that into my virtualenv, will communicate the outcome. @willthames @s-hertel Cheers!
Within the virtualenv I installed boto3 and validated botocore is available. Same error. Is there a specific rev if boto3 and botocore that I need to be running?

`-> % pip list
ansible (2.4.0)
...(truncated)
boto3 (1.4.7)
botocore (1.7.4)
...(truncated)`
Hrm. I had some issues initially with that too, but I can't remember how I fixed it. Your versions should be fine...
Is your virtualenv python3? (if so, might need to check or install with pip3?)
```
$ pip freeze | grep boto
boto==2.47.0
boto3==1.4.6
botocore==1.6.3
```
I believe I follow you, I installed virtualenv with pip (not pip3). python3 is achievable within the env.

```
pip3 freeze | grep boto
boto==2.48.0
boto3==1.4.6
botocore==1.6.5
```

```
pip freeze | grep boto
boto==2.48.0
boto3==1.4.7
botocore==1.7.4
```

```
python3 --version
Python 3.6.2
```
Is this the issue related to Ansible using `/usr/bin/python` in modules?

@shdobxr Do you have something like this in your inventory file?

```
localhost ansible_python_interpreter=python
```

This should force it to use the virtualenv python and its packages.
@shdobxr Sorry for spamming the thread. I meant how you installed the boto libraries. A while ago I was running a python3-as-the-default-python virtualenv and had used pip to install boto3. I had a boto3 file elsewhere, and things were working weirdly. Uninstalling and then reinstalling with pip3 fixed that issue for me.

Are you using the local connection type? If not, perhaps it's grabbing your system python instead of the venv.

You could try running `$ source hacking/env-setup` again.
@s-hertel No harm in getting more attention on this issue as it's important to me and potentially other cloudtrail automators. Spam cannon away! I rebuild my virtualenv with pyenv for 3.6.2 for those that care to try I'm on a mac (brew).

```
brew install pyenv
pyenv install 3.6.2
virtualenv -p ~/.pyenv/versions/3.6.2/bin/python3.6 .env
source .env/bin/activate
python --version (3.6.2)
cd ansible (pull'd from github)
source hacking/env-setup
pip install -r requirements.txt
python setup.py install
```


Ahhh the missing bit was having boto3 installed with pip3.
within the env
```
pip3 install boto3
```

```
PLAY RECAP **********************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0
```
Looking forward to 2.4.0.0 having this fixed, I will leave this to @s-hertel  and @willthames  to potentially track if logging needs to be fixed for that traceback, or that ansible modules properly error if a dependency like a boto3 module being present. 
cc @shepdelacreme @tedder
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@shdobxr Thanks for opening the issue. Trying to understand what is happening here...it appears the profile sbx has been passed in successfully with with_items and is used by the module. Is the profile sbx by appearance only and the default actually utilized anyway?
I tried reproducing this by passing in two different profiles, neither of which has Cloudtrail permissions. The traceback displays the profile that got the AccessDeniedException, and the module appears to be correctly using profiles for me (and never used the default). Can you share the format of ~/.aws/credentials? Are you using source profiles?
I tested simply with something like:
```
[default]
aws_access_key_id = #######
aws_secret_access_key = #######

[testprofile]
aws_access_key_id = #######
aws_secret_access_key = #######

[shertel]
aws_access_key_id = #######
aws_secret_access_key = #######
```

needs_info
@shdobxr Also wanting to note that this module has been ported to boto3 for 2.4, which should get rid of issues with boto3-style credentials (which I think could be the problem here). The nonexistent bucket issue has also been fixed for 2.4.
Hey Shertel,

My layout of ~/.aws/credentials is much like yours obv without the
[testprofile] and [shertel] profiles.

My role for cloudtrails looks precisely like:

   ---
   
   # roles/cloudtrails/tasks/main.yml
   
   
   - name:              Create cloudtrail
     with_items:         "{{ trails }}"
     cloudtrail:
       profile:          "{{ item.profile }}"
       name:             "{{ item.trailname }}"
       region:           "{{ item.region }}"
       state:            "{{ item.stream_state }}"
       s3_bucket_name:   "{{ item.s3 }}"
       s3_key_prefix:    ""

And as you can see I source my vars from a list called trails

   trails:
         - { profile: "sbx", trailname: "cloudtrail-sbx", s3:"cloudtrail-sbx", region: "us-east-1", stream_state: "enabled" }

With verbosity pumped up I do see the same as you, the successful loading
of the profile. But it still creates the trail in my default account. I can
only direct the cloudtrail module to produce the trail in the proper
account by modifying my default keys within ~/.aws/credentials

When is 2.4 supposed to be released? I'd prefer to stay with prod and not run test versions of ansible.


@shdobxr This is an unfortunate bug. I just checked in the AWS console and I have reproduced the issue. Thanks for the quick reply.
@s-hertel No problem, thanks for taking this on. Any idea when 2.4 will be release for pub consumption?
@shdobxr Within a couple weeks I think. https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/roadmap/ROADMAP_2_4.rst
I guess I could pull 2.4 and test. Were you able to get the same bug in 2.4?

@s-hertel if this is fixed in devel can we link to the PR fixing this?

2.3.2.0 same issue. Pulling 2.4.0.0 to see if this works as designed. Thanks for the comments and direction folks (tips hat). I installed a dev branch awhile back but my brain isn't working this morning. How would I go about installing 2.4? I see that pip shows the latest 2.3.2.0

nvmind, coffee wins

`git clone https://github.com/ansible/ansible.git`
`cd ansible`
`pip install -r requirements.txt`
`python setup.py install`

Ahh I get an error 

`    "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/kr/pdz2r4pd43q4f4bwfsx6d_20b2sqwb/T/ansible_NZkGyd/ansible_module_cloudtrail.py\", line 248, in <module>\n    from botocore.exceptions import ClientError\nImportError: No module named botocore.exceptions\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0`

`~/.env/bin/ansible-playbook --version
ansible-playbook 2.4.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/username/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/username/.env/lib/python2.7/site-packages/ansible-2.4.0-py2.7.egg/ansible
  executable location = /Users/username/.env/bin/ansible-playbook
  python version = 2.7.10 (default, Feb  7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]`
@shdobxr The refactored module in 2.4 uses boto3 and requires the boto3 and botocore pip packages. Even so it probably should be giving a nice error instead of that traceback like that.

@willthames This is the PR for the boto3 refactor. #22884 

@willthames Porting this to boto3 seems to have fixed the issue, but I'm not sure exactly what changed between the way boto2 reads the credentials and boto3. Need to look through the source code. There are enough modules still using boto2 though that I'm concerned the underlying problem is still a bug needing to be fixed.
@shepdelacreme I was thinking it was most definitely a boto3 dependency. Working to get that into my virtualenv, will communicate the outcome. @willthames @s-hertel Cheers!
Within the virtualenv I installed boto3 and validated botocore is available. Same error. Is there a specific rev if boto3 and botocore that I need to be running?

`-> % pip list
ansible (2.4.0)
...(truncated)
boto3 (1.4.7)
botocore (1.7.4)
...(truncated)`
Hrm. I had some issues initially with that too, but I can't remember how I fixed it. Your versions should be fine...
Is your virtualenv python3? (if so, might need to check or install with pip3?)
```
$ pip freeze | grep boto
boto==2.47.0
boto3==1.4.6
botocore==1.6.3
```
I believe I follow you, I installed virtualenv with pip (not pip3). python3 is achievable within the env.

```
pip3 freeze | grep boto
boto==2.48.0
boto3==1.4.6
botocore==1.6.5
```

```
pip freeze | grep boto
boto==2.48.0
boto3==1.4.7
botocore==1.7.4
```

```
python3 --version
Python 3.6.2
```
Is this the issue related to Ansible using `/usr/bin/python` in modules?

@shdobxr Do you have something like this in your inventory file?

```
localhost ansible_python_interpreter=python
```

This should force it to use the virtualenv python and its packages.
@shdobxr Sorry for spamming the thread. I meant how you installed the boto libraries. A while ago I was running a python3-as-the-default-python virtualenv and had used pip to install boto3. I had a boto3 file elsewhere, and things were working weirdly. Uninstalling and then reinstalling with pip3 fixed that issue for me.

Are you using the local connection type? If not, perhaps it's grabbing your system python instead of the venv.

You could try running `$ source hacking/env-setup` again.
@s-hertel No harm in getting more attention on this issue as it's important to me and potentially other cloudtrail automators. Spam cannon away! I rebuild my virtualenv with pyenv for 3.6.2 for those that care to try I'm on a mac (brew).

```
brew install pyenv
pyenv install 3.6.2
virtualenv -p ~/.pyenv/versions/3.6.2/bin/python3.6 .env
source .env/bin/activate
python --version (3.6.2)
cd ansible (pull'd from github)
source hacking/env-setup
pip install -r requirements.txt
python setup.py install
```


Ahhh the missing bit was having boto3 installed with pip3.
within the env
```
pip3 install boto3
```

```
PLAY RECAP **********************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0
```
Looking forward to 2.4.0.0 having this fixed, I will leave this to @s-hertel  and @willthames  to potentially track if logging needs to be fixed for that traceback, or that ansible modules properly error if a dependency like a boto3 module being present. 

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/cloudtrail.py b/lib/ansible/modules/cloud/amazon/cloudtrail.py
--- a/lib/ansible/modules/cloud/amazon/cloudtrail.py
+++ b/lib/ansible/modules/cloud/amazon/cloudtrail.py
@@ -119,7 +119,7 @@
     HAS_BOTO = False
 
 from ansible.module_utils.basic import AnsibleModule
-from ansible.module_utils.ec2 import connect_to_aws, ec2_argument_spec, get_ec2_creds
+from ansible.module_utils.ec2 import connect_to_aws, ec2_argument_spec, get_aws_connection_info
 
 
 class CloudTrailManager:
@@ -186,9 +186,7 @@ def main():
     if not HAS_BOTO:
         module.fail_json(msg='boto is required.')
 
-    ec2_url, access_key, secret_key, region = get_ec2_creds(module)
-    aws_connect_params = dict(aws_access_key_id=access_key,
-                              aws_secret_access_key=secret_key)
+    region, ec2_url, aws_connect_params = get_aws_connection_info(module)
 
     if not region:
         module.fail_json(msg="Region must be specified as a parameter, in EC2_REGION or AWS_REGION environment variables or in boto configuration file")


Test patch: