Repository: ansible/ansible

Issue:
aws_ec2 inventory plugin doesn't find/use server IAM role
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
Always add information AFTER of these html comments. -->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report
 

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->

aws_ec2.py 

(aws_ec2.py / all.aws_ec2.yml)

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->

ansible-config dump --only-changed
DEFAULT_HOST_LIST(/etc/ansible/ansible.cfg) = [u'/etc/ansible/plugins/inventory']
DEFAULT_REMOTE_USER(/etc/ansible/ansible.cfg) = ansible
INVENTORY_ENABLED(/etc/ansible/ansible.cfg) = ['aws_ec2', 'host_list', 'virtualbox', 'yaml', 'constructed']

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->

ubuntu 16.04 (ansible host)
ubuntu 16.04 (target host)

##### SUMMARY
<!--- Explain the problem briefly -->
Ansible server has IAM role that works with dynamic inventory script (ec2.py / ec2.ini). Switching dynamic inventory source to aws_ec2 plugin (aws.ec2.py / all.aws_ec2.yml) fails with WARNING message "Insufficient boto credentials found". 

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```
inventory_test.yml
---
- hosts: "{{ target }}"
  remote_user: ansible
  become: yes
  tasks:
    - name: test connection
      ping:
      remote_user: ansible

all.aws_ec2.yml

plugin: aws_ec2
regions: # populate inventory with instances in these regions
  - us-east-1
filters:
  
# ignores 403 errors rather than failing
strict_permissions: False
hostnames:
  - tag:Name
  - private-ip-address    

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->

Expect ansible to find/use server IAM role to return test-instance with aws_ec2 inventory plugin for playbook execution

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
Ansible warns of "Insufficient boto credentials found" and prompts to place credentials in inventory configuration file or environment variables. 

Ansible output warns : Insufficient boto credentials found. Please provide them in your inventory
configuration file or set them as environment variables.

<!--- Paste verbatim command output between quotes below -->
```
ansible-playbook -e "target=test-instance" inventory_test.yml -vvvv
ansible-playbook 2.5.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
 [WARNING]:  * Failed to parse /etc/ansible/plugins/inventory/all.aws_ec2.yml with aws_ec2 plugin: Insufficient boto credentials found. Please provide them in your inventory
configuration file or set them as environment variables.

  File "/usr/lib/python2.7/dist-packages/ansible/inventory/manager.py", line 264, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/lib/python2.7/dist-packages/ansible/plugins/inventory/aws_ec2.py", line 505, in parse
    self._set_credentials()
  File "/usr/lib/python2.7/dist-packages/ansible/plugins/inventory/aws_ec2.py", line 444, in _set_credentials
    raise AnsibleError("Insufficient boto credentials found. Please provide them in your "

 [WARNING]:  * Failed to parse /etc/ansible/plugins/inventory/all.aws_ec2.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory

  File "/usr/lib/python2.7/dist-packages/ansible/inventory/manager.py", line 264, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/lib/python2.7/dist-packages/ansible/plugins/inventory/aws_ec2.py", line 505, in parse
    self._set_credentials()
  File "/usr/lib/python2.7/dist-packages/ansible/plugins/inventory/aws_ec2.py", line 444, in _set_credentials
    raise AnsibleError("Insufficient boto credentials found. Please provide them in your "

 [WARNING]:  * Failed to parse /etc/ansible/plugins/inventory/all.aws_ec2.yml with constructed plugin: Incorrect plugin name in file: aws_ec2

  File "/usr/lib/python2.7/dist-packages/ansible/inventory/manager.py", line 264, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/lib/python2.7/dist-packages/ansible/plugins/inventory/yaml.py", line 103, in parse
    raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')

 [WARNING]: Unable to parse /etc/ansible/plugins/inventory/all.aws_ec2.yml as an inventory source

 [WARNING]: Unable to parse /etc/ansible/plugins/inventory as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Loading callback plugin default of type stdout, v2.0 from /usr/lib/python2.7/dist-packages/ansible/plugins/callback/default.pyc

PLAYBOOK: inventory_test.yml *************************************************************************************************************************************************************
1 plays in inventory_test.yml
 [WARNING]: Could not match supplied host pattern, ignoring: test-instance


PLAY [test-instance] *********************************************************************************************************************************************************************
skipping: no hosts matched

PLAY RECAP *******************************************************************************************************************************************************************************

```



Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Files identified in the description:
* [lib/ansible/plugins/inventory/aws_ec2.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/inventory/aws_ec2.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc/ @s-hertel 
It's sometimes possible if you set some `AWS_*_KEY` variables that boto may not actually look for the EC2 instance role. Can you double-check `env | grep -i AWS` doesn't show anything credential-related?
@ryansb 

Nothing returned from env | grep -i AWS  
It works if add param `boto_profile: my_profile` to all.aws_ec2.yml

@luvs 

The server uses an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), so there is no boto profile to add. I tried using the name of the IAM role but that produces a "config profile could not be found" error. 

I was able to continue testing by adding an extra check to _set_credentials in aws_ec2.py that retrieves the temporary credentials:

`
def _set_credentials(self):
        '''
            :param config_data: contents of the inventory config file
        '''

        self.boto_profile = self._options.get('boto_profile')
        self.aws_access_key_id = self._options.get('aws_access_key_id')
        self.aws_secret_access_key = self._options.get('aws_secret_access_key')
        self.aws_security_token = self._options.get('aws_security_token')

        if not self.boto_profile and not (self.aws_access_key_id and self.aws_secret_access_key):
            session = botocore.session.get_session()
            self.aws_access_key_id = session.get_credentials().access_key
            self.aws_secret_access_key = session.get_credentials().secret_key
            self.aws_security_token = session.get_credentials().token

        if not self.boto_profile and not (self.aws_access_key_id and self.aws_secret_access_key):
            raise AnsibleError("Insufficient boto credentials found. Please provide them in your "
`
@jsutter55 - You should send a PR before someone else does and takes the credit!
Merging into jsutter/devel branch doesn't close this for everyone ;) 
@ryansb 

Sorry. Newb on parade. Just sent again... hopefully. :)

Resolving patch:
diff --git a/lib/ansible/plugins/inventory/aws_ec2.py b/lib/ansible/plugins/inventory/aws_ec2.py
--- a/lib/ansible/plugins/inventory/aws_ec2.py
+++ b/lib/ansible/plugins/inventory/aws_ec2.py
@@ -451,6 +451,13 @@ def _set_credentials(self):
         self.aws_secret_access_key = self._options.get('aws_secret_access_key')
         self.aws_security_token = self._options.get('aws_security_token')
 
+        if not self.boto_profile and not (self.aws_access_key_id and self.aws_secret_access_key):
+            session = botocore.session.get_session()
+            if session.get_credentials() is not None:
+                self.aws_access_key_id = session.get_credentials().access_key
+                self.aws_secret_access_key = session.get_credentials().secret_key
+                self.aws_security_token = session.get_credentials().token
+
         if not self.boto_profile and not (self.aws_access_key_id and self.aws_secret_access_key):
             raise AnsibleError("Insufficient boto credentials found. Please provide them in your "
                                "inventory configuration file or set them as environment variables.")


Test patch: