Repository: ansible-collections/amazon.aws

Issue:
Passing both a profile and access tokens is not supported
### Summary

Using amazon.aws.ec2_instance module with the profile parameter and AWS credentials stored in the environment variables results in an error `Passing both a profile and access tokens is not supported`

This behaviour is not shown in Ansible 7.2 using amazon.aws collection version 5.1.0

### Issue Type

Bug Report

### Component Name

amazon.aws.ec2_instance

### Ansible Version

ansible [core 2.14.2]
  config file = /home/centos/ansible/ansible.cfg
  configured module search path = ['/home/centos/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/centos/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/centos/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/centos/.local/bin/ansible
  python version = 3.9.6 (default, Aug 25 2021, 16:22:38) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True


Name: ansible
Version: 7.2.0
Summary: Radically simple IT automation
Home-page: https://ansible.com/
Author: Ansible, Inc.
Author-email: info@ansible.com
License: GPLv3+
Location: /home/centos/.local/lib/python3.9/site-packages
Requires: ansible-core
Required-by:



### Collection Versions

Collection                    Version
----------------------------- -------
amazon.aws                    5.2.0
ansible.netcommon             4.1.0
ansible.posix                 1.5.1
ansible.utils                 2.9.0
ansible.windows               1.13.0
arista.eos                    6.0.0
awx.awx                       21.11.0
azure.azcollection            1.14.0
check_point.mgmt              4.0.0
chocolatey.chocolatey         1.4.0
cisco.aci                     2.3.0
cisco.asa                     4.0.0
cisco.dnac                    6.6.3
cisco.intersight              1.0.23
cisco.ios                     4.3.1
cisco.iosxr                   4.1.0
cisco.ise                     2.5.12
cisco.meraki                  2.15.0
cisco.mso                     2.2.1
cisco.nso                     1.0.3
cisco.nxos                    4.0.1
cisco.ucs                     1.8.0
cloud.common                  2.1.2
cloudscale_ch.cloud           2.2.4
community.aws                 5.2.0
community.azure               2.0.0
community.ciscosmb            1.0.5
community.crypto              2.10.0
community.digitalocean        1.23.0
community.dns                 2.5.0
community.docker              3.4.0
community.fortios             1.0.0
community.general             6.3.0
community.google              1.0.0
community.grafana             1.5.3
community.hashi_vault         4.1.0
community.hrobot              1.7.0
community.libvirt             1.2.0
community.mongodb             1.4.2
community.mysql               3.5.1
community.network             5.0.0
community.okd                 2.2.0
community.postgresql          2.3.2
community.proxysql            1.5.1
community.rabbitmq            1.2.3
community.routeros            2.7.0
community.sap                 1.0.0
community.sap_libs            1.4.0
community.skydive             1.0.0
community.sops                1.6.0
community.vmware              3.3.0
community.windows             1.12.0
community.zabbix              1.9.1
containers.podman             1.10.1
cyberark.conjur               1.2.0
cyberark.pas                  1.0.17
dellemc.enterprise_sonic      2.0.0
dellemc.openmanage            6.3.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
dellemc.powerflex             1.5.0
dellemc.unity                 1.5.0
f5networks.f5_modules         1.22.0
fortinet.fortimanager         2.1.7
fortinet.fortios              2.2.2
frr.frr                       2.0.0
gluster.gluster               1.0.2
google.cloud                  1.1.2
grafana.grafana               1.1.0
hetzner.hcloud                1.9.1
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.11.0
infinidat.infinibox           1.3.12
infoblox.nios_modules         1.4.1
inspur.ispim                  1.2.0
inspur.sm                     2.3.0
junipernetworks.junos         4.1.0
kubernetes.core               2.3.2
lowlydba.sqlserver            1.3.1
mellanox.onyx                 1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0
netapp.ontap                  22.2.0
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0
netapp_eseries.santricity     1.4.0
netbox.netbox                 3.10.0
ngine_io.cloudstack           2.3.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.3
openstack.cloud               1.10.0
openvswitch.openvswitch       2.1.0
ovirt.ovirt                   2.4.1
purestorage.flasharray        1.16.2
purestorage.flashblade        1.10.0
purestorage.fusion            1.3.0
sensu.sensu_go                1.13.2
splunk.es                     2.1.0
t_systems_mms.icinga_director 1.32.0
theforeman.foreman            3.8.0
vmware.vmware_rest            2.2.0
vultr.cloud                   1.7.0
vyos.vyos                     4.0.0
wti.remote                    1.0.4


### AWS SDK versions

Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /home/centos/.local/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.26.66
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/centos/.local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.29.66
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/centos/.local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: awscli, boto3, s3transfer


### Configuration

CONFIG_FILE() = /home/centos/ansible/ansible.cfg
DEFAULT_HOST_LIST(/home/centos/ansible/ansible.cfg) = ['/home/centos/ansible/inventory']
DEFAULT_ROLES_PATH(/home/centos/ansible/ansible.cfg) = ['/home/centos/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/home/centos/ansible/ansible.cfg) = yaml
HOST_KEY_CHECKING(/home/centos/ansible/ansible.cfg) = False
INVENTORY_ENABLED(/home/centos/ansible/ansible.cfg) = ['ini']


### OS / Environment

CentOS Stream 8

### Steps to Reproduce

Using Ansible 7.2 and amazon.aws collection version 5.2.0

Store AWS creds in environment vars
```
export AWS_ACCESS_KEY_ID="XXXXXXX"
export AWS_SECRET_KEY="XXXXXX"
export AWS_REGION="XXXXXX"
```

Setup AWS profile config file:
```
[default]
region = xxxxxxxxxxxxxxxxx
[profile testing]
role_arn = arn:aws:iam::xxxxxxxxxxxxx:role/testing-role
credential_source = Environment
region =xxxxxxxxxxxxxxxx
[profile testing2]
role_arn = arn:aws:iam::xxxxxxxxxxx:role/testing-role
credential_source = Environment
region = xxxxxxxxxxxxxxxxxx
```

Add Ansible code
```
---
- name: Example play for test
  hosts: localhost
  gather_facts: no
  become: no


  tasks:
 
    - name: Test EC2 State Change
      amazon.aws.ec2_instance:
        profile: testing
        state: running
        instance_ids: i-xxxxxxxx
```
Run ansible-playbook

`ansible-playbook example-play.yml`


### Expected Results

Authenticate to AWS using the profile instead of the AWS credentials stored in environment vars
Successfully run the actions in the amazon.aws.ec2_instance task

### Actual Results

Using amazon.aws 5.2.0
```fatal: [localhost]: FAILED! => changed=false
  msg: Passing both a profile and access tokens is not supported.
```
Using amazon.aws 5.1.0
Task completes successfully as expected

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Thanks for taking the time to open this issue.

The triggering PR was #1224.  However, support for passing both profile and credentials was deprecated in release 1.2.0 (back in 2020), and officially "removed" in release 5.0.0.  This includes passing them through both environment variables and parameters.

Unfortunately, due to some messy logic paths in the original code it looks like when removing support we only partially dropped support.  With #1224 the logic was simplified and the change exposed the dropped support that you've now encountered.

Since this breaking change appeared in a non-major release I'm going to revert it for the remainder of the 5.x release cycle.  However, please note that this behaviour will return in release 6.0.0.

Resolving patch:
diff --git a/plugins/doc_fragments/aws.py b/plugins/doc_fragments/aws.py
--- a/plugins/doc_fragments/aws.py
+++ b/plugins/doc_fragments/aws.py
@@ -19,7 +19,9 @@ class ModuleDocFragment(object):
         U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
       - The C(AWS_ACCESS_KEY_ID), C(AWS_ACCESS_KEY) or C(EC2_ACCESS_KEY)
         environment variables may also be used in decreasing order of
-        preference.
+        preference.  Prior to release 6.0.0 these environment variables will be
+        ignored if the I(profile) parameter is passed.  After release 6.0.0
+        I(access_key) will always fall back to the environment variables if set.
       - The I(aws_access_key) and I(profile) options are mutually exclusive.
       - The I(aws_access_key_id) alias was added in release 5.1.0 for
         consistency with the AWS botocore SDK.
@@ -36,7 +38,9 @@ class ModuleDocFragment(object):
         U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
       - The C(AWS_SECRET_ACCESS_KEY), C(AWS_SECRET_KEY), or C(EC2_SECRET_KEY)
         environment variables may also be used in decreasing order of
-        preference.
+        preference.  Prior to release 6.0.0 these environment variables will be
+        ignored if the I(profile) parameter is passed.  After release 6.0.0
+        I(secret_key) will always fall back to the environment variables if set.
       - The I(secret_key) and I(profile) options are mutually exclusive.
       - The I(aws_secret_access_key) alias was added in release 5.1.0 for
         consistency with the AWS botocore SDK.
@@ -53,6 +57,9 @@ class ModuleDocFragment(object):
         U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
       - The C(AWS_SESSION_TOKEN), C(AWS_SECURITY_TOKEN) or C(EC2_SECURITY_TOKEN)
         environment variables may also be used in decreasing order of preference.
+        Prior to release 6.0.0 these environment variables will be
+        ignored if the I(profile) parameter is passed.  After release 6.0.0
+        I(session_token) will always fall back to the environment variables if set.
       - The I(security_token) and I(profile) options are mutually exclusive.
       - Aliases I(aws_session_token) and I(session_token) were added in release
         3.2.0, with the parameter being renamed from I(security_token) to
@@ -70,7 +77,10 @@ class ModuleDocFragment(object):
       - A named AWS profile to use for authentication.
       - See the AWS documentation for more information about named profiles
         U(https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html).
-      - The C(AWS_PROFILE) environment variable may also be used.
+      - The C(AWS_PROFILE) environment variable may also be used.  Prior to release 6.0.0 the
+        C(AWS_PROFILE) environment variable will be ignored if any of I(access_key), I(secret_key),
+        or I(session_token) are passed.  After release 6.0.0 I(profile) will always fall back to the
+        C(AWS_PROFILE) environment variable if set.
       - The I(profile) option is mutually exclusive with the I(aws_access_key),
         I(aws_secret_key) and I(security_token) options.
     type: str
diff --git a/plugins/module_utils/botocore.py b/plugins/module_utils/botocore.py
--- a/plugins/module_utils/botocore.py
+++ b/plugins/module_utils/botocore.py
@@ -148,9 +148,7 @@ def get_aws_region(module, boto3=None):
 
     # here we don't need to make an additional call, will default to 'us-east-1' if the below evaluates to None.
     try:
-        # Botocore doesn't like empty strings, make sure we default to None in the case of an empty
-        # string.
-        profile_name = module.params.get('profile') or None
+        profile_name = module.params.get('profile')
         return botocore.session.Session(profile=profile_name).get_config_variable('region')
     except botocore.exceptions.ProfileNotFound:
         return None
@@ -171,25 +169,64 @@ def get_aws_connection_info(module, boto3=None):
     ca_bundle = module.params.get('aws_ca_bundle')
     config = module.params.get('aws_config')
 
+    # Only read the profile environment variables if we've *not* been passed
+    # any credentials as parameters.
+    if not profile_name and not access_key and not secret_key:
+        if os.environ.get('AWS_PROFILE'):
+            profile_name = os.environ.get('AWS_PROFILE')
+        if os.environ.get('AWS_DEFAULT_PROFILE'):
+            profile_name = os.environ.get('AWS_DEFAULT_PROFILE')
+
     if profile_name and (access_key or secret_key or session_token):
         module.fail_json(msg="Passing both a profile and access tokens is not supported.")
 
     # Botocore doesn't like empty strings, make sure we default to None in the case of an empty
     # string.
     if not access_key:
-        access_key = None
+        # AWS_ACCESS_KEY_ID is the one supported by the AWS CLI
+        # AWS_ACCESS_KEY is to match up with our parameter name
+        if os.environ.get('AWS_ACCESS_KEY_ID'):
+            access_key = os.environ['AWS_ACCESS_KEY_ID']
+        elif os.environ.get('AWS_ACCESS_KEY'):
+            access_key = os.environ['AWS_ACCESS_KEY']
+        # Deprecated - 'EC2' implies just EC2, but is global
+        elif os.environ.get('EC2_ACCESS_KEY'):
+            access_key = os.environ['EC2_ACCESS_KEY']
+        else:
+            # in case access_key came in as empty string
+            access_key = None
+
     if not secret_key:
-        secret_key = None
+        # AWS_SECRET_ACCESS_KEY is the one supported by the AWS CLI
+        # AWS_SECRET_KEY is to match up with our parameter name
+        if os.environ.get('AWS_SECRET_ACCESS_KEY'):
+            secret_key = os.environ['AWS_SECRET_ACCESS_KEY']
+        elif os.environ.get('AWS_SECRET_KEY'):
+            secret_key = os.environ['AWS_SECRET_KEY']
+        # Deprecated - 'EC2' implies just EC2, but is global
+        elif os.environ.get('EC2_SECRET_KEY'):
+            secret_key = os.environ['EC2_SECRET_KEY']
+        else:
+            # in case secret_key came in as empty string
+            secret_key = None
+
     if not session_token:
-        session_token = None
+        # AWS_SESSION_TOKEN is supported by the AWS CLI
+        if os.environ.get('AWS_SESSION_TOKEN'):
+            session_token = os.environ['AWS_SESSION_TOKEN']
+        # Deprecated - boto
+        elif os.environ.get('AWS_SECURITY_TOKEN'):
+            session_token = os.environ['AWS_SECURITY_TOKEN']
+        # Deprecated - 'EC2' implies just EC2, but is global
+        elif os.environ.get('EC2_SECURITY_TOKEN'):
+            session_token = os.environ['EC2_SECURITY_TOKEN']
+        else:
+            # in case secret_token came in as empty string
+            session_token = None
 
     if profile_name:
-        boto_params = dict(
-            aws_access_key_id=None,
-            aws_secret_access_key=None,
-            aws_session_token=None,
-            profile_name=profile_name,
-        )
+        boto_params = dict(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None)
+        boto_params['profile_name'] = profile_name
     else:
         boto_params = dict(
             aws_access_key_id=access_key,
diff --git a/plugins/module_utils/modules.py b/plugins/module_utils/modules.py
--- a/plugins/module_utils/modules.py
+++ b/plugins/module_utils/modules.py
@@ -380,7 +380,6 @@ def _aws_common_argument_spec():
             deprecated_aliases=[
                 dict(name='ec2_access_key', date='2024-12-01', collection_name='amazon.aws'),
             ],
-            fallback=(env_fallback, ['AWS_ACCESS_KEY_ID', 'AWS_ACCESS_KEY', 'EC2_ACCESS_KEY']),
             no_log=False,
         ),
         secret_key=dict(
@@ -388,7 +387,6 @@ def _aws_common_argument_spec():
             deprecated_aliases=[
                 dict(name='ec2_secret_key', date='2024-12-01', collection_name='amazon.aws'),
             ],
-            fallback=(env_fallback, ['AWS_SECRET_ACCESS_KEY', 'AWS_SECRET_KEY', 'EC2_SECRET_KEY']),
             no_log=True,
         ),
         session_token=dict(
@@ -398,12 +396,10 @@ def _aws_common_argument_spec():
                 dict(name='security_token', date='2024-12-01', collection_name='amazon.aws'),
                 dict(name='aws_security_token', date='2024-12-01', collection_name='amazon.aws'),
             ],
-            fallback=(env_fallback, ['AWS_SESSION_TOKEN', 'AWS_SECURITY_TOKEN', 'EC2_SECURITY_TOKEN']),
             no_log=True,
         ),
         profile=dict(
             aliases=['aws_profile'],
-            fallback=(env_fallback, ['AWS_PROFILE', 'AWS_DEFAULT_PROFILE']),
         ),
 
         endpoint_url=dict(


Test patch: