Repository: ansible-collections/community.aws

Issue:
unexpected output from Python interpreter discovery with aws_ssm connection plugin
### Summary

I had good luck with the aws_ssm plugin until attempting to use it against the latest Amazon Linux AMI.  Simple commands that work well with a CentOS 7 host fails when trying to run them against AMI.   It appears to be something to do with interpreting shell output. 

A 'raw' command like this works fine:

```
ansible -i inventory.aws_ec2.yaml -m 'raw' -a 'whoami' tag_role_FAKEMX
```

However when running the equivalent 'command' module it fails for Amazon Linux, but works on CentOS 7.

```
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ec2-13-58-203-89.us-east-2.compute.amazonaws.com
[WARNING]: Unhandled error in Python interpreter discovery for host ec2-13-58-203-89.us-east-2.compute.amazonaws.com: unexpected output from Python interpreter discovery
ec2-13-58-203-89.us-east-2.compute.amazonaws.com | FAILED | rc=-1 >>
failed to transfer file to /Users/username/.ansible/tmp/ansible-local-78807y4966nip/tmpqn_vbdbk /AnsiballZ_command.py:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file
Warning: or directory
  1  129k    1  1531    0     0  19698      0  0:00:06 --:--:--  0:00:06 19883
curl: (23) Failure writing output to destination
```

Versus CentOS 7:

```
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ip-10-240-172-59.us-east-2.compute.internal
[WARNING]: Platform linux on host ip-10-240-172-59.us-east-2.compute.internal is using the discovered Python interpreter at /usr/libexec/platform-python, but future installation of another Python interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-
core/2.14/reference_appendices/interpreter_discovery.html for more information.
ip-10-240-172-59.us-east-2.compute.internal | CHANGED | rc=0 >>
root
```

From '-vvvv' output I see things like this:

```
<i-0cc859c89f4aaf5f4> ssm_retry: (success) (0, '\x1b[?2004h\x1b[?2004l\r\r\r\nPLATFORM\r\r\nLinux\r\r\nFOUND\r\r\n/usr/bin/python3.9\r\r\n/usr/bin/python3\r\r\nENDFOUND\r\r\n\x1b[?2004h\x1b[?2004l\r\r\r', '')
[WARNING]: Unhandled error in Python interpreter discovery for host ec2-13-58-203-89.us-east-2.compute.amazonaws.com: unexpected output from Python interpreter discovery
Using module file /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/ansible/modules/command.py
```

That's a failure to find the python version.  Versus: 

```
<i-03c9cbe64572b3eb0> ssm_retry: (success) (0, 'PLATFORM\r\r\nLinux\r\r\nFOUND\r\r\n/usr/libexec/platform-python\r\r\n/usr/bin/python2.7\r\r\n/usr/bin/python\r\r\n/usr/bin/python\r\r\nENDFOUND\r\r', '')
<ip-10-240-172-59.us-east-2.compute.internal> Python interpreter discovery fallback (pipelining support required for extended interpreter discovery)`
```

Where it didn't complain and seems to be successful. Notice the additional "x1b[?2004h\x1b[?2004" strings in the output.

I see the similar string in other problems, like "Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py"




Note that if I set the python interpreter it will remove the warning, but it will still error out on the "Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file Warning: or directory" 


Tried this with community.aws collection versions 4.5.0, 5.2.0, and 5.3.0 and the errors is the same every time.   Also tried a few different amazon.aws collection versions and had the same error. 

These examples are taken from a Mac running python 3.11.2 and ansible 2.14.2, but the same errors occured in a Linux-based Execution Environment for AWX running in EKS. 

### Issue Type

Bug Report

### Component Name

aws_ssm connection

### Ansible Version

```
ansible [core 2.14.2]
  config file = None
  configured module search path = ['/Users/username/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/ansible
  ansible collection location = /Users/username/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/username/.asdf/installs/python/3.11.2/bin/ansible
  python version = 3.11.2 (main, Feb 21 2023, 11:07:56) [Clang 13.1.6 (clang-1316.0.21.2.5)] (/Users/username/.asdf/installs/python/3.11.2/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True
```


### Collection Versions

```# /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
ansible.netcommon             4.1.0
ansible.posix                 1.5.1
ansible.utils                 2.9.0
ansible.windows               1.13.0
arista.eos                    6.0.0
awx.awx                       21.11.0
azure.azcollection            1.14.0
check_point.mgmt              4.0.0
chocolatey.chocolatey         1.4.0
cisco.aci                     2.3.0
cisco.asa                     4.0.0
cisco.dnac                    6.6.3
cisco.intersight              1.0.23
cisco.ios                     4.3.1
cisco.iosxr                   4.1.0
cisco.ise                     2.5.12
cisco.meraki                  2.15.0
cisco.mso                     2.2.1
cisco.nso                     1.0.3
cisco.nxos                    4.0.1
cisco.ucs                     1.8.0
cloud.common                  2.1.2
cloudscale_ch.cloud           2.2.4
community.azure               2.0.0
community.ciscosmb            1.0.5
community.crypto              2.10.0
community.digitalocean        1.23.0
community.dns                 2.5.0
community.docker              3.4.0
community.fortios             1.0.0
community.general             6.3.0
community.google              1.0.0
community.grafana             1.5.3
community.hashi_vault         4.1.0
community.hrobot              1.7.0
community.libvirt             1.2.0
community.mongodb             1.4.2
community.mysql               3.5.1
community.network             5.0.0
community.okd                 2.2.0
community.postgresql          2.3.2
community.proxysql            1.5.1
community.rabbitmq            1.2.3
community.routeros            2.7.0
community.sap                 1.0.0
community.sap_libs            1.4.0
community.skydive             1.0.0
community.sops                1.6.0
community.vmware              3.3.0
community.windows             1.12.0
community.zabbix              1.9.1
containers.podman             1.10.1
cyberark.conjur               1.2.0
cyberark.pas                  1.0.17
dellemc.enterprise_sonic      2.0.0
dellemc.openmanage            6.3.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
dellemc.powerflex             1.5.0
dellemc.unity                 1.5.0
f5networks.f5_modules         1.22.0
fortinet.fortimanager         2.1.7
fortinet.fortios              2.2.2
frr.frr                       2.0.0
gluster.gluster               1.0.2
google.cloud                  1.1.2
grafana.grafana               1.1.0
hetzner.hcloud                1.9.1
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.11.0
infinidat.infinibox           1.3.12
infoblox.nios_modules         1.4.1
inspur.ispim                  1.2.0
inspur.sm                     2.3.0
junipernetworks.junos         4.1.0
kubernetes.core               2.3.2
lowlydba.sqlserver            1.3.1
mellanox.onyx                 1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0
netapp.ontap                  22.2.0
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0
netapp_eseries.santricity     1.4.0
netbox.netbox                 3.10.0
ngine_io.cloudstack           2.3.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.3
openstack.cloud               1.10.0
openvswitch.openvswitch       2.1.0
ovirt.ovirt                   2.4.1
purestorage.flasharray        1.16.2
purestorage.flashblade        1.10.0
purestorage.fusion            1.3.0
sensu.sensu_go                1.13.2
splunk.es                     2.1.0
t_systems_mms.icinga_director 1.32.0
theforeman.foreman            3.8.0
vmware.vmware_rest            2.2.0
vultr.cloud                   1.7.0
vyos.vyos                     4.0.0
wti.remote                    1.0.4

# /Users/username/.ansible/collections/ansible_collections
Collection    Version
------------- -------
amazon.aws    4.3.0
community.aws 5.2.0
```


### AWS SDK versions

```
❯ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.26.76
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.29.76
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
```


### Configuration

```
❯ ansible-config dump --only-changed
CONFIG_FILE() = None
```


### OS / Environment

Darwin ENG-NMOSEMAN-MB 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 PST 2023; root:xnu-8792.81.3~2/RELEASE_X86_64 x86_64 i386 Darwin

and 

modified version of quay.io/ansible/awx-ee:latest to include community.aws, and ssm stuff.

### Steps to Reproduce

```
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ec2-13-58-203-89.us-east-2.compute.amazonaws.com
[WARNING]: Unhandled error in Python interpreter discovery for host ec2-13-58-203-89.us-east-2.compute.amazonaws.com: unexpected output from Python interpreter discovery
ec2-13-58-203-89.us-east-2.compute.amazonaws.com | FAILED | rc=-1 >>
failed to transfer file to /Users/username/.ansible/tmp/ansible-local-78807y4966nip/tmpqn_vbdbk /AnsiballZ_command.py:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file
Warning: or directory
  1  129k    1  1531    0     0  19698      0  0:00:06 --:--:--  0:00:06 19883
curl: (23) Failure writing output to destination
```


### Expected Results

```
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ip-10-240-172-59.us-east-2.compute.internal
ip-10-240-172-59.us-east-2.compute.internal | CHANGED | rc=0 >>
root
```

### Actual Results

```failed to transfer file to /Users/username/.ansible/tmp/ansible-local-78807y4966nip/tmpqn_vbdbk /AnsiballZ_command.py:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file
Warning: or directory
  1  129k    1  1531    0     0  19698      0  0:00:06 --:--:--  0:00:06 19883
curl: (23) Failure writing output to destination
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
We just ran into this issue here. It seems that Amazon Linux outputs colorized text when Ansible runs any remote shell commands which causes parsing of the result to fail. Our solution was to not use aws_ssm connection - instead we setup SSH to make connections through Session Manager: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html
I was facing this same issue with hosts running both Ubuntu 22.04 and Amazon Linux 2023, and I was finally able to trace the extra output to a root cause. This is due to newer versions of Bash/readline turning on the option `enable-bracketed-paste` by default (more details [here](https://utcc.utoronto.ca/~cks/space/blog/unix/BashBracketedPasteChange)).

I have a patch that will disable this option (will send a PR later today), which allows `ansible -m ping` to work on several hosts I have access to, including Ubuntu 18.04, Amazon Linux 2023, and Ubuntu 22.04.
> https://github.com/ansible-collections/community.aws/pull/1839

Confirmed it's working with Amazon Linux
I've ended up having the same issue switching from amazon Linux 2 to amazon Linux 2023, my error was:
```
service-use1-bh | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo \u001b[?2004h\u001b[?2004l/.ansible/tmp `\"&& mkdir \"` echo \u001b[?2004h\u001b[?2004l/.ansible/tmp/ansible-tmp-1694091988.5261781-5952-41494157774678 `\" && echo ansible-tmp-1694091988.5261781-5952-41494157774678=\"` echo \u001b[?2004h\u001b[?2004l/.ansible/tmp/ansible-tmp-1694091988.5261781-5952-41494157774678 `\" ), exited with result 1, stdout output: \u001b[?2004h\u001b[?2004l\r\r\r\nmkdir: cannot create directory ‘2004h2004l’: Permission denied\r\r\n\u001b[?2004h\u001b[?2004l\r\r\r",
    "unreachable": true
}
```

https://github.com/ansible-collections/community.aws/pull/1839 fixes it, but it's taking ages to be merged for unknown reasons. In the meantime I've set `set enable-bracketed-paste off` into `/etc/inputrc` which is, needless to say, not a fix at all since you need to configure all servers this way, which is exactly what ansible is meant to do. In my case was just one so _for now_ it's sorted, thanks to @dennisjlee !


Resolving patch:
diff --git a/plugins/connection/aws_ssm.py b/plugins/connection/aws_ssm.py
--- a/plugins/connection/aws_ssm.py
+++ b/plugins/connection/aws_ssm.py
@@ -627,7 +627,7 @@ def _prepare_terminal(self):
         disable_prompt_complete = None
         end_mark = "".join([random.choice(string.ascii_letters) for i in xrange(self.MARK_LENGTH)])
         disable_prompt_cmd = to_bytes(
-            "PS1='' ; printf '\\n%s\\n' '" + end_mark + "'\n",
+            "PS1='' ; bind 'set enable-bracketed-paste off'; printf '\\n%s\\n' '" + end_mark + "'\n",
             errors="surrogate_or_strict",
         )
         disable_prompt_reply = re.compile(r"\r\r\n" + re.escape(end_mark) + r"\r\r\n", re.MULTILINE)


Test patch: