Repository: ansible/ansible

Issue:
ec2_metadata_facts.py seems doesn't properly handle spaces in SSH keys
##### SUMMARY
ec2_metadata_facts.py fails with HTTP Error 400: Bad Request response.

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
lib/ansible/modules/cloud/amazon/ec2_metadata_facts.py

##### ANSIBLE VERSION
```
ansible --version
ansible 2.6.0
```

##### CONFIGURATION
pretty standard config.

##### OS / ENVIRONMENT
Latest CentOS 7

##### STEPS TO REPRODUCE
```yaml
- action: ec2_metadata_facts
```

##### EXPECTED RESULTS
No errors.

##### ACTUAL RESULTS
```
 [WARNING]: Retrying query to metadata service. First attempt failed: HTTP
Error 400: Bad Request
fatal: [websites_demo]: FAILED! => {"changed": false, "msg": "Failed to retrieve metadata from AWS: HTTP Error 400: Bad Request",
"response": {"body": "<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\"\n         \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"en\" lang=\"en\">\n <head>\n  <title>400 - Bad Request</title>\n </head>\n <body>\n  <h1>400 - Bad Request</h1>\n </body>\n</html>\n", "connection": "close", "content-length": "349", "content-type": "text/html", "date": "Thu, 05 Jul 2018 17:14:16 GMT", "msg": "HTTP Error 400: Bad Request", "server": "EC2ws", "status": 400, "url": "http://169.254.169.254/latest/meta-data/public-keys/0=Super Europe Admin Keys"}}
...

```



Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/ec2_metadata_facts.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/ec2_metadata_facts.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @roadmapper @silviud
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
I believe I've narrowed down the issue; prior to the change made by @ryansb (to retry when the metadata service might be not available), the module just ignored the bad request (400) on public keys that had a space in the name. Now that the retry logic checks for 200 or 404 explicitly, the 400 is not ignored silently.

@ryansb do you think it is ok to accept a 400 status as well? Or should we prevent the call being made with the space in the field name (which would be problematic anyway since none of the other EC2 metadata fields have a space in them)? Either option seems ok to me.
This seems like it's not making the proper request for the public key metadata. Shouldn't it be `http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key` instead of `http://169.254.169.254/latest/meta-data/public-keys/0={key name}`? That's what the docs here indicate: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

Or is the plan to just ignore the public key info?
@dmertl The docs state that the public key can be requested by hitting `http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key`. This is the only way of getting the public key from the metadata service.

Hitting `http://169.254.169.254/latest/meta-data/public-keys/` will enumerate the public keys (which is not really necessary since there can only be one key attached to an EC2 instance as of today). 

Hitting `http://169.254.169.254/latest/meta-data/public-keys/0={key name}` will result in a 404 since metadata service doesn't serve anything on that path.

Where `Test Key` is my SSH key name:
```
curl http://169.254.169.254/latest/meta-data/public-keys/0=Test%20Key
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>
```

Where `test-key` is my SSH key name:
```
curl http://169.254.169.254/latest/meta-data/public-keys/0=test-key
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>
```

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/ec2_metadata_facts.py b/lib/ansible/modules/cloud/amazon/ec2_metadata_facts.py
--- a/lib/ansible/modules/cloud/amazon/ec2_metadata_facts.py
+++ b/lib/ansible/modules/cloud/amazon/ec2_metadata_facts.py
@@ -423,7 +423,7 @@
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils._text import to_text
 from ansible.module_utils.urls import fetch_url
-
+from ansible.module_utils.six.moves.urllib.parse import quote
 
 socket.setdefaulttimeout(5)
 
@@ -444,13 +444,14 @@ def __init__(self, module, ec2_metadata_uri=None, ec2_sshdata_uri=None, ec2_user
         self._prefix = 'ansible_ec2_%s'
 
     def _fetch(self, url):
-        response, info = fetch_url(self.module, url, force=True)
+        encoded_url = quote(url, safe='%/:=&?~#+!$,;\'@()*[]')
+        response, info = fetch_url(self.module, encoded_url, force=True)
 
         if info.get('status') not in (200, 404):
             time.sleep(3)
             # request went bad, retry once then raise
             self.module.warn('Retrying query to metadata service. First attempt failed: {0}'.format(info['msg']))
-            response, info = fetch_url(self.module, url, force=True)
+            response, info = fetch_url(self.module, encoded_url, force=True)
             if info.get('status') not in (200, 404):
                 # fail out now
                 self.module.fail_json(msg='Failed to retrieve metadata from AWS: {0}'.format(info['msg']), response=info)


Test patch: