Repository: ansible/ansible

Issue:
Ansible not IPv6 ready.
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
Issue appears in modules `get_url` and `apt_key` but they seem to rely on shared code so this issue might be broader than these two modules we've encountered it in.

##### ANSIBLE VERSION
```
ansible 2.4.0 (devel 41c1457823) last updated 2017/07/13 10:59:46 (GMT +200)
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
Ubuntu Xenial Xerus 16.04.2
**The managed box only has IPv6 network connectivity**

##### SUMMARY
Trying to use the `get_url` module on a managed box that only has IPv6 network connectivity to retrieve a file from an IPv6 connected webserver on the internet results in <code>[Errno 101] Network is unreachable</code>.

If the webserver on the internet *ONLY* has an IPv6 record the get_url module returns <code>[Errno -2] Name or service not known</code>.

##### STEPS TO REPRODUCE
Have an IPv6 only connected box, run the following playbook on it:
```yaml
- name: Test!
  gather_facts: no
  hosts: all
  tasks:
   - name: get_url
     get_url:
       url: "https://packagecloud.io/intelsdi-x/snap/gpgkey"
       dest: /root/gpgkey-intelsdi-x
       mode: 0440
```

This exact same playbook works fine on hosts that have both IPv4 and IPv6 connectivity.
Running the <code>apt_key</code> module on this IPv6-only connected box results in the same <code>network unreachable</code> message.

To test this, one could use <code>ip4.8n1.org</code> (only IPv4) <code>ip6.8n1.org</code> (only IPv6) and <code>ip.8n1.org</code> (both).

##### EXPECTED RESULTS
Fetches the resource pointed to, regardless of IP version.

##### ACTUAL RESULTS
Fails to fetch the resource pointed to, because it only supports IPv4.

```
root@box:~/sanders/.ansible/lib/ansible/modules/network/basics# python get_url.py
{"ANSIBLE_MODULE_ARGS": {"directory_mode": null, "force": false, "backup": false, "remote_src": null, "owner": null, "follow": false, "group": null, "use_proxy": true, "unsafe_writes": null, "setype": null, "content": null, "serole": null, "timeout": 10, "src": null, "dest": "/root/gpgkey-intelsdi-x", "selevel": null, "force_basic_auth": false, "sha256sum": "", "http_agent": "ansible-httpget", "regexp": null, "url_password": null, "url": "https://packagecloud.io/intelsdi-x/snap/gpgkey", "checksum": "", "seuser": null, "headers": null, "delimiter": null, "mode": 288, "url_username": null, "attributes": null, "validate_certs": true, "tmp_dest": ""}}

{"msg": "Failed to connect to packagecloud.io at port 443: [Errno 101] Network is unreachable", "failed": true, "invocation": {"module_args": {"directory_mode": null, "force": false, "remote_src": null, "owner": null, "follow": false, "group": null, "use_proxy": true, "unsafe_writes": null, "setype": null, "content": null, "validate_certs": true, "serole": null, "url_username": null, "url_password": null, "dest": "/root/gpgkey-intelsdi-x", "selevel": null, "force_basic_auth": false, "sha256sum": "", "http_agent": "ansible-httpget", "regexp": null, "src": null, "url": "https://packagecloud.io/intelsdi-x/snap/gpgkey", "checksum": "", "seuser": null, "headers": null, "delimiter": null, "mode": 288, "timeout": 10, "attributes": null, "backup": false, "tmp_dest": ""}}}
```

**Please let me know if you need more information, or to test stuff out on IPv6 only boxes**


Discussion:
@sndrsmnk I don't like to point people to Google searches (you may as well have done this yourself), but in this case there were some pages hinting to the IPV6 setup and things to try: [google: python "Network is unreachable" ipv6](https://www.google.be/search?q=python+%22Network+is+unreachable%22+ipv6)

PS There is also a link to issue #4482
<code>apt_key</code> and <code>get_url</code> each eventually call <code>fetch_url</code> from <code>lib/ansible/module_utils/urls.py</code>.

I'm using Python 2.7.12

This snippet works fine:
```sh
root@mds1:~/sanders/ansible/lib/ansible/module_utils# python --version
Python 2.7.12

root@mds1:~# cat test 
import urllib2
response = urllib2.urlopen('https://ip6.8n1.org')
html = response.read()
print html

root@mds1:~# python test
2001:db8:1::2:3:4
```
So i doubt it's Python, or the connectivity on this box.

cc @jpmens
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@sndrsmnk I wasn't saying python was the problem.

```
[dag@moria ansible.git]$ python test.py
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    response = urllib2.urlopen('https://ip6.8n1.org')
  File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 1258, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 101] Network is unreachable>
```

So there may still be something wrong with the target node.

```
[dag@moria ansible.git]$ wget https://ip6.8n1.org
--2017-07-13 12:03:39--  https://ip6.8n1.org/
Resolving ip6.8n1.org (ip6.8n1.org)... 2001:7b8:633:1:213:154:236:181
Connecting to ip6.8n1.org (ip6.8n1.org)|2001:7b8:633:1:213:154:236:181|:443... failed: Network is unreachable.
```
Leuk dat jullie in groep elkaar steunen op Github, trouwens ;-)
The likes-sprinkling supporters are my colleagues. (Hi colleagues!)
Which goes to show this isn't just me digging around what's going on. ;-)

I ran a HTTP test on <code>ip6.8n1.org</code> via the [NLNOG Ring Project](https://ring.nlnog.net/).
It reports 360 servers all over the world can reach the site just fine.

If you doubt my m4d n3tw0rk sk1llz, which i understand, perhaps try any of the [test-ipv6](https://test-ipv6.com) provided IPv6 only hostnames like for example <code>ipv6.test-ipv6.{com,net,nl,...}</code>.

Also, if you want me to prove the IPv6 connectivity on the managed box is OK, i can run any test you like. The likes-sprinkling supporters and me work for an internet provider and umm... we think we know what we're doing. ;)
This is an issue about the https:// certificate validation being excitability IPv4.

Let me first demonstrate __http://__ vs. __https://__.

```
andreas@i6o:~$ ansible localhost --module-name=get_url --args="url=http://ip6.8n1.org/ dest=$(mktemp -u)"
localhost | SUCCESS => {
    "changed": true, 
    "checksum_dest": null, 
    "checksum_src": "9aea580ed1badc619e6743ac6d1d42702582e610", 
    "dest": "/tmp/tmp.iPtO5bOsgw", 
    "failed": false, 
    "gid": 1000, 
    "group": "andreas", 
    "md5sum": "afa229236886a7d2cd5317ee18b31453", 
    "mode": "0664", 
    "msg": "OK (26 bytes)", 
    "owner": "andreas", 
    "size": 26, 
    "src": "/tmp/tmpRAzAcN", 
    "state": "file", 
    "status_code": 200, 
    "uid": 1000, 
    "url": "http://ip6.8n1.org/"
}
andreas@i6o:~$ ansible localhost --module-name=get_url --args="url=https://ip6.8n1.org/ dest=$(mktemp -u)"
localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "msg": "Failed to connect to ip6.8n1.org at port 443: [Errno -2] Name or service not known"
}
andreas@i6o:~$
```

Followed by __unvalidated https://__ vs. __validated https://__.

```
andreas@i6o:~$ ansible localhost --module-name=get_url --args="url=https://ip6.8n1.org/ dest=$(mktemp -u) validate_certs=no"
localhost | SUCCESS => {
    "changed": true, 
    "checksum_dest": null, 
    "checksum_src": "9aea580ed1badc619e6743ac6d1d42702582e610", 
    "dest": "/tmp/tmp.OfNw9KuqYE", 
    "failed": false, 
    "gid": 1000, 
    "group": "andreas", 
    "md5sum": "afa229236886a7d2cd5317ee18b31453", 
    "mode": "0664", 
    "msg": "OK (26 bytes)", 
    "owner": "andreas", 
    "size": 26, 
    "src": "/tmp/tmpIbPWBr", 
    "state": "file", 
    "status_code": 200, 
    "uid": 1000, 
    "url": "https://ip6.8n1.org/"
}
andreas@i6o:~$ ansible localhost --module-name=get_url --args="url=https://ip6.8n1.org/ dest=$(mktemp -u) validate_certs=yes"
localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "msg": "Failed to connect to ip6.8n1.org at port 443: [Errno -2] Name or service not known"
}
andreas@i6o:~$
```

The culprit being the `ansible.module_utils.urls` `SSLValidationHandler` class wanting to open an `AF_INET` (aka: IPv4) socket.

https://github.com/ansible/ansible/blob/41c14578230272bf5ebc336ceb1519c128b81e0b/lib/ansible/module_utils/urls.py#L723



Resolving patch:
diff --git a/lib/ansible/module_utils/urls.py b/lib/ansible/module_utils/urls.py
--- a/lib/ansible/module_utils/urls.py
+++ b/lib/ansible/module_utils/urls.py
@@ -720,11 +720,10 @@ def http_request(self, req):
             return req
 
         try:
-            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
             if https_proxy:
                 proxy_parts = generic_urlparse(urlparse(https_proxy))
                 port = proxy_parts.get('port') or 443
-                s.connect((proxy_parts.get('hostname'), port))
+                s = socket.create_connection((proxy_parts.get('hostname'), port))
                 if proxy_parts.get('scheme') == 'http':
                     s.sendall(self.CONNECT_COMMAND % (self.hostname, self.port))
                     if proxy_parts.get('username'):
@@ -748,7 +747,7 @@ def http_request(self, req):
                 else:
                     raise ProxyError('Unsupported proxy scheme: %s. Currently ansible only supports HTTP proxies.' % proxy_parts.get('scheme'))
             else:
-                s.connect((self.hostname, self.port))
+                s = socket.create_connection((self.hostname, self.port))
                 if context:
                     ssl_s = context.wrap_socket(s, server_hostname=self.hostname)
                 elif HAS_URLLIB3_SSL_WRAP_SOCKET:


Test patch: