Repository: ansible/ansible

Issue:
Yum module fails when installing via URL if a yum http_proxy is required
_From @JamesUoM on November 29, 2016 13:38_

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
yum

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.1.2.0 (v2.1.2.0-1 29f2f26278) last updated 2016/11/28 11:09:19 (GMT +100)
  lib/ansible/modules/core: (detached HEAD 17ee1cfaf9) last updated 2016/11/28 11:58:46 (GMT +100)
  lib/ansible/modules/extras: (detached HEAD d312f34d9b) last updated 2016/11/28 11:58:47 (GMT +100)
  config file = /home/ansible/opencast-ansible/ansible.cfg
  configured module search path = ['/home/ansible/src/ansible/lib/ansible/modules/core', '/home/ansible/src/ansible/lib/anisble/modules/extra']

```

##### OS / ENVIRONMENT
Red Hat 6

##### SUMMARY
The Yum module fails when trying make sure an RPM is installed via a URL if it must do so via a http proxy.

I found that line 246 in the yum module it tries to fetch the url directly possibly circumventing the proxy settings: 
https://github.com/ansible/ansible-modules-core/blob/c67315fc4e0c3bc5cb519ef2651cccf4bc659780/packaging/os/yum.py#L246

Note: the proxy settings are also set in /etc/profile.d/proxy.sh but I guess these are not picked up when when the remote module code is executed.

##### STEPS TO REPRODUCE
myserver must use a http_proxy which has been correctly set in  /etc/yum.conf

I'm using a playbook but the single command also fails:

```
ansible myserver -i inventory/ -s -m yum -a "state=present name=http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm"
```

Note: it doesn't matter if the rpm is already cached  in /var/tmp/yum-### or already installed or not.
Note: running the yum command below on myserver works as expected.
 
```
sudo yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
````
##### EXPECTED RESULTS
SUCCESS or Changed

##### ACTUAL RESULTS
Fails to confirm that it is installed
```
fatal: [myserver]: FAILED! => {"changed": false, "failed": true, "msg": "Failure downloading http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm, 'NoneType' object has no attribute 'read'"}
```


_Copied from original issue: ansible/ansible-modules-core#5757_


Discussion:
_From @ansibot on November 29, 2016 13:48_

<!-- @JamesUoM: Thanks for filing this bug report! -->
@ansible ping, this issue is waiting for your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: issue_bug_new --->
A workaround it to specify the http_proxy as an environment variable.
@JamesUoM just to clarify, does the download (wget?) works when you download the package locally?

> A workaround it to specify the http_proxy as an environment variable.

Alternatively (the same concept):
I was using a command module like:
`  command: http_proxy=http:/somehost:3128 wget http://something`
Yes, wget works but also needs the http_proxy set.

The problem is that I think the yum module is getting the url by itself and not via yum and therefore bypassing the proxy set in yum.conf
I am experiencing the same issue with a CentOS 7 host. I'm using ansible 2.2.1.0, and my proxy has been set both in `/etc/yum.conf` and the `(http|https|ftp)_proxy` environment variables (via `/etc/profile.d/proxy.sh`).
I mentioned a workaround but realise it is not perhaps clear that you can set the http_proxy env for the task:

```
- name: install foo
  yum: name=http://foorepo.com/foo.rpm state=present
  environment:
      http_proxy: http://myproxy.com
```
This is still a problem in ansible 2.3.1.0, from RHEL host to RHEL nodes.
cc @berenddeschouwer @verm666
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
cc @mkrizek
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
cc @akasurde
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Also Experiencing this issue, yum proxy is set in /etc/yum.conf and working, but not with Ansible. 
- Centos 7.4
- Ansible 2.3.2.0
It's because in ansible, for installing from internet, used **fetch_rpm_from_url** function, which doesn't work with proxy setting
https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/packaging/os/yum.py#L735-L736
and for proxy used http(s)_proxy environment

This is a major issue for us, as only parts of the hosts connect via proxies, others don't.  The proxies are correctly configured in /etc/yum.conf but ignored by the yum module.  Having the playbook to set (or not set) this via environment seems too much of an overhead.  Why configure again which is already configured?  That's not the proper way to do it.

Resolving patch:
diff --git a/lib/ansible/modules/packaging/os/yum.py b/lib/ansible/modules/packaging/os/yum.py
--- a/lib/ansible/modules/packaging/os/yum.py
+++ b/lib/ansible/modules/packaging/os/yum.py
@@ -265,6 +265,8 @@
 except ImportError:
     transaction_helpers = False
 
+from contextlib import contextmanager
+
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils._text import to_native
 from ansible.module_utils.urls import fetch_url
@@ -639,6 +641,37 @@ def local_envra(path):
                                header[rpm.RPMTAG_ARCH])
 
 
+@contextmanager
+def set_env_proxy(conf_file, installroot):
+    # setting system proxy environment and saving old, if exists
+    my = yum_base(conf_file, installroot)
+    namepass = ""
+    scheme = ["http", "https"]
+    old_proxy_env = [os.getenv("http_proxy"), os.getenv("https_proxy")]
+    try:
+        if my.conf.proxy:
+            if my.conf.proxy_username:
+                namepass = namepass + my.conf.proxy_username
+                if my.conf.proxy_password:
+                    namepass = namepass + ":" + my.conf.proxy_password
+            namepass = namepass + '@'
+            for item in scheme:
+                os.environ[item + "_proxy"] = re.sub(r"(http://)",
+                                                     r"\1" + namepass, my.conf.proxy)
+        yield
+    except yum.Errors.YumBaseError:
+        raise
+    finally:
+        # revert back to previously system configuration
+        for item in scheme:
+            if os.getenv("{0}_proxy".format(item)):
+                del os.environ["{0}_proxy".format(item)]
+        if old_proxy_env[0]:
+            os.environ["http_proxy"] = old_proxy_env[0]
+        if old_proxy_env[1]:
+            os.environ["https_proxy"] = old_proxy_env[1]
+
+
 def pkg_to_dict(pkgstr):
     if pkgstr.strip():
         n, e, v, r, a, repo = pkgstr.split('|')
@@ -761,7 +794,8 @@ def install(module, items, repoq, yum_basecmd, conf_file, en_repos, dis_repos, i
                 module.fail_json(**res)
 
             if '://' in spec:
-                package = fetch_rpm_from_url(spec, module=module)
+                with set_env_proxy(conf_file, installroot):
+                    package = fetch_rpm_from_url(spec, module=module)
             else:
                 package = spec
 
@@ -1074,7 +1108,8 @@ def latest(module, items, repoq, yum_basecmd, conf_file, en_repos, dis_repos, up
             # URL
             elif '://' in spec:
                 # download package so that we can check if it's already installed
-                package = fetch_rpm_from_url(spec, module=module)
+                with set_env_proxy(conf_file, installroot):
+                    package = fetch_rpm_from_url(spec, module=module)
                 envra = local_envra(package)
 
                 if envra is None:


Test patch: