Repository: conan-io/conan

Issue:
[proxies] section in conan.conf does not permit proxies for specific targets
Using conan 1.14 on Linux and Windows

In a large organization, let's call it acme, I have a conan setup with a private shared repository residing in a virtual private cloud. The URL of the repo (cloud.acme.com) makes it appear to be internal, but the only way of accessing it is through a proxy (e.g. http://proxy.acme.com).

Setting HTTPS_PROXY env var prior to executing any conan command works fine and I can access the repo however, if I need to e.g. download some code from an internal source over https as part of a recipe, I have to remember to reset HTTPS_PROXY before doing that download or the downloader will attempt to also use the cloud proxy.

I can't use no_proxy_match as far as I can see because the target URL appears to be local to the organization so there's no simple fnmatch that will exclude all others.

A solution to this problem is suggested in http://docs.python-requests.org/en/latest/user/advanced/#proxies which states that I can include a specific host as well as a scheme when listing proxies. This would imply that adding:

[proxies]
https://conan-repo.cloud.acme.com = http://proxy.acme.com

should do the trick. The problem is that conan.conf has been parsed by ConfigParser which sees the first ':' as a name/value separator and so takes everything to the right (including the =) as being the name of the proxy.

Do you know of another workaround to this?

I guess the simplest fix would be to omit ':' as a valid deliminator for the ConfigParser - and hope that no-one has chosen to use ':'s in their config (or write a migration step). Maybe that could be controlled via an env var?

Alternatively allow multi-line entries and parse them - assuming I've correctly understood how those work?
```
[proxies]
https = http://default-proxy.acme.com
  conan-repo.cloud.acme.com = http://cloud-proxy.acme.com
```
which should be parsed to result in a python 'proxies' argument like:
```python
proxies = {
    'https': 'http://default-proxy.acme.com'
    'https://conan-repo.cloud.acme.com': 'http://cloud-proxy.acme.com'
}
```
In my case I'd be wanting to omit the 'default https proxy' entirely:
```
[proxies]
https =
  conan-repo.cloud.acme.com = http://cloud-proxy.acme.com
```
Yielding
```python
proxies = {
    'https://conan-repo.cloud.acme.com': 'http://cloud-proxy.acme.com'
}
```


Discussion:

Resolving patch:
diff --git a/conans/client/conf/__init__.py b/conans/client/conf/__init__.py
--- a/conans/client/conf/__init__.py
+++ b/conans/client/conf/__init__.py
@@ -150,10 +150,14 @@
 [proxies]
 # Empty section will try to use system proxies.
 # If don't want proxy at all, remove section [proxies]
-# As documented in http://docs.python-requests.org/en/latest/user/advanced/#proxies
+# As documented in http://docs.python-requests.org/en/latest/user/advanced/#proxies - but see below
+# for proxies to specific hosts
 # http = http://user:pass@10.10.1.10:3128/
 # http = http://10.10.1.10:3128
 # https = http://10.10.1.10:1080
+# To specify a proxy for a specific host or hosts, use multiple lines each specifying host = proxy-spec
+# http =
+#   hostname.to.be.proxied.com = http://user:pass@10.10.1.10:3128
 # You can skip the proxy for the matching (fnmatch) urls (comma-separated)
 # no_proxy_match = *bintray.com*, https://myserver.*
 
@@ -418,16 +422,26 @@ def storage_path(self):
 
     @property
     def proxies(self):
-        """ optional field, might not exist
-        """
-        try:
+        try:  # optional field, might not exist
             proxies = self.get_conf("proxies")
-            # If there is proxies section, but empty, it will try to use system proxy
-            if proxies:
-                return {k: (None if v == "None" else v) for k, v in proxies}
-            return {}
         except Exception:
             return None
+        result = {}
+        # Handle proxy specifications of the form:
+        # http = http://proxy.xyz.com
+        #   special-host.xyz.com = http://special-proxy.xyz.com
+        # (where special-proxy.xyz.com is only used as a proxy when special-host.xyz.com)
+        for scheme, proxy_string in proxies or []:
+            if proxy_string is None or proxy_string == "None":
+                result[scheme] = None
+            else:
+                for line in proxy_string.splitlines():
+                    proxy_value = [t.strip() for t in line.split("=", 1)]
+                    if len(proxy_value) == 2:
+                        result[scheme+"://"+proxy_value[0]] = proxy_value[1]
+                    elif proxy_value[0]:
+                        result[scheme] = proxy_value[0]
+        return result
 
     @property
     def cacert_path(self):


Test patch:
diff --git a/conans/test/functional/configuration/proxies_conf_test.py b/conans/test/functional/configuration/proxies_conf_test.py
--- a/conans/test/functional/configuration/proxies_conf_test.py
+++ b/conans/test/functional/configuration/proxies_conf_test.py
@@ -5,6 +5,7 @@
 from conans.client.rest.conan_requester import ConanRequester
 from conans.test.utils.tools import TestClient
 from conans.util.files import save
+import textwrap
 
 
 class ProxiesConfTest(unittest.TestCase):
@@ -21,14 +22,14 @@ def test_requester(self):
 [proxies]
 https=None
 no_proxy=http://someurl,http://otherurl.com
-http=http:/conan.url
+http=http://conan.url
         """
         save(client.cache.conan_conf_path, conf)
         client.cache.invalidate()
         requester = ConanRequester(client.cache)
 
         def verify_proxies(url, **kwargs):
-            self.assertEqual(kwargs["proxies"], {"https": None, "http": "http:/conan.url"})
+            self.assertEqual(kwargs["proxies"], {"https": None, "http": "http://conan.url"})
             return "mocked ok!"
 
         requester._http_requester.get = verify_proxies
@@ -36,11 +37,39 @@ def verify_proxies(url, **kwargs):
 
         self.assertEqual(requester.get("MyUrl"), "mocked ok!")
 
+    def test_requester_with_host_specific_proxies(self):
+        client = TestClient()
+        conf = textwrap.dedent("""
+            [proxies]
+            https=http://conan.url
+              only.for.this.conan.url = http://special.url
+              only.for.that.conan.url = http://user:pass@extra.special.url
+            http=
+              only.for.the.other.conan.url = http://other.special.url
+                    """)
+        save(client.cache.conan_conf_path, conf)
+        client.cache.invalidate()
+        requester = ConanRequester(client.cache)
+
+        def verify_proxies(url, **kwargs):
+            self.assertEqual(kwargs["proxies"],
+                             {"http://only.for.the.other.conan.url": "http://other.special.url",
+                              "https": "http://conan.url",
+                              "https://only.for.this.conan.url": "http://special.url",
+                              "https://only.for.that.conan.url":
+                              "http://user:pass@extra.special.url"})
+            return "mocked ok!"
+
+        requester._http_requester.get = verify_proxies
+        self.assertFalse("NO_PROXY" in os.environ)
+
+        self.assertEqual(requester.get("MyUrl"), "mocked ok!")
+
     def new_proxy_exclude_test(self):
 
         class MyRequester(object):
 
-            def __init__(*args, **kwargs):
+            def __init__(self, *args, **kwargs):
                 pass
 
             def get(self, _, **kwargs):