Repository: conda/conda

Issue:
conda sends bad HTTP headers when querying noarch repos
``conda 4.3`` requires package repositories to provide the ``noarch`` platform directory. After adding ``noarch`` to my custom repository served by Microsoft ``IIS 7.5``, conda started failing after a few runs of ``conda install``:

```
$ conda install conda
Fetching package metadata ....
CondaHTTPError: HTTP 400 BAD REQUEST for url <http://myrepo/packages/noarch/repodata.json>
An HTTP error occurred when trying to retrieve this URL.
HTTPError(u'400 Client Error: Bad Request for url: http://myrepo/packages/noarch/repodata.json',)
```

Inspecting the HTTP headers sent by conda, it seems that the ETag ``If-None-Match`` contains bad characters:

```
$ conda install conda --debug
DEBUG conda.core.index:fetch_repodata(368): Locally invalidating cached repodata for http://myrepo/packages/noarch at C:\Anaconda2\pkgs\cache\8beb19b1.json
DEBUG requests.packages.urllib3.connectionpool:_make_request(400): http://dproxy:8080 "GET http://myrepo/packages/noarch/repodata.json HTTP/1.1" 400 311
DEBUG conda.core.index:fetch_repodata_remote_request(193):
>>GET /packages/noarch/repodata.json HTTP
> User-Agent: conda/4.3.4 requests/2.12.4 CPython/2.7.13 Windows/7 Windows/6.1.7601
> Accept: */*
> Accept-Encoding: gzip, deflate, compress, identity
> Connection: keep-alive
> Content-Type: application/json
> If-Modified-Since: Mon, 16 Jan 2017 15:27:55 GMT
> If-None-Match: \"b7eece1bd70d21:0\"

<<HTTP 400 Bad Request
< Content-Type: text/html; charset=us-ascii
< Date: Mon, 16 Jan 2017 15:30:47 GMT
< Proxy-Connection: Keep-Alive
< Server: Microsoft-HTTPAPI/2.0
< Content-Length: 311
< Elapsed: 00:00.006000

```
As you can see, the value of ``If-None-Match`` contains extra ``\"``, which IIS interprets as invalid. Stripping out these characters makes the request go through the server without any problems. 

Note that this happens only when conda queries the ``noarch`` subdirectory. All other platform subdirs work fine. Also note that this does not happen immediately; it takes a few retries until conda starts sending the malformed header. 


Discussion:
Thanks for the report.  I've tracked down the problem.  Working on the solution now.

Resolving patch:
diff --git a/conda/common/compat.py b/conda/common/compat.py
--- a/conda/common/compat.py
+++ b/conda/common/compat.py
@@ -155,6 +155,10 @@ def ensure_text_type(value):
     return value.decode('utf-8') if hasattr(value, 'decode') else value
 
 
+def ensure_unicode(value):
+    return value.decode('unicode_escape') if hasattr(value, 'decode') else value
+
+
 # TODO: move this somewhere else
 # work-around for python bug on Windows prior to python 3.2
 # https://bugs.python.org/issue10027
diff --git a/conda/core/index.py b/conda/core/index.py
--- a/conda/core/index.py
+++ b/conda/core/index.py
@@ -11,13 +11,12 @@
 from os import makedirs
 from os.path import getmtime, join
 import re
+from requests.exceptions import ConnectionError, HTTPError, SSLError
+from requests.packages.urllib3.exceptions import InsecureRequestWarning
 from textwrap import dedent
 from time import time
 import warnings
 
-from requests.exceptions import ConnectionError, HTTPError, SSLError
-from requests.packages.urllib3.exceptions import InsecureRequestWarning
-
 from .linked_data import linked_data
 from .package_cache import PackageCache
 from .._vendor.auxlib.entity import EntityEncoder
@@ -25,7 +24,7 @@
 from .._vendor.auxlib.logz import stringify
 from ..base.constants import (CONDA_HOMEPAGE_URL, MAX_CHANNEL_PRIORITY)
 from ..base.context import context
-from ..common.compat import ensure_text_type, iteritems, iterkeys, itervalues
+from ..common.compat import ensure_text_type, ensure_unicode, iteritems, iterkeys, itervalues
 from ..common.url import join_url
 from ..connection import CondaSession
 from ..exceptions import CondaHTTPError, CondaRuntimeError
@@ -149,7 +148,7 @@ def read_mod_and_etag(path):
         try:
             with closing(mmap(f.fileno(), 0, access=ACCESS_READ)) as m:
                 match_objects = take(3, re.finditer(REPODATA_HEADER_RE, m))
-                result = dict(map(ensure_text_type, mo.groups()) for mo in match_objects)
+                result = dict(map(ensure_unicode, mo.groups()) for mo in match_objects)
                 return result
         except ValueError:
             # ValueError: cannot mmap an empty file


Test patch: