Repository: mitmproxy/mitmproxy

Issue:
Proxyauth metadata not present for HTTPS requests
##### Steps to reproduce the problem:

1. Run `mitmproxy --nonanonymous`
2. Set up the proxy in `System Preferences`. Enter a username and password (here I used `laura` and `pass` respectively).
3. Navigate to a HTTP site. Verify that you can see the proxyauth credentials in the `Detail` tab when viewing the request in mitmproxy. See image below:
<img width="1440" alt="screen shot 2017-03-07 at 11 18 52" src="https://cloud.githubusercontent.com/assets/9434500/23654470/b927f98e-0328-11e7-832c-b008a37943de.png">
4. Navigate to a HTTPS site. Verify that the proxyauth credentials are not visible in the `Detail` tab. 
Example: 
<img width="1440" alt="screen shot 2017-03-07 at 11 19 18" src="https://cloud.githubusercontent.com/assets/9434500/23654556/1e8a0704-0329-11e7-8aa9-0e576a1e341d.png">

##### Any other comments? What have you tried so far?
The proxyauth metadata is not present in the `request` method of inline scripts either. It is present when the request is HTTP. I have not checked other events. Adding 

```python
mitmproxy.ctx.log(f.request.pretty_host + "- Adding credientials to metadata")
```
 to [line 109 of the proxyauth addon](https://github.com/mitmproxy/mitmproxy/blob/master/mitmproxy/addons/proxyauth.py#L109), and using a simple inline script:

```python
def request(self, flow):
        ctx.log("Request recieved: " + flow.request.pretty_host + ". Metadata: " + str(flow.metadata))
```

I get the following output in the event log:

<img width="1440" alt="screen shot 2017-03-07 at 11 42 39" src="https://cloud.githubusercontent.com/assets/9434500/23655118/a88d2010-032b-11e7-86fd-78fa45cdc83b.png">
<img width="1440" alt="screen shot 2017-03-07 at 11 42 34" src="https://cloud.githubusercontent.com/assets/9434500/23655119/a88e4ef4-032b-11e7-8115-ef853d8e31fa.png">



##### System information

Mitmproxy version: 3.0.0 (1.0.1dev0299-0x37c1c55)
Python version: 3.6.0
Platform: Darwin-15.6.0-x86_64-i386-64bit
SSL version: OpenSSL 1.0.2k  26 Jan 2017
Mac version: 10.11.6 ('', '', '') x86_64



Discussion:
Side note: I previously added
```python
f.metadata['user'] = username
```
to [line 91 of the proxyauth addon](https://github.com/mitmproxy/mitmproxy/blob/master/mitmproxy/addons/proxyauth.py#L91) and I was always able to access that metadata from the `request` method of my inline script. 

That is no longer present for HTTPS requests either, and I'm not sure at when it broke since I have been away for a couple of weeks (and pulled when I got back). Just letting you know in case it helps you trace down the issue, but I shouldn't need to do this anymore if the proxyauth metadata is added correctly 😄 
Thanks for the extensive report! The issue here is that the client won't send authentication headers anymore after it has issued a `CONNECT` request to establish a TLS connection. If we detect this [here](https://github.com/mitmproxy/mitmproxy/blob/cb6240974d16e9f2923543fe28c201d39a63f60f/mitmproxy/addons/proxyauth.py#L163) we should still add the credentials as metadata to the flow. I think the correct approach is to use a `weakref.WeakKeyDictionary` for `.authenticated` instead of a `WeakSet`, so that we can look up the credentials belonging to a client connection there.

Resolving patch:
diff --git a/mitmproxy/addons/proxyauth.py b/mitmproxy/addons/proxyauth.py
--- a/mitmproxy/addons/proxyauth.py
+++ b/mitmproxy/addons/proxyauth.py
@@ -1,7 +1,7 @@
 import binascii
 import weakref
 from typing import Optional
-from typing import Set  # noqa
+from typing import MutableMapping  # noqa
 from typing import Tuple
 
 import passlib.apache
@@ -46,7 +46,7 @@ def __init__(self):
         self.htpasswd = None
         self.singleuser = None
         self.mode = None
-        self.authenticated = weakref.WeakSet()  # type: Set[connections.ClientConnection]
+        self.authenticated = weakref.WeakKeyDictionary()  # type: MutableMapping[connections.ClientConnection, Tuple[str, str]]
         """Contains all connections that are permanently authenticated after an HTTP CONNECT"""
 
     def enabled(self) -> bool:
@@ -155,11 +155,12 @@ def configure(self, options, updated):
     def http_connect(self, f: http.HTTPFlow) -> None:
         if self.enabled():
             if self.authenticate(f):
-                self.authenticated.add(f.client_conn)
+                self.authenticated[f.client_conn] = f.metadata["proxyauth"]
 
     def requestheaders(self, f: http.HTTPFlow) -> None:
         if self.enabled():
             # Is this connection authenticated by a previous HTTP CONNECT?
             if f.client_conn in self.authenticated:
+                f.metadata["proxyauth"] = self.authenticated[f.client_conn]
                 return
             self.authenticate(f)


Test patch:
diff --git a/test/mitmproxy/addons/test_proxyauth.py b/test/mitmproxy/addons/test_proxyauth.py
--- a/test/mitmproxy/addons/test_proxyauth.py
+++ b/test/mitmproxy/addons/test_proxyauth.py
@@ -171,3 +171,4 @@ def test_handlers():
         f2 = tflow.tflow(client_conn=f.client_conn)
         up.requestheaders(f2)
         assert not f2.response
+        assert f2.metadata["proxyauth"] == ('test', 'test')