Repository: mitmproxy/mitmproxy

Issue:
proxyauth option breaks flow state access in addons
##### Steps to reproduce the problem:

1. Create `test.py` addon
```py
from mitmproxy import ctx
class AuthTest:
    def response(self, flow):
        ctx.log.info('Getting flow state')
        flow.get_state()
        ctx.log.info('Got flow state')

addons = [
    AuthTest()
]
```
2. Start `mitmdump`, navigate to example.com and observe the log
```bash
mitmdump -s server/test.py
Loading script server/test.py
Proxy server listening at http://*:8080
...
[::1]:56410: GET http://example.com/
          << 200 OK 606b
Getting flow state
Got flow state
```
3. Re-start `mitmdump` with proxyauth applied and navigate to example.com again
```bash
mitmdump -s server/test.py --set proxyauth=foo:bar
Loading script server/test.py
Proxy server listening at http://*:8080
...
[::1]:56290: GET http://example.com/
          << 200 OK 606b
Getting flow state
Addon error: Traceback (most recent call last):
  File "server/test.py", line 5, in response
    flow.get_state()
  File "/usr/local/Cellar/mitmproxy/4.0.1/libexec/lib/python3.6/site-packages/mitmproxy/flow.py", line 94, in get_state
    d = super().get_state()
  File "/usr/local/Cellar/mitmproxy/4.0.1/libexec/lib/python3.6/site-packages/mitmproxy/stateobject.py", line 31, in get_state
    state[attr] = get_state(cls, val)
  File "/usr/local/Cellar/mitmproxy/4.0.1/libexec/lib/python3.6/site-packages/mitmproxy/stateobject.py", line 94, in get_state
    return _process(typeinfo, val, False)
  File "/usr/local/Cellar/mitmproxy/4.0.1/libexec/lib/python3.6/site-packages/mitmproxy/stateobject.py", line 77, in _process
    for k, v in val.items()
  File "/usr/local/Cellar/mitmproxy/4.0.1/libexec/lib/python3.6/site-packages/mitmproxy/stateobject.py", line 77, in <dictcomp>
    for k, v in val.items()
  File "/usr/local/Cellar/mitmproxy/4.0.1/libexec/lib/python3.6/site-packages/mitmproxy/stateobject.py", line 81, in _process
    assert isinstance(val, (int, str, bool, bytes))
AssertionError
```

##### Any other comments? What have you tried so far?

There's a FIXME [right near the code that's breaking](https://github.com/mitmproxy/mitmproxy/blob/master/mitmproxy/stateobject.py#L80), but I think it's a red herring. My best guess is that proxy auth adds something to the flow state that isn't in the list of allowed types `int, str, bool, bytes`—possibly a `dict`.

##### System information

```bash
mitmdump --version
Mitmproxy: 4.0.1
Python:    3.6.5
OpenSSL:   OpenSSL 1.0.2o  27 Mar 2018
Platform:  Darwin-17.5.0-x86_64-i386-64bit
```

<!-- Please use the mitmproxy forums (https://discourse.mitmproxy.org/) for support/how-to questions. Thanks! :) -->



Discussion:
Thanks for the report! 👍  Yes, this is a bug!

The offending type is: `typing.Any`
`val = ('foo', 'bar')`,  which is a tuple of strings.
I think that assert can basically go. We want values to be serialisable, but we don't want to implement a type checker in that spot. This will change once we have the planned session store in place. Thoughts?

Resolving patch:
diff --git a/mitmproxy/stateobject.py b/mitmproxy/stateobject.py
--- a/mitmproxy/stateobject.py
+++ b/mitmproxy/stateobject.py
@@ -1,6 +1,7 @@
 import typing
 from typing import Any  # noqa
 from typing import MutableMapping  # noqa
+import json
 
 from mitmproxy.coretypes import serializable
 from mitmproxy.utils import typecheck
@@ -77,8 +78,14 @@ def _process(typeinfo: typecheck.Type, val: typing.Any, make: bool) -> typing.An
             for k, v in val.items()
         }
     elif typename.startswith("typing.Any"):
-        # FIXME: Remove this when we remove flow.metadata
-        assert isinstance(val, (int, str, bool, bytes))
+        # This requires a bit of explanation. We can't import our IO layer here,
+        # because it causes a circular import. Rather than restructuring the
+        # code for this, we use JSON serialization, which has similar primitive
+        # type restrictions as tnetstring, to check for conformance.
+        try:
+            json.dumps(val)
+        except TypeError:
+            raise ValueError(f"Data not serializable: {val}")
         return val
     else:
         return typeinfo(val)


Test patch:
diff --git a/test/mitmproxy/test_stateobject.py b/test/mitmproxy/test_stateobject.py
--- a/test/mitmproxy/test_stateobject.py
+++ b/test/mitmproxy/test_stateobject.py
@@ -125,7 +125,7 @@ def test_any():
     assert a.x == b.x
 
     a = TAny(object())
-    with pytest.raises(AssertionError):
+    with pytest.raises(ValueError):
         a.get_state()