Repository: mitmproxy/mitmproxy

Issue:
Mitmproxy crashes, when setting only value in cookies/set-cookies grideditor
##### Steps to reproduce the problem:

1. Run mitmproxy.
2. Press `n` -> `enter` -> `enter` -> `e` -> `cookies` (or `set-cookies`)
3. You are in grideditor. Press `a`. Choose `Value` column and add any value.
You will see something like:
![image](https://user-images.githubusercontent.com/20267977/35590192-be00b434-060e-11e8-8ebd-e5bb20952aef.png)

4. Press `q` -> `e` -> `cookies` (or `set-cookies` if you're doing that for it)

Traceback: 
```
Traceback (most recent call last):
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/master.py", line 216, in run
    self.loop.run()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 278, in run
    self._run()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 376, in _run
    self.event_loop.run()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 682, in run
    self._loop()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 710, in _loop
    self._entering_idle()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 671, in _entering_idle
    callback()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 564, in entering_idle
    self.draw_screen()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 578, in draw_screen
    canvas = self._topmost_widget.render(self.screen_size, focus=True)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/container.py", line 1083, in render
    focus and self.focus_part == 'body')
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/decoration.py", line 225, in render
    canv = self._original_widget.render(size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/container.py", line 1083, in render
    focus and self.focus_part == 'body')
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 1750, in render
    canv = get_delegate(self).render(size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 1750, in render
    canv = get_delegate(self).render(size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/container.py", line 1083, in render
    focus and self.focus_part == 'body')
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 455, in render
    (maxcol, maxrow), focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 337, in calculate_visible
    self._set_focus_complete( (maxcol, maxrow), focus )
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 702, in _set_focus_complete
    (maxcol,maxrow), focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 672, in _set_focus_first_selectable
    (maxcol, maxrow), focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 340, in calculate_visible
    focus_widget, focus_pos = self.body.get_focus()
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/base.py", line 227, in get_focus
    self.lst[self.focus]
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/base.py", line 83, in __init__
    w = self.editor.columns[i].Display(v)
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/col_text.py", line 18, in Display
    return TDisplay(data, self.encoding_args)
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/col_text.py", line 31, in __init__
    super().__init__(data.encode(*self.encoding_args))
AttributeError: 'NoneType' object has no attribute 'encode'
```
You can also reproduce it for `set-cookies attributes`. Just add value without name into `set-cookies attributes`. Press `q` -> `q` to get into `flowview` tab. Press `e` -> `set-cookies`. You will see the same traceback.
##### Any other comments? What have you tried so far?

This issue is relevant for 2.0.x as well.


##### System information

Mitmproxy: 3.0.0.dev75 (commit 02d2ef7) 
Python:    3.5.2
OpenSSL:   OpenSSL 1.1.0g  2 Nov 2017
Platform:  Linux-4.4.0-112-generic-x86_64-with-Ubuntu-16.04-xenial
Mitmproxy crashes, when setting only value in cookies/set-cookies grideditor
##### Steps to reproduce the problem:

1. Run mitmproxy.
2. Press `n` -> `enter` -> `enter` -> `e` -> `cookies` (or `set-cookies`)
3. You are in grideditor. Press `a`. Choose `Value` column and add any value.
You will see something like:
![image](https://user-images.githubusercontent.com/20267977/35590192-be00b434-060e-11e8-8ebd-e5bb20952aef.png)

4. Press `q` -> `e` -> `cookies` (or `set-cookies` if you're doing that for it)

Traceback: 
```
Traceback (most recent call last):
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/master.py", line 216, in run
    self.loop.run()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 278, in run
    self._run()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 376, in _run
    self.event_loop.run()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 682, in run
    self._loop()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 710, in _loop
    self._entering_idle()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 671, in _entering_idle
    callback()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 564, in entering_idle
    self.draw_screen()
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/main_loop.py", line 578, in draw_screen
    canvas = self._topmost_widget.render(self.screen_size, focus=True)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/container.py", line 1083, in render
    focus and self.focus_part == 'body')
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/decoration.py", line 225, in render
    canv = self._original_widget.render(size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/container.py", line 1083, in render
    focus and self.focus_part == 'body')
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 1750, in render
    canv = get_delegate(self).render(size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 1750, in render
    canv = get_delegate(self).render(size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/container.py", line 1083, in render
    focus and self.focus_part == 'body')
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/widget.py", line 141, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 455, in render
    (maxcol, maxrow), focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 337, in calculate_visible
    self._set_focus_complete( (maxcol, maxrow), focus )
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 702, in _set_focus_complete
    (maxcol,maxrow), focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 672, in _set_focus_first_selectable
    (maxcol, maxrow), focus=focus)
  File "/home/kajoj/Mitmproxy/mitmproxy/venv/lib/python3.5/site-packages/urwid/listbox.py", line 340, in calculate_visible
    focus_widget, focus_pos = self.body.get_focus()
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/base.py", line 227, in get_focus
    self.lst[self.focus]
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/base.py", line 83, in __init__
    w = self.editor.columns[i].Display(v)
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/col_text.py", line 18, in Display
    return TDisplay(data, self.encoding_args)
  File "/home/kajoj/Mitmproxy/mitmproxy/mitmproxy/tools/console/grideditor/col_text.py", line 31, in __init__
    super().__init__(data.encode(*self.encoding_args))
AttributeError: 'NoneType' object has no attribute 'encode'
```
You can also reproduce it for `set-cookies attributes`. Just add value without name into `set-cookies attributes`. Press `q` -> `q` to get into `flowview` tab. Press `e` -> `set-cookies`. You will see the same traceback.
##### Any other comments? What have you tried so far?

This issue is relevant for 2.0.x as well.


##### System information

Mitmproxy: 3.0.0.dev75 (commit 02d2ef7) 
Python:    3.5.2
OpenSSL:   OpenSSL 1.1.0g  2 Nov 2017
Platform:  Linux-4.4.0-112-generic-x86_64-with-Ubuntu-16.04-xenial


Discussion:
@kajojify Hi, are you working on this? :) 
@kira0204 hi! I am not working on it. Take it, if you want :)
Update: the root cause of this is because the list that is maintaining cookies is initially like `['','value']` but is getting updated to `['value',None]`, I still don't know what is causing this but will revert soon.
@kajojify Hi, are you working on this? :) 
@kira0204 hi! I am not working on it. Take it, if you want :)
Update: the root cause of this is because the list that is maintaining cookies is initially like `['','value']` but is getting updated to `['value',None]`, I still don't know what is causing this but will revert soon.

Resolving patch:
diff --git a/mitmproxy/net/http/cookies.py b/mitmproxy/net/http/cookies.py
--- a/mitmproxy/net/http/cookies.py
+++ b/mitmproxy/net/http/cookies.py
@@ -114,11 +114,10 @@ def _read_cookie_pairs(s, off=0):
         lhs, off = _read_key(s, off)
         lhs = lhs.lstrip()
 
-        if lhs:
-            rhs = None
-            if off < len(s) and s[off] == "=":
-                rhs, off = _read_value(s, off + 1, ";")
-
+        rhs = ""
+        if off < len(s) and s[off] == "=":
+            rhs, off = _read_value(s, off + 1, ";")
+        if rhs or lhs:
             pairs.append([lhs, rhs])
 
         off += 1
@@ -143,25 +142,24 @@ def _read_set_cookie_pairs(s: str, off=0) -> Tuple[List[TPairs], int]:
         lhs, off = _read_key(s, off, ";=,")
         lhs = lhs.lstrip()
 
-        if lhs:
-            rhs = None
-            if off < len(s) and s[off] == "=":
-                rhs, off = _read_value(s, off + 1, ";,")
-
-                # Special handliing of attributes
-                if lhs.lower() == "expires":
-                    # 'expires' values can contain commas in them so they need to
-                    # be handled separately.
+        rhs = ""
+        if off < len(s) and s[off] == "=":
+            rhs, off = _read_value(s, off + 1, ";,")
 
-                    # We actually bank on the fact that the expires value WILL
-                    # contain a comma. Things will fail, if they don't.
+            # Special handling of attributes
+            if lhs.lower() == "expires":
+                # 'expires' values can contain commas in them so they need to
+                # be handled separately.
 
-                    # '3' is just a heuristic we use to determine whether we've
-                    # only read a part of the expires value and we should read more.
-                    if len(rhs) <= 3:
-                        trail, off = _read_value(s, off + 1, ";,")
-                        rhs = rhs + "," + trail
+                # We actually bank on the fact that the expires value WILL
+                # contain a comma. Things will fail, if they don't.
 
+                # '3' is just a heuristic we use to determine whether we've
+                # only read a part of the expires value and we should read more.
+                if len(rhs) <= 3:
+                    trail, off = _read_value(s, off + 1, ";,")
+                    rhs = rhs + "," + trail
+        if rhs or lhs:
             pairs.append([lhs, rhs])
 
             # comma marks the beginning of a new cookie
@@ -196,13 +194,10 @@ def _format_pairs(pairs, specials=(), sep="; "):
     """
     vals = []
     for k, v in pairs:
-        if v is None:
-            vals.append(k)
-        else:
-            if k.lower() not in specials and _has_special(v):
-                v = ESCAPE.sub(r"\\\1", v)
-                v = '"%s"' % v
-            vals.append("%s=%s" % (k, v))
+        if k.lower() not in specials and _has_special(v):
+            v = ESCAPE.sub(r"\\\1", v)
+            v = '"%s"' % v
+        vals.append("%s=%s" % (k, v))
     return sep.join(vals)
 
 


Test patch:
diff --git a/test/mitmproxy/net/http/test_cookies.py b/test/mitmproxy/net/http/test_cookies.py
--- a/test/mitmproxy/net/http/test_cookies.py
+++ b/test/mitmproxy/net/http/test_cookies.py
@@ -6,6 +6,10 @@
 
 
 cookie_pairs = [
+    [
+        "=uno",
+        [["", "uno"]]
+    ],
     [
         "",
         []
@@ -16,7 +20,7 @@
     ],
     [
         "one",
-        [["one", None]]
+        [["one", ""]]
     ],
     [
         "one=uno; two=due",
@@ -36,7 +40,7 @@
     ],
     [
         "one=uno; two; three=tre",
-        [["one", "uno"], ["two", None], ["three", "tre"]]
+        [["one", "uno"], ["two", ""], ["three", "tre"]]
     ],
     [
         "_lvs2=zHai1+Hq+Tc2vmc2r4GAbdOI5Jopg3EwsdUT9g=; "
@@ -78,9 +82,13 @@ def test_read_quoted_string():
 
 def test_read_cookie_pairs():
     vals = [
+        [
+            "=uno",
+            [["", "uno"]]
+        ],
         [
             "one",
-            [["one", None]]
+            [["one", ""]]
         ],
         [
             "one=two",
@@ -100,7 +108,7 @@ def test_read_cookie_pairs():
         ],
         [
             'one="two"; three=four; five',
-            [["one", "two"], ["three", "four"], ["five", None]]
+            [["one", "two"], ["three", "four"], ["five", ""]]
         ],
         [
             'one="\\"two"; three=four',
@@ -134,6 +142,12 @@ def test_cookie_roundtrips():
 
 def test_parse_set_cookie_pairs():
     pairs = [
+        [
+            "=uno",
+            [[
+                ["", "uno"]
+            ]]
+        ],
         [
             "one=uno",
             [[
@@ -150,7 +164,7 @@ def test_parse_set_cookie_pairs():
             "one=uno; foo",
             [[
                 ["one", "uno"],
-                ["foo", None]
+                ["foo", ""]
             ]]
         ],
         [
@@ -199,6 +213,12 @@ def set_cookie_equal(obs, exp):
         [
             ";", []
         ],
+        [
+            "=uno",
+            [
+                ("", "uno", ())
+            ]
+        ],
         [
             "one=uno",
             [
diff --git a/test/mitmproxy/net/http/test_response.py b/test/mitmproxy/net/http/test_response.py
--- a/test/mitmproxy/net/http/test_response.py
+++ b/test/mitmproxy/net/http/test_response.py
@@ -113,7 +113,7 @@ def test_get_cookies_with_parameters(self):
         assert attrs["domain"] == "example.com"
         assert attrs["expires"] == "Wed Oct  21 16:29:41 2015"
         assert attrs["path"] == "/"
-        assert attrs["httponly"] is None
+        assert attrs["httponly"] == ""
 
     def test_get_cookies_no_value(self):
         resp = tresp()