Repository: mitmproxy/mitmproxy

Issue:
crached when resumeall after duplicate flow
##### Steps to reproduce the problem:
1.mitmproxy -p 7778
2.chrome use proxy switchysharp to charge to 127.0.0.1:7778
3.Intercept filter:~q
4.Intercept a resquest,click into it,press `shirt+d` to duplicate one or more flow，press `shift+a`to resume all
5.carsh and show this
```shell
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/master.py", line 200, in run
    self.loop.run()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 278, in run
    self._run()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 376, in _run
    self.event_loop.run()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 682, in run
    self._loop()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 719, in _loop
    self._watch_files[fd]()
  File "/usr/lib/python3.6/site-packages/urwid/raw_display.py", line 393, in <lambda>
    event_loop, callback, self.get_available_raw_input())
  File "/usr/lib/python3.6/site-packages/urwid/raw_display.py", line 493, in parse_input
    callback(processed, processed_codes)
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 403, in _update
    self.process_input(keys)
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 503, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/window.py", line 276, in keypress
    return self.master.keymap.handle(fs.keyctx, k)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/keymap.py", line 123, in handle
    return self.executor(b.command)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/commandeditor.py", line 24, in __call__
    ret = self.master.commands.call(cmd)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 144, in call
    return self.call_args(parts[0], parts[1:])
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 135, in call_args
    return self.commands[path].call(args)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 106, in call
    ret = self.func(*pargs)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 197, in wrapper
    return function(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/addons/core.py", line 34, in resume
    f.resume()
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/flow.py", line 182, in resume
    self.reply.commit()
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/controller.py", line 100, in commit
    "Reply is {}, but expected it to be taken.".format(self.state)
mitmproxy.exceptions.ControlException: Reply is start, but expected it to be taken.

mitmproxy has crashed!
Please lodge a bug report at:
	https://github.com/mitmproxy/mitmproxy
Shutting down...

```
##### Any other comments? What have you tried so far?
searched but not help

##### System information

<!-- Paste the output of "mitmproxy --version" here. -->
```shell
Mitmproxy version: 3.0.0 (release version) 
Python version: 3.6.2
Platform: Linux-4.12.6-1-ARCH-x86_64-with-arch-Arch-Linux
SSL version: OpenSSL 1.1.0f  25 May 2017
Linux distro: arch Arch Linux
```
crached when resumeall after duplicate flow
##### Steps to reproduce the problem:
1.mitmproxy -p 7778
2.chrome use proxy switchysharp to charge to 127.0.0.1:7778
3.Intercept filter:~q
4.Intercept a resquest,click into it,press `shirt+d` to duplicate one or more flow，press `shift+a`to resume all
5.carsh and show this
```shell
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/master.py", line 200, in run
    self.loop.run()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 278, in run
    self._run()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 376, in _run
    self.event_loop.run()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 682, in run
    self._loop()
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 719, in _loop
    self._watch_files[fd]()
  File "/usr/lib/python3.6/site-packages/urwid/raw_display.py", line 393, in <lambda>
    event_loop, callback, self.get_available_raw_input())
  File "/usr/lib/python3.6/site-packages/urwid/raw_display.py", line 493, in parse_input
    callback(processed, processed_codes)
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 403, in _update
    self.process_input(keys)
  File "/usr/lib/python3.6/site-packages/urwid/main_loop.py", line 503, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/window.py", line 276, in keypress
    return self.master.keymap.handle(fs.keyctx, k)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/keymap.py", line 123, in handle
    return self.executor(b.command)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/tools/console/commandeditor.py", line 24, in __call__
    ret = self.master.commands.call(cmd)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 144, in call
    return self.call_args(parts[0], parts[1:])
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 135, in call_args
    return self.commands[path].call(args)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 106, in call
    ret = self.func(*pargs)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/command.py", line 197, in wrapper
    return function(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/addons/core.py", line 34, in resume
    f.resume()
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/flow.py", line 182, in resume
    self.reply.commit()
  File "/usr/lib/python3.6/site-packages/mitmproxy-3.0.0-py3.6.egg/mitmproxy/controller.py", line 100, in commit
    "Reply is {}, but expected it to be taken.".format(self.state)
mitmproxy.exceptions.ControlException: Reply is start, but expected it to be taken.

mitmproxy has crashed!
Please lodge a bug report at:
	https://github.com/mitmproxy/mitmproxy
Shutting down...

```
##### Any other comments? What have you tried so far?
searched but not help

##### System information

<!-- Paste the output of "mitmproxy --version" here. -->
```shell
Mitmproxy version: 3.0.0 (release version) 
Python version: 3.6.2
Platform: Linux-4.12.6-1-ARCH-x86_64-with-arch-Arch-Linux
SSL version: OpenSSL 1.1.0f  25 May 2017
Linux distro: arch Arch Linux
```


Discussion:
Thanks! This is a recent 3.0 snapshot from master?
@mhils Yes, this is a still a problem on master (1d55d241f8f3c3c49a67777de42c9636a39c93db).
Looking into this, there's a more general issue. When we duplicate a flow, it still has `flow.intercepted` set to True. This is a bit confusing as there is no corresponding connection that we could "resume". Do we want to

1. Make "resume" a no-op in these cases, which just sets `intercepted = False`
2. Set `.intercepted = False` on duplication *and file save* (where we have the same issue)?

@cortesi, @Kriechi, any opinion?
Thanks! This is a recent 3.0 snapshot from master?
@mhils Yes, this is a still a problem on master (1d55d241f8f3c3c49a67777de42c9636a39c93db).
Looking into this, there's a more general issue. When we duplicate a flow, it still has `flow.intercepted` set to True. This is a bit confusing as there is no corresponding connection that we could "resume". Do we want to

1. Make "resume" a no-op in these cases, which just sets `intercepted = False`
2. Set `.intercepted = False` on duplication *and file save* (where we have the same issue)?

@cortesi, @Kriechi, any opinion?

Resolving patch:
diff --git a/mitmproxy/flow.py b/mitmproxy/flow.py
--- a/mitmproxy/flow.py
+++ b/mitmproxy/flow.py
@@ -179,5 +179,7 @@ def resume(self):
         if not self.intercepted:
             return
         self.intercepted = False
-        self.reply.ack()
-        self.reply.commit()
+        # If a flow is intercepted and then duplicated, the duplicated one is not taken.
+        if self.reply.state == "taken":
+            self.reply.ack()
+            self.reply.commit()


Test patch:
diff --git a/test/mitmproxy/test_http.py b/test/mitmproxy/test_http.py
--- a/test/mitmproxy/test_http.py
+++ b/test/mitmproxy/test_http.py
@@ -203,6 +203,15 @@ def test_resume(self):
         f.resume()
         assert f.reply.state == "committed"
 
+    def test_resume_duplicated(self):
+        f = tflow.tflow()
+        f.intercept()
+        f2 = f.copy()
+        assert f.intercepted is f2.intercepted is True
+        f.resume()
+        f2.resume()
+        assert f.intercepted is f2.intercepted is False
+
     def test_replace_unicode(self):
         f = tflow.tflow(resp=True)
         f.response.content = b"\xc2foo"