internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: mitmproxy/mitmproxy

Issue:
Cannot inject websocket with binary or string content.
#### Problem Description
mitmproxy attempts to parse the injected websocket binary data as text, and replaces the binary data if it cannot be decoded into text, changing the injected websocket message. Also, when a string is given instead, there is error:
File "\mitmproxy\proxy\layers\websocket.py", line 230, in msg
    data_str = data.decode(errors="replace")
AttributeError: 'str' object has no attribute 'decode'

#### Steps to reproduce the behavior:
1. from mitmproxy.ctx import master
2. master.commands.call("inject.websocket", flow, False, bytes([i for i in range(256)])) # No error, but message contents are changed
3. master.commands.call("inject.websocket", flow, False, "string") # AttributeError

#### System Information
Mitmproxy: 7.0.2
Python:    3.9.6
OpenSSL:   OpenSSL 1.1.1k  25 Mar 2021
Platform:  Windows-10-10.0.19043-SP0



Discussion:
In both cases `bytes` is expected and the last argument `is_text` specifies what type of message you want to inject:

https://github.com/mitmproxy/mitmproxy/blob/ab6f1ebb4415620ce02d61d0ff1c72e6b84b5198/mitmproxy/addons/proxyserver.py#L193

In your first example the bytes are treated as TEXT frame, hence why "mitmproxy attempts to parse the injected websocket binary data as text". In the second example you are passing a string, but utf-8 bytes are expected.

I'm not even sure where this is in the docs, but I'll mark it as a docs issue
The only place where I can find how to inject websocket in the docs is [websocket-inject-message.py example](https://docs.mitmproxy.org/stable/addons-examples/#websocket-inject-message). In the example, the string "ssssssh" is injected, which should result in error.
Yep, this was an oversight here https://github.com/mitmproxy/mitmproxy/pull/4650 The example needs to be fixed.
@Prinzhorn: Would you mind sending a quick PR? ❤️ 

Resolving patch:
diff --git a/examples/addons/websocket-inject-message.py b/examples/addons/websocket-inject-message.py
--- a/examples/addons/websocket-inject-message.py
+++ b/examples/addons/websocket-inject-message.py
@@ -13,9 +13,9 @@
 def websocket_message(flow: http.HTTPFlow):
     assert flow.websocket is not None  # make type checker happy
     last_message = flow.websocket.messages[-1]
-    if b"secret" in last_message.content:
+    if last_message.is_text and "secret" in last_message.text:
         last_message.drop()
-        ctx.master.commands.call("inject.websocket", flow, last_message.from_client, "ssssssh")
+        ctx.master.commands.call("inject.websocket", flow, last_message.from_client, "ssssssh".encode())
 
 
 # Complex example: Schedule a periodic timer
@@ -24,7 +24,7 @@ async def inject_async(flow: http.HTTPFlow):
     msg = "hello from mitmproxy! "
     assert flow.websocket is not None  # make type checker happy
     while flow.websocket.timestamp_end is None:
-        ctx.master.commands.call("inject.websocket", flow, True, msg)
+        ctx.master.commands.call("inject.websocket", flow, True, msg.encode())
         await asyncio.sleep(1)
         msg = msg[1:] + msg[:1]
 


Test patch:
Repository: mitmproxy/mitmproxy

Issue:
When too many requests come simultaneously, mitmdump called an error and quited [ValueError: too many file descriptors in select()]
#### Problem Description
A clear and concise description of what the bug is.
When too many requests come simultaneously, mitmdump called an error and quited.
Traceback (most recent call last):
  File "mitmdump", line 3, in <module>
  File "mitmproxy\tools\main.py", line 147, in mitmdump
  File "mitmproxy\tools\main.py", line 114, in run
  File "mitmproxy\master.py", line 76, in run
  File "mitmproxy\master.py", line 59, in run_loop
  File "mitmproxy\master.py", line 95, in shutdown
  File "asyncio\base_events.py", line 629, in run_until_complete
  File "asyncio\base_events.py", line 596, in run_forever
  File "asyncio\base_events.py", line 1854, in _run_once
  File "selectors.py", line 324, in select
  File "selectors.py", line 315, in _select
ValueError: too many file descriptors in select()
[77436] Failed to execute script 'mitmdump' due to unhandled exception!

I googled the error message, and found the following answer. Don't know if it's related.
https://stackoverflow.com/questions/57182009/why-am-i-getting-an-valueerror-too-many-file-descriptors-in-select

#### Steps to reproduce the behavior:
1. I use the following command
`mitmdump.exe -p 8080 --anticomp -q -s "d:\redirect-router.py"`
In the script, I re-write the host for a specific URL
2. 
3. 

#### System Information
Paste the output of "mitmproxy --version" here.
mitmproxy --version
Mitmproxy: 7.0.2 binary
Python:    3.9.6
OpenSSL:   OpenSSL 1.1.1k  25 Mar 2021
Platform:  Windows-10-10.0.18363-SP0


Discussion:

Resolving patch:
diff --git a/mitmproxy/__init__.py b/mitmproxy/__init__.py
--- a/mitmproxy/__init__.py
+++ b/mitmproxy/__init__.py
@@ -1,9 +0,0 @@
-import asyncio
-import sys
-
-if sys.platform == 'win32':
-    # workaround for
-    # https://github.com/tornadoweb/tornado/issues/2751
-    # https://www.tornadoweb.org/en/stable/index.html#installation
-    # (copied multiple times in the codebase, please remove all occurrences)
-    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())


Test patch:
diff --git a/test/mitmproxy/tools/web/test_app.py b/test/mitmproxy/tools/web/test_app.py
--- a/test/mitmproxy/tools/web/test_app.py
+++ b/test/mitmproxy/tools/web/test_app.py
@@ -2,18 +2,10 @@
 import json as _json
 import logging
 import os
-import sys
 from unittest import mock
 
 import pytest
 
-if sys.platform == 'win32':
-    # workaround for
-    # https://github.com/tornadoweb/tornado/issues/2751
-    # https://www.tornadoweb.org/en/stable/index.html#installation
-    # (copied multiple times in the codebase, please remove all occurrences)
-    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
-
 import tornado.testing  # noqa
 from tornado import httpclient  # noqa
 from tornado import websocket  # noqa
Repository: mitmproxy/mitmproxy

Issue:
WebSocket view jumps to top on new message
The WebSocket view keeps jumping to the top every time a new message arrives. This makes it basically impossible to work with while the connection is open. I hold down arrow -> it scrolls a bit -> message arrives -> I'm back at the top

_Originally posted by @Prinzhorn in https://github.com/mitmproxy/mitmproxy/issues/4486#issuecomment-796578909_


Discussion:
Hi @mhils how do I reproduce this behaviour? I could have a look into this.
@aaron-tan  Open a website with heavy WebSocket traffic, e.g. a twitch stream. When you open the websocket tab you cannot scroll down. Every time a new message arrives it jumps to the top. In the video I'm holding DOWN ARROW. Towards the end of the video I have closed the stream and I can scroll.

https://user-images.githubusercontent.com/679144/131297482-6606c88a-f145-463b-8493-566354758872.mp4



Resolving patch:
diff --git a/mitmproxy/tools/console/searchable.py b/mitmproxy/tools/console/searchable.py
--- a/mitmproxy/tools/console/searchable.py
+++ b/mitmproxy/tools/console/searchable.py
@@ -19,6 +19,7 @@ class Searchable(urwid.ListBox):
     def __init__(self, contents):
         self.walker = urwid.SimpleFocusListWalker(contents)
         urwid.ListBox.__init__(self, self.walker)
+        self.set_focus(len(self.walker) - 1)
         self.search_offset = 0
         self.current_highlight = None
         self.search_term = None


Test patch:
Repository: mitmproxy/mitmproxy

Issue:
Crash when invoking replay.server.count from console
#### Problem Description
Invoking `replay.server.count` from the console causes a crash. I don't think it happens all the time, but see below for repeatable reproduction.

#### Steps to reproduce the behavior:
1. Start `mitmproxy`
2. Hit `n` to create a new flow
3. Hit Enter
4. Hit `r` to issue the request
5. With the same flow selected, issue the command `:replay.server @focus`
6. Issue the command `:replay.server.count`

Sample stack trace:
```python
  File "/home/elespike/venvs/mitmproxy/lib/python3.9/site-packages/mitmproxy/tools/console/grideditor/editors.py", line 222, in __init__
    if not isinstance(vals[0], list):
TypeError: 'int' object is not subscriptable
```

#### System Information
```
Mitmproxy: 7.0.4
Python:    3.9.2
```


Discussion:

Resolving patch:
diff --git a/mitmproxy/tools/console/commandexecutor.py b/mitmproxy/tools/console/commandexecutor.py
--- a/mitmproxy/tools/console/commandexecutor.py
+++ b/mitmproxy/tools/console/commandexecutor.py
@@ -19,7 +19,7 @@ def __call__(self, cmd):
             except exceptions.CommandError as e:
                 ctx.log.error(str(e))
             else:
-                if ret:
+                if ret is not None:
                     if type(ret) == typing.Sequence[flow.Flow]:
                         signals.status_message.send(
                             message="Command returned %s flows" % len(ret)
diff --git a/mitmproxy/tools/console/grideditor/editors.py b/mitmproxy/tools/console/grideditor/editors.py
--- a/mitmproxy/tools/console/grideditor/editors.py
+++ b/mitmproxy/tools/console/grideditor/editors.py
@@ -213,11 +213,11 @@ def __init__(
             vals: typing.Union[
                 typing.List[typing.List[typing.Any]],
                 typing.List[typing.Any],
-                str,
+                typing.Any,
             ]) -> None:
-        if vals:
+        if vals is not None:
             # Whatever vals is, make it a list of rows containing lists of column values.
-            if isinstance(vals, str):
+            if not isinstance(vals, list):
                 vals = [vals]
             if not isinstance(vals[0], list):
                 vals = [[i] for i in vals]


Test patch:
diff --git a/test/mitmproxy/tools/console/test_integration.py b/test/mitmproxy/tools/console/test_integration.py
--- a/test/mitmproxy/tools/console/test_integration.py
+++ b/test/mitmproxy/tools/console/test_integration.py
@@ -54,3 +54,8 @@ def test_options_home_end(console):
 @pytest.mark.asyncio
 def test_keybindings_home_end(console):
     console.type("K<home><end>")
+
+
+@pytest.mark.asyncio
+def test_replay_count(console):
+    console.type(":replay.server.count<enter>")
Repository: mitmproxy/mitmproxy

Issue:
Crash when invoking replay.server.count from console
#### Problem Description
Invoking `replay.server.count` from the console causes a crash. I don't think it happens all the time, but see below for repeatable reproduction.

#### Steps to reproduce the behavior:
1. Start `mitmproxy`
2. Hit `n` to create a new flow
3. Hit Enter
4. Hit `r` to issue the request
5. With the same flow selected, issue the command `:replay.server @focus`
6. Issue the command `:replay.server.count`

Sample stack trace:
```python
  File "/home/elespike/venvs/mitmproxy/lib/python3.9/site-packages/mitmproxy/tools/console/grideditor/editors.py", line 222, in __init__
    if not isinstance(vals[0], list):
TypeError: 'int' object is not subscriptable
```

#### System Information
```
Mitmproxy: 7.0.4
Python:    3.9.2
```


Discussion:

Resolving patch:
diff --git a/mitmproxy/addons/dumper.py b/mitmproxy/addons/dumper.py
--- a/mitmproxy/addons/dumper.py
+++ b/mitmproxy/addons/dumper.py
@@ -38,10 +38,10 @@ def load(self, loader):
         loader.add_option(
             "flow_detail", int, 1,
             """
-            The display detail level for flows in mitmdump: 0 (almost quiet) to 3 (very verbose).
-              0: shortened request URL, response status code, WebSocket and TCP message notifications.
-              1: full request URL with response status code
-              2: 1 + HTTP headers
+            The display detail level for flows in mitmdump: 0 (quiet) to 4 (very verbose).
+              0: no output
+              1: shortened request URL with response status code
+              2: full request URL with response status code and HTTP headers
               3: 2 + truncated response content, content of WebSocket and TCP messages
               4: 3 + nothing is truncated
             """
@@ -156,7 +156,7 @@ def _echo_request_line(self, flow: http.HTTPFlow) -> None:
         else:
             url = flow.request.url
 
-        if ctx.options.flow_detail <= 1:
+        if ctx.options.flow_detail == 1:
             # We need to truncate before applying styles, so we just focus on the URL.
             terminal_width_limit = max(shutil.get_terminal_size()[0] - 25, 50)
             if len(url) > terminal_width_limit:


Test patch:
diff --git a/test/mitmproxy/addons/test_dumper.py b/test/mitmproxy/addons/test_dumper.py
--- a/test/mitmproxy/addons/test_dumper.py
+++ b/test/mitmproxy/addons/test_dumper.py
@@ -158,7 +158,7 @@ def test_echo_request_line():
         assert "nonstandard" in sio.getvalue()
         sio.truncate(0)
 
-        ctx.configure(d, flow_detail=0, showhost=True)
+        ctx.configure(d, flow_detail=1, showhost=True)
         f = tflow.tflow(resp=True)
         terminalWidth = max(shutil.get_terminal_size()[0] - 25, 50)
         f.request.url = "http://address:22/" + ("x" * terminalWidth) + "textToBeTruncated"
Repository: mitmproxy/mitmproxy

Issue:
RuntimeWarning: coroutine 'BaseEventLoop._create_server_getaddrinfo' was never awaited
#### Problem Description
I get this error when i run the mitmproxy command
```

Traceback (most recent call last):
  File "mitmproxy\master.py", line 54, in run_loop
  File "urwid\main_loop.py", line 287, in run
  File "urwid\main_loop.py", line 377, in _run
  File "urwid\main_loop.py", line 338, in start
  File "urwid\display_common.py", line 813, in start
  File "mitmproxy\contrib\urwid\raw_display.py", line 263, in _start
AssertionError

mitmproxy has crashed!
Please lodge a bug report at:
        https://github.com/mitmproxy/mitmproxy/issues
7[?47hsys:1: RuntimeWarning: coroutine 'BaseEventLoop._create_server_getaddrinfo' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
```
#### Steps to reproduce the behavior:
1. Open CMD on windows and run mitmproxy


#### System Information
Mitmproxy: 7.0.4 binary
Python:    3.9.7
OpenSSL:   OpenSSL 1.1.1l  24 Aug 2021
Platform:  Windows-10-10.0.10240-SP0


Discussion:
How do you run the mitmproxy command? In other words, which console are you using? cmd, powershell, something else?
> How do you run the mitmproxy command? In other words, which console are you using? cmd, powershell, something else?

I am using CMD. I can't install windows terminal (my windows version is lower)
Which Windows version are you on? :)
Windows-10 build 1024
Thanks for confirming. mitmproxy's console interface won't work before build 10586, which I think is the first build that includes ENABLE_VIRTUAL_TERMINAL_PROCESSING. mitmweb and mitmdump should be fine. I'll make sure that we have a better error message here.

Resolving patch:
diff --git a/mitmproxy/contrib/urwid/raw_display.py b/mitmproxy/contrib/urwid/raw_display.py
--- a/mitmproxy/contrib/urwid/raw_display.py
+++ b/mitmproxy/contrib/urwid/raw_display.py
@@ -260,7 +260,9 @@ def _start(self, alternate_buffer=True):
             )
 
             ok = win32.SetConsoleMode(hOut, dwOutMode)
-            assert ok
+            if not ok:
+                raise RuntimeError("Error enabling virtual terminal processing, "
+                                   "mitmproxy's console interface requires Windows 10 Build 10586 or above.")
             ok = win32.SetConsoleMode(hIn, dwInMode)
             assert ok
         else:


Test patch:
Repository: mitmproxy/mitmproxy

Issue:
Event type error says mitmproxy has crashed without actually crashing
#### Problem Description
Some sort of event type error...? This happened when I was testing `modify_body` and `modify_headers` options (see reproduction steps) and it results in repeated error messages with really long tracebacks.

The error messages begin by saying that mitmproxy has crashed, but it doesn't actually crash.

```
[... lots more traceback before the below]
  File "/home/elespike/venvs/mitmproxy/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 144, inhandle_event
    command = command_generator.send(send)
  File
"/home/elespike/venvs/mitmproxy/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py", line136, in _handle_event
    yield from self.client_state(event)
  File "/home/elespike/venvs/mitmproxy/lib/python3.9/site-packages/mitmproxy/proxy/utils.py", line 23, in
_check_event_type
    raise AssertionError(
AssertionError: Unexpected event type at HttpStream.state_done: Expected no events, got
RequestEndOfMessage(stream_id=1).
```

#### Steps to reproduce the behavior:
1. Start with default options
2. Set `modify_body` and `modify_headers` as follows:
```
modify_body:
  - :~u webhook.site ~q::<h1>hey</h1>

modify_headers:
  - :~u webhook.site ~s:content-type:text/html
```
3. Visit https://webhook.site

#### System Information
Mitmproxy: 7.0.4
Python:    3.9.2



Discussion:

Resolving patch:
diff --git a/mitmproxy/proxy/layers/websocket.py b/mitmproxy/proxy/layers/websocket.py
--- a/mitmproxy/proxy/layers/websocket.py
+++ b/mitmproxy/proxy/layers/websocket.py
@@ -86,7 +86,6 @@ class WebsocketLayer(layer.Layer):
     def __init__(self, context: Context, flow: http.HTTPFlow):
         super().__init__(context)
         self.flow = flow
-        assert context.server.connected
 
     @expect(events.Start)
     def start(self, _) -> layer.CommandGenerator[None]:


Test patch:
Repository: mitmproxy/mitmproxy

Issue:
ValueError: mutable default <class 'mitmproxy.contentviews.grpc.ProtoParser.ParserOptions'> for field parser_options is not allowed: use default_factory on python 3.11
#### Problem Description
mitmproxy fails to start throwing a `ValueError` exception:
```
ValueError: mutable default <class 'mitmproxy.contentviews.grpc.ProtoParser.ParserOptions'> for field parser_options is not allowed: use default_factory
```

#### Steps to reproduce the behavior:
1. Install mitmproxy 8.1.1 on Fedora rawhide (37)
2. run the binary

#### System Information
```
$ /usr/bin/mitmproxy --version
Traceback (most recent call last):
  File "/usr/bin/mitmproxy", line 8, in <module>
    sys.exit(mitmproxy())
             ^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mitmproxy/tools/main.py", line 118, in mitmproxy
    from mitmproxy.tools import console
  File "/usr/lib/python3.11/site-packages/mitmproxy/tools/console/__init__.py", line 1, in <module>
    from mitmproxy.tools.console import master
  File "/usr/lib/python3.11/site-packages/mitmproxy/tools/console/master.py", line 26, in <module>
    from mitmproxy.tools.console import consoleaddons
  File "/usr/lib/python3.11/site-packages/mitmproxy/tools/console/consoleaddons.py", line 6, in <module>
    from mitmproxy import contentviews
  File "/usr/lib/python3.11/site-packages/mitmproxy/contentviews/__init__.py", line 23, in <module>
    from . import (
  File "/usr/lib/python3.11/site-packages/mitmproxy/contentviews/grpc.py", line 952, in <module>
    @dataclass
     ^^^^^^^^^
  File "/usr/lib64/python3.11/dataclasses.py", line 1221, in dataclass
    return wrap(cls)
           ^^^^^^^^^
  File "/usr/lib64/python3.11/dataclasses.py", line 1211, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/dataclasses.py", line 959, in _process_class
    cls_fields.append(_get_field(cls, name, type, kw_only))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/dataclasses.py", line 816, in _get_field
    raise ValueError(f'mutable default {type(f.default)} for field '
ValueError: mutable default <class 'mitmproxy.contentviews.grpc.ProtoParser.ParserOptions'> for field parser_options is not allowed: use default_factory
```



Discussion:
We don't officially support Python 3.11 yet, but thanks for the pointer!

Resolving patch:
diff --git a/mitmproxy/contentviews/grpc.py b/mitmproxy/contentviews/grpc.py
--- a/mitmproxy/contentviews/grpc.py
+++ b/mitmproxy/contentviews/grpc.py
@@ -951,7 +951,7 @@ def format_grpc(
 
 @dataclass
 class ViewConfig:
-    parser_options: ProtoParser.ParserOptions = ProtoParser.ParserOptions()
+    parser_options: ProtoParser.ParserOptions = field(default_factory=ProtoParser.ParserOptions)
     parser_rules: list[ProtoParser.ParserRule] = field(default_factory=list)
 
 


Test patch:
Repository: mitmproxy/mitmproxy

Issue:
Latest release broken on Ubuntu 22.04 - ImportError: libffi.so.7: cannot open shared object file: No such file or directory
#### Problem Description

https://askubuntu.com/questions/1286772/libffi-so-7-cannot-open-shared-object-file-no-such-file-or-directory

I don't think our users should do anything with their system to make the binary run, correct?

```
$ ./mitmdump 
Traceback (most recent call last):
  File "mitmdump", line 3, in <module>
  File "mitmproxy/tools/main.py", line 131, in mitmdump
  File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module
  File "mitmproxy/tools/dump.py", line 1, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module
  File "mitmproxy/addons/__init__.py", line 10, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module
  File "mitmproxy/addons/cut.py", line 14, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module
  File "pyperclip/__init__.py", line 52, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module
  File "ctypes/__init__.py", line 8, in <module>
ImportError: libffi.so.7: cannot open shared object file: No such file or directory
[50935] Failed to execute script 'mitmdump' due to unhandled exception!
```

#### Steps to reproduce the behavior:

1. `mitmdump`

Repros in a clean `docker run --rm -it ubuntu:22.04` as well

#### System Information

9.0.0 just downloaded (I cannot run `mitmdump --version` as it doesn't launch)



Discussion:
Huh, thanks for raising this. The compiled binary _does_ include `libffi.so.7.1.0`, so I'm not sure what exactly is broken here.
pyinstaller 5.1 with Python 3.10 works. Let's see who's at fault here.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -108,7 +108,7 @@
             "hypothesis>=5.8,<7",
             "parver>=0.1,<2.0",
             "pdoc>=4.0.0",
-            "pyinstaller==5.6.1",
+            "pyinstaller==5.5",
             "pytest-asyncio>=0.17,<0.21",
             "pytest-cov>=2.7.1,<4.1",
             "pytest-timeout>=1.3.3,<2.2",


Test patch: