Repository: mitmproxy/mitmproxy

Issue:
Logging verbosity impacts logs from a script
#### Problem Description
When I set the log verbosity in the config file to `warn`, i.e.

```
console_eventlog_verbosity: warn
termlog_verbosity: warn
```

logs that I'm writing in a script at that log level or higher are not outputted to stdout, whereas they are if I leave the verbosity at `info`, they are.

#### Steps to reproduce the behavior:
1. Set the `eventlog` and `termlog` verbosity to `warn`
2. Run a script with something like the following:

```python
import logging

class Handler
  def __init__(self):
    logging.basicConfig(level=logging.WARNING, 
                        format='%(asctime)s %(levelname)s %(name)s %(message)s',
                        filename="/var/log/mitmproxy.log"
                       )
    self.logger = logging.getLogger(__name__)

  def request(self, flow: http.HTTPFlow):
    self.logger.warning("A warning here")
```

#### System Information
Mitmproxy: 9.0.1
Python:    3.10.10
OpenSSL:   OpenSSL 3.0.7 1 Nov 2022
Platform:  Linux-5.10.162-141.675.amzn2.x86_64-x86_64-with



Discussion:
I can't reproduce this with what you provided. The following works as expected for me:

```python
import logging
from mitmproxy import http

class Handler:
  def __init__(self):
    logging.basicConfig(level=logging.WARNING, 
                        format='%(asctime)s %(levelname)s %(name)s %(message)s',
                        filename="/var/log/mitmproxy.log"
                       )
    self.logger = logging.getLogger(__name__)

  def request(self, flow: http.HTTPFlow):
    self.logger.warning("A warning here")

addons = [Handler()]
```
```shell
mitmdump -s foo.py --set termlog_verbosity=warn
```
```shell
curl -x localhost:8080 -k https://example.com
```

![image](https://user-images.githubusercontent.com/1019198/219645592-33c95649-b46d-477f-b361-d56846110a46.png)

<hr>

Mitmproxy: 10.0.0.dev (+217, commit 2663b42)
Python:    3.11.2
OpenSSL:   OpenSSL 3.0.8 7 Feb 2023
Platform:  Linux-4.4.0-22621-Microsoft-x86_64-with-glibc2.35
Sorry, I think my problem was stemming from a different behavior. If I call `self.logger.error("My error")`, it terminates the mitmproxy process. Is this intended, any logged error stops the process?
Absolutely not, except during startup. Again this needs a full repro. :)
```
#!/usr/bin/env python3

from mitmproxy import http
from mitmproxy import coretypes
import logging

class Throttling:
  
  def __init__(self):
    logging.basicConfig(level=logging.INFO, 
                        format='%(asctime)s %(levelname)s %(name)s %(message)s',
                        filename = "/var/log/mitmprox/mitmproxy.log",
                        filemode= "a"
                       )
    self.logger = logging.getLogger(__name__)
    self.logger.error("Init called")

  def request(self, flow: http.HTTPFlow):
    self.logger.warning("received request")

addons = [Throttling()]
```
Logging an error during ``__init__`` terminates the container I'm running this in.
Yes, errors during startup are intentionally fatal. The idea here is that those likely point to mistakes in the invocation, and it would be worse to silently swallow those.
The relevant addon is here: https://github.com/mitmproxy/mitmproxy/blob/main/mitmproxy/addons/errorcheck.py I'd be open to making this configurable, but it seems like it would be a reasonable alternative to just `.warn` in your case?
Using `.warn` is acceptable, just wasn't expecting logging an error during init to be fatal.
If you have ideas how how we can improve the dev experience here I'm all ears. Would an explicit message on stdout help? I'm thinking of something like "Errors logged during startup, exiting...".
An explicit message would definitely help, or even just a note in the documentation to expect this behavior, both would be ok for me.

Resolving patch:
diff --git a/mitmproxy/addons/errorcheck.py b/mitmproxy/addons/errorcheck.py
--- a/mitmproxy/addons/errorcheck.py
+++ b/mitmproxy/addons/errorcheck.py
@@ -8,8 +8,14 @@
 class ErrorCheck:
     """Monitor startup for error log entries, and terminate immediately if there are some."""
 
-    def __init__(self, log_to_stderr: bool = False) -> None:
-        self.log_to_stderr = log_to_stderr
+    repeat_errors_on_stderr: bool
+    """
+    Repeat all errors on stderr before exiting.
+    This is useful for the console UI, which otherwise swallows all output.
+    """
+
+    def __init__(self, repeat_errors_on_stderr: bool = False) -> None:
+        self.repeat_errors_on_stderr = repeat_errors_on_stderr
 
         self.logger = ErrorCheckHandler()
         self.logger.install()
@@ -21,10 +27,14 @@ async def shutdown_if_errored(self):
         # don't run immediately, wait for all logging tasks to finish.
         await asyncio.sleep(0)
         if self.logger.has_errored:
-            if self.log_to_stderr:
-                plural = "s" if len(self.logger.has_errored) > 1 else ""
+            plural = "s" if len(self.logger.has_errored) > 1 else ""
+            if self.repeat_errors_on_stderr:
                 msg = "\n".join(r.msg for r in self.logger.has_errored)
-                print(f"Error{plural} on startup: {msg}", file=sys.stderr)
+                print(f"Error{plural} logged during startup: {msg}", file=sys.stderr)
+            else:
+                print(
+                    f"Error{plural} logged during startup, exiting...", file=sys.stderr
+                )
 
             sys.exit(1)
 
diff --git a/mitmproxy/tools/console/master.py b/mitmproxy/tools/console/master.py
--- a/mitmproxy/tools/console/master.py
+++ b/mitmproxy/tools/console/master.py
@@ -55,7 +55,7 @@ def __init__(self, opts: options.Options) -> None:
             readfile.ReadFile(),
             consoleaddons.ConsoleAddon(self),
             keymap.KeymapConfig(self),
-            errorcheck.ErrorCheck(log_to_stderr=True),
+            errorcheck.ErrorCheck(repeat_errors_on_stderr=True),
         )
 
         self.window: window.Window | None = None


Test patch:
diff --git a/test/mitmproxy/addons/test_errorcheck.py b/test/mitmproxy/addons/test_errorcheck.py
--- a/test/mitmproxy/addons/test_errorcheck.py
+++ b/test/mitmproxy/addons/test_errorcheck.py
@@ -1,3 +1,5 @@
+import logging
+
 import pytest
 
 from mitmproxy.addons.errorcheck import ErrorCheck
@@ -13,10 +15,19 @@ def test_errorcheck(tdata, capsys):
                 tdata.path("mitmproxy/data/addonscripts/load_error.py"),
             ]
         )
-    assert "Error on startup" in capsys.readouterr().err
+    assert "Error logged during startup" in capsys.readouterr().err
 
 
 async def test_no_error():
     e = ErrorCheck()
     await e.shutdown_if_errored()
     e.finish()
+
+
+async def test_error_message(capsys):
+    e = ErrorCheck()
+    logging.error("wat")
+    logging.error("wat")
+    with pytest.raises(SystemExit):
+        await e.shutdown_if_errored()
+    assert "Errors logged during startup" in capsys.readouterr().err