Repository: streamlink/streamlink

Issue:
UnicodeEncodeError: 'charmap' codec can't encode characters
### Checklist

- [X] This is a [plugin issue](https://streamlink.github.io/plugins.html) and not [a different kind of issue](https://github.com/streamlink/streamlink/issues/new/choose)
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed plugin issues](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22)
- [X] [I have checked the commit log of the master branch](https://github.com/streamlink/streamlink/commits/master)

### Streamlink version

latest

### Description

Setting the loglevel to "info", "debug", "trace" or "all" results in an error.

See logfile
[unicode error maybe.txt](https://github.com/streamlink/streamlink/files/12506748/unicode.error.maybe.txt)


Loglevel "warning" or lower works fine.

Maybe `encoding="utf-8"` fixes this?

### Debug log

```text
streamlink -o "~/recordings/{author}/{title}-{time:%Y-%m-%d-%H-%M-%S}.ts"  --logfile FILE.txt --loglevel all https://live.nicovideo.jp/watch/lv342661855 best
```

```
--- Logging error ---
Traceback (most recent call last):
  File "logging\__init__.py", line 1113, in emit
  File "encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode characters in position 79-84: character maps to <undefined>
Call stack:
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\bin\streamlink.exe\__main__.py", line 18, in <module>
    sys.exit(main())
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink_cli\main.py", line 924, in main
    handle_url()
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink_cli\main.py", line 564, in handle_url
    handle_stream(plugin, streams, stream_name)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink_cli\main.py", line 436, in handle_stream
    success = output_stream(stream, formatter)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink_cli\main.py", line 332, in output_stream
    output = create_output(formatter)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink_cli\main.py", line 103, in create_output
    return check_file_output(formatter.path(args.output, args.fs_safe_rules), args.force)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink_cli\main.py", line 69, in check_file_output
    log.info(f"Writing output to\n{path.resolve()}")
  File "logging\__init__.py", line 1489, in info
  File "logging\__init__.py", line 1634, in _log
  File "logging\__init__.py", line 1644, in handle
  File "logging\__init__.py", line 1706, in callHandlers
  File "logging\__init__.py", line 978, in handle
  File "logging\__init__.py", line 1230, in emit
  File "logging\__init__.py", line 1118, in emit
Message: 'Writing output to\nC:\\Users\\SecondaryH\\recordings\\大ちゃんマン(大雀士)\\22歳無職ゴミ絶望-2023-09-03-19-24-20.ts'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "logging\__init__.py", line 1113, in emit
  File "encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode characters in position 84-87: character maps to <undefined>
Call stack:
  File "threading.py", line 995, in _bootstrap
  File "threading.py", line 1038, in _bootstrap_inner
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink\plugin\api\websocket.py", line 112, in run
    self.ws.run_forever(**self._ws_rundata)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\websocket\_app.py", line 501, in run_forever
    setSock()
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\websocket\_app.py", line 428, in setSock
    dispatcher.read(self.sock.sock, read, check)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\websocket\_app.py", line 99, in read
    if not read_callback():
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\websocket\_app.py", line 461, in read
    self._callback(self.on_message, data)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\websocket\_app.py", line 547, in _callback
    callback(self, *args)
  File "C:\Users\SecondaryH\AppData\Local\Programs\Streamlink\pkgs\streamlink\plugins\nicolive.py", line 47, in on_message
    log.debug(f"Received: {data}")
  File "logging\__init__.py", line 1477, in debug
  File "logging\__init__.py", line 1634, in _log
  File "logging\__init__.py", line 1644, in handle
  File "logging\__init__.py", line 1706, in callHandlers
  File "logging\__init__.py", line 978, in handle
  File "logging\__init__.py", line 1230, in emit
  File "logging\__init__.py", line 1118, in emit
Message: 'Received: {"type":"room","data":{"name":"アリーナ","messageServer":{"uri":"wss://msgd.live2.nicovideo.jp/websocket","type":"niwavided"},"threadId":"SENSITIVE INFO","yourPostKey":"DELETED CUZ SENSiTIVE INFO I THINK","isFirst":true,"waybackkey":"waybackkey","vposBaseTime":"2023-09-03T17:42:23+09:00"}}'
Arguments: ()
```



Discussion:
Not a plugin issue.

The error is triggered by the logging module and it's caused by your system's default encoding.
This got me confused for a moment, because it's actually only the logfile that's causing issues here. The regular log stream-output should be working fine.

[`logging.FileHandler()`](https://docs.python.org/3/library/logging.handlers.html#logging.FileHandler) needs the `encoding="utf-8"` arg:
https://github.com/streamlink/streamlink/blob/79b8497add48a7068ece0a0b2bf538db2a7d61aa/src/streamlink/logger.py#L194-L198

Resolving patch:
diff --git a/src/streamlink/logger.py b/src/streamlink/logger.py
--- a/src/streamlink/logger.py
+++ b/src/streamlink/logger.py
@@ -193,7 +193,7 @@ def basicConfig(
     with _config_lock:
         handler: logging.StreamHandler
         if filename is not None:
-            handler = logging.FileHandler(filename, filemode)
+            handler = logging.FileHandler(filename, filemode, encoding="utf-8")
         else:
             handler = logging.StreamHandler(stream)
 


Test patch:
diff --git a/tests/test_logger.py b/tests/test_logger.py
--- a/tests/test_logger.py
+++ b/tests/test_logger.py
@@ -5,7 +5,6 @@
 from io import StringIO
 from pathlib import Path
 from typing import Iterable, Tuple, Type
-from unittest.mock import patch
 
 import freezegun
 import pytest
@@ -20,29 +19,53 @@ def output():
 
 
 @pytest.fixture()
-def log(request, output: StringIO):
+def logfile(tmp_path: Path) -> str:
+    return str(tmp_path / "log.txt")
+
+
+@pytest.fixture()
+def log(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch, output: StringIO):
     params = getattr(request, "param", {})
     params.setdefault("format", "[{name}][{levelname}] {message}")
     params.setdefault("style", "{")
+
+    if "logfile" in request.fixturenames:
+        params["filename"] = request.getfixturevalue("logfile")
+
     fakeroot = logging.getLogger("streamlink.test")
-    with patch("streamlink.logger.root", fakeroot), \
-         patch("streamlink.utils.times.LOCAL", timezone.utc):
-        handler = logger.basicConfig(stream=output, **params)
-        assert isinstance(handler, logging.Handler)
-        yield fakeroot
-        logger.capturewarnings(False)
-        fakeroot.removeHandler(handler)
-        assert not fakeroot.handlers
+
+    monkeypatch.setattr("streamlink.logger.root", fakeroot)
+    monkeypatch.setattr("streamlink.utils.times.LOCAL", timezone.utc)
+
+    handler = logger.basicConfig(stream=output, **params)
+    assert isinstance(handler, logging.Handler)
+
+    yield fakeroot
+
+    logger.capturewarnings(False)
+
+    handler.close()
+    fakeroot.removeHandler(handler)
+    assert not fakeroot.handlers
 
 
 class TestLogging:
     @pytest.fixture()
-    def log_failure(self, request, log: logging.Logger, output: StringIO):
+    def log_failure(
+        self,
+        request: pytest.FixtureRequest,
+        monkeypatch: pytest.MonkeyPatch,
+        log: logging.Logger,
+        output: StringIO,
+    ):
         params = getattr(request, "param", {})
+
         root = logging.getLogger("streamlink")
-        with patch("streamlink.logger.root", root):
-            with pytest.raises(Exception) as cm:  # noqa: PT011
-                logger.basicConfig(stream=output, **params)
+        monkeypatch.setattr("streamlink.logger.root", root)
+
+        with pytest.raises(Exception) as cm:  # noqa: PT011
+            logger.basicConfig(stream=output, **params)
+
         return cm.value
 
     @pytest.mark.parametrize(("name", "level"), [
@@ -164,6 +187,13 @@ def test_datefmt_custom(self, log: logging.Logger, output: StringIO):
         log.info("test")
         assert output.getvalue() == "[03:04:05.123456+0000][test][info] test\n"
 
+    def test_logfile(self, logfile: str, log: logging.Logger, output: StringIO):
+        log.setLevel("info")
+        log.info("Hello world, Γειά σου Κόσμε, こんにちは世界")
+        log.handlers[0].flush()
+        with open(logfile, "r", encoding="utf-8") as fh:
+            assert fh.read() == "[test][info] Hello world, Γειά σου Κόσμε, こんにちは世界\n"
+
 
 class TestCaptureWarnings:
     @staticmethod