Repository: streamlink/streamlink

Issue:
 File name too long error when using metadata in output filename
### Checklist

- [X] This is a bug report and not [a different kind of issue](https://github.com/streamlink/streamlink/issues/new/choose)
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed bug reports](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22bug%22)
- [X] [I have checked the commit log of the master branch](https://github.com/streamlink/streamlink/commits/master)

### Streamlink version

streamlink 6.7.2

### Description

The plugin should slice the output filename generated by metadata when the string exceed 255 bytes which will cause File name too long error.

I have solve it by adding the following to 'check_file_output'

```
root, ext = os.path.splitext(os.path.realpath(path).encode())
realpath = Path((root[:255 - len(ext)] + ext).decode(errors='ignore'))
```

### Debug log

```text
[10:57:48.870241][session][debug] Loading plugin: twitch
[10:57:48.936603][cli][debug] OS:         Linux-5.15.0-101-generic-x86_64-with-glibc2.29
[10:57:48.944412][cli][debug] Python:     3.8.10
[10:57:48.945094][cli][debug] OpenSSL:    OpenSSL 1.1.1f  31 Mar 2020
[10:57:48.945723][cli][debug] Streamlink: 6.7.2
[10:57:48.948742][cli][debug] Dependencies:
[10:57:48.976956][cli][debug]  certifi: 2024.2.2
[10:57:48.997520][cli][debug]  exceptiongroup: 1.2.0
[10:57:49.007621][cli][debug]  isodate: 0.6.1
[10:57:49.025597][cli][debug]  lxml: 4.9.4
[10:57:49.042283][cli][debug]  pycountry: 23.12.11
[10:57:49.057904][cli][debug]  pycryptodome: 3.20.0
[10:57:49.067723][cli][debug]  PySocks: 1.7.1
[10:57:49.081707][cli][debug]  requests: 2.31.0
[10:57:49.091640][cli][debug]  trio: 0.24.0
[10:57:49.105944][cli][debug]  trio-websocket: 0.11.1
[10:57:49.116662][cli][debug]  typing-extensions: 4.9.0
[10:57:49.136825][cli][debug]  urllib3: 2.2.0
[10:57:49.148906][cli][debug]  websocket-client: 1.7.0
[10:57:49.149938][cli][debug] Arguments:
[10:57:49.150537][cli][debug]  url=https://twitch.tv/solfi02
[10:57:49.151088][cli][debug]  stream=['best']
[10:57:49.151736][cli][debug]  --loglevel=debug
[10:57:49.152431][cli][debug]  --output=/home/NAS/broadcast/streams/twitch_solfi02/240403_105748_solfi02_{title}_{category}.ts
[10:57:49.155160][cli][debug]  --twitch-disable-ads=True
[10:57:49.157319][cli][info] Found matching plugin twitch for URL https://twitch.tv/solfi02
[10:57:49.158011][plugins.twitch][debug] Getting live HLS streams for solfi02
[10:57:49.472515][plugins.twitch][debug] {'adblock': False, 'geoblock_reason': '', 'hide_ads': False, 'server_ads': True, 'show_ads': True}
[10:57:49.707814][utils.l10n][debug] Language code: en_GB
[10:57:51.383465][cli][info] Available streams: audio_only, 160p (worst), 360p, 480p, 720p60, 1080p60 (best)
[10:57:51.383988][cli][info] Opening stream: 1080p60 (hls)
[10:57:51.394701][cli][info] Writing output to
/home/NAS/broadcast/streams/twitch_solfi02/240403_105748_solfi02_43950115467_해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘_Art.ts
[10:57:51.395080][cli][debug] Checking file output
debugback (most recent call last):
  File "/home/admin/.local/bin/streamlink", line 8, in <module>
    sys.exit(main())
  File "/home/admin/.local/lib/python3.8/site-packages/streamlink_cli/main.py", line 963, in main
    error_code = run(parser)
  File "/home/admin/.local/lib/python3.8/site-packages/streamlink_cli/main.py", line 936, in run
    handle_url()
  File "/home/admin/.local/lib/python3.8/site-packages/streamlink_cli/main.py", line 573, in handle_url
    handle_stream(plugin, streams, stream_name)
  File "/home/admin/.local/lib/python3.8/site-packages/streamlink_cli/main.py", line 445, in handle_stream
    success = output_stream(stream, formatter)
  File "/home/admin/.local/lib/python3.8/site-packages/streamlink_cli/main.py", line 341, in output_stream
    output = create_output(formatter)
  File "/home/admin/.local/lib/python3.8/site-packages/streamlink_cli/main.py", line 109, in create_output
    return check_file_output(formatter.path(args.output, args.fs_safe_rules), args.force)
  File "/home/admin/.local/lib/python3.8/site-packages/streamlink_cli/main.py", line 78, in check_file_output
    if realpath.is_file() and not force:
  File "/usr/lib/python3.8/pathlib.py", line 1439, in is_file
    return S_ISREG(self.stat().st_mode)
  File "/usr/lib/python3.8/pathlib.py", line 1198, in stat
    return self._accessor.stat(self)
OSError: [Errno 36] File name too long: '/home/NAS/broadcast/streams/twitch_solfi02/240403_105748_solfi02_43950115467_해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘!해묘_Art.ts'
```



Discussion:
The limitation doesn't come from the max path length but from the max file name length of the underlying file system (in addition to the OS's max path length - which should not be of relevance here).

Most file systems restrict their file name length to 255 bytes, so hardcoding this limitation should be good enough:
https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

My initial idea was truncating each variable substitution of the user's path template in [`streamlink.utils.formatter.Formatter._format()`](https://github.com/streamlink/streamlink/blob/6.7.2/src/streamlink/utils/formatter.py#L46-L59) similar to what [`streamlink_cli.utils.progress.ProgressFormatter.format()`](https://github.com/streamlink/streamlink/blob/6.7.2/src/streamlink_cli/utils/progress.py#L113-L160) does when formatting the download progress output, but this won't work, because the path's file name can always contain non-variable content, and this data can also exceed the 255 byte limitation on its own. This means that the file name always needs to be truncated after variables were substituted with stream metadata.

So what needs to be done is updating [`streamlink_cli.utils.formatter.Formatter.path()`](https://github.com/streamlink/streamlink/blob/6.7.2/src/streamlink_cli/utils/formatter.py#L13) and truncating each `part`, similar to your suggestion in the main CLI module. This will also truncate parent directory names, because those are allowed to have metadata as well. Simply truncating the whole path is the wrong solution. On the resolved path's final part, the user's file name extension must be preserved.

However, as you have already implied in your code suggestion, unicode strings can't easily be truncated, because characters can consist of multiple bytes, so the input string needs to be encoded, truncated and then decoded with any errors being ignored.

Unicode characters can include optional modifier characters though, and those modifiers should not be truncated on their own, otherwise the meaning of the base character will be changed. This isn't just limited to just emojis (e.g. skin tone, etc), but also to regular typography. Whole ranges of unicode modifier characters would need to be checked for the path's trailing character(s) when truncating. But considering that this is an edge-case within an edge-case, this could probably be ignored.

```py
>>> hand_light_skin_tone = "👋🏻"  # Waving Hand: Light Skin Tone (U+1F44B U+1F3FB)
>>> len(hand_light_skin_tone)
2
>>> hand_light_skin_tone.encode("utf-32-be").hex(" ", 2)
'0001 f44b 0001 f3fb'
>>> hand_light_skin_tone.encode("utf-8").hex(" ", 2)
'f09f 918b f09f 8fbb'

>>> for i in range(4, 12):
...     truncated = (hand_light_skin_tone*10).encode("utf-8")[:i].decode("utf-8", errors="ignore")
...     truncated, truncated.encode().hex(" ", 2)
... 
('👋', 'f09f 918b')
('👋', 'f09f 918b')
('👋', 'f09f 918b')
('👋', 'f09f 918b')
('👋🏻', 'f09f 918b f09f 8fbb')
('👋🏻', 'f09f 918b f09f 8fbb')
('👋🏻', 'f09f 918b f09f 8fbb')
('👋🏻', 'f09f 918b f09f 8fbb')
```

Resolving patch:
diff --git a/src/streamlink_cli/utils/formatter.py b/src/streamlink_cli/utils/formatter.py
--- a/src/streamlink_cli/utils/formatter.py
+++ b/src/streamlink_cli/utils/formatter.py
@@ -2,17 +2,18 @@
 from typing import Optional
 
 from streamlink.utils.formatter import Formatter as _BaseFormatter
-from streamlink_cli.utils.path import replace_chars, replace_path
+from streamlink_cli.utils.path import replace_chars, replace_path, truncate_path
 
 
 class Formatter(_BaseFormatter):
     title = _BaseFormatter.format
 
-    def path(self, pathlike: str, charmap: Optional[str] = None) -> Path:
+    def path(self, pathlike: str, charmap: Optional[str] = None, max_filename_length: int = 255) -> Path:
         def mapper(s):
             return replace_chars(s, charmap)
 
-        def format_part(part):
-            return self._format(part, mapper, {})
+        def format_part(part: str, isfile: bool) -> str:
+            formatted = self._format(part, mapper, {})
+            return truncate_path(formatted, max_filename_length, keep_extension=isfile)
 
         return replace_path(pathlike, format_part)
diff --git a/src/streamlink_cli/utils/path.py b/src/streamlink_cli/utils/path.py
--- a/src/streamlink_cli/utils/path.py
+++ b/src/streamlink_cli/utils/path.py
@@ -35,9 +35,34 @@ def replace_chars(path: str, charmap: Optional[str] = None, replacement: str = R
     return pattern.sub(replacement, path)
 
 
-def replace_path(pathlike: Union[str, Path], mapper: Callable[[str], str]) -> Path:
-    def get_part(part):
-        newpart = mapper(part)
+# This method does not take care of unicode modifier characters when truncating
+def truncate_path(path: str, length: int = 255, keep_extension: bool = True) -> str:
+    if len(path) <= length:
+        return path
+
+    parts = path.rsplit(".", 1)
+
+    # no file name extension (no dot separator in path or file name extension too long):
+    # truncate the whole thing
+    if not keep_extension or len(parts) == 1 or len(parts[1]) > 10:
+        encoded = path.encode("utf-8")
+        truncated = encoded[:length]
+        decoded = truncated.decode("utf-8", errors="ignore")
+        return decoded
+
+    # truncate file name, but keep file name extension
+    encoded = parts[0].encode("utf-8")
+    truncated = encoded[:length - len(parts[1]) - 1]
+    decoded = truncated.decode("utf-8", errors="ignore")
+    return f"{decoded}.{parts[1]}"
+
+
+def replace_path(pathlike: Union[str, Path], mapper: Callable[[str, bool], str]) -> Path:
+    def get_part(part: str, isfile: bool) -> str:
+        newpart = mapper(part, isfile)
         return REPLACEMENT if part != newpart and newpart in SPECIAL_PATH_PARTS else newpart
 
-    return Path(*(get_part(part) for part in Path(pathlike).expanduser().parts))
+    parts = Path(pathlike).expanduser().parts
+    last = len(parts) - 1
+
+    return Path(*(get_part(part, i == last) for i, part in enumerate(parts)))


Test patch:
diff --git a/tests/cli/utils/test_formatter.py b/tests/cli/utils/test_formatter.py
--- a/tests/cli/utils/test_formatter.py
+++ b/tests/cli/utils/test_formatter.py
@@ -1,6 +1,7 @@
 from datetime import datetime
 from os.path import sep
 from pathlib import Path
+from string import ascii_letters as alphabet
 from unittest.mock import Mock, call, patch
 
 import pytest
@@ -79,3 +80,19 @@ def test_path_substitute(self, formatter: Formatter):
         path = formatter.path(f"{{current}}{sep}{{parent}}{sep}{{dots}}{sep}{{separator}}{sep}foo{sep}.{sep}..{sep}bar")
         assert path == Path("_", "_", "...", "_", "foo", ".", "..", "bar"), \
             "Formats the path's parts separately and ignores current and parent directories in substitutions only"
+
+    def test_path_truncation_ascii(self, formatter: Formatter):
+        formatter.mapping.update({
+            "dir": lambda: alphabet * 10,
+            "file": lambda: alphabet * 10,
+        })
+        path = formatter.path(f"{{dir}}.fakeext{sep}{{file}}.ext")
+        assert path == Path((alphabet * 10)[:255], f"{(alphabet * 10)[:251]}.ext")
+
+    def test_path_truncation_unicode(self, formatter: Formatter):
+        formatter.mapping.update({
+            "dir": lambda: "🐻" * 512,
+            "file": lambda: "🐻" * 512,
+        })
+        path = formatter.path(f"{{dir}}.fakeext{sep}{{file}}.ext")
+        assert path == Path("🐻" * 63, f"{'🐻' * 62}.ext")
diff --git a/tests/cli/utils/test_path.py b/tests/cli/utils/test_path.py
--- a/tests/cli/utils/test_path.py
+++ b/tests/cli/utils/test_path.py
@@ -1,10 +1,10 @@
-import os
 from pathlib import Path
-from unittest.mock import patch
+from string import ascii_lowercase as alphabet
+from typing import Tuple
 
 import pytest
 
-from streamlink_cli.utils.path import replace_chars, replace_path
+from streamlink_cli.utils.path import replace_chars, replace_path, truncate_path
 
 
 @pytest.mark.parametrize("char", list(range(32)))
@@ -59,7 +59,7 @@ def test_replace_chars_replacement():
 
 
 def test_replace_path():
-    def mapper(s):
+    def mapper(s, *_):
         return dict(foo=".", bar="..").get(s, s)
 
     path = Path("foo", ".", "bar", "..", "baz")
@@ -68,14 +68,128 @@ def mapper(s):
 
 
 @pytest.mark.posix_only()
-def test_replace_path_expanduser_posix():
-    with patch.object(os, "environ", {"HOME": "/home/foo"}):
-        assert replace_path("~/bar", lambda s: s) == Path("/home/foo/bar")
-        assert replace_path("foo/bar", lambda s: dict(foo="~").get(s, s)) == Path("~/bar")
+@pytest.mark.parametrize("os_environ", [pytest.param({"HOME": "/home/foo"}, id="posix")], indirect=True)
+def test_replace_path_expanduser_posix(os_environ):
+    assert replace_path("~/bar", lambda s, *_: s) == Path("/home/foo/bar")
+    assert replace_path("foo/bar", lambda s, *_: dict(foo="~").get(s, s)) == Path("~/bar")
 
 
 @pytest.mark.windows_only()
-def test_replace_path_expanduser_windows():
-    with patch.object(os, "environ", {"USERPROFILE": "C:\\Users\\foo"}):
-        assert replace_path("~\\bar", lambda s: s) == Path("C:\\Users\\foo\\bar")
-        assert replace_path("foo\\bar", lambda s: dict(foo="~").get(s, s)) == Path("~\\bar")
+@pytest.mark.parametrize("os_environ", [pytest.param({"USERPROFILE": "C:\\Users\\foo"}, id="windows")], indirect=True)
+def test_replace_path_expanduser_windows(os_environ):
+    assert replace_path("~\\bar", lambda s, *_: s) == Path("C:\\Users\\foo\\bar")
+    assert replace_path("foo\\bar", lambda s, *_: dict(foo="~").get(s, s)) == Path("~\\bar")
+
+
+text = alphabet * 10
+bear = "🐻"  # Unicode character: "Bear Face" (U+1F43B)
+bears = bear * 512
+
+
+@pytest.mark.parametrize(("args", "expected"), [
+    pytest.param(
+        (alphabet, 255, True),
+        alphabet,
+        id="text - no truncation",
+    ),
+    pytest.param(
+        (text, 255, True),
+        text[:255],
+        id="text - truncate",
+    ),
+    pytest.param(
+        (text, 50, True),
+        text[:50],
+        id="text - truncate at 50",
+    ),
+    pytest.param(
+        (f"{alphabet}.ext", 255, True),
+        f"{alphabet}.ext",
+        id="text+ext1 - no truncation",
+    ),
+    pytest.param(
+        (f"{text}.ext", 255, True),
+        f"{text[:251]}.ext",
+        id="text+ext1 - truncate",
+    ),
+    pytest.param(
+        (f"{text}.ext", 50, True),
+        f"{text[:46]}.ext",
+        id="text+ext1 - truncate at 50",
+    ),
+    pytest.param(
+        (f"{text}.ext", 255, False),
+        text[:255],
+        id="text+ext1+nokeep - truncate",
+    ),
+    pytest.param(
+        (f"{text}.ext", 50, False),
+        text[:50],
+        id="text+ext1+nokeep - truncate at 50",
+    ),
+    pytest.param(
+        (f"{text}.notafilenameextension", 255, True),
+        text[:255],
+        id="text+ext2 - truncate",
+    ),
+    pytest.param(
+        (f"{text}.notafilenameextension", 50, True),
+        text[:50],
+        id="text+ext2 - truncate at 50",
+    ),
+    pytest.param(
+        (bear, 255, True),
+        bear,
+        id="bear - no truncation",
+    ),
+    pytest.param(
+        (bears, 255, True),
+        bear * 63,
+        id="bear - truncate",
+    ),
+    pytest.param(
+        (bears, 50, True),
+        bear * 12,
+        id="bear - truncate at 50",
+    ),
+    pytest.param(
+        (f"{bear}.ext", 255, True),
+        f"{bear}.ext",
+        id="bear+ext1 - no truncation",
+    ),
+    pytest.param(
+        (f"{bears}.ext", 255, True),
+        f"{bear * 62}.ext",
+        id="bear+ext1 - truncate",
+    ),
+    pytest.param(
+        (f"{bears}.ext", 50, True),
+        f"{bear * 11}.ext",
+        id="bear+ext1 - truncate at 50",
+    ),
+    pytest.param(
+        (f"{bears}.ext", 255, False),
+        bear * 63,
+        id="bear+ext1+nokeep - truncate",
+    ),
+    pytest.param(
+        (f"{bears}.ext", 50, False),
+        bear * 12,
+        id="bear+ext1+nokeep - truncate at 50",
+    ),
+    pytest.param(
+        (f"{bears}.notafilenameextension", 255, True),
+        bear * 63,
+        id="bear+ext2 - truncate",
+    ),
+    pytest.param(
+        (f"{bears}.notafilenameextension", 50, True),
+        bear * 12,
+        id="bear+ext2 - truncate at 50",
+    ),
+])
+def test_truncate_path(args: Tuple[str, int, bool], expected: str):
+    path, length, keep_extension = args
+    result = truncate_path(path, length, keep_extension)
+    assert len(result.encode("utf-8")) <= length
+    assert result == expected