Repository: streamlink/streamlink

Issue:
UnicodeDecodeError when opening streams with non-ascii characters in the title
<!--
Thanks for reporting a bug!
USE THE TEMPLATE. Otherwise your bug report may be rejected.

First, see the contribution guidelines:
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink

Also check the list of open and closed bug reports:
https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22bug%22

Please see the text preview to avoid unnecessary formatting errors.
-->


## Bug Report

<!-- Replace [ ] with [x] in order to check the box -->
- [x] This is a bug report and I have read the contribution guidelines.


### Description

<!-- Explain the bug as thoroughly as you can. Don't leave out information which is necessary for us to reproduce and debug this issue. -->
When opening a stream with a non-ascii character in the stream title there is a “UnicodeDecodeError” and the stream doesn’t open.

The steam I tried to open had an ‘ö’ in the title, but trying other streams with non-ascii characters gave the same error (tested on random Korean streams). Other streams opened as expected.

### Expected / Actual behavior

<!-- What do you expect to happen, and what is actually happening? -->
Expect: Stream to open.
Actual: It doesn’t.

### Reproduction steps / Explicit stream URLs to test

<!-- How can we reproduce this? Please note the exact steps below using the list format supplied. If you need more steps please add them. -->

1. Find a stream with a non-ascii title and attempt to open it.

### Log output

<!--
TEXT LOG OUTPUT IS REQUIRED for a bug report!
Use the `--loglevel debug` parameter and avoid using parameters which suppress log output.
https://streamlink.github.io/cli.html#cmdoption-l

Make sure to **remove usernames and passwords**
You can copy the output to https://gist.github.com/ or paste it below.
-->

```
		OS: macOS 10.14.2
		Python: 2.7.14
		Streamlink: 1.1.1
		Requests(2.21.0), Socks(1.6.7), Websocket(0.47.0)
		Found matching plugin twitch for URL twitch.tv/quickybaby
		Plugin specific arguments:
		--twitch-oauth-token=******** (oauth_token)
		--twitch-disable-hosting=True (disable_hosting)
		--twitch-disable-ads=True (disable_ads)
		Getting live HLS streams for quickybaby
		Attempting to authenticate using OAuth token
		Successfully logged in as DELETE
		[utils.l10n][debug] Language code: en_US
		Available streams: audio_only, 160p (worst), 360p, 480p, 720p, 720p60, 1080p60 (best)
		Opening stream: 1080p60 (hls)
		Starting player: /Applications/mpv.app/Contents/MacOS/mpv
		Traceback (most recent call last):
		File "/usr/local/bin/streamlink", line 11, in <module>
		load_entry_point('streamlink==1.1.1', 'console_scripts', 'streamlink')()
		File "/usr/local/lib/python2.7/site-packages/streamlink-1.1.1-py2.7.egg/streamlink_cli/main.py", line 1033, in main
		handle_url()
		File "/usr/local/lib/python2.7/site-packages/streamlink-1.1.1-py2.7.egg/streamlink_cli/main.py", line 594, in handle_url
		handle_stream(plugin, streams, stream_name)
		File "/usr/local/lib/python2.7/site-packages/streamlink-1.1.1-py2.7.egg/streamlink_cli/main.py", line 437, in handle_stream
		success = output_stream_passthrough(plugin, stream)
		File "/usr/local/lib/python2.7/site-packages/streamlink-1.1.1-py2.7.egg/streamlink_cli/main.py", line 262, in output_stream_passthrough
		output.open()
		File "/usr/local/lib/python2.7/site-packages/streamlink-1.1.1-py2.7.egg/streamlink_cli/output.py", line 24, in open
		self._open()
		File "/usr/local/lib/python2.7/site-packages/streamlink-1.1.1-py2.7.egg/streamlink_cli/output.py", line 219, in _open
		self._open_call()
		File "/usr/local/lib/python2.7/site-packages/streamlink-1.1.1-py2.7.egg/streamlink_cli/output.py", line 234, in _open_call
		log.debug(u"Calling: {0}".format(fargs))
		UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 141: ordinal not in range(128)
```


### Additional comments, screenshots, etc.
I can't link to a stream that will give an error as titles always change, but it should be easy enough to find a Korean stream, and they usually have Korean titles.

[Love Streamlink? Please consider supporting our collective. Thanks!](https://opencollective.com/streamlink/donate)



Discussion:
Hi there.
Noticed this bug on Elementary OS 0.4.1 (based on Ubuntu 16.04.2). It can easily be reproduced by using the title parameter, e.g.
`streamlink -t "★" ((any stream url)) best`

When using Streamlink Twitch GUI, go to Settings > Player > Window title and disable the custom title option as a temporary workaround.

Regards,
Addi
**can't reproduce** on ubuntu 18.10 / python2

---

easiest fix is to use **python3** instead of python2

Resolving patch:
diff --git a/src/streamlink/utils/encoding.py b/src/streamlink/utils/encoding.py
--- a/src/streamlink/utils/encoding.py
+++ b/src/streamlink/utils/encoding.py
@@ -15,7 +15,10 @@ def get_filesystem_encoding():
 
 def maybe_encode(text, encoding="utf8"):
     if is_py2:
-        return text.encode(encoding)
+        if isinstance(text, unicode):
+            return text.encode(encoding)
+        else:
+            return text
     else:
         return text
 
diff --git a/src/streamlink_cli/main.py b/src/streamlink_cli/main.py
--- a/src/streamlink_cli/main.py
+++ b/src/streamlink_cli/main.py
@@ -30,7 +30,7 @@
 import streamlink.logger as logger
 from .argparser import build_parser
 from .compat import stdout, is_win32
-from streamlink.utils.encoding import maybe_encode
+from streamlink.utils.encoding import maybe_decode, get_filesystem_encoding
 from .console import ConsoleOutput, ConsoleUserInputRequester
 from .constants import CONFIG_FILES, PLUGINS_DIR, STREAM_SYNONYMS, DEFAULT_STREAM_METADATA
 from .output import FileOutput, PlayerOutput
@@ -146,7 +146,7 @@ def create_http_server(host=None, port=0):
 def create_title(plugin=None):
     if args.title and plugin:
         title = LazyFormatter.format(
-            maybe_encode(args.title),
+            maybe_decode(args.title, get_filesystem_encoding()),
             title=lambda: plugin.get_title() or DEFAULT_STREAM_METADATA["title"],
             author=lambda: plugin.get_author() or DEFAULT_STREAM_METADATA["author"],
             category=lambda: plugin.get_category() or DEFAULT_STREAM_METADATA["category"],
diff --git a/src/streamlink_cli/output.py b/src/streamlink_cli/output.py
--- a/src/streamlink_cli/output.py
+++ b/src/streamlink_cli/output.py
@@ -185,13 +185,13 @@ def _create_arguments(self):
             if self.player_name == "vlc":
                 # see https://wiki.videolan.org/Documentation:Format_String/, allow escaping with \$
                 self.title = self.title.replace("$", "$$").replace(r'\$$', "$")
-                extra_args.extend(["--input-title-format", self.title])
+                extra_args.extend([u"--input-title-format", self.title])
 
             # mpv
             if self.player_name == "mpv":
                 # see https://mpv.io/manual/stable/#property-expansion, allow escaping with \$, respect mpv's $>
                 self.title = self._mpv_title_escape(self.title)
-                extra_args.append("--title={}".format(self.title))
+                extra_args.append(u"--title={}".format(self.title))
 
             # potplayer
             if self.player_name == "potplayer":
@@ -209,8 +209,7 @@ def _create_arguments(self):
         if is_win32:
             eargs = maybe_decode(subprocess.list2cmdline(extra_args))
             # do not insert and extra " " when there are no extra_args
-            return maybe_encode(u' '.join([cmd] + ([eargs] if eargs else []) + [args]),
-                                encoding=get_filesystem_encoding())
+            return u' '.join([cmd] + ([eargs] if eargs else []) + [args])
         return shlex.split(cmd) + extra_args + shlex.split(args)
 
     def _open(self):
@@ -234,7 +233,8 @@ def _open_call(self):
         else:
             fargs = subprocess.list2cmdline(args)
         log.debug(u"Calling: {0}".format(fargs))
-        subprocess.call(args,
+
+        subprocess.call(maybe_encode(args, get_filesystem_encoding()),
                         stdout=self.stdout,
                         stderr=self.stderr)
 
@@ -247,7 +247,8 @@ def _open_subprocess(self):
         else:
             fargs = subprocess.list2cmdline(args)
         log.debug(u"Opening subprocess: {0}".format(fargs))
-        self.player = subprocess.Popen(args,
+
+        self.player = subprocess.Popen(maybe_encode(args, get_filesystem_encoding()),
                                        stdin=self.stdin, bufsize=0,
                                        stdout=self.stdout,
                                        stderr=self.stderr)


Test patch:
diff --git a/tests/__init__.py b/tests/__init__.py
--- a/tests/__init__.py
+++ b/tests/__init__.py
@@ -1,17 +1,29 @@
+import os
 import warnings
 
+import pytest
+
+from streamlink.compat import is_py2, is_py3
+
 
 def catch_warnings(record=False, module=None):
     def _catch_warnings_wrapper(f):
         def _catch_warnings(*args, **kwargs):
             with warnings.catch_warnings(record=True, module=module) as w:
                 if record:
-                    return f(*(args + (w, )), **kwargs)
+                    return f(*(args + (w,)), **kwargs)
                 else:
                     return f(*args, **kwargs)
+
         return _catch_warnings
 
     return _catch_warnings_wrapper
 
 
-__all__ = ['ignore_warnings']
+windows_only = pytest.mark.skipif(os.name != "nt", reason="test only applicable on Window")
+posix_only = pytest.mark.skipif(os.name != "posix", reason="test only applicable on a POSIX OS")
+py3_only = pytest.mark.skipif(not is_py3, reason="test only applicable for Python 3")
+py2_only = pytest.mark.skipif(not is_py2, reason="test only applicable for Python 2")
+
+
+__all__ = ['catch_warnings', 'windows_only', 'posix_only', 'py2_only', 'py3_only']
diff --git a/tests/test_cli_playerout.py b/tests/test_cli_playerout.py
new file mode 100644
--- /dev/null
+++ b/tests/test_cli_playerout.py
@@ -0,0 +1,71 @@
+# -*- coding: utf-8 -*-
+from streamlink.utils import get_filesystem_encoding
+from streamlink_cli.output import PlayerOutput
+from tests import posix_only, windows_only, py2_only, py3_only
+from tests.mock import patch, ANY
+
+UNICODE_TITLE = u"기타치는소율 with UL섬 "
+
+
+@posix_only
+@patch('subprocess.Popen')
+def test_output_mpv_unicode_title_posix(popen):
+    po = PlayerOutput("mpv", title=UNICODE_TITLE)
+    popen().poll.side_effect = lambda: None
+    po.open()
+    popen.assert_called_with(['mpv', u"--title=" + UNICODE_TITLE, '-'],
+                             bufsize=ANY, stderr=ANY, stdout=ANY, stdin=ANY)
+
+
+@posix_only
+@patch('subprocess.Popen')
+def test_output_vlc_unicode_title_posix(popen):
+    po = PlayerOutput("vlc", title=UNICODE_TITLE)
+    popen().poll.side_effect = lambda: None
+    po.open()
+    popen.assert_called_with(['vlc', u'--input-title-format', UNICODE_TITLE, '-'],
+                             bufsize=ANY, stderr=ANY, stdout=ANY, stdin=ANY)
+
+
+@py2_only
+@windows_only
+@patch('subprocess.Popen')
+def test_output_mpv_unicode_title_windows_py2(popen):
+    po = PlayerOutput("mpv.exe", title=UNICODE_TITLE)
+    popen().poll.side_effect = lambda: None
+    po.open()
+    popen.assert_called_with("mpv.exe \"--title=" + UNICODE_TITLE.encode(get_filesystem_encoding()) + "\" -",
+                             bufsize=ANY, stderr=ANY, stdout=ANY, stdin=ANY)
+
+
+@py2_only
+@windows_only
+@patch('subprocess.Popen')
+def test_output_vlc_unicode_title_windows_py2(popen):
+    po = PlayerOutput("vlc.exe", title=UNICODE_TITLE)
+    popen().poll.side_effect = lambda: None
+    po.open()
+    popen.assert_called_with("vlc.exe --input-title-format \"" + UNICODE_TITLE.encode(get_filesystem_encoding()) + "\" -",
+                             bufsize=ANY, stderr=ANY, stdout=ANY, stdin=ANY)
+
+
+@py3_only
+@windows_only
+@patch('subprocess.Popen')
+def test_output_mpv_unicode_title_windows_py3(popen):
+    po = PlayerOutput("mpv.exe", title=UNICODE_TITLE)
+    popen().poll.side_effect = lambda: None
+    po.open()
+    popen.assert_called_with("mpv.exe \"--title=" + UNICODE_TITLE + "\" -",
+                             bufsize=ANY, stderr=ANY, stdout=ANY, stdin=ANY)
+
+
+@py3_only
+@windows_only
+@patch('subprocess.Popen')
+def test_output_vlc_unicode_title_windows_py3(popen):
+    po = PlayerOutput("vlc.exe", title=UNICODE_TITLE)
+    popen().poll.side_effect = lambda: None
+    po.open()
+    popen.assert_called_with("vlc.exe --input-title-format \"" + UNICODE_TITLE + "\" -",
+                             bufsize=ANY, stderr=ANY, stdout=ANY, stdin=ANY)
diff --git a/tests/test_cmdline_title.py b/tests/test_cmdline_title.py
--- a/tests/test_cmdline_title.py
+++ b/tests/test_cmdline_title.py
@@ -1,7 +1,7 @@
 # -*- coding: utf-8 -*-
 import unittest
 
-from streamlink.compat import is_win32, is_py3
+from streamlink.compat import is_win32, is_py3, is_py2
 from streamlink.utils import get_filesystem_encoding
 from tests.test_cmdline import CommandLineTestCase
 
@@ -28,6 +28,10 @@ def test_open_player_with_title_mpv(self):
         self._test_args(["streamlink", "-p", "/usr/bin/mpv", "--title", "{title}", "http://test.se", "test"],
                         ["/usr/bin/mpv", "--title=Test Title", "-"])
 
+    def test_unicode_title_2444(self):
+        self._test_args(["streamlink", "-p", "mpv", "-t", "★", "http://test.se", "test"],
+                        ["mpv", u'--title=\u2605', "-"])
+
 
 @unittest.skipIf(not is_win32, "test only applicable on Windows")
 class TestCommandLineWithTitleWindows(CommandLineTestCase):
@@ -77,7 +81,7 @@ def test_open_player_with_title_pot(self):
             passthrough=True
         )
 
-    @unittest.skipIf(is_py3, "Encoding is different in Python 2")
+    @unittest.skipUnless(is_py2, "Encoding is different in Python 2")
     def test_open_player_with_unicode_author_pot_py2(self):
         self._test_args(
             ["streamlink", "-p", "\"c:\\Program Files\\DAUM\\PotPlayer\\PotPlayerMini64.exe\"",
@@ -88,7 +92,7 @@ def test_open_player_with_unicode_author_pot_py2(self):
             passthrough=True
         )
 
-    @unittest.skipIf(not is_py3, "Encoding is different in Python 2")
+    @unittest.skipUnless(is_py3, "Encoding is different in Python 3")
     def test_open_player_with_unicode_author_pot_py3(self):
         self._test_args(
             ["streamlink", "-p", "\"c:\\Program Files\\DAUM\\PotPlayer\\PotPlayerMini64.exe\"",
@@ -106,3 +110,13 @@ def test_open_player_with_default_title_pot(self):
             + "\"http://test.se/playlist.m3u8\\http://test.se/stream\"",
             passthrough=True
         )
+
+    @unittest.skipUnless(is_py2, "test only valid for Python 2")
+    def test_unicode_title_2444_py2(self):
+        self._test_args(["streamlink", "-p", "mpv", "-t", u"★".encode(get_filesystem_encoding()), "http://test.se", "test"],
+                        "mpv --title=" + u"★".encode(get_filesystem_encoding()) + " -")
+
+    @unittest.skipUnless(is_py3, "test only valid for Python 3")
+    def test_unicode_title_2444_py3(self):
+        self._test_args(["streamlink", "-p", "mpv", "-t", "★", "http://test.se", "test"],
+                        "mpv --title=★ -")