Repository: yt-dlp/yt-dlp

Issue:
[Bug] aria2c error on Unicode character
```
Exception: [../../aria2-release-1.35.0/src/AbstractOptionHandler.cc:69] errorCode=28 We encountered a problem while processing the option '--console-log-level'.
  -> [../../aria2-release-1.35.0/src/OptionHandlerImpl.cc:435] errorCode=1 console-log-level must be one of the following:'debug' 'info' 'notice' 'warn' 'error'
Usage:
 --console-log-level=LEVEL    Set log level to output to console.
                              Possible Values: debug, info, notice, warn, error
                              Default: notice
                              Tags: #advanced
[download] Finished downloading playlist: 1
```

_Originally posted by @candrapersada in https://github.com/pukkandan/yt-dlp/issues/55#issuecomment-776899332_


Discussion:
@candrapersada Please give the full verbose log
```$ youtube-dlc --external-downloader aria2c https://www.youtube.com/playlist?list=PLN5V6aiYlgnvVxc9_QcD_3BVSbeRw5vr4 -v --playlist-items 1
[debug] Command-line config: ['--external-downloader', 'aria2c', 'https://www.youtube.com/playlist?list=PLN5V6aiYlgnvVxc9_QcD_3BVSbeRw5vr4', '-v', '--playlist-items', '1']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] yt-dlp version 2021.02.09
[debug] Plugin Extractors: ['SamplePlugin']
[debug] Git HEAD: 8d801631c
[debug] Python version 3.8.4 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg 4.3.1-2020-11-19-full_build-www.gyan.dev, ffprobe 4.3.1-2020-11-19-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube:tab] PLN5V6aiYlgnvVxc9_QcD_3BVSbeRw5vr4: Downloading webpage
[download] Downloading playlist: アルトネリコ1
[youtube:tab] playlist アルトネリコ1: Downloading 1 videos
[download] Downloading video 1 of 1
[youtube] s3ErYKn70Zw: Downloading webpage
[youtube] s3ErYKn70Zw: Downloading MPD manifest
[debug] Sort order given by user:
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, vcodec:vp9(9), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] Downloading format(s) 244+140
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on 'https://manifest.googlevideo.com/api/manifest/dash/expire/1613005337/ei/uS0kYPvVJOGq8QOnlpG4CA/ip/2405%3A201%3Af003%3Ab167%3A357a%3Ab92d%3A59b8%3A8b3a/id/b3712b60a9fbd19c/source/youtube/requiressl/yes/playback_host/r1---sn-gwpa-jv36.googlevideo.com/mh/cG/mm/31%2C29/mn/sn-gwpa-jv36%2Csn-gwpa-h55r/ms/au%2Crdu/mv/m/mvi/1/pl/38/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/253750/vprv/1/mt/1612983399/fvip/1/keepalive/yes/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRAIgEX6UtsO6okKGRWPyUtdIP7Fk_tvRBHxwVs_86ZJHUbMCIFVkTd4BVcjAUwN8Q6xQhDfHIkKe0EAARlP7a-h4-BAS/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgWAuzq57p68-nhhUmjoKDua2UrfbphfnmuDk9DQw0uqoCIBs5hc4iVl6khdaK4aTc67-Wk16NbZG5cpxGMTfftSD2'
[dashsegments] Total fragments: 193
[download] Destination: アルトネリコ　Phase 0 序 [s3ErYKn70Zw].f244.webm
ERROR: 'charmap' codec can't encode characters in position 751-757: character maps to <undefined>
Traceback (most recent call last):
  File "<REDACTED>\youtube_dlc\YoutubeDL.py", line 1004, in wrapper
    return func(self, *args, **kwargs)
  File "<REDACTED>\youtube_dlc\YoutubeDL.py", line 1312, in __process_iterable_entry
    return self.process_ie_result(
  File "<REDACTED>\youtube_dlc\YoutubeDL.py", line 1110, in process_ie_result
    return self.process_ie_result(
  File "<REDACTED>\youtube_dlc\YoutubeDL.py", line 1073, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "<REDACTED>\youtube_dlc\YoutubeDL.py", line 1943, in process_video_result
    self.process_info(new_info)
  File "<REDACTED>\youtube_dlc\YoutubeDL.py", line 2347, in process_info
    partial_success, real_download = dl(fname, new_info)
  File "<REDACTED>\youtube_dlc\YoutubeDL.py", line 2120, in dl
    return fd.download(name, info, subtitle)
  File "<REDACTED>\youtube_dlc\downloader\common.py", line 375, in download
    return self.real_download(filename, info_dict), True
  File "<REDACTED>\youtube_dlc\downloader\dash.py", line 98, in real_download
    success = fd.real_download(filename, info_copy)
  File "<REDACTED>\youtube_dlc\downloader\external.py", line 47, in real_download
    retval = self._call_downloader(tmpfilename, info_dict)
  File "<REDACTED>\youtube_dlc\downloader\external.py", line 114, in _call_downloader
    cmd = [encodeArgument(a) for a in self._make_cmd(tmpfilename, info_dict)]
  File "<REDACTED>\youtube_dlc\downloader\external.py", line 267, in _make_cmd
    f.write('\n'.join(url_list))
  File "<REDACTED>\Python\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 751-757: character maps to <undefined>
```

@shirt-dev, can you have a look

Possibly related: https://www.reddit.com/r/youtubedl/comments/lgconf/ytdlp_release_20210209/gmtp1aa?context=3


Resolving patch:
diff --git a/youtube_dlc/downloader/external.py b/youtube_dlc/downloader/external.py
--- a/youtube_dlc/downloader/external.py
+++ b/youtube_dlc/downloader/external.py
@@ -5,7 +5,6 @@
 import subprocess
 import sys
 import time
-import shutil
 
 try:
     from Crypto.Cipher import AES
@@ -32,6 +31,7 @@
     is_outdated_version,
     process_communicate_or_kill,
     sanitized_Request,
+    sanitize_open,
 )
 
 
@@ -126,23 +126,27 @@ def _call_downloader(self, tmpfilename, info_dict):
             for [i, url] in enumerate(info_dict['url_list']):
                 tmpsegmentname = '%s_%s.frag' % (tmpfilename, i)
                 file_list.append(tmpsegmentname)
-            with open(tmpfilename, 'wb') as dest:
-                for i in file_list:
-                    if 'decrypt_info' in info_dict:
-                        decrypt_info = info_dict['decrypt_info']
-                        with open(i, 'rb') as src:
-                            if decrypt_info['METHOD'] == 'AES-128':
-                                iv = decrypt_info.get('IV')
-                                decrypt_info['KEY'] = decrypt_info.get('KEY') or self.ydl.urlopen(
-                                    self._prepare_url(info_dict, info_dict.get('_decryption_key_url') or decrypt_info['URI'])).read()
-                                encrypted_data = src.read()
-                                decrypted_data = AES.new(
-                                    decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(encrypted_data)
-                                dest.write(decrypted_data)
-                            else:
-                                shutil.copyfileobj(open(i, 'rb'), dest)
+            dest, _ = sanitize_open(tmpfilename, 'wb')
+            for i in file_list:
+                src, _ = sanitize_open(i, 'rb')
+                if 'decrypt_info' in info_dict:
+                    decrypt_info = info_dict['decrypt_info']
+                    if decrypt_info['METHOD'] == 'AES-128':
+                        iv = decrypt_info.get('IV')
+                        decrypt_info['KEY'] = decrypt_info.get('KEY') or self.ydl.urlopen(
+                            self._prepare_url(info_dict, info_dict.get('_decryption_key_url') or decrypt_info['URI'])).read()
+                        encrypted_data = src.read()
+                        decrypted_data = AES.new(
+                            decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(encrypted_data)
+                        dest.write(decrypted_data)
                     else:
-                        shutil.copyfileobj(open(i, 'rb'), dest)
+                        fragment_data = src.read()
+                        dest.write(fragment_data)
+                else:
+                    fragment_data = src.read()
+                    dest.write(fragment_data)
+                src.close()
+            dest.close()
             if not self.params.get('keep_fragments', False):
                 for file_path in file_list:
                     try:
@@ -263,8 +267,9 @@ def _make_cmd(self, tmpfilename, info_dict):
             for [i, url] in enumerate(info_dict['url_list']):
                 tmpsegmentname = '%s_%s.frag' % (os.path.basename(tmpfilename), i)
                 url_list.append('%s\n\tout=%s' % (url, tmpsegmentname))
-            with open(url_list_file, 'w') as f:
-                f.write('\n'.join(url_list))
+            stream, _ = sanitize_open(url_list_file, 'wb')
+            stream.write('\n'.join(url_list).encode('utf-8'))
+            stream.close()
 
             cmd += ['-i', url_list_file]
         else:


Test patch: