Repository: yt-dlp/yt-dlp

Issue:
[Broken] YouTube said: Unknown error.
<!--

######################################################################
  WARNING!
  IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################

-->


## Checklist

<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.10. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
- Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
- Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
-->

- [x] I'm reporting a broken site support
- [x] I've verified that I'm running yt-dlp version **2021.08.10**
- [x] I've checked that all provided URLs are alive and playable in a browser
- [x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [x] I've searched the bugtracker for similar issues including closed ones


## Verbose log

<!--
Provide the complete verbose output of yt-dlp that clearly demonstrates the problem.
Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
 [debug] System config: []
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
 [debug] yt-dlp version 2021.08.10
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
 <more lines>
-->

```
[debug] Command-line config: ['--verbose', '-o', '/mnt/sdb1/Videos/DanceSportTotal/%(title)s.%(ext)s', '-c', '-i', 'https://www.youtube.com/c/DanceSportTotal/videos', '--cookies', '/mnt/sdb1/cookies.txt', '--extractor-args', 'youtube:player_client=android', '--playlist-start=5000']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] yt-dlp version 2021.08.10 
[debug] Python version 3.8.10 (CPython 64bit) - Linux-5.4.0-81-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4
[debug] Optional libraries: keyring, mutagen, pycryptodome, sqlite, websockets
[debug] Proxy map: {}
[debug] [youtube:tab] Extracting URL: https://www.youtube.com/c/DanceSportTotal/videos
[youtube:tab] DanceSportTotal: Downloading webpage
[debug] [youtube:tab] Final URL: https://www.youtube.com/c/DanceSportTotal/videos
[download] Downloading playlist: DanceSportTotal - Videos
[youtube:tab] UC7nIDQ_jiPsbuK8D1fCjcLA page 1: Downloading API JSON
[youtube:tab] UC7nIDQ_jiPsbuK8D1fCjcLA page 2: Downloading API JSON
[youtube:tab] UC7nIDQ_jiPsbuK8D1fCjcLA page 3: Downloading API JSON
ERROR: YouTube said: Unknown error.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/YoutubeDL.py", line 1199, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/YoutubeDL.py", line 1424, in <lambda>
    lambda self, i: ie_entries[i - 1],
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/utils.py", line 4018, in __getitem__
    self.__cache.extend(itertools.islice(self.__iterable, n))
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/extractor/youtube.py", line 4200, in _smuggle_data
    for entry in entries:
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/extractor/youtube.py", line 3885, in _entries
    response = self._extract_response(
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/extractor/youtube.py", line 813, in _extract_response
    self._extract_and_report_alerts(response, expected=False)
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/extractor/youtube.py", line 741, in _extract_and_report_alerts
    return self._report_alerts(self._extract_alerts(data), *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/yt_dlp/extractor/youtube.py", line 738, in _report_alerts
    raise ExtractorError('YouTube said: %s' % errors[-1][1], expected=expected)
yt_dlp.utils.ExtractorError: YouTube said: Unknown error.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
```
<!--
Do not remove the above ```
-->


## Description

<!--
Provide an explanation of your issue in an arbitrary form. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->

WRITE DESCRIPTION HERE



Discussion:
유튜브 왭싸이트에서 접속을 시도해야할것이고 wifi를 거쳐 동영상을 받아야한다
라즈베리파이4인가요
Weird server side issue with YouTube (that message is coming from their API). I've seen this a few times myself.

Retrying should work.
아
코드를 보여줘
서버측의문제가 확실한가요
> 서버측의문제가 확실한가요
"Are server-side issues obvious?"

It depends. e.g. A 5XX HTTP response is usually a server side issue.
Same problem
> Weird server side issue with YouTube (that message is coming from their API). I've seen this a few times myself.
> 
> Retrying should work.

What's annoying about this error is that it is contained within an HTTP 200 response. We usually retry on network errors, however, this is not considered one.

In other words, this error is showing up in the same place we expect the "Playlist does not exist" type errors to be (that we do not want to retry on). So to retry, we might have to hard code this particular one?
maybe we can retry except in the first page? since we'd expect to see any normal error only there
> maybe we can retry except in the first page? since we'd expect to see any normal error only there


not sure. I think we ideally want this to be within extract_response, which doesn't know about pages.

Looking here: https://github.com/yt-dlp/yt-dlp/blob/a7429aa9fa3bc6616d9861a8ce5584a241a93ecc/yt_dlp/extractor/youtube.py#L816-L822
Since this error is caught and re-thrown here, I think we can get away with adding a check here for this error so it can retry?

Resolving patch:
diff --git a/yt_dlp/extractor/youtube.py b/yt_dlp/extractor/youtube.py
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -48,6 +48,7 @@
     parse_iso8601,
     parse_qs,
     qualities,
+    remove_end,
     remove_start,
     smuggle_url,
     str_or_none,
@@ -720,7 +721,7 @@ def _extract_alerts(cls, data):
                 if message:
                     yield alert_type, message
 
-    def _report_alerts(self, alerts, expected=True, fatal=True):
+    def _report_alerts(self, alerts, expected=True, fatal=True, only_once=False):
         errors = []
         warnings = []
         for alert_type, alert_message in alerts:
@@ -730,7 +731,7 @@ def _report_alerts(self, alerts, expected=True, fatal=True):
                 warnings.append([alert_type, alert_message])
 
         for alert_type, alert_message in (warnings + errors[:-1]):
-            self.report_warning('YouTube said: %s - %s' % (alert_type, alert_message))
+            self.report_warning('YouTube said: %s - %s' % (alert_type, alert_message), only_once=only_once)
         if errors:
             raise ExtractorError('YouTube said: %s' % errors[-1][1], expected=expected)
 
@@ -779,7 +780,7 @@ def _extract_response(self, item_id, query, note='Downloading API JSON', headers
         while count < retries:
             count += 1
             if last_error:
-                self.report_warning('%s. Retrying ...' % last_error)
+                self.report_warning('%s. Retrying ...' % remove_end(last_error, '.'))
             try:
                 response = self._call_api(
                     ep=ep, fatal=True, headers=headers,
@@ -814,8 +815,13 @@ def _extract_response(self, item_id, query, note='Downloading API JSON', headers
             else:
                 # Youtube may send alerts if there was an issue with the continuation page
                 try:
-                    self._extract_and_report_alerts(response, expected=False)
+                    self._extract_and_report_alerts(response, expected=False, only_once=True)
                 except ExtractorError as e:
+                    # YouTube servers may return errors we want to retry on in a 200 OK response
+                    # See: https://github.com/yt-dlp/yt-dlp/issues/839
+                    if 'unknown error' in e.msg.lower():
+                        last_error = e.msg
+                        continue
                     if fatal:
                         raise
                     self.report_warning(error_to_compat_str(e))
@@ -4285,7 +4291,7 @@ def get_mobj(url):
         # YouTube sometimes provides a button to reload playlist with unavailable videos.
         if 'no-youtube-unavailable-videos' not in compat_opts:
             data = self._reload_with_unavailable_videos(item_id, data, webpage) or data
-        self._extract_and_report_alerts(data)
+        self._extract_and_report_alerts(data, only_once=True)
         tabs = try_get(
             data, lambda x: x['contents']['twoColumnBrowseResultsRenderer']['tabs'], list)
         if tabs:


Test patch: