Repository: yt-dlp/yt-dlp
Pull request #7765 (2023-08-03 18:59:12)

TBS Free (Japan) extractor
**IMPORTANT**: PRs without the template will be CLOSED

### Description of your *pull request* and other information

<!--

Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible

-->

This PR adds extractors for Japanese [TBS TV](https://en.wikipedia.org/wiki/TBS_Television_(Japan))'s catch-up site, TBS FREE.
It supports individual episodes of a programme, whole programme playlists, and category playlists that can contain both episodes and whole programmes.
TBS FREE is only available in Japan, but some metadata can be extracted internationally. The geoblock can be bypassed with a VPN.

Subtitle downloading won't work unless #7681 is merged.


<details open><summary>Template</summary> <!-- OPEN is intentional -->

<!--

# PLEASE FOLLOW THE GUIDE BELOW

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check all of the following options that apply:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [ ] Fix or improvement to an extractor (Make sure to add/update tests)
- [x] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [ ] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))


<!-- Do NOT edit/remove anything below this! -->
</details><details><summary>Copilot Summary</summary>  

<!--
copilot:all
-->
### <samp>🤖 Generated by Copilot at 9cd12f7</samp>

### Summary
🇯🇵📺🐍

<!--
1.  🇯🇵 - This emoji represents Japan, the country where TBS is based and where the videos are geo-restricted to. It also indicates that the extractors are specific to the Japanese version of TBS, not the American one.
2. 📺 - This emoji represents TV, the medium that TBS produces and streams. It also indicates that the extractors deal with TV shows and episodes, not movies or other types of videos.
3. 🐍 - This emoji represents Python, the programming language that yt-dlp and the extractors are written in. It also indicates that the change involves adding a new module and importing classes.
-->
This pull request adds support for downloading videos from the Japanese streaming service TBS. It introduces a new module `tbsjp.py` that defines three extractors for different types of TBS webpages. It also imports these extractors in `_extractors.py` to register them with yt-dlp.

> _`yt-dlp` can now_
> _download from TBS Japan_
> _autumn leaves streaming_

### Walkthrough
*  Add support for downloading videos from TBS, a Japanese streaming service ([link](https://github.com/yt-dlp/yt-dlp/pull/7765/files?diff=unified&w=0#diff-780b22dc7eb280f5a7b2bbf79aff17826de88ddcbf2fc1116ba19901827aa4e3R1904-R1908), [link](https://github.com/yt-dlp/yt-dlp/pull/7765/files?diff=unified&w=0#diff-bc8c5e84ec259edccabfecfaad24147e884278f8062045302c2caf8084a6db38R1-R161))
  * Import TBS extractor classes from `tbsjp.py` in `_extractors.py` ([link](https://github.com/yt-dlp/yt-dlp/pull/7765/files?diff=unified&w=0#diff-780b22dc7eb280f5a7b2bbf79aff17826de88ddcbf2fc1116ba19901827aa4e3R1904-R1908))
  * Define TBS extractor classes in `tbsjp.py` that inherit from InfoExtractor and implement _real_extract ([link](https://github.com/yt-dlp/yt-dlp/pull/7765/files?diff=unified&w=0#diff-bc8c5e84ec259edccabfecfaad24147e884278f8062045302c2caf8084a6db38R1-R161))
  * Handle geo-restriction, JSON parsing, and URL construction in the TBS extractor classes ([link](https://github.com/yt-dlp/yt-dlp/pull/7765/files?diff=unified&w=0#diff-bc8c5e84ec259edccabfecfaad24147e884278f8062045302c2caf8084a6db38R1-R161))



</details>



Patch:
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index 87bca5bbe..666d89b46 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -34,7 +34,7 @@
 from .extractor.openload import PhantomJSwrapper
 from .minicurses import format_text
 from .networking import HEADRequest, Request, RequestDirector
-from .networking.common import _REQUEST_HANDLERS
+from .networking.common import _REQUEST_HANDLERS, _RH_PREFERENCES
 from .networking.exceptions import (
     HTTPError,
     NoSupportingHandlers,
@@ -683,7 +683,7 @@ def process_color_policy(stream):
         self.params['http_headers'] = HTTPHeaderDict(std_headers, self.params.get('http_headers'))
         self._load_cookies(self.params['http_headers'].get('Cookie'))  # compat
         self.params['http_headers'].pop('Cookie', None)
-        self._request_director = self.build_request_director(_REQUEST_HANDLERS.values())
+        self._request_director = self.build_request_director(_REQUEST_HANDLERS.values(), _RH_PREFERENCES)
 
         if auto_init and auto_init != 'no_verbose_header':
             self.print_debug_header()
@@ -4077,7 +4077,7 @@ def urlopen(self, req):
         except HTTPError as e:  # TODO: Remove in a future release
             raise _CompatHTTPError(e) from e
 
-    def build_request_director(self, handlers):
+    def build_request_director(self, handlers, preferences=None):
         logger = _YDLLogger(self)
         headers = self.params['http_headers'].copy()
         proxies = self.proxies.copy()
@@ -4106,6 +4106,7 @@ def build_request_director(self, handlers):
                     },
                 }),
             ))
+        director.preferences.update(preferences or [])
         return director
 
     def encode(self, s):
diff --git a/yt_dlp/cookies.py b/yt_dlp/cookies.py
index 157f5b0c2..a71fbc28b 100644
--- a/yt_dlp/cookies.py
+++ b/yt_dlp/cookies.py
@@ -138,7 +138,7 @@ def _extract_firefox_cookies(profile, container, logger):
         containers_path = os.path.join(os.path.dirname(cookie_database_path), 'containers.json')
         if not os.path.isfile(containers_path) or not os.access(containers_path, os.R_OK):
             raise FileNotFoundError(f'could not read containers.json in {search_root}')
-        with open(containers_path) as containers:
+        with open(containers_path, encoding='utf8') as containers:
             identities = json.load(containers).get('identities', [])
         container_id = next((context.get('userContextId') for context in identities if container in (
             context.get('name'),
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
index 63bb55ea7..d4d3b6074 100644
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -1901,6 +1901,11 @@
 from .tagesschau import TagesschauIE
 from .tass import TassIE
 from .tbs import TBSIE
+from .tbsjp import (
+    TBSJPEpisodeIE,
+    TBSJPProgramIE,
+    TBSJPPlaylistIE,
+)
 from .tdslifeway import TDSLifewayIE
 from .teachable import (
     TeachableIE,
diff --git a/yt_dlp/extractor/cbc.py b/yt_dlp/extractor/cbc.py
index 41e092422..9413281a5 100644
--- a/yt_dlp/extractor/cbc.py
+++ b/yt_dlp/extractor/cbc.py
@@ -161,7 +161,7 @@ class CBCPlayerIE(InfoExtractor):
             'upload_date': '20160210',
             'uploader': 'CBCC-NEW',
         },
-        'skip': 'Geo-restricted to Canada',
+        'skip': 'Geo-restricted to Canada and no longer available',
     }, {
         # Redirected from http://www.cbc.ca/player/AudioMobile/All%20in%20a%20Weekend%20Montreal/ID/2657632011/
         'url': 'http://www.cbc.ca/player/play/2657631896',
@@ -174,6 +174,9 @@ class CBCPlayerIE(InfoExtractor):
             'timestamp': 1425704400,
             'upload_date': '20150307',
             'uploader': 'CBCC-NEW',
+            'thumbnail': 'http://thumbnails.cbc.ca/maven_legacy/thumbnails/sonali-karnick-220.jpg',
+            'chapters': [],
+            'duration': 494.811,
         },
     }, {
         'url': 'http://www.cbc.ca/player/play/2164402062',
@@ -186,6 +189,28 @@ class CBCPlayerIE(InfoExtractor):
             'timestamp': 1320410746,
             'upload_date': '20111104',
             'uploader': 'CBCC-NEW',
+            'thumbnail': 'https://thumbnails.cbc.ca/maven_legacy/thumbnails/277/67/cancer_852x480_2164412612.jpg',
+            'chapters': [],
+            'duration': 186.867,
+        },
+    }, {
+        # Has subtitles
+        # These broadcasts expire after ~1 month, can find new test URL here:
+        # https://www.cbc.ca/player/news/TV%20Shows/The%20National/Latest%20Broadcast
+        'url': 'http://www.cbc.ca/player/play/2249992771553',
+        'md5': '2f2fb675dd4f0f8a5bb7588d1b13bacd',
+        'info_dict': {
+            'id': '2249992771553',
+            'ext': 'mp4',
+            'title': 'The National | Women’s soccer pay, Florida seawater, Swift quake',
+            'description': 'md5:adba28011a56cfa47a080ff198dad27a',
+            'timestamp': 1690596000,
+            'duration': 2716.333,
+            'subtitles': {'eng': [{'ext': 'vtt', 'protocol': 'm3u8_native'}]},
+            'thumbnail': 'https://thumbnails.cbc.ca/maven_legacy/thumbnails/481/326/thumbnail.jpeg',
+            'uploader': 'CBCC-NEW',
+            'chapters': 'count:5',
+            'upload_date': '20230729',
         },
     }]
 
@@ -199,6 +224,7 @@ def _real_extract(self, url):
                     'force_smil_url': True
                 }),
             'id': video_id,
+            '_format_sort_fields': ('res', 'proto')  # Prioritize direct http formats over HLS
         }
 
 
diff --git a/yt_dlp/extractor/scrippsnetworks.py b/yt_dlp/extractor/scrippsnetworks.py
index c3cee6e4a..adfd7e5f2 100644
--- a/yt_dlp/extractor/scrippsnetworks.py
+++ b/yt_dlp/extractor/scrippsnetworks.py
@@ -115,6 +115,7 @@ class ScrippsNetworksIE(InfoExtractor):
             'uploader': 'SCNI-SCND',
         },
         'add_ie': ['ThePlatform'],
+        'expected_warnings': ['No HLS formats found'],
     }, {
         'url': 'https://www.diynetwork.com/videos/diy-barnwood-tablet-stand-0265790',
         'only_matching': True,
diff --git a/yt_dlp/extractor/tbsjp.py b/yt_dlp/extractor/tbsjp.py
new file mode 100644
index 000000000..77ddeca32
--- /dev/null
+++ b/yt_dlp/extractor/tbsjp.py
@@ -0,0 +1,152 @@
+from .common import InfoExtractor
+from ..networking.exceptions import HTTPError
+from ..utils import (
+    ExtractorError,
+    clean_html,
+    get_element_text_and_html_by_tag,
+    int_or_none,
+    str_or_none,
+    traverse_obj,
+    try_call,
+    unified_timestamp,
+    urljoin,
+)
+
+
+class TBSJPEpisodeIE(InfoExtractor):
+    _VALID_URL = r'https?://cu\.tbs\.co\.jp/episode/(?P<id>[\d_]+)'
+    _GEO_BYPASS = False
+    _TESTS = [{
+        'url': 'https://cu.tbs.co.jp/episode/23613_2044134_1000049010',
+        'skip': 'streams geo-restricted, Japan only. Also, will likely expire eventually',
+        'info_dict': {
+            'title': 'VIVANT 第三話 誤送金完結へ!絶体絶命の反撃開始',
+            'id': '23613_2044134_1000049010',
+            'ext': 'mp4',
+            'upload_date': '20230728',
+            'duration': 3517,
+            'release_timestamp': 1691118230,
+            'episode': '第三話 誤送金完結へ!絶体絶命の反撃開始',
+            'release_date': '20230804',
+            'categories': 'count:11',
+            'episode_number': 3,
+            'timestamp': 1690522538,
+            'description': 'md5:2b796341af1ef772034133174ba4a895',
+            'series': 'VIVANT',
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        meta = self._search_json(r'window\.app\s*=', webpage, 'episode info', video_id, fatal=False)
+        episode = traverse_obj(meta, ('falcorCache', 'catalog', 'episode', video_id, 'value'))
+
+        tf_path = self._search_regex(
+            r'<script[^>]+src=["\'](/assets/tf\.[^"\']+\.js)["\']', webpage, 'stream API config')
+        tf_js = self._download_webpage(urljoin(url, tf_path), video_id, note='Downloading stream API config')
+        video_url = self._search_regex(r'videoPlaybackUrl:\s*[\'"]([^\'"]+)[\'"]', tf_js, 'stream API url')
+        api_key = self._search_regex(r'api_key:\s*[\'"]([^\'"]+)[\'"]', tf_js, 'stream API key')
+
+        try:
+            source_meta = self._download_json(f'{video_url}ref:{video_id}', video_id,
+                                              headers={'X-Streaks-Api-Key': api_key},
+                                              note='Downloading stream metadata')
+        except ExtractorError as e:
+            if isinstance(e.cause, HTTPError) and e.cause.status == 403:
+                self.raise_geo_restricted(countries=['JP'])
+            raise
+
+        formats, subtitles = [], {}
+        for src in traverse_obj(source_meta, ('sources', ..., 'src')):
+            fmts, subs = self._extract_m3u8_formats_and_subtitles(src, video_id, fatal=False)
+            formats.extend(fmts)
+            self._merge_subtitles(subs, target=subtitles)
+
+        return {
+            'title': try_call(lambda: clean_html(get_element_text_and_html_by_tag('h3', webpage)[0])),
+            'id': video_id,
+            **traverse_obj(episode, {
+                'categories': ('keywords', {list}),
+                'id': ('content_id', {str}),
+                'description': ('description', 0, 'value'),
+                'timestamp': ('created_at', {unified_timestamp}),
+                'release_timestamp': ('pub_date', {unified_timestamp}),
+                'duration': ('tv_episode_info', 'duration', {int_or_none}),
+                'episode_number': ('tv_episode_info', 'episode_number', {int_or_none}),
+                'episode': ('title', lambda _, v: not v.get('is_phonetic'), 'value'),
+                'series': ('custom_data', 'program_name'),
+            }, get_all=False),
+            'formats': formats,
+            'subtitles': subtitles,
+        }
+
+
+class TBSJPProgramIE(InfoExtractor):
+    _VALID_URL = r'https?://cu\.tbs\.co\.jp/program/(?P<id>\d+)'
+    _TESTS = [{
+        'url': 'https://cu.tbs.co.jp/program/23601',
+        'playlist_mincount': 4,
+        'info_dict': {
+            'id': '23601',
+            'categories': ['エンタメ', 'ミライカプセル', '会社', '働く', 'バラエティ', '動画'],
+            'description': '幼少期の夢は大人になって、どう成長したのだろうか？\nそしてその夢は今後、どのように広がっていくのか？\nいま話題の会社で働く人の「夢の成長」を描く',
+            'series': 'ミライカプセル　-I have a dream-',
+            'title': 'ミライカプセル　-I have a dream-'
+        }
+    }]
+
+    def _real_extract(self, url):
+        programme_id = self._match_id(url)
+        webpage = self._download_webpage(url, programme_id)
+        meta = self._search_json(r'window\.app\s*=', webpage, 'programme info', programme_id)
+
+        programme = traverse_obj(meta, ('falcorCache', 'catalog', 'program', programme_id, 'false', 'value'))
+
+        return {
+            '_type': 'playlist',
+            'entries': [self.url_result(f'https://cu.tbs.co.jp/episode/{video_id}', TBSJPEpisodeIE, video_id)
+                        for video_id in traverse_obj(programme, ('custom_data', 'seriesList', 'episodeCode', ...))],
+            'id': programme_id,
+            **traverse_obj(programme, {
+                'categories': ('keywords', ...),
+                'id': ('tv_episode_info', 'show_content_id', {str_or_none}),
+                'description': ('custom_data', 'program_description'),
+                'series': ('custom_data', 'program_name'),
+                'title': ('custom_data', 'program_name'),
+            }),
+        }
+
+
+class TBSJPPlaylistIE(InfoExtractor):
+    _VALID_URL = r'https?://cu\.tbs\.co\.jp/playlist/(?P<id>[\da-f]+)'
+    _TESTS = [{
+        'url': 'https://cu.tbs.co.jp/playlist/184f9970e7ba48e4915f1b252c55015e',
+        'playlist_mincount': 4,
+        'info_dict': {
+            'title': 'まもなく配信終了',
+            'id': '184f9970e7ba48e4915f1b252c55015e',
+        }
+    }]
+
+    def _real_extract(self, url):
+        playlist_id = self._match_id(url)
+        page = self._download_webpage(url, playlist_id)
+        meta = self._search_json(r'window\.app\s*=', page, 'playlist info', playlist_id)
+        playlist = traverse_obj(meta, ('falcorCache', 'playList', playlist_id))
+
+        def entries():
+            for entry in traverse_obj(playlist, ('catalogs', 'value', lambda _, v: v['content_id'])):
+                # TODO: it's likely possible to get all metadata from the playlist page json instead
+                content_id = entry['content_id']
+                content_type = entry.get('content_type')
+                if content_type == 'tv_show':
+                    yield self.url_result(
+                        f'https://cu.tbs.co.jp/program/{content_id}', TBSJPProgramIE, content_id)
+                elif content_type == 'tv_episode':
+                    yield self.url_result(
+                        f'https://cu.tbs.co.jp/episode/{content_id}', TBSJPEpisodeIE, content_id)
+                else:
+                    self.report_warning(f'Skipping "{content_id}" with unsupported content_type "{content_type}"')
+
+        return self.playlist_result(entries(), playlist_id, traverse_obj(playlist, ('display_name', 'value')))
diff --git a/yt_dlp/extractor/theplatform.py b/yt_dlp/extractor/theplatform.py
index 8307b912d..99caeb5f9 100644
--- a/yt_dlp/extractor/theplatform.py
+++ b/yt_dlp/extractor/theplatform.py
@@ -19,7 +19,11 @@
     xpath_with_ns,
     mimetype2ext,
     find_xpath_attr,
+    traverse_obj,
+    update_url,
+    urlhandle_detect_ext,
 )
+from ..networking import HEADRequest
 
 default_ns = 'http://www.w3.org/2005/SMIL21/Language'
 _x = lambda p: xpath_with_ns(p, {'smil': default_ns})
@@ -162,7 +166,8 @@ class ThePlatformIE(ThePlatformBaseIE, AdobePassIE):
         'params': {
             # rtmp download
             'skip_download': True,
-        }
+        },
+        'skip': '404 Not Found',
     }, {
         'url': 'https://player.theplatform.com/p/D6x-PC/pulse_preview/embed/select/media/yMBg9E8KFxZD',
         'info_dict': {
@@ -171,7 +176,8 @@ class ThePlatformIE(ThePlatformBaseIE, AdobePassIE):
             'description': 'md5:644ad9188d655b742f942bf2e06b002d',
             'title': 'HIGHLIGHTS: USA bag first ever series Cup win',
             'uploader': 'EGSM',
-        }
+        },
+        'skip': '404 Not Found',
     }, {
         'url': 'http://player.theplatform.com/p/NnzsPC/widget/select/media/4Y0TlYUr_ZT7',
         'only_matching': True,
@@ -189,6 +195,7 @@ class ThePlatformIE(ThePlatformBaseIE, AdobePassIE):
             'upload_date': '20150701',
             'uploader': 'NBCU-NEWS',
         },
+        'skip': '404 Not Found',
     }, {
         # From http://www.nbc.com/the-blacklist/video/sir-crispin-crandall/2928790?onid=137781#vc137781=1
         # geo-restricted (US), HLS encrypted with AES-128
@@ -295,6 +302,17 @@ def _real_extract(self, url):
 
         formats, subtitles = self._extract_theplatform_smil(smil_url, video_id)
 
+        # With some sites, manifest URL must be forced to extract HLS formats
+        if not traverse_obj(formats, lambda _, v: v['format_id'].startswith('hls')):
+            m3u8_url = update_url(url, query='mbr=true&manifest=m3u', fragment=None)
+            urlh = self._request_webpage(
+                HEADRequest(m3u8_url), video_id, 'Checking for HLS formats', 'No HLS formats found', fatal=False)
+            if urlh and urlhandle_detect_ext(urlh) == 'm3u8':
+                m3u8_fmts, m3u8_subs = self._extract_m3u8_formats_and_subtitles(
+                    m3u8_url, video_id, m3u8_id='hls', fatal=False)
+                formats.extend(m3u8_fmts)
+                self._merge_subtitles(m3u8_subs, target=subtitles)
+
         ret = self._extract_theplatform_metadata(path, video_id)
         combined_subtitles = self._merge_subtitles(ret.get('subtitles', {}), subtitles)
         ret.update({
diff --git a/yt_dlp/extractor/youtube.py b/yt_dlp/extractor/youtube.py
index 1e16631b1..023d8fd8c 100644
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -496,16 +496,10 @@ def _initialize_consent(self):
         cookies = self._get_cookies('https://www.youtube.com/')
         if cookies.get('__Secure-3PSID'):
             return
-        consent_id = None
-        consent = cookies.get('CONSENT')
-        if consent:
-            if 'YES' in consent.value:
-                return
-            consent_id = self._search_regex(
-                r'PENDING\+(\d+)', consent.value, 'consent', default=None)
-        if not consent_id:
-            consent_id = random.randint(100, 999)
-        self._set_cookie('.youtube.com', 'CONSENT', 'YES+cb.20210328-17-p0.en+FX+%s' % consent_id)
+        socs = cookies.get('SOCS')
+        if socs and not socs.value.startswith('CAA'):  # not consented
+            return
+        self._set_cookie('.youtube.com', 'SOCS', 'CAI', secure=True)  # accept all (required for mixes)
 
     def _initialize_pref(self):
         cookies = self._get_cookies('https://www.youtube.com/')
diff --git a/yt_dlp/networking/common.py b/yt_dlp/networking/common.py
index 8fba8c1c5..584c7bb4d 100644
--- a/yt_dlp/networking/common.py
+++ b/yt_dlp/networking/common.py
@@ -31,8 +31,19 @@
 )
 from ..utils.networking import HTTPHeaderDict, normalize_url
 
-if typing.TYPE_CHECKING:
-    RequestData = bytes | Iterable[bytes] | typing.IO | None
+
+def register_preference(*handlers: type[RequestHandler]):
+    assert all(issubclass(handler, RequestHandler) for handler in handlers)
+
+    def outer(preference: Preference):
+        @functools.wraps(preference)
+        def inner(handler, *args, **kwargs):
+            if not handlers or isinstance(handler, handlers):
+                return preference(handler, *args, **kwargs)
+            return 0
+        _RH_PREFERENCES.add(inner)
+        return inner
+    return outer
 
 
 class RequestDirector:
@@ -40,12 +51,17 @@ class RequestDirector:
 
     Helper class that, when given a request, forward it to a RequestHandler that supports it.
 
+    Preference functions in the form of func(handler, request) -> int
+    can be registered into the `preferences` set. These are used to sort handlers
+    in order of preference.
+
     @param logger: Logger instance.
     @param verbose: Print debug request information to stdout.
     """
 
     def __init__(self, logger, verbose=False):
         self.handlers: dict[str, RequestHandler] = {}
+        self.preferences: set[Preference] = set()
         self.logger = logger  # TODO(Grub4k): default logger
         self.verbose = verbose
 
@@ -58,6 +74,16 @@ def add_handler(self, handler: RequestHandler):
         assert isinstance(handler, RequestHandler), 'handler must be a RequestHandler'
         self.handlers[handler.RH_KEY] = handler
 
+    def _get_handlers(self, request: Request) -> list[RequestHandler]:
+        """Sorts handlers by preference, given a request"""
+        preferences = {
+            rh: sum(pref(rh, request) for pref in self.preferences)
+            for rh in self.handlers.values()
+        }
+        self._print_verbose('Handler preferences for this request: %s' % ', '.join(
+            f'{rh.RH_NAME}={pref}' for rh, pref in preferences.items()))
+        return sorted(self.handlers.values(), key=preferences.get, reverse=True)
+
     def _print_verbose(self, msg):
         if self.verbose:
             self.logger.stdout(f'director: {msg}')
@@ -73,8 +99,7 @@ def send(self, request: Request) -> Response:
 
         unexpected_errors = []
         unsupported_errors = []
-        # TODO (future): add a per-request preference system
-        for handler in reversed(list(self.handlers.values())):
+        for handler in self._get_handlers(request):
             self._print_verbose(f'Checking if "{handler.RH_NAME}" supports this request.')
             try:
                 handler.validate(request)
@@ -530,3 +555,10 @@ def info(self):
     def getheader(self, name, default=None):
         deprecation_warning('Response.getheader() is deprecated, use Response.get_header', stacklevel=2)
         return self.get_header(name, default)
+
+
+if typing.TYPE_CHECKING:
+    RequestData = bytes | Iterable[bytes] | typing.IO | None
+    Preference = typing.Callable[[RequestHandler, Request], int]
+
+_RH_PREFERENCES: set[Preference] = set()


Test patch:
diff --git a/test/test_networking.py b/test/test_networking.py
index 9c33b0d4c..2622d24da 100644
--- a/test/test_networking.py
+++ b/test/test_networking.py
@@ -1035,17 +1035,17 @@ def test_send(self):
         assert isinstance(director.send(Request('http://')), FakeResponse)
 
     def test_unsupported_handlers(self):
-        director = RequestDirector(logger=FakeLogger())
-        director.add_handler(FakeRH(logger=FakeLogger()))
-
         class SupportedRH(RequestHandler):
             _SUPPORTED_URL_SCHEMES = ['http']
 
             def _send(self, request: Request):
                 return Response(fp=io.BytesIO(b'supported'), headers={}, url=request.url)
 
-        # This handler should by default take preference over FakeRH
+        director = RequestDirector(logger=FakeLogger())
         director.add_handler(SupportedRH(logger=FakeLogger()))
+        director.add_handler(FakeRH(logger=FakeLogger()))
+
+        # First should take preference
         assert director.send(Request('http://')).read() == b'supported'
         assert director.send(Request('any://')).read() == b''
 
@@ -1072,6 +1072,27 @@ def _send(self, request: Request):
         director.add_handler(UnexpectedRH(logger=FakeLogger))
         assert director.send(Request('any://'))
 
+    def test_preference(self):
+        director = RequestDirector(logger=FakeLogger())
+        director.add_handler(FakeRH(logger=FakeLogger()))
+
+        class SomeRH(RequestHandler):
+            _SUPPORTED_URL_SCHEMES = ['http']
+
+            def _send(self, request: Request):
+                return Response(fp=io.BytesIO(b'supported'), headers={}, url=request.url)
+
+        def some_preference(rh, request):
+            return (0 if not isinstance(rh, SomeRH)
+                    else 100 if 'prefer' in request.headers
+                    else -1)
+
+        director.add_handler(SomeRH(logger=FakeLogger()))
+        director.preferences.add(some_preference)
+
+        assert director.send(Request('http://')).read() == b''
+        assert director.send(Request('http://', headers={'prefer': '1'})).read() == b'supported'
+
 
 # XXX: do we want to move this to test_YoutubeDL.py?
 class TestYoutubeDLNetworking: