Repository: yt-dlp/yt-dlp
Pull request #7667 (2023-07-22 16:16:14)

[ie] Extract subtitles from SMIL manifests
`_extract_smil_formats_and_subtitles` and `_parse_smil_subtitles` only extract subtitles from the SMIL itself; `_parse_smil_formats` was ignoring all of the HLS/DASH/ISM manifest subtitles

Extractors that are directly impacted by this change:

- **`MediasetIE`**
   - it had redefined `_parse_smil_formats`, so PR needed to adapt it to redefine `_parse_smil_formats_and_subtitles`instead
   
- **`LivestreamIE`**
   - it had redefined `_parse_smil_formats`, so PR needed to adapt it to redefine `_parse_smil_formats_and_subtitles`instead
   
- **`SBSIE`**
   - no fix needed
   
- **`ThePlatformIE`**
   - PR adapts `_extract_theplatform_smil` to use `_parse_smil_formats_and_subtitles` instead of `_parse_smil_formats`
   - **indirectly impacted** are all extractors subclassed from `ThePlatformIE` or any extractor returning `url_result`s to it


Related #7484


<details open><summary>Template</summary> <!-- OPEN is intentional -->

<!--

# PLEASE FOLLOW THE GUIDE BELOW

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check all of the following options that apply:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [x] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [x] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))


<!-- Do NOT edit/remove anything below this! -->
</details><details><summary>Copilot Summary</summary>  

<!--
copilot:all
-->
### <samp>🤖 Generated by Copilot at 075e3ce</samp>

### Summary
🎞️🗣️🛠️

<!--
1.  🎞️ This emoji represents the SMIL format, which is a multimedia synchronization language that can be used to describe the temporal and spatial relationships between media objects. It is often used to create interactive presentations or animations. This emoji is relevant for the first, third, and fifth changes, which involve parsing and extracting subtitles from SMIL files.
2. 🗣️ This emoji represents the subtitles or captions that are added to videos to provide additional information or translation for the viewers. It is relevant for all the changes, which enhance the subtitle extraction and merging capabilities of the extractors.
3. 🛠️ This emoji represents the tools or modifications that are made to the code to improve its functionality or performance. It is relevant for the second and fourth changes, which remove or disable some features that are no longer needed or desired.
-->
This pull request improves the subtitle extraction and merging for various `InfoExtractor` subclasses that handle SMIL, M3U8, MPD, and ISM files. It adds a new parameter `extract_subs` to the superclass and overrides or modifies some functions in the subclasses. It also updates the test cases to match the new behavior.

> _`InfoExtractor`_
> _Enhances subtitles in spring_
> _`parse_subs` is new_

### Walkthrough
*  Add a new parameter `extract_subs` to the function `_extract_smil_formats_and_subtitles` in `common.py` to control whether to extract subtitles from SMIL files or not ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b7f6633815316063666fc3ecebb970e68fdcb5ee01beba03d04c01830e55e8a6L2243-R2244))
*  Pass the `extract_subs` parameter to the function `_parse_smil_formats_and_subtitles` and receive the subtitles returned by that function in `common.py` ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b7f6633815316063666fc3ecebb970e68fdcb5ee01beba03d04c01830e55e8a6L2257-R2269))
*  Rename the function `_parse_smil_formats` to `_parse_smil_formats_and_subtitles` in `common.py` to reflect its new functionality ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b7f6633815316063666fc3ecebb970e68fdcb5ee01beba03d04c01830e55e8a6L2329-R2337))
*  Add a new variable `subtitles` to the function `_parse_smil_formats_and_subtitles` in `common.py` to store the subtitles extracted from the SMIL file ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b7f6633815316063666fc3ecebb970e68fdcb5ee01beba03d04c01830e55e8a6L2337-R2345))
*  Use the functions `_extract_m3u8_formats_and_subtitles`, `_extract_mpd_formats_and_subtitles`, and `_extract_ism_formats_and_subtitles` instead of their counterparts without subtitles in `common.py` to enable the extraction and merging of subtitles from M3U8, MPD, and ISM files ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b7f6633815316063666fc3ecebb970e68fdcb5ee01beba03d04c01830e55e8a6L2385-R2397), [link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b7f6633815316063666fc3ecebb970e68fdcb5ee01beba03d04c01830e55e8a6L2407-R2430))
*  Return both the formats and the subtitles extracted from the SMIL file in the function `_parse_smil_formats_and_subtitles` in `common.py` ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b7f6633815316063666fc3ecebb970e68fdcb5ee01beba03d04c01830e55e8a6L2442-R2461))
*  Override the function `_parse_smil_formats_and_subtitles` in `livestream.py` and `mediaset.py` to return an empty dictionary for subtitles, since these subclasses do not extract subtitles from the SMIL file ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-de759949ea8a712263ea95214b4c24064f314128d2bcffbad76eca9e07073560L83-R84), [link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-de759949ea8a712263ea95214b4c24064f314128d2bcffbad76eca9e07073560L107-R108), [link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-b8ea51677c6dbdff3872a4ce0de126407a6f12dc5d4541e32371fb3364765ca5L157-R162))
*  Use the function `_parse_smil_formats_and_subtitles` and merge the subtitles from the SMIL file with the subtitles from the metadata in the function `_extract_theplatform_smil` in `theplatform.py` ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-665418b61f97d16d95c134f49e50b95c175a7b154685876b26f468916710e43aL48-R48), [link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-665418b61f97d16d95c134f49e50b95c175a7b154685876b26f468916710e43aR69))
*  Remove the expected warning `Ignoring subtitle tracks` from the test case of the `NBCIE` class in `nbc.py`, since this subclass no longer ignores subtitles from the SMIL file ([link](https://github.com/yt-dlp/yt-dlp/pull/7667/files?diff=unified&w=0#diff-db8d06cca3b41f67d5615e9bcf4c7f22a052e1d5a408d90a53c6579c7972ff9bL134))



</details>



Patch:
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index 64a280dc0..b69ac1d65 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -2248,18 +2248,10 @@ def _extract_smil_formats_and_subtitles(self, smil_url, video_id, fatal=True, f4
         if res is False:
             assert not fatal
             return [], {}
-
         smil, urlh = res
-        smil_url = urlh.url
-
-        namespace = self._parse_smil_namespace(smil)
 
-        fmts = self._parse_smil_formats(
-            smil, smil_url, video_id, namespace=namespace, f4m_params=f4m_params)
-        subs = self._parse_smil_subtitles(
-            smil, namespace=namespace)
-
-        return fmts, subs
+        return self._parse_smil_formats_and_subtitles(smil, urlh.url, video_id, f4m_params=f4m_params,
+                                                      namespace=self._parse_smil_namespace(smil))
 
     def _extract_smil_formats(self, *args, **kwargs):
         fmts, subs = self._extract_smil_formats_and_subtitles(*args, **kwargs)
@@ -2285,9 +2277,8 @@ def _download_smil(self, smil_url, video_id, fatal=True, transform_source=None):
     def _parse_smil(self, smil, smil_url, video_id, f4m_params=None):
         namespace = self._parse_smil_namespace(smil)
 
-        formats = self._parse_smil_formats(
+        formats, subtitles = self._parse_smil_formats_and_subtitles(
             smil, smil_url, video_id, namespace=namespace, f4m_params=f4m_params)
-        subtitles = self._parse_smil_subtitles(smil, namespace=namespace)
 
         video_id = os.path.splitext(url_basename(smil_url))[0]
         title = None
@@ -2326,7 +2317,14 @@ def _parse_smil_namespace(self, smil):
         return self._search_regex(
             r'(?i)^{([^}]+)?}smil$', smil.tag, 'namespace', default=None)
 
-    def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_params=None, transform_rtmp_url=None):
+    def _parse_smil_formats(self, *args, **kwargs):
+        fmts, subs = self._parse_smil_formats_and_subtitles(*args, **kwargs)
+        if subs:
+            self._report_ignoring_subs('SMIL')
+        return fmts
+
+    def _parse_smil_formats_and_subtitles(
+            self, smil, smil_url, video_id, namespace=None, f4m_params=None, transform_rtmp_url=None):
         base = smil_url
         for meta in smil.findall(self._xpath_ns('./head/meta', namespace)):
             b = meta.get('base') or meta.get('httpBase')
@@ -2334,7 +2332,7 @@ def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_para
                 base = b
                 break
 
-        formats = []
+        formats, subtitles = [], {}
         rtmp_count = 0
         http_count = 0
         m3u8_count = 0
@@ -2382,8 +2380,9 @@ def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_para
             src_url = src_url.strip()
 
             if proto == 'm3u8' or src_ext == 'm3u8':
-                m3u8_formats = self._extract_m3u8_formats(
+                m3u8_formats, m3u8_subs = self._extract_m3u8_formats_and_subtitles(
                     src_url, video_id, ext or 'mp4', m3u8_id='hls', fatal=False)
+                self._merge_subtitles(m3u8_subs, target=subtitles)
                 if len(m3u8_formats) == 1:
                     m3u8_count += 1
                     m3u8_formats[0].update({
@@ -2404,11 +2403,15 @@ def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_para
                 f4m_url += urllib.parse.urlencode(f4m_params)
                 formats.extend(self._extract_f4m_formats(f4m_url, video_id, f4m_id='hds', fatal=False))
             elif src_ext == 'mpd':
-                formats.extend(self._extract_mpd_formats(
-                    src_url, video_id, mpd_id='dash', fatal=False))
+                mpd_formats, mpd_subs = self._extract_mpd_formats_and_subtitles(
+                    src_url, video_id, mpd_id='dash', fatal=False)
+                formats.extend(mpd_formats)
+                self._merge_subtitles(mpd_subs, target=subtitles)
             elif re.search(r'\.ism/[Mm]anifest', src_url):
-                formats.extend(self._extract_ism_formats(
-                    src_url, video_id, ism_id='mss', fatal=False))
+                ism_formats, ism_subs = self._extract_ism_formats_and_subtitles(
+                    src_url, video_id, ism_id='mss', fatal=False)
+                formats.extend(ism_formats)
+                self._merge_subtitles(ism_subs, target=subtitles)
             elif src_url.startswith('http') and self._is_valid_url(src, video_id):
                 http_count += 1
                 formats.append({
@@ -2439,7 +2442,10 @@ def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_para
                 'format_note': 'SMIL storyboards',
             })
 
-        return formats
+        smil_subs = self._parse_smil_subtitles(smil, namespace=namespace)
+        self._merge_subtitles(smil_subs, target=subtitles)
+
+        return formats, subtitles
 
     def _parse_smil_subtitles(self, smil, namespace=None, subtitles_lang='en'):
         urls = []
diff --git a/yt_dlp/extractor/livestream.py b/yt_dlp/extractor/livestream.py
index 692d6ab3a..a05a0fa9e 100644
--- a/yt_dlp/extractor/livestream.py
+++ b/yt_dlp/extractor/livestream.py
@@ -80,7 +80,8 @@ class LivestreamIE(InfoExtractor):
     }]
     _API_URL_TEMPLATE = 'http://livestream.com/api/accounts/%s/events/%s'
 
-    def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_params=None, transform_rtmp_url=None):
+    def _parse_smil_formats_and_subtitles(
+            self, smil, smil_url, video_id, namespace=None, f4m_params=None, transform_rtmp_url=None):
         base_ele = find_xpath_attr(
             smil, self._xpath_ns('.//meta', namespace), 'name', 'httpBase')
         base = base_ele.get('content') if base_ele is not None else 'http://livestreamvod-f.akamaihd.net/'
@@ -104,7 +105,7 @@ def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_para
                 'tbr': tbr,
                 'preference': -1000,  # Strictly inferior than all other formats?
             })
-        return formats
+        return formats, {}
 
     def _extract_video_info(self, video_data):
         video_id = compat_str(video_data['id'])
diff --git a/yt_dlp/extractor/mediaset.py b/yt_dlp/extractor/mediaset.py
index 1fa529914..e3b728dca 100644
--- a/yt_dlp/extractor/mediaset.py
+++ b/yt_dlp/extractor/mediaset.py
@@ -154,10 +154,12 @@ class MediasetIE(ThePlatformBaseIE):
         }
     }]
 
-    def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_params=None, transform_rtmp_url=None):
+    def _parse_smil_formats_and_subtitles(
+            self, smil, smil_url, video_id, namespace=None, f4m_params=None, transform_rtmp_url=None):
         for video in smil.findall(self._xpath_ns('.//video', namespace)):
             video.attrib['src'] = re.sub(r'(https?://vod05)t(-mediaset-it\.akamaized\.net/.+?.mpd)\?.+', r'\1\2', video.attrib['src'])
-        return super(MediasetIE, self)._parse_smil_formats(smil, smil_url, video_id, namespace, f4m_params, transform_rtmp_url)
+        return super(MediasetIE, self)._parse_smil_formats_and_subtitles(
+            smil, smil_url, video_id, namespace, f4m_params, transform_rtmp_url)
 
     def _check_drm_formats(self, tp_formats, video_id):
         has_nondrm, drm_manifest = False, ''
diff --git a/yt_dlp/extractor/nbc.py b/yt_dlp/extractor/nbc.py
index 299b05174..b3c28ab55 100644
--- a/yt_dlp/extractor/nbc.py
+++ b/yt_dlp/extractor/nbc.py
@@ -131,7 +131,6 @@ class NBCIE(ThePlatformIE):  # XXX: Do not subclass from concrete IE
                 'age_limit': 0,
                 'thumbnail': r're:https?://.+\.jpg',
             },
-            'expected_warnings': ['Ignoring subtitle tracks'],
             'params': {
                 'skip_download': 'm3u8',
             },
diff --git a/yt_dlp/extractor/theplatform.py b/yt_dlp/extractor/theplatform.py
index 537f6f6cd..8307b912d 100644
--- a/yt_dlp/extractor/theplatform.py
+++ b/yt_dlp/extractor/theplatform.py
@@ -45,7 +45,7 @@ def _extract_theplatform_smil(self, smil_url, video_id, note='Downloading SMIL d
                     raise ExtractorError(
                         error_element.attrib['abstract'], expected=True)
 
-        smil_formats = self._parse_smil_formats(
+        smil_formats, subtitles = self._parse_smil_formats_and_subtitles(
             meta, smil_url, video_id, namespace=default_ns,
             # the parameters are from syfy.com, other sites may use others,
             # they also work for nbc.com
@@ -65,8 +65,6 @@ def _extract_theplatform_smil(self, smil_url, video_id, note='Downloading SMIL d
 
                 formats.append(_format)
 
-        subtitles = self._parse_smil_subtitles(meta, default_ns)
-
         return formats, subtitles
 
     def _download_theplatform_metadata(self, path, video_id):
diff --git a/yt_dlp/extractor/triller.py b/yt_dlp/extractor/triller.py
index c5d01c827..56e51fea8 100644
--- a/yt_dlp/extractor/triller.py
+++ b/yt_dlp/extractor/triller.py
@@ -66,13 +66,6 @@ def _get_comments(self, video_id, limit=15):
             'timestamp': ('timestamp', {unified_timestamp}),
         }))
 
-    def _check_user_info(self, user_info):
-        if user_info.get('private') and not user_info.get('followed_by_me'):
-            raise ExtractorError('This video is private', expected=True)
-        elif traverse_obj(user_info, 'blocked_by_user', 'blocking_user'):
-            raise ExtractorError('The author of the video is blocked', expected=True)
-        return user_info
-
     def _parse_video_info(self, video_info, username, user_id, display_id=None):
         video_id = str(video_info['id'])
         display_id = display_id or video_info.get('video_uuid')
@@ -231,8 +224,6 @@ def _real_extract(self, url):
             f'{self._API_BASE_URL}/api/videos/{display_id}', display_id,
             headers=self._API_HEADERS)['videos'][0]
 
-        self._check_user_info(video_info.get('user') or {})
-
         return self._parse_video_info(video_info, username, None, display_id)
 
 
@@ -287,9 +278,14 @@ def _entries(self, username, user_id, limit=6):
     def _real_extract(self, url):
         username = self._match_id(url)
 
-        user_info = self._check_user_info(self._download_json(
+        user_info = traverse_obj(self._download_json(
             f'{self._API_BASE_URL}/api/users/by_username/{username}',
-            username, note='Downloading user info', headers=self._API_HEADERS)['user'])
+            username, note='Downloading user info', headers=self._API_HEADERS), ('user', {dict})) or {}
+
+        if user_info.get('private') and user_info.get('followed_by_me') not in (True, 'true'):
+            raise ExtractorError('This user profile is private', expected=True)
+        elif traverse_obj(user_info, (('blocked_by_user', 'blocking_user'), {bool}), get_all=False):
+            raise ExtractorError('The author of the video is blocked', expected=True)
 
         user_id = str_or_none(user_info.get('user_id'))
         if not user_id:
diff --git a/yt_dlp/networking/_urllib.py b/yt_dlp/networking/_urllib.py
index ff3a22c8c..3fe5fa52e 100644
--- a/yt_dlp/networking/_urllib.py
+++ b/yt_dlp/networking/_urllib.py
@@ -385,6 +385,11 @@ def __init__(self, *, enable_file_urls: bool = False, **kwargs):
         if self.enable_file_urls:
             self._SUPPORTED_URL_SCHEMES = (*self._SUPPORTED_URL_SCHEMES, 'file')
 
+    def _check_extensions(self, extensions):
+        super()._check_extensions(extensions)
+        extensions.pop('cookiejar', None)
+        extensions.pop('timeout', None)
+
     def _create_instance(self, proxies, cookiejar):
         opener = urllib.request.OpenerDirector()
         handlers = [
diff --git a/yt_dlp/networking/common.py b/yt_dlp/networking/common.py
index 7f7457978..ab26a0628 100644
--- a/yt_dlp/networking/common.py
+++ b/yt_dlp/networking/common.py
@@ -21,6 +21,7 @@
     TransportError,
     UnsupportedRequest,
 )
+from ..compat.types import NoneType
 from ..utils import (
     bug_reports_message,
     classproperty,
@@ -147,6 +148,7 @@ class RequestHandler(abc.ABC):
         a proxy url with an url scheme not in this list will raise an UnsupportedRequest.
 
     - `_SUPPORTED_FEATURES`: a tuple of supported features, as defined in Features enum.
+
     The above may be set to None to disable the checks.
 
     Parameters:
@@ -169,9 +171,14 @@ class RequestHandler(abc.ABC):
     Requests may have additional optional parameters defined as extensions.
      RequestHandler subclasses may choose to support custom extensions.
 
+    If an extension is supported, subclasses should extend _check_extensions(extensions)
+    to pop and validate the extension.
+    - Extensions left in `extensions` are treated as unsupported and UnsupportedRequest will be raised.
+
     The following extensions are defined for RequestHandler:
-    - `cookiejar`: Cookiejar to use for this request
-    - `timeout`: socket timeout to use for this request
+    - `cookiejar`: Cookiejar to use for this request.
+    - `timeout`: socket timeout to use for this request.
+    To enable these, add extensions.pop('<extension>', None) to _check_extensions
 
     Apart from the url protocol, proxies dict may contain the following keys:
     - `all`: proxy to use for all protocols. Used as a fallback if no proxy is set for a specific protocol.
@@ -263,26 +270,19 @@ def _check_proxies(self, proxies):
             if scheme not in self._SUPPORTED_PROXY_SCHEMES:
                 raise UnsupportedRequest(f'Unsupported proxy type: "{scheme}"')
 
-    def _check_cookiejar_extension(self, extensions):
-        if not extensions.get('cookiejar'):
-            return
-        if not isinstance(extensions['cookiejar'], CookieJar):
-            raise UnsupportedRequest('cookiejar is not a CookieJar')
-
-    def _check_timeout_extension(self, extensions):
-        if extensions.get('timeout') is None:
-            return
-        if not isinstance(extensions['timeout'], (float, int)):
-            raise UnsupportedRequest('timeout is not a float or int')
-
     def _check_extensions(self, extensions):
-        self._check_cookiejar_extension(extensions)
-        self._check_timeout_extension(extensions)
+        """Check extensions for unsupported extensions. Subclasses should extend this."""
+        assert isinstance(extensions.get('cookiejar'), (CookieJar, NoneType))
+        assert isinstance(extensions.get('timeout'), (float, int, NoneType))
 
     def _validate(self, request):
         self._check_url_scheme(request)
         self._check_proxies(request.proxies or self.proxies)
-        self._check_extensions(request.extensions)
+        extensions = request.extensions.copy()
+        self._check_extensions(extensions)
+        if extensions:
+            # TODO: add support for optional extensions
+            raise UnsupportedRequest(f'Unsupported extensions: {", ".join(extensions.keys())}')
 
     @wrap_request_errors
     def validate(self, request: Request):


Test patch:
diff --git a/test/test_networking.py b/test/test_networking.py
index d4eba2a5d..1bd6afc88 100644
--- a/test/test_networking.py
+++ b/test/test_networking.py
@@ -804,10 +804,10 @@ def test_httplib_validation_errors(self, handler):
             assert not isinstance(exc_info.value, TransportError)
 
 
-def run_validation(handler, fail, req, **handler_kwargs):
+def run_validation(handler, error, req, **handler_kwargs):
     with handler(**handler_kwargs) as rh:
-        if fail:
-            with pytest.raises(UnsupportedRequest):
+        if error:
+            with pytest.raises(error):
                 rh.validate(req)
         else:
             rh.validate(req)
@@ -824,6 +824,9 @@ class NoCheckRH(ValidationRH):
         _SUPPORTED_PROXY_SCHEMES = None
         _SUPPORTED_URL_SCHEMES = None
 
+        def _check_extensions(self, extensions):
+            extensions.clear()
+
     class HTTPSupportedRH(ValidationRH):
         _SUPPORTED_URL_SCHEMES = ('http',)
 
@@ -834,26 +837,26 @@ class HTTPSupportedRH(ValidationRH):
             ('https', False, {}),
             ('data', False, {}),
             ('ftp', False, {}),
-            ('file', True, {}),
+            ('file', UnsupportedRequest, {}),
             ('file', False, {'enable_file_urls': True}),
         ]),
         (NoCheckRH, [('http', False, {})]),
-        (ValidationRH, [('http', True, {})])
+        (ValidationRH, [('http', UnsupportedRequest, {})])
     ]
 
     PROXY_SCHEME_TESTS = [
         # scheme, expected to fail
         ('Urllib', [
             ('http', False),
-            ('https', True),
+            ('https', UnsupportedRequest),
             ('socks4', False),
             ('socks4a', False),
             ('socks5', False),
             ('socks5h', False),
-            ('socks', True),
+            ('socks', UnsupportedRequest),
         ]),
         (NoCheckRH, [('http', False)]),
-        (HTTPSupportedRH, [('http', True)]),
+        (HTTPSupportedRH, [('http', UnsupportedRequest)]),
     ]
 
     PROXY_KEY_TESTS = [
@@ -863,8 +866,22 @@ class HTTPSupportedRH(ValidationRH):
             ('unrelated', False),
         ]),
         (NoCheckRH, [('all', False)]),
-        (HTTPSupportedRH, [('all', True)]),
-        (HTTPSupportedRH, [('no', True)]),
+        (HTTPSupportedRH, [('all', UnsupportedRequest)]),
+        (HTTPSupportedRH, [('no', UnsupportedRequest)]),
+    ]
+
+    EXTENSION_TESTS = [
+        ('Urllib', [
+            ({'cookiejar': 'notacookiejar'}, AssertionError),
+            ({'cookiejar': CookieJar()}, False),
+            ({'timeout': 1}, False),
+            ({'timeout': 'notatimeout'}, AssertionError),
+            ({'unsupported': 'value'}, UnsupportedRequest),
+        ]),
+        (NoCheckRH, [
+            ({'cookiejar': 'notacookiejar'}, False),
+            ({'somerandom': 'test'}, False),  # but any extension is allowed through
+        ]),
     ]
 
     @pytest.mark.parametrize('handler,scheme,fail,handler_kwargs', [
@@ -907,15 +924,16 @@ def test_empty_proxy(self, handler):
     @pytest.mark.parametrize('proxy_url', ['//example.com', 'example.com', '127.0.0.1'])
     @pytest.mark.parametrize('handler', ['Urllib'], indirect=True)
     def test_missing_proxy_scheme(self, handler, proxy_url):
-        run_validation(handler, True, Request('http://', proxies={'http': 'example.com'}))
-
-    @pytest.mark.parametrize('handler', ['Urllib'], indirect=True)
-    def test_cookiejar_extension(self, handler):
-        run_validation(handler, True, Request('http://', extensions={'cookiejar': 'notacookiejar'}))
+        run_validation(handler, UnsupportedRequest, Request('http://', proxies={'http': 'example.com'}))
 
-    @pytest.mark.parametrize('handler', ['Urllib'], indirect=True)
-    def test_timeout_extension(self, handler):
-        run_validation(handler, True, Request('http://', extensions={'timeout': 'notavalidtimeout'}))
+    @pytest.mark.parametrize('handler,extensions,fail', [
+        (handler_tests[0], extensions, fail)
+        for handler_tests in EXTENSION_TESTS
+        for extensions, fail in handler_tests[1]
+    ], indirect=['handler'])
+    def test_extension(self, handler, extensions, fail):
+        run_validation(
+            handler, fail, Request('http://', extensions=extensions))
 
     def test_invalid_request_type(self):
         rh = self.ValidationRH(logger=FakeLogger())