Repository: yt-dlp/yt-dlp
Pull request #5213 (2022-10-12 08:18:47)

[extr/generic:quoted-html] Extract embeds within commonly escaped HTML
**IMPORTANT**: PRs without the template will be CLOSED

### Description of your *pull request* and other information

</details>

<!--

Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible

-->

ADD DESCRIPTION HERE

It seems that on some sites the iframe etc. embeds may be within escaped html -  such as in an attribute or json payload. https://github.com/yt-dlp/yt-dlp/pull/5176 added support so we can write extractors that extract, unquote and run the embed detection for such cases.

This extractor aims to add support for common cases. One I have found is the `data-html` attribute, which is both used by squarespace sites and 24tv.ua. I am not sure if this is related to any kind of web framework or is just a convention.

If anyone knows of any other common cases, we should consider adding them too.

<details open><summary>Template</summary> <!-- OPEN is intentional -->

<!--

# PLEASE FOLLOW THE GUIDE BELOW

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [X] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [X] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [X] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [X] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [X] Fix or improvement to an extractor (Make sure to add/update tests)
- [X] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [ ] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))



Patch:
diff --git a/README.md b/README.md
index 9b59e096a..7374e0e94 100644
--- a/README.md
+++ b/README.md
@@ -1724,7 +1724,7 @@ #### youtube
 
 #### youtubetab (YouTube playlists, channels, feeds, etc.)
 * `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details)
-* `approximate_date`: Extract approximate `upload_date` in flat-playlist. This may cause date-based filters to be slightly off
+* `approximate_date`: Extract approximate `upload_date` and `timestamp` in flat-playlist. This may cause date-based filters to be slightly off
 
 #### funimation
 * `language`: Audio languages to extract, e.g. `funimation:language=english,japanese`
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index e1c24b892..39df79a3f 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -1621,6 +1621,7 @@ def process_ie_result(self, ie_result, download=True, extra_info=None):
                 self.add_default_extra_info(info_copy, ie, ie_result['url'])
                 self.add_extra_info(info_copy, extra_info)
                 info_copy, _ = self.pre_process(info_copy)
+                self._fill_common_fields(info_copy, False)
                 self.__forced_printings(info_copy, self.prepare_filename(info_copy), incomplete=True)
                 self._raise_pending_errors(info_copy)
                 if self.params.get('force_write_download_archive', False):
@@ -2379,10 +2380,9 @@ def check_thumbnails(thumbnails):
         else:
             info_dict['thumbnails'] = thumbnails
 
-    def _fill_common_fields(self, info_dict, is_video=True):
+    def _fill_common_fields(self, info_dict, final=True):
         # TODO: move sanitization here
-        if is_video:
-            # playlists are allowed to lack "title"
+        if final:
             title = info_dict.get('title', NO_DEFAULT)
             if title is NO_DEFAULT:
                 raise ExtractorError('Missing "title" field in extractor result',
@@ -2432,7 +2432,7 @@ def _fill_common_fields(self, info_dict, is_video=True):
         # Auto generate title fields corresponding to the *_number fields when missing
         # in order to always have clean titles. This is very common for TV series.
         for field in ('chapter', 'season', 'episode'):
-            if info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
+            if final and info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
                 info_dict[field] = '%s %d' % (field.capitalize(), info_dict['%s_number' % field])
 
     def _raise_pending_errors(self, info):
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
index d514f9a89..8652ec54e 100644
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -65,6 +65,7 @@
     HistoryPlayerIE,
     BiographyIE,
 )
+from .aeonco import AeonCoIE
 from .afreecatv import (
     AfreecaTVIE,
     AfreecaTVLiveIE,
@@ -697,7 +698,10 @@
     HSEShowIE,
     HSEProductIE,
 )
-from .genericembeds import HTML5MediaEmbedIE
+from .genericembeds import (
+    HTML5MediaEmbedIE,
+    QuotedHTMLIE,
+)
 from .huajiao import HuajiaoIE
 from .huya import HuyaLiveIE
 from .huffpost import HuffPostIE
@@ -1883,7 +1887,6 @@
 )
 from .tv24ua import (
     TV24UAVideoIE,
-    TV24UAGenericPassthroughIE
 )
 from .tv2dk import (
     TV2DKIE,
diff --git a/yt_dlp/extractor/aeonco.py b/yt_dlp/extractor/aeonco.py
new file mode 100644
index 000000000..4655862e3
--- /dev/null
+++ b/yt_dlp/extractor/aeonco.py
@@ -0,0 +1,40 @@
+from .common import InfoExtractor
+from .vimeo import VimeoIE
+
+
+class AeonCoIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?aeon\.co/videos/(?P<id>[^/?]+)'
+    _TESTS = [{
+        'url': 'https://aeon.co/videos/raw-solar-storm-footage-is-the-punk-rock-antidote-to-sleek-james-webb-imagery',
+        'md5': 'e5884d80552c9b6ea8d268a258753362',
+        'info_dict': {
+            'id': '1284717',
+            'ext': 'mp4',
+            'title': 'Brilliant Noise',
+            'thumbnail': 'https://i.vimeocdn.com/video/21006315-1a1e49da8b07fd908384a982b4ba9ff0268c509a474576ebdf7b1392f4acae3b-d_960',
+            'uploader': 'Semiconductor',
+            'uploader_id': 'semiconductor',
+            'uploader_url': 'https://vimeo.com/semiconductor',
+            'duration': 348
+        }
+    }, {
+        'url': 'https://aeon.co/videos/dazzling-timelapse-shows-how-microbes-spoil-our-food-and-sometimes-enrich-it',
+        'md5': '4e5f3dad9dbda0dbfa2da41a851e631e',
+        'info_dict': {
+            'id': '728595228',
+            'ext': 'mp4',
+            'title': 'Wrought',
+            'thumbnail': 'https://i.vimeocdn.com/video/1484618528-c91452611f9a4e4497735a533da60d45b2fe472deb0c880f0afaab0cd2efb22a-d_1280',
+            'uploader': 'Biofilm Productions',
+            'uploader_id': 'user140352216',
+            'uploader_url': 'https://vimeo.com/user140352216',
+            'duration': 1344
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        vimeo_id = self._search_regex(r'hosterId":\s*"(?P<id>[0-9]+)', webpage, 'vimeo id')
+        vimeo_url = VimeoIE._smuggle_referrer(f'https://player.vimeo.com/video/{vimeo_id}', 'https://aeon.co')
+        return self.url_result(vimeo_url, VimeoIE)
diff --git a/yt_dlp/extractor/cda.py b/yt_dlp/extractor/cda.py
index 6d01c60d5..2a12b054b 100644
--- a/yt_dlp/extractor/cda.py
+++ b/yt_dlp/extractor/cda.py
@@ -1,4 +1,8 @@
+import base64
 import codecs
+import datetime
+import hashlib
+import hmac
 import json
 import re
 
@@ -12,6 +16,8 @@
     multipart_encode,
     parse_duration,
     random_birthday,
+    traverse_obj,
+    try_call,
     try_get,
     urljoin,
 )
@@ -19,7 +25,18 @@
 
 class CDAIE(InfoExtractor):
     _VALID_URL = r'https?://(?:(?:www\.)?cda\.pl/video|ebd\.cda\.pl/[0-9]+x[0-9]+)/(?P<id>[0-9a-z]+)'
+    _NETRC_MACHINE = 'cdapl'
+
     _BASE_URL = 'http://www.cda.pl/'
+    _BASE_API_URL = 'https://api.cda.pl'
+    _API_HEADERS = {
+        'Accept': 'application/vnd.cda.public+json',
+        'User-Agent': 'pl.cda 1.0 (version 1.2.88 build 15306; Android 9; Xiaomi Redmi 3S)',
+    }
+    # hardcoded in the app
+    _LOGIN_REQUEST_AUTH = 'Basic YzU3YzBlZDUtYTIzOC00MWQwLWI2NjQtNmZmMWMxY2Y2YzVlOklBTm95QlhRRVR6U09MV1hnV3MwMW0xT2VyNWJNZzV4clRNTXhpNGZJUGVGZ0lWUlo5UGVYTDhtUGZaR1U1U3Q'
+    _BEARER_CACHE = 'cda-bearer'
+
     _TESTS = [{
         'url': 'http://www.cda.pl/video/5749950c',
         'md5': '6f844bf51b15f31fae165365707ae970',
@@ -83,8 +100,73 @@ def _download_age_confirm_page(self, url, video_id, *args, **kwargs):
                 'Content-Type': content_type,
             }, **kwargs)
 
+    def _perform_login(self, username, password):
+        cached_bearer = self.cache.load(self._BEARER_CACHE, username) or {}
+        if cached_bearer.get('valid_until', 0) > datetime.datetime.now().timestamp() + 5:
+            self._API_HEADERS['Authorization'] = f'Bearer {cached_bearer["token"]}'
+            return
+
+        password_hash = base64.urlsafe_b64encode(hmac.new(
+            b's01m1Oer5IANoyBXQETzSOLWXgWs01m1Oer5bMg5xrTMMxRZ9Pi4fIPeFgIVRZ9PeXL8mPfXQETZGUAN5StRZ9P',
+            ''.join(f'{bytes((bt & 255, )).hex():0>2}'
+                    for bt in hashlib.md5(password.encode()).digest()).encode(),
+            hashlib.sha256).digest()).decode().replace('=', '')
+
+        token_res = self._download_json(
+            f'{self._BASE_API_URL}/oauth/token', None, 'Logging in', data=b'',
+            headers={**self._API_HEADERS, 'Authorization': self._LOGIN_REQUEST_AUTH},
+            query={
+                'grant_type': 'password',
+                'login': username,
+                'password': password_hash,
+            })
+        self.cache.store(self._BEARER_CACHE, username, {
+            'token': token_res['access_token'],
+            'valid_until': token_res['expires_in'] + datetime.datetime.now().timestamp(),
+        })
+        self._API_HEADERS['Authorization'] = f'Bearer {token_res["access_token"]}'
+
     def _real_extract(self, url):
         video_id = self._match_id(url)
+
+        if 'Authorization' in self._API_HEADERS:
+            return self._api_extract(video_id)
+        else:
+            return self._web_extract(video_id, url)
+
+    def _api_extract(self, video_id):
+        meta = self._download_json(
+            f'{self._BASE_API_URL}/video/{video_id}', video_id, headers=self._API_HEADERS)['video']
+
+        if meta.get('premium') and not meta.get('premium_free'):
+            self.report_drm(video_id)
+
+        uploader = traverse_obj(meta, 'author', 'login')
+
+        formats = [{
+            'url': quality['file'],
+            'format': quality.get('title'),
+            'resolution': quality.get('name'),
+            'height': try_call(lambda: int(quality['name'][:-1])),
+            'filesize': quality.get('length'),
+        } for quality in meta['qualities'] if quality.get('file')]
+
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': meta.get('title'),
+            'description': meta.get('description'),
+            'uploader': None if uploader == 'anonim' else uploader,
+            'average_rating': float_or_none(meta.get('rating')),
+            'thumbnail': meta.get('thumb'),
+            'formats': formats,
+            'duration': meta.get('duration'),
+            'age_limit': 18 if meta.get('for_adults') else 0,
+            'view_count': meta.get('views'),
+        }
+
+    def _web_extract(self, video_id, url):
         self._set_cookie('cda.pl', 'cda.player', 'html5')
         webpage = self._download_webpage(
             self._BASE_URL + '/video/' + video_id, video_id)
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index 10d44d95a..ab8def57d 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -3843,8 +3843,8 @@ def _configuration_arg(self, key, default=NO_DEFAULT, *, ie_key=None, casesense=
         @param default      The default value to return when the key is not present (default: [])
         @param casesense    When false, the values are converted to lower case
         '''
-        val = traverse_obj(
-            self._downloader.params, ('extractor_args', (ie_key or self.ie_key()).lower(), key))
+        ie_key = ie_key if isinstance(ie_key, str) else (ie_key or self).ie_key()
+        val = traverse_obj(self._downloader.params, ('extractor_args', ie_key.lower(), key))
         if val is None:
             return [] if default is NO_DEFAULT else default
         return list(val) if casesense else [x.lower() for x in val]
diff --git a/yt_dlp/extractor/generic.py b/yt_dlp/extractor/generic.py
index ad4e3c5b8..b7a5ffb5b 100644
--- a/yt_dlp/extractor/generic.py
+++ b/yt_dlp/extractor/generic.py
@@ -1980,22 +1980,6 @@ class GenericIE(InfoExtractor):
             },
             'playlist_count': 6,
         },
-        {
-            # Squarespace video embed, 2019-08-28
-            'url': 'http://ootboxford.com',
-            'info_dict': {
-                'id': 'Tc7b_JGdZfw',
-                'title': 'Out of the Blue, at Childish Things 10',
-                'ext': 'mp4',
-                'description': 'md5:a83d0026666cf5ee970f8bd1cfd69c7f',
-                'uploader_id': 'helendouglashouse',
-                'uploader': 'Helen & Douglas House',
-                'upload_date': '20140328',
-            },
-            'params': {
-                'skip_download': True,
-            },
-        },
         # {
         #     # Zype embed
         #     'url': 'https://www.cookscountry.com/episode/554-smoky-barbecue-favorites',
@@ -2784,12 +2768,6 @@ def _extract_embeds(self, url, webpage, *, urlh=None, info_dict={}):
         # There probably should be a second run of generic extractor on unescaped webpage.
         # webpage = urllib.parse.unquote(webpage)
 
-        # Unescape squarespace embeds to be detected by generic extractor,
-        # see https://github.com/ytdl-org/youtube-dl/issues/21294
-        webpage = re.sub(
-            r'<div[^>]+class=[^>]*?\bsqs-video-wrapper\b[^>]*>',
-            lambda x: unescapeHTML(x.group(0)), webpage)
-
         # TODO: Move to respective extractors
         bc_urls = BrightcoveLegacyIE._extract_brightcove_urls(webpage)
         if bc_urls:
diff --git a/yt_dlp/extractor/genericembeds.py b/yt_dlp/extractor/genericembeds.py
index 64bd20e3a..1bffe275a 100644
--- a/yt_dlp/extractor/genericembeds.py
+++ b/yt_dlp/extractor/genericembeds.py
@@ -1,5 +1,8 @@
+import re
+import urllib.parse
+
 from .common import InfoExtractor
-from ..utils import make_archive_id
+from ..utils import make_archive_id, unescapeHTML
 
 
 class HTML5MediaEmbedIE(InfoExtractor):
@@ -29,3 +32,84 @@ def _extract_from_webpage(self, url, webpage):
             })
             self._sort_formats(entry['formats'])
             yield entry
+
+
+class QuotedHTMLIE(InfoExtractor):
+    """For common cases of quoted/escaped html parts in the webpage"""
+    _VALID_URL = False
+    IE_NAME = 'generic:quoted-html'
+    IE_DESC = False  # Do not list
+    _WEBPAGE_TESTS = [{
+        # 2 YouTube embeds in data-html
+        'url': 'https://24tv.ua/bronetransporteri-ozbroyenni-zsu-shho-vidomo-pro-bronovik-wolfhound_n2167966',
+        'info_dict': {
+            'id': 'bronetransporteri-ozbroyenni-zsu-shho-vidomo-pro-bronovik-wolfhound_n2167966',
+            'title': 'Броньовик Wolfhound: гігант, який допомагає ЗСУ знищувати окупантів на фронті',
+            'thumbnail': r're:^https?://.*\.jpe?g',
+            'timestamp': float,
+            'upload_date': str,
+            'description': 'md5:6816e1e5a65304bd7898e4c7eb1b26f7',
+            'age_limit': 0,
+        },
+        'playlist_count': 2
+    }, {
+        # Generic iframe embed of TV24UAPlayerIE within data-html
+        'url': 'https://24tv.ua/harkivyani-zgaduyut-misto-do-viyni-shhemlive-video_n1887584',
+        'info_dict': {
+            'id': '1887584',
+            'ext': 'mp4',
+            'title': 'Харків\'яни згадують місто до війни: щемливе відео',
+            'thumbnail': r're:^https?://.*\.jpe?g',
+        },
+        'params': {'skip_download': True}
+    }, {
+        # YouTube embeds on Squarespace (data-html): https://github.com/ytdl-org/youtube-dl/issues/21294
+        'url': 'https://www.harvardballetcompany.org/past-productions',
+        'info_dict': {
+            'id': 'past-productions',
+            'title': 'Productions — Harvard Ballet Company',
+            'age_limit': 0,
+            'description': 'Past Productions',
+        },
+        'playlist_mincount': 26
+    }, {
+        # Squarespace video embed, 2019-08-28, data-html
+        'url': 'http://ootboxford.com',
+        'info_dict': {
+            'id': 'Tc7b_JGdZfw',
+            'title': 'Out of the Blue, at Childish Things 10',
+            'ext': 'mp4',
+            'description': 'md5:a83d0026666cf5ee970f8bd1cfd69c7f',
+            'uploader_id': 'helendouglashouse',
+            'uploader': 'Helen & Douglas House',
+            'upload_date': '20140328',
+            'availability': 'public',
+            'view_count': int,
+            'channel': 'Helen & Douglas House',
+            'comment_count': int,
+            'uploader_url': 'http://www.youtube.com/user/helendouglashouse',
+            'duration': 253,
+            'channel_url': 'https://www.youtube.com/channel/UCTChGezrZVmlYlpMlkmulPA',
+            'playable_in_embed': True,
+            'age_limit': 0,
+            'channel_follower_count': int,
+            'channel_id': 'UCTChGezrZVmlYlpMlkmulPA',
+            'tags': 'count:6',
+            'categories': ['Nonprofits & Activism'],
+            'like_count': int,
+            'thumbnail': 'https://i.ytimg.com/vi/Tc7b_JGdZfw/hqdefault.jpg',
+        },
+        'params': {
+            'skip_download': True,
+        },
+    }]
+
+    def _extract_from_webpage(self, url, webpage):
+        combined = ''
+        for _, html in re.findall(r'(?s)\bdata-html=(["\'])((?:(?!\1).)+)\1', webpage):
+            # unescapeHTML can handle &quot; etc., unquote can handle percent encoding
+            unquoted_html = unescapeHTML(urllib.parse.unquote(html))
+            if unquoted_html != html:
+                combined += unquoted_html
+        if combined:
+            yield from self._extract_generic_embeds(url, combined)
diff --git a/yt_dlp/extractor/iqiyi.py b/yt_dlp/extractor/iqiyi.py
index 6a43846c1..bb77647f8 100644
--- a/yt_dlp/extractor/iqiyi.py
+++ b/yt_dlp/extractor/iqiyi.py
@@ -588,8 +588,9 @@ def _real_extract(self, url):
             ut_list = ['0']
 
         # bid 0 as an initial format checker
-        dash_paths = self._parse_json(PhantomJSwrapper(self).get(
-            url, html='<!DOCTYPE html>', video_id=video_id, note2='Executing signature code', jscode=self._DASH_JS % {
+        dash_paths = self._parse_json(PhantomJSwrapper(self, timeout=120_000).get(
+            url, note2='Executing signature code (this may take a couple minutes)',
+            html='<!DOCTYPE html>', video_id=video_id, jscode=self._DASH_JS % {
                 'tvid': video_info['tvId'],
                 'vid': video_info['vid'],
                 'src': traverse_obj(next_props, ('initialProps', 'pageProps', 'ptid'),
diff --git a/yt_dlp/extractor/tv24ua.py b/yt_dlp/extractor/tv24ua.py
index 723049e78..553a70b6b 100644
--- a/yt_dlp/extractor/tv24ua.py
+++ b/yt_dlp/extractor/tv24ua.py
@@ -1,15 +1,10 @@
-import base64
 import re
-import urllib.parse
 
 from .common import InfoExtractor
 from ..utils import (
     determine_ext,
-    extract_attributes,
-    get_elements_html_by_class,
     js_to_json,
     mimetype2ext,
-    smuggle_url,
     traverse_obj,
 )
 
@@ -87,60 +82,3 @@ def _real_extract(self, url):
             'title': self._html_extract_title(webpage) or self._og_search_title(webpage),
             'description': self._og_search_description(webpage, default=None),
         }
-
-
-class TV24UAGenericPassthroughIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:[a-zA-Z0-9]+?\.)?24tv\.ua/(?P<id>[^/]+?_n\d+)'
-
-    _TESTS = [{
-        # Generic iframe, not within media_embed
-        'url': 'https://24tv.ua/vipalyuyut-nashi-mista-sela-dsns-pokazali-motoroshni-naslidki_n1883966',
-        'info_dict': {
-            'id': '1883966',
-            'ext': 'mp4',
-            'title': 'Випалюють наші міста та села, – моторошні наслідки обстрілів на Чернігівщині',
-            'thumbnail': r're:^https?://.*\.jpe?g',
-        }
-    }, {
-        # Generic iframe embed of TV24UAPlayerIE, within media_embed
-        'url': 'https://24tv.ua/harkivyani-zgaduyut-misto-do-viyni-shhemlive-video_n1887584',
-        'info_dict': {
-            'id': 'harkivyani-zgaduyut-misto-do-viyni-shhemlive-video_n1887584',
-            'title': 'Харків\'яни згадують місто до війни: щемливе відео'
-        },
-        'playlist': [{
-            'info_dict': {
-                'id': '1887584',
-                'ext': 'mp4',
-                'title': 'Харків\'яни згадують місто до війни: щемливе відео',
-                'thumbnail': r're:^https?://.*\.jpe?g',
-            },
-        }]
-    }, {
-        # 2 media_embeds with YouTube iframes
-        'url': 'https://24tv.ua/bronetransporteri-ozbroyenni-zsu-shho-vidomo-pro-bronovik-wolfhound_n2167966',
-        'info_dict': {
-            'id': 'bronetransporteri-ozbroyenni-zsu-shho-vidomo-pro-bronovik-wolfhound_n2167966',
-            'title': 'Броньовик Wolfhound: гігант, який допомагає ЗСУ знищувати окупантів на фронті',
-        },
-        'playlist_count': 2
-    }, {
-        'url': 'https://men.24tv.ua/fitnes-bloger-sprobuvav-vikonati-trenuvannya-naysilnishoyi-lyudini_n2164538',
-        'only_matching': True,
-    }]
-
-    def _real_extract(self, url):
-        display_id = self._match_id(url)
-        webpage = self._download_webpage(url, display_id)
-        data_urls = []
-        # The site contains escaped iframe embeds within an attribute.
-        # Once escaped, generic can handle them, so we use a data url to pass the escaped html back.
-        for html in get_elements_html_by_class('media_embed', webpage):
-            data = urllib.parse.unquote(extract_attributes(html).get('data-html'))
-            data_urls.append(f'data:text/html;base64,{base64.b64encode(data.encode("utf-8")).decode("utf-8")}')
-
-        if not data_urls:
-            return self.url_result(url, 'Generic')
-        return self.playlist_from_matches(
-            [smuggle_url(url, {'to_generic': True}) for url in data_urls], display_id, ie='Generic',
-            playlist_title=self._og_search_title(webpage) or self._html_extract_title(webpage))
diff --git a/yt_dlp/extractor/youtube.py b/yt_dlp/extractor/youtube.py
index 35e41753a..857c9670c 100644
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -948,9 +948,9 @@ def _extract_video(self, renderer):
             'uploader': uploader,
             'channel_id': channel_id,
             'thumbnails': thumbnails,
-            'upload_date': (strftime_or_none(self._parse_time_text(time_text), '%Y%m%d')
-                            if self._configuration_arg('approximate_date', ie_key='youtubetab')
-                            else None),
+            'timestamp': (self._parse_time_text(time_text)
+                          if self._configuration_arg('approximate_date', ie_key=YoutubeTabIE)
+                          else None),
             'release_timestamp': scheduled_timestamp,
             'availability':
                 'public' if self._has_badge(badges, BadgeType.AVAILABILITY_PUBLIC)
@@ -3684,17 +3684,13 @@ def _list_formats(self, video_id, microformats, video_details, player_responses,
             is_live = get_first(live_broadcast_details, 'isLiveNow')
         live_content = get_first(video_details, 'isLiveContent')
         is_upcoming = get_first(video_details, 'isUpcoming')
-        if is_live is None and is_upcoming or live_content is False:
-            is_live = False
-        if is_upcoming is None and (live_content or is_live):
-            is_upcoming = False
         post_live = get_first(video_details, 'isPostLiveDvr')
         live_status = ('post_live' if post_live
                        else 'is_live' if is_live
                        else 'is_upcoming' if is_upcoming
-                       else None if None in (is_live, is_upcoming, live_content)
-                       else 'was_live' if live_content else 'not_live')
-
+                       else 'was_live' if live_content
+                       else 'not_live' if False in (is_live, live_content)
+                       else None)
         streaming_data = traverse_obj(player_responses, (..., 'streamingData'), default=[])
         *formats, subtitles = self._extract_formats_and_subtitles(streaming_data, video_id, player_url, live_status, duration)
 
@@ -6105,9 +6101,9 @@ def _extract_notification_renderer(self, notification):
         title = self._search_regex(
             rf'{re.escape(channel or "")}[^:]+: (.+)', notification_title,
             'video title', default=None)
-        upload_date = (strftime_or_none(self._parse_time_text(self._get_text(notification, 'sentTimeText')), '%Y%m%d')
-                       if self._configuration_arg('approximate_date', ie_key=YoutubeTabIE.ie_key())
-                       else None)
+        timestamp = (self._parse_time_text(self._get_text(notification, 'sentTimeText'))
+                     if self._configuration_arg('approximate_date', ie_key=YoutubeTabIE)
+                     else None)
         return {
             '_type': 'url',
             'url': url,
@@ -6117,7 +6113,7 @@ def _extract_notification_renderer(self, notification):
             'channel_id': channel_id,
             'channel': channel,
             'thumbnails': self._extract_thumbnails(notification, 'videoThumbnail'),
-            'upload_date': upload_date,
+            'timestamp': timestamp,
         }
 
     def _notification_menu_entries(self, ytcfg):
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index 6cfbcdb8d..adb7c0e8c 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -3275,6 +3275,8 @@ def strip_jsonp(code):
 
 def js_to_json(code, vars={}, *, strict=False):
     # vars is a dict of var, val pairs to substitute
+    STRING_QUOTES = '\'"'
+    STRING_RE = '|'.join(rf'{q}(?:\\.|[^\\{q}])*{q}' for q in STRING_QUOTES)
     COMMENT_RE = r'/\*(?:(?!\*/).)*?\*/|//[^\n]*\n'
     SKIP_RE = fr'\s*(?:{COMMENT_RE})?\s*'
     INTEGER_TABLE = (
@@ -3282,6 +3284,15 @@ def js_to_json(code, vars={}, *, strict=False):
         (fr'(?s)^(0+[0-7]+){SKIP_RE}:?$', 8),
     )
 
+    def process_escape(match):
+        JSON_PASSTHROUGH_ESCAPES = R'"\bfnrtu'
+        escape = match.group(1) or match.group(2)
+
+        return (Rf'\{escape}' if escape in JSON_PASSTHROUGH_ESCAPES
+                else R'\u00' if escape == 'x'
+                else '' if escape == '\n'
+                else escape)
+
     def fix_kv(m):
         v = m.group(0)
         if v in ('true', 'false', 'null'):
@@ -3289,28 +3300,25 @@ def fix_kv(m):
         elif v in ('undefined', 'void 0'):
             return 'null'
         elif v.startswith('/*') or v.startswith('//') or v.startswith('!') or v == ',':
-            return ""
-
-        if v[0] in ("'", '"'):
-            v = re.sub(r'(?s)\\.|"', lambda m: {
-                '"': '\\"',
-                "\\'": "'",
-                '\\\n': '',
-                '\\x': '\\u00',
-            }.get(m.group(0), m.group(0)), v[1:-1])
-        else:
-            for regex, base in INTEGER_TABLE:
-                im = re.match(regex, v)
-                if im:
-                    i = int(im.group(1), base)
-                    return '"%d":' % i if v.endswith(':') else '%d' % i
+            return ''
+
+        if v[0] in STRING_QUOTES:
+            escaped = re.sub(r'(?s)(")|\\(.)', process_escape, v[1:-1])
+            return f'"{escaped}"'
+
+        for regex, base in INTEGER_TABLE:
+            im = re.match(regex, v)
+            if im:
+                i = int(im.group(1), base)
+                return f'"{i}":' if v.endswith(':') else str(i)
+
+        if v in vars:
+            return json.dumps(vars[v])
 
-            if v in vars:
-                return json.dumps(vars[v])
-            if strict:
-                raise ValueError(f'Unknown value: {v}')
+        if not strict:
+            return f'"{v}"'
 
-        return '"%s"' % v
+        raise ValueError(f'Unknown value: {v}')
 
     def create_map(mobj):
         return json.dumps(dict(json.loads(js_to_json(mobj.group(1) or '[]', vars=vars))))
@@ -3320,15 +3328,14 @@ def create_map(mobj):
         code = re.sub(r'new Date\((".+")\)', r'\g<1>', code)
         code = re.sub(r'new \w+\((.*?)\)', lambda m: json.dumps(m.group(0)), code)
 
-    return re.sub(r'''(?sx)
-        "(?:[^"\\]*(?:\\\\|\\['"nurtbfx/\n]))*[^"\\]*"|
-        '(?:[^'\\]*(?:\\\\|\\['"nurtbfx/\n]))*[^'\\]*'|
-        {comment}|,(?={skip}[\]}}])|
+    return re.sub(rf'''(?sx)
+        {STRING_RE}|
+        {COMMENT_RE}|,(?={SKIP_RE}[\]}}])|
         void\s0|(?:(?<![0-9])[eE]|[a-df-zA-DF-Z_$])[.a-zA-Z_$0-9]*|
-        \b(?:0[xX][0-9a-fA-F]+|0+[0-7]+)(?:{skip}:)?|
-        [0-9]+(?={skip}:)|
+        \b(?:0[xX][0-9a-fA-F]+|0+[0-7]+)(?:{SKIP_RE}:)?|
+        [0-9]+(?={SKIP_RE}:)|
         !+
-        '''.format(comment=COMMENT_RE, skip=SKIP_RE), fix_kv, code)
+        ''', fix_kv, code)
 
 
 def qualities(quality_ids):


Test patch:
diff --git a/test/test_utils.py b/test/test_utils.py
index df23f1f47..49ab3796b 100644
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -1100,6 +1100,12 @@ def test_js_to_json_edgecases(self):
         on = js_to_json('[1,//{},\n2]')
         self.assertEqual(json.loads(on), [1, 2])
 
+        on = js_to_json(R'"\^\$\#"')
+        self.assertEqual(json.loads(on), R'^$#', msg='Unnecessary escapes should be stripped')
+
+        on = js_to_json('\'"\\""\'')
+        self.assertEqual(json.loads(on), '"""', msg='Unnecessary quote escape should be escaped')
+
     def test_js_to_json_malformed(self):
         self.assertEqual(js_to_json('42a1'), '42"a1"')
         self.assertEqual(js_to_json('42a-1'), '42"a"-1')