Repository: yt-dlp/yt-dlp
Pull request #8295 (2023-10-06 12:42:34)

[core] Use lazy quantifier in regex for sanitizing dates in js_to_json
Given the following JavaScript code as string:

    [new Date("foobar"), '("baz")']

The greedy quantifier's usage matches the string till the last quote instead of just the text inside the parenthesis. This patch fixes that using the `STRING_RE` regex for proper string literal matching.

Also updated tests.

<details open><summary>Template</summary> <!-- OPEN is intentional -->

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check all of the following options that apply:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)

### What is the purpose of your *pull request*?
- [x] Core bug fix/improvement

<!-- Do NOT edit/remove anything below this! -->
</details><details><summary>Copilot Summary</summary>  

<!--
copilot:all
-->
### <samp>🤖 Generated by Copilot at c3dadad</samp>

### Summary
🐛🕒🎬

<!--
1.  🐛 - This emoji represents a bug, and can be used to indicate that the change fixes a bug in the code.
2.  🕒 - This emoji represents a clock, and can be used to indicate that the change relates to the `new Date` constructor, which creates a date object based on a string or a number.
3.  🎬 - This emoji represents a clapper board, and can be used to indicate that the change is part of a feature that extracts metadata from YouTube videos.
-->
Fix `js_to_json` bug with single-quoted strings in `new Date` arguments. This improves YouTube metadata extraction.

> _`js_to_json` fixed_
> _Single quotes now accepted_
> _Winter of errors_

### Walkthrough
* Fix bug in `js_to_json` function that caused parsing errors for single-quoted strings in `new Date` constructor ([link](https://github.com/yt-dlp/yt-dlp/pull/8295/files?diff=unified&w=0#diff-feda8f56946dc048e754111850baaef0eec4b9f8bbc2d3f04b1a785626ea5c0eL2747-R2748))



</details>


coletdjnz: Also update/add tests if feasible (see `test/test_utils.py` - possibly `test_js_to_json_edgecases`).
Grub4K: We can instead use proper greedy string matching by doing this:
```diff
-        code = re.sub(r'new Date\((".+")\)', r'\g<1>', code)
+        code = re.sub(rf'new Date\(({STRING_RE})\)', r'\g<1>', code)
```
It passes the test you provided:
```py
        on = js_to_json('[new Date("spam"), \'("eggs")\']')
        self.assertEqual(json.loads(on), ['spam', '("eggs")'], msg='Date regex should match a single string')
```
awalgarg: Thanks both, updated the PR. Using `STRING_RE` for matching string literals and updated tests.

Patch:
diff --git a/yt_dlp/utils/_utils.py b/yt_dlp/utils/_utils.py
index ba6242380..3dc17bf59 100644
--- a/yt_dlp/utils/_utils.py
+++ b/yt_dlp/utils/_utils.py
@@ -2744,7 +2744,7 @@ def create_map(mobj):
     code = re.sub(r'(?:new\s+)?Array\((.*?)\)', r'[\g<1>]', code)
     code = re.sub(r'new Map\((\[.*?\])?\)', create_map, code)
     if not strict:
-        code = re.sub(r'new Date\((".+")\)', r'\g<1>', code)
+        code = re.sub(rf'new Date\(({STRING_RE})\)', r'\g<1>', code)
         code = re.sub(r'new \w+\((.*?)\)', lambda m: json.dumps(m.group(0)), code)
         code = re.sub(r'parseInt\([^\d]+(\d+)[^\d]+\)', r'\1', code)
         code = re.sub(r'\(function\([^)]*\)\s*\{[^}]*\}\s*\)\s*\(\s*(["\'][^)]*["\'])\s*\)', r'\1', code)


Test patch:
diff --git a/test/test_utils.py b/test/test_utils.py
index fd612ff86..77040f29c 100644
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -1209,6 +1209,9 @@ def test_js_to_json_edgecases(self):
         on = js_to_json('\'"\\""\'')
         self.assertEqual(json.loads(on), '"""', msg='Unnecessary quote escape should be escaped')
 
+        on = js_to_json('[new Date("spam"), \'("eggs")\']')
+        self.assertEqual(json.loads(on), ['spam', '("eggs")'], msg='Date regex should match a single string')
+
     def test_js_to_json_malformed(self):
         self.assertEqual(js_to_json('42a1'), '42"a1"')
         self.assertEqual(js_to_json('42a-1'), '42"a"-1')
@@ -1220,11 +1223,13 @@ def test_js_to_json_template_literal(self):
         self.assertEqual(js_to_json('`${name}"${name}"`', {'name': '5'}), '"5\\"5\\""')
         self.assertEqual(js_to_json('`${name}`', {}), '"name"')
 
-    def test_js_to_json_map_array_constructors(self):
+    def test_js_to_json_common_constructors(self):
         self.assertEqual(json.loads(js_to_json('new Map([["a", 5]])')), {'a': 5})
         self.assertEqual(json.loads(js_to_json('Array(5, 10)')), [5, 10])
         self.assertEqual(json.loads(js_to_json('new Array(15,5)')), [15, 5])
         self.assertEqual(json.loads(js_to_json('new Map([Array(5, 10),new Array(15,5)])')), {'5': 10, '15': 5})
+        self.assertEqual(json.loads(js_to_json('new Date("123")')), "123")
+        self.assertEqual(json.loads(js_to_json('new Date(\'2023-10-19\')')), "2023-10-19")
 
     def test_extract_attributes(self):
         self.assertEqual(extract_attributes('<e x="y">'), {'x': 'y'})