Repository: yt-dlp/yt-dlp

Issue:
[TikTok] Invalid URL
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

France

### Provide a description that is worded well enough to be understood

I've had trouble downloading TikTok links saying "The read operation timed out". While searching for issues about this i came across #2396 which had a "fix" of spoofing the user agent, which i did. I'm running programmatically and changed the user agent using `yt_dlp.utils.std_headers["User-Agent"] = "facebookexternalhit/1.1"`. This worked fine for a while, and now i'm getting a new error that I can't seem to find a fix for.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['--user-agent', 'facebookexternalhit/1.1', '-vU', 'https://vm.tiktok.com/ZTR45GpSF/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (zip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-56-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2020.06.20, pyxattr-0.7.2, secretstorage-3.3.1, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vm.tiktok] Extracting URL: https://vm.tiktok.com/ZTR45GpSF/
[vm.tiktok] ZTR45GpSF: Downloading webpage
[debug] [generic] Extracting URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy
[generic] 7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG+60x/lLtJURz/Ry5GL7fxK7hqI5ZC/RnX9lm6YXtN4ajMomGgA=&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] 7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG+60x/lLtJURz/Ry5GL7fxK7hqI5ZC/RnX9lm6YXtN4ajMomGgA=&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy
Traceback (most recent call last):
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1485, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1561, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/generic.py", line 2721, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy
```



Discussion:
Just in case, this is the log without the user agent change:
```shell
[debug] Command-line config: ['-vU', 'https://vm.tiktok.com/ZTR45GpSF/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (zip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-56-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2020.06.20, pyxattr-0.7.2, secretstorage-3.3.1, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vm.tiktok] Extracting URL: https://vm.tiktok.com/ZTR45GpSF/
[vm.tiktok] ZTR45GpSF: Downloading webpage
[debug] [generic] Extracting URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy
[generic] 7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG+60x/lLtJURz/Ry5GL7fxK7hqI5ZC/RnX9lm6YXtN4ajMomGgA=&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy: Downloading webpage
ERROR: [generic] Unable to download webpage: The read operation timed out (caused by TimeoutError('The read operation timed out')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/generic.py", line 2582, in _real_extract
    full_response = self._request_webpage(url, video_id, headers={
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 825, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 807, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3692, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/usr/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/bin/yt-dlp/yt_dlp/utils.py", line 1527, in https_open
    return self.do_open(
  File "/usr/lib/python3.10/urllib/request.py", line 1352, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.10/ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.10/ssl.py", line 1130, in read
    return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out
```
TikTok has changed the redirect target locations of the `vm.tiktok.com` and `vt.tiktok.com` shortlinks to completely omit the `user_id`, e.g.:
`https://www.tiktok.com/@/video/7106798200794926362`
instead of
`https://www.tiktok.com/@user_id/video/7106798200794926362`

This URL format does not match `TikTokIE`'s `_VALID_URL` pattern, so the returned `url_result` falls back to the generic extractor. And while I can't reproduce this, some users clearly aren't able to download `www.tiktok.com` webpages at all without using the facebook bot user-agent, which is why OP's second example fails. The first example succeeds at downloading the webpage with the fb-bot UA, but the generic extractor doesn't know how to do anything with the data.

I'll have a PR up in a bit to fix this

Resolving patch:
diff --git a/yt_dlp/extractor/tiktok.py b/yt_dlp/extractor/tiktok.py
--- a/yt_dlp/extractor/tiktok.py
+++ b/yt_dlp/extractor/tiktok.py
@@ -30,11 +30,15 @@ class TikTokBaseIE(InfoExtractor):
     _WORKING_APP_VERSION = None
     _APP_NAME = 'trill'
     _AID = 1180
-    _API_HOSTNAME = 'api16-normal-c-useast1a.tiktokv.com'
     _UPLOADER_URL_FORMAT = 'https://www.tiktok.com/@%s'
     _WEBPAGE_HOST = 'https://www.tiktok.com/'
     QUALITIES = ('360p', '540p', '720p', '1080p')
 
+    @property
+    def _API_HOSTNAME(self):
+        return self._configuration_arg(
+            'api_hostname', ['api16-normal-c-useast1a.tiktokv.com'], ie_key=TikTokIE)[0]
+
     @staticmethod
     def _create_url(user_id, video_id):
         return f'https://www.tiktok.com/@{user_id or "_"}/video/{video_id}'
@@ -398,7 +402,7 @@ def _parse_aweme_video_web(self, aweme_detail, webpage_url):
 
 
 class TikTokIE(TikTokBaseIE):
-    _VALID_URL = r'https?://www\.tiktok\.com/(?:embed|@(?P<user_id>[\w\.-]+)/video)/(?P<id>\d+)'
+    _VALID_URL = r'https?://www\.tiktok\.com/(?:embed|@(?P<user_id>[\w\.-]+)?/video)/(?P<id>\d+)'
     _EMBED_REGEX = [rf'<(?:script|iframe)[^>]+\bsrc=(["\'])(?P<url>{_VALID_URL})']
 
     _TESTS = [{
@@ -944,8 +948,27 @@ class TikTokVMIE(InfoExtractor):
             'creator': 'SigmaChad',
         },
     }, {
-        'url': 'https://vm.tiktok.com/ZSe4FqkKd',
-        'only_matching': True,
+        'url': 'https://vm.tiktok.com/ZTR45GpSF/',
+        'info_dict': {
+            'id': '7106798200794926362',
+            'ext': 'mp4',
+            'title': 'md5:edc3e7ea587847f8537468f2fe51d074',
+            'uploader_id': '6997695878846268418',
+            'upload_date': '20220608',
+            'view_count': int,
+            'like_count': int,
+            'comment_count': int,
+            'thumbnail': r're:https://.+\.webp.*',
+            'uploader_url': 'https://www.tiktok.com/@MS4wLjABAAAAdZ_NcPPgMneaGrW0hN8O_J_bwLshwNNERRF5DxOw2HKIzk0kdlLrR8RkVl1ksrMO',
+            'duration': 29,
+            'timestamp': 1654680400,
+            'repost_count': int,
+            'artist': 'Akihitoko',
+            'track': 'original sound',
+            'description': 'md5:edc3e7ea587847f8537468f2fe51d074',
+            'uploader': 'akihitoko1',
+            'creator': 'Akihitoko',
+        },
     }, {
         'url': 'https://vt.tiktok.com/ZSe4FqkKd',
         'only_matching': True,


Test patch: