Repository: yt-dlp/yt-dlp

Issue:
add --wait-for-video suport twitter/x
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm requesting a feature unrelated to a specific site
- [X] I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
- [X] I've verified that I'm running yt-dlp version **2023.10.13** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Provide a description that is worded well enough to be understood

Twiiter have from today a support for schedule a livestream.
for testing you can see this one (live at 00:00 GMT) https://twitter.com/i/broadcasts/1ynKOygEOPyJR

### Provide verbose output that clearly demonstrates the problem

- [x] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [ ] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
$ yt-dlp -vU --wait-for-video 0 --cookies-from-browser firefox https://twitter.com/i/broadcasts/1ynKOygEOPyJR
[debug] Command-line config: ['-vU', '--wait-for-video', '0', '--cookies-from-browser', 'firefox', 'https://twitter.com/i/broadcasts/1ynKOygEOPyJR']
[debug] User config "/home/user-name/.config/yt-dlp/config": []
Extracting cookies from firefox
[debug] Extracting cookies from: "/home/user-name/.mozilla/firefox/<firefoxID>.default/cookies.sqlite"
Extracted 2816 cookies from firefox
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 [b634ba742] (pip)
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-6.2.0-35-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2020.06.20, mutagen-1.45.1, pyxattr-0.7.2, secretstorage-3.3.1, sqlite3-3.37.2, websockets-9.1
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.10.13, Current version: stable@2023.10.13
yt-dlp is up to date (stable@2023.10.13)
[twitter:broadcast] Extracting URL: https://twitter.com/i/broadcasts/1ynKOygEOPyJR
[twitter:broadcast] 1ynKOygEOPyJR: Downloading legacy API JSON
[twitter:broadcast] 28_1718025875026042880: Downloading legacy API JSON
ERROR: [twitter:broadcast] 1ynKOygEOPyJR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 715, in extract
    ie_result = self._real_extract(url)
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/twitter.py", line 1589, in _real_extract
    source = self._call_api(
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/twitter.py", line 312, in _call_api
    result = self._download_json(
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1069, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1033, in download_handle
    res = self._download_webpage_handle(
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 860, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/networking/_urllib.py", line 410, in _send
    res = opener.open(urllib_req, timeout=float(request.extensions.get('timeout') or self.timeout))
  File "/usr/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 4051, in urlopen
    return self._request_director.send(req)
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/networking/common.py", line 114, in send
    response = handler.send(request)
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/networking/_helper.py", line 204, in wrapper
    return func(self, *args, **kwargs)
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/networking/common.py", line 325, in send
    return self._send(request)
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/networking/_urllib.py", line 415, in _send
    raise HTTPError(UrllibResponseAdapter(e.fp), redirect_loop='redirect error' in str(e)) from e
yt_dlp.networking.exceptions.HTTPError: HTTP Error 404: Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 847, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
  File "/home/user-name/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 4070, in urlopen
    raise _CompatHTTPError(e) from e
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 404: Not Found
```



Discussion:

Resolving patch:
diff --git a/yt_dlp/extractor/periscope.py b/yt_dlp/extractor/periscope.py
--- a/yt_dlp/extractor/periscope.py
+++ b/yt_dlp/extractor/periscope.py
@@ -4,6 +4,7 @@
     parse_iso8601,
     unescapeHTML,
 )
+from ..utils.traversal import traverse_obj
 
 
 class PeriscopeBaseIE(InfoExtractor):
@@ -20,8 +21,6 @@ def _parse_broadcast_data(self, broadcast, video_id):
         title = broadcast.get('status') or 'Periscope Broadcast'
         uploader = broadcast.get('user_display_name') or broadcast.get('username')
         title = '%s - %s' % (uploader, title) if uploader else title
-        is_live = broadcast.get('state').lower() == 'running'
-
         thumbnails = [{
             'url': broadcast[image],
         } for image in ('image_url', 'image_url_medium', 'image_url_small') if broadcast.get(image)]
@@ -31,12 +30,16 @@ def _parse_broadcast_data(self, broadcast, video_id):
             'title': title,
             'timestamp': parse_iso8601(broadcast.get('created_at')) or int_or_none(
                 broadcast.get('created_at_ms'), scale=1000),
+            'release_timestamp': int_or_none(broadcast.get('scheduled_start_ms'), scale=1000),
             'uploader': uploader,
             'uploader_id': broadcast.get('user_id') or broadcast.get('username'),
             'thumbnails': thumbnails,
             'view_count': int_or_none(broadcast.get('total_watched')),
             'tags': broadcast.get('tags'),
-            'is_live': is_live,
+            'live_status': {
+                'running': 'is_live',
+                'not_started': 'is_upcoming',
+            }.get(traverse_obj(broadcast, ('state', {str.lower}))) or 'was_live'
         }
 
     @staticmethod
diff --git a/yt_dlp/extractor/twitter.py b/yt_dlp/extractor/twitter.py
--- a/yt_dlp/extractor/twitter.py
+++ b/yt_dlp/extractor/twitter.py
@@ -1619,6 +1619,9 @@ def _real_extract(self, url):
         info['title'] = broadcast.get('status') or info.get('title')
         info['uploader_id'] = broadcast.get('twitter_username') or info.get('uploader_id')
         info['uploader_url'] = format_field(broadcast, 'twitter_username', 'https://twitter.com/%s', default=None)
+        if info['live_status'] == 'is_upcoming':
+            return info
+
         media_key = broadcast['media_key']
         source = self._call_api(
             f'live_video_stream/status/{media_key}', media_key)['source']


Test patch: