Repository: yt-dlp/yt-dlp

Issue:
[TVer] Extractor error (caused by KeyError('publisher_id')) for some video URLs
### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2021.12.27**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
- [X] I've checked that all provided URLs are alive and playable in a browser
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [x] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Japan

### Description

I get extractor errors when downloading certain videos from TVer. There are some video URLs (such as https://tver.jp/corner/f0095338) that can be downloaded successfully, but there are also some live video URLs (such as https://tver.jp/feature/f0095053) that are now videos, but instead is a LIVE broadcast prepared for the future. The URLs I just introduced have an expiration date for viewing, so please be careful.

### Verbose log

```shell
[debug] Command-line config: ['-Uv', 'https://tver.jp/feature/f0095053']
[debug] Encodings: locale cp932, fs utf-8, out utf-8, err utf-8, pref cp932
[debug] yt-dlp version 2021.12.27 [6223f67a8]
[debug] Python version 3.8.12 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev (setts), ffprobe 4.4-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2021.12.27, Current version: 2021.12.27
yt-dlp is up to date (2021.12.27)
[TVer] Downloading JSON metadata
[debug] [TVer] Extracting URL: https://tver.jp/feature/f0095053
[TVer] f0095053: Downloading JSON metadata
ERROR: f0095053: An extractor error has occurred. (caused by KeyError('publisher_id')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\common.py", line 599, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\tver.py", line 44, in _real_extract
    p_id = main['publisher_id']
KeyError: 'publisher_id'
Traceback (most recent call last):
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\common.py", line 599, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\tver.py", line 44, in _real_extract
    p_id = main['publisher_id']
KeyError: 'publisher_id'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\YoutubeDL.py", line 1353, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\YoutubeDL.py", line 1423, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\common.py", line 629, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e, video_id=self.get_temp_id(url))
yt_dlp.utils.ExtractorError: f0095053: An extractor error has occurred. (caused by KeyError('publisher_id')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
```

[TVer] Extractor error (caused by KeyError('publisher_id')) for some video URLs
### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2021.12.27**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
- [X] I've checked that all provided URLs are alive and playable in a browser
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [x] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Japan

### Description

I get extractor errors when downloading certain videos from TVer. There are some video URLs (such as https://tver.jp/corner/f0095338) that can be downloaded successfully, but there are also some live video URLs (such as https://tver.jp/feature/f0095053) that are now videos, but instead is a LIVE broadcast prepared for the future. The URLs I just introduced have an expiration date for viewing, so please be careful.

### Verbose log

```shell
[debug] Command-line config: ['-Uv', 'https://tver.jp/feature/f0095053']
[debug] Encodings: locale cp932, fs utf-8, out utf-8, err utf-8, pref cp932
[debug] yt-dlp version 2021.12.27 [6223f67a8]
[debug] Python version 3.8.12 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev (setts), ffprobe 4.4-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2021.12.27, Current version: 2021.12.27
yt-dlp is up to date (2021.12.27)
[TVer] Downloading JSON metadata
[debug] [TVer] Extracting URL: https://tver.jp/feature/f0095053
[TVer] f0095053: Downloading JSON metadata
ERROR: f0095053: An extractor error has occurred. (caused by KeyError('publisher_id')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\common.py", line 599, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\tver.py", line 44, in _real_extract
    p_id = main['publisher_id']
KeyError: 'publisher_id'
Traceback (most recent call last):
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\common.py", line 599, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\tver.py", line 44, in _real_extract
    p_id = main['publisher_id']
KeyError: 'publisher_id'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\YoutubeDL.py", line 1353, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\YoutubeDL.py", line 1423, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Users\User\anaconda3\lib\site-packages\yt_dlp\extractor\common.py", line 629, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e, video_id=self.get_temp_id(url))
yt_dlp.utils.ExtractorError: f0095053: An extractor error has occurred. (caused by KeyError('publisher_id')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
```



Discussion:
That looks "working" since it's not airing as of you ran it. Do you expect something like `Premieres in 5 hours` for example?
At least YouTube live videos says like that so why not TVer too?
maybe like this?
```
$ ./yt-dlp.sh -s https://tver.jp/feature/f0095053
[TVer] Downloading JSON metadata
[TVer] f0095053: Downloading JSON metadata
ERROR: [TVer] f0095053: 1月19日(水) 19:00頃 配信開始
```
Yeah that'll be great!
Thanks, will make PR
That looks "working" since it's not airing as of you ran it. Do you expect something like `Premieres in 5 hours` for example?
At least YouTube live videos says like that so why not TVer too?
maybe like this?
```
$ ./yt-dlp.sh -s https://tver.jp/feature/f0095053
[TVer] Downloading JSON metadata
[TVer] f0095053: Downloading JSON metadata
ERROR: [TVer] f0095053: 1月19日(水) 19:00頃 配信開始
```
Yeah that'll be great!
Thanks, will make PR

Resolving patch:
diff --git a/yt_dlp/extractor/tver.py b/yt_dlp/extractor/tver.py
--- a/yt_dlp/extractor/tver.py
+++ b/yt_dlp/extractor/tver.py
@@ -5,10 +5,11 @@
 from .common import InfoExtractor
 from ..compat import compat_str
 from ..utils import (
+    ExtractorError,
     int_or_none,
     remove_start,
     smuggle_url,
-    try_get,
+    traverse_obj,
 )
 
 
@@ -38,13 +39,18 @@ def _real_initialize(self):
 
     def _real_extract(self, url):
         path, video_id = self._match_valid_url(url).groups()
-        main = self._download_json(
+        api_response = self._download_json(
             'https://api.tver.jp/v4/' + path, video_id,
-            query={'token': self._TOKEN})['main']
-        p_id = main['publisher_id']
-        service = remove_start(main['service'], 'ts_')
+            query={'token': self._TOKEN})
+        p_id = traverse_obj(api_response, ('main', 'publisher_id'))
+        if not p_id:
+            error_msg, expected = traverse_obj(api_response, ('episode', 0, 'textbar', 0, ('text', 'longer')), get_all=False), True
+            if not error_msg:
+                error_msg, expected = 'Failed to extract publisher ID', False
+            raise ExtractorError(error_msg, expected=expected)
+        service = remove_start(traverse_obj(api_response, ('main', 'service')), 'ts_')
 
-        r_id = main['reference_id']
+        r_id = traverse_obj(api_response, ('main', 'reference_id'))
         if service not in ('tx', 'russia2018', 'sebare2018live', 'gorin'):
             r_id = 'ref:' + r_id
         bc_url = smuggle_url(
@@ -53,8 +59,8 @@ def _real_extract(self, url):
 
         return {
             '_type': 'url_transparent',
-            'description': try_get(main, lambda x: x['note'][0]['text'], compat_str),
-            'episode_number': int_or_none(try_get(main, lambda x: x['ext']['episode_number'])),
+            'description': traverse_obj(api_response, ('main', 'note', 0, 'text'), expected_type=compat_str),
+            'episode_number': int_or_none(traverse_obj(api_response, ('main', 'ext', 'episode_number'), expected_type=compat_str)),
             'url': bc_url,
             'ie_key': 'BrightcoveNew',
         }


Test patch: