Repository: yt-dlp/yt-dlp

Issue:
[Site Request] wetv.vip
- [x] I'm reporting a new site support request
- [x] I've verified that I'm running yt-dlp version **2021.09.25**
- [x] I've checked that all provided URLs are alive and playable in a browser
- [x] I've checked that none of provided URLs violate any copyrights
- [x] The provided URLs do not contain any DRM to the best of my knowledge
- [x] I've searched the bugtracker for similar site support requests including closed ones


## Example URLs

- Single video: https://wetv.vip/play/air11ooo2rdsdi3-Cute-Programmer/v0040pr89t9-EP1-Cute-Programmer
- Single video: https://wetv.vip/play/bi3mcnv6qg0os9k-Manner-of-Death/b0035046n4p-EP1-Manner-of-Death
- Playlist: https://wetv.vip/play/air11ooo2rdsdi3-Cute-Programmer


## Description

Website with a lot of dramas and "anime" shows and movies. As mgtv they have a lot of original content.




Discussion:
@Tithen-Firion wrote a PR for that, last year.
https://github.com/Tithen-Firion
https://github.com/ytdl-org/youtube-dl/pull/26809

Ah you also the one who requested
https://github.com/ytdl-org/youtube-dl/issues/22040

> @Tithen-Firion wrote a PR for that, last year. https://github.com/Tithen-Firion [ytdl-org/youtube-dl#26809](https://github.com/ytdl-org/youtube-dl/pull/26809)
> 
> Ah you also the one who requested [ytdl-org/youtube-dl#22040](https://github.com/ytdl-org/youtube-dl/issues/22040)

oh sry, I totally forgot about it.. its been a year :( and for some reason search didnt find me anything here.

edit: oh its on old repo.. sry I didnt check it.. but seems like that PR is probably dead anyway
It is a bit too complex for me to easily port it over myself. Perhaps @Tithen-Firion would be willing to port it themselves?
> > @Tithen-Firion wrote a PR for that, last year. https://github.com/Tithen-Firion [ytdl-org/youtube-dl#26809](https://github.com/ytdl-org/youtube-dl/pull/26809)
> > Ah you also the one who requested [ytdl-org/youtube-dl#22040](https://github.com/ytdl-org/youtube-dl/issues/22040)
> 
> oh sry, I totally forgot about it.. its been a year :( and for some reason search didnt find me anything here.
> 
> edit: oh its on old repo.. sry I didnt check it.. but seems like that PR is probably dead anyway

The ckey algo still works (which I've been using also) that's just the very important.
Maybe the api are already outdated or just needed a little modification.
Tithen-Firion seems to be inactive, so ytdl-org/youtube-dl#26809 should be ported by another user
Not familiar with this project, not interested with porting. BTW, porting? I thought this was a youtube-dl fork. Are extractors not compatible with each other?

As for the wetv - like I said in the original PR, most of the `CKey` code is just AES-CBC encryption so you can simplify it greatly.

But there's one issue. Downloading with youtube-dl was painfully slow. That's because no matter which downloader I used (native or ffmpeg) it was downloading one chunk at a time. And I think wetv is throttling the download speed. So I gave up on that. Instead I grabbed the `.m3u8` playlist from wetv website and downloaded with some M3U8 downloader (don't remember which one but it let you download multiple chunks at once).

So unless this fork is modified to download multiple chunks at once it will be a pretty bad experience.
> So unless this fork is modified to download multiple chunks at once it will be a pretty bad experience.

I think it already does that, via `-N` or `--downloader aria2c`.
>  porting? I thought this was a youtube-dl fork. Are extractors not compatible with each other?

ok, porting may not be the right word. The extractors are mostly compatible. But more importantly, it is helpful for the original author to copy the PR and address reviews. If not, I can only make any changes based on how much I understand without any feedback on what the intend of some of the code may be

> As for the wetv - like I said in the original PR, most of the CKey code is just AES-CBC encryption so you can simplify it greatly.

This is one example of the same. I havent yet looked at how complex the CKey code is, but I (or whoever "ports" it) will need to go through and thoroughly understand the entire thing to decide which parts can be replaced by the existing CBC decryptor

> So unless this fork is modified to download multiple chunks at once it will be a pretty bad experience.

Yes, we have the ability to download fragments in parallel

> Not familiar with this project, not interested with porting. 

That's fine. Just pinged you in case you were interested. Thanks for replying anyway
so it seems there is no hope for this site to be supported? :(

Resolving patch:
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -2088,6 +2088,7 @@
     WeiboMobileIE
 )
 from .weiqitv import WeiqiTVIE
+from .wetv import WeTvEpisodeIE, WeTvSeriesIE
 from .wikimedia import WikimediaIE
 from .willow import WillowIE
 from .wimtv import WimTVIE
diff --git a/yt_dlp/extractor/wetv.py b/yt_dlp/extractor/wetv.py
new file mode 100644
--- /dev/null
+++ b/yt_dlp/extractor/wetv.py
@@ -0,0 +1,209 @@
+import re
+import time
+
+from .common import InfoExtractor
+from ..aes import aes_cbc_encrypt
+from ..utils import bytes_to_intlist, determine_ext, intlist_to_bytes, int_or_none, traverse_obj
+
+
+class WeTvBaseIE(InfoExtractor):
+    _VALID_URL_BASE = r'https?://(?:www\.)?wetv\.vip/(?:[^?#]+/)?play'
+
+    def _get_ckey(self, video_id, url, app_version, platform):
+        ua = self.get_param('http_headers')['User-Agent']
+
+        payload = (f'{video_id}|{int(time.time())}|mg3c3b04ba|{app_version}|0000000000000000|'
+                   f'{platform}|{url[:48]}|{ua.lower()[:48]}||Mozilla|Netscape|Win32|00|')
+
+        ciphertext_int_bytes = aes_cbc_encrypt(
+            bytes_to_intlist(bytes(f'|{sum(map(ord, payload))}|{payload}', 'utf-8')),
+            bytes_to_intlist(b'Ok\xda\xa3\x9e/\x8c\xb0\x7f^r-\x9e\xde\xf3\x14'),
+            bytes_to_intlist(b'\x01PJ\xf3V\xe6\x19\xcf.B\xbb\xa6\x8c?p\xf9'),
+            'whitespace')
+
+        return intlist_to_bytes(ciphertext_int_bytes).hex()
+
+    def _get_video_api_response(self, video_url, video_id, series_id, subtitle_format, video_format, video_quality):
+        app_version = '3.5.57'
+        platform = '4830201'
+
+        ckey = self._get_ckey(video_id, video_url, app_version, platform)
+        query = {
+            'vid': video_id,
+            'cid': series_id,
+            'cKey': ckey,
+            'encryptVer': '8.1',
+            'spcaptiontype': '1' if subtitle_format == 'vtt' else '0',  # 0 - SRT, 1 - VTT
+            'sphls': '1' if video_format == 'hls' else '0',  # 0 - MP4, 1 - HLS
+            'defn': video_quality,  # '': 480p, 'shd': 720p, 'fhd': 1080p
+            'spsrt': '1',  # Enable subtitles
+            'sphttps': '1',  # Enable HTTPS
+            'otype': 'json',  # Response format: xml, json,
+            'dtype': '1',
+            'spwm': '1',
+            'host': 'wetv.vip',  # These three values are needed for SHD
+            'referer': 'wetv.vip',
+            'ehost': video_url,
+            'appVer': app_version,
+            'platform': platform,
+        }
+
+        return self._search_json(r'QZOutputJson=', self._download_webpage(
+            'https://play.wetv.vip/getvinfo', video_id, query=query), 'api_response', video_id)
+
+    def _get_webpage_metadata(self, webpage, video_id):
+        return self._parse_json(
+            traverse_obj(self._search_nextjs_data(webpage, video_id), ('props', 'pageProps', 'data')),
+            video_id, fatal=False)
+
+
+class WeTvEpisodeIE(WeTvBaseIE):
+    IE_NAME = 'wetv:episode'
+    _VALID_URL = WeTvBaseIE._VALID_URL_BASE + r'/(?P<series_id>\w+)(?:-[^?#]+)?/(?P<id>\w+)(?:-[^?#]+)?'
+
+    _TESTS = [{
+        'url': 'https://wetv.vip/en/play/air11ooo2rdsdi3-Cute-Programmer/v0040pr89t9-EP1-Cute-Programmer',
+        'md5': 'a046f565c9dce9b263a0465a422cd7bf',
+        'info_dict': {
+            'id': 'v0040pr89t9',
+            'ext': 'mp4',
+            'title': 'EP1: Cute Programmer',
+            'description': 'md5:e87beab3bf9f392d6b9e541a63286343',
+            'thumbnail': r're:^https?://[^?#]+air11ooo2rdsdi3',
+            'series': 'Cute Programmer',
+            'episode': 'Episode 1',
+            'episode_number': 1,
+            'duration': 2835,
+        },
+    }, {
+        'url': 'https://wetv.vip/en/play/u37kgfnfzs73kiu/p0039b9nvik',
+        'md5': '4d9d69bcfd11da61f4aae64fc6b316b3',
+        'info_dict': {
+            'id': 'p0039b9nvik',
+            'ext': 'mp4',
+            'title': 'EP1: You Are My Glory',
+            'description': 'md5:831363a4c3b4d7615e1f3854be3a123b',
+            'thumbnail': r're:^https?://[^?#]+u37kgfnfzs73kiu',
+            'series': 'You Are My Glory',
+            'episode': 'Episode 1',
+            'episode_number': 1,
+            'duration': 2454,
+        },
+    }, {
+        'url': 'https://wetv.vip/en/play/lcxgwod5hapghvw-WeTV-PICK-A-BOO/i0042y00lxp-Zhao-Lusi-Describes-The-First-Experiences-She-Had-In-Who-Rules-The-World-%7C-WeTV-PICK-A-BOO',
+        'md5': '71133f5c2d5d6cad3427e1b010488280',
+        'info_dict': {
+            'id': 'i0042y00lxp',
+            'ext': 'mp4',
+            'title': 'md5:f7a0857dbe5fbbe2e7ad630b92b54e6a',
+            'description': 'md5:76260cb9cdc0ef76826d7ca9d92fadfa',
+            'thumbnail': r're:^https?://[^?#]+lcxgwod5hapghvw',
+            'series': 'WeTV PICK-A-BOO',
+            'episode': 'Episode 0',
+            'episode_number': 0,
+            'duration': 442,
+        },
+    }]
+
+    def _extract_video_formats_and_subtitles(self, api_response, video_id, video_quality):
+        video_response = api_response['vl']['vi'][0]
+        video_width = video_response.get('vw')
+        video_height = video_response.get('vh')
+
+        formats, subtitles = [], {}
+        for video_format in video_response['ul']['ui']:
+            if video_format.get('hls'):
+                fmts, subs = self._extract_m3u8_formats_and_subtitles(
+                    video_format['url'] + video_format['hls']['pname'], video_id, 'mp4', fatal=False)
+                for f in fmts:
+                    f['width'] = video_width
+                    f['height'] = video_height
+
+                formats.extend(fmts)
+                self._merge_subtitles(subs, target=subtitles)
+            else:
+                formats.append({
+                    'url': f'{video_format["url"]}{video_response["fn"]}?vkey={video_response["fvkey"]}',
+                    'width': video_width,
+                    'height': video_height,
+                    'ext': 'mp4',
+                })
+
+        return formats, subtitles
+
+    def _extract_video_subtitles(self, api_response, subtitles_format):
+        subtitles = {}
+        for subtitle in traverse_obj(api_response, ('sfl', 'fi')):
+            subtitles.setdefault(subtitle['lang'].lower(), []).append({
+                'url': subtitle['url'],
+                'ext': subtitles_format,
+                'protocol': 'm3u8_native' if determine_ext(subtitle['url']) == 'm3u8' else 'http',
+            })
+
+        return subtitles
+
+    def _real_extract(self, url):
+        video_id, series_id = self._match_valid_url(url).group('id', 'series_id')
+        webpage = self._download_webpage(url, video_id)
+
+        formats, subtitles = [], {}
+        for video_format, subtitle_format, video_quality in (('mp4', 'srt', ''), ('hls', 'vtt', 'shd'), ('hls', 'vtt', 'fhd')):
+            api_response = self._get_video_api_response(url, video_id, series_id, subtitle_format, video_format, video_quality)
+
+            fmts, subs = self._extract_video_formats_and_subtitles(api_response, video_id, video_quality)
+            native_subtitles = self._extract_video_subtitles(api_response, subtitle_format)
+
+            formats.extend(fmts)
+            self._merge_subtitles(subs, native_subtitles, target=subtitles)
+
+        self._sort_formats(formats)
+        webpage_metadata = self._get_webpage_metadata(webpage, video_id)
+
+        return {
+            'id': video_id,
+            'title': (self._og_search_title(webpage)
+                      or traverse_obj(webpage_metadata, ('coverInfo', 'description'))),
+            'description': (self._og_search_description(webpage)
+                            or traverse_obj(webpage_metadata, ('coverInfo', 'description'))),
+            'formats': formats,
+            'subtitles': subtitles,
+            'thumbnail': self._og_search_thumbnail(webpage),
+            'duration': int_or_none(traverse_obj(webpage_metadata, ('videoInfo', 'duration'))),
+            'series': traverse_obj(webpage_metadata, ('coverInfo', 'title')),
+            'episode_number': int_or_none(traverse_obj(webpage_metadata, ('videoInfo', 'episode'))),
+        }
+
+
+class WeTvSeriesIE(WeTvBaseIE):
+    _VALID_URL = WeTvBaseIE._VALID_URL_BASE + r'/(?P<id>\w+)(?:-[^/?#]+)?/?(?:[?#]|$)'
+
+    _TESTS = [{
+        'url': 'https://wetv.vip/play/air11ooo2rdsdi3-Cute-Programmer',
+        'info_dict': {
+            'id': 'air11ooo2rdsdi3',
+            'title': 'Cute Programmer',
+            'description': 'md5:e87beab3bf9f392d6b9e541a63286343',
+        },
+        'playlist_count': 30,
+    }, {
+        'url': 'https://wetv.vip/en/play/u37kgfnfzs73kiu-You-Are-My-Glory',
+        'info_dict': {
+            'id': 'u37kgfnfzs73kiu',
+            'title': 'You Are My Glory',
+            'description': 'md5:831363a4c3b4d7615e1f3854be3a123b',
+        },
+        'playlist_count': 32,
+    }]
+
+    def _real_extract(self, url):
+        series_id = self._match_id(url)
+        webpage = self._download_webpage(url, series_id)
+        webpage_metadata = self._get_webpage_metadata(webpage, series_id)
+
+        episode_paths = (re.findall(r'<a[^>]+class="play-video__link"[^>]+href="(?P<path>[^"]+)', webpage)
+                         or [f'/{series_id}/{episode["vid"]}' for episode in webpage_metadata.get('videoList')])
+
+        return self.playlist_from_matches(
+            episode_paths, series_id, ie=WeTvEpisodeIE,
+            title=traverse_obj(webpage_metadata, ('coverInfo', 'title')) or self._og_search_title(webpage),
+            description=traverse_obj(webpage_metadata, ('coverInfo', 'description')) or self._og_search_description(webpage))


Test patch: