Repository: yt-dlp/yt-dlp

Issue:
[RUTV] https://smotrim.ru is broken now
### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.05.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Russian Federation

### Description

The site https://smotrim.ru has become inoperative after changing the advertising policy, which occurred on 06/18/2022.

### Verbose log

```shell
[debug] Command-line config: ['-vU', '-f', '1200', '-o', 'other\\%(upload_date)s_%(title)s.%(ext)s', '-a', 'listurls.txt']
[debug] Batch file urls: ['https://smotrim.ru/video/2427602']
[debug] Encodings: locale cp1251, fs utf-8, pref cp1251, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version 2022.05.18 [b14d523] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-7-6.1.7601-SP1
[debug] Checking exe version: ffprobe -bsfs
[debug] Checking exe version: ffmpeg -bsfs
[debug] exe versions: ffmpeg 2022-02-07-git-04cc7a5548-full_build-www.gyan.dev (setts), ffprobe 2022-02-07-git-04cc7a5548-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2021.10.08, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
Latest version: 2022.05.18, Current version: 2022.05.18
yt-dlp is up to date (2022.05.18)
[debug] [generic] Extracting URL: https://smotrim.ru/video/2427602
[generic] 2427602: Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] 2427602: Downloading webpage
[generic] 2427602: Extracting information
[debug] Looking for video embeds
ERROR: Unsupported URL: https://smotrim.ru/video/2427602
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1395, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1465, in __extract_info
  File "yt_dlp\extractor\common.py", line 642, in extract
  File "yt_dlp\extractor\generic.py", line 4031, in _real_extract
yt_dlp.utils.UnsupportedError: Unsupported URL: https://smotrim.ru/video/2427602
```



Discussion:
Does this issue is still valid to be `site-request`?. When i visited the website, the website return `HTTP 502`
Related issue in youtube-dl: https://github.com/ytdl-org/youtube-dl/issues/31045
The site opens for me. I assume it's blocked in western countries due to the current political situation
It opens here in Australia. Is it still showing a 502 error for you HobbyistDev?
It also opens in JP and US (via VPS). Probably your ISP is blocking the website?
> Probably your ISP is blocking the website?

@HobbyistDev Is there's a lock icon next to the URL in your address bar? Otherwise I don't think an ISP block would result in a 502 error, but I might be wrong
> It opens here in Australia. Is it still showing a 502 error for you HobbyistDev?

Yes, it's still showing 502 Error
Here's the log of curl
```
C:\Users\USER\Downloads>curl "https://smotrim.ru/video/2427602"
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>HTTP 502</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <style>
        html {font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;}
        body {background-color:#fff;padding:15px;}
        div.title {font-size:32px;font-weight:bold;line-height:1.2em;}
        div.sub-title {font-size:25px;}
        div.descr {margin-top:40px;}
        div.footer {margin-top:80px;color:#777;}
        div.guru {font-size:12px;color:#ccc;}
    </style>
</head>
<body>
    <div class="container">
        <div class="title">502 Error</div>
        <div class="sub-title">Bad gateway</div>

        <div class="descr">
            <p>The connection to site smotrim.ru is not established - it does not respond.</p>
            <p>Try accessing the site smotrim.ru later or contact administration of the site.</p>
        </div>

        <div class="footer">
            <div class="guru">
                IP: [REDACTED]<br/>
                Request: GET L3ZpZGVvLzI0Mjc2MDI=<br/>
                Guru meditation: OXBjZ0t4VUNJN2JwQWpJTThNMlFyYVhGeWNZODM5Nzk=<br/>
            </div>
        </div>

    </div>
</body>
</html>
```
Or in browser:
![Screenshot (1553)](https://user-images.githubusercontent.com/105957301/177060646-8a9af8a9-1579-43d0-85f5-d0a48453ffbd.png)

@HobbyistDev What country do you live in (feel free not to answer)
I live in a country in Southeast Asia, sorry if this didn't answer your question
Some words about situation with smotrim.ru now: it looks like it was a dns-spoofing attack ([proof](https://habr.com/ru/post/680816/)). I'm using Yandex Safe DNS and DoH in a browser, so has no such problems. But to download video I had to add |smotrim\\.ru in rutv.py and with little modification in lazy_extractors.py. But since version 2022.8.8 it need more changes. As a temporary solution I added to generic.py such code:
```
        if not found:
            # smotrim.ru
            found = re.findall(r'name="smotrim_player_(\d+)"(?:\s+[^=]+="[^"]*")*?\s+src="([^"]+\1)[^"]*"', webpage)
            if found:
                new_url = urllib.parse.urljoin(url, found[0][-1])
                if new_url != url:
                    self.report_following_redirect(new_url)
                    return {
                        '_type': 'url',
                        'url': new_url,
                    }
                else:
                    found = None
```
It works for me, but maybe more complex solution reqired.
@kvk-2015 @pukkandan 
Unfortunately, I can’t say by software code, but the URL can be changed as follows:

https://smotrim.ru/video/2427602 = https://player.vgtrk.com/iframe/video/id/2427602 (**work**) = https://player.smotrim.ru/iframe/video/id/2427602 (**not work**)

And the first URL replaced will work:

```
[debug] Command-line config: ['-vU', '-a', 'listurls.txt', '-F']
[debug] Batch file urls: ['https://player.vgtrk.com/iframe/video/id/2427602']
[debug] Encodings: locale cp1251, fs utf-8, pref cp1251, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2022.08.08 [3157158] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-7-6.1.7601-SP1
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2022-08-08-git-56973eb687-full_build-www.gyan.dev (setts), ffprobe 2022-08-08-git-56973eb687-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
[debug] [RUTV] Extracting URL: https://player.vgtrk.com/iframe/video/id/2427602
[RUTV] 2427602: Downloading JSON
[RUTV] 2427602: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7),channels, vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext,aext, hasaud, source, id
[info] Available formats for 2427602:
ID        EXT RESOLUTION |    FILESIZE   TBR PROTO | VCODEC    VBR ACODEC  ABR
------------------------------------------------------------------------------
hls-400   mp4 unknown    | ~ 503.47MiB  400k m3u8  | unknown  400k unknown  0k
hls-800   mp4 unknown    | ~1006.93MiB  800k m3u8  | unknown  800k unknown  0k
hls-1200  mp4 unknown    | ~   1.48GiB 1200k m3u8  | unknown 1200k unknown  0k
hls-1800  mp4 unknown    | ~   2.21GiB 1800k m3u8  | unknown 1800k unknown  0k
hls-4050  mp4 unknown    | ~   4.98GiB 4050k m3u8  | unknown 4050k unknown  0k
http-234  mp4 416x234    |                   https | unknown       unknown
http-360  mp4 640x360    |                   https | unknown       unknown
http-540  mp4 960x540    |                   https | unknown       unknown
http-720  mp4 1280x720   |                   https | unknown       unknown
http-1080 mp4 1920x1080  |                   https | unknown       unknown
```
Also on smotrim.ru there are realtime broadcasts of television channels, for example https://smotrim.ru/live/63338. But this URL also does not work, however, it can be replaced in accordance with the following:

https://smotrim.ru/live/63338 = https://player.vgtrk.com/iframe/live/id/63338 (**work**) = https://player.smotrim.ru/iframe/live/id/63338 (**not work**)

And also the first URL replaced will work:

```
[debug] Command-line config: ['-vU', '-a', 'listurls.txt', '-F']
[debug] Batch file urls: ['https://player.vgtrk.com/iframe/live/id/63338']
[debug] Encodings: locale cp1251, fs utf-8, pref cp1251, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2022.08.08 [3157158] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-7-6.1.7601-SP1
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2022-08-08-git-56973eb687-full_build-www.gyan.dev (setts), ffprobe 2022-08-08-git-56973eb687-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
[debug] [RUTV] Extracting URL: https://player.vgtrk.com/iframe/live/id/63338
[RUTV] 63338: Downloading JSON
[RUTV] 63338: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7),channels, vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto,vext, aext, hasaud, source, id
WARNING: "view_count" field is not numeric - forcing int conversion, there is an error in extractor
[info] Available formats for 63338:
ID       EXT RESOLUTION FPS |   TBR PROTO | VCODEC        VBR ACODEC    ABR
---------------------------------------------------------------------------
hls-765  mp4 1024x576    25 |  766k m3u8  | avc1.42c01f  766k mp4a.40.2  0k
hls-1425 mp4 1024x576    25 | 1426k m3u8  | avc1.42c01f 1426k mp4a.40.2  0k
hls-2120 mp4 1024x576    25 | 2121k m3u8  | avc1.4d401f 2121k mp4a.40.2  0k
hls-3256 mp4 1280x720    25 | 3256k m3u8  | avc1.4d401f 3256k mp4a.40.2  0k
hls-8976 mp4 1920x1080   25 | 8976k m3u8  | avc1.4d4028 8976k mp4a.40.2  0k
rtmp-    flv unknown        |       rtmp  | unknown           unknown
```
And this stream can even be saved, it will dump through FFMpeg. Why does https: //player.smotrim.ru/iframe not work ... I don't know.

Resolving patch:
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -1585,6 +1585,7 @@
 from .slideshare import SlideshareIE
 from .slideslive import SlidesLiveIE
 from .slutload import SlutloadIE
+from .smotrim import SmotrimIE
 from .snotr import SnotrIE
 from .sohu import SohuIE
 from .sonyliv import (
diff --git a/yt_dlp/extractor/smotrim.py b/yt_dlp/extractor/smotrim.py
new file mode 100644
--- /dev/null
+++ b/yt_dlp/extractor/smotrim.py
@@ -0,0 +1,65 @@
+from .common import InfoExtractor
+from ..utils import ExtractorError
+
+
+class SmotrimIE(InfoExtractor):
+    _VALID_URL = r'https?://smotrim\.ru/(?P<type>brand|video|article|live)/(?P<id>[0-9]+)'
+    _TESTS = [{  # video
+        'url': 'https://smotrim.ru/video/1539617',
+        'md5': 'b1923a533c8cab09679789d720d0b1c5',
+        'info_dict': {
+            'id': '1539617',
+            'ext': 'mp4',
+            'title': 'Полиглот. Китайский с нуля за 16 часов! Урок №16',
+            'description': '',
+        },
+        'add_ie': ['RUTV'],
+    }, {  # article (geo-restricted? plays fine from the US and JP)
+        'url': 'https://smotrim.ru/article/2813445',
+        'md5': 'e0ac453952afbc6a2742e850b4dc8e77',
+        'info_dict': {
+            'id': '2431846',
+            'ext': 'mp4',
+            'title': 'Новости культуры. Съёмки первой программы "Большие и маленькие"',
+            'description': 'md5:94a4a22472da4252bf5587a4ee441b99',
+        },
+        'add_ie': ['RUTV'],
+    }, {  # brand, redirect
+        'url': 'https://smotrim.ru/brand/64356',
+        'md5': '740472999ccff81d7f6df79cecd91c18',
+        'info_dict': {
+            'id': '2354523',
+            'ext': 'mp4',
+            'title': 'Большие и маленькие. Лучшее. 4-й выпуск',
+            'description': 'md5:84089e834429008371ea41ea3507b989',
+        },
+        'add_ie': ['RUTV'],
+    }, {  # live
+        'url': 'https://smotrim.ru/live/19201',
+        'info_dict': {
+            'id': '19201',
+            'ext': 'mp4',
+            # this looks like a TV channel name
+            'title': 'Россия Культура. Прямой эфир',
+            'description': '',
+        },
+        'add_ie': ['RUTV'],
+    }]
+
+    def _real_extract(self, url):
+        video_id, typ = self._match_valid_url(url).group('id', 'type')
+        rutv_type = 'video'
+        if typ not in ('video', 'live'):
+            webpage = self._download_webpage(url, video_id, f'Resolving {typ} link')
+            # there are two cases matching regex:
+            # 1. "embedUrl" in JSON LD (/brand/)
+            # 2. "src" attribute from iframe (/article/)
+            video_id = self._search_regex(
+                r'"https://player.smotrim.ru/iframe/video/id/(?P<video_id>\d+)/',
+                webpage, 'video_id', default=None)
+            if not video_id:
+                raise ExtractorError('There are no video in this page.', expected=True)
+        elif typ == 'live':
+            rutv_type = 'live'
+
+        return self.url_result(f'https://player.vgtrk.com/iframe/{rutv_type}/id/{video_id}')


Test patch: