Repository: yt-dlp/yt-dlp

Issue:
[rai] 1080p quality and original MP4 files no longer working for many videos
### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.03.08.1**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
- [X] I've checked that all provided URLs are alive and playable in a browser
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Italy

### Description

Recently something has changed with RAI where the ability to directly download the original MP4 files in higher quality is no longer possible for many videos where it previously was.  The originals were always labeled as `https-`, so 1080p for example was `https-5000`, whereas currently only `hls-` streams are available for a lot of the site, though there are definite exceptions.

Here's an easy example:

https://www.raiplay.it/video/2019/10/Volevo-fare-la-rockstar-S1E1-86984292-1f9f-4947-a0d3-5d636b02e1ec.html
```
yt-dlp.exe --version
2022.03.08.1

yt-dlp --list-formats https://www.raiplay.it/video/2019/10/Volevo-fare-la-rockstar-S1E1-86984292-1f9f-4947-a0d3-5d636b02e1ec.html
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading video JSON
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading XML metadata for platform mon
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading m3u8 information
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading XML metadata for platform flash
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading XML metadata for platform native
[info] Available formats for 86984292-1f9f-4947-a0d3-5d636b02e1ec:
ID                       EXT RESOLUTION │   FILESIZE   TBR PROTO  │ VCODEC       VBR ACODEC  ABR MORE INFO
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hls-aac-Audiodescrizione mp4 audio only │                  m3u8_n │ audio only       unknown     [Audiodescrizione] Audiodescrizione
hls-aac-Italiano         mp4 audio only │                  m3u8_n │ audio only       unknown     [Italiano] Italiano
hls-1200                 mp4 736x414    │ ~502.50MiB 1288k m3u8_n │ unknown    1288k unknown  0k
hls-1800                 mp4 1024x576   │ ~685.86MiB 1758k m3u8_n │ unknown    1758k unknown  0k
hls-2400                 mp4 1280x720   │ ~914.48MiB 2344k m3u8_n │ unknown    2344k unknown  0k
```

I'm using this as example, because I already have this series in 1080p from November.  Here's the mediainfo for each (with the original remuxed to MKV):

<details>
  <summary>mediainfo from original 1080p http-5000 format from November 2021</summary>
  <p>
General

Unique ID                                : 297726856945746280844857093181880217146 (0xDFFC1B5E0FAE0D0328AF36EA24112E3A)
Complete name                            : Volevo.fare.la.rockstar.S01E01.1080p.RAI.WEB-DL.mkv
Format                                   : Matroska
Format version                           : Version 4 / Version 2
File size                                : 1.90 GiB
Duration                                 : 53 min 15 s
Overall bit rate                         : 5 099 kb/s
Encoded date                             : UTC 2021-11-24 22:02:31
Writing application                      : mkvmerge v62.0.0 ('Apollo') 64-bit
Writing library                          : libebml v1.4.2 + libmatroska v1.6.4

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 53 min 15 s
Bit rate                                 : 5 000 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.096
Stream size                              : 1.85 GiB (97%)
Writing library                          : x264 core 142 r2495 6a301b6
Encoding settings                        : cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x3:0x111 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=72 / lookahead_threads=6 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=50 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=5000 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.41 / aq=1:1.00
Default                                  : Yes
Forced                                   : No

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : A_AAC-2
Duration                                 : 53 min 15 s
Bit rate                                 : 128 kb/s
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -46 ms
Stream size                              : 48.8 MiB (3%)
Default                                  : Yes
Forced                                   : No
</p>
</details>


<details>
  <summary>mediainfo from current maximum hls-720 format from April 2022</summary>
  <p>
General

Complete name                            : Volevo fare la rockstar - S1E1 [86984292-1f9f-4947-a0d3-5d636b02e1ec].mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 916 MiB
Duration                                 : 53 min 16 s
Overall bit rate mode                    : Constant
Overall bit rate                         : 2 403 kb/s
Writing application                      : Lavf58.25.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L4
Format settings                          : CABAC / 5 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 5 frames
Format settings, GOP                     : M=3, N=25
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 53 min 16 s
Bit rate mode                            : Constant
Bit rate                                 : 2 400 kb/s
Width                                    : 1 280 pixels
Height                                   : 720 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.104
Stream size                              : 915 MiB (100%)
</p>
</details>

I have the entire first season of that show in 1080p from using the `https-5000` format, but now the highest quality available is the 720p `hls-2400` video-only stream.  There are still some things where `https-5000` shows up, however plenty of things that used to work no longer do.

I believe this to be a change on RAI's end and nothing to do with yt-dlp as I tried [version 2021.11.10](https://github.com/yt-dlp/yt-dlp/releases/tag/2021.11.10) and it also only finds `hls-2400` now.

Also of interest is that you now have to sometimes download both the video `hls-2400` and audio `hls-aac-Italiano` streams separately and then combine them yourself.

Maybe we can loop @nixxo in on this as they did the original implementation to get the higher quality raw MP4s from RAI to begin with.

### Verbose log

```shell
yt-dlp.exe -vU https://www.raiplay.it/video/2019/10/Volevo-fare-la-rockstar-S1E1-86984292-1f9f-4947-a0d3-5d636b02e1ec.html
[debug] Command-line config: ['-vU', 'https://www.raiplay.it/video/2019/10/Volevo-fare-la-rockstar-S1E1-86984292-1f9f-4947-a0d3-5d636b02e1ec.html']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2022.03.08.1 [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-92926-g2d580d727c, ffprobe N-92926-g2d580d727c
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2022.03.08.1, Current version: 2022.03.08.1
yt-dlp is up to date (2022.03.08.1)
[debug] [RaiPlay] Extracting URL: https://www.raiplay.it/video/2019/10/Volevo-fare-la-rockstar-S1E1-86984292-1f9f-4947-a0d3-5d636b02e1ec.html
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading video JSON
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading XML metadata for platform mon
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading m3u8 information
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading XML metadata for platform flash
[RaiPlay] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading XML metadata for platform native
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 86984292-1f9f-4947-a0d3-5d636b02e1ec: Downloading 1 format(s): hls-2400
[debug] Invoking downloader on "https://StreamCdnB1-b70cb04c54ab478189e9d8ee45637b13.msvdn.net/ostr1/podcastcdn/teche_root/YT_ITALIA_TECHE_HD_multiaudio/Q72tfYXSYrGcwS8iJmv9NweeqqEEqualeeqqEEqual_,1200,1800,2400/chunklist_b2344000_vo_slita_t64MjQwMA==.m3u8"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 320
[download] Destination: Volevo fare la rockstar - S1E1 [86984292-1f9f-4947-a0d3-5d636b02e1ec].mp4
[download] 100% of 944.03MiB in 12:13
[debug] ffprobe command line: ffprobe -hide_banner -show_format -show_streams -print_format json "file:Volevo fare la rockstar - S1E1 [86984292-1f9f-4947-a0d3-5d636b02e1ec].mp4"
[FixupM3u8] Fixing MPEG-TS in MP4 container of "Volevo fare la rockstar - S1E1 [86984292-1f9f-4947-a0d3-5d636b02e1ec].mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Volevo fare la rockstar - S1E1 [86984292-1f9f-4947-a0d3-5d636b02e1ec].mp4" -map 0 -dn -ignore_unknown -c copy -f mp4 "-bsf:a" aac_adtstoasc -movflags "+faststart" "file:Volevo fare la rockstar - S1E1 [86984292-1f9f-4947-a0d3-5d636b02e1ec].temp.mp4"
```



Discussion:
Thank for reporting it. It's an error in the extractor.
Rai updated the link structure of the manifest/mp4 files and the extractor regex does not catch all the media available.

I updated the regex inside to catch better the relinker url for mp4 files.

Resolving patch:
diff --git a/yt_dlp/extractor/rai.py b/yt_dlp/extractor/rai.py
--- a/yt_dlp/extractor/rai.py
+++ b/yt_dlp/extractor/rai.py
@@ -118,7 +118,7 @@ def _extract_relinker_info(self, relinker_url, video_id, audio_only=False):
         })
 
     def _create_http_urls(self, relinker_url, fmts):
-        _RELINKER_REG = r'https?://(?P<host>[^/]+?)/(?:i/)?(?P<extra>[^/]+?)/(?P<path>.+?)/(?P<id>\d+)(?:_(?P<quality>[\d\,]+))?(?:\.mp4|/playlist\.m3u8).+?'
+        _RELINKER_REG = r'https?://(?P<host>[^/]+?)/(?:i/)?(?P<extra>[^/]+?)/(?P<path>.+?)/(?P<id>\w+)(?:_(?P<quality>[\d\,]+))?(?:\.mp4|/playlist\.m3u8).+?'
         _MP4_TMPL = '%s&overrideUserAgentRule=mp4-%s'
         _QUALITY = {
             # tbr: w, h


Test patch: