Repository: yt-dlp/yt-dlp

Issue:
[TV4] HTTP Error 503: Service Unavailable
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Sweden

### Provide a description that is worded well enough to be understood

www.tv4play.se not working:
Unable to download JSON metadata: HTTP Error 503: Service Unavailable

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['https://www.tv4play.se/program/next-level-chef-us/del-5-s%C3%A4song-1/13772514', '-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (pip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-52-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2020.06.20, mutagen-1.46.0, pyxattr-0.7.2, secretstorage-3.3.1, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] Using fake IP 78.67.182.65 (SE) as X-Forwarded-For
[debug] [TV4] Extracting URL: https://www.tv4play.se/program/next-level-chef-us/del-5-s%C3%A4song-1/13772514
[TV4] 13772514: Downloading video info JSON
ERROR: [TV4] 13772514: Unable to download JSON metadata: HTTP Error 503: Service Unavailable (caused by <HTTPError 503: 'Service Unavailable'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/extractor/tv4.py", line 75, in _real_extract
    info = self._download_json(
  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1034, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 998, in download_handle
    res = self._download_webpage_handle(
  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 868, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 825, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 807, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
  File "/home/user/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3692, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: Service Unavailable
```



Discussion:
I got the same issue. My guess it has to do with the fact that you have to login using a free account on tv4play now. This seems to have happened just recently.

![image](https://user-images.githubusercontent.com/10449598/201760420-1df3b169-3473-4c30-928b-9f468b48169a.png)


I don't get any login prompt, but the provided link appears to be DRM protected
Should be the same issue as this: https://github.com/spaam/svtplay-dl/issues/1509
Fix: https://github.com/spaam/svtplay-dl/commit/8950be8eec917b3bf206c3d32d6df2a6fe4797be

And also, just to clarify, there was no need to log in or anything. Just the URL that was changed.
The extractor makes two API calls and the failing one (the first) is the metadata retrieval, while the linked change corresponds to the second call. These use different domains in the b17g.net domain. Apparently svtplay-dl already used the hydration JSON to get the metadata (a subset is also in the ld+json VideoObject).

Anyhow: https://playback2.a2d.tv/docs/swagger.json

The yt-dl extractor is identical apart from sttl processing and conventions: unsurprisingly it has the same issue.

The new API is more strictly blocked (XFF doesn't work) and returns this with 401 for the Brexited UK, making it difficult to check the returned data (which from the above link may combine the two results that we were fetching before):
```
{"errorCode":"ASSET_PLAYBACK_PORTABILITY_REQUIRED_FOR_GEO_LOCATION","message":"Ip address 51.6.64.223 does not have access to platform \'tv4\'"}
```
Possibly login cookies would bypass this.
So is anyone working on a solution for this? And with my little knowledge I don't think it is DRM-protected.
> And with my little knowledge I don't think it is DRM-protected.

Why do you say so?

![image](https://user-images.githubusercontent.com/6623388/202781346-bc8b6fb1-9955-4ada-90ab-53be43332922.png)

Since It is possible to download with https://github.com/spaam/svtplay-dl/ with the latest fix and that program does not "decode" DRM as far as I know.
But maybe I have misunderstood that.


> Fix: [spaam/svtplay-dl@8950be8](https://github.com/spaam/svtplay-dl/commit/8950be8eec917b3bf206c3d32d6df2a6fe4797be)

Ah, my bad. I did not notice this part in previous comment 😓

Does this work with cookies?

```diff
diff --git a/yt_dlp/extractor/tv4.py b/yt_dlp/extractor/tv4.py
index 1378a6f57..22ee7e2f3 100644
--- a/yt_dlp/extractor/tv4.py
+++ b/yt_dlp/extractor/tv4.py
@@ -73,12 +73,14 @@ def _real_extract(self, url):
         video_id = self._match_id(url)

         info = self._download_json(
-            'https://playback-api.b17g.net/asset/%s' % video_id,
+            f'https://playback2.a2d.tv/play/{video_id}',
             video_id, 'Downloading video info JSON', query={
                 'service': 'tv4',
                 'device': 'browser',
                 'protocol': 'hls,dash',
                 'drm': 'widevine',
+                'browser': 'GoogleChrome',
+                'capabilities': 'live-drm-adstitch-2,expired_assets',
             })['metadata']

         title = info['title']
```

PS: Related https://github.com/spaam/svtplay-dl/issues/1509
It does not work for me with fix, with or without cookies.
Latest svtplay-dl does not work either.

502 Bad Gateway. Does it work for anyone else?
So it might work now. The 502 was for the manifest_url still pointing to the b17g domain. 

> > Fix: [spaam/svtplay-dl@8950be8](https://github.com/spaam/svtplay-dl/commit/8950be8eec917b3bf206c3d32d6df2a6fe4797be)
> 
> Ah, my bad. I did not notice this part in previous comment 😓
> 
> Does this work with cookies?
> 
> ```diff
> diff --git a/yt_dlp/extractor/tv4.py b/yt_dlp/extractor/tv4.py
> index 1378a6f57..22ee7e2f3 100644
> --- a/yt_dlp/extractor/tv4.py
> +++ b/yt_dlp/extractor/tv4.py
> @@ -73,12 +73,14 @@ def _real_extract(self, url):
>          video_id = self._match_id(url)
> 
>          info = self._download_json(
> -            'https://playback-api.b17g.net/asset/%s' % video_id,
> +            f'https://playback2.a2d.tv/play/{video_id}',
>              video_id, 'Downloading video info JSON', query={
>                  'service': 'tv4',
>                  'device': 'browser',
>                  'protocol': 'hls,dash',
>                  'drm': 'widevine',
> +                'browser': 'GoogleChrome',
> +                'capabilities': 'live-drm-adstitch-2,expired_assets',
>              })['metadata']
> 
>          title = info['title']
> ```
> 
> PS: Related [spaam/svtplay-dl#1509](https://github.com/spaam/svtplay-dl/issues/1509)


This fix only changes the info_url. The manifest_url also has the be changed to 'playback2.a2d.tv/play/'. The info_url might be 'playback2.a2d.tv/assets' actually?

Resolving patch:
diff --git a/yt_dlp/extractor/tv4.py b/yt_dlp/extractor/tv4.py
--- a/yt_dlp/extractor/tv4.py
+++ b/yt_dlp/extractor/tv4.py
@@ -2,8 +2,11 @@
 
 from .common import InfoExtractor
 from ..utils import (
+    bool_or_none,
     int_or_none,
     parse_iso8601,
+    traverse_obj,
+    url_or_none,
 )
 
 
@@ -20,19 +23,25 @@ class TV4IE(InfoExtractor):
                 sport/|
             )
         )(?P<id>[0-9]+)'''
-    _GEO_COUNTRIES = ['SE']
+    _GEO_BYPASS = False
     _TESTS = [
         {
+            # not geo-restricted
             'url': 'http://www.tv4.se/kalla-fakta/klipp/kalla-fakta-5-english-subtitles-2491650',
             'md5': 'cb837212f342d77cec06e6dad190e96d',
             'info_dict': {
                 'id': '2491650',
                 'ext': 'mp4',
                 'title': 'Kalla Fakta 5 (english subtitles)',
-                'thumbnail': r're:^https?://.*\.jpg$',
-                'timestamp': int,
+                'description': '2491650',
+                'series': 'Kalla fakta',
+                'duration': 1335,
+                'thumbnail': r're:^https?://[^/?#]+/api/v2/img/',
+                'timestamp': 1385373240,
                 'upload_date': '20131125',
             },
+            'params': {'skip_download': 'm3u8'},
+            'expected_warnings': ['Unable to download f4m manifest'],
         },
         {
             'url': 'http://www.tv4play.se/iframe/video/3054113',
@@ -46,6 +55,7 @@ class TV4IE(InfoExtractor):
                 'timestamp': int,
                 'upload_date': '20150130',
             },
+            'skip': '404 Not Found',
         },
         {
             'url': 'http://www.tv4play.se/sport/3060959',
@@ -69,29 +79,28 @@ class TV4IE(InfoExtractor):
         }
     ]
 
+    def _call_api(self, endpoint, video_id, headers=None, query={}):
+        return self._download_json(
+            f'https://playback2.a2d.tv/{endpoint}/{video_id}', video_id,
+            f'Downloading {endpoint} API JSON', headers=headers, query={
+                'service': 'tv4',
+                'device': 'browser',
+                'protocol': 'hls',
+                **query,
+            })
+
     def _real_extract(self, url):
         video_id = self._match_id(url)
 
-        info = self._download_json(
-            'https://playback-api.b17g.net/asset/%s' % video_id,
-            video_id, 'Downloading video info JSON', query={
-                'service': 'tv4',
-                'device': 'browser',
-                'protocol': 'hls,dash',
-                'drm': 'widevine',
-            })['metadata']
+        info = traverse_obj(self._call_api('asset', video_id, query={
+            'protocol': 'hls,dash',
+            'drm': 'widevine',
+        }), ('metadata', {dict})) or {}
 
-        title = info['title']
+        manifest_url = self._call_api(
+            'play', video_id, headers=self.geo_verification_headers())['playbackItem']['manifestUrl']
 
-        manifest_url = self._download_json(
-            'https://playback-api.b17g.net/media/' + video_id,
-            video_id, query={
-                'service': 'tv4',
-                'device': 'browser',
-                'protocol': 'hls',
-            })['playbackItem']['manifestUrl']
-        formats = []
-        subtitles = {}
+        formats, subtitles = [], {}
 
         fmts, subs = self._extract_m3u8_formats_and_subtitles(
             manifest_url, video_id, 'mp4',
@@ -117,20 +126,24 @@ def _real_extract(self, url):
         subtitles = self._merge_subtitles(subtitles, subs)
 
         if not formats and info.get('is_geo_restricted'):
-            self.raise_geo_restricted(countries=self._GEO_COUNTRIES, metadata_available=True)
+            self.raise_geo_restricted(
+                'This video is not available from your location due to geo-restriction, or not being authenticated',
+                countries=['SE'])
 
         return {
             'id': video_id,
-            'title': title,
             'formats': formats,
             'subtitles': subtitles,
-            'description': info.get('description'),
-            'timestamp': parse_iso8601(info.get('broadcast_date_time')),
-            'duration': int_or_none(info.get('duration')),
-            'thumbnail': info.get('image'),
-            'is_live': info.get('isLive') is True,
-            'series': info.get('seriesTitle'),
-            'season_number': int_or_none(info.get('seasonNumber')),
-            'episode': info.get('episodeTitle'),
-            'episode_number': int_or_none(info.get('episodeNumber')),
+            **traverse_obj(info, {
+                'title': ('title', {str}),
+                'description': ('description', {str}),
+                'timestamp': (('broadcast_date_time', 'broadcastDateTime'), {parse_iso8601}),
+                'duration': ('duration', {int_or_none}),
+                'thumbnail': ('image', {url_or_none}),
+                'is_live': ('isLive', {bool_or_none}),
+                'series': ('seriesTitle', {str}),
+                'season_number': ('seasonNumber', {int_or_none}),
+                'episode': ('episodeTitle', {str}),
+                'episode_number': ('episodeNumber', {int_or_none}),
+            }, get_all=False),
         }


Test patch: