Repository: ytdl-org/youtube-dl

Issue:
Apple/google podcasts --get-thumbnail
<!--

######################################################################
  WARNING!
  IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################

-->


## Checklist

<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2021.05.16. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->

- [x] I'm reporting a site feature request
- [x] I've verified that I'm running youtube-dl version **2021.05.16**
- [x] I've searched the bugtracker for similar site feature requests including closed ones


## Description

<!--
Provide an explanation of your site feature request in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->

Just wondering if there's a way to make --get-thumbnail work for Apple and Google podcasts? This would be a great addition to the existing extractors that are working just great. 



Discussion:
Ok, since this was marked as incomplete, maybe my explanations of a feature request were unclear. 
So here's a test link:
https://podcasts.apple.com/ua/podcast/the-jordan-b-peterson-podcast/id1184022695?i=1000524122414

Webpage contains podcast thumbnail, but ytdl JSON output has no trace of it. I was wondering if it's possible to also get a link for the podcast's thumbnail via --get-thumbnail command or within -J command. 

Used inspect element on the image and looks like it's possible to make it work. Here's a link to one of the thumbnails:
https://is5-ssl.mzstatic.com/image/thumb/Podcasts125/v4/14/31/c2/1431c2f2-1be3-c717-cdcb-02e660c0fc55/mza_4126462245463073008.jpeg/313x0w.webp

Here's current output ytdl output for --get-thumbnail and the link above:

youtube-dl --get-thumbnail -v https://podcasts.apple.com/ua/podcast/the-jordan-b-peterson-podcast/id1184022695?i=1000524122414
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--get-thumbnail', u'-v', u'https://podcasts.apple.com/ua/podcast/the-jordan-b-peterson-podcast/id1184022695?i=1000524122414']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 2.7.16 (CPython) - Darwin-20.5.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[debug] Default format spec: bestvideo+bestaudio/best


Here's the JSON output:

youtube-dl -J -v https://podcasts.apple.com/ua/podcast/the-jordan-b-peterson-podcast/id1184022695?i=1000524122414
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-J', u'-v', u'https://podcasts.apple.com/ua/podcast/the-jordan-b-peterson-podcast/id1184022695?i=1000524122414']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 2.7.16 (CPython) - Darwin-20.5.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[debug] Default format spec: bestvideo+bestaudio/best
{"extractor": "ApplePodcasts", "protocol": "https", "upload_date": "20210603", "series": "The Jordan B. Peterson Podcast", "duration": 8333, "playlist_index": null, "playlist": null, "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.89 Safari/537.36"}, "id": "1000524122414", "timestamp": 1622739420, "webpage_url_basename": "id1184022695", "display_id": "1000524122414", "description": "On this episode of the Jordan Peterson Podcast, Jordan is joined by Rex Murphy. Rex is a Canadian commentator and author who deals primarily with Canadian political and social matters. He is best known for working on and for CBC Here and Now, CBC Radio 1\u2019s Cross Country Checkup, writing for The Globe and Mail, and writing for The National Post. He is a well-recognized and loved figure.\n\nRex Murphy and I sit down to discuss a variety of topics including his impressive career, Canadian politics, western culture, the woke culture wars, changes in universities, the crumbling study of the humanities, New Finland, Toronto, and more. Find more Rex Murphy by searching his name for articles and in his book\nLearn more about your ad choices. Visit megaphone.fm/adchoices", "format": "0 - unknown", "requested_subtitles": null, "format_id": "0", "url": "https://traffic.megaphone.fm/ADV6683168388.mp3?updated=1622745430", "extractor_key": "ApplePodcasts", "title": "The Education of a Journalist | Rex Murphy", "ext": "mp3", "webpage_url": "https://podcasts.apple.com/ua/podcast/the-jordan-b-peterson-podcast/id1184022695?i=1000524122414"}




Resolving patch:
diff --git a/youtube_dl/extractor/applepodcasts.py b/youtube_dl/extractor/applepodcasts.py
--- a/youtube_dl/extractor/applepodcasts.py
+++ b/youtube_dl/extractor/applepodcasts.py
@@ -3,8 +3,11 @@
 
 from .common import InfoExtractor
 from ..utils import (
+    clean_html,
     clean_podcast_url,
+    get_element_by_class,
     int_or_none,
+    parse_codecs,
     parse_iso8601,
     try_get,
 )
@@ -14,16 +17,17 @@ class ApplePodcastsIE(InfoExtractor):
     _VALID_URL = r'https?://podcasts\.apple\.com/(?:[^/]+/)?podcast(?:/[^/]+){1,2}.*?\bi=(?P<id>\d+)'
     _TESTS = [{
         'url': 'https://podcasts.apple.com/us/podcast/207-whitney-webb-returns/id1135137367?i=1000482637777',
-        'md5': 'df02e6acb11c10e844946a39e7222b08',
+        'md5': '41dc31cd650143e530d9423b6b5a344f',
         'info_dict': {
             'id': '1000482637777',
             'ext': 'mp3',
             'title': '207 - Whitney Webb Returns',
-            'description': 'md5:13a73bade02d2e43737751e3987e1399',
+            'description': 'md5:75ef4316031df7b41ced4e7b987f79c6',
             'upload_date': '20200705',
-            'timestamp': 1593921600,
-            'duration': 6425,
+            'timestamp': 1593932400,
+            'duration': 6454,
             'series': 'The Tim Dillon Show',
+            'thumbnail': 're:.+[.](png|jpe?g|webp)',
         }
     }, {
         'url': 'https://podcasts.apple.com/podcast/207-whitney-webb-returns/id1135137367?i=1000482637777',
@@ -39,19 +43,40 @@ class ApplePodcastsIE(InfoExtractor):
     def _real_extract(self, url):
         episode_id = self._match_id(url)
         webpage = self._download_webpage(url, episode_id)
-        ember_data = self._parse_json(self._search_regex(
-            r'id="shoebox-ember-data-store"[^>]*>\s*({.+?})\s*<',
-            webpage, 'ember data'), episode_id)
-        ember_data = ember_data.get(episode_id) or ember_data
-        episode = ember_data['data']['attributes']
+        episode_data = {}
+        ember_data = {}
+        # new page type 2021-11
+        amp_data = self._parse_json(self._search_regex(
+            r'(?s)id="shoebox-media-api-cache-amp-podcasts"[^>]*>\s*({.+?})\s*<',
+            webpage, 'AMP data', default='{}'), episode_id, fatal=False) or {}
+        amp_data = try_get(amp_data,
+                           lambda a: self._parse_json(
+                               next(a[x] for x in iter(a) if episode_id in x),
+                               episode_id),
+                           dict) or {}
+        amp_data = amp_data.get('d') or []
+        episode_data = try_get(
+            amp_data,
+            lambda a: next(x for x in a
+                           if x['type'] == 'podcast-episodes' and x['id'] == episode_id),
+            dict)
+        if not episode_data:
+            # try pre 2021-11 page type: TODO: consider deleting if no longer used
+            ember_data = self._parse_json(self._search_regex(
+                r'(?s)id="shoebox-ember-data-store"[^>]*>\s*({.+?})\s*<',
+                webpage, 'ember data'), episode_id) or {}
+            ember_data = ember_data.get(episode_id) or ember_data
+            episode_data = try_get(ember_data, lambda x: x['data'], dict)
+        episode = episode_data['attributes']
         description = episode.get('description') or {}
 
         series = None
-        for inc in (ember_data.get('included') or []):
+        for inc in (amp_data or ember_data.get('included') or []):
             if inc.get('type') == 'media/podcast':
                 series = try_get(inc, lambda x: x['attributes']['name'])
+        series = series or clean_html(get_element_by_class('podcast-header__identity', webpage))
 
-        return {
+        info = [{
             'id': episode_id,
             'title': episode['name'],
             'url': clean_podcast_url(episode['assetUrl']),
@@ -59,4 +84,10 @@ def _real_extract(self, url):
             'timestamp': parse_iso8601(episode.get('releaseDateTime')),
             'duration': int_or_none(episode.get('durationInMilliseconds'), 1000),
             'series': series,
-        }
+            'thumbnail': self._og_search_thumbnail(webpage),
+        }]
+        self._sort_formats(info)
+        info = info[0]
+        codecs = parse_codecs(info.get('ext', 'mp3'))
+        info.update(codecs)
+        return info


Test patch: