Repository: yt-dlp/yt-dlp

Issue:
[pinterest] Failed extract some videos
### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2021.11.10.1**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
- [X] I've checked that all provided URLs are alive and playable in a browser
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

_No response_

### Description

Failed extract for https://pin.it/1IDcxxi link

### Verbose log

```shell
yt-dlp -v 'https://pin.it/1IDcxxi'
[debug] Command-line config: ['-v', 'https://pin.it/1IDcxxi']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.11.10.1
[debug] Python version 3.9.8 (CPython 64bit) - macOS-10.15.7-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://pin.it/1IDcxxi
[generic] 1IDcxxi: Requesting header
[redirect] Following redirect to https://www.pinterest.com/pin/1084663891475263837/sent/?invite_code=54ee875476024653944c129295a21dc1&sender=658651651661399279&sfo=1
[debug] [Pinterest] Extracting URL: https://www.pinterest.com/pin/1084663891475263837/sent/?invite_code=54ee875476024653944c129295a21dc1&sender=658651651661399279&sfo=1
[Pinterest] 1084663891475263837: Download Pin JSON metadata
ERROR: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "./lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1311, in wrapper
    return func(self, *args, **kwargs)
  File "./lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1336, in __extract_info
    ie_result = ie.extract(url)
  File "./lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 595, in extract
    ie_result = self._real_extract(url)
  File "./lib/python3.9/site-packages/yt_dlp/extractor/pinterest.py", line 140, in _real_extract
    return self._extract_video(data)
  File "./lib/python3.9/site-packages/yt_dlp/extractor/pinterest.py", line 38, in _extract_video
    for format_id, format_dict in data['videos']['video_list'].items():
TypeError: 'NoneType' object is not subscriptable
```



Discussion:
Another link that fails for some reason, same error message:
https://www.pinterest.fr/pin/616430267754192999/

Here's also a sample link that fails (embedded Vimeo video there):
https://www.pinterest.ca/pin/441282463481903715/

Other pins seem to be working fine... strange
https://www.pinterest.com/pin/56013589110815406/
https://www.pinterest.com/pin/673499319268887982/
https://www.pinterest.ca/pin/63402307245271106/

Resolving patch:
diff --git a/yt_dlp/extractor/pinterest.py b/yt_dlp/extractor/pinterest.py
--- a/yt_dlp/extractor/pinterest.py
+++ b/yt_dlp/extractor/pinterest.py
@@ -1,19 +1,24 @@
 import json
 
 from .common import InfoExtractor
-from ..compat import compat_str
 from ..utils import (
     determine_ext,
     float_or_none,
     int_or_none,
-    try_get,
+    str_or_none,
+    strip_or_none,
+    traverse_obj,
     unified_timestamp,
     url_or_none,
 )
 
 
 class PinterestBaseIE(InfoExtractor):
-    _VALID_URL_BASE = r'https?://(?:[^/]+\.)?pinterest\.(?:com|fr|de|ch|jp|cl|ca|it|co\.uk|nz|ru|com\.au|at|pt|co\.kr|es|com\.mx|dk|ph|th|com\.uy|co|nl|info|kr|ie|vn|com\.vn|ec|mx|in|pe|co\.at|hu|co\.in|co\.nz|id|com\.ec|com\.py|tw|be|uk|com\.bo|com\.pe)'
+    _VALID_URL_BASE = r'''(?x)
+        https?://(?:[^/]+\.)?pinterest\.(?:
+            com|fr|de|ch|jp|cl|ca|it|co\.uk|nz|ru|com\.au|at|pt|co\.kr|es|com\.mx|
+            dk|ph|th|com\.uy|co|nl|info|kr|ie|vn|com\.vn|ec|mx|in|pe|co\.at|hu|
+            co\.in|co\.nz|id|com\.ec|com\.py|tw|be|uk|com\.bo|com\.pe)'''
 
     def _call_api(self, resource, video_id, options):
         return self._download_json(
@@ -24,14 +29,53 @@ def _call_api(self, resource, video_id, options):
 
     def _extract_video(self, data, extract_formats=True):
         video_id = data['id']
+        thumbnails = []
+        images = data.get('images')
+        if isinstance(images, dict):
+            for thumbnail_id, thumbnail in images.items():
+                if not isinstance(thumbnail, dict):
+                    continue
+                thumbnail_url = url_or_none(thumbnail.get('url'))
+                if not thumbnail_url:
+                    continue
+                thumbnails.append({
+                    'url': thumbnail_url,
+                    'width': int_or_none(thumbnail.get('width')),
+                    'height': int_or_none(thumbnail.get('height')),
+                })
 
-        title = (data.get('title') or data.get('grid_title') or video_id).strip()
+        info = {
+            'title': strip_or_none(traverse_obj(data, 'title', 'grid_title', default='')),
+            'description': traverse_obj(data, 'seo_description', 'description'),
+            'timestamp': unified_timestamp(data.get('created_at')),
+            'thumbnails': thumbnails,
+            'uploader': traverse_obj(data, ('closeup_attribution', 'full_name')),
+            'uploader_id': str_or_none(traverse_obj(data, ('closeup_attribution', 'id'))),
+            'repost_count': int_or_none(data.get('repin_count')),
+            'comment_count': int_or_none(data.get('comment_count')),
+            'categories': traverse_obj(data, ('pin_join', 'visual_annotation'), expected_type=list),
+            'tags': traverse_obj(data, 'hashtags', expected_type=list),
+        }
 
         urls = []
         formats = []
         duration = None
-        if extract_formats:
-            for format_id, format_dict in data['videos']['video_list'].items():
+        domain = data.get('domain', '')
+        if domain.lower() != 'uploaded by user' and traverse_obj(data, ('embed', 'src')):
+            if not info['title']:
+                info['title'] = None
+            return {
+                '_type': 'url_transparent',
+                'url': data['embed']['src'],
+                **info,
+            }
+
+        elif extract_formats:
+            video_list = traverse_obj(
+                data, ('videos', 'video_list'),
+                ('story_pin_data', 'pages', ..., 'blocks', ..., 'video', 'video_list'),
+                expected_type=dict, get_all=False, default={})
+            for format_id, format_dict in video_list.items():
                 if not isinstance(format_dict, dict):
                     continue
                 format_url = url_or_none(format_dict.get('url'))
@@ -53,72 +97,79 @@ def _extract_video(self, data, extract_formats=True):
                         'duration': duration,
                     })
 
-        description = data.get('description') or data.get('description_html') or data.get('seo_description')
-        timestamp = unified_timestamp(data.get('created_at'))
-
-        def _u(field):
-            return try_get(data, lambda x: x['closeup_attribution'][field], compat_str)
-
-        uploader = _u('full_name')
-        uploader_id = _u('id')
-
-        repost_count = int_or_none(data.get('repin_count'))
-        comment_count = int_or_none(data.get('comment_count'))
-        categories = try_get(data, lambda x: x['pin_join']['visual_annotation'], list)
-        tags = data.get('hashtags')
-
-        thumbnails = []
-        images = data.get('images')
-        if isinstance(images, dict):
-            for thumbnail_id, thumbnail in images.items():
-                if not isinstance(thumbnail, dict):
-                    continue
-                thumbnail_url = url_or_none(thumbnail.get('url'))
-                if not thumbnail_url:
-                    continue
-                thumbnails.append({
-                    'url': thumbnail_url,
-                    'width': int_or_none(thumbnail.get('width')),
-                    'height': int_or_none(thumbnail.get('height')),
-                })
-
         return {
             'id': video_id,
-            'title': title,
-            'description': description,
-            'duration': duration,
-            'timestamp': timestamp,
-            'thumbnails': thumbnails,
-            'uploader': uploader,
-            'uploader_id': uploader_id,
-            'repost_count': repost_count,
-            'comment_count': comment_count,
-            'categories': categories,
-            'tags': tags,
             'formats': formats,
+            'duration': duration,
+            'webpage_url': f'https://www.pinterest.com/pin/{video_id}/',
             'extractor_key': PinterestIE.ie_key(),
+            'extractor': PinterestIE.IE_NAME,
+            **info,
         }
 
 
 class PinterestIE(PinterestBaseIE):
     _VALID_URL = r'%s/pin/(?P<id>\d+)' % PinterestBaseIE._VALID_URL_BASE
     _TESTS = [{
+        # formats found in data['videos']
         'url': 'https://www.pinterest.com/pin/664281013778109217/',
         'md5': '6550c2af85d6d9f3fe3b88954d1577fc',
         'info_dict': {
             'id': '664281013778109217',
             'ext': 'mp4',
             'title': 'Origami',
-            'description': 'md5:b9d90ddf7848e897882de9e73344f7dd',
+            'description': 'md5:e29801cab7d741ea8c741bc50c8d00ab',
             'duration': 57.7,
             'timestamp': 1593073622,
             'upload_date': '20200625',
-            'uploader': 'Love origami -I am Dafei',
-            'uploader_id': '586523688879454212',
-            'repost_count': 50,
-            'comment_count': 0,
+            'repost_count': int,
+            'comment_count': int,
             'categories': list,
             'tags': list,
+            'thumbnail': r're:^https?://.*\.(?:jpg|png)$',
+        },
+    }, {
+        # formats found in data['story_pin_data']
+        'url': 'https://www.pinterest.com/pin/1084663891475263837/',
+        'md5': '069ac19919ab9e1e13fa60de46290b03',
+        'info_dict': {
+            'id': '1084663891475263837',
+            'ext': 'mp4',
+            'title': 'Gadget, Cool products, Amazon product, technology, Kitchen gadgets',
+            'description': 'md5:d0a4b6ae996ff0c6eed83bc869598d13',
+            'uploader': 'CoolCrazyGadgets',
+            'uploader_id': '1084664028912989237',
+            'upload_date': '20211003',
+            'timestamp': 1633246654.0,
+            'duration': 14.9,
+            'comment_count': int,
+            'repost_count': int,
+            'categories': 'count:9',
+            'tags': list,
+            'thumbnail': r're:^https?://.*\.(?:jpg|png)$',
+        },
+    }, {
+        # vimeo.com embed
+        'url': 'https://www.pinterest.ca/pin/441282463481903715/',
+        'info_dict': {
+            'id': '111691128',
+            'ext': 'mp4',
+            'title': 'Tonite Let\'s All Make Love In London (1967)',
+            'description': 'md5:8190f37b3926807809ec57ec21aa77b2',
+            'uploader': 'Vimeo',
+            'uploader_id': '473792960706651251',
+            'upload_date': '20180120',
+            'timestamp': 1516409040,
+            'duration': 3404,
+            'comment_count': int,
+            'repost_count': int,
+            'categories': 'count:9',
+            'tags': [],
+            'thumbnail': r're:^https?://.*\.(?:jpg|png)$',
+            'uploader_url': 'https://vimeo.com/willardandrade',
+        },
+        'params': {
+            'skip_download': 'm3u8',
         },
     }, {
         'url': 'https://co.pinterest.com/pin/824721750502199491/',


Test patch: