Repository: ytdl-org/youtube-dl

Issue:
Nuvid failing - RegExNotFound error
## Checklist

- [x] I'm reporting a broken site support
- [x] I've verified that I'm running youtube-dl version **2021.06.06**
- [x] I've checked that all provided URLs are alive and playable in a browser
- [x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [x] I've searched the bugtracker for similar issues including closed ones


## Verbose log

```
tracy@tracy-HP17:/media/tracy/Testing/DL$ youtube-dl --config-location ~/.config/youtube-dl/config.sex -v https://www.nuvid.com/video/2070161/teen-girl-with-a-fetish-for-balloons
[debug] System config: []
[debug] User config: []
[debug] Custom config: [u'-o', u'/media/tracy/Testing/DL/sex/%(title)s.%(ext)s', u'--netrc', u'--sleep-interval', u'0', u'--max-sleep-interval', u'5', u'--no-overwrites', u'--download-archive', u'~/.config/youtube-dl/archive-list', u'--yes-playlist', u'--mark-watched', u'--ignore-errors']
[debug] Command-line args: [u'--config-location', u'/home/tracy/.config/youtube-dl/config.sex', u'-v', u'https://www.nuvid.com/video/2070161/teen-girl-with-a-fetish-for-balloons']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 2.7.18 (CPython) - Linux-5.8.0-55-generic-x86_64-with-Ubuntu-20.04-focal
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[Nuvid] 2070161: Downloading video page
[Nuvid] 2070161: Downloading video page for MP4 format
ERROR: Unable to extract 2070161; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/nuvid.py", line 39, in _real_extract
    video_url = self._html_search_regex(html5_video_re, webpage, video_id)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1021, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1012, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract 2070161; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
```


## Description

The Nuvid extractor seems to be failing - apparently something has changed on the site, as the error coming back is RegExNotFound. I tested against 52 URLs for the site, an example is:

https://www.nuvid.com/video/2070161/teen-girl-with-a-fetish-for-balloons

I have verified that these URLs (including the sample listed above) work correctly in Chrome Version 91.0.4472.101 (Official Build) (64-bit), running on Ubuntu 20.04

Other open bug reports on this issue exist, specifically:

https://github.com/ytdl-org/youtube-dl/issues/27578
https://github.com/ytdl-org/youtube-dl/issues/15355

[Nuvid] Unable to extract 1310741
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2018.01.18**

### Before submitting an *issue* make sure you have:
- [x] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones

### What is the purpose of your *issue*?
- [x] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

http://m.nuvid.com/video/1310741/

```
ERROR: Unable to extract 1310741; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line 784, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 438, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/nuvid.py", line 39, in _real_extract
    video_url = self._html_search_regex(html5_video_re, webpage, video_id)
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 803, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 794, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract 1310741; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
```


Discussion:

Still the case in 2018.08.04.
Also an issue in 2019.07.16.  The associated PR resolves this nicely.

Resolving patch:
diff --git a/youtube_dl/extractor/nuvid.py b/youtube_dl/extractor/nuvid.py
--- a/youtube_dl/extractor/nuvid.py
+++ b/youtube_dl/extractor/nuvid.py
@@ -1,71 +1,113 @@
+# coding: utf-8
 from __future__ import unicode_literals
 
-import re
-
 from .common import InfoExtractor
+from ..compat import compat_str
 from ..utils import (
     parse_duration,
+    int_or_none,
+    try_get,
+    url_or_none,
 )
 
+import re
+
 
 class NuvidIE(InfoExtractor):
     _VALID_URL = r'https?://(?:www|m)\.nuvid\.com/video/(?P<id>[0-9]+)'
-    _TEST = {
-        'url': 'http://m.nuvid.com/video/1310741/',
-        'md5': 'eab207b7ac4fccfb4e23c86201f11277',
+    _TESTS = [{
+        'url': 'https://www.nuvid.com/video/6513023/italian-babe',
+        'md5': '772d2f8288f3d3c5c45f7a41761c7844',
+        'info_dict': {
+            'id': '6513023',
+            'ext': 'mp4',
+            'title': 'italian babe',
+            'format_id': '360p',
+            'duration': 321.0,
+            'age_limit': 18,
+            'thumbnail': r're:https?://.+\.jpg',
+            'thumbnails': list,
+        }
+    }, {
+        'url': 'https://m.nuvid.com/video/6523263',
+        'md5': 'ebd22ce8e47e1d9a4d0756a15c67da52',
         'info_dict': {
-            'id': '1310741',
+            'id': '6523263',
             'ext': 'mp4',
-            'title': 'Horny babes show their awesome bodeis and',
-            'duration': 129,
+            'title': 'Slut brunette college student anal dorm',
+            'format_id': '720p',
+            'duration': 421.0,
             'age_limit': 18,
+            'thumbnail': r're:https?://.+\.jpg',
+            'thumbnails': list,
         }
-    }
+    }, {
+        'url': 'http://m.nuvid.com/video/6415801/',
+        'md5': '638d5ececb138d5753593f751ae3f697',
+        'info_dict': {
+            'id': '6415801',
+            'ext': 'mp4',
+            'title': 'My best friend wanted to fuck my wife for a long time',
+            'format_id': '720p',
+            'duration': 1882,
+            'age_limit': 18,
+            'thumbnail': r're:https?://.+\.jpg',
+            'thumbnails': list,
+        }
+    }]
 
     def _real_extract(self, url):
         video_id = self._match_id(url)
 
-        page_url = 'http://m.nuvid.com/video/%s' % video_id
+        qualities = {
+            'lq': '360p',
+            'hq': '720p',
+        }
+
+        json_url = 'https://www.nuvid.com/player_config_json/?vid={video_id}&aid=0&domain_id=0&embed=0&check_speed=0'.format(**locals())
+        video_data = self._download_json(
+            json_url, video_id, headers={
+                'Accept': 'application/json, text/javascript, */*; q = 0.01',
+                'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8',
+            }) or {}
+
+        # nice to have, not required
         webpage = self._download_webpage(
-            page_url, video_id, 'Downloading video page')
-        # When dwnld_speed exists and has a value larger than the MP4 file's
-        # bitrate, Nuvid returns the MP4 URL
-        # It's unit is 100bytes/millisecond, see mobile-nuvid-min.js for the algorithm
-        self._set_cookie('nuvid.com', 'dwnld_speed', '10.0')
-        mp4_webpage = self._download_webpage(
-            page_url, video_id, 'Downloading video page for MP4 format')
+            'http://m.nuvid.com/video/%s' % (video_id, ),
+            video_id, 'Downloading video page', fatal=False) or ''
+
+        title = (
+            try_get(video_data, lambda x: x['title'], compat_str)
+            or self._html_search_regex(
+                (r'''<span\s[^>]*?\btitle\s*=\s*(?P<q>"|'|\b)(?P<title>[^"]+)(?P=q)\s*>''',
+                 r'''<div\s[^>]*?\bclass\s*=\s*(?P<q>"|'|\b)thumb-holder video(?P=q)>\s*<h5\b[^>]*>(?P<title>[^<]+)</h5''',
+                 r'''<span\s[^>]*?\bclass\s*=\s*(?P<q>"|'|\b)title_thumb(?P=q)>(?P<title>[^<]+)</span'''),
+                webpage, 'title', group='title')).strip()
 
-        html5_video_re = r'(?s)<(?:video|audio)[^<]*(?:>.*?<source[^>]*)?\s+src=["\'](.*?)["\']',
-        video_url = self._html_search_regex(html5_video_re, webpage, video_id)
-        mp4_video_url = self._html_search_regex(html5_video_re, mp4_webpage, video_id)
         formats = [{
-            'url': video_url,
-        }]
-        if mp4_video_url != video_url:
-            formats.append({
-                'url': mp4_video_url,
-            })
+            'url': source,
+            'format_id': qualities.get(quality),
+            'height': int_or_none(qualities.get(quality)[:-1]),
+        } for quality, source in video_data.get('files').items() if source]
+
+        self._check_formats(formats, video_id)
+        self._sort_formats(formats)
 
-        title = self._html_search_regex(
-            [r'<span title="([^"]+)">',
-             r'<div class="thumb-holder video">\s*<h5[^>]*>([^<]+)</h5>',
-             r'<span[^>]+class="title_thumb">([^<]+)</span>'], webpage, 'title').strip()
+        duration = parse_duration(video_data.get('duration') or video_data.get('duration_format'))
         thumbnails = [
-            {
-                'url': thumb_url,
-            } for thumb_url in re.findall(r'<img src="([^"]+)" alt="" />', webpage)
+            {'url': thumb_url, }
+            for thumb_url in (
+                url_or_none(src) for src in re.findall(
+                    r'<div\s+class\s*=\s*"video-tmb-wrap"\s*>\s*<img\s+src\s*=\s*"([^"]+)"\s*/>',
+                    webpage))
         ]
-        thumbnail = thumbnails[0]['url'] if thumbnails else None
-        duration = parse_duration(self._html_search_regex(
-            [r'<i class="fa fa-clock-o"></i>\s*(\d{2}:\d{2})',
-             r'<span[^>]+class="view_time">([^<]+)</span>'], webpage, 'duration', fatal=False))
 
         return {
             'id': video_id,
+            'formats': formats,
             'title': title,
+            'thumbnail': url_or_none(video_data.get('poster')),
             'thumbnails': thumbnails,
-            'thumbnail': thumbnail,
             'duration': duration,
             'age_limit': 18,
-            'formats': formats,
         }


Test patch: