Repository: yt-dlp/yt-dlp
Pull request #695 (2021-08-14 06:08:21)

[bilibili] Category Extractor
## Please follow the guide below

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *pull request* (like that [x])
- Use *Preview* tab to see how your *pull request* will actually look like

---

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [adding new extractor tutorial](https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site) and [youtube-dl coding conventions](https://github.com/ytdl-org/youtube-dl#youtube-dl-coding-conventions) sections
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)

### In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [ ] Bug fix
- [ ] Improvement
- [x] New extractor
- [ ] New feature

---

### Description of your *pull request* and other information

This adds an extractor to match URLs like https://www.bilibili.com/v/kichiku/mad , predefined categories on Bilibili, and download all videos.



Patch:
diff --git a/README.md b/README.md
index dcd870dec..bd7d31c13 100644
--- a/README.md
+++ b/README.md
@@ -64,7 +64,7 @@ # NEW FEATURES
 
 * **[SponSkrub Integration](#sponskrub-sponsorblock-options)**: You can use [SponSkrub](https://github.com/yt-dlp/SponSkrub) to mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
 
-* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))
+* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples))
 
 * **Merged with youtube-dl [commit/379f52a](https://github.com/ytdl-org/youtube-dl/commit/379f52a4954013767219d25099cce9e0f9401961)**: (v2021.06.06) You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
 
@@ -90,7 +90,7 @@ # NEW FEATURES
 
 * **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, SlingTV MSO, Cablevision MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live, douyin, pornflip, ParamountPlusSeries, ScienceChannel, Utreon, OpenRec, BandcampMusic, blackboardcollaborate, eroprofile albums, mirrativ
 
-* **Fixed/improved extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki, Hotstar, vidio, vimeo, mediaset, Mxplayer, nbcolympics, ParamountPlus, Newgrounds, 
+* **Fixed/improved extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki, Hotstar, vidio, vimeo, mediaset, Mxplayer, nbcolympics, ParamountPlus, Newgrounds
 
 * **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [commit/be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
 
@@ -213,7 +213,7 @@ ### COMPILE
 You can also build the executable without any version info or metadata by using:
 
     pyinstaller.exe yt_dlp\__main__.py --onefile --name yt-dlp
-    
+
 Note that pyinstaller [does not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment
 
 **For Unix**:
@@ -273,7 +273,7 @@ ## General Options:
     --no-mark-watched                Do not mark videos watched (default)
     --no-colors                      Do not emit color codes in output
     --compat-options OPTS            Options that can help keep compatibility
-                                     with youtube-dl and youtube-dlc
+                                     with youtube-dl or youtube-dlc
                                      configurations by reverting some of the
                                      changes made in yt-dlp. See "Differences in
                                      default behavior" for details
@@ -1405,7 +1405,7 @@ # EXTRACTOR ARGUMENTS
     * `include_live_dash`: Include live dash formats (These formats don't download properly)
     * `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side).
     * `max_comments`: Maximum amount of comments to download (default all).
-    * `max_comment_depth`: Maximum depth for nested comments. YouTube supports depths 1 or 2 (default). 
+    * `max_comment_depth`: Maximum depth for nested comments. YouTube supports depths 1 or 2 (default).
 
 * **funimation**
     * `language`: Languages to extract. Eg: `funimation:language=english,japanese`
@@ -1439,6 +1439,10 @@ #### Redundant options
     -e, --get-title                  --print title
     -g, --get-url                    --print urls
     -j, --dump-json                  --print "%()j"
+    --match-title REGEX              --match-filter "title ~= (?i)REGEX"
+    --reject-title REGEX             --match-filter "title !~= (?i)REGEX"
+    --min-views COUNT                --match-filter "view_count >=? COUNT"
+    --max-views COUNT                --match-filter "view_count <=? COUNT"
 
 
 #### Not recommended
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index 5937e85bd..f180483a3 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -1117,12 +1117,15 @@ def check_filter():
             if age_restricted(info_dict.get('age_limit'), self.params.get('age_limit')):
                 return 'Skipping "%s" because it is age restricted' % video_title
 
-            if not incomplete:
-                match_filter = self.params.get('match_filter')
-                if match_filter is not None:
-                    ret = match_filter(info_dict)
-                    if ret is not None:
-                        return ret
+            match_filter = self.params.get('match_filter')
+            if match_filter is not None:
+                try:
+                    ret = match_filter(info_dict, incomplete=incomplete)
+                except TypeError:
+                    # For backward compatibility
+                    ret = None if incomplete else match_filter(info_dict)
+                if ret is not None:
+                    return ret
             return None
 
         if self.in_download_archive(info_dict):
@@ -1251,7 +1254,7 @@ def add_default_extra_info(self, ie_result, ie, url):
                 'extractor_key': ie.ie_key(),
             })
 
-    def process_ie_result(self, ie_result, download=True, extra_info={}):
+    def process_ie_result(self, ie_result, download=True, extra_info=None):
         """
         Take the result of the ie(may be modified) and resolve all unresolved
         references (URLs, playlist items).
@@ -1259,6 +1262,8 @@ def process_ie_result(self, ie_result, download=True, extra_info={}):
         It will also download the videos if 'download'.
         Returns the resolved ie_result.
         """
+        if extra_info is None:
+            extra_info = {}
         result_type = ie_result.get('_type', 'video')
 
         if result_type in ('url', 'url_transparent'):
@@ -1449,7 +1454,7 @@ def get_entry(i):
 
         # Save playlist_index before re-ordering
         entries = [
-            ((playlistitems[i - 1] if playlistitems else i), entry)
+            ((playlistitems[i - 1] if playlistitems else i + playliststart - 1), entry)
             for i, entry in enumerate(entries, 1)
             if entry is not None]
         n_entries = len(entries)
@@ -1514,7 +1519,7 @@ def get_entry(i):
         max_failures = self.params.get('skip_playlist_after_errors') or float('inf')
         for i, entry_tuple in enumerate(entries, 1):
             playlist_index, entry = entry_tuple
-            if 'playlist_index' in self.params.get('compat_options', []):
+            if 'playlist-index' in self.params.get('compat_options', []):
                 playlist_index = playlistitems[i - 1] if playlistitems else i
             self.to_screen('[download] Downloading video %s of %s' % (i, n_entries))
             # This __x_forwarded_for_ip thing is a bit ugly but requires
@@ -2231,7 +2236,7 @@ def is_wellformed(f):
         if self.params.get('list_thumbnails'):
             self.list_thumbnails(info_dict)
         if self.params.get('listformats'):
-            if not info_dict.get('formats'):
+            if not info_dict.get('formats') and not info_dict.get('url'):
                 raise ExtractorError('No video formats found', expected=True)
             self.list_formats(info_dict)
         if self.params.get('listsubtitles'):
@@ -2663,7 +2668,6 @@ def existing_file(*filepaths):
                             os.remove(encodeFilename(file))
                         return None
 
-                    self.report_file_already_downloaded(existing_files[0])
                     info_dict['ext'] = os.path.splitext(existing_files[0])[1][1:]
                     return existing_files[0]
 
@@ -2718,7 +2722,7 @@ def correct_ext(filename, ext=new_ext):
                         info_dict['protocol'] = _protocols.pop()
                     directly_mergable = FFmpegFD.can_merge_formats(info_dict)
                     if dl_filename is not None:
-                        pass
+                        self.report_file_already_downloaded(dl_filename)
                     elif (directly_mergable and get_suitable_downloader(
                             info_dict, self.params, to_stdout=(temp_filename == '-')) == FFmpegFD):
                         info_dict['url'] = '\n'.join(f['url'] for f in requested_formats)
@@ -2770,9 +2774,13 @@ def correct_ext(filename, ext=new_ext):
                 else:
                     # Just a single file
                     dl_filename = existing_file(full_filename, temp_filename)
-                    if dl_filename is None:
+                    if dl_filename is None or dl_filename == temp_filename:
+                        # dl_filename == temp_filename could mean that the file was partially downloaded with --no-part.
+                        # So we should try to resume the download
                         success, real_download = self.dl(temp_filename, info_dict)
                         info_dict['__real_download'] = real_download
+                    else:
+                        self.report_file_already_downloaded(dl_filename)
 
                 dl_filename = dl_filename or temp_filename
                 info_dict['__finaldir'] = os.path.dirname(os.path.abspath(encodeFilename(full_filename)))
@@ -2870,13 +2878,13 @@ def download(self, url_list):
             except UnavailableVideoError:
                 self.report_error('unable to download video')
             except MaxDownloadsReached:
-                self.to_screen('[info] Maximum number of downloaded files reached')
+                self.to_screen('[info] Maximum number of downloads reached')
                 raise
             except ExistingVideoReached:
-                self.to_screen('[info] Encountered a file that is already in the archive, stopping due to --break-on-existing')
+                self.to_screen('[info] Encountered a video that is already in the archive, stopping due to --break-on-existing')
                 raise
             except RejectedVideoReached:
-                self.to_screen('[info] Encountered a file that did not match filter, stopping due to --break-on-reject')
+                self.to_screen('[info] Encountered a video that did not match filter, stopping due to --break-on-reject')
                 raise
             else:
                 if self.params.get('dump_single_json', False):
@@ -2905,6 +2913,8 @@ def download_with_info_file(self, info_filename):
     @staticmethod
     def sanitize_info(info_dict, remove_private_keys=False):
         ''' Sanitize the infodict for converting to json '''
+        if info_dict is None:
+            return info_dict
         info_dict.setdefault('epoch', int(time.time()))
         remove_keys = {'__original_infodict'}  # Always remove this since this may contain a copy of the entire dict
         keep_keys = ['_type'],  # Always keep this to facilitate load-info-json
diff --git a/yt_dlp/__init__.py b/yt_dlp/__init__.py
index d6479b3ff..36b2eddba 100644
--- a/yt_dlp/__init__.py
+++ b/yt_dlp/__init__.py
@@ -257,35 +257,7 @@ def parse_retries(retries, name=''):
     else:
         date = DateRange(opts.dateafter, opts.datebefore)
 
-    def parse_compat_opts():
-        parsed_compat_opts, compat_opts = set(), opts.compat_opts[::-1]
-        while compat_opts:
-            actual_opt = opt = compat_opts.pop().lower()
-            if opt == 'youtube-dl':
-                compat_opts.extend(['-multistreams', 'all'])
-            elif opt == 'youtube-dlc':
-                compat_opts.extend(['-no-youtube-channel-redirect', '-no-live-chat', 'all'])
-            elif opt == 'all':
-                parsed_compat_opts.update(all_compat_opts)
-            elif opt == '-all':
-                parsed_compat_opts = set()
-            else:
-                if opt[0] == '-':
-                    opt = opt[1:]
-                    parsed_compat_opts.discard(opt)
-                else:
-                    parsed_compat_opts.update([opt])
-                if opt not in all_compat_opts:
-                    parser.error('Invalid compatibility option %s' % actual_opt)
-        return parsed_compat_opts
-
-    all_compat_opts = [
-        'filename', 'format-sort', 'abort-on-error', 'format-spec', 'no-playlist-metafiles',
-        'multistreams', 'no-live-chat', 'playlist-index', 'list-formats', 'no-direct-merge',
-        'no-youtube-channel-redirect', 'no-youtube-unavailable-videos', 'no-attach-info-json',
-        'embed-thumbnail-atomicparsley', 'seperate-video-versions', 'no-clean-infojson', 'no-keep-subs',
-    ]
-    compat_opts = parse_compat_opts()
+    compat_opts = opts.compat_opts
 
     def _unused_compat_opt(name):
         if name not in compat_opts:
diff --git a/yt_dlp/downloader/common.py b/yt_dlp/downloader/common.py
index 3846a7ee5..f5f6393a6 100644
--- a/yt_dlp/downloader/common.py
+++ b/yt_dlp/downloader/common.py
@@ -204,9 +204,12 @@ def ytdl_filename(self, filename):
         return filename + '.ytdl'
 
     def try_rename(self, old_filename, new_filename):
+        if old_filename == new_filename:
+            return
         try:
-            if old_filename == new_filename:
-                return
+            if self.params.get('overwrites', False):
+                if os.path.isfile(encodeFilename(new_filename)):
+                    os.remove(encodeFilename(new_filename))
             os.rename(encodeFilename(old_filename), encodeFilename(new_filename))
         except (IOError, OSError) as err:
             self.report_error('unable to rename file: %s' % error_to_compat_str(err))
diff --git a/yt_dlp/downloader/hls.py b/yt_dlp/downloader/hls.py
index 9cfc191cb..779658b70 100644
--- a/yt_dlp/downloader/hls.py
+++ b/yt_dlp/downloader/hls.py
@@ -254,8 +254,14 @@ def is_ad_fragment_end(s):
             def pack_fragment(frag_content, frag_index):
                 output = io.StringIO()
                 adjust = 0
+                overflow = False
+                mpegts_last = None
                 for block in webvtt.parse_fragment(frag_content):
                     if isinstance(block, webvtt.CueBlock):
+                        extra_state['webvtt_mpegts_last'] = mpegts_last
+                        if overflow:
+                            extra_state['webvtt_mpegts_adjust'] += 1
+                            overflow = False
                         block.start += adjust
                         block.end += adjust
 
@@ -296,9 +302,9 @@ def pack_fragment(frag_content, frag_index):
                         extra_state.setdefault('webvtt_mpegts_adjust', 0)
                         block.mpegts += extra_state['webvtt_mpegts_adjust'] << 33
                         if block.mpegts < extra_state.get('webvtt_mpegts_last', 0):
-                            extra_state['webvtt_mpegts_adjust'] += 1
+                            overflow = True
                             block.mpegts += 1 << 33
-                        extra_state['webvtt_mpegts_last'] = block.mpegts
+                        mpegts_last = block.mpegts
 
                         if frag_index == 1:
                             extra_state['webvtt_mpegts'] = block.mpegts or 0
diff --git a/yt_dlp/extractor/bilibili.py b/yt_dlp/extractor/bilibili.py
index 92c988274..743fc501e 100644
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -4,13 +4,16 @@
 import hashlib
 import itertools
 import json
+import functools
 import re
+import math
 
 from .common import InfoExtractor, SearchInfoExtractor
 from ..compat import (
     compat_str,
     compat_parse_qs,
     compat_urlparse,
+    compat_urllib_parse_urlparse
 )
 from ..utils import (
     ExtractorError,
@@ -24,6 +27,7 @@
     unified_timestamp,
     unsmuggle_url,
     urlencode_postdata,
+    OnDemandPagedList
 )
 
 
@@ -535,6 +539,75 @@ def _real_extract(self, url):
         return self.playlist_result(self._entries(list_id), list_id)
 
 
+class BilibiliCategoryIE(InfoExtractor):
+    IE_NAME = 'Bilibili category extractor'
+    _MAX_RESULTS = 1000000
+    _VALID_URL = r'https?://www\.bilibili\.com/v/[a-zA-Z]+\/[a-zA-Z]+'
+    _TESTS = [{
+        'url': 'https://www.bilibili.com/v/kichiku/mad',
+        'info_dict': {
+            'id': 'kichiku: mad',
+            'title': 'kichiku: mad'
+        },
+        'playlist_mincount': 45,
+        'params': {
+            'playlistend': 45
+        }
+    }]
+
+    def _fetch_page(self, api_url, num_pages, query, page_num):
+        parsed_json = self._download_json(
+            api_url, query, query={'Search_key': query, 'pn': page_num},
+            note='Extracting results from page %s of %s' % (page_num, num_pages))
+
+        video_list = try_get(parsed_json, lambda x: x['data']['archives'], list)
+        if not video_list:
+            raise ExtractorError('Failed to retrieve video list for page %d' % page_num)
+
+        for video in video_list:
+            yield self.url_result(
+                'https://www.bilibili.com/video/%s' % video['bvid'], 'BiliBili', video['bvid'])
+
+    def _entries(self, category, subcategory, query):
+        # map of categories : subcategories : RIDs
+        rid_map = {
+            'kichiku': {
+                'mad': 26,
+                'manual_vocaloid': 126,
+                'guide': 22,
+                'theatre': 216,
+                'course': 127
+            },
+        }
+
+        if category not in rid_map:
+            raise ExtractorError('The supplied category, %s, is not supported. List of supported categories: %s' % (category, list(rid_map.keys())))
+
+        if subcategory not in rid_map[category]:
+            raise ExtractorError('The subcategory, %s, isn\'t supported for this category. Supported subcategories: %s' % (subcategory, list(rid_map[category].keys())))
+
+        rid_value = rid_map[category][subcategory]
+
+        api_url = 'https://api.bilibili.com/x/web-interface/newlist?rid=%d&type=1&ps=20&jsonp=jsonp' % rid_value
+        page_json = self._download_json(api_url, query, query={'Search_key': query, 'pn': '1'})
+        page_data = try_get(page_json, lambda x: x['data']['page'], dict)
+        count, size = int_or_none(page_data.get('count')), int_or_none(page_data.get('size'))
+        if count is None or not size:
+            raise ExtractorError('Failed to calculate either page count or size')
+
+        num_pages = math.ceil(count / size)
+
+        return OnDemandPagedList(functools.partial(
+            self._fetch_page, api_url, num_pages, query), size)
+
+    def _real_extract(self, url):
+        u = compat_urllib_parse_urlparse(url)
+        category, subcategory = u.path.split('/')[2:4]
+        query = '%s: %s' % (category, subcategory)
+
+        return self.playlist_result(self._entries(category, subcategory, query), query, query)
+
+
 class BiliBiliSearchIE(SearchInfoExtractor):
     IE_DESC = 'Bilibili video search, "bilisearch" keyword'
     _MAX_RESULTS = 100000
diff --git a/yt_dlp/extractor/eroprofile.py b/yt_dlp/extractor/eroprofile.py
index 54ed9a49e..a8396f1d3 100644
--- a/yt_dlp/extractor/eroprofile.py
+++ b/yt_dlp/extractor/eroprofile.py
@@ -114,11 +114,13 @@ def _entries(self, playlist_id, first_page):
         yield from self._extract_from_page(first_page)
 
         page_urls = re.findall(rf'href=".*?(/m/videos/album/{playlist_id}\?pnum=(\d+))"', first_page)
+        max_page = max(int(n) for _, n in page_urls)
 
-        for url, n in page_urls[1:]:
-            yield from self._extract_from_page(self._download_webpage(
-                f'https://www.eroprofile.com{url}',
-                playlist_id, note=f'Downloading playlist page {int(n) - 1}'))
+        for n in range(2, max_page + 1):
+            url = f'https://www.eroprofile.com/m/videos/album/{playlist_id}?pnum={n}'
+            yield from self._extract_from_page(
+                self._download_webpage(url, playlist_id,
+                                       note=f'Downloading playlist page {int(n) - 1}'))
 
     def _real_extract(self, url):
         playlist_id = self._match_id(url)
diff --git a/yt_dlp/extractor/extractors.py b/yt_dlp/extractor/extractors.py
index 5b15bb8e7..077fb7503 100644
--- a/yt_dlp/extractor/extractors.py
+++ b/yt_dlp/extractor/extractors.py
@@ -140,6 +140,7 @@
 from .bilibili import (
     BiliBiliIE,
     BiliBiliSearchIE,
+    BilibiliCategoryIE,
     BiliBiliBangumiIE,
     BilibiliAudioIE,
     BilibiliAudioAlbumIE,
@@ -420,6 +421,7 @@
     FC2EmbedIE,
 )
 from .fczenit import FczenitIE
+from .filmmodu import FilmmoduIE
 from .filmon import (
     FilmOnIE,
     FilmOnChannelIE,
@@ -537,6 +539,7 @@
 from .hungama import (
     HungamaIE,
     HungamaSongIE,
+    HungamaAlbumPlaylistIE,
 )
 from .hypem import HypemIE
 from .ign import (
@@ -1043,6 +1046,7 @@
 from .pornhub import (
     PornHubIE,
     PornHubUserIE,
+    PornHubPlaylistIE,
     PornHubPagedVideoListIE,
     PornHubUserVideosUploadIE,
 )
@@ -1604,6 +1608,10 @@
 from .vodpl import VODPlIE
 from .vodplatform import VODPlatformIE
 from .voicerepublic import VoiceRepublicIE
+from .voicy import (
+    VoicyIE,
+    VoicyChannelIE,
+)
 from .voot import (
     VootIE,
     VootSeriesIE,
diff --git a/yt_dlp/extractor/filmmodu.py b/yt_dlp/extractor/filmmodu.py
new file mode 100644
index 000000000..2746876d5
--- /dev/null
+++ b/yt_dlp/extractor/filmmodu.py
@@ -0,0 +1,74 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import int_or_none
+
+
+class FilmmoduIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www.)?filmmodu.org/(?P<id>[^/]+-(?:turkce-dublaj-izle|altyazili-izle))'
+    _TESTS = [{
+        'url': 'https://www.filmmodu.org/f9-altyazili-izle',
+        'md5': 'aeefd955c2a508a5bdaa3bcec8eeb0d4',
+        'info_dict': {
+            'id': '10804',
+            'ext': 'mp4',
+            'title': 'F9',
+            'description': 'md5:2713f584a4d65afa2611e2948d0b953c',
+            'subtitles': {
+                'tr': [{
+                    'ext': 'vtt',
+                }],
+            },
+            'thumbnail': r're:https://s[0-9]+.filmmodu.org/uploads/movie/cover/10804/xXHZeb1yhJvnSHPzZDqee0zfMb6.jpg',
+        },
+    }, {
+        'url': 'https://www.filmmodu.org/the-godfather-turkce-dublaj-izle',
+        'md5': '109f2fcb9c941330eed133971c035c00',
+        'info_dict': {
+            'id': '3646',
+            'ext': 'mp4',
+            'title': 'Baba',
+            'description': 'md5:d43fd651937cd75cc650883ebd8d8461',
+            'thumbnail': r're:https://s[0-9]+.filmmodu.org/uploads/movie/cover/3646/6xKCYgH16UuwEGAyroLU6p8HLIn.jpg',
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        title = self._og_search_title(webpage, fatal=True)
+        description = self._og_search_description(webpage)
+        thumbnail = self._og_search_thumbnail(webpage)
+        real_video_id = self._search_regex(r'var\s*videoId\s*=\s*\'([0-9]+)\'', webpage, 'video_id')
+        video_type = self._search_regex(r'var\s*videoType\s*=\s*\'([a-z]+)\'', webpage, 'video_type')
+        data = self._download_json('https://www.filmmodu.org/get-source', real_video_id, query={
+            'movie_id': real_video_id,
+            'type': video_type,
+        })
+        formats = [{
+            'url': source['src'],
+            'ext': 'mp4',
+            'format_id': source['label'],
+            'height': int_or_none(source.get('res')),
+            'protocol': 'm3u8_native',
+        } for source in data['sources']]
+
+        self._sort_formats(formats)
+
+        subtitles = {}
+
+        if data.get('subtitle'):
+            subtitles['tr'] = [{
+                'url': data['subtitle'],
+            }]
+
+        return {
+            'id': real_video_id,
+            'display_id': video_id,
+            'title': title,
+            'description': description,
+            'formats': formats,
+            'subtitles': subtitles,
+            'thumbnail': thumbnail,
+        }
diff --git a/yt_dlp/extractor/hungama.py b/yt_dlp/extractor/hungama.py
index 3fdaac5b6..c33e387e7 100644
--- a/yt_dlp/extractor/hungama.py
+++ b/yt_dlp/extractor/hungama.py
@@ -1,9 +1,12 @@
 # coding: utf-8
 from __future__ import unicode_literals
 
+import re
+
 from .common import InfoExtractor
 from ..utils import (
     int_or_none,
+    try_get,
     urlencode_postdata,
 )
 
@@ -71,14 +74,14 @@ class HungamaSongIE(InfoExtractor):
     _VALID_URL = r'https?://(?:www\.)?hungama\.com/song/[^/]+/(?P<id>\d+)'
     _TEST = {
         'url': 'https://www.hungama.com/song/kitni-haseen-zindagi/2931166/',
-        'md5': 'a845a6d1ebd08d80c1035126d49bd6a0',
+        'md5': 'd4a6a05a394ad0453a9bea3ca00e6024',
         'info_dict': {
             'id': '2931166',
-            'ext': 'mp4',
+            'ext': 'mp3',
             'title': 'Lucky Ali - Kitni Haseen Zindagi',
             'track': 'Kitni Haseen Zindagi',
             'artist': 'Lucky Ali',
-            'album': 'Aks',
+            'album': None,
             'release_year': 2000,
         }
     }
@@ -89,18 +92,20 @@ def _real_extract(self, url):
         data = self._download_json(
             'https://www.hungama.com/audio-player-data/track/%s' % audio_id,
             audio_id, query={'_country': 'IN'})[0]
-
         track = data['song_name']
         artist = data.get('singer_name')
-
-        m3u8_url = self._download_json(
-            data.get('file') or data['preview_link'],
-            audio_id)['response']['media_url']
-
-        formats = self._extract_m3u8_formats(
-            m3u8_url, audio_id, ext='mp4', entry_protocol='m3u8_native',
-            m3u8_id='hls')
-        self._sort_formats(formats)
+        formats = []
+        media_json = self._download_json(data.get('file') or data['preview_link'], audio_id)
+        media_url = try_get(media_json, lambda x: x['response']['media_url'], str)
+        media_type = try_get(media_json, lambda x: x['response']['type'], str)
+
+        if media_url:
+            formats.append({
+                'url': media_url,
+                'ext': media_type,
+                'vcodec': 'none',
+                'acodec': media_type,
+            })
 
         title = '%s - %s' % (artist, track) if artist else track
         thumbnail = data.get('img_src') or data.get('album_image')
@@ -111,7 +116,32 @@ def _real_extract(self, url):
             'thumbnail': thumbnail,
             'track': track,
             'artist': artist,
-            'album': data.get('album_name'),
+            'album': data.get('album_name') or None,
             'release_year': int_or_none(data.get('date')),
             'formats': formats,
         }
+
+
+class HungamaAlbumPlaylistIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?hungama\.com/(?:playlists|album)/[^/]+/(?P<id>\d+)'
+    _TESTS = [{
+        'url': 'https://www.hungama.com/album/bhuj-the-pride-of-india/69481490/',
+        'playlist_mincount': 7,
+        'info_dict': {
+            'id': '69481490',
+        },
+    }, {
+        'url': 'https://www.hungama.com/playlists/hindi-jan-to-june-2021/123063/',
+        'playlist_mincount': 50,
+        'info_dict': {
+            'id': '123063',
+        },
+    }]
+
+    def _real_extract(self, url):
+        id = self._match_id(url)
+        webpage = self._download_webpage(url, id)
+        ptrn = r'<meta[^>]+?property=[\"\']?music:song:url[\"\']?[^>]+?content=[\"\']?([^\"\']+)'
+        items = re.findall(ptrn, webpage)
+        entries = [self.url_result(item, ie=HungamaSongIE.ie_key()) for item in items]
+        return self.playlist_result(entries, id)
diff --git a/yt_dlp/extractor/instagram.py b/yt_dlp/extractor/instagram.py
index d8acef5ba..27deb0af0 100644
--- a/yt_dlp/extractor/instagram.py
+++ b/yt_dlp/extractor/instagram.py
@@ -277,6 +277,9 @@ def get_count(keys, kind):
             'like_count': like_count,
             'comment_count': comment_count,
             'comments': comments,
+            'http_headers': {
+                'Referer': 'https://www.instagram.com/',
+            }
         }
 
 
diff --git a/yt_dlp/extractor/kakao.py b/yt_dlp/extractor/kakao.py
index fefd8a215..97c986d8c 100644
--- a/yt_dlp/extractor/kakao.py
+++ b/yt_dlp/extractor/kakao.py
@@ -7,6 +7,7 @@
 from ..utils import (
     int_or_none,
     strip_or_none,
+    traverse_obj,
     unified_timestamp,
 )
 
@@ -73,40 +74,35 @@ def _real_extract(self, url):
 
         formats = []
         for fmt in clip.get('videoOutputList', []):
-            try:
-                profile_name = fmt['profile']
-                if profile_name == 'AUDIO':
-                    continue
-                query.update({
-                    'profile': profile_name,
-                    'fields': '-*,url',
-                })
-
-                fmt_url_json = self._download_json(
-                    cdn_api_base, video_id,
-                    'Downloading video URL for profile %s' % profile_name,
-                    query=query, fatal=False)
-
-                if fmt_url_json is None:
-                    continue
+            profile_name = fmt.get('profile')
+            if not profile_name or profile_name == 'AUDIO':
+                continue
+            query.update({
+                'profile': profile_name,
+                'fields': '-*,url',
+            })
 
-                fmt_vidLocation = fmt_url_json['videoLocation']
-                fmt_url = fmt_vidLocation['url']
-                formats.append({
-                    'url': fmt_url,
-                    'format_id': profile_name,
-                    'width': int_or_none(fmt.get('width')),
-                    'height': int_or_none(fmt.get('height')),
-                    'format_note': fmt.get('label'),
-                    'filesize': int_or_none(fmt.get('filesize')),
-                    'tbr': int_or_none(fmt.get('kbps')),
-                })
-            except KeyError:
-                pass
+            fmt_url_json = self._download_json(
+                cdn_api_base, video_id,
+                'Downloading video URL for profile %s' % profile_name,
+                query=query, fatal=False)
+            fmt_url = traverse_obj(fmt_url_json, ('videoLocation', 'url'))
+            if not fmt_url:
+                continue
+
+            formats.append({
+                'url': fmt_url,
+                'format_id': profile_name,
+                'width': int_or_none(fmt.get('width')),
+                'height': int_or_none(fmt.get('height')),
+                'format_note': fmt.get('label'),
+                'filesize': int_or_none(fmt.get('filesize')),
+                'tbr': int_or_none(fmt.get('kbps')),
+            })
         self._sort_formats(formats)
 
         thumbs = []
-        for thumb in clip.get('clipChapterThumbnailList', []):
+        for thumb in clip.get('clipChapterThumbnailList') or []:
             thumbs.append({
                 'url': thumb.get('thumbnailUrl'),
                 'id': compat_str(thumb.get('timeInSec')),
@@ -123,7 +119,7 @@ def _real_extract(self, url):
             'id': video_id,
             'title': title,
             'description': strip_or_none(clip.get('description')),
-            'uploader': clip_link.get('channel', {}).get('name'),
+            'uploader': traverse_obj(clip_link, ('channel', 'name')),
             'uploader_id': clip_link.get('channelId'),
             'thumbnails': thumbs,
             'timestamp': unified_timestamp(clip_link.get('createTime')),
diff --git a/yt_dlp/extractor/paramountplus.py b/yt_dlp/extractor/paramountplus.py
index e1d479588..338b84d5b 100644
--- a/yt_dlp/extractor/paramountplus.py
+++ b/yt_dlp/extractor/paramountplus.py
@@ -90,7 +90,7 @@ class ParamountPlusIE(CBSBaseIE):
     def _extract_video_info(self, content_id, mpx_acc=2198311517):
         items_data = self._download_json(
             'https://www.paramountplus.com/apps-api/v2.0/androidtv/video/cid/%s.json' % content_id,
-            content_id, query={'locale': 'en-us', 'at': 'ABCqWNNSwhIqINWIIAG+DFzcFUvF8/vcN6cNyXFFfNzWAIvXuoVgX+fK4naOC7V8MLI='})
+            content_id, query={'locale': 'en-us', 'at': 'ABCqWNNSwhIqINWIIAG+DFzcFUvF8/vcN6cNyXFFfNzWAIvXuoVgX+fK4naOC7V8MLI='}, headers=self.geo_verification_headers())
 
         asset_types = {
             item.get('assetType'): {
diff --git a/yt_dlp/extractor/pornhub.py b/yt_dlp/extractor/pornhub.py
index c525505d1..c2b20ecfd 100644
--- a/yt_dlp/extractor/pornhub.py
+++ b/yt_dlp/extractor/pornhub.py
@@ -3,6 +3,7 @@
 
 import functools
 import itertools
+import math
 import operator
 import re
 
@@ -638,7 +639,7 @@ def _real_extract(self, url):
 
 
 class PornHubPagedVideoListIE(PornHubPagedPlaylistBaseIE):
-    _VALID_URL = r'https?://(?:[^/]+\.)?%s/(?P<id>(?:[^/]+/)*[^/?#&]+)' % PornHubBaseIE._PORNHUB_HOST_RE
+    _VALID_URL = r'https?://(?:[^/]+\.)?%s/(?!playlist/)(?P<id>(?:[^/]+/)*[^/?#&]+)' % PornHubBaseIE._PORNHUB_HOST_RE
     _TESTS = [{
         'url': 'https://www.pornhub.com/model/zoe_ph/videos',
         'only_matching': True,
@@ -731,18 +732,6 @@ class PornHubPagedVideoListIE(PornHubPagedPlaylistBaseIE):
     }, {
         'url': 'https://www.pornhub.com/video/incategories/60fps-1/hd-porn',
         'only_matching': True,
-    }, {
-        'url': 'https://www.pornhub.com/playlist/44121572',
-        'info_dict': {
-            'id': 'playlist/44121572',
-        },
-        'playlist_mincount': 132,
-    }, {
-        'url': 'https://www.pornhub.com/playlist/4667351',
-        'only_matching': True,
-    }, {
-        'url': 'https://de.pornhub.com/playlist/4667351',
-        'only_matching': True,
     }, {
         'url': 'https://pornhubthbh7ap3u.onion/model/zoe_ph/videos',
         'only_matching': True,
@@ -770,3 +759,59 @@ class PornHubUserVideosUploadIE(PornHubPagedPlaylistBaseIE):
         'url': 'http://pornhubthbh7ap3u.onion/pornstar/jenny-blighe/videos/upload',
         'only_matching': True,
     }]
+
+
+class PornHubPlaylistIE(PornHubPlaylistBaseIE):
+    _VALID_URL = r'(?P<url>https?://(?:[^/]+\.)?%s/playlist/(?P<id>[^/?#&]+))' % PornHubBaseIE._PORNHUB_HOST_RE
+    _TESTS = [{
+        'url': 'https://www.pornhub.com/playlist/44121572',
+        'info_dict': {
+            'id': '44121572',
+        },
+        'playlist_count': 77,
+    }, {
+        'url': 'https://www.pornhub.com/playlist/4667351',
+        'only_matching': True,
+    }, {
+        'url': 'https://de.pornhub.com/playlist/4667351',
+        'only_matching': True,
+    }, {
+        'url': 'https://de.pornhub.com/playlist/4667351?page=2',
+        'only_matching': True,
+    }]
+
+    def _entries(self, url, host, item_id):
+        webpage = self._download_webpage(url, item_id, 'Downloading page 1')
+        playlist_id = self._search_regex(r'var\s+playlistId\s*=\s*"([^"]+)"', webpage, 'playlist_id')
+        video_count = int_or_none(
+            self._search_regex(r'var\s+itemsCount\s*=\s*([0-9]+)\s*\|\|', webpage, 'video_count'))
+        token = self._search_regex(r'var\s+token\s*=\s*"([^"]+)"', webpage, 'token')
+        page_count = math.ceil((video_count - 36) / 40.) + 1
+        page_entries = self._extract_entries(webpage, host)
+
+        def download_page(page_num):
+            note = 'Downloading page {}'.format(page_num)
+            page_url = 'https://www.{}/playlist/viewChunked'.format(host)
+            return self._download_webpage(page_url, item_id, note, query={
+                'id': playlist_id,
+                'page': page_num,
+                'token': token,
+            })
+
+        for page_num in range(1, page_count + 1):
+            if page_num > 1:
+                webpage = download_page(page_num)
+                page_entries = self._extract_entries(webpage, host)
+            if not page_entries:
+                break
+            for e in page_entries:
+                yield e
+
+    def _real_extract(self, url):
+        mobj = re.match(self._VALID_URL, url)
+        host = mobj.group('host')
+        item_id = mobj.group('id')
+
+        self._login(host)
+
+        return self.playlist_result(self._entries(mobj.group('url'), host, item_id), item_id)
diff --git a/yt_dlp/extractor/soundcloud.py b/yt_dlp/extractor/soundcloud.py
index 84ab4bcdf..13ec96d82 100644
--- a/yt_dlp/extractor/soundcloud.py
+++ b/yt_dlp/extractor/soundcloud.py
@@ -295,7 +295,7 @@ def _download_json(self, *args, **kwargs):
             try:
                 return super(SoundcloudIE, self)._download_json(*args, **compat_kwargs(kwargs))
             except ExtractorError as e:
-                if isinstance(e.cause, compat_HTTPError) and e.cause.code == 401:
+                if isinstance(e.cause, compat_HTTPError) and e.cause.code in (401, 403):
                     self._store_client_id(None)
                     self._update_client_id()
                     continue
@@ -305,7 +305,7 @@ def _download_json(self, *args, **kwargs):
                 raise
 
     def _real_initialize(self):
-        self._CLIENT_ID = self._downloader.cache.load('soundcloud', 'client_id') or 'fXuVKzsVXlc6tzniWWS31etd7VHWFUuN'  # persistent `client_id`
+        self._CLIENT_ID = self._downloader.cache.load('soundcloud', 'client_id') or 'fSSdm5yTnDka1g0Fz1CO5Yx6z0NbeHAj'
         self._login()
 
     _USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36'
diff --git a/yt_dlp/extractor/trovo.py b/yt_dlp/extractor/trovo.py
index 0e842b9ca..7d6b2b88e 100644
--- a/yt_dlp/extractor/trovo.py
+++ b/yt_dlp/extractor/trovo.py
@@ -36,7 +36,7 @@ def _real_extract(self, url):
                 'query': '''{
   getLiveInfo(params: {userName: "%s"}) {
     isLive
-    programInfo	{
+    programInfo {
       coverUrl
       id
       streamInfo {
diff --git a/yt_dlp/extractor/voicy.py b/yt_dlp/extractor/voicy.py
new file mode 100644
index 000000000..ae29c3de2
--- /dev/null
+++ b/yt_dlp/extractor/voicy.py
@@ -0,0 +1,148 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import (
+    ExtractorError,
+    smuggle_url,
+    traverse_obj,
+    unsmuggle_url,
+    unified_strdate,
+)
+
+import re
+import itertools
+
+
+class VoicyBaseIE(InfoExtractor):
+    def _extract_from_playlist_data(self, value):
+        voice_id = compat_str(value.get('PlaylistId'))
+        upload_date = unified_strdate(value.get('Published'), False)
+        items = [self._extract_single_article(voice_data) for voice_data in value['VoiceData']]
+        return {
+            '_type': 'multi_video',
+            'entries': items,
+            'id': voice_id,
+            'title': compat_str(value.get('PlaylistName')),
+            'uploader': value.get('SpeakerName'),
+            'uploader_id': compat_str(value.get('SpeakerId')),
+            'channel': value.get('ChannelName'),
+            'channel_id': compat_str(value.get('ChannelId')),
+            'upload_date': upload_date,
+        }
+
+    def _extract_single_article(self, entry):
+        formats = [{
+            'url': entry['VoiceHlsFile'],
+            'format_id': 'hls',
+            'ext': 'm4a',
+            'acodec': 'aac',
+            'vcodec': 'none',
+            'protocol': 'm3u8_native',
+        }, {
+            'url': entry['VoiceFile'],
+            'format_id': 'mp3',
+            'ext': 'mp3',
+            'acodec': 'mp3',
+            'vcodec': 'none',
+        }]
+        self._sort_formats(formats)
+        return {
+            'id': compat_str(entry.get('ArticleId')),
+            'title': entry.get('ArticleTitle'),
+            'description': entry.get('MediaName'),
+            'formats': formats,
+        }
+
+    def _call_api(self, url, video_id, **kwargs):
+        response = self._download_json(url, video_id, **kwargs)
+        if response.get('Status') != 0:
+            message = traverse_obj(response, ('Value', 'Error', 'Message'), expected_type=compat_str)
+            if not message:
+                message = 'There was a error in the response: %d' % response.get('Status')
+            raise ExtractorError(message, expected=False)
+        return response.get('Value')
+
+
+class VoicyIE(VoicyBaseIE):
+    IE_NAME = 'voicy'
+    _VALID_URL = r'https?://voicy\.jp/channel/(?P<channel_id>\d+)/(?P<id>\d+)'
+    ARTICLE_LIST_API_URL = 'https://vmw.api.voicy.jp/articles_list?channel_id=%s&pid=%s'
+    _TESTS = [{
+        'url': 'https://voicy.jp/channel/1253/122754',
+        'info_dict': {
+            'id': '122754',
+            'title': '1/21(木)声日記：ついに原稿終わった！！',
+            'uploader': 'ちょまど@ ITエンジニアなオタク',
+            'uploader_id': '7339',
+        },
+        'playlist_mincount': 9,
+    }]
+
+    def _real_extract(self, url):
+        mobj = re.match(self._VALID_URL, url)
+        assert mobj
+        voice_id = mobj.group('id')
+        channel_id = mobj.group('channel_id')
+        url, article_list = unsmuggle_url(url)
+        if not article_list:
+            article_list = self._call_api(self.ARTICLE_LIST_API_URL % (channel_id, voice_id), voice_id)
+        return self._extract_from_playlist_data(article_list)
+
+
+class VoicyChannelIE(VoicyBaseIE):
+    IE_NAME = 'voicy:channel'
+    _VALID_URL = r'https?://voicy\.jp/channel/(?P<id>\d+)'
+    PROGRAM_LIST_API_URL = 'https://vmw.api.voicy.jp/program_list/all?channel_id=%s&limit=20&public_type=3%s'
+    _TESTS = [{
+        'url': 'https://voicy.jp/channel/1253/',
+        'info_dict': {
+            'id': '7339',
+            'title': 'ゆるふわ日常ラジオ #ちょまラジ',
+            'uploader': 'ちょまど@ ITエンジニアなオタク',
+            'uploader_id': '7339',
+        },
+        'playlist_mincount': 54,
+    }]
+
+    @classmethod
+    def suitable(cls, url):
+        return not VoicyIE.suitable(url) and super(VoicyChannelIE, cls).suitable(url)
+
+    def _entries(self, channel_id):
+        pager = ''
+        for count in itertools.count(1):
+            article_list = self._call_api(self.PROGRAM_LIST_API_URL % (channel_id, pager), channel_id, note='Paging #%d' % count)
+            playlist_data = article_list.get('PlaylistData')
+            if not playlist_data:
+                break
+            yield from playlist_data
+            last = playlist_data[-1]
+            pager = '&pid=%d&p_date=%s&play_count=%s' % (last['PlaylistId'], last['Published'], last['PlayCount'])
+
+    def _real_extract(self, url):
+        channel_id = self._match_id(url)
+        articles = self._entries(channel_id)
+
+        first_article = next(articles, None)
+        title = traverse_obj(first_article, ('ChannelName', ), expected_type=compat_str)
+        speaker_name = traverse_obj(first_article, ('SpeakerName', ), expected_type=compat_str)
+        if not title and speaker_name:
+            title = 'Uploads from %s' % speaker_name
+        if not title:
+            title = 'Uploads from channel ID %s' % channel_id
+
+        articles = itertools.chain([first_article], articles) if first_article else articles
+
+        playlist = (
+            self.url_result(smuggle_url('https://voicy.jp/channel/%s/%d' % (channel_id, value['PlaylistId']), value), VoicyIE.ie_key())
+            for value in articles)
+        return {
+            '_type': 'playlist',
+            'entries': playlist,
+            'id': channel_id,
+            'title': title,
+            'channel': speaker_name,
+            'channel_id': channel_id,
+        }
diff --git a/yt_dlp/options.py b/yt_dlp/options.py
index f2d5deb68..ef821eb11 100644
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -122,6 +122,30 @@ def _list_from_options_callback(option, opt_str, value, parser, append=True, del
             parser.values, option.dest,
             current + value if append is True else value + current)
 
+    def _set_from_options_callback(
+            option, opt_str, value, parser,
+            delim=',', allowed_values=None, process=str.lower, aliases={}):
+        current = getattr(parser.values, option.dest)
+        values = [process(value)] if delim is None else process(value).split(delim)[::-1]
+        while values:
+            actual_val = val = values.pop()
+            if val == 'all':
+                current.update(allowed_values)
+            elif val == '-all':
+                current = set()
+            elif val in aliases:
+                values.extend(aliases[val])
+            else:
+                if val[0] == '-':
+                    val = val[1:]
+                    current.discard(val)
+                else:
+                    current.update([val])
+                if allowed_values is not None and val not in allowed_values:
+                    raise optparse.OptionValueError(f'wrong {option.metavar} for {opt_str}: {actual_val}')
+
+        setattr(parser.values, option.dest, current)
+
     def _dict_from_options_callback(
             option, opt_str, value, parser,
             allowed_keys=r'[\w-]+', delimiter=':', default_key=None, process=None, multiple_keys=True):
@@ -241,10 +265,21 @@ def _dict_from_options_callback(
         help='Do not emit color codes in output')
     general.add_option(
         '--compat-options',
-        metavar='OPTS', dest='compat_opts', default=[],
-        action='callback', callback=_list_from_options_callback, type='str',
-        help=(
-            'Options that can help keep compatibility with youtube-dl and youtube-dlc '
+        metavar='OPTS', dest='compat_opts', default=set(), type='str',
+        action='callback', callback=_set_from_options_callback,
+        callback_kwargs={
+            'allowed_values': {
+                'filename', 'format-sort', 'abort-on-error', 'format-spec', 'no-playlist-metafiles',
+                'multistreams', 'no-live-chat', 'playlist-index', 'list-formats', 'no-direct-merge',
+                'no-youtube-channel-redirect', 'no-youtube-unavailable-videos', 'no-attach-info-json',
+                'embed-thumbnail-atomicparsley', 'seperate-video-versions', 'no-clean-infojson', 'no-keep-subs',
+            },
+            'aliases': {
+                'youtube-dl': ['-multistreams', 'all'],
+                'youtube-dlc': ['-no-youtube-channel-redirect', '-no-live-chat', 'all'],
+            }
+        }, help=(
+            'Options that can help keep compatibility with youtube-dl or youtube-dlc '
             'configurations by reverting some of the changes made in yt-dlp. '
             'See "Differences in default behavior" for details'))
 
@@ -321,11 +356,11 @@ def _dict_from_options_callback(
     selection.add_option(
         '--match-title',
         dest='matchtitle', metavar='REGEX',
-        help='Download only matching titles (regex or caseless sub-string)')
+        help=optparse.SUPPRESS_HELP)
     selection.add_option(
         '--reject-title',
         dest='rejecttitle', metavar='REGEX',
-        help='Skip download for matching titles (regex or caseless sub-string)')
+        help=optparse.SUPPRESS_HELP)
     selection.add_option(
         '--max-downloads',
         dest='max_downloads', metavar='NUMBER', type=int, default=None,
@@ -360,11 +395,11 @@ def _dict_from_options_callback(
     selection.add_option(
         '--min-views',
         metavar='COUNT', dest='min_views', default=None, type=int,
-        help='Do not download any videos with less than COUNT views')
+        help=optparse.SUPPRESS_HELP)
     selection.add_option(
         '--max-views',
         metavar='COUNT', dest='max_views', default=None, type=int,
-        help='Do not download any videos with more than COUNT views')
+        help=optparse.SUPPRESS_HELP)
     selection.add_option(
         '--match-filter',
         metavar='FILTER', dest='match_filter', default=None,
diff --git a/yt_dlp/postprocessor/ffmpeg.py b/yt_dlp/postprocessor/ffmpeg.py
index 1d21ffaf2..be6cc9f09 100644
--- a/yt_dlp/postprocessor/ffmpeg.py
+++ b/yt_dlp/postprocessor/ffmpeg.py
@@ -802,7 +802,7 @@ def run(self, info):
                 }
 
             info['__files_to_move'][new_file] = replace_extension(
-                info['__files_to_move'][old_file], new_ext)
+                info['__files_to_move'][sub['filepath']], new_ext)
 
         return sub_filenames, info
 
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index 0e8392fdf..6276ac726 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -4657,7 +4657,7 @@ def filter_using_list(row, filterArray):
     return '\n'.join(format_str % tuple(row) for row in table)
 
 
-def _match_one(filter_part, dct):
+def _match_one(filter_part, dct, incomplete):
     # TODO: Generalize code with YoutubeDL._build_format_filter
     STRING_OPERATORS = {
         '*=': operator.contains,
@@ -4718,7 +4718,7 @@ def _match_one(filter_part, dct):
                         'Invalid integer value %r in filter part %r' % (
                             m.group('intval'), filter_part))
         if actual_value is None:
-            return m.group('none_inclusive')
+            return incomplete or m.group('none_inclusive')
         return op(actual_value, comparison_value)
 
     UNARY_OPERATORS = {
@@ -4733,22 +4733,25 @@ def _match_one(filter_part, dct):
     if m:
         op = UNARY_OPERATORS[m.group('op')]
         actual_value = dct.get(m.group('key'))
+        if incomplete and actual_value is None:
+            return True
         return op(actual_value)
 
     raise ValueError('Invalid filter part %r' % filter_part)
 
 
-def match_str(filter_str, dct):
-    """ Filter a dictionary with a simple string syntax. Returns True (=passes filter) or false """
-
+def match_str(filter_str, dct, incomplete=False):
+    """ Filter a dictionary with a simple string syntax. Returns True (=passes filter) or false
+        When incomplete, all conditions passes on missing fields
+    """
     return all(
-        _match_one(filter_part.replace(r'\&', '&'), dct)
+        _match_one(filter_part.replace(r'\&', '&'), dct, incomplete)
         for filter_part in re.split(r'(?<!\\)&', filter_str))
 
 
 def match_filter_func(filter_str):
-    def _match_func(info_dict):
-        if match_str(filter_str, info_dict):
+    def _match_func(info_dict, *args, **kwargs):
+        if match_str(filter_str, info_dict, *args, **kwargs):
             return None
         else:
             video_title = info_dict.get('title', info_dict.get('id', 'video'))


Test patch:
diff --git a/test/test_YoutubeDL.py b/test/test_YoutubeDL.py
index 7e0133027..e689978fd 100644
--- a/test/test_YoutubeDL.py
+++ b/test/test_YoutubeDL.py
@@ -978,54 +978,31 @@ def get_downloaded_info_dicts(params):
             ydl.process_ie_result(copy.deepcopy(playlist))
             return ydl.downloaded_info_dicts
 
-        def get_ids(params):
-            return [int(v['id']) for v in get_downloaded_info_dicts(params)]
-
-        result = get_ids({})
-        self.assertEqual(result, [1, 2, 3, 4])
-
-        result = get_ids({'playlistend': 10})
-        self.assertEqual(result, [1, 2, 3, 4])
-
-        result = get_ids({'playlistend': 2})
-        self.assertEqual(result, [1, 2])
-
-        result = get_ids({'playliststart': 10})
-        self.assertEqual(result, [])
-
-        result = get_ids({'playliststart': 2})
-        self.assertEqual(result, [2, 3, 4])
-
-        result = get_ids({'playlist_items': '2-4'})
-        self.assertEqual(result, [2, 3, 4])
-
-        result = get_ids({'playlist_items': '2,4'})
-        self.assertEqual(result, [2, 4])
-
-        result = get_ids({'playlist_items': '10'})
-        self.assertEqual(result, [])
-
-        result = get_ids({'playlist_items': '3-10'})
-        self.assertEqual(result, [3, 4])
-
-        result = get_ids({'playlist_items': '2-4,3-4,3'})
-        self.assertEqual(result, [2, 3, 4])
+        def test_selection(params, expected_ids):
+            results = [
+                (v['playlist_autonumber'] - 1, (int(v['id']), v['playlist_index']))
+                for v in get_downloaded_info_dicts(params)]
+            self.assertEqual(results, list(enumerate(zip(expected_ids, expected_ids))))
+
+        test_selection({}, [1, 2, 3, 4])
+        test_selection({'playlistend': 10}, [1, 2, 3, 4])
+        test_selection({'playlistend': 2}, [1, 2])
+        test_selection({'playliststart': 10}, [])
+        test_selection({'playliststart': 2}, [2, 3, 4])
+        test_selection({'playlist_items': '2-4'}, [2, 3, 4])
+        test_selection({'playlist_items': '2,4'}, [2, 4])
+        test_selection({'playlist_items': '10'}, [])
 
         # Tests for https://github.com/ytdl-org/youtube-dl/issues/10591
-        # @{
-        result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
-        self.assertEqual(result[0]['playlist_index'], 2)
-        self.assertEqual(result[1]['playlist_index'], 3)
-
-        result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
-        self.assertEqual(result[0]['playlist_index'], 2)
-        self.assertEqual(result[1]['playlist_index'], 3)
-        self.assertEqual(result[2]['playlist_index'], 4)
-
-        result = get_downloaded_info_dicts({'playlist_items': '4,2'})
-        self.assertEqual(result[0]['playlist_index'], 4)
-        self.assertEqual(result[1]['playlist_index'], 2)
-        # @}
+        test_selection({'playlist_items': '2-4,3-4,3'}, [2, 3, 4])
+        test_selection({'playlist_items': '4,2'}, [4, 2])
+
+        # Tests for https://github.com/yt-dlp/yt-dlp/issues/720
+        # https://github.com/yt-dlp/yt-dlp/issues/302
+        test_selection({'playlistreverse': True}, [4, 3, 2, 1])
+        test_selection({'playliststart': 2, 'playlistreverse': True}, [4, 3, 2])
+        test_selection({'playlist_items': '2,4', 'playlistreverse': True}, [4, 2])
+        test_selection({'playlist_items': '4,2'}, [4, 2])
 
     def test_urlopen_no_file_protocol(self):
         # see https://github.com/ytdl-org/youtube-dl/issues/8227
diff --git a/test/test_utils.py b/test/test_utils.py
index aef59e491..dedc598f7 100644
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -1285,9 +1285,15 @@ def test_match_str(self):
         self.assertTrue(match_str(r'x="foo \& bar" & x^=foo', {'x': 'foo & bar'}))
 
         # Example from docs
-        self.assertTrue(
-            r'!is_live & like_count>?100 & description~=\'(?i)\bcats \& dogs\b\'',
-            {'description': 'Raining Cats & Dogs'})
+        self.assertTrue(match_str(
+            r"!is_live & like_count>?100 & description~='(?i)\bcats \& dogs\b'",
+            {'description': 'Raining Cats & Dogs'}))
+
+        # Incomplete
+        self.assertFalse(match_str('id!=foo', {'id': 'foo'}, True))
+        self.assertTrue(match_str('x', {'id': 'foo'}, True))
+        self.assertTrue(match_str('!x', {'id': 'foo'}, True))
+        self.assertFalse(match_str('x', {'id': 'foo'}, False))
 
     def test_parse_dfxp_time_expr(self):
         self.assertEqual(parse_dfxp_time_expr(None), None)