Repository: ytdl-org/youtube-dl

Issue:
Add support for downloading audio from Callin
## Checklist

- [x] I'm reporting a new site support request
- [x] I've verified that I'm running youtube-dl version **2021.12.17**
- [x] I've checked that all provided URLs are alive and playable in a browser
- [x] I've checked that none of provided URLs violate any copyrights
- [x] I've searched the bugtracker for similar site support requests including closed ones

## Example URLs

- Single audio file: https://www.callin.com/episode/fcc-commissioner-brendan-carr-on-elons-PrumRdSQJW
- Single audio file: https://www.callin.com/episode/episode-81-elites-melt-down-over-student-debt-lzxMidUnjA

## Description

Pretty self-explanatory, the site hosts audio tracks of call-in shows that you can listen to after the fact. Compare Soundcloud as another example of an extractor for audio-only content.


Discussion:
The page uses NextJS. Metadata and what appears to be a valid m3u8 (although the URL contains the word `preview`) can be found in the hydration JSON.
Hey there! I'm currently a student in college and as part of my final project, I'm required to make contributions to an open-source project and write a report on my experiences (and inform the maintainers) . Could I take this issue?
@JohnnySunUmich I can't speak for the project, but I can't imagine any reason why you couldn't take this issue -- it was filed a while ago and there's no sign of anyone else working on it.

As the issue reporter, I would certainly appreciate it!
So is this all the same class "volunteering" to solve open-source issues?

@JohnnySunUmich, please go ahead. Do note the general points in the advice I gave [here](https://github.com/ytdl-org/youtube-dl/issues/31336#issuecomment-1306389725).

For this specific issue, what I'd suggest is to back-port the method `InfoExtractor._search_nextjs_data()` from yt-dlp, which is under the same licence: you'll have to adjust the signature to make it work in Python 2, as I'll discuss below. Then you can navigate the JSON returned by that method to populate the `info_dict` that the extractor has to return.
* The mandatory and optional elements are listed in the class documentation comments in `InfoExtractor`.
* There are various utility functions that you should use to ensure the correct sort of value (`xxx_or_none()`) and to get values that may sometimes be absent (`try_get()` and now `traverse_obj()`).
* Look at existing extractor code for examples.

[Python 3](https://docs.python.org/3/whatsnew/3.0.html) introduced the [dummy argument `*`](https://peps.python.org/pep-3102/) to mean that subsequent arguments can only be passed by keyword, or equivalently that there are no more positional parameters.

A cross-version equivalent is like this:
```py
# Python 3 only
def foo(arg1, arg2, *, kwonly1=val1, kwonly2=val2, **kwargs):
    ...

#Python 2+3
def foo(arg1, arg2, **kwargs):
    # instantiate keyword-only args
    kwonly1 = kwargs.pop('kwonly1', val1)
    # or if kwonly2 is going to be read and passed down
    # in kwargs to another function (eg `fatal` in the mentioned method) 
    if 'kwonly2' in kwargs:
        kwonly2 = kwargs.get('kwonly2')
    else:
        kwargs['kwonly2'] = kwonly2 = val2
        kwargs = compat_kwargs(kwargs)
    ...
```
You can see the benefit of the new syntax, but that doesn't make it backward-compatible. Also, the one (?) person trying to run yt-dl on Python 2.6 will thank you for passing an updated `kwargs` using `compat_kwargs()`!

> A cross-version equivalent is like this:

I use `*` in yt-dlp because I can - it's in no way essential to the working of the code. Rather than the hack with `kwargs`, I'd recommend to just remove the `*` and mention in docstring that it is a keyword only argument (see `_download_webpage_handle`)
@dirkf Thank you so much for your detailed advice! Will keep you updated if there's any progress
What are you saying, "hack"?

There may also be this issue:
```
def foo(arg1, arg2, kwonly1='val1', kwonly2='val2', **kwargs):
    pass
    
foo(1, 2, 'val1_a', kwonly1='val1_b')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: foo() got multiple values for keyword argument 'kwonly1'
```




Resolving patch:
diff --git a/youtube_dl/extractor/callin.py b/youtube_dl/extractor/callin.py
new file mode 100644
--- /dev/null
+++ b/youtube_dl/extractor/callin.py
@@ -0,0 +1,74 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import (
+    ExtractorError,
+    traverse_obj,
+    try_get,
+)
+
+
+class CallinIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?callin\.com/episode/(?:[^/#?-]+-)*(?P<id>[^/#?-]+)'
+    _TESTS = [{
+        'url': 'https://www.callin.com/episode/fcc-commissioner-brendan-carr-on-elons-PrumRdSQJW',
+        'md5': '14ede27ee2c957b7e4db93140fc0745c',
+        'info_dict': {
+            'id': 'PrumRdSQJW',
+            'ext': 'mp4',
+            'title': 'FCC Commissioner Brendan Carr on Elon’s Starlink',
+            'description': 'Or, why the government doesn’t like SpaceX',
+            'channel': 'The Pull Request',
+            'channel_url': 'https://callin.com/show/the-pull-request-ucnDJmEKAa',
+        }
+    }, {
+        'url': 'https://www.callin.com/episode/episode-81-elites-melt-down-over-student-debt-lzxMidUnjA',
+        'md5': '16f704ddbf82a27e3930533b12062f07',
+        'info_dict': {
+            'id': 'lzxMidUnjA',
+            'ext': 'mp4',
+            'title': 'Episode 81- Elites MELT DOWN over Student Debt Victory? Rumble in NYC?',
+            'description': 'Let’s talk todays episode about the primary election shake up in NYC and the elites melting down over student debt cancelation.',
+            'channel': 'The DEBRIEF With Briahna Joy Gray',
+            'channel_url': 'https://callin.com/show/the-debrief-with-briahna-joy-gray-siiFDzGegm',
+        }
+    }]
+
+    def _search_nextjs_data(self, webpage, video_id, transform_source=None, fatal=True, **kw):
+        return self._parse_json(
+            self._search_regex(
+                r'(?s)<script[^>]+id=[\'"]__NEXT_DATA__[\'"][^>]*>([^<]+)</script>',
+                webpage, 'next.js data', fatal=fatal, **kw),
+            video_id, transform_source=transform_source, fatal=fatal)
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        next_data = self._search_nextjs_data(webpage, video_id)
+        episode = traverse_obj(next_data, ('props', 'pageProps', 'episode'), expected_type=dict)
+        if not episode:
+            raise ExtractorError('Failed to find episode data')
+
+        title = episode.get('title') or self._og_search_title(webpage)
+        description = episode.get('description') or self._og_search_description(webpage)
+
+        formats = []
+        formats.extend(self._extract_m3u8_formats(
+            episode.get('m3u8'), video_id, 'mp4',
+            entry_protocol='m3u8_native', fatal=False))
+        self._sort_formats(formats)
+
+        channel = try_get(episode, lambda x: x['show']['title'], compat_str)
+        channel_url = try_get(episode, lambda x: x['show']['linkObj']['resourceUrl'], compat_str)
+
+        return {
+            'id': video_id,
+            'title': title,
+            'description': description,
+            'formats': formats,
+            'channel': channel,
+            'channel_url': channel_url,
+        }
diff --git a/youtube_dl/extractor/extractors.py b/youtube_dl/extractor/extractors.py
--- a/youtube_dl/extractor/extractors.py
+++ b/youtube_dl/extractor/extractors.py
@@ -158,6 +158,7 @@
 from .buzzfeed import BuzzFeedIE
 from .byutv import BYUtvIE
 from .c56 import C56IE
+from .callin import CallinIE
 from .camdemy import (
     CamdemyIE,
     CamdemyFolderIE


Test patch: