Repository: streamlink/streamlink
Pull request #2927 (2020-04-21 17:19:05)

plugins.nbcnews: new plugin for http://nbcnews.com/now
NBC News NOW is a streaming network from NBC News. Get all the latest on the day’s biggest stories and all aspects of the coronavirus crisis from the global resources of NBC News.

codecov[bot]: # [Codecov](https://codecov.io/gh/streamlink/streamlink/pull/2927?src=pr&el=h1) Report
> Merging [#2927](https://codecov.io/gh/streamlink/streamlink/pull/2927?src=pr&el=desc) into [master](https://codecov.io/gh/streamlink/streamlink/commit/39f64c0917bb8747a8871b7d4312b285822b70cb&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `73.33%`.


```diff
@@            Coverage Diff             @@
##           master    #2927      +/-   ##
==========================================
+ Coverage   52.61%   52.62%   +0.01%     
==========================================
  Files         250      251       +1     
  Lines       15720    15735      +15     
==========================================
+ Hits         8271     8281      +10     
- Misses       7449     7454       +5     
```

bastimeyer: Thanks for the plugin submission!

Not sure how safe it is to use a static API ID here (the `220018` value), but extracting it from the script files is problematic, too, as they're minified. This is where the value is coming from:

`main` js script loaded by the site's script tag (with appended hash):
`https://ndassets.s-nbcnews.com/main-35a7208c5c9dd9c46a24.js`
```js
var g = a("j9g7")
          , m = n(g)
          , N = a("y3TV")
          , T = n(N)
          , L = t.NEWS_NOW_PID = "220018"
          , h = t.MSNBC_TVE_PID = "2000762"
          , w = 0;
        t.playmakerConfiguration = {
            news: {
                leapConfigUrl: "https://api.leap.nbcsports.com/pid/NBCNews/{PID}/v4/{PLATFORM}"
            },
            msnbc: {
                leapConfigUrl: "https://api.leap.nbcsports.com/pid/MSNBC/{PID}/v4/{PLATFORM}"
            }
        }
```

There's also `https://stream.nbcsports.com/data/event_config_220018.json` which includes a link to `https://stream.nbcnews.com/data/live_sources_220018.json` which has all the m3u8 URLs as well.

Also, regarding the implementation, it'd be better to use Streamlink's validation API for getting the HLS URLs from the JSON.
vstavrinov: @bastimeyer :
> extracting it from the script files is problematic, too, as they're minified. 

Here is a solution: #2534
bastimeyer: Parsing a minimized JS file and getting the value from an AST is just a little bit better than reading it from a regex. It's still unstable and can break very easily. They just have to change a variable name for example.

I personally don't have anything against a JS parser, but such a dependency should be maintained, which esprima doesn't seem to be, especially in the JS world which changes so rapidly.
back-to: the channel is not live, but the plugin will play a 4h long VOD

this should probably ignored, instead of playing it.
vstavrinov: > the channel is not live, but the plugin will play a 4h long VOD

Sad surprise.  There is no live stream. The live stream is here: 

https://pluto.tv/live-tv/nbc-news-now

Waiting for  #2747
vstavrinov: Weird: there was no live stream on weekend but it has emerged now. Does it mean the live stream is not permanent? But nevertheless, for now, the plugin produces the same live stream (really live, not VOD) as the site does. So You can safely merge it.
vstavrinov: > check if the stream is live with `type` or `status`

What for? For debug info and don't play if VOD? It looks like this VOD is the most recent stream that was live before terminated (or maybe digest or the last fragment)
back-to: > What for? ... don't play if VOD?

Yes don't open the Stream if it is a VOD only play it as a Livestream.
vstavrinov: The interesting thing: when you get stream URL and play it directly You can see it is VOD, but streamlink itself has no such indicator. So when You play VOD with streamlink You could think it is live.  Maybe it is worth to add such feature.

Patch:
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index b821f3fa..378222a8 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -43,6 +43,8 @@ jobs:
       - name: Upload coverage data
         continue-on-error: ${{ matrix.continue || false }}
         uses: codecov/codecov-action@v1
+        with:
+          name: os:${{ matrix.os }} py:${{ matrix.python }}
 
   documentation:
     name: Test docs
@@ -52,7 +54,7 @@ jobs:
         with:
           fetch-depth: 300
       - name: Fetch tags
-        run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+        run: git fetch --depth=300 origin +refs/tags/*:refs/tags/*
       - uses: actions/setup-python@v1
         with:
           python-version: 3.8
@@ -71,7 +73,7 @@ jobs:
         with:
           fetch-depth: 300
       - name: Fetch tags
-        run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+        run: git fetch --depth=300 origin +refs/tags/*:refs/tags/*
       - uses: actions/setup-python@v1
         with:
           python-version: 3.8
@@ -106,7 +108,7 @@ jobs:
         with:
           fetch-depth: 300
       - name: Fetch tags
-        run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+        run: git fetch --depth=300 origin +refs/tags/*:refs/tags/*
       - uses: actions/setup-python@v1
         with:
           python-version: 3.8
@@ -132,7 +134,7 @@ jobs:
         with:
           fetch-depth: 300
       - name: Fetch tags
-        run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+        run: git fetch --depth=300 origin +refs/tags/*:refs/tags/*
       - uses: actions/setup-python@v1
         with:
           python-version: 3.8
diff --git a/docs/plugin_matrix.rst b/docs/plugin_matrix.rst
index a088fca0..0d003504 100644
--- a/docs/plugin_matrix.rst
+++ b/docs/plugin_matrix.rst
@@ -126,6 +126,7 @@ mitele                  mitele.es            Yes   No    Streams may be geo-rest
 mixer                   mixer.com            Yes   Yes
 mjunoon                 mjunoon.tv           Yes   Yes
 nbc                     nbc.com              No    Yes   Streams are geo-restricted to USA. Authentication is not supported.
+nbcnews                 nbcnews.com          Yes   No
 nbcsports               nbcsports.com        No    Yes   Streams maybe be geo-restricted to USA. Authentication is not supported.
 nhkworld                nhk.or.jp/nhkworld   Yes   No
 nicolive                live.nicovideo.jp    Yes   Yes   Timeshift is supported. Some content may require login.
diff --git a/src/streamlink/plugins/nbcnews.py b/src/streamlink/plugins/nbcnews.py
new file mode 100644
index 00000000..865219d6
--- /dev/null
+++ b/src/streamlink/plugins/nbcnews.py
@@ -0,0 +1,46 @@
+import logging
+import re
+
+from streamlink.plugin import Plugin
+from streamlink.plugin.api import validate
+from streamlink.stream import HLSStream
+from streamlink.utils import parse_json
+
+log = logging.getLogger(__name__)
+
+
+class NBCNews(Plugin):
+    url_re = re.compile(r'https?://(?:www\.)?nbcnews\.com/now')
+    js_re = re.compile(r'https://ndassets\.s-nbcnews\.com/main-[0-9a-f]{20}\.js')
+    api_re = re.compile(r'NEWS_NOW_PID="([0-9]+)"')
+    api_url = 'https://stream.nbcnews.com/data/live_sources_{0}.json'
+    api_schema = validate.Schema(validate.transform(parse_json), {
+        'videoSources': [{
+            'sourceUrl': validate.url(),
+            'type': validate.text
+        }]
+    }, validate.get('videoSources'), validate.get(0))
+
+    @classmethod
+    def can_handle_url(cls, url):
+        return cls.url_re.match(url) is not None
+
+    def get_title(self):
+        return 'NBC News Now'
+
+    def _get_streams(self):
+        html = self.session.http.get(self.url).text
+        match = self.js_re.search(html)
+        js = self.session.http.get(match.group(0)).text
+        match = self.api_re.search(js)
+        log.debug('API ID: {0}'.format(match.group(1)))
+        api_url = self.api_url.format(match.group(1))
+        stream = self.session.http.get(api_url, schema=self.api_schema)
+        log.trace('{0!r}'.format(stream))
+        if stream['type'].lower() != 'live':
+            log.error('invalid stream type "{0}"'.format(stream['type']))
+            return
+        return HLSStream.parse_variant_playlist(self.session, stream['sourceUrl'])
+
+
+__plugin__ = NBCNews


Test patch:
diff --git a/tests/plugins/test_nbcnews.py b/tests/plugins/test_nbcnews.py
new file mode 100644
index 00000000..174f3dda
--- /dev/null
+++ b/tests/plugins/test_nbcnews.py
@@ -0,0 +1,21 @@
+import unittest
+
+from streamlink.plugins.nbcnews import NBCNews
+
+
+class TestPluginNBCNews(unittest.TestCase):
+    def test_can_handle_url(self):
+        should_match = [
+            'https://www.nbcnews.com/now/',
+            'http://www.nbcnews.com/now/'
+        ]
+        for url in should_match:
+            self.assertTrue(NBCNews.can_handle_url(url))
+
+    def test_can_handle_url_negative(self):
+        should_not_match = [
+            'https://www.nbcnews.com/',
+            'http://www.nbcnews.com/'
+        ]
+        for url in should_not_match:
+            self.assertFalse(NBCNews.can_handle_url(url))