internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: streamlink/streamlink

Issue:
[bug] TVN new links not openable
<!-- Replace [ ] with [x] in order to check the box -->
- [x] This is a bug report and I have read the contribution guidelines.

### Description
TVN.bg has moved its livestream links and the new ones can't be opened by the plugin.

### Reproduction steps / Explicit stream URLs to test
https://play.tvn.bg/TVNweb/embed.html?realtime=true


Discussion:
Site uses websockets now. Requires a plugin rewrite.
static URL
`https://play.tvn.bg/TVNweb/index.m3u8`
That static URL is still working and it's not geo-IP blocked.  Does this mean the tvnbg plugin should be scheduled for removal, @back-to?  Am I right in thinking this plugin is not supporting VODs?  It seems that way...

Resolving patch:
diff --git a/src/streamlink/plugins/tvnbg.py b/src/streamlink/plugins/tvnbg.py
deleted file mode 100644
--- a/src/streamlink/plugins/tvnbg.py
+++ /dev/null
@@ -1,36 +0,0 @@
-import re
-
-from streamlink.plugin import Plugin
-from streamlink.compat import urljoin
-from streamlink.stream import HLSStream
-
-
-class TVNBG(Plugin):
-    url_re = re.compile(r"https?://(?:live\.)?tvn\.bg/(?:live)?")
-    iframe_re = re.compile(r'<iframe.*?src="([^"]+)".*?></iframe>')
-    src_re = re.compile(r'<source.*?src="([^"]+)".*?/>')
-
-    @classmethod
-    def can_handle_url(cls, url):
-        return cls.url_re.match(url) is not None
-
-    def _get_streams(self):
-        base_url = self.url
-        res = self.session.http.get(self.url)
-
-        # Search for the iframe in the page
-        iframe_m = self.iframe_re.search(res.text)
-        if iframe_m:
-            # If the iframe is found, load the embedded page
-            base_url = iframe_m.group(1)
-            res = self.session.http.get(iframe_m.group(1))
-
-        # Search the page (original or embedded) for the stream URL
-        src_m = self.src_re.search(res.text)
-        if src_m:
-            stream_url = urljoin(base_url, src_m.group(1))
-            # There is no variant playlist, only a plain HLS Stream
-            yield "live", HLSStream(self.session, stream_url)
-
-
-__plugin__ = TVNBG


Test patch:
diff --git a/tests/plugins/test_tvnbg.py b/tests/plugins/test_tvnbg.py
deleted file mode 100644
--- a/tests/plugins/test_tvnbg.py
+++ /dev/null
@@ -1,19 +0,0 @@
-import unittest
-
-from streamlink.plugins.tvnbg import TVNBG
-
-
-class TestPluginTVNBG(unittest.TestCase):
-    def test_can_handle_url(self):
-        should_match = [
-            'https://tvn.bg/live/',
-        ]
-        for url in should_match:
-            self.assertTrue(TVNBG.can_handle_url(url))
-
-    def test_can_handle_url_negative(self):
-        should_not_match = [
-            'https://example.com/index.html',
-        ]
-        for url in should_not_match:
-            self.assertFalse(TVNBG.can_handle_url(url))
Repository: streamlink/streamlink

Issue:
Unable to play bloomberg with streamlink - No playable streams found on this URL
<!--
Thanks for reporting a plugin issue!
USE THE TEMPLATE. Otherwise your plugin issue may be rejected.

First, see the contribution guidelines:
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink

Also check the list of open and closed plugin issues:
https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22

Please see the text preview to avoid unnecessary formatting errors.
-->


## Plugin Issue

<!-- Replace [ ] with [x] in order to check the box -->
- [X] This is a plugin issue and I have read the contribution guidelines.


### Description

<!-- Explain the plugin issue as thoroughly as you can. -->
I am attempting to use streamlink to play bloomberg live.  However, when attempting to load the URL https://www.bloomberg.com/live/us I receive an error stating there are no playable streams found on this URL.  

### Reproduction steps / Explicit stream URLs to test

<!-- How can we reproduce this? Please note the exact steps below using the list format supplied. If you need more steps please add them. -->

1. I am running this command: streamlink https://www.bloomberg.com/live/us

### Log output

<!--
TEXT LOG OUTPUT IS REQUIRED for a plugin issue!
Use the `--loglevel debug` parameter and avoid using parameters which suppress log output.
https://streamlink.github.io/cli.html#cmdoption-l

Make sure to **remove usernames and passwords**
You can copy the output to https://gist.github.com/ or paste it below.
-->

```
C:\Windows\system32>streamlink https://www.bloomberg.com/live/us -l debug
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.6.6
[cli][debug] Streamlink: 1.5.0
[cli][debug] Requests(2.24.0), Socks(1.7.1), Websocket(0.57.0)
[cli][info] Found matching plugin bloomberg for URL https://www.bloomberg.com/live/us
[plugin.bloomberg][debug] Found liveId: PHOENIX_US
[plugin.bloomberg][debug] Player URL: https://cdn.gotraffic.net/projector/v0.11.192/app.js
[plugin.bloomberg][debug] Webpack module ID: 203
error: No playable streams found on this URL: https://www.bloomberg.com/live/us
```

### Additional comments, screenshots, etc.



[Love Streamlink? Please consider supporting our collective. Thanks!](https://opencollective.com/streamlink/donate)



Discussion:

Resolving patch:
diff --git a/src/streamlink/plugins/bloomberg.py b/src/streamlink/plugins/bloomberg.py
--- a/src/streamlink/plugins/bloomberg.py
+++ b/src/streamlink/plugins/bloomberg.py
@@ -37,7 +37,7 @@ class Bloomberg(Plugin):
     _live_stream_info_module_id_re = re.compile(
         r'{(?:(?:"[^"]+"|\w+):(?:\d+|"[^"]+"),)*"(?:[^"]*/)?config/livestreams":(\d+)(?:,(?:"[^"]+"|\w+):(?:\d+|"[^"]+"))*}'
     )
-    _live_stream_info_re = r'],{id}:\[function\(.+var n=({{.+}});r.default=n}},{{"[^"]+":\d+}}],{next}:\['
+    _live_stream_info_re = r'],{id}:\[function\(.+var n=({{.+}});r.default=n}},{{}}],{next}:\['
 
     _live_streams_schema = validate.Schema(
         validate.transform(_js_to_json_re),


Test patch:
Repository: streamlink/streamlink

Issue:
tv360.com.tr no playable stream
## Bug Report
- [x] This is a bug report and I have read the contribution guidelines.

### Description

can't find playable stream.

### Expected / Actual behavior

stream supposed to be found

### Reproduction steps / Explicit stream URLs to test

``` 1. streamlink https://www.tv360.com.tr/canli-yayin ```

### Log output

```
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.8.2
[cli][debug] Streamlink: 1.5.0
[cli][debug] Requests(2.24.0), Socks(1.7.1), Websocket(0.57.0)
[cli][info] Found matching plugin tv360 for URL tv360.com.tr/canli-yayin
error: No playable streams found on this URL: tv360.com.tr/canli-yayin
```


### Additional comments, screenshots, etc.



[Love Streamlink? Please consider supporting our collective. Thanks!](https://opencollective.com/streamlink/donate)



Discussion:

Resolving patch:
diff --git a/src/streamlink/plugins/tv360.py b/src/streamlink/plugins/tv360.py
--- a/src/streamlink/plugins/tv360.py
+++ b/src/streamlink/plugins/tv360.py
@@ -1,5 +1,3 @@
-from __future__ import print_function
-
 import re
 
 from streamlink.plugin import Plugin
@@ -9,11 +7,11 @@
 
 class TV360(Plugin):
     url_re = re.compile(r"https?://(?:www.)?tv360.com.tr/canli-yayin")
-    hls_re = re.compile(r'''hls.loadSource\(["'](http.*m3u8)["']\)''', re.DOTALL)
+    hls_re = re.compile(r'''src="(http.*m3u8)"''')
 
     hls_schema = validate.Schema(
         validate.transform(hls_re.search),
-        validate.any(None, validate.all(validate.get(1)))
+        validate.any(None, validate.all(validate.get(1), validate.url()))
     )
 
     @classmethod
@@ -21,11 +19,10 @@ def can_handle_url(cls, url):
         return cls.url_re.match(url) is not None
 
     def _get_streams(self):
-        res = self.session.http.get(self.url)
-        hls_url = self.hls_re.search(res.text)
+        hls_url = self.session.http.get(self.url, schema=self.hls_schema)
 
         if hls_url:
-            return HLSStream.parse_variant_playlist(self.session, hls_url.group(1))
+            return HLSStream.parse_variant_playlist(self.session, hls_url)
 
 
 __plugin__ = TV360


Test patch:
Repository: streamlink/streamlink

Issue:
[New Plugin] Add TV999
## Plugin Request
- [x] This is a plugin request and I have read the contribution guidelines and plugin request requirements.

### Description
Bulgarian TV channel that mainly features Russian content with Bulgarian subtitles.

### Example stream URLs
http://tv999.bg/live.html


Discussion:

Resolving patch:
diff --git a/src/streamlink/plugins/tv999.py b/src/streamlink/plugins/tv999.py
new file mode 100644
--- /dev/null
+++ b/src/streamlink/plugins/tv999.py
@@ -0,0 +1,51 @@
+import logging
+import re
+
+from streamlink.plugin import Plugin
+from streamlink.plugin.api import validate
+from streamlink.stream import HLSStream
+from streamlink.utils.url import update_scheme
+
+log = logging.getLogger(__name__)
+
+
+class TV999(Plugin):
+    url_re = re.compile(r'https?://(?:www\.)?tv999\.bg/live\.html')
+    iframe_re = re.compile(r'<iframe.*src="([^"]+)"')
+    hls_re = re.compile(r'src="([^"]+)"\s+type="application/x-mpegURL"')
+
+    iframe_schema = validate.Schema(
+        validate.transform(iframe_re.search),
+        validate.any(None, validate.all(
+            validate.get(1),
+            validate.url(),
+        )),
+    )
+
+    hls_schema = validate.Schema(
+        validate.transform(hls_re.search),
+        validate.any(None, validate.all(
+            validate.get(1),
+            validate.transform(lambda x: update_scheme('http:', x)),
+            validate.url(),
+        )),
+    )
+
+    @classmethod
+    def can_handle_url(cls, url):
+        return cls.url_re.match(url) is not None
+
+    def _get_streams(self):
+        iframe_url = self.session.http.get(self.url, schema=self.iframe_schema)
+
+        if not iframe_url:
+            log.error('Failed to find IFRAME URL')
+            return
+
+        hls_url = self.session.http.get(iframe_url, schema=self.hls_schema)
+
+        if hls_url:
+            return {'live': HLSStream(self.session, hls_url)}
+
+
+__plugin__ = TV999


Test patch:
diff --git a/tests/plugins/test_tv999.py b/tests/plugins/test_tv999.py
new file mode 100644
--- /dev/null
+++ b/tests/plugins/test_tv999.py
@@ -0,0 +1,24 @@
+import unittest
+
+from streamlink.plugins.tv999 import TV999
+
+
+class TestPluginTV999(unittest.TestCase):
+    def test_can_handle_url(self):
+        should_match = [
+            'http://tv999.bg/live.html',
+            'http://www.tv999.bg/live.html',
+            'https://tv999.bg/live.html',
+            'https://www.tv999.bg/live.html',
+        ]
+        for url in should_match:
+            self.assertTrue(TV999.can_handle_url(url))
+
+    def test_can_handle_url_negative(self):
+        should_not_match = [
+            'http://tv999.bg/',
+            'https://tv999.bg/live',
+            'https://example.com/index.html',
+        ]
+        for url in should_not_match:
+            self.assertFalse(TV999.can_handle_url(url))
Repository: streamlink/streamlink

Issue:
Change `author_email` in setup.py
https://github.com/streamlink/streamlink/blob/08e582580f3411b2de2c368f8b0cc7108264f990/setup.py#L83

@gravyboat 
you've registered `streamlink@protonmail.com` a couple of years ago, right? Can this be used instead?

What's the email address of the `streamlink` account on pypi?
https://pypi.org/user/streamlink/


Discussion:
@bastimeyer Yes `streamlink@protonmail.com` is registered and can be used. I believe @beardypig also has access to this address. I thought I sent you the password as well but maybe not?

For the `streamlink` repo on pypi there is a `streamlink` user that has the same protonmail address, but that user doesn't have direct manage access to the pypi project since it's shared, they're just a maintainer. It was created by @beardypig. Cdrage and I both have owner access. We should get you added on there as well.
> can be used

Open a PR then.

> I thought I sent you the password as well but maybe not?

No, I don't have access to any Streamlink accounts.

> We should get you added on there as well.

Not really important. I've created an acc today though, so if you want to add me, you can.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -80,7 +80,7 @@ def is_wheel_for_windows():
       },
       author="Streamlink",
       # temp until we have a mailing list / global email
-      author_email="charlie@charliedrage.com",
+      author_email="streamlink@protonmail.com",
       license="Simplified BSD",
       packages=find_packages("src"),
       package_dir={"": "src"},


Test patch:
Repository: streamlink/streamlink

Issue:
Turkuvaz Plugin missing Streams
Hi,

first of all to be sure installed Streamlink 2.0.0 via Python3 again.

After that tested all streams one by one with the turkuvaz.py

Most of them are working, only 2 of 9 channels missing, "error: No plugin can handle URL"

A2 and A Haber TV:

https://www.atv.com.tr/a2tv/canli-yayin
https://www.ahaber.com.tr/video/canli-yayin




Discussion:
Probably just needs a regex update. I'll take a look at it tomorrow if nobody else does before.

Resolving patch:
diff --git a/src/streamlink/plugins/turkuvaz.py b/src/streamlink/plugins/turkuvaz.py
--- a/src/streamlink/plugins/turkuvaz.py
+++ b/src/streamlink/plugins/turkuvaz.py
@@ -20,6 +20,10 @@ class Turkuvaz(Plugin):
             |
             (atv|a2tv|ahaber|aspor|minikago|minikacocuk|anews)\.com\.tr
         )/webtv/(?:live-broadcast|canli-yayin)
+    |
+        (ahaber)\.com\.tr/video/canli-yayin
+    |
+        atv\.com\.tr/(a2tv)/canli-yayin
     |
         sabah\.com\.tr/(apara)/canli-yayin
     )""")
@@ -39,7 +43,7 @@ def can_handle_url(cls, url):
 
     def _get_streams(self):
         url_m = self._url_re.match(self.url)
-        domain = url_m.group(1) or url_m.group(2) or url_m.group(3)
+        domain = url_m.group(1) or url_m.group(2) or url_m.group(3) or url_m.group(4) or url_m.group(5)
         # remap the domain to channel
         channel = {"atv": "atvhd",
                    "ahaber": "ahaberhd",


Test patch:
diff --git a/tests/plugins/test_turkuvaz.py b/tests/plugins/test_turkuvaz.py
--- a/tests/plugins/test_turkuvaz.py
+++ b/tests/plugins/test_turkuvaz.py
@@ -6,8 +6,12 @@
 class TestPluginTurkuvaz(unittest.TestCase):
     def test_can_handle_url(self):
         should_match = [
+            'http://www.atv.com.tr/a2tv/canli-yayin',
+            'https://www.atv.com.tr/a2tv/canli-yayin',
             'https://www.atv.com.tr/webtv/canli-yayin',
             'http://www.a2tv.com.tr/webtv/canli-yayin',
+            'http://www.ahaber.com.tr/video/canli-yayin',
+            'https://www.ahaber.com.tr/video/canli-yayin',
             'https://www.ahaber.com.tr/webtv/canli-yayin',
             'https://www.aspor.com.tr/webtv/canli-yayin',
             'http://www.anews.com.tr/webtv/live-broadcast',
Repository: streamlink/streamlink

Issue:
Dogus.py missing stream
https://www.ntvspor.net/ntvspor-canli-yayin

This is normally handled by dogus.py but it seems to be outdated. The other streams are working ok only this one.


Discussion:
I'll take a look tomorrow.
@Rotarum Make sure to provide all the information required by the template. You didn't here, you didn't on https://github.com/streamlink/streamlink/issues/3477, and you didn't on https://github.com/streamlink/streamlink/issues/3475. You've also been commenting multiple times on closed issues generating unnecessary noise. If this continues I'll have to ban you from the repo in the future. This is an open source project and you're doing less than the bare minimum to help the people trying to fix issues and not contributing in a positive way.

Resolving patch:
diff --git a/src/streamlink/plugins/dogus.py b/src/streamlink/plugins/dogus.py
--- a/src/streamlink/plugins/dogus.py
+++ b/src/streamlink/plugins/dogus.py
@@ -12,16 +12,15 @@
 
 class Dogus(Plugin):
     """
-    Support for live streams from Dogus sites include ntv, ntvspor, and kralmuzik
+    Support for live streams from Dogus sites include ntv, and kralmuzik
     """
 
-    url_re = re.compile(r"""https?://(?:www.)?
+    url_re = re.compile(r"""https?://(?:www\.)?
         (?:
-            ntv.com.tr/canli-yayin/ntv|
-            ntvspor.net/canli-yayin|
-            kralmuzik.com.tr/tv/|
-            eurostartv.com.tr/canli-izle|
-            startv.com.tr/canli-yayin
+            eurostartv\.com\.tr/canli-izle|
+            kralmuzik\.com\.tr/tv/|
+            ntv\.com\.tr/canli-yayin/ntv|
+            startv\.com\.tr/canli-yayin
         )/?""", re.VERBOSE)
     mobile_url_re = re.compile(r"""(?P<q>["'])(?P<url>(https?:)?//[^'"]*?/live/hls/[^'"]*?\?token=)
                                    (?P<token>[^'"]*?)(?P=q)""", re.VERBOSE)


Test patch:
diff --git a/tests/plugins/test_dogus.py b/tests/plugins/test_dogus.py
--- a/tests/plugins/test_dogus.py
+++ b/tests/plugins/test_dogus.py
@@ -6,8 +6,9 @@
 class TestPluginDogus(unittest.TestCase):
     def test_can_handle_url(self):
         should_match = [
-            'http://www.ntvspor.net/canli-yayin',
             'http://eurostartv.com.tr/canli-izle',
+            'http://kralmuzik.com.tr/tv/',
+            'http://ntv.com.tr/canli-yayin/ntv',
             'http://startv.com.tr/canli-yayin',
         ]
         for url in should_match:
@@ -16,6 +17,7 @@ def test_can_handle_url(self):
     def test_can_handle_url_negative(self):
         should_not_match = [
             'https://example.com/index.html',
+            'http://www.ntvspor.net/canli-yayin',
         ]
         for url in should_not_match:
             self.assertFalse(Dogus.can_handle_url(url))
Repository: streamlink/streamlink

Issue:
plugins.bbciplayer: Not working
<!--
Thanks for reporting a plugin issue!
USE THE TEMPLATE. Otherwise your plugin issue may be rejected.

First, see the contribution guidelines:
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink

Plugin issues describe broken functionality within a plugin's code base, eg. the streaming site has made breaking changes, streams don't get resolved correctly, or authentication has stopped working, etc.

Also check the list of open and closed plugin issues:
https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22

Please see the text preview to avoid unnecessary formatting errors.
-->


## Plugin Issue

<!-- Replace the space character between the square brackets with an x in order to check the boxes -->
- [ x] This is a plugin issue and I have read the contribution guidelines.
- [ x] I am using the latest development version from the master branch.


### Description

If I try to run the plugin, for example with streamlink -p "C:\ProgramData\chocolatey\lib\mpv.install\tools\mpv" --bbciplayer-username="*USERNAME HERE*" --bbciplayer-password="*PASSWORD HERE*" bbc.co.uk/iplayer/live/bbcparliament 288p_dash, I receive the following:

```
error: Unable to parse JSON: Extra data: line 1 column 23991 (char 23990) ('{"navigation":{"items":[{"id":"cha ...)
```

### Reproduction steps / Explicit stream URLs to test

I updated Streamlink to 2.1.0 to try and resolve, but that didn't work.

1. Running using the command posted above


### Additional comments, etc.

It seems to error here: https://github.com/streamlink/streamlink/blob/master/src/streamlink/plugins/bbciplayer.py#L114

I am using Windows 10

I presume it may be an issue with the JSON being returned by BBC iPlayer, as there is a semicolon after a closing bracket?


Discussion:

Resolving patch:
diff --git a/src/streamlink/plugins/bbciplayer.py b/src/streamlink/plugins/bbciplayer.py
--- a/src/streamlink/plugins/bbciplayer.py
+++ b/src/streamlink/plugins/bbciplayer.py
@@ -28,7 +28,7 @@ class BBCiPlayer(Plugin):
     """, re.VERBOSE)
     mediator_re = re.compile(
         r'window\.__IPLAYER_REDUX_STATE__\s*=\s*({.*?});', re.DOTALL)
-    state_re = re.compile(r'window.__IPLAYER_REDUX_STATE__\s*=\s*({.*});')
+    state_re = re.compile(r'window.__IPLAYER_REDUX_STATE__\s*=\s*({.*?});</script>')
     account_locals_re = re.compile(r'window.bbcAccount.locals\s*=\s*({.*?});')
     hash = base64.b64decode(b"N2RmZjc2NzFkMGM2OTdmZWRiMWQ5MDVkOWExMjE3MTk5MzhiOTJiZg==")
     api_url = "https://open.live.bbc.co.uk/mediaselector/6/select/version/2.0/mediaset/" \


Test patch:
Repository: streamlink/streamlink

Issue:
Add lxml dependency
### Checklist

- [X] This is a feature request and not a different kind of issue
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed plugin requests](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22feature+request%22)

### Description

Streamlink should finally switch to a proper HTML/XML parser for extracting data instead of using cheap regex workarounds which don't work properly. I've already commented on this issue last year:
https://github.com/streamlink/streamlink/issues/3241#issuecomment-706486239

The reason why I'm suggesting this again right now is that I was trying to fix the deutschewelle plugin (https://dw.com) yesterday and ran into issues with the `itertags` utility method, which is based on simple regexes for iterating HTML nodes and their attributes+body. `itertags` for example does not work with nested nodes, which makes adding ridiculous custom regexes necessary. Just take a look at this madness:
https://github.com/streamlink/streamlink/blob/3668770d608f0fab54d40a46acd6720a97f63775/src/streamlink/plugins/deutschewelle.py#L18-L29

With `lxml` (https://lxml.de/), HTML page contents can be parsed and the data extracted via XPath queries and/or the respective API methods. The methods are similar to python's native `xml.etree.ElementTree`, which itself is considered too slow and unsafe in certain cases. I am by no means an expert regarding python's standard library though, so if someone has better insight here, please share. In regards to packaging, this lib is available on basically every packaging system and adding it as a dependency here only has benefits.

I'd suggest that we add `lxml` as a dependency now and start using it for extracting data from HTML documents. The validation schema methods could be improved for this as well. There's also the `parse_xml` utility method, which is currently based on the native module.

Comments?


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -12,6 +12,7 @@
 deps = [
     "requests>=2.26.0,<3.0",
     "isodate",
+    "lxml>=4.6.3",
     "websocket-client>=0.58.0",
     # Support for SOCKS proxies
     "PySocks!=1.5.7,>=1.5.6",


Test patch: