Repository: streamlink/streamlink

Issue:
plugins.vinhlongtv: update backend API URL
<!--
Thanks for opening a pull request!

Before you continue, please make sure that you have read and understood the contribution guidelines, otherwise your changes may be rejected:
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink

If possible, run the tests, perform code linting and build the documentation locally on your system first to avoid unnecessary build failures:
https://streamlink.github.io/latest/developing.html#validating-changes

Also don't forget to add a meaningful description of your changes, so that the reviewing process is as simple as possible for the maintainers.

Thank you very much!
-->
This is the new backend URL to get the links from Vinhlong TV. Slight change.

Proof picture : 
![image](https://user-images.githubusercontent.com/30985701/192102124-2e4af4ca-08ee-4aff-8056-5636810d7dfd.png)



Discussion:
Thanks. Before I'm going to merge this, let me quickly fix some other issues in the plugin file, so I don't have to open another pull request.
My intention was to quickly fix the validation schema definition and the plugin's code style, but apparently there's more to the API URL change.

The site appears to be setting not only the timezone name for accessing streams outside of the target region, but it also sets two additional headers, `x-sfd-date` and `x-sfd-key`. If the timezone or additional headers are not set, then the API response is 403.

```
$ curl -IL 'https://api.thvli.vn/backend/cm/get_detail/thvl1-hd'
HTTP/2 301 
access-control-allow-origin: *
content-type: text/html; charset=utf-8
location: https://api.thvli.vn/backend/cm/get_detail/thvl1-hd/
x-frame-options: SAMEORIGIN
date: Sat, 24 Sep 2022 16:35:47 GMT

HTTP/2 403 
access-control-allow-origin: *
allow: GET, HEAD, OPTIONS
content-type: application/json
x-frame-options: SAMEORIGIN
date: Sat, 24 Sep 2022 16:35:47 GMT
```

```
$ curl -IL 'https://api.thvli.vn/backend/cm/get_detail/thvl1-hd/?timezone=Europe/Paris'
HTTP/2 403 
access-control-allow-origin: *
allow: GET, HEAD, OPTIONS
content-type: application/json
x-frame-options: SAMEORIGIN
date: Sat, 24 Sep 2022 16:46:58 GMT
```

```
$ curl -IL -H 'X-SFD-Date: 20220924184513' -H 'X-SFD-Key: d13957d58ca0ae35568214470629f0b5' 'https://api.thvli.vn/backend/cm/get_detail/thvl1-hd/?timezone=Europe/Paris'
HTTP/2 200 
access-control-allow-origin: *
allow: GET, HEAD, OPTIONS
cache-control: max-age=600
content-type: application/json
expires: Sat, 24 Sep 2022 16:56:41 GMT
last-modified: Sat, 24 Sep 2022 16:46:41 GMT
x-frame-options: SAMEORIGIN
date: Sat, 24 Sep 2022 16:46:41 GMT
```

This is the code responsible for generating the headers, without the signing key part (`I()(h).toString()`)

```js
function(e, t, n) {
    var c = Date.now()
      , a = 0;
    localStorage.getItem("TIME_DIFF") && (a = parseInt(localStorage.getItem("TIME_DIFF"))),
    c -= a;
    var s = new Date(c)
      , i = s.getDate() < 10 ? "0" + s.getDate() : s.getDate().toString()
      , r = s.getMonth() + 1 < 10 ? "0" + (s.getMonth() + 1) : (s.getMonth() + 1).toString()
      , l = s.getFullYear().toString() + r + i
      , o = (s.getHours() < 10 ? "0" + s.getHours() : s.getHours().toString()) + (s.getMinutes() < 10 ? "0" + s.getMinutes() : s.getMinutes().toString()) + (s.getSeconds() < 10 ? "0" + s.getSeconds() : s.getSeconds().toString())
      , d = I()(l + o).toString()
      , h = "Kh0ngDuLieu" + l + "C0R0i" + o + "Kh0aAnT0an" + (d.substring(0, 3) + d.substring(d.length - 3));
    return E()({
        headers: {
            "X-SFD-Key": I()(h).toString(),
            "X-SFD-Date": l + o
        },
        method: e,
        url: "".concat("https://api.thvli.vn/backend/cm/").concat(t),
        data: n
    })
}
```

As you can see, this involves the current date, as well as hardcoded strings in the minified/obfuscated JS. And the resulting string is then "signed" (haven't checked in detail yet), so that the server can check it back and verify the client's signature.

Since I don't have access to a VPN to Vietnam, I don't know if the plugin works from this region. It does not from a different region when the timezone and headers are unset, and the result is a 403 API response.
If we can't get any confirmation that the plugin is working from within Vietnam (with the updated API URL), then it'll have to be removed.

Otherwise, the whole `x-sdf-{date,key}` header logic would need to be re-implemented, in addition to the timezone names. The timezone names are another problem, because Python's standard library doesn't provide methods for getting the human-readable timezone names, unlike the "ECMAScript Internationalization API" in NodeJS or in the browser's DOM via `Intl.DateTimeFormat().resolvedOptions().timeZone`, so adding the `pytz` dependency would be required, which is not ideal.
It is also the only endpoint that is verified by authentification. All the others do not seem to have it.
So, may I ask, did you actually test the plugin yourself, or did you just compare the API URL with your browser's dev tools? Because it looks like the headers are always required.

----

I had another look at it, and apparently the site publishes JS source maps, so here's the non-minified code for making API calls:

```js
const callApi = (method, endpoint, data) => {
  let currentTimestamp = Date.now()
  let timeDiff = 0

  if (localStorage.getItem('TIME_DIFF')) {
    timeDiff = parseInt(localStorage.getItem('TIME_DIFF'))
  }

  currentTimestamp = currentTimestamp - timeDiff

  const date = new Date(currentTimestamp)
  const day = date.getDate() < 10 ? ('0' + date.getDate()) : (date.getDate()).toString()
  const month = (date.getMonth() + 1) < 10 ? ('0' + (date.getMonth() + 1)) : (date.getMonth() + 1).toString()
  const year = date.getFullYear().toString()
  const hour = date.getHours() < 10 ? ('0' + date.getHours()) : (date.getHours()).toString()
  const minute = date.getMinutes() < 10 ? ('0' + date.getMinutes()) : (date.getMinutes()).toString()
  const second = date.getSeconds() < 10 ? ('0' + date.getSeconds()) : (date.getSeconds()).toString()

  const dateValue = year + month + day
  const timeValue = hour + minute + second
  const md5Value = (MD5(dateValue + timeValue)).toString()
  const keyValue = md5Value.substring(0, 3) + md5Value.substring(md5Value.length - 3)

  const keyAccess = process.env.REACT_APP_ACCESS_KEY_DATE + dateValue + process.env.REACT_APP_ACCESS_KEY_TIME + timeValue + process.env.REACT_APP_ACCESS_KEY_SECRET + keyValue

  return axios({
    headers: {
      'X-SFD-Key': MD5(keyAccess).toString(),
      'X-SFD-Date': dateValue + timeValue
    },
    method: method,
    url: `${process.env.REACT_APP_API_URL}${endpoint}`,
    data: data
  })
}
```

with the following env vars set while they've built their production code:

- `REACT_APP_ACCESS_KEY_DATE=Kh0ngDuLieu`
- `REACT_APP_ACCESS_KEY_TIME=C0R0i`
- `REACT_APP_ACCESS_KEY_SECRET=Kh0aAnT0an`
- `REACT_APP_API_URL=https://api.thvli.vn/backend/cm/`

The `MD5` function comes from here and works as expected with byte inputs:
https://github.com/brix/crypto-js/blob/4.1.1/md5.js

However, as said, in order to implement the right API calls, we need human-readable names of the user's timezone, and that requires adding another dependency to Streamlink.

Since the timezone is most likely used for setting the right time on the server for checking the provided MD5 (as it depends on the user's current time), it's possible that generating a hash from the UTC timezone and setting the `Europe/London` timezone string will work regardless of the user's local time.

Resolving patch:
diff --git a/src/streamlink/plugins/vinhlongtv.py b/src/streamlink/plugins/vinhlongtv.py
--- a/src/streamlink/plugins/vinhlongtv.py
+++ b/src/streamlink/plugins/vinhlongtv.py
@@ -7,39 +7,67 @@
 
 import logging
 import re
+from datetime import datetime
+from hashlib import md5
+
+from isodate import UTC  # type: ignore[import]
 
 from streamlink.plugin import Plugin, pluginmatcher
 from streamlink.plugin.api import validate
 from streamlink.stream.hls import HLSStream
 
+
 log = logging.getLogger(__name__)
 
 
 @pluginmatcher(re.compile(
-    r'https?://(?:www\.)?thvli\.vn/live/(?P<channel>[^/]+)'
+    r"https?://(?:www\.)?thvli\.vn/live/(?P<channel>[^/]+)"
 ))
 class VinhLongTV(Plugin):
-    api_url = 'http://api.thvli.vn/backend/cm/detail/{0}/'
+    _API_URL = "https://api.thvli.vn/backend/cm/get_detail/{channel}/"
+    _API_KEY_DATE = "Kh0ngDuLieu"
+    _API_KEY_TIME = "C0R0i"
+    _API_KEY_SECRET = "Kh0aAnT0an"
+
+    def _get_headers(self):
+        now = datetime.now(tz=UTC)
+        date = now.strftime("%Y%m%d")
+        time = now.strftime("%H%M%S")
+        dtstr = f"{date}{time}"
+        dthash = md5(dtstr.encode()).hexdigest()
+        key_value = f"{dthash[:3]}{dthash[-3:]}"
+        key_access = f"{self._API_KEY_DATE}{date}{self._API_KEY_TIME}{time}{self._API_KEY_SECRET}{key_value}"
 
-    _data_schema = validate.Schema(
-        {
-            'link_play': validate.text,
-        },
-        validate.get('link_play')
-    )
+        return {
+            "X-SFD-Date": dtstr,
+            "X-SFD-Key": md5(key_access.encode()).hexdigest(),
+        }
 
     def _get_streams(self):
-        channel = self.match.group('channel')
+        channel = self.match.group("channel")
+        params = {"timezone": "UTC"}
+        headers = self._get_headers()
 
-        res = self.session.http.get(self.api_url.format(channel))
-        hls_url = self.session.http.json(res, schema=self._data_schema)
-        log.debug('URL={0}'.format(hls_url))
+        self.id, self.title, hls_url = self.session.http.get(
+            self._API_URL.format(channel=channel),
+            params=params,
+            headers=headers,
+            schema=validate.Schema(
+                validate.parse_json(),
+                {
+                    "id": str,
+                    "title": str,
+                    "link_play": str,
+                },
+                validate.union_get(
+                    "id",
+                    "title",
+                    "link_play",
+                ),
+            ),
+        )
 
-        streams = HLSStream.parse_variant_playlist(self.session, hls_url)
-        if not streams:
-            return {'live': HLSStream(self.session, hls_url)}
-        else:
-            return streams
+        return HLSStream.parse_variant_playlist(self.session, hls_url)
 
 
 __plugin__ = VinhLongTV


Test patch:
diff --git a/tests/plugins/test_vinhlongtv.py b/tests/plugins/test_vinhlongtv.py
--- a/tests/plugins/test_vinhlongtv.py
+++ b/tests/plugins/test_vinhlongtv.py
@@ -1,3 +1,7 @@
+from unittest.mock import Mock
+
+from freezegun import freeze_time
+
 from streamlink.plugins.vinhlongtv import VinhLongTV
 from tests.plugins import PluginCanHandleUrl
 
@@ -6,7 +10,17 @@ class TestPluginCanHandleUrlVinhLongTV(PluginCanHandleUrl):
     __plugin__ = VinhLongTV
 
     should_match = [
-        'http://thvli.vn/live/thvl1-hd/aab94d1f-44e1-4992-8633-6d46da08db42',
-        'http://thvli.vn/live/thvl2-hd/bc60bddb-99ac-416e-be26-eb4d0852f5cc',
-        'http://thvli.vn/live/phat-thanh/c87174ba-7aeb-4cb4-af95-d59de715464c',
+        "https://www.thvli.vn/live/thvl1-hd",
+        "https://www.thvli.vn/live/thvl2-hd",
+        "https://www.thvli.vn/live/thvl3-hd",
+        "https://www.thvli.vn/live/thvl4-hd",
     ]
+
+
+@freeze_time("2022-09-25T00:04:45Z")
+def test_headers():
+    # noinspection PyUnresolvedReferences
+    assert VinhLongTV(Mock(), "")._get_headers() == {
+        "X-SFD-Date": "20220925000445",
+        "X-SFD-Key": "3507c190ae8befda3bfa8e2c00af3c7a",
+    }