Repository: streamlink/streamlink

Issue:
webbrowser: Cloak headless Chrome user agent string
### Checklist

- [X] This is a feature request and not [a different kind of issue](https://github.com/streamlink/streamlink/issues/new/choose)
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed feature requests](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22feature+request%22)

### Collaboration

- [X] My request is genuine and in the best interest of the project
- [X] [I will provide feedback should a pull request be opened with a new feature implementation](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#pull-request-feedback)

### Description

Since a few months ago, headless Chrome reports itself using the `HeadlessChrome` user agent, I believe this is what caused sites like Twitch to stop working in headless mode (https://github.com/streamlink/streamlink/issues/5600). I believe it may also affect recent additions like the [Amazon WAF tokens](https://github.com/streamlink/streamlink/pull/6102).

The current user agent for headless Chrome (on Windows 10) is 
`Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.0.0 Safari/537.36`, 
while non-headless is 
`Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36`. Simply replacing "HeadlessChrome" with "Chrome" seems to work.

Also see:
https://github.com/seleniumbase/SeleniumBase/issues/2106
https://github.com/seleniumbase/SeleniumBase/issues/2523
https://github.com/seleniumbase/SeleniumBase/pull/2524
https://github.com/seleniumbase/SeleniumBase/blob/feca61a3c283858c185596ef3d0941e519c2be31/seleniumbase/core/browser_launcher.py#L4183-L4203

Discussion:
Hakkin: Tested this with the `vtvgo` plugin:
Before, git master patched to set `headless=True` on AWSWAF:
```
[plugins.vtvgo][info] Getting new AWS WAF token (bot-detection)
[webbrowser.webbrowser][info] Launching web browser: /usr/bin/chromium
[webbrowser.webbrowser][warning] Web browser task group has timed out
[plugin.api.aws_waf][error] No AWS WAF token has been acquired
```

After:
```
[cli][info] Found matching plugin vtvgo for URL https://vtvgo.vn/xem-truc-tuyen-kenh-vtv3-3.html
[plugins.vtvgo][info] Getting new AWS WAF token (bot-detection)
[webbrowser.webbrowser][info] Launching web browser: /usr/bin/chromium
[cli][info] Available streams: 480p (worst), 720p (best)
```

Works well for me with chromium at least.
bastimeyer: > Works well for me with chromium at least.

Well, yes, that's what I'm using here locally. I asked for Chromium forks, because that's what I currently don't know yet. Thanks for testing though. Btw, the PR was just updated again, and now it doesn't set the default UA, which is firefox. This required some module changes in #6112.
Hakkin: Works on Brave as well.

```
[plugins.vtvgo][info] Getting new AWS WAF token (bot-detection)
[webbrowser.webbrowser][info] Launching web browser: /usr/bin/brave
[cli][info] Available streams: 480p (worst), 720p (best)
```
Hakkin: Microsoft Edge (on Linux).

```
[plugins.vtvgo][info] Getting new AWS WAF token (bot-detection)
[webbrowser.webbrowser][info] Launching web browser: /usr/bin/microsoft-edge-stable
[cli][info] Available streams: 480p (worst), 720p (best)
```

Opera
```
[plugins.vtvgo][info] Getting new AWS WAF token (bot-detection)
[webbrowser.webbrowser][info] Launching web browser: /usr/bin/opera
[cli][info] Available streams: 480p (worst), 720p (best)
```
bastimeyer: Thanks. Your help is appreciated.

I'm back now and quickly checked my Windows VM. MSEdge and Brave work, Opera and Vivaldi don't, but that's an issue with the CDP bridge and not the `--user-agent` launch arg.

Going to merge this now.

Patch:
diff --git a/src/streamlink/plugin/api/aws_waf.py b/src/streamlink/plugin/api/aws_waf.py
index d9594dce..fd7e84f4 100644
--- a/src/streamlink/plugin/api/aws_waf.py
+++ b/src/streamlink/plugin/api/aws_waf.py
@@ -24,9 +24,8 @@ class AWSWAF:
     TOKEN = "aws-waf-token"
     EXPIRATION = 3600 * 24 * 4
 
-    def __init__(self, session: Streamlink, headless: bool = False):
+    def __init__(self, session: Streamlink):
         self.session = session
-        self.headless = headless
 
     def acquire(self, url: str) -> bool:
         send: trio.MemorySendChannel[Optional[str]]
@@ -63,11 +62,7 @@ class AWSWAF:
                         return await receive.receive()
 
         try:
-            data = CDPClient.launch(
-                self.session,
-                acquire_token,
-                headless=self.headless,
-            )
+            data = CDPClient.launch(self.session, acquire_token)
         except BaseExceptionGroup:
             log.exception("Failed acquiring AWS WAF token")
         except Exception as err:
diff --git a/src/streamlink/webbrowser/chromium.py b/src/streamlink/webbrowser/chromium.py
index 6ce9f948..82dde51e 100644
--- a/src/streamlink/webbrowser/chromium.py
+++ b/src/streamlink/webbrowser/chromium.py
@@ -8,6 +8,7 @@ import trio
 from streamlink.compat import is_darwin, is_win32
 from streamlink.plugin.api import validate
 from streamlink.session import Streamlink
+from streamlink.session.http_useragents import CHROME
 from streamlink.utils.socket import find_free_port_ipv4, find_free_port_ipv6
 from streamlink.webbrowser.webbrowser import Webbrowser
 
@@ -151,6 +152,8 @@ class ChromiumWebbrowser(Webbrowser):
         self.port = port
         if headless:
             self.arguments.append("--headless=new")
+            # When in headless mode, a headless hint is added to the User-Agent header, which we want to avoid
+            self.arguments.append(f"--user-agent={CHROME}")
 
     @asynccontextmanager
     async def launch(self, timeout: Optional[float] = None) -> AsyncGenerator[trio.Nursery, None]:


Test patch:
diff --git a/tests/webbrowser/test_chromium.py b/tests/webbrowser/test_chromium.py
index cacece58..e516c813 100644
--- a/tests/webbrowser/test_chromium.py
+++ b/tests/webbrowser/test_chromium.py
@@ -11,6 +11,7 @@ from requests import Timeout
 from streamlink.compat import is_win32
 from streamlink.exceptions import PluginError
 from streamlink.session import Streamlink
+from streamlink.session.http_useragents import CHROME
 from streamlink.webbrowser.chromium import ChromiumWebbrowser
 from streamlink.webbrowser.exceptions import WebbrowserError
 
@@ -107,6 +108,7 @@ class TestLaunchArgs:
     def test_headless(self, headless: bool):
         webbrowser = ChromiumWebbrowser(headless=headless)
         assert ("--headless=new" in webbrowser.arguments) is headless
+        assert (f"--user-agent={CHROME}" in webbrowser.arguments) is headless
 
 
 @pytest.mark.trio()