Repository: python-telegram-bot/python-telegram-bot

Issue:
Native support for TGs test environment
By now, TGs [dedicated test environment](https://core.telegram.org/bots/features#dedicated-test-environment) is more prominently documented. Adding native support for it in PTB could be a desribale feature.

## ⚠️  Current status

This feature is currently not on any roadmap, since it doesn't seem to be widely used. If you would like for PTB to have this feature, please react with a 👍 to the comment. If we see a large number of requests, this issue will likely get put on a roadmap.

Edit: One can trick PTB into using the test env by setting `token='<token>/test'`. But this hack is not guaranteed to keep working in future versions of PTB …

Count of thumbs up over time, status 2024-12-31:

![Image](https://github.com/user-attachments/assets/8b48070c-6379-4330-a336-9d4a351be2a0)

<details><summary>For future reference: GraphQL query to get the data</summary>

```gql
{
  repository(owner: "python-telegram-bot", name: "python-telegram-bot") {
    issue(number: 3355) {
      reactions(first: 100) {
        nodes{
          content
          createdAt
          user {
            login
          }
        }
      }
    }
  }
}
```

</details>

## Open Questions

We should check how the url returned by `get_file` behave, i.e. if they are still `https://api.telegram.org/file/bot<Token>/…` or if they are e.g. `https://api.telegram.org/file/bot<token>/test/…`.

## Implementation options

Just quickly listing two options that come to my mind

### A `test_env` flag

That would work via `ApplicationBuilder().token(…).test_env(True)` or `Bot(…, test_env=True)` . This would be non-breaking. However, if may not play nicely with custom `base_file_url` depending on what we find out about file urls.

### Allow custom insertion of token into `base(_file)_url`

That could either work by requesting the user to pass something like `base_url = 'api.telegram.org/bot{token}/'` (this would be a breaking change) or by allowing the user to pass a `Callable[[str], str]` that accepts the token and returns the (file_)url (this could be implemented in a non-breaking manner)




Discussion:
It's currently the only way to test topics without having 200 members in a test group
> It's currently the only way to test topics without having 200 members in a test group

As of Telegram 9.3 (Dec 30, 2022), we can now make a group of any size into forums.
For future reference: A first draft PR was already made in #3972 and if we ever decide to go ahead with this, a PR may possibly build up on that PR.
I've started implementing payments in telegram and this feature might be more and more requested in order to test payments in the test environment.
@OlegNovosad if you can answer the open question and ideally get TG to also document the result, I'm open to take up discussions on this again :)
Using the test environment is the only way to test payments for digital goods & services (using TG stars) AFAIU. 

Testing (& receiving payments) is a very important feature for many bot projects.

@Bibo-Joshi in regards to the open question. The [docs](https://core.telegram.org/bots/features#testing-your-bot) indicate that `file_id` is unique for the different bots/environments. I read this as the url being `https://api.telegram.org/file/bot<Token>/…`
Hi @maschlr 

>  I read this as

That is the problem though ;) As validation that the url is indeed working for the test environment, I would need to see e.g. an exceprt of two curl commands, one for a `getFile` request to the test environment and one for downloading the file with the corresponding ID. If you can provide this (and as indicated above, ideally get TG to add documentation about it), I'm happy to continue the discussion.
I would. It seems as if you [need an iOS device to create a TG account in the test environment](https://core.telegram.org/bots/features#creating-a-bot-in-the-test-environment) that can then be imported into the desktop app. Unfortunately, I don't have access to an iOS device :smirk_cat: 


@maschlr Mh, for the TG Desktop Variant, it looks like it&#x27;s available independently of the OS ...
@Bibo-Joshi And android beta also has test backend, but since this is not official released they don&#x27;t document it. A search engine will give you a link though.
> @maschlr Mh, for the TG Desktop Variant, it looks like it&#x27;s available independently of the OS ...

Yeah I tried it according to the docs. After choosing "Test Server" on the desktop app, it wants me to scan a QR with my mobile device (Android). There, I get "invalid token". 

Debug menu on mobile also has no test environment option...
@maschlr log in using phone number. Scroll down on this page - https://github.com/telegramdesktop/tdesktop/wiki/Cheat-Codes for more info on test backend. My personal phone number didn't work for me, but the test one did. 

Also if you have an alternative android Telegram client, like nekogram or exteragram, you can use the test server on them too.



Patch:
diff --git a/telegram/_bot.py b/telegram/_bot.py
index a47e7c2137c..6cba7dbf803 100644
--- a/telegram/_bot.py
+++ b/telegram/_bot.py
@@ -96,7 +96,14 @@
 from telegram._utils.logging import get_logger
 from telegram._utils.repr import build_repr_with_selected_attrs
 from telegram._utils.strings import to_camel_case
-from telegram._utils.types import CorrectOptionID, FileInput, JSONDict, ODVInput, ReplyMarkup
+from telegram._utils.types import (
+    BaseUrl,
+    CorrectOptionID,
+    FileInput,
+    JSONDict,
+    ODVInput,
+    ReplyMarkup,
+)
 from telegram._utils.warnings import warn
 from telegram._webhookinfo import WebhookInfo
 from telegram.constants import InlineQueryLimit, ReactionEmoji
@@ -126,6 +133,35 @@
 BT = TypeVar("BT", bound="Bot")
 
 
+# Even though we document only {token} as supported insertion, we are a bit more flexible
+# internally and support additional variants. At the very least, we don't want the insertion
+# to be case sensitive.
+_SUPPORTED_INSERTIONS = {"token", "TOKEN", "bot_token", "BOT_TOKEN", "bot-token", "BOT-TOKEN"}
+_INSERTION_STRINGS = {f"{{{insertion}}}" for insertion in _SUPPORTED_INSERTIONS}
+
+
+class _TokenDict(dict):
+    __slots__ = ("token",)
+
+    # small helper to make .format_map work without knowing which exact insertion name is used
+    def __init__(self, token: str):
+        self.token = token
+        super().__init__()
+
+    def __missing__(self, key: str) -> str:
+        if key in _SUPPORTED_INSERTIONS:
+            return self.token
+        raise KeyError(f"Base URL string contains unsupported insertion: {key}")
+
+
+def _parse_base_url(value: BaseUrl, token: str) -> str:
+    if callable(value):
+        return value(token)
+    if any(insertion in value for insertion in _INSERTION_STRINGS):
+        return value.format_map(_TokenDict(token))
+    return value + token
+
+
 class Bot(TelegramObject, contextlib.AbstractAsyncContextManager["Bot"]):
     """This object represents a Telegram Bot.
 
@@ -193,8 +229,40 @@ class Bot(TelegramObject, contextlib.AbstractAsyncContextManager["Bot"]):
 
     Args:
         token (:obj:`str`): Bot's unique authentication token.
-        base_url (:obj:`str`, optional): Telegram Bot API service URL.
+        base_url (:obj:`str` | Callable[[:obj:`str`], :obj:`str`], optional): Telegram Bot API
+            service URL. If the string contains ``{token}``, it will be replaced with the bot's
+            token. If a callable is passed, it will be called with the bot's token as the only
+            argument and must return the base URL. Otherwise, the token will be appended to the
+            string. Defaults to ``"https://api.telegram.org/bot"``.
+
+            Tip:
+                Customizing the base URL can be used to run a bot against
+                :wiki:`Local Bot API Server <Local-Bot-API-Server>` or using Telegrams
+                `test environment \
+                <https://core.telegram.org/bots/features#dedicated-test-environment>`_.
+
+                Example:
+                    ``"https://api.telegram.org/bot{token}/test"``
+
+            .. versionchanged:: NEXT.VERSION
+               Supports callable input and string formatting.
         base_file_url (:obj:`str`, optional): Telegram Bot API file URL.
+            If the string contains ``{token}``, it will be replaced with the bot's
+            token. If a callable is passed, it will be called with the bot's token as the only
+            argument and must return the base URL. Otherwise, the token will be appended to the
+            string. Defaults to ``"https://api.telegram.org/bot"``.
+
+            Tip:
+                Customizing the base URL can be used to run a bot against
+                :wiki:`Local Bot API Server <Local-Bot-API-Server>` or using Telegrams
+                `test environment \
+                <https://core.telegram.org/bots/features#dedicated-test-environment>`_.
+
+                Example:
+                    ``"https://api.telegram.org/file/bot{token}/test"``
+
+            .. versionchanged:: NEXT.VERSION
+               Supports callable input and string formatting.
         request (:class:`telegram.request.BaseRequest`, optional): Pre initialized
             :class:`telegram.request.BaseRequest` instances. Will be used for all bot methods
             *except* for :meth:`get_updates`. If not passed, an instance of
@@ -239,8 +307,8 @@ class Bot(TelegramObject, contextlib.AbstractAsyncContextManager["Bot"]):
     def __init__(
         self,
         token: str,
-        base_url: str = "https://api.telegram.org/bot",
-        base_file_url: str = "https://api.telegram.org/file/bot",
+        base_url: BaseUrl = "https://api.telegram.org/bot",
+        base_file_url: BaseUrl = "https://api.telegram.org/file/bot",
         request: Optional[BaseRequest] = None,
         get_updates_request: Optional[BaseRequest] = None,
         private_key: Optional[bytes] = None,
@@ -252,8 +320,11 @@ def __init__(
             raise InvalidToken("You must pass the token you received from https://t.me/Botfather!")
         self._token: str = token
 
-        self._base_url: str = base_url + self._token
-        self._base_file_url: str = base_file_url + self._token
+        self._base_url: str = _parse_base_url(base_url, self._token)
+        self._base_file_url: str = _parse_base_url(base_file_url, self._token)
+        self._LOGGER.debug("Set Bot API URL: %s", self._base_url)
+        self._LOGGER.debug("Set Bot API File URL: %s", self._base_file_url)
+
         self._local_mode: bool = local_mode
         self._bot_user: Optional[User] = None
         self._private_key: Optional[bytes] = None
@@ -264,7 +335,7 @@ def __init__(
             HTTPXRequest() if request is None else request,
         )
 
-        # this section is about issuing a warning when using HTTP/2 and connect to a self hosted
+        # this section is about issuing a warning when using HTTP/2 and connect to a self-hosted
         # bot api instance, which currently only supports HTTP/1.1. Checking if a custom base url
         # is set is the best way to do that.
 
@@ -273,14 +344,14 @@ def __init__(
         if (
             isinstance(self._request[0], HTTPXRequest)
             and self._request[0].http_version == "2"
-            and not base_url.startswith("https://api.telegram.org/bot")
+            and not self.base_url.startswith("https://api.telegram.org/bot")
         ):
             warning_string = "get_updates_request"
 
         if (
             isinstance(self._request[1], HTTPXRequest)
             and self._request[1].http_version == "2"
-            and not base_url.startswith("https://api.telegram.org/bot")
+            and not self.base_url.startswith("https://api.telegram.org/bot")
         ):
             if warning_string:
                 warning_string += " and request"
diff --git a/telegram/_utils/types.py b/telegram/_utils/types.py
index e1bf77d34ea..29377df5bae 100644
--- a/telegram/_utils/types.py
+++ b/telegram/_utils/types.py
@@ -25,7 +25,7 @@
 """
 from collections.abc import Collection
 from pathlib import Path
-from typing import IO, TYPE_CHECKING, Any, Literal, Optional, TypeVar, Union
+from typing import IO, TYPE_CHECKING, Any, Callable, Literal, Optional, TypeVar, Union
 
 if TYPE_CHECKING:
     from telegram import (
@@ -91,3 +91,5 @@
     tuple[int, int, Union[bytes, bytearray]],
     tuple[int, int, None, int],
 ]
+
+BaseUrl = Union[str, Callable[[str], str]]
diff --git a/telegram/ext/_applicationbuilder.py b/telegram/ext/_applicationbuilder.py
index d0ade0492b0..6c0cd3b009d 100644
--- a/telegram/ext/_applicationbuilder.py
+++ b/telegram/ext/_applicationbuilder.py
@@ -26,7 +26,15 @@
 
 from telegram._bot import Bot
 from telegram._utils.defaultvalue import DEFAULT_FALSE, DEFAULT_NONE, DefaultValue
-from telegram._utils.types import DVInput, DVType, FilePathInput, HTTPVersion, ODVInput, SocketOpt
+from telegram._utils.types import (
+    BaseUrl,
+    DVInput,
+    DVType,
+    FilePathInput,
+    HTTPVersion,
+    ODVInput,
+    SocketOpt,
+)
 from telegram._utils.warnings import warn
 from telegram.ext._application import Application
 from telegram.ext._baseupdateprocessor import BaseUpdateProcessor, SimpleUpdateProcessor
@@ -164,8 +172,8 @@ class ApplicationBuilder(Generic[BT, CCT, UD, CD, BD, JQ]):
 
     def __init__(self: "InitApplicationBuilder"):
         self._token: DVType[str] = DefaultValue("")
-        self._base_url: DVType[str] = DefaultValue("https://api.telegram.org/bot")
-        self._base_file_url: DVType[str] = DefaultValue("https://api.telegram.org/file/bot")
+        self._base_url: DVType[BaseUrl] = DefaultValue("https://api.telegram.org/bot")
+        self._base_file_url: DVType[BaseUrl] = DefaultValue("https://api.telegram.org/file/bot")
         self._connection_pool_size: DVInput[int] = DEFAULT_NONE
         self._proxy: DVInput[Union[str, httpx.Proxy, httpx.URL]] = DEFAULT_NONE
         self._socket_options: DVInput[Collection[SocketOpt]] = DEFAULT_NONE
@@ -378,15 +386,19 @@ def token(self: BuilderType, token: str) -> BuilderType:
         self._token = token
         return self
 
-    def base_url(self: BuilderType, base_url: str) -> BuilderType:
+    def base_url(self: BuilderType, base_url: BaseUrl) -> BuilderType:
         """Sets the base URL for :attr:`telegram.ext.Application.bot`. If not called,
         will default to ``'https://api.telegram.org/bot'``.
 
         .. seealso:: :paramref:`telegram.Bot.base_url`,
             :wiki:`Local Bot API Server <Local-Bot-API-Server>`, :meth:`base_file_url`
 
+        .. versionchanged:: NEXT.VERSION
+           Supports callable input and string formatting.
+
         Args:
-            base_url (:obj:`str`): The URL.
+            base_url (:obj:`str` | Callable[[:obj:`str`], :obj:`str`]): The URL or
+                input for the URL as accepted by :paramref:`telegram.Bot.base_url`.
 
         Returns:
             :class:`ApplicationBuilder`: The same builder with the updated argument.
@@ -396,15 +408,19 @@ def base_url(self: BuilderType, base_url: str) -> BuilderType:
         self._base_url = base_url
         return self
 
-    def base_file_url(self: BuilderType, base_file_url: str) -> BuilderType:
+    def base_file_url(self: BuilderType, base_file_url: BaseUrl) -> BuilderType:
         """Sets the base file URL for :attr:`telegram.ext.Application.bot`. If not
         called, will default to ``'https://api.telegram.org/file/bot'``.
 
         .. seealso:: :paramref:`telegram.Bot.base_file_url`,
             :wiki:`Local Bot API Server <Local-Bot-API-Server>`, :meth:`base_url`
 
+        .. versionchanged:: NEXT.VERSION
+           Supports callable input and string formatting.
+
         Args:
-            base_file_url (:obj:`str`): The URL.
+            base_file_url (:obj:`str` | Callable[[:obj:`str`], :obj:`str`]): The URL or
+                input for the URL as accepted by :paramref:`telegram.Bot.base_file_url`.
 
         Returns:
             :class:`ApplicationBuilder`: The same builder with the updated argument.
diff --git a/telegram/ext/_extbot.py b/telegram/ext/_extbot.py
index 80eede9d841..db19ec806bb 100644
--- a/telegram/ext/_extbot.py
+++ b/telegram/ext/_extbot.py
@@ -93,7 +93,14 @@
 from telegram._utils.defaultvalue import DEFAULT_NONE, DefaultValue
 from telegram._utils.logging import get_logger
 from telegram._utils.repr import build_repr_with_selected_attrs
-from telegram._utils.types import CorrectOptionID, FileInput, JSONDict, ODVInput, ReplyMarkup
+from telegram._utils.types import (
+    BaseUrl,
+    CorrectOptionID,
+    FileInput,
+    JSONDict,
+    ODVInput,
+    ReplyMarkup,
+)
 from telegram.ext._callbackdatacache import CallbackDataCache
 from telegram.ext._utils.types import RLARGS
 from telegram.request import BaseRequest
@@ -184,8 +191,8 @@ class ExtBot(Bot, Generic[RLARGS]):
     def __init__(
         self: "ExtBot[None]",
         token: str,
-        base_url: str = "https://api.telegram.org/bot",
-        base_file_url: str = "https://api.telegram.org/file/bot",
+        base_url: BaseUrl = "https://api.telegram.org/bot",
+        base_file_url: BaseUrl = "https://api.telegram.org/file/bot",
         request: Optional[BaseRequest] = None,
         get_updates_request: Optional[BaseRequest] = None,
         private_key: Optional[bytes] = None,
@@ -199,8 +206,8 @@ def __init__(
     def __init__(
         self: "ExtBot[RLARGS]",
         token: str,
-        base_url: str = "https://api.telegram.org/bot",
-        base_file_url: str = "https://api.telegram.org/file/bot",
+        base_url: BaseUrl = "https://api.telegram.org/bot",
+        base_file_url: BaseUrl = "https://api.telegram.org/file/bot",
         request: Optional[BaseRequest] = None,
         get_updates_request: Optional[BaseRequest] = None,
         private_key: Optional[bytes] = None,
@@ -214,8 +221,8 @@ def __init__(
     def __init__(
         self,
         token: str,
-        base_url: str = "https://api.telegram.org/bot",
-        base_file_url: str = "https://api.telegram.org/file/bot",
+        base_url: BaseUrl = "https://api.telegram.org/bot",
+        base_file_url: BaseUrl = "https://api.telegram.org/file/bot",
         request: Optional[BaseRequest] = None,
         get_updates_request: Optional[BaseRequest] = None,
         private_key: Optional[bytes] = None,


Test patch:
diff --git a/tests/test_bot.py b/tests/test_bot.py
index 1245b5b8575..6f5dc0ade0a 100644
--- a/tests/test_bot.py
+++ b/tests/test_bot.py
@@ -235,7 +235,9 @@ def _reset(self):
 
     @pytest.mark.parametrize("bot_class", [Bot, ExtBot])
     def test_slot_behaviour(self, bot_class, offline_bot):
-        inst = bot_class(offline_bot.token)
+        inst = bot_class(
+            offline_bot.token, request=OfflineRequest(1), get_updates_request=OfflineRequest(1)
+        )
         for attr in inst.__slots__:
             assert getattr(inst, attr, "err") != "err", f"got extra slot '{attr}'"
         assert len(mro_slots(inst)) == len(set(mro_slots(inst))), "duplicate slot"
@@ -244,6 +246,71 @@ async def test_no_token_passed(self):
         with pytest.raises(InvalidToken, match="You must pass the token"):
             Bot("")
 
+    def test_base_url_parsing_basic(self, caplog):
+        with caplog.at_level(logging.DEBUG):
+            bot = Bot(
+                token="!!Test String!!",
+                base_url="base/",
+                base_file_url="base/",
+                request=OfflineRequest(1),
+                get_updates_request=OfflineRequest(1),
+            )
+
+        assert bot.base_url == "base/!!Test String!!"
+        assert bot.base_file_url == "base/!!Test String!!"
+
+        assert len(caplog.records) >= 2
+        messages = [record.getMessage() for record in caplog.records]
+        assert "Set Bot API URL: base/!!Test String!!" in messages
+        assert "Set Bot API File URL: base/!!Test String!!" in messages
+
+    @pytest.mark.parametrize(
+        "insert_key", ["token", "TOKEN", "bot_token", "BOT_TOKEN", "bot-token", "BOT-TOKEN"]
+    )
+    def test_base_url_parsing_string_format(self, insert_key, caplog):
+        string = f"{{{insert_key}}}"
+
+        with caplog.at_level(logging.DEBUG):
+            bot = Bot(
+                token="!!Test String!!",
+                base_url=string,
+                base_file_url=string,
+                request=OfflineRequest(1),
+                get_updates_request=OfflineRequest(1),
+            )
+
+        assert bot.base_url == "!!Test String!!"
+        assert bot.base_file_url == "!!Test String!!"
+
+        assert len(caplog.records) >= 2
+        messages = [record.getMessage() for record in caplog.records]
+        assert "Set Bot API URL: !!Test String!!" in messages
+        assert "Set Bot API File URL: !!Test String!!" in messages
+
+        with pytest.raises(KeyError, match="unsupported insertion: unknown"):
+            Bot("token", base_url="{unknown}{token}")
+
+    def test_base_url_parsing_callable(self, caplog):
+        def build_url(_: str) -> str:
+            return "!!Test String!!"
+
+        with caplog.at_level(logging.DEBUG):
+            bot = Bot(
+                token="some-token",
+                base_url=build_url,
+                base_file_url=build_url,
+                request=OfflineRequest(1),
+                get_updates_request=OfflineRequest(1),
+            )
+
+        assert bot.base_url == "!!Test String!!"
+        assert bot.base_file_url == "!!Test String!!"
+
+        assert len(caplog.records) >= 2
+        messages = [record.getMessage() for record in caplog.records]
+        assert "Set Bot API URL: !!Test String!!" in messages
+        assert "Set Bot API File URL: !!Test String!!" in messages
+
     async def test_repr(self):
         offline_bot = Bot(token="some_token", base_file_url="")
         assert repr(offline_bot) == "Bot[token=some_token]"
@@ -409,9 +476,10 @@ def test_bot_deepcopy_error(self, offline_bot):
         ("cls", "logger_name"), [(Bot, "telegram.Bot"), (ExtBot, "telegram.ext.ExtBot")]
     )
     async def test_bot_method_logging(self, offline_bot: PytestExtBot, cls, logger_name, caplog):
+        instance = cls(offline_bot.token)
         # Second argument makes sure that we ignore logs from e.g. httpx
         with caplog.at_level(logging.DEBUG, logger="telegram"):
-            await cls(offline_bot.token).get_me()
+            await instance.get_me()
             # Only for stabilizing this test-
             if len(caplog.records) == 4:
                 for idx, record in enumerate(caplog.records):