Repository: python-telegram-bot/python-telegram-bot

Issue:
Review types of optional arguments in methods and classes 
There are certain parameters in our bot methods which optional with the default as `DEFAULT_NONE` (as specified by the `Defaults` class). The type hinting of such parameters is not consistent. For e.g. 
https://github.com/python-telegram-bot/python-telegram-bot/blob/8cc55c338161e71a640b24a382a259cff6e7a715/telegram/_bot.py#L7197
and 
https://github.com/python-telegram-bot/python-telegram-bot/blob/8cc55c338161e71a640b24a382a259cff6e7a715/telegram/_bot.py#L7201

both parameters accept `bool`, but one of them is type hinted to also accept `None` (ODV -> Optional DefaultValue). Here the type hint for `protect_content` should be corrected to `DVInput` imo.

We should look for other cases like this, and correct it in both the type hints and documentation.


Discussion:
I did some investigation on this. First, regarding arguments of bot methods:

Let's take `disable_notification` as example from above. This is an optional argument. We expect that users either do not pass this argument or when they pass it, they pass a bool. This would lead to `DVInput[bool]`. However, one can argue that passing `None` should be equal to not passing the argument. This is useful e.g. when passing kwargs that are generated at runtime (`send_message(**build_kwargs())`). Moreover, all other optional arguments are typed to actually accept `None` (e.g. `reply_to_message_id: Optional[int] = None`).
I would hence vote that all arguments of bot methods with default values should be using `ODVInput`.

Moreover, I collected all arguments in our code base with one of these type hints:

<details><summary>Script</summary>

```python
import re
from pathlib import Path
from pprint import pprint

REGEX = re.compile(r"\s([_\w]+: O?DV(Input|Type)\[\w+\] = .*)")
MATCHES = set()

for file_path in Path("telegram").rglob("*.py"):
    content = file_path.read_text(encoding="utf-8")
    for match in REGEX.finditer(content):
        MATCHES.add(match.group(1))


pprint(sorted(MATCHES))
```

</details>

Results:

```python
['allow_sending_without_reply: DVInput[bool] = DEFAULT_NONE,',
 'allow_sending_without_reply: ODVInput[bool] = DEFAULT_NONE,',
 'block: DVType[bool] = DEFAULT_TRUE,',
 'connect_timeout: ODVInput[float] = BaseRequest.DEFAULT_NONE,',
 'connect_timeout: ODVInput[float] = DEFAULT_NONE,',
 'disable_notification: DVInput[bool] = DEFAULT_NONE,',
 'disable_notification: ODVInput[bool] = DEFAULT_NONE,',
 'disable_web_page_preview: ODVInput[bool] = DEFAULT_NONE,',
 'explanation_parse_mode: ODVInput[str] = DEFAULT_NONE,',
 'parse_mode: ODVInput[str] = DEFAULT_NONE,',
 'pool_timeout: ODVInput[float] = BaseRequest.DEFAULT_NONE,',
 'pool_timeout: ODVInput[float] = DEFAULT_NONE,',
 'protect_content: ODVInput[bool] = DEFAULT_NONE,',
 'read_timeout: ODVInput[float] = BaseRequest.DEFAULT_NONE,',
 'read_timeout: ODVInput[float] = DEFAULT_NONE,',
 'write_timeout: ODVInput[float] = 20,',
 'write_timeout: ODVInput[float] = BaseRequest.DEFAULT_NONE,',
 'write_timeout: ODVInput[float] = DEFAULT_NONE,']
```

In this, I see several notable points.

Firstly, we have inconsistencies in the typing of `allow_sending_without_reply` and `disable_notification` that should be unified
Secondly, most arguments are already using `ODVInput` (which fits my above reasoning 😎 )

Thirdly, there is a problem with `write_timeout` defaulting to an integer 20 (not `DefaultValue(20)`!). In `HTTPXRequest.do_request`

https://github.com/python-telegram-bot/python-telegram-bot/blob/a0decdac282507aafd0ee84b0bbcea32cca687e8/telegram/request/_httpxrequest.py#L177-L200

this leads to `write_timeout` being 20 seconds - as desired. However, when specifying a custom `write_timeout` value via `HTTPXRequest(write_timeout=42)` or `ApplicationBuilder().write_timeout(42)`, the value 42 will *not* be used by `send_photo` & friends. Moreover, even if we fix this problem, I notice that we can't specify different default values for media operations and non-media operations. Since already use different values for these in the standard configuration, I think this would make sense.

My first idea would be the following:

1. In the bot methods make `write_timeout` default to `DEFAULT_NONE` as with the other timeout parameters
2. Add a new argument `media_write_timeout` to `HTTPXRequest` that defaults to 20 seconds.
3. Make `HTTPXRequest.do_request` select the `write_timeout` as follows:
  
   ```python
    if write_timeout is not DEFAULT_NONE:
        effective_write_timeout = write_timeout
    elif files:
        effective_write_timeout = self.media_write_timeout
    else:
        effective_write_timeout = self.write_timeout
    ```

This would be a backward compatible change, further decouple the `Bot` from the networking backend and add an additional feature.

What do you think?

can you assign this to me please .
@UjjwalPardeshi this issue is still in discussion and not ready to be pred for
> Add a new argument media_write_timeout to HTTPXRequest that defaults to 20 seconds.

Shouldn't that be higher?

Your logic looks sound to me. Would go ahead.
> However, when specifying a custom `write_timeout` value via `HTTPXRequest(write_timeout=42)` or `ApplicationBuilder().write_timeout(42)`, the value 42 will _not_ be used by `send_photo` & friends. 

kinda shocking that this went unnoticed by tests and everyone (maybe that's why ppl keep posting the timeout errors even after changing the timeout vals)

> My first idea would be the following:

We would also need to add a new method to AppBuilder for the new arg right?

> 1. In the bot methods make `write_timeout` default to `DEFAULT_NONE` as with the other timeout parameters
> 2. Add a new argument `media_write_timeout` to `HTTPXRequest` that defaults to 20 seconds.
> 3. Make `HTTPXRequest.do_request` select the `write_timeout` as follows:

Idea sounds good 👍🏽 


> > Add a new argument media_write_timeout to HTTPXRequest that defaults to 20 seconds.
> 
> Shouldn't that be higher?

I agree, should be like 30 seconds min to accomodate for slower internet speeds (https://www.speedtest.net/global-index)


Actually I'm backtracking on adding `media_write_timeout` (point 2.). After reading #3893 I'm not sure if adding yet another argument would make it easier for the user (and us!). How about internally setting the timeout to `30` seconds for file-related methods only? The user can simply change that 30 second behaviour by using the same `write_timeout` argument in a bot method. So the logic would now be: 

``` python
 if write_timeout is not DEFAULT_NONE:
     effective_write_timeout = write_timeout
 elif files:
     effective_write_timeout = 30
 else:
     effective_write_timeout = self.write_timeout
```

We would continue to document in the send_* methods that the write_timeout param defaults to 30 seconds.
> > > Add a new argument media_write_timeout to HTTPXRequest that defaults to 20 seconds.
>>
> > Shouldn't that be higher?
>
> I agree, should be like 30 seconds min to accomodate for slower internet speeds (https://www.speedtest.net/global-index)

20 is the value that has been used for eons (minimum since v6.0) and is documented everywhere. If there are no clear indications that this value is too short for most people, I would prefer to stick to the 20 …

> We would also need to add a new method to AppBuilder for the new arg right?

yes, of course `AppBuilder.media_write_timeout()` would be required

>> I'm not sure if adding yet another argument would make it easier for the user (and us!). How about internally setting the timeout to 30 seconds for file-related methods only? The user can simply change that 30 second behaviour by using the same `write_timeout` argument in a bot method.

Just to clarify: This would steeps 1 & 3, only step 2 would be omitted.
Personally, I see no reason to make this particular timeout hard-coded while all other timeouts are configurable. Setting the value of every call to `send_*` is probably not easier than setting it via `AppBuilder.media_write_timeout`. In terms of maintenance, I would argue that an additional timeout setting is not much added overhead as long as it works just like the other ones - which it would :)
Still, one could get started with steps 1 & 3 and add 2 later on in a follow-up PR.


@Bibo-Joshi 20 seconds is a sane default. We worked it out the hard way. 

It is a trade off between identifying problems asap and between being too aggressive and/or slow internet connections. 

The default should not be supportive of those with horrible internet.
> 20 is the value that has been used for eons (minimum since v6.0) and is documented everywhere. If there are no clear indications that this value is too short for most people, I would prefer to stick to the 20 …

> It is a trade off between identifying problems asap and between being too aggressive and/or slow internet connections.

ok, but tbf I've only ever seen people _increase_ the timeouts, not decrease them.

> Setting the value of every call to `send_*` is probably not easier than setting it via `AppBuilder.media_write_timeout`. 

Right, I was thinking `AppBuilder.write_timeout` should override the 20 (or 30) seconds timeout for media operations as well. But maybe that behaviour is obscure and not in line with having everything configurable via the builder..

> Still, one could get started with steps 1 & 3 and add 2 later on in a follow-up PR.

👍🏽 

ALl right, then I think we have a consensus here and there are 3 PRs to be done:

1. Make all arguments of bot methods with default values use ODVInput
2. Steps 1 & 3 from https://github.com/python-telegram-bot/python-telegram-bot/issues/3864#issuecomment-1712796820
3. As follow up to 2, step 2
> ok, but tbf I've only ever seen people _increase_ the timeouts, not decrease them.

@Bibo-Joshi  you’ve out seen those who complained. Think of all those which it worked good for them. Increasing the timeout by default will change the behavior and raise a wave of complaints. 
the locations of errors varied between getUpdates receive a message to establishing a connection. 

However, that said and ptb is in a differed state today. It is using httpx which greatly superior to urllib3 and all the alternatives available back at the day. 
Afair now you can have different timeouts to connection establishment, ssl negotiation, read & write. You can also easily set them per operation rather than on a connection and/or globally. 


If you want something which controls the timeouts in a smarter way it might be a good idea to check what is the payload size to send and decide on the timeout accordingly 



Patch:
diff --git a/telegram/ext/_applicationbuilder.py b/telegram/ext/_applicationbuilder.py
index 71c4bd31286..106c73462b8 100644
--- a/telegram/ext/_applicationbuilder.py
+++ b/telegram/ext/_applicationbuilder.py
@@ -78,6 +78,7 @@
     ("connect_timeout", "connect_timeout"),
     ("read_timeout", "read_timeout"),
     ("write_timeout", "write_timeout"),
+    ("media_write_timeout", "media_write_timeout"),
     ("http_version", "http_version"),
     ("get_updates_connection_pool_size", "get_updates_connection_pool_size"),
     ("get_updates_proxy", "get_updates_proxy"),
@@ -152,6 +153,7 @@ class ApplicationBuilder(Generic[BT, CCT, UD, CD, BD, JQ]):
         "_http_version",
         "_job_queue",
         "_local_mode",
+        "_media_write_timeout",
         "_persistence",
         "_pool_timeout",
         "_post_init",
@@ -181,6 +183,7 @@ def __init__(self: "InitApplicationBuilder"):
         self._connect_timeout: ODVInput[float] = DEFAULT_NONE
         self._read_timeout: ODVInput[float] = DEFAULT_NONE
         self._write_timeout: ODVInput[float] = DEFAULT_NONE
+        self._media_write_timeout: ODVInput[float] = DEFAULT_NONE
         self._pool_timeout: ODVInput[float] = DEFAULT_NONE
         self._request: DVInput[BaseRequest] = DEFAULT_NONE
         self._get_updates_connection_pool_size: DVInput[int] = DEFAULT_NONE
@@ -243,6 +246,10 @@ def _build_request(self, get_updates: bool) -> BaseRequest:
             "write_timeout": getattr(self, f"{prefix}write_timeout"),
             "pool_timeout": getattr(self, f"{prefix}pool_timeout"),
         }
+
+        if not get_updates:
+            timeouts["media_write_timeout"] = self._media_write_timeout
+
         # Get timeouts that were actually set-
         effective_timeouts = {
             key: value for key, value in timeouts.items() if not isinstance(value, DefaultValue)
@@ -424,9 +431,13 @@ def _request_check(self, get_updates: bool) -> None:
         prefix = "get_updates_" if get_updates else ""
         name = prefix + "request"
 
+        timeouts = ["connect_timeout", "read_timeout", "write_timeout", "pool_timeout"]
+        if not get_updates:
+            timeouts.append("media_write_timeout")
+
         # Code below tests if it's okay to set a Request object. Only okay if no other request args
         # or instances containing a Request were set previously
-        for attr in ("connect_timeout", "read_timeout", "write_timeout", "pool_timeout"):
+        for attr in timeouts:
             if not isinstance(getattr(self, f"_{prefix}{attr}"), DefaultValue):
                 raise RuntimeError(_TWO_ARGS_REQ.format(name, attr))
 
@@ -617,6 +628,26 @@ def write_timeout(self: BuilderType, write_timeout: Optional[float]) -> BuilderT
         self._write_timeout = write_timeout
         return self
 
+    def media_write_timeout(
+        self: BuilderType, media_write_timeout: Optional[float]
+    ) -> BuilderType:
+        """Sets the media write operation timeout for the
+        :paramref:`~telegram.request.HTTPXRequest.media_write_timeout` parameter of
+        :attr:`telegram.Bot.request`. Defaults to ``20``.
+
+        .. versionadded:: NEXT.VERSION
+
+        Args:
+            media_write_timeout (:obj:`float`): See
+                :paramref:`telegram.request.HTTPXRequest.media_write_timeout` for more information.
+
+        Returns:
+            :class:`ApplicationBuilder`: The same builder with the updated argument.
+        """
+        self._request_param_check(name="media_write_timeout", get_updates=False)
+        self._media_write_timeout = media_write_timeout
+        return self
+
     def pool_timeout(self: BuilderType, pool_timeout: Optional[float]) -> BuilderType:
         """Sets the connection pool's connection freeing timeout for the
         :paramref:`~telegram.request.HTTPXRequest.pool_timeout` parameter of
diff --git a/telegram/request/_httpxrequest.py b/telegram/request/_httpxrequest.py
index 7ca54c5c534..8c18d063012 100644
--- a/telegram/request/_httpxrequest.py
+++ b/telegram/request/_httpxrequest.py
@@ -64,6 +64,10 @@ class HTTPXRequest(BaseRequest):
             a network socket; i.e. POSTing a request or uploading a file).
             This value is used unless a different value is passed to :meth:`do_request`.
             Defaults to ``5``.
+
+            Hint:
+                This timeout is used for all requests except for those that upload media/files.
+                For the latter, :paramref:`media_write_timeout` is used.
         connect_timeout (:obj:`float` | :obj:`None`, optional): If passed, specifies the
             maximum amount of time (in seconds) to wait for a connection attempt to a server
             to succeed. This value is used unless a different value is passed to
@@ -112,10 +116,16 @@ class HTTPXRequest(BaseRequest):
             .. _the docs of httpx: https://www.python-httpx.org/environment_variables/#proxies
 
             .. versionadded:: 20.7
+        media_write_timeout (:obj:`float` | :obj:`None`, optional): Like :paramref:`write_timeout`,
+            but used only for requests that upload media/files. This value is used unless a
+            different value is passed to :paramref:`do_request.write_timeout` of
+            :meth:`do_request`. Defaults to ``20`` seconds.
+
+            .. versionadded:: NEXT.VERSION
 
     """
 
-    __slots__ = ("_client", "_client_kwargs", "_http_version")
+    __slots__ = ("_client", "_client_kwargs", "_http_version", "_media_write_timeout")
 
     def __init__(
         self,
@@ -128,6 +138,7 @@ def __init__(
         http_version: HTTPVersion = "1.1",
         socket_options: Optional[Collection[SocketOpt]] = None,
         proxy: Optional[Union[str, httpx.Proxy, httpx.URL]] = None,
+        media_write_timeout: Optional[float] = 20.0,
     ):
         if proxy_url is not None and proxy is not None:
             raise ValueError("The parameters `proxy_url` and `proxy` are mutually exclusive.")
@@ -142,6 +153,7 @@ def __init__(
             )
 
         self._http_version = http_version
+        self._media_write_timeout = media_write_timeout
         timeout = httpx.Timeout(
             connect=connect_timeout,
             read=read_timeout,
@@ -251,11 +263,7 @@ async def do_request(
             pool_timeout = self._client.timeout.pool
 
         if isinstance(write_timeout, DefaultValue):
-            # Making the networking backend decide on the proper timeout values instead of doing
-            # it via the default values of the Bot methods was introduced in version 20.7.
-            # We hard-code the value here for now until we add additional parameters to this
-            # class to control the media_write_timeout separately.
-            write_timeout = self._client.timeout.write if not files else 20
+            write_timeout = self._client.timeout.write if not files else self._media_write_timeout
 
         timeout = httpx.Timeout(
             connect=connect_timeout,


Test patch:
diff --git a/tests/ext/test_applicationbuilder.py b/tests/ext/test_applicationbuilder.py
index 89990601bc5..9fcbc140141 100644
--- a/tests/ext/test_applicationbuilder.py
+++ b/tests/ext/test_applicationbuilder.py
@@ -76,6 +76,9 @@ def test_all_methods_request(self, builder, get_updates):
         for argument in arguments:
             if argument == "self":
                 continue
+            if argument == "media_write_timeout" and get_updates:
+                # get_updates never makes media requests
+                continue
             assert hasattr(builder, prefix + argument), f"missing method {prefix}{argument}"
 
     @pytest.mark.parametrize("bot_class", [Bot, ExtBot])
@@ -202,6 +205,7 @@ def test_mutually_exclusive_for_bot(self, builder, method, description):
             "pool_timeout",
             "read_timeout",
             "write_timeout",
+            "media_write_timeout",
             "proxy",
             "proxy_url",
             "socket_options",
@@ -272,6 +276,7 @@ def test_mutually_exclusive_for_get_updates_request(self, builder, method):
             "pool_timeout",
             "read_timeout",
             "write_timeout",
+            "media_write_timeout",
             "proxy",
             "proxy_url",
             "socket_options",
@@ -316,6 +321,7 @@ def test_mutually_exclusive_for_updater(self, builder, method):
             "pool_timeout",
             "read_timeout",
             "write_timeout",
+            "media_write_timeout",
             "proxy",
             "proxy_url",
             "socket_options",
@@ -384,12 +390,20 @@ class Client:
             http2: object
             transport: object = None
 
+        original_init = HTTPXRequest.__init__
+        media_write_timeout = []
+
+        def init_httpx_request(self_, *args, **kwargs):
+            media_write_timeout.append(kwargs.get("media_write_timeout"))
+            original_init(self_, *args, **kwargs)
+
         monkeypatch.setattr(httpx, "AsyncClient", Client)
+        monkeypatch.setattr(HTTPXRequest, "__init__", init_httpx_request)
 
         builder = ApplicationBuilder().token(bot.token)
         builder.connection_pool_size(1).connect_timeout(2).pool_timeout(3).read_timeout(
             4
-        ).write_timeout(5).http_version("1.1")
+        ).write_timeout(5).media_write_timeout(6).http_version("1.1")
         getattr(builder, proxy_method)("proxy")
         app = builder.build()
         client = app.bot.request._client
@@ -399,7 +413,9 @@ class Client:
         assert client.proxy == "proxy"
         assert client.http1 is True
         assert client.http2 is False
+        assert media_write_timeout == [6, None]
 
+        media_write_timeout.clear()
         builder = ApplicationBuilder().token(bot.token)
         builder.get_updates_connection_pool_size(1).get_updates_connect_timeout(
             2
@@ -417,6 +433,7 @@ class Client:
         assert client.proxy == "get_updates_proxy"
         assert client.http1 is True
         assert client.http2 is False
+        assert media_write_timeout == [None, None]
 
     def test_custom_socket_options(self, builder, monkeypatch, bot):
         httpx_request_kwargs = []
diff --git a/tests/request/test_request.py b/tests/request/test_request.py
index 8dddd0e3f7a..47e7d2125f6 100644
--- a/tests/request/test_request.py
+++ b/tests/request/test_request.py
@@ -738,6 +738,39 @@ async def request(_, **kwargs):
         # other than HTTPXRequest
         assert len(recwarn) == 0
 
+    @pytest.mark.parametrize("init", [True, False])
+    async def test_setting_media_write_timeout(
+        self, monkeypatch, init, input_media_photo, recwarn  # noqa: F811
+    ):
+        httpx_request = HTTPXRequest(media_write_timeout=42) if init else HTTPXRequest()
+
+        async def request(_, **kwargs):
+            self.test_flag = kwargs["timeout"].write
+            return httpx.Response(HTTPStatus.OK, content=b'{"ok": "True", "result": {}}')
+
+        monkeypatch.setattr(httpx.AsyncClient, "request", request)
+
+        data = {"string": "string", "int": 1, "float": 1.0, "media": input_media_photo}
+        request_data = RequestData(
+            parameters=[RequestParameter.from_input(key, value) for key, value in data.items()],
+        )
+
+        # First make sure that custom timeouts are always respected
+        await httpx_request.post(
+            "url",
+            request_data,
+            write_timeout=43,
+        )
+        assert self.test_flag == 43
+
+        # Now also ensure that the init value is respected
+        await httpx_request.post("url", request_data)
+        assert self.test_flag == 42 if init else 20
+
+        # Just for double-checking, since warnings are issued for implementations of BaseRequest
+        # other than HTTPXRequest
+        assert len(recwarn) == 0
+
     async def test_socket_opts(self, monkeypatch):
         transport_kwargs = {}
         transport_init = AsyncHTTPTransport.__init__