Repository: aio-libs/aiohttp
Pull request: Add full request typing signatures for session HTTP methods
<!-- Thank you for your contribution! -->

## What do these changes do?

Improve the request typing by using 3.11's new `Unpack`  operator to properly type the kwargs that live on each HTTP method. Using `typing_extensions`, this is supported on older versions of Python as well.

To do this, I did add `typing-extensions` as a requirement on versions of Python before 3.11. I think this trade-off is well worth it, as it drastically improves the static typing checking experience, and the intellisense experience for VSCode users on Pylance. `typing-extensions` will also let `aiohttp` use other typing backports without having to wait multiple years until the minimum supported version Python catches up.

<!-- Please give a short brief about these changes. -->

## Are there changes in behavior for the user?

Users will be improved type-checking when making requests, if improper/unexpected types are being used, they might notice new typing warnings.

<!-- Outline any notable behaviour for the end users. -->

## Is it a substantial burden for the maintainers to support this?

No, updates to the request arguments simply need to be mirrored with the typed dict added, which will update all of the core HTTP methods.

<!--
Stop right there! Pause. Just for a minute... Can you think of anything
obvious that would complicate the ongoing development of this project?

Try to consider if you'd be able to maintain it throughout the next
5 years. Does it seem viable? Tell us your thoughts! We'd very much
love to hear what the consequences of merging this patch might be...

This will help us assess if your change is something we'd want to
entertain early in the review process. Thank you in advance!
-->

## Related issue number

Fixes https://github.com/aio-libs/aiohttp/issues/4749

<!-- Are there any issues opened that will be resolved by merging this change? -->
<!-- Remember to prefix with 'Fixes' if it should close the issue (e.g. 'Fixes #123'). -->

## Checklist

- [x] I think the code is well written
- [x] Unit tests for the changes exist (not applicable, no logic changes)
- [x] Documentation reflects the changes (type-checking changes)
- [x] If you provide code modification, please add yourself to `CONTRIBUTORS.txt`
  * The format is &lt;Name&gt; &lt;Surname&gt;.
  * Please keep alphabetical order, the file is sorted by names.
- [x] Add a new news fragment into the `CHANGES/` folder
  * name it `<issue_or_pr_num>.<type>.rst` (e.g. `588.bugfix.rst`)
  * if you don't have an issue number, change it to the pull request
    number after creating the PR
    * `.bugfix`: A bug fix for something the maintainers deemed an
      improper undesired behavior that got corrected to match
      pre-agreed expectations.
    * `.feature`: A new behavior, public APIs. That sort of stuff.
    * `.deprecation`: A declaration of future API removals and breaking
      changes in behavior.
    * `.breaking`: When something public is removed in a breaking way.
      Could be deprecated in an earlier release.
    * `.doc`: Notable updates to the documentation structure or build
      process.
    * `.packaging`: Notes for downstreams about unobvious side effects
      and tooling. Changes in the test invocation considerations and
      runtime assumptions.
    * `.contrib`: Stuff that affects the contributor experience. e.g.
      Running tests, building the docs, setting up the development
      environment.
    * `.misc`: Changes that are hard to assign to any of the above
      categories.
  * Make sure to use full sentences with correct case and punctuation,
    for example:
    ```rst
    Fixed issue with non-ascii contents in doctest text files
    -- by :user:`contributor-gh-handle`.
    ```

    Use the past tense or the present tense a non-imperative mood,
    referring to what's changed compared to the last released version
    of this project.


Problem statement:
ClientSession post and get methods doesn't have full signature
🐣 **Is your feature request related to a problem? Please describe.**
It seems inconvenient that you don't know the signature of a method when using it. If you are coming from different libraries, like request or maybe you've used the aiohttp Client some time ago, you kinda know that there is probably params like `auth`, `timeout`, `headers`, and some other basic stuff, so it would be really convenient to not check the docs or source code, and have IDE to suggest you these params.
The other thing with this is that auth, for example, should be an instance of a BasicAuth, wherein requests it could be just a tuple. If we would have a signature beforehand, the IDE or mypy would be able to detect the error before executing the code.

💡 **Describe the solution you'd like**
One possibility is to duplicate the whole signature, instead of using kwargs. From the code perspective probably not the best idea, but it could help end-users.


❓ **Describe alternatives you've considered**
I've also thought about making kwargs fully typed, but it doesn't look like mypy can handle this. I definitely can check this again if you think that this is a good idea.


📋 **Additional context**
Additionally, I would like to say that I'm willing to work on this if the community decides that this feature could add value to the project.
Thanks for the great library!

Hints from issue discussion:
If anybody would like to work on this, we can now use `Unpack` with mypy 1.7:
https://docs.python.org/3/library/typing.html#typing.Unpack

This will allow us to specify all the kwarg types in one place and have them used for all the methods needed.

Commit message: Avoid overloading

Patch to review:
diff --git a/CHANGES/8463.misc.rst b/CHANGES/8463.misc.rst
new file mode 100644
index 00000000000..1d42136ebd7
--- /dev/null
+++ b/CHANGES/8463.misc.rst
@@ -0,0 +1 @@
+Added a 3.11-specific overloads to ``ClientSession``  -- by :user:`max-muoto`.
diff --git a/aiohttp/client.py b/aiohttp/client.py
index 7a4db0db476..9fc6b0fb1da 100644
--- a/aiohttp/client.py
+++ b/aiohttp/client.py
@@ -29,9 +29,11 @@
     Set,
     Tuple,
     Type,
+    TypedDict,
     TypeVar,
     Union,
     final,
+    overload,
 )
 
 from multidict import CIMultiDict, MultiDict, MultiDictProxy, istr
@@ -150,6 +152,34 @@
     SSLContext = None
 
 
+class _RequestOptions(TypedDict, total=False):
+    params: Union[Mapping[str, str], None]
+    data: Any
+    json: Any
+    cookies: Union[LooseCookies, None]
+    headers: Union[LooseHeaders, None]
+    skip_auto_headers: Union[Iterable[str], None]
+    auth: Union[BasicAuth, None]
+    allow_redirects: bool
+    max_redirects: int
+    compress: Union[str, None]
+    chunked: Union[bool, None]
+    expect100: bool
+    raise_for_status: Union[None, bool, Callable[[ClientResponse], Awaitable[None]]]
+    read_until_eof: bool
+    proxy: Union[StrOrURL, None]
+    proxy_auth: Union[BasicAuth, None]
+    timeout: "Union[ClientTimeout, _SENTINEL, None]"
+    ssl: Union[SSLContext, bool, Fingerprint]
+    server_hostname: Union[str, None]
+    proxy_headers: Union[LooseHeaders, None]
+    trace_request_ctx: Union[SimpleNamespace, None]
+    read_bufsize: Union[int, None]
+    auto_decompress: Union[bool, None]
+    max_line_size: Union[int, None]
+    max_field_size: Union[int, None]
+
+
 @dataclasses.dataclass(frozen=True)
 class ClientTimeout:
     total: Optional[float] = None
@@ -986,61 +1016,112 @@ def _prepare_headers(self, headers: Optional[LooseHeaders]) -> "CIMultiDict[str]
                     added_names.add(key)
         return result
 
-    def get(
-        self, url: StrOrURL, *, allow_redirects: bool = True, **kwargs: Any
-    ) -> "_RequestContextManager":
-        """Perform HTTP GET request."""
-        return _RequestContextManager(
-            self._request(hdrs.METH_GET, url, allow_redirects=allow_redirects, **kwargs)
-        )
+    if sys.version_info >= (3, 11) and TYPE_CHECKING:
+        import typing
+
+        def get(
+            self,
+            url: StrOrURL,
+            **kwargs: typing.Unpack[_RequestOptions],
+        ) -> "_RequestContextManager": ...
+
+        def options(
+            self,
+            url: StrOrURL,
+            **kwargs: typing.Unpack[_RequestOptions],
+        ) -> "_RequestContextManager": ...
+
+        def head(
+            self,
+            url: StrOrURL,
+            **kwargs: typing.Unpack[_RequestOptions],
+        ) -> "_RequestContextManager": ...
+
+        def post(
+            self,
+            url: StrOrURL,
+            **kwargs: typing.Unpack[_RequestOptions],
+        ) -> "_RequestContextManager": ...
+
+        def put(
+            self,
+            url: StrOrURL,
+            **kwargs: typing.Unpack[_RequestOptions],
+        ) -> "_RequestContextManager": ...
+
+        def patch(
+            self,
+            url: StrOrURL,
+            **kwargs: typing.Unpack[_RequestOptions],
+        ) -> "_RequestContextManager": ...
+
+        def delete(
+            self,
+            url: StrOrURL,
+            **kwargs: typing.Unpack[_RequestOptions],
+        ) -> "_RequestContextManager": ...
+
+    else:
+
+        def get(
+            self, url: StrOrURL, *, allow_redirects: bool = True, **kwargs: Any
+        ) -> "_RequestContextManager":
+            """Perform HTTP GET request."""
+            return _RequestContextManager(
+                self._request(
+                    hdrs.METH_GET, url, allow_redirects=allow_redirects, **kwargs
+                )
+            )
 
-    def options(
-        self, url: StrOrURL, *, allow_redirects: bool = True, **kwargs: Any
-    ) -> "_RequestContextManager":
-        """Perform HTTP OPTIONS request."""
-        return _RequestContextManager(
-            self._request(
-                hdrs.METH_OPTIONS, url, allow_redirects=allow_redirects, **kwargs
+        def options(
+            self, url: StrOrURL, *, allow_redirects: bool = True, **kwargs: Any
+        ) -> "_RequestContextManager":
+            """Perform HTTP OPTIONS request."""
+            return _RequestContextManager(
+                self._request(
+                    hdrs.METH_OPTIONS, url, allow_redirects=allow_redirects, **kwargs
+                )
             )
-        )
 
-    def head(
-        self, url: StrOrURL, *, allow_redirects: bool = False, **kwargs: Any
-    ) -> "_RequestContextManager":
-        """Perform HTTP HEAD request."""
-        return _RequestContextManager(
-            self._request(
-                hdrs.METH_HEAD, url, allow_redirects=allow_redirects, **kwargs
+        def head(
+            self, url: StrOrURL, *, allow_redirects: bool = False, **kwargs: Any
+        ) -> "_RequestContextManager":
+            """Perform HTTP HEAD request."""
+            return _RequestContextManager(
+                self._request(
+                    hdrs.METH_HEAD, url, allow_redirects=allow_redirects, **kwargs
+                )
             )
-        )
 
-    def post(
-        self, url: StrOrURL, *, data: Any = None, **kwargs: Any
-    ) -> "_RequestContextManager":
-        """Perform HTTP POST request."""
-        return _RequestContextManager(
-            self._request(hdrs.METH_POST, url, data=data, **kwargs)
-        )
+        def post(
+            self, url: StrOrURL, *, data: Any = None, **kwargs: Any
+        ) -> "_RequestContextManager":
+            """Perform HTTP POST request."""
+            return _RequestContextManager(
+                self._request(hdrs.METH_POST, url, data=data, **kwargs)
+            )
 
-    def put(
-        self, url: StrOrURL, *, data: Any = None, **kwargs: Any
-    ) -> "_RequestContextManager":
-        """Perform HTTP PUT request."""
-        return _RequestContextManager(
-            self._request(hdrs.METH_PUT, url, data=data, **kwargs)
-        )
+        def put(
+            self, url: StrOrURL, *, data: Any = None, **kwargs: Any
+        ) -> "_RequestContextManager":
+            """Perform HTTP PUT request."""
+            return _RequestContextManager(
+                self._request(hdrs.METH_PUT, url, data=data, **kwargs)
+            )
 
-    def patch(
-        self, url: StrOrURL, *, data: Any = None, **kwargs: Any
-    ) -> "_RequestContextManager":
-        """Perform HTTP PATCH request."""
-        return _RequestContextManager(
-            self._request(hdrs.METH_PATCH, url, data=data, **kwargs)
-        )
+        def patch(
+            self, url: StrOrURL, *, data: Any = None, **kwargs: Any
+        ) -> "_RequestContextManager":
+            """Perform HTTP PATCH request."""
+            return _RequestContextManager(
+                self._request(hdrs.METH_PATCH, url, data=data, **kwargs)
+            )
 
-    def delete(self, url: StrOrURL, **kwargs: Any) -> "_RequestContextManager":
-        """Perform HTTP DELETE request."""
-        return _RequestContextManager(self._request(hdrs.METH_DELETE, url, **kwargs))
+        def delete(self, url: StrOrURL, **kwargs: Any) -> "_RequestContextManager":
+            """Perform HTTP DELETE request."""
+            return _RequestContextManager(
+                self._request(hdrs.METH_DELETE, url, **kwargs)
+            )
 
     async def close(self) -> None:
         """Close underlying connector.


Code review comments:
diff_hunk: @@ -986,61 +1016,112 @@ def _prepare_headers(self, headers: Optional[LooseHeaders]) -> "CIMultiDict[str]
                     added_names.add(key)
         return result
 
-    def get(
-        self, url: StrOrURL, *, allow_redirects: bool = True, **kwargs: Any
-    ) -> "_RequestContextManager":
-        """Perform HTTP GET request."""
-        return _RequestContextManager(
-            self._request(hdrs.METH_GET, url, allow_redirects=allow_redirects, **kwargs)
-        )
+    if sys.version_info >= (3, 11) and TYPE_CHECKING:
+        import typing
path: aiohttp/client.py
text: @author:
MyPy weirdly doesn't like a direct import of `Unpack` from `typing`, but is fine with `typing.Unpack`.

@user1:
I don't see any errors when I test it locally?

@author:
Let me show you an example:

@author:
The same example I posted above (as is now):

```bash
mypy --version
mypy 1.10.0 (compiled: yes)
```

```bash
mypy aiohttp/test.py --python-version=3.11

Success: no issues found in 1 source file
```

Using `from typing import Unpack`

**Now:**

```python
    if sys.version_info >= (3, 11) and TYPE_CHECKING:
        from typing import Unpack

        def get(
            self,
            url: StrOrURL,
            **kwargs: Unpack[_RequestOptions],
        ) -> "_RequestContextManager": ...

        def options(
            self,
            url: StrOrURL,
            **kwargs: Unpack[_RequestOptions],
        ) -> "_RequestContextManager": ...

        def head(
            self,
            url: StrOrURL,
            **kwargs: Unpack[_RequestOptions],
        ) -> "_RequestContextManager": ...

        def post(
            self,
            url: StrOrURL,
            **kwargs: Unpack[_RequestOptions],
        ) -> "_RequestContextManager": ...

        def put(
            self,
            url: StrOrURL,
            **kwargs: Unpack[_RequestOptions],
        ) -> "_RequestContextManager": ...

        def patch(
            self,
            url: StrOrURL,
            **kwargs: Unpack[_RequestOptions],
        ) -> "_RequestContextManager": ...

        def delete(
            self,
            url: StrOrURL,
            **kwargs: Unpack[_RequestOptions],
        ) -> "_RequestContextManager": ...
```


```bash
aiohttp/client.py:1025:23: error: Variable "aiohttp.client.ClientSession.Unpack" is not valid as a type  [valid-type]
                **kwargs: Unpack[_RequestOptions],
                          ^
aiohttp/client.py:1025:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
aiohttp/client.py:1025:23: note: See https://mypy.rtfd.io/en/stable/_refs.html#code-valid-type for more info
aiohttp/client.py:1031:23: error: Variable "aiohttp.client.ClientSession.Unpack" is not valid as a type  [valid-type]
                **kwargs: Unpack[_RequestOptions],
                          ^
aiohttp/client.py:1031:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
aiohttp/client.py:1037:23: error: Variable "aiohttp.client.ClientSession.Unpack" is not valid as a type  [valid-type]
                **kwargs: Unpack[_RequestOptions],
                          ^
aiohttp/client.py:1037:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
aiohttp/client.py:1043:23: error: Variable "aiohttp.client.ClientSession.Unpack" is not valid as a type  [valid-type]
                **kwargs: Unpack[_RequestOptions],
                          ^
aiohttp/client.py:1043:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
aiohttp/client.py:1049:23: error: Variable "aiohttp.client.ClientSession.Unpack" is not valid as a type  [valid-type]
                **kwargs: Unpack[_RequestOptions],
                          ^
aiohttp/client.py:1049:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
aiohttp/client.py:1055:23: error: Variable "aiohttp.client.ClientSession.Unpack" is not valid as a type  [valid-type]
                **kwargs: Unpack[_RequestOptions],
                          ^
aiohttp/client.py:1055:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
aiohttp/client.py:1061:23: error: Variable "aiohttp.client.ClientSession.Unpack" is not valid as a type  [valid-type]
                **kwargs: Unpack[_RequestOptions],
                          ^
aiohttp/client.py:1061:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
aiohttp/test.py:10:25: note: Revealed type is "def (url: Union[builtins.str, yarl.URL], **kwargs: Unpack?[TypedDict('aiohttp.client._RequestOptions', {'params'?: Union[typing.Mapping[builtins.str, builtins.str], None], 'data'?: Any, 'json'?: Any, 'cookies'?: Union[typing.Mapping[builtins.str, Union[builtins.str, http.cookies.BaseCookie[builtins.str], http.cookies.Morsel[Any]]], typing.Iterable[tuple[builtins.str, Union[builtins.str, http.cookies.BaseCookie[builtins.str], http.cookies.Morsel[Any]]]], http.cookies.BaseCookie[builtins.str], None], 'headers'?: Union[typing.Mapping[Union[builtins.str, multidict.istr], builtins.str], multidict.CIMultiDict[builtins.str], multidict.CIMultiDictProxy[builtins.str], None], 'skip_auto_headers'?: Union[typing.Iterable[builtins.str], None], 'auth'?: Union[tuple[Any, Any, Any, fallback=aiohttp.helpers.BasicAuth], None], 'allow_redirects'?: builtins.bool, 'max_redirects'?: builtins.int, 'compress'?: Union[builtins.str, None], 'chunked'?: Union[builtins.bool, None], 'expect100'?: builtins.bool, 'raise_for_status'?: Union[None, builtins.bool, def (aiohttp.client_reqrep.ClientResponse) -> typing.Awaitable[None]], 'read_until_eof'?: builtins.bool, 'proxy'?: Union[builtins.str, yarl.URL, None], 'proxy_auth'?: Union[tuple[Any, Any, Any, fallback=aiohttp.helpers.BasicAuth], None], 'timeout'?: Union[aiohttp.client.ClientTimeout, aiohttp.helpers._SENTINEL, None], 'ssl'?: Union[ssl.SSLContext, builtins.bool, aiohttp.client_reqrep.Fingerprint], 'server_hostname'?: Union[builtins.str, None], 'proxy_headers'?: Union[typing.Mapping[Union[builtins.str, multidict.istr], builtins.str], multidict.CIMultiDict[builtins.str], multidict.CIMultiDictProxy[builtins.str], None], 'trace_request_ctx'?: Union[types.SimpleNamespace, None], 'read_bufsize'?: Union[builtins.int, None], 'auto_decompress'?: Union[builtins.bool, None], 'max_line_size'?: Union[builtins.int, None], 'max_field_size'?: Union[builtins.int, None]})]) -> aiohttp.client._RequestContextManager"
Found 7 errors in 1 file (checked 1 source file)
```



@author:
The revealed type is still fine, but it causes MyPy issues in the source file.

@user1:
Right, I missed the python-version. That error makes sense though, because you're creating a `ClientSession.Unpack` class attribute. I think it's best to move the import to the top of the module and just have 2 if conditions.

@author:
Fixed.

@author:
https://github.com/aio-libs/aiohttp/pull/8463/commits/e4af6b13c7e4d7ebc5dcac5a7d38bdf6a13619fc