Repository: aio-libs-abandoned/aioredis-py

Issue:
[2.0] Type annotations break mypy
I tried porting an existing project to aioredis 2.0. I've got it almost working, but the type annotations that have been added are too strict (and in some cases just wrong) and break mypy. The main problem is that all the functions that take keys annotate them as `str`, when `bytes` (and I think several other types) are perfectly acceptable and are used in my code. The same applies to `register_script`.

The `min` and `max` arguments of `zrangebylex` and `zrevrangebylex` are annotated as int, but they're used for lexicographical sorting so are string-like.

Getting the type annotations right is a fair large undertaking. If there is a desire to release 2.0 soon I'd suggest deleting `py.typed` so that mypy doesn't see this package as annotated. There are annotations for redis-py in typeshed; perhaps that would be a good place to start, although I've occasionally also had issues there.


Discussion:
Hey @bmerry - thanks for checking this out. Would you be willing to collect the mypy complaints you're finding and add them to this ticket? I'd much prefer to get the typing right before we do an official release. 
Yes, I agree it would be nice to have type annotations - I'm a big fan of them.

I've listed the ones I've run into so far. I'll let you know if I encounter any more, but the project where I use aioredis only touches about a dozen commands so that's not great coverage. Just going through all the commands changing keys from `str` to `Union[str, bytes]` would be a good start.

Specifying the types does require imposing some opinions. One extreme is to use EncodableT for any argument that gets passed down to `send_command`. That is technically correct and shouldn't produce any false positives from mypy, but is probably too relaxed: if someone is passing an integer as a key they maybe did something wrong (and can easily cast it to string themselves), and conversely if someone is passing a string for something that a TTL that needs an integer, then possibly it's correct and they're passing a string containing an integer, but more likely it is an error.

Where a string-like is expecting I consider it important to accept both `str` and `bytes`. While `str` is more common, the Redis data model treats all keys and other strings as uninterpreted binary blobs, and given all the complications of UTF-8 it's not good to ask the user to turn their binary blob into a Unicode string and hope that it will round-trip back to `bytes`.

What I would suggest is defining some type aliases for different semantics: key, arbitrary value, integer, float etc; and using those to annotate the individual commands. That serves as handy documentation about the semantic types, and also makes it easier to late fine-tune the concrete types that will be accepted by each of them.

Let me know if you'd like some help with a PR to illustrate what I mean. I don't think I'll have time to go through the hundreds of commands, but I can provide some examples.
I've made a small PR (#1022) to fix a few specific issues. Once that's in I've got a much larger one waiting that implements some of my suggestions above and fixes a lot more issues I noticed.
Think it'd be a good idea to setup a mypy CI that always succeeds regardless of exit status (so that it won't be a blocker for future PRs).
@bmerry I've merged a couple of your recent PRs. Thanks for all your contributions to the type annotations! 

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -54,6 +54,7 @@ def read_version():
     extras_require={
         "hiredis": 'hiredis>=1.0; implementation_name=="cpython"',
     },
+    package_data={"aioredis": ["py.typed"]},
     python_requires=">=3.6",
     include_package_data=True,
 )


Test patch: