Repository: python/mypy

Issue:
Make any callable compatible with (*args: Any, **kwargs: Any)?
Perhaps a callable that accepts arbitrary arguments by having `(*args: Any, **kwargs: Any)` could be treated similarly to `Callable[..., X]` (with explicit `...`), i.e. a callable with arbitrary arguments would be compatible with it.

The reason for this is that there's no easy way to annotate a function so that the type would be `Callable[..., X]`, even though this seems to be a somewhat common need. Users sometimes assume that using `(*args: Any, **kwargs: Any)` will work for this purpose and are confused when it doesn't. https://github.com/python/typing/issues/264#issuecomment-436525286 is a recent example.

Example:

```py
from typing import Any

class A:
    # A method that accepts unspecified arguments and returns int
    def f(self, *args: Any, **kwargs: Any) -> int: pass

class B(A):
    def f(self, x: int) -> int: pass  # Should be okay
```

We could perhaps extend this to handle decorators as well:

```py
from typing import Any, Callable, TypeVar

T = TypeVar('T', bound=Callable[..., Any])

def deco(f: T) -> T:
    def wrapper(*args: Any, **kwargs: Any) -> Any:
        print('called')
        return f(*args, **kwargs)
    return wrapper  # Maybe this should be okay
```


Discussion:
I actually like this idea, I have seen this confusion several times, and although it is a bit unsafe, most of the time when people write `(*args, **kwargs)` it means "don't care", rather than "should work for all calls".
Agreed, this is a case where practicality beats purity.

I just stumbled upon this trying to build a Protocol to require a callable that takes an Event as the first parameter:

```
class SubprocessTarget(Protocol):
    def __call__(self, quit_event: MultiprocessEvent, *args: Any, **kwargs: Any) -> int: ...
```

Just throwing this out here, but maybe a variant of Any could be useful for defining callback signatures:

`...*args: AnyOrMissing, **kwargs: AnyOrMissing` (sorry for the poor choice of name)

Or maybe the __call__ could be decorated with something like `@OptionalVarArgs` to indicate that we really don't care about args/kwargs? Or the other way around, `@PartialSignature` to indicate that the method may include additional arguments, then the protocol signature only lists the required arguments (it doesn't have to include args/kwargs anymore).

Another similar (but different) use case would be to require the method to include a specific kwarg (or **kwargs), regardless of its position. I feel these will be hard to implement without the help of some decorators?
I do read the following words in [mypy docs](https://mypy.readthedocs.io/en/latest/kinds_of_types.html#callable-types-and-lambdas)

> Mypy recognizes a special form **Callable[..., T] (with a literal ...)** which can be used in less typical cases. It is compatible with arbitrary callable objects that return a type compatible with T, independent of the number, types or kinds of arguments. Mypy lets you call such callable values with arbitrary arguments, without any checking – in this respect they are **treated similar to a (\*args: Any, \*\*kwargs: Any) function signature**. 

Does it mean this issue has been resolved? Correct me if I am wrong.
@ckyogg: no, the issue here is to be able to enforce partial signatures to improve duck typing possibilities. Your excerpt is asking for "any callback whatsoever" but i want to define "a callback with a quit flag as first argument and anything else after" 
Has anyone found a good solution for this? My usecase is exactly the same as @jonapich I want to enforce the first argument and allow anything afterwards.
+1.  It seems like the only solution right now is to create a union type alias covering all cases, which is obviously something I'd like to avoid doing!

```python
class CallbackWithKwargs1(Protocol[T_contra, T_co]):
    def __call__(self, __element: T_contra, __arg1=...) -> T_co:
        pass

class CallbackWithKwargs2(Protocol[T_contra, T_co]):
    def __call__(self, __element: T_contra, __arg1=..., __arg2=...) -> T_co:
        pass

...

CallbackWithKwargs=Union[
    CallbackWithKwargs1, CallbackWithKwargs2, ...]
```
Another common real-world example is trying to create a type for a functional view in Django, where the first argument is always a `Request` and there might be more arguments for URL parameters.
@antonagestam perhaps https://github.com/python/mypy/issues/8263 could help resolve this issue?
**I** really just care about typing a function (in a prototype) with *arbitrary* arguments (except for perhaps `self`), and `def foo(self, *args: Any, **kwargs: Any) -> Any: ...` just doesn't work. What am I to do?
@jmehnle does something like `foo: Callable[..., Any]` work?
Tip for achieving this at present:

```python
from typing import Callable, Any


class Parent:
    method: Callable[..., int]
    def method(  # type: ignore
        self, *args: Any, **kwargs: Any,
    ) -> int:
        ...

class Child(Parent):
    def method(self, x: str) -> int:
        ...
```

Try it: https://mypy-play.net/?mypy=latest&python=3.8&gist=ba254920ee62608c5696c29610b2a379
> @jmehnle does something like `foo: Callable[..., Any]` work?

Alas, no. I get: `note: Protocol member Connection.cursor expected settable variable, got read-only attribute`.

And unfortunately @rmorshea's trick still generates that very same message (see also https://github.com/python/mypy/issues/9560).
Doesn't Guido's trick in #9560 work?
It does. I saw that only just now. I'm going to use that for the time being, but of course that still doesn't make any callable match the `(*args: Any, **kwargs: Any)` type (*this* issue).
I've got the same issue here, currently writing an interface that accepts a callback that has at least one specific parameter, the others are optional and can be handed to a registering function beforehand. I thus defined my protocol as follows:
```
from typing import Any

from typing_extensions import Protocol

class Callback(Protocol):
    def __call__(self, gain: bool, *args: Any, **kwargs: Any) -> None:
        ...
```

But now, every function passed with type Callback needs to explicitly have the structure 
```
def func(param: bool, *args: Any, **kwargs: Any)
```
although a function
```
def func(param: bool)
```
is totally valid as long as no additional arguments were supplied to the registering function before.
Is there any solution to this problem yet?
@Timtam I don't see `def func(param: bool)` as compatible. IMV, an object that satifies the given Callback protocol must accept a sequence of arguments and keyword arguments without raising a TypeError.
@Timtam I think that use case might be addressed here: https://github.com/python/typing/issues/696
The working workaround solution for now is to enumerate all possible calls. This works for me:

```python
from typing import Protocol, TypeVar, Callable, cast

class IRequest(Protocol):
    body: str
    
class IResponse(Protocol):
    code: int

Handler0 = Callable[[IRequest], IResponse]
Handler1 = Callable[[IRequest, str], IResponse]
Handler2 = Callable[[IRequest, str, str], IResponse]
Handler = TypeVar('Handler', Handler0, Handler1, Handler2)
    

def route(path: str) -> Callable[[Handler], Handler]:
    def wrapper(handler: Handler) -> Handler:
        return handler
    return wrapper

@route("/users") # <---------- pass
def api(req: IRequest) -> IResponse:
    return cast(IResponse, None)

@route("/user/{name}") # <---------- pass 
def user_by_name(req: IRequest, name: str) -> IResponse:
    return cast(IResponse, None)

@route("/user/{uid}") # <---------- fail
def user_by_id(req: IRequest, uid: int) -> IResponse:
    return cast(IResponse, None)
```
```zsh
➜ python -mmypy test.py
test.py:28: error: Value of type variable "Handler" of function cannot be "Callable[[IRequest, int], IResponse]"
Found 1 error in 1 file (checked 1 source file)
```



Resolving patch:
diff --git a/mypy/subtypes.py b/mypy/subtypes.py
--- a/mypy/subtypes.py
+++ b/mypy/subtypes.py
@@ -12,6 +12,8 @@
 # Circular import; done in the function instead.
 # import mypy.solve
 from mypy.nodes import (
+    ARG_STAR,
+    ARG_STAR2,
     CONTRAVARIANT,
     COVARIANT,
     Decorator,
@@ -1291,6 +1293,16 @@ def are_parameters_compatible(
     right_star = right.var_arg()
     right_star2 = right.kw_arg()
 
+    # Treat "def _(*a: Any, **kw: Any) -> X" similarly to "Callable[..., X]"
+    if (
+        right.arg_kinds == [ARG_STAR, ARG_STAR2]
+        and right_star
+        and isinstance(get_proper_type(right_star.typ), AnyType)
+        and right_star2
+        and isinstance(get_proper_type(right_star2.typ), AnyType)
+    ):
+        return True
+
     # Match up corresponding arguments and check them for compatibility. In
     # every pair (argL, argR) of corresponding arguments from L and R, argL must
     # be "more general" than argR if L is to be a subtype of R.


Test patch:
diff --git a/test-data/unit/check-classes.test b/test-data/unit/check-classes.test
--- a/test-data/unit/check-classes.test
+++ b/test-data/unit/check-classes.test
@@ -7136,6 +7136,16 @@ class B(A):  # E: Final class __main__.B has abstract attributes "foo"
 class C:
     class C1(XX): pass  # E: Name "XX" is not defined
 
+[case testArgsKwargsInheritance]
+from typing import Any
+
+class A(object):
+    def f(self, *args: Any, **kwargs: Any) -> int: ...
+
+class B(A):
+    def f(self, x: int) -> int: ...
+[builtins fixtures/dict.pyi]
+
 [case testClassScopeImports]
 class Foo:
     from mod import plain_function  # E: Unsupported class scoped import
diff --git a/test-data/unit/check-functions.test b/test-data/unit/check-functions.test
--- a/test-data/unit/check-functions.test
+++ b/test-data/unit/check-functions.test
@@ -188,9 +188,9 @@ if int():
     ee_var = everywhere
 
 if int():
-    ee_var = specific_1 # The difference between Callable[..., blah] and one with a *args: Any, **kwargs: Any is that the ... goes loosely both ways.
+    ee_var = specific_1
 if int():
-    ee_def = specific_1 # E: Incompatible types in assignment (expression has type "Callable[[int, str], None]", variable has type "Callable[[VarArg(Any), KwArg(Any)], None]")
+    ee_def = specific_1
 
 [builtins fixtures/dict.pyi]
 
@@ -1787,7 +1787,7 @@ def f2(*args, **kwargs) -> int: pass
 d(f1)
 e(f2)
 d(f2)
-e(f1)  # E: Argument 1 to "e" has incompatible type "Callable[[VarArg(Any)], int]"; expected "Callable[[VarArg(Any), KwArg(Any)], int]"
+e(f1)
 
 [builtins fixtures/dict.pyi]
 
diff --git a/test-data/unit/check-modules.test b/test-data/unit/check-modules.test
--- a/test-data/unit/check-modules.test
+++ b/test-data/unit/check-modules.test
@@ -3160,7 +3160,7 @@ from test1 import aaaa  # E: Module "test1" has no attribute "aaaa"
 import b
 [file a.py]
 class Foo:
-    def frobnicate(self, *args, **kwargs): pass
+    def frobnicate(self, x, *args, **kwargs): pass
 [file b.py]
 from a import Foo
 class Bar(Foo):
@@ -3178,12 +3178,12 @@ class Bar(Foo):
 [out1]
 tmp/b.py:3: error: Signature of "frobnicate" incompatible with supertype "Foo"
 tmp/b.py:3: note:      Superclass:
-tmp/b.py:3: note:          def frobnicate(self, *args: Any, **kwargs: Any) -> Any
+tmp/b.py:3: note:          def frobnicate(self, x: Any, *args: Any, **kwargs: Any) -> Any
 tmp/b.py:3: note:      Subclass:
 tmp/b.py:3: note:          def frobnicate(self) -> None
 [out2]
 tmp/b.py:3: error: Signature of "frobnicate" incompatible with supertype "Foo"
 tmp/b.py:3: note:      Superclass:
-tmp/b.py:3: note:          def frobnicate(self, *args: Any, **kwargs: Any) -> Any
+tmp/b.py:3: note:          def frobnicate(self, x: Any, *args: Any, **kwargs: Any) -> Any
 tmp/b.py:3: note:      Subclass:
 tmp/b.py:3: note:          def frobnicate(self, *args: Any) -> None