Repository: python/mypy

Issue:
Some combos of *args/**kwds in caller/callee not supported
This spawned off #1553. The mapping of argument type from a call site to a function definition is complicated by the existence of `*args`, `**kwds`, optional args, and keyword-only args.

For example, Python 3.5+ support multiple `*args` in a call! (But not in a definition.) So this (valid) code gives two errors while there should be none:

``` python
def f(*a: int) -> int: return sum(a)
f(*[1, 2, 3], *[4, 5], 6)  # E: Too many arguments for "f"
f(*(1, 2, 3), *(4, 5), 6)  # E: Too many arguments for "f"
```

(It also doesn't give a SyntaxError with `--python-version=3.4` as it should.)

Star argument to format string in logging gives internal error
Hi,

the following code causes an internal error:

``` python
import logging
log=logging.getLogger()
v=("a", "b")
log.warning("Test (%s, %s)", *v)
```



Discussion:
Here's another example of something that doesn't work:

``` python
def f(*a: int) -> None:
    print(a)

f(1, 2, 3, 4)  # ok
f(1, 2, *[3, 4])  # ok
f(*[1, 2], 3, 4)  # E: Too many arguments for "f"
```

The reason in this case is that `[1, 2]` is treated as just `List[int]` and a `*x` argument where `x` is a sequence type, when the corresponding formal is also a `*` argument, is assumed to "fill up" the entire `*args`.

Could this also be related to #1553?

I guess I should have added the traceback(?):

```
*** INTERNAL ERROR ***

x.py:4: error: Internal error -- please report a bug at https://github.com/python/mypy/issues

NOTE: you can use "mypy --pdb ..." to drop into the debugger when this happens.
Traceback (most recent call last):
  File ".../bin/mypy", line 6, in <module>
    main(__file__)
  File ".../lib/python3.4/site-packages/mypy/main.py", line 55, in main
    res = type_check_only(sources, bin_dir, options)
  File ".../lib/python3.4/site-packages/mypy/main.py", line 103, in type_check_only
    python_path=options.python_path)
  File ".../lib/python3.4/site-packages/mypy/build.py", line 217, in build
    dispatch(sources, manager)
  File ".../lib/python3.4/site-packages/mypy/build.py", line 1383, in dispatch
    process_graph(graph, manager)
  File ".../lib/python3.4/site-packages/mypy/build.py", line 1521, in process_graph
    process_stale_scc(graph, scc)
  File ".../lib/python3.4/site-packages/mypy/build.py", line 1598, in process_stale_scc
    graph[id].type_check()
  File ".../lib/python3.4/site-packages/mypy/build.py", line 1361, in type_check
    manager.type_checker.visit_file(self.tree, self.xpath)
  File ".../lib/python3.4/site-packages/mypy/checker.py", line 169, in visit_file
    self.accept(d)
  File ".../lib/python3.4/site-packages/mypy/checker.py", line 218, in accept
    typ = node.accept(self)
  File ".../lib/python3.4/site-packages/mypy/nodes.py", line 728, in accept
    return visitor.visit_expression_stmt(self)
  File ".../lib/python3.4/site-packages/mypy/checker.py", line 1353, in visit_expression_stmt
    self.accept(s.expr)
  File ".../lib/python3.4/site-packages/mypy/checker.py", line 218, in accept
    typ = node.accept(self)
  File ".../lib/python3.4/site-packages/mypy/nodes.py", line 1184, in accept
    return visitor.visit_call_expr(self)
  File ".../lib/python3.4/site-packages/mypy/checker.py", line 1767, in visit_call_expr
    return self.expr_checker.visit_call_expr(e)
  File ".../lib/python3.4/site-packages/mypy/checkexpr.py", line 143, in visit_call_expr
    return self.check_call_expr_with_callee_type(callee_type, e)
  File ".../lib/python3.4/site-packages/mypy/checkexpr.py", line 201, in check_call_expr_with_callee_type
    e.arg_names, callable_node=e.callee)[0]
  File ".../lib/python3.4/site-packages/mypy/checkexpr.py", line 252, in check_call
    messages=arg_messages)
  File ".../lib/python3.4/site-packages/mypy/checkexpr.py", line 668, in check_argument_types
    tuple_counter)
  File ".../lib/python3.4/site-packages/mypy/constraints.py", line 86, in get_actual_type
    return arg_type.items[tuple_counter[0] - 1]
IndexError: list index out of range
```

Version is 9a92899e7c8416d43196d8bf424e7e2d86f27f0d. The traceback differs from your linked bug (which I found before), so I hope I figured correctly that this isn't just a dupe and I am not just pestering you guys with redundancy...

The only way to find out is to fix the bug and see how many issues it also
fixes.

--Guido (mobile)

Let's say it's a dupe of #1553.

The key clue that suggests this is likely a dupe is that the repro involves a `*`-argument which is a tuple.

It wasn't a duplicate, I'm still able to reproduce.

``` python
import logging
logger = logging.getLogger(__name__)
a = 1, 2
logger.debug('%s %s', *a)
```

gives

```
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/mypy", line 6, in <module>
    main(__file__)
  File "/usr/lib64/python3.4/site-packages/mypy/main.py", line 40, in main
    res = type_check_only(sources, bin_dir, options)
  File "/usr/lib64/python3.4/site-packages/mypy/main.py", line 81, in type_check_only
    options=options)
  File "/usr/lib64/python3.4/site-packages/mypy/build.py", line 178, in build
    dispatch(sources, manager)
  File "/usr/lib64/python3.4/site-packages/mypy/build.py", line 1325, in dispatch
    process_graph(graph, manager)
  File "/usr/lib64/python3.4/site-packages/mypy/build.py", line 1468, in process_graph
    process_stale_scc(graph, scc)
  File "/usr/lib64/python3.4/site-packages/mypy/build.py", line 1545, in process_stale_scc
    graph[id].type_check()
  File "/usr/lib64/python3.4/site-packages/mypy/build.py", line 1303, in type_check
    manager.type_checker.visit_file(self.tree, self.xpath)
  File "/usr/lib64/python3.4/site-packages/mypy/checker.py", line 152, in visit_file
    self.accept(d)
  File "/usr/lib64/python3.4/site-packages/mypy/checker.py", line 201, in accept
    typ = node.accept(self)
  File "/usr/lib64/python3.4/site-packages/mypy/nodes.py", line 740, in accept
    return visitor.visit_expression_stmt(self)
  File "/usr/lib64/python3.4/site-packages/mypy/checker.py", line 1412, in visit_expression_stmt
    self.accept(s.expr)
  File "/usr/lib64/python3.4/site-packages/mypy/checker.py", line 201, in accept
    typ = node.accept(self)
  File "/usr/lib64/python3.4/site-packages/mypy/nodes.py", line 1200, in accept
    return visitor.visit_call_expr(self)
  File "/usr/lib64/python3.4/site-packages/mypy/checker.py", line 1870, in visit_call_expr
    return self.expr_checker.visit_call_expr(e)
  File "/usr/lib64/python3.4/site-packages/mypy/checkexpr.py", line 144, in visit_call_expr
    return self.check_call_expr_with_callee_type(callee_type, e)
  File "/usr/lib64/python3.4/site-packages/mypy/checkexpr.py", line 202, in check_call_expr_with_callee_type
    e.arg_names, callable_node=e.callee)[0]
  File "/usr/lib64/python3.4/site-packages/mypy/checkexpr.py", line 253, in check_call
    messages=arg_messages)
  File "/usr/lib64/python3.4/site-packages/mypy/checkexpr.py", line 676, in check_argument_types
    tuple_counter)
  File "/usr/lib64/python3.4/site-packages/mypy/constraints.py", line 86, in get_actual_type
    return arg_type.items[tuple_counter[0] - 1]
IndexError: list index out of range

*** INTERNAL ERROR ***

/tmp/asd9.py:6: error: Internal error -- please report a bug at https://github.com/python/mypy/issues

NOTE: you can use "mypy --pdb ..." to drop into the debugger when this happens.
```

Yeah, still crashes. 


Resolving patch:
diff --git a/mypy/checkexpr.py b/mypy/checkexpr.py
--- a/mypy/checkexpr.py
+++ b/mypy/checkexpr.py
@@ -672,6 +672,11 @@ def check_argument_types(self, arg_types: List[Type], arg_kinds: List[int],
                     messages.invalid_keyword_var_arg(arg_type, context)
                 # Get the type of an individual actual argument (for *args
                 # and **args this is the item type, not the collection type).
+                if (isinstance(arg_type, TupleType)
+                        and tuple_counter[0] >= len(arg_type.items)
+                        and arg_kinds[actual] == nodes.ARG_STAR):
+                    # The tuple is exhausted. Continue with further arguments.
+                    continue
                 actual_type = get_actual_type(arg_type, arg_kinds[actual],
                                               tuple_counter)
                 check_arg(actual_type, arg_type, arg_kinds[actual],
@@ -1667,7 +1672,8 @@ def map_actuals_to_formals(caller_kinds: List[int],
                             map[j].append(i)
                         else:
                             break
-                        j += 1
+                        if callee_kinds[j] != nodes.ARG_STAR:
+                            j += 1
             else:
                 # Assume that it is an iterable (if it isn't, there will be
                 # an error later).
@@ -1676,6 +1682,8 @@ def map_actuals_to_formals(caller_kinds: List[int],
                         break
                     else:
                         map[j].append(i)
+                    if callee_kinds[j] == nodes.ARG_STAR:
+                        break
                     j += 1
         elif kind == nodes.ARG_NAMED:
             name = caller_names[i]


Test patch:
diff --git a/mypy/test/data.py b/mypy/test/data.py
--- a/mypy/test/data.py
+++ b/mypy/test/data.py
@@ -405,7 +405,7 @@ def repr_failure(self, excinfo):
             self.parent._prunetraceback(excinfo)
             excrepr = excinfo.getrepr(style='short')
 
-        return "data: {}:{}\n{}".format(self.obj.file, self.obj.line, excrepr)
+        return "data: {}:{}:\n{}".format(self.obj.file, self.obj.line, excrepr)
 
 
 class DataSuite:
diff --git a/test-data/unit/check-varargs.test b/test-data/unit/check-varargs.test
--- a/test-data/unit/check-varargs.test
+++ b/test-data/unit/check-varargs.test
@@ -112,6 +112,25 @@ f(*it1)
 f(*it2) # E: Argument 1 to "f" has incompatible type *Iterable[str]; expected "int"
 [builtins fixtures/for.py]
 
+[case testCallVarargsFunctionWithIterableAndPositional]
+# options: fast_parser
+from typing import Iterable
+it1 = None  # type: Iterable[int]
+def f(*x: int) -> None: pass
+f(*it1, 1, 2)
+f(*it1, 1, *it1, 2)
+f(*it1, '') # E: Argument 2 to "f" has incompatible type "str"; expected "int"
+[builtins fixtures/for.py]
+
+[case testCallVarargsFunctionWithTupleAndPositional]
+# options: fast_parser
+def f(*x: int) -> None: pass
+it1 = (1, 2)
+f(*it1, 1, 2)
+f(*it1, 1, *it1, 2)
+f(*it1, '') # E: Argument 2 to "f" has incompatible type "str"; expected "int"
+[builtins fixtures/for.py]
+
 
 -- Calling varargs function + type inference
 -- -----------------------------------------