Repository: python/mypy

Issue:
Add support for typing_extensions.Annotated
It would be great to support `Annotated` types as specified by [PEP 593](https://www.python.org/dev/peps/pep-0593/). This should be pretty easy to implement.


Discussion:
If nobody else is working on this, I'd be happy to do it.
OK, go ahead and try it!
Doing this right turned out to be significantly more difficult than I anticipated.

It's trivial to blindly strip away annotations when encountered, or to support anything that can be put in a `Literal`... But the PEP seems to allow literally any expression, such as function calls like `struct2.ctype("<10s")` and `ValueRange(-20, 3)`. A proper implementation would most likely require deferring these expressions until a new, later pass, or somehow validating and re-stringifying the expressions... and I don't know enough about mypy's internals to implement that.

If we're okay with just relying on plugins to handle any fancy business, then dropping all annotations and resolving to the inner type, I can still do that fairly easily. But I'm not quite sure what the vision is here going forward. 
For a first pass, I think it is totally fine to just discard the additional annotations. The important bit is that we are able to process code with `Annotated`, ignoring any annotations. (Given that we don't support any of them yet!)

In order to support actually using `Annotated` things, we probably want to be more liberal in what we shove into `RawExpressionType`, and actually store an `Expression` there. Then we would push back evaluation of that into actual literal values back into type analysis. Pinging @Michael0x2a to see if that makes sense?
But that doesn't need to be present in the initial pass.
Okay, thanks for the clarification. It looks like plugins will be able to use `get_type_analyze_hook` to make any changes before annotations are discarded, too.

I'll have something up this week for review, hopefully.
> In order to support actually using Annotated things, we probably want to be more liberal in what we shove into RawExpressionType, and actually store an Expression there. Then we would push back evaluation of that into actual literal values back into type analysis.

Yeah, this seems pretty reasonable to me. The only caveat is that we currently do a bit of hackery in the earlier parsing and semantic analysis layers to handle some str/unicode/bytes nonsense when constructing RawExpressionTypes. I'm not sure how complicated pushing this logic up into type analysis will be -- probably it won't be too bad? 

But if it gets too annoying, we could maybe sidestep the issue by having RawExpressionType contain a *union* of Expression and the existing allowed values. That would let us leave most of the existing hackery alone, just at the cost of introducing even more hackery. ¯\\\_(ツ)\_/¯

The only other main constraint is that we need to make sure RawExpressionTypes don't leak out of the semantic analysis/type analysis phase -- but the strategy of delegating everything to plugins and/or dropping all annotations ought to handle that.

Resolving patch:
diff --git a/mypy/newsemanal/typeanal.py b/mypy/newsemanal/typeanal.py
--- a/mypy/newsemanal/typeanal.py
+++ b/mypy/newsemanal/typeanal.py
@@ -41,6 +41,7 @@
     'typing.Union',
     'typing.Literal',
     'typing_extensions.Literal',
+    'typing_extensions.Annotated',
 }  # type: Final
 
 ARG_KINDS_BY_CONSTRUCTOR = {
@@ -302,6 +303,12 @@ def try_analyze_special_unbound_type(self, t: UnboundType, fullname: str) -> Opt
             return UninhabitedType(is_noreturn=True)
         elif fullname in ('typing_extensions.Literal', 'typing.Literal'):
             return self.analyze_literal_type(t)
+        elif fullname == 'typing_extensions.Annotated':
+            if len(t.args) < 2:
+                self.fail("Annotated[...] must have exactly one type argument"
+                          " and at least one annotation", t)
+                return AnyType(TypeOfAny.from_error)
+            return self.anal_type(t.args[0])
         return None
 
     def get_omitted_any(self, typ: Type, fullname: Optional[str] = None) -> AnyType:
diff --git a/mypy/typeanal.py b/mypy/typeanal.py
--- a/mypy/typeanal.py
+++ b/mypy/typeanal.py
@@ -40,6 +40,7 @@
     'typing.Union',
     'typing.Literal',
     'typing_extensions.Literal',
+    'typing_extensions.Annotated',
 }  # type: Final
 
 ARG_KINDS_BY_CONSTRUCTOR = {
@@ -312,6 +313,12 @@ def try_analyze_special_unbound_type(self, t: UnboundType, fullname: str) -> Opt
             return UninhabitedType(is_noreturn=True)
         elif fullname in ('typing_extensions.Literal', 'typing.Literal'):
             return self.analyze_literal_type(t)
+        elif fullname == 'typing_extensions.Annotated':
+            if len(t.args) < 2:
+                self.fail("Annotated[...] must have exactly one type argument"
+                          " and at least one annotation", t)
+                return AnyType(TypeOfAny.from_error)
+            return self.anal_type(t.args[0])
         return None
 
     def analyze_unbound_type_with_type_info(self, t: UnboundType, info: TypeInfo) -> Type:


Test patch:
diff --git a/mypy/test/testcheck.py b/mypy/test/testcheck.py
--- a/mypy/test/testcheck.py
+++ b/mypy/test/testcheck.py
@@ -86,6 +86,7 @@
     'check-newsemanal.test',
     'check-inline-config.test',
     'check-reports.test',
+    'check-annotated.test',
 ]
 
 # Tests that use Python 3.8-only AST features (like expression-scoped ignores):
diff --git a/test-data/unit/check-annotated.test b/test-data/unit/check-annotated.test
new file mode 100644
--- /dev/null
+++ b/test-data/unit/check-annotated.test
@@ -0,0 +1,65 @@
+[case testAnnotated0]
+from typing_extensions import Annotated
+x: Annotated[int, ...]
+reveal_type(x)  # N: Revealed type is 'builtins.int'
+
+[case testAnnotated1]
+from typing import Union
+from typing_extensions import Annotated
+x: Annotated[Union[int, str], ...]
+reveal_type(x)  # N: Revealed type is 'Union[builtins.int, builtins.str]'
+
+[case testAnnotated2]
+from typing_extensions import Annotated
+x: Annotated[int, THESE, ARE, IGNORED, FOR, NOW]
+reveal_type(x)  # N: Revealed type is 'builtins.int'
+
+[case testAnnotated3]
+from typing_extensions import Annotated
+x: Annotated[int, -+~12.3, "som"[e], more(anno+a+ions, that=[are]), (b"ignored",), 4, N.O.W, ...]
+reveal_type(x)  # N: Revealed type is 'builtins.int'
+
+[case testAnnotatedBadType]
+from typing_extensions import Annotated
+x: Annotated[XXX, ...]  # E: Name 'XXX' is not defined
+reveal_type(x)  # N: Revealed type is 'Any'
+
+[case testAnnotatedBadNoArgs]
+from typing_extensions import Annotated
+x: Annotated  # E: Annotated[...] must have exactly one type argument and at least one annotation
+reveal_type(x)  # N: Revealed type is 'Any'
+
+[case testAnnotatedBadOneArg]
+from typing_extensions import Annotated
+x: Annotated[int]  # E: Annotated[...] must have exactly one type argument and at least one annotation
+reveal_type(x)  # N: Revealed type is 'Any'
+
+[case testAnnotatedNested0]
+from typing_extensions import Annotated
+x: Annotated[Annotated[int, ...], ...]
+reveal_type(x)  # N: Revealed type is 'builtins.int'
+
+[case testAnnotatedNested1]
+from typing import Union
+from typing_extensions import Annotated
+x: Annotated[Annotated[Union[int, str], ...], ...]
+reveal_type(x)  # N: Revealed type is 'Union[builtins.int, builtins.str]'
+
+[case testAnnotatedNestedBadType]
+from typing_extensions import Annotated
+x: Annotated[Annotated[XXX, ...], ...]  # E: Name 'XXX' is not defined
+reveal_type(x)  # N: Revealed type is 'Any'
+
+[case testAnnotatedNestedBadNoArgs]
+from typing_extensions import Annotated
+x: Annotated[Annotated, ...]  # E: Annotated[...] must have exactly one type argument and at least one annotation
+reveal_type(x)  # N: Revealed type is 'Any'
+
+[case testAnnotatedNestedBadOneArg]
+from typing_extensions import Annotated
+x: Annotated[Annotated[int], ...]  # E: Annotated[...] must have exactly one type argument and at least one annotation
+reveal_type(x)  # N: Revealed type is 'Any'
+
+[case testAnnotatedNoImport]
+x: Annotated[int, ...]  # E: Name 'Annotated' is not defined
+reveal_type(x)  # N: Revealed type is 'Any'