Repository: python/mypy
Pull request: Add more details for "Invalid type" errors
Fixes https://github.com/python/mypy/issues/4030

This adds some more details to various `Invalid type` errors, and similar such as `Invalid type alias`, and `Invalid base class`. Unfortunately, `MessageBuilder` is not available in `typeanal.py`, so I put some error message formatting logic there.

Please add ideas about how to improve these errors.

Problem statement:
Better error message for "Invalid type"
There several situations where a quite terse error message is given: ``Invalid type "__main__.T"`` -- this could be an attempt to use a variable as a type, attempt to use function as a type, unsupported forward reference, maybe some other. I propose to make this error message more specific, or add a note:
```
error: Invalid type "__main__.f"
note: Functions can't be used as a type
```
and
```
error: Invalid type "__main__.Alias"
note: Variables can't be used as a type
```
The second can be also useful if someone accidentally created a variable instead of an alias. In this case we can even append a link to docs, if we detect that variable has type ``Type[...]``.

Another similar terse error message is ``Invalid base class``. We can say which base is invalid, this will be useful in case of multiple bases on the same line, and maybe also add a note in some cases explaining why it is invalid, like ``note: Union types can't be subclassed``, so that it is clear what went wrong.

Hints from issue discussion:
Also e.g.
```py
class C:
  A = int

a: C.A  # error: Invalid type "_.C.A"
```
This case is actually quite important, especially while https://github.com/python/mypy/issues/3494 (document variables vs aliases) is not fixed.
A rare case where adding "Invalid type" would be actually helpful is the error message for using subscriptable metaclasses in type context, see #6313.
A similar example where we should give a better error message is (recently closed) https://github.com/python/mypy/issues/2477
One more example is cast targets, see https://github.com/python/mypy/issues/6862

Commit message: More test updates

Patch to review:
diff --git a/mypy/newsemanal/semanal.py b/mypy/newsemanal/semanal.py
index 3fe7d03f5090..898145fe4999 100644
--- a/mypy/newsemanal/semanal.py
+++ b/mypy/newsemanal/semanal.py
@@ -1348,6 +1348,18 @@ def make_empty_type_info(self, defn: ClassDef) -> TypeInfo:
             info.set_line(defn)
         return info
 
+    def get_name_repr_of_expr(self, expr: Expression) -> Optional[str]:
+        """Try finding a short simplified textual representation of a base class expression."""
+        if isinstance(expr, NameExpr):
+            return expr.name
+        if isinstance(expr, MemberExpr):
+            return get_member_expr_fullname(expr)
+        if isinstance(expr, IndexExpr):
+            return self.get_name_repr_of_expr(expr.base)
+        if isinstance(expr, CallExpr):
+            return self.get_name_repr_of_expr(expr.callee)
+        return None
+
     def analyze_base_classes(
             self,
             base_type_exprs: List[Expression]) -> Optional[Tuple[List[Tuple[Type, Expression]],
@@ -1371,7 +1383,13 @@ def analyze_base_classes(
             try:
                 base = self.expr_to_analyzed_type(base_expr, allow_placeholder=True)
             except TypeTranslationError:
-                self.fail('Invalid base class', base_expr)
+                msg = 'Invalid base class'
+                name = self.get_name_repr_of_expr(base_expr)
+                if name:
+                    msg += ' "{}"'.format(name)
+                if isinstance(base_expr, CallExpr):
+                    msg += ': Unsupported dynamic base class'
+                self.fail(msg, base_expr)
                 is_error = True
                 continue
             if base is None:
@@ -1409,7 +1427,11 @@ def configure_base_classes(self,
                     self.fail(msg, base_expr)
                 info.fallback_to_any = True
             else:
-                self.fail('Invalid base class', base_expr)
+                msg = 'Invalid base class'
+                name = self.get_name_repr_of_expr(base_expr)
+                if name:
+                    msg += ' "{}"'.format(name)
+                self.fail(msg, base_expr)
                 info.fallback_to_any = True
             if self.options.disallow_any_unimported and has_any_from_unimported_type(base):
                 if isinstance(base_expr, (NameExpr, MemberExpr)):
@@ -1421,7 +1443,7 @@ def configure_base_classes(self,
                                    context=base_expr)
 
         # Add 'object' as implicit base if there is no other base class.
-        if (not base_types and defn.fullname != 'builtins.object'):
+        if not base_types and defn.fullname != 'builtins.object':
             base_types.append(self.object_type())
 
         info.bases = base_types
@@ -3186,7 +3208,7 @@ def visit_with_stmt(self, s: WithStmt) -> None:
             actual_targets = [t for t in s.target if t is not None]
             if len(actual_targets) == 0:
                 # We have a type for no targets
-                self.fail('Invalid type comment', s)
+                self.fail('Invalid type comment: `with` statement has no targets', s)
             elif len(actual_targets) == 1:
                 # We have one target and one type
                 types = [s.unanalyzed_type]
diff --git a/mypy/newsemanal/typeanal.py b/mypy/newsemanal/typeanal.py
index 61290e8ee822..52709220448b 100644
--- a/mypy/newsemanal/typeanal.py
+++ b/mypy/newsemanal/typeanal.py
@@ -22,7 +22,7 @@
     TypeInfo, Context, SymbolTableNode, Var, Expression,
     nongen_builtins, check_arg_names, check_arg_kinds, ARG_POS, ARG_NAMED,
     ARG_OPT, ARG_NAMED_OPT, ARG_STAR, ARG_STAR2, TypeVarExpr,
-    TypeAlias, PlaceholderNode
+    TypeAlias, PlaceholderNode, SYMBOL_FUNCBASE_TYPES, Decorator, MypyFile
 )
 from mypy.typetraverser import TypeTraverserVisitor
 from mypy.tvar_scope import TypeVarScope
@@ -72,7 +72,7 @@ def analyze_type_alias(node: Expression,
     try:
         type = expr_to_unanalyzed_type(node)
     except TypeTranslationError:
-        api.fail('Invalid type alias', node)
+        api.fail('Invalid type alias: cannot interpret right hand side as a type', node)
         return None
     analyzer = TypeAnalyser(api, tvar_scope, plugin, options, is_typeshed_stub,
                             allow_unnormalized=allow_unnormalized, defining_alias=True,
@@ -401,7 +401,18 @@ def analyze_unbound_type_without_type_info(self, t: UnboundType, sym: SymbolTabl
         # None of the above options worked. We parse the args (if there are any)
         # to make sure there are no remaining semanal-only types, then give up.
         t = t.copy_modified(args=self.anal_array(t.args))
-        self.fail('Invalid type "{}"'.format(name), t)
+        # TODO: Move this message building logic to messages.py.
+        if isinstance(sym.node, Var):
+            reason = 'Cannot use variable as type'  # TODO: add link to alias docs, see #3494
+        elif isinstance(sym.node, (SYMBOL_FUNCBASE_TYPES, Decorator)):
+            reason = 'Cannot use function as type, use Callable[...] or a protocol instead'
+        elif isinstance(sym.node, MypyFile):
+            reason = 'Module cannot be used as type'  # TODO: suggest a protocol when supported
+        elif unbound_tvar:
+            reason = 'Can only use bound type variables as types'
+        else:
+            reason = 'Cannot interpret reference as a type'
+        self.fail('Invalid type "{}": {}'.format(name, reason), t)
 
         # TODO: Would it be better to always return Any instead of UnboundType
         # in case of an error? On one hand, UnboundType has a name so error messages
@@ -422,7 +433,7 @@ def visit_deleted_type(self, t: DeletedType) -> Type:
         return t
 
     def visit_type_list(self, t: TypeList) -> Type:
-        self.fail('Invalid type', t)
+        self.fail('Invalid type: Did you want to use "List[...]"?', t)
         return AnyType(TypeOfAny.from_error)
 
     def visit_callable_argument(self, t: CallableArgument) -> Type:
diff --git a/test-data/unit/check-basic.test b/test-data/unit/check-basic.test
index fa13f6d5d7bd..fda04c2022f6 100644
--- a/test-data/unit/check-basic.test
+++ b/test-data/unit/check-basic.test
@@ -325,13 +325,14 @@ import e
 1+'no'  # E: Unsupported operand types for + ("int" and "str")
 
 [case testModuleAsTypeNoCrash]
+# flags: --new-semantic-analyzer
 import mock
 from typing import Union
 
 class A: ...
 class B: ...
 
-x: Union[mock, A]  # E: Invalid type "mock"
+x: Union[mock, A]  # E: Invalid type "mock": Module cannot be used as type
 
 if isinstance(x, B):
     pass
@@ -340,13 +341,14 @@ if isinstance(x, B):
 [out]
 
 [case testModuleAsTypeNoCrash2]
+# flags: --new-semantic-analyzer
 import mock
 from typing import overload, Any, Union
 
 @overload
 def f(x: int) -> int: ...
 @overload
-def f(x: str) -> Union[mock, str]: ...  # E: Invalid type "mock"
+def f(x: str) -> Union[mock, str]: ...  # E: Invalid type "mock": Module cannot be used as type
 def f(x):
     pass
 
diff --git a/test-data/unit/check-classes.test b/test-data/unit/check-classes.test
index 563149956ea1..0984b5ba3386 100644
--- a/test-data/unit/check-classes.test
+++ b/test-data/unit/check-classes.test
@@ -4833,12 +4833,13 @@ reveal_type(Arc1[MyDestr]())  # N: Revealed type is '__main__.Arc1[__main__.MyDe
 [typing fixtures/typing-full.pyi]
 
 [case testSixMetaclassErrors]
+# flags: --new-semantic-analyzer
 import six
 class M(type): pass
 class A(object): pass
 def f() -> type: return M
-class C1(six.with_metaclass(M), object): pass  # E: Invalid base class
-class C2(C1, six.with_metaclass(M)): pass  # E: Invalid base class
+class C1(six.with_metaclass(M), object): pass  # E: Invalid base class "six.with_metaclass": Unsupported dynamic base class
+class C2(C1, six.with_metaclass(M)): pass  # E: Invalid base class "six.with_metaclass": Unsupported dynamic base class
 class C3(six.with_metaclass(A)): pass  # E: Metaclasses not inheriting from 'type' are not supported
 @six.add_metaclass(A)  # E: Argument 1 to "add_metaclass" has incompatible type "Type[A]"; expected "Type[type]" \
                        # E: Metaclasses not inheriting from 'type' are not supported
diff --git a/test-data/unit/check-columns.test b/test-data/unit/check-columns.test
index 17fb8d0695db..d7d2a6a91be2 100644
--- a/test-data/unit/check-columns.test
+++ b/test-data/unit/check-columns.test
@@ -69,32 +69,34 @@ if int():
     (f(b=object())) # E:6: Unexpected keyword argument "b" for "f"
 
 [case testColumnInvalidType]
+# flags: --new-semantic-analyzer
 from typing import Iterable
 
 bad = 0
 
-def f(x: bad): # E:10: Invalid type "__main__.bad"
-    y: bad # E:8: Invalid type "__main__.bad"
+def f(x: bad): # E:10: Invalid type "__main__.bad": Cannot use variable as type
+    y: bad # E:8: Invalid type "__main__.bad": Cannot use variable as type
 
 if int():
-    def g(x): # E:5: Invalid type "__main__.bad"
+    def g(x): # E:5: Invalid type "__main__.bad": Cannot use variable as type
         # type: (bad) -> None
-        y = 0  # type: bad  # E:9: Invalid type "__main__.bad"
+        y = 0  # type: bad  # E:9: Invalid type "__main__.bad": Cannot use variable as type
 
-z: Iterable[bad] # E:13: Invalid type "__main__.bad"
-h: bad[int] # E:4: Invalid type "__main__.bad"
+z: Iterable[bad] # E:13: Invalid type "__main__.bad": Cannot use variable as type
+h: bad[int] # E:4: Invalid type "__main__.bad": Cannot use variable as type
 
 [case testColumnInvalidType_python2]
+# flags: --new-semantic-analyzer
 from typing import Iterable
 
 bad = 0
 
 if int():
-    def g(x): # E:5: Invalid type "__main__.bad"
+    def g(x): # E:5: Invalid type "__main__.bad": Cannot use variable as type
         # type: (bad) -> None
-        y = 0  # type: bad  # E:9: Invalid type "__main__.bad"
+        y = 0  # type: bad  # E:9: Invalid type "__main__.bad": Cannot use variable as type
 
-    z = ()  # type: Iterable[bad] # E:5: Invalid type "__main__.bad"
+    z = ()  # type: Iterable[bad] # E:5: Invalid type "__main__.bad": Cannot use variable as type
 
 [case testColumnFunctionMissingTypeAnnotation]
 # flags: --disallow-untyped-defs
diff --git a/test-data/unit/check-custom-plugin.test b/test-data/unit/check-custom-plugin.test
index ce3b1c4e665e..3bd3f08182d0 100644
--- a/test-data/unit/check-custom-plugin.test
+++ b/test-data/unit/check-custom-plugin.test
@@ -496,12 +496,12 @@ from mod import declarative_base, Column, Instr, non_declarative_base
 Bad1 = non_declarative_base()
 Bad2 = Bad3 = declarative_base()
 
-class C1(Bad1): ...  # E: Invalid type "__main__.Bad1" \
-                     # E: Invalid base class
-class C2(Bad2): ...  # E: Invalid type "__main__.Bad2" \
-                     # E: Invalid base class
-class C3(Bad3): ...  # E: Invalid type "__main__.Bad3" \
-                     # E: Invalid base class
+class C1(Bad1): ...  # E: Invalid type "__main__.Bad1": Cannot use variable as type \
+                     # E: Invalid base class "Bad1"
+class C2(Bad2): ...  # E: Invalid type "__main__.Bad2": Cannot use variable as type \
+                     # E: Invalid base class "Bad2"
+class C3(Bad3): ...  # E: Invalid type "__main__.Bad3": Cannot use variable as type \
+                     # E: Invalid base class "Bad3"
 [file mod.py]
 from typing import Generic, TypeVar
 def declarative_base(): ...
diff --git a/test-data/unit/check-functions.test b/test-data/unit/check-functions.test
index 2d96022cb057..62b9b808a31b 100644
--- a/test-data/unit/check-functions.test
+++ b/test-data/unit/check-functions.test
@@ -1720,6 +1720,7 @@ def Arg(x, y): pass
 F = Callable[[Arg(int, 'x')], int]  # E: Invalid argument constructor "__main__.Arg"
 
 [case testCallableParsingFromExpr]
+# flags: --new-semantic-analyzer
 from typing import Callable, List
 from mypy_extensions import Arg, VarArg, KwArg
 import mypy_extensions
@@ -1737,12 +1738,12 @@ J = Callable[[VarArg(), KwArg()], int] # ok
 K = Callable[[VarArg(), int], int] # E: Required positional args may not appear after default, named or var args
 L = Callable[[Arg(name='x', type=int)], int] # ok
 # I have commented out the following test because I don't know how to expect the "defined here" note part of the error.
-# M = Callable[[Arg(gnome='x', type=int)], int]   E: Invalid type alias   E: Unexpected keyword argument "gnome" for "Arg"
+# M = Callable[[Arg(gnome='x', type=int)], int]   E: Invalid type alias: cannot interpret right hand side as a type   E: Unexpected keyword argument "gnome" for "Arg"
 N = Callable[[Arg(name=None, type=int)], int] # ok
-O = Callable[[List[Arg(int)]], int]  # E: Invalid type alias  # E: Value of type "int" is not indexable  # E: Type expected within [...] # E: The type "Type[List[Any]]" is not generic and not indexable
+O = Callable[[List[Arg(int)]], int]  # E: Invalid type alias: cannot interpret right hand side as a type  # E: Value of type "int" is not indexable  # E: Type expected within [...] # E: The type "Type[List[Any]]" is not generic and not indexable
 P = Callable[[mypy_extensions.VarArg(int)], int] # ok
-Q = Callable[[Arg(int, type=int)], int]  # E: Invalid type alias  # E: Value of type "int" is not indexable  # E: "Arg" gets multiple values for keyword argument "type"
-R = Callable[[Arg(int, 'x', name='y')], int]  # E: Invalid type alias  # E: Value of type "int" is not indexable  # E: "Arg" gets multiple values for keyword argument "name"
+Q = Callable[[Arg(int, type=int)], int]  # E: Invalid type alias: cannot interpret right hand side as a type  # E: Value of type "int" is not indexable  # E: "Arg" gets multiple values for keyword argument "type"
+R = Callable[[Arg(int, 'x', name='y')], int]  # E: Invalid type alias: cannot interpret right hand side as a type  # E: Value of type "int" is not indexable  # E: "Arg" gets multiple values for keyword argument "name"
 
 [builtins fixtures/dict.pyi]
 
diff --git a/test-data/unit/check-generics.test b/test-data/unit/check-generics.test
index cbf483db7303..9f729d19c889 100644
--- a/test-data/unit/check-generics.test
+++ b/test-data/unit/check-generics.test
@@ -722,6 +722,7 @@ reveal_type(f('a'))  # N: Revealed type is '__main__.D[builtins.str*]'
 main:15: error: Argument 1 to "D" has incompatible type "int"; expected "Tuple[T, T]"
 
 [case testGenericTypeAliasesSubclassingBad]
+# flags: --new-semantic-analyzer
 from typing import TypeVar, Generic, Tuple, Union
 T = TypeVar('T')
 class Node(Generic[T]):
@@ -733,7 +734,7 @@ UNode = Union[int, Node[T]]
 
 class C(TupledNode): ... # Same as TupledNode[Any]
 class D(TupledNode[T]): ...
-class E(Generic[T], UNode[T]): ... # E: Invalid base class
+class E(Generic[T], UNode[T]): ... # E: Invalid base class "UNode"
 
 reveal_type(D((1, 1))) # N: Revealed type is '__main__.D[builtins.int*]'
 [builtins fixtures/list.pyi]
@@ -962,6 +963,7 @@ O[int]  # E: Bad number of arguments for type alias, expected: 0, given: 1  # E:
 [out]
 
 [case testAliasesInClassBodyNormalVsSubscripted]
+# flags: --new-semantic-analyzer
 from typing import Union, Type, Iterable
 
 class A: pass
@@ -976,9 +978,9 @@ class C:
         b = int  # E: Cannot assign multiple types to name "b" without an explicit "Type[...]" annotation
     if int():
         c = int
-    def f(self, x: a) -> None: pass  # E: Invalid type "__main__.C.a"
+    def f(self, x: a) -> None: pass  # E: Invalid type "__main__.C.a": Cannot use variable as type
     def g(self, x: b) -> None: pass
-    def h(self, x: c) -> None: pass  # E: Invalid type "__main__.C.c"
+    def h(self, x: c) -> None: pass  # E: Invalid type "__main__.C.c": Cannot use variable as type
     x: b
     reveal_type(x)  # N: Revealed type is 'Union[builtins.int, builtins.str]'
 [out]
diff --git a/test-data/unit/check-incremental.test b/test-data/unit/check-incremental.test
index 02a7077f4fe2..819580852503 100644
--- a/test-data/unit/check-incremental.test
+++ b/test-data/unit/check-incremental.test
@@ -2180,6 +2180,7 @@ tmp/b.py:1: error: Module 'c' has no attribute 'x'
 tmp/b.py:1: error: Module 'c' has no attribute 'x'
 
 [case testCacheDeletedAfterErrorsFound2]
+# flags: --new-semantic-analyzer
 import a
 [file a.py]
 from b import x
@@ -2195,9 +2196,9 @@ from b import x
 1 + 1
 [out]
 [out2]
-tmp/b.py:2: error: Invalid type "c.C"
+tmp/b.py:2: error: Invalid type "c.C": Cannot use function as type, use Callable[...] or a protocol instead
 [out3]
-tmp/b.py:2: error: Invalid type "c.C"
+tmp/b.py:2: error: Invalid type "c.C": Cannot use function as type, use Callable[...] or a protocol instead
 
 [case testCacheDeletedAfterErrorsFound3]
 import a
diff --git a/test-data/unit/check-literal.test b/test-data/unit/check-literal.test
index b2cffb02f1f6..4cfa56204765 100644
--- a/test-data/unit/check-literal.test
+++ b/test-data/unit/check-literal.test
@@ -694,8 +694,9 @@ y: Foo[Foo]     # E: Literal[...] must have at least one parameter
 [out]
 
 [case testLiteralBadRawExpressionWithBadType]
+# flags: --new-semantic-analyzer
 NotAType = 3
-def f() -> NotAType['also' + 'not' + 'a' + 'type']: ... # E: Invalid type "__main__.NotAType" \
+def f() -> NotAType['also' + 'not' + 'a' + 'type']: ... # E: Invalid type "__main__.NotAType": Cannot use variable as type \
                                                         # E: Invalid type comment or annotation
 
 # Note: this makes us re-inspect the type (e.g. via '_patch_indirect_dependencies'
@@ -876,6 +877,7 @@ reveal_type(d)      # N: Revealed type is 'Any'
 [out]
 
 [case testLiteralDisallowFloatsAndComplex]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal
 a1: Literal[3.14]    # E: Parameter 1 of Literal[...] cannot be of type "float"
 b1: 3.14             # E: Invalid type: float literals cannot be used as a type
@@ -889,10 +891,10 @@ d2t = 3j
 
 a2: a2t
 reveal_type(a2)      # N: Revealed type is 'Any'
-b2: b2t              # E: Invalid type "__main__.b2t"
+b2: b2t              # E: Invalid type "__main__.b2t": Cannot use variable as type
 c2: c2t
 reveal_type(c2)      # N: Revealed type is 'Any'
-d2: d2t              # E: Invalid type "__main__.d2t"
+d2: d2t              # E: Invalid type "__main__.d2t": Cannot use variable as type
 [builtins fixtures/complex_tuple.pyi]
 [out]
 
@@ -914,28 +916,31 @@ c: {"a": 1, "b": 2}             # E: Invalid type comment or annotation
 d: {1, 2, 3}                    # E: Invalid type comment or annotation
 
 [case testLiteralDisallowCollections2]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal
 a: (1, 2, 3)                    # E: Syntax error in type annotation \
                                 # N: Suggestion: Use Tuple[T1, ..., Tn] instead of (T1, ..., Tn)
 b: Literal[[1, 2, 3]]           # E: Parameter 1 of Literal[...] is invalid
-c: [1, 2, 3]                    # E: Invalid type
+c: [1, 2, 3]                    # E: Invalid type: Did you want to use "List[...]"?
 [out]
 
 [case testLiteralDisallowCollectionsTypeAlias]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal
-at = Literal[{"a": 1, "b": 2}]  # E: Invalid type alias
+at = Literal[{"a": 1, "b": 2}]  # E: Invalid type alias: cannot interpret right hand side as a type
 bt = {"a": 1, "b": 2}
-a: at                           # E: Invalid type "__main__.at"
-b: bt                           # E: Invalid type "__main__.bt"
+a: at                           # E: Invalid type "__main__.at": Cannot use variable as type
+b: bt                           # E: Invalid type "__main__.bt": Cannot use variable as type
 [builtins fixtures/dict.pyi]
 [out]
 
 [case testLiteralDisallowCollectionsTypeAlias2]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal
-at = Literal[{1, 2, 3}]         # E: Invalid type alias
+at = Literal[{1, 2, 3}]         # E: Invalid type alias: cannot interpret right hand side as a type
 bt = {1, 2, 3}
-a: at                           # E: Invalid type "__main__.at"
-b: bt                           # E: Invalid type "__main__.bt"
+a: at                           # E: Invalid type "__main__.at": Cannot use variable as type
+b: bt                           # E: Invalid type "__main__.bt": Cannot use variable as type
 [builtins fixtures/set.pyi]
 [out]
 
@@ -1776,16 +1781,17 @@ issubclass(int, indirect.Literal[3])  # E: Cannot use issubclass() with a Litera
 [out]
 
 [case testLiteralErrorsWhenSubclassed]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal
 from typing_extensions import Literal as Renamed
 import typing_extensions as indirect
 
 Alias = Literal[3]
 
-class Bad1(Literal[3]): pass            # E: Invalid base class
-class Bad2(Renamed[3]): pass            # E: Invalid base class
-class Bad3(indirect.Literal[3]): pass   # E: Invalid base class
-class Bad4(Alias): pass                 # E: Invalid base class
+class Bad1(Literal[3]): pass            # E: Invalid base class "Literal"
+class Bad2(Renamed[3]): pass            # E: Invalid base class "Renamed"
+class Bad3(indirect.Literal[3]): pass   # E: Invalid base class "indirect.Literal"
+class Bad4(Alias): pass                 # E: Invalid base class "Alias"
 [out]
 
 [case testLiteralErrorsWhenInvoked-skip]
@@ -2650,6 +2656,7 @@ over_literal(reveal_type(WrapperClass(var3)))   # N: Revealed type is '__main__.
 [out]
 
 [case testLiteralFinalUsedInLiteralType]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal, Final
 a: Final[int] = 3
 b: Final = 3
@@ -2662,13 +2669,13 @@ d: Literal[3]
 # "3" wherever it's used and get the same behavior -- so maybe we do need to support
 # at least case "b" for consistency?
 a_wrap: Literal[4, a]  # E: Parameter 2 of Literal[...] is invalid \
-                       # E: Invalid type "__main__.a"
+                       # E: Invalid type "__main__.a": Cannot use variable as type
 b_wrap: Literal[4, b]  # E: Parameter 2 of Literal[...] is invalid \
-                       # E: Invalid type "__main__.b"
+                       # E: Invalid type "__main__.b": Cannot use variable as type
 c_wrap: Literal[4, c]  # E: Parameter 2 of Literal[...] is invalid \
-                       # E: Invalid type "__main__.c"
+                       # E: Invalid type "__main__.c": Cannot use variable as type
 d_wrap: Literal[4, d]  # E: Parameter 2 of Literal[...] is invalid \
-                       # E: Invalid type "__main__.d"
+                       # E: Invalid type "__main__.d": Cannot use variable as type
 [out]
 
 [case testLiteralWithFinalPropagation]
@@ -2715,6 +2722,7 @@ expect_2(final_set_2.pop())  # E: Argument 1 to "expect_2" has incompatible type
 --
 
 [case testLiteralWithEnumsBasic]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal
 from enum import Enum
 
@@ -2729,7 +2737,7 @@ r: Literal[Color.RED]
 g: Literal[Color.GREEN]
 b: Literal[Color.BLUE]
 bad1: Literal[Color]         # E: Parameter 1 of Literal[...] is invalid
-bad2: Literal[Color.func]    # E: Invalid type "__main__.Color.func" \
+bad2: Literal[Color.func]    # E: Invalid type "__main__.Color.func": Cannot use function as type, use Callable[...] or a protocol instead \
                              # E: Parameter 1 of Literal[...] is invalid
 bad3: Literal[Color.func()]  # E: Invalid type: Literal[...] cannot contain arbitrary expressions
 
diff --git a/test-data/unit/check-modules.test b/test-data/unit/check-modules.test
index c390a66cf0ba..d78762a34d9d 100644
--- a/test-data/unit/check-modules.test
+++ b/test-data/unit/check-modules.test
@@ -807,6 +807,7 @@ import m.a
 [out]
 
 [case testCheckDecoratedFuncAsAnnotWithImportCycle]
+# flags: --new-semantic-analyzer
 import a
 [file a.py]
 from typing import TypeVar
@@ -823,7 +824,7 @@ MYPY = False
 if MYPY:
     from a import Session
 
-def f(self, session: Session) -> None:  # E: Invalid type "a.Session"
+def f(self, session: Session) -> None:  # E: Invalid type "a.Session": Cannot use function as type, use Callable[...] or a protocol instead
     pass
 [builtins fixtures/bool.pyi]
 
diff --git a/test-data/unit/check-newsemanal.test b/test-data/unit/check-newsemanal.test
index 7483d2513b76..00b8840ef46f 100644
--- a/test-data/unit/check-newsemanal.test
+++ b/test-data/unit/check-newsemanal.test
@@ -2546,7 +2546,7 @@ class C:
     def str(self) -> str:
         return 0 # E: Incompatible return value type (got "int", expected "str")
 
-    zz: str # E: Invalid type "__main__.C.str"
+    zz: str # E: Invalid type "__main__.C.str": Cannot use function as type, use Callable[...] or a protocol instead
 
 reveal_type(C().x()) # N: Revealed type is 'builtins.int'
 reveal_type(C().y()) # N: Revealed type is 'builtins.int'
@@ -2571,7 +2571,7 @@ class C:
     def str(self) -> str:
         return 0 # E: Incompatible return value type (got "int", expected "str")
 
-    zz: str # E: Invalid type "__main__.C.str"
+    zz: str # E: Invalid type "__main__.C.str": Cannot use function as type, use Callable[...] or a protocol instead
 
 reveal_type(C().x()) # N: Revealed type is 'builtins.int'
 reveal_type(C().y()) # N: Revealed type is 'builtins.int'
diff --git a/test-data/unit/check-newtype.test b/test-data/unit/check-newtype.test
index c35610705104..3d9eb535eda6 100644
--- a/test-data/unit/check-newtype.test
+++ b/test-data/unit/check-newtype.test
@@ -295,15 +295,16 @@ a = Foo(type(3))
 [out]
 
 [case testNewTypeWithTypeVarsFails]
+# flags: --new-semantic-analyzer
 from typing import NewType, TypeVar, List
 T = TypeVar('T')
 A = NewType('A', T)
 B = NewType('B', List[T])
 [builtins fixtures/list.pyi]
 [out]
-main:3: error: Argument 2 to NewType(...) must be subclassable (got T?)
-main:3: error: Invalid type "__main__.T"
-main:4: error: Invalid type "__main__.T"
+main:4: error: Argument 2 to NewType(...) must be subclassable (got T?)
+main:4: error: Invalid type "__main__.T": Can only use bound type variables as types
+main:5: error: Invalid type "__main__.T": Can only use bound type variables as types
 
 [case testNewTypeRedefiningVariablesFails]
 # flags: --new-semantic-analyzer
diff --git a/test-data/unit/check-protocols.test b/test-data/unit/check-protocols.test
index 0072502969d6..ea48b055e37a 100644
--- a/test-data/unit/check-protocols.test
+++ b/test-data/unit/check-protocols.test
@@ -306,6 +306,7 @@ main:11: note:     x: expected "int", got "None"
 -- --------------------------------
 
 [case testBasicSemanalErrorsInProtocols]
+# flags: --new-semantic-analyzer
 from typing import Protocol, Generic, TypeVar, Iterable
 
 T = TypeVar('T', covariant=True)
@@ -324,7 +325,7 @@ class P3(Protocol[T], Generic[S]): # E: Only single Generic[...] or Protocol[...
         pass
 
 class P4(Protocol[T]):
-    attr: Iterable[S] # E: Invalid type "__main__.S"
+    attr: Iterable[S] # E: Invalid type "__main__.S": Can only use bound type variables as types
 
 class P5(Iterable[S], Protocol[T]): # E: If Generic[...] or Protocol[...] is present it should list all type variables
     def meth(self) -> T:
diff --git a/test-data/unit/check-redefine.test b/test-data/unit/check-redefine.test
index 769f33cfe433..77b1d5ab2b1d 100644
--- a/test-data/unit/check-redefine.test
+++ b/test-data/unit/check-redefine.test
@@ -266,7 +266,7 @@ def f() -> None:
     class y: pass # E: Name 'y' already defined on line 5
 
 [case testRedefineVarAsTypeVar]
-# flags: --allow-redefinition
+# flags: --allow-redefinition --new-semantic-analyzer
 from typing import TypeVar
 def f() -> None:
     x = TypeVar('x')
@@ -276,7 +276,7 @@ def f() -> None:
     # NOTE: '"int" not callable' is due to test stubs
     y = TypeVar('y')  # E: Cannot redefine 'y' as a type variable \
                       # E: "int" not callable
-    def h(a: y) -> y: return a # E: Invalid type "y"
+    def h(a: y) -> y: return a # E: Invalid type "y": Cannot use variable as type
 
 [case testCannotRedefineVarAsModule]
 # flags: --allow-redefinition
diff --git a/test-data/unit/check-semanal-error.test b/test-data/unit/check-semanal-error.test
index 3be47bade386..3fa5445cc57a 100644
--- a/test-data/unit/check-semanal-error.test
+++ b/test-data/unit/check-semanal-error.test
@@ -49,15 +49,16 @@ A().foo(1)
 A().x = '' # E: Incompatible types in assignment (expression has type "str", variable has type "int")
 
 [case testInvalidBaseClass2]
+# flags: --new-semantic-analyzer
 X = 1
 class A(X): # E
     x = 1
 A().foo(1)
 A().x = '' # E
 [out]
-main:2: error: Invalid type "__main__.X"
-main:2: error: Invalid base class
-main:5: error: Incompatible types in assignment (expression has type "str", variable has type "int")
+main:3: error: Invalid type "__main__.X": Cannot use variable as type
+main:3: error: Invalid base class "X"
+main:6: error: Incompatible types in assignment (expression has type "str", variable has type "int")
 
 
 [case testInvalidNumberOfTypeArgs]
diff --git a/test-data/unit/check-statements.test b/test-data/unit/check-statements.test
index e51079e18e76..16d2ef06b71c 100644
--- a/test-data/unit/check-statements.test
+++ b/test-data/unit/check-statements.test
@@ -1351,13 +1351,13 @@ with A() as (a, b):
 [builtins fixtures/tuple.pyi]
 
 [case testWithStmtTypeComment]
-
+# flags: --new-semantic-analyzer
 from typing import Union
 class A:
     def __enter__(self) -> int: pass
     def __exit__(self, x, y, z): pass
 
-with A():  # type: int  # E: Invalid type comment
+with A():  # type: int  # E: Invalid type comment: `with` statement has no targets
     pass
 
 with A() as a:  # type: int
diff --git a/test-data/unit/check-type-aliases.test b/test-data/unit/check-type-aliases.test
index ae3ef301686a..f829f0b7870b 100644
--- a/test-data/unit/check-type-aliases.test
+++ b/test-data/unit/check-type-aliases.test
@@ -85,14 +85,15 @@ if int():
 [out]
 
 [case testProhibitUsingVariablesAsTypesAndAllowAliasesAsTypes]
+# flags: --new-semantic-analyzer
 from typing import TypeVar, Sequence, Type
 T = TypeVar('T')
 
 A: Type[float] = int
 if int():
     A = float  # OK
-x: A  # E: Invalid type "__main__.A"
-def bad(tp: A) -> None:  # E: Invalid type "__main__.A"
+x: A  # E: Invalid type "__main__.A": Cannot use variable as type
+def bad(tp: A) -> None:  # E: Invalid type "__main__.A": Cannot use variable as type
     pass
 
 Alias = int
diff --git a/test-data/unit/fine-grained-cycles.test b/test-data/unit/fine-grained-cycles.test
index e4206496b589..0da152118b25 100644
--- a/test-data/unit/fine-grained-cycles.test
+++ b/test-data/unit/fine-grained-cycles.test
@@ -204,7 +204,7 @@ def h() -> None:
 
 [out]
 ==
-a.py:3: error: Invalid type "b.C"
+a.py:3: error: Invalid type "b.C": Cannot use function as type, use Callable[...] or a protocol instead
 b.py:7: error: C? has no attribute "g"
 
 -- TODO: More import cycle:
diff --git a/test-data/unit/fine-grained.test b/test-data/unit/fine-grained.test
index 9a8301463fc7..24d93853af42 100644
--- a/test-data/unit/fine-grained.test
+++ b/test-data/unit/fine-grained.test
@@ -108,6 +108,7 @@ def g(x: int) -> None: pass
 main:2: error: Module has no attribute "f"
 
 [case testClassChangedIntoFunction]
+# flags: --new-semantic-analyzer
 import m
 def f(a: m.A) -> None:
     pass
@@ -117,9 +118,10 @@ class A: pass
 def A() -> None: pass
 [out]
 ==
-main:2: error: Invalid type "m.A"
+main:3: error: Invalid type "m.A": Cannot use function as type, use Callable[...] or a protocol instead
 
 [case testClassChangedIntoFunction2]
+# flags: --new-semantic-analyzer
 import m
 class B:
     def f(self, a: m.A) -> None: pass
@@ -130,9 +132,9 @@ def A() -> None: pass
 [file n.py.3]
 [out]
 ==
-main:3: error: Invalid type "m.A"
+main:4: error: Invalid type "m.A": Cannot use function as type, use Callable[...] or a protocol instead
 ==
-main:3: error: Invalid type "m.A"
+main:4: error: Invalid type "m.A": Cannot use function as type, use Callable[...] or a protocol instead
 
 [case testAttributeTypeChanged]
 import m
@@ -5331,10 +5333,10 @@ def T() -> None:
 [out]
 ==
 main:4: error: "C" expects no type arguments, but 1 given
-main:4: error: Invalid type "a.T"
+main:4: error: Invalid type "a.T": Cannot use function as type, use Callable[...] or a protocol instead
 main:6: error: Free type variable expected in Generic[...]
-main:7: error: Invalid type "a.T"
-main:10: error: Invalid type "a.T"
+main:7: error: Invalid type "a.T": Cannot use function as type, use Callable[...] or a protocol instead
+main:10: error: Invalid type "a.T": Cannot use function as type, use Callable[...] or a protocol instead
 main:10: error: Bad number of arguments for type alias, expected: 0, given: 1
 
 [case testChangeTypeVarToModule]
@@ -5360,13 +5362,14 @@ import T
 ==
 ==
 main:4: error: "C" expects no type arguments, but 1 given
-main:4: error: Invalid type "T"
+main:4: error: Invalid type "T": Module cannot be used as type
 main:6: error: Free type variable expected in Generic[...]
-main:7: error: Invalid type "T"
-main:10: error: Invalid type "T"
+main:7: error: Invalid type "T": Module cannot be used as type
+main:10: error: Invalid type "T": Module cannot be used as type
 main:10: error: Bad number of arguments for type alias, expected: 0, given: 1
 
 [case testChangeClassToModule]
+# flags: --new-semantic-analyzer
 import a
 x: a.C
 def f() -> None:
@@ -5385,9 +5388,9 @@ import C
 [out]
 ==
 ==
-main:2: error: Invalid type "C"
-main:4: error: Module not callable
-main:7: error: Invalid type "C"
+main:3: error: Invalid type "C": Module cannot be used as type
+main:5: error: Module not callable
+main:8: error: Invalid type "C": Module cannot be used as type
 
 [case testChangeTypeVarToTypeAlias]
 # flags: --new-semantic-analyzer
@@ -5414,6 +5417,7 @@ main:6: error: Free type variable expected in Generic[...]
 main:10: error: Bad number of arguments for type alias, expected: 0, given: 1
 
 [case testChangeTypeAliasToModule]
+# flags: --new-semantic-analyzer
 import a
 x: a.C
 def f() -> None:
@@ -5435,11 +5439,12 @@ import D
 [out]
 ==
 ==
-main:2: error: Invalid type "D"
-main:4: error: Module not callable
-main:7: error: Invalid type "D"
+main:3: error: Invalid type "D": Module cannot be used as type
+main:5: error: Module not callable
+main:8: error: Invalid type "D": Module cannot be used as type
 
 [case testChangeTypeAliasToModuleUnqualified]
+# flags: --new-semantic-analyzer
 from a import C
 x: C
 def f() -> None:
@@ -5461,9 +5466,9 @@ import D
 [out]
 ==
 ==
-main:2: error: Invalid type "D"
-main:4: error: Module not callable
-main:7: error: Invalid type "D"
+main:3: error: Invalid type "D": Module cannot be used as type
+main:5: error: Module not callable
+main:8: error: Invalid type "D": Module cannot be used as type
 
 [case testChangeFunctionToVariableAndRefreshUsingStaleDependency]
 import a
@@ -7842,15 +7847,15 @@ x = 1
 [out]
 a.py:1: error: Name 'TypeVar' is not defined
 a.py:1: note: Did you forget to import it from "typing"? (Suggestion: "from typing import TypeVar")
-a.py:7: error: Invalid type "a.T"
+a.py:7: error: Invalid type "a.T": Cannot use variable as type
 a.py:10: error: Name 'bar' already defined on line 6
-a.py:11: error: Invalid type "a.T"
+a.py:11: error: Invalid type "a.T": Cannot use variable as type
 ==
 a.py:1: error: Name 'TypeVar' is not defined
 a.py:1: note: Did you forget to import it from "typing"? (Suggestion: "from typing import TypeVar")
-a.py:7: error: Invalid type "a.T"
+a.py:7: error: Invalid type "a.T": Cannot use variable as type
 a.py:10: error: Name 'bar' already defined on line 6
-a.py:11: error: Invalid type "a.T"
+a.py:11: error: Invalid type "a.T": Cannot use variable as type
 
 [case testRefreshForWithTypeComment1]
 [file a.py]
@@ -8301,7 +8306,7 @@ x = 'no way'
 main:10: error: Incompatible types in assignment (expression has type "str", variable has type "int")
 
 [case testNamedTupleForwardFunctionDirect]
-# flags: --ignore-missing-imports
+# flags: --ignore-missing-imports --new-semantic-analyzer
 from typing import NamedTuple
 from b import B
 
@@ -8311,10 +8316,10 @@ def func(x): pass
 B = func
 [out]
 ==
-main:5: error: Invalid type "b.B"
+main:5: error: Invalid type "b.B": Cannot use variable as type
 
 [case testNamedTupleForwardFunctionIndirect]
-# flags: --ignore-missing-imports
+# flags: --ignore-missing-imports --new-semantic-analyzer
 from typing import NamedTuple
 from a import A
 
@@ -8327,10 +8332,10 @@ def func(x): pass
 B = func
 [out]
 ==
-main:5: error: Invalid type "a.A"
+main:5: error: Invalid type "a.A": Cannot use variable as type
 
 [case testNamedTupleForwardFunctionIndirectReveal]
-# flags: --ignore-missing-imports
+# flags: --ignore-missing-imports --new-semantic-analyzer
 import m
 [file m.py]
 from typing import NamedTuple
@@ -8353,14 +8358,14 @@ def func(x): pass
 B = func
 [out]
 ==
-m.py:4: error: Invalid type "a.A"
+m.py:4: error: Invalid type "a.A": Cannot use variable as type
 ==
-m.py:4: error: Invalid type "a.A"
+m.py:4: error: Invalid type "a.A": Cannot use variable as type
 m.py:5: note: Revealed type is 'A?'
 m.py:7: note: Revealed type is 'A?'
 
 [case testAliasForwardFunctionDirect]
-# flags: --ignore-missing-imports
+# flags: --ignore-missing-imports --new-semantic-analyzer
 from typing import Optional
 from b import B
 
@@ -8370,10 +8375,10 @@ def func(x): pass
 B = int()
 [out]
 ==
-main:5: error: Invalid type "b.B"
+main:5: error: Invalid type "b.B": Cannot use variable as type
 
 [case testAliasForwardFunctionIndirect]
-# flags: --ignore-missing-imports
+# flags: --ignore-missing-imports --new-semantic-analyzer
 from typing import Optional
 from a import A
 
@@ -8386,7 +8391,7 @@ def func(x): pass
 B = func
 [out]
 ==
-main:5: error: Invalid type "a.A"
+main:5: error: Invalid type "a.A": Cannot use variable as type
 
 [case testLiteralFineGrainedVarConversion]
 import mod
diff --git a/test-data/unit/merge.test b/test-data/unit/merge.test
index 68d3ca9b6ad0..71ded43a74a0 100644
--- a/test-data/unit/merge.test
+++ b/test-data/unit/merge.test
@@ -770,6 +770,7 @@ TypeVarExpr:2: Any
 NameExpr:4: T`1(upper_bound=target.A[Any]<1>)
 
 [case testUnboundType_types]
+# flags: --new-semantic-analyzer
 import target
 [file target.py]
 from typing import TypeVar, Generic
@@ -782,7 +783,7 @@ class A: pass
 foo: int
 x: foo[A]
 [out]
-tmp/target.py:4: error: Invalid type "target.foo"
+tmp/target.py:4: error: Invalid type "target.foo": Cannot use variable as type
 ## target
 TempNode:-1: Any
 TempNode:-1: Any
diff --git a/test-data/unit/pythoneval-asyncio.test b/test-data/unit/pythoneval-asyncio.test
index 64f0d63eb656..cd2aae26567a 100644
--- a/test-data/unit/pythoneval-asyncio.test
+++ b/test-data/unit/pythoneval-asyncio.test
@@ -485,7 +485,7 @@ loop.close()
 [out]
 _program.py:16: error: Incompatible types in assignment (expression has type "A", variable has type "B")
 
-[case testForwardRefToBadAsyncShouldNotCrash]
+[case testForwardRefToBadAsyncShouldNotCrash_newsemanal]
 from typing import TypeVar
 import asyncio
 
@@ -501,4 +501,4 @@ def bad(arg: P) -> T:
     pass
 [out]
 _program.py:8: note: Revealed type is 'def [T] (arg: P?) -> T`-1'
-_program.py:12: error: Invalid type "_testForwardRefToBadAsyncShouldNotCrash.P"
+_program.py:12: error: Invalid type "_testForwardRefToBadAsyncShouldNotCrash_newsemanal.P": Cannot use variable as type
diff --git a/test-data/unit/semanal-classes.test b/test-data/unit/semanal-classes.test
index 5969fabf94ad..b9d03f7af0d5 100644
--- a/test-data/unit/semanal-classes.test
+++ b/test-data/unit/semanal-classes.test
@@ -555,13 +555,14 @@ MypyFile:1(
         PassStmt:4()))))
 
 [case testInvalidBaseClass]
+# flags: --new-semantic-analyzer
 from typing import Any, Callable
 class A(None): pass
 class B(Any): pass
 class C(Callable[[], int]): pass
 [out]
-main:2: error: Invalid base class
-main:4: error: Invalid base class
+main:3: error: Invalid base class "None"
+main:5: error: Invalid base class "Callable"
 
 [case testTupleAsBaseClass]
 # flags: --new-semantic-analyzer
diff --git a/test-data/unit/semanal-classvar.test b/test-data/unit/semanal-classvar.test
index bdbcb0381fc4..c2895e733aec 100644
--- a/test-data/unit/semanal-classvar.test
+++ b/test-data/unit/semanal-classvar.test
@@ -55,12 +55,13 @@ MypyFile:1(
       Any)))
 
 [case testClassVarWithTypeVar]
+# flags: --new-semantic-analyzer
 from typing import ClassVar, TypeVar
 T = TypeVar('T')
 class A:
     x = None  # type: ClassVar[T]
 [out]
-main:4: error: Invalid type "__main__.T"
+main:5: error: Invalid type "__main__.T": Can only use bound type variables as types
 
 [case testClassVarInFunctionArgs]
 from typing import ClassVar
diff --git a/test-data/unit/semanal-errors.test b/test-data/unit/semanal-errors.test
index 6dc94e35a1c3..00dbe32cc835 100644
--- a/test-data/unit/semanal-errors.test
+++ b/test-data/unit/semanal-errors.test
@@ -126,14 +126,15 @@ y = None # type: Callable[[], A[int]]  # E: "A" expects no type arguments, but 1
 [out]
 
 [case testVarOrFuncAsType]
+# flags: --new-semantic-analyzer
 import typing
 def f(): pass
 x = 1
 y = 0 # type: f
 z = 0 # type: x
 [out]
-main:4: error: Invalid type "__main__.f"
-main:5: error: Invalid type "__main__.x"
+main:5: error: Invalid type "__main__.f": Cannot use function as type, use Callable[...] or a protocol instead
+main:6: error: Invalid type "__main__.x": Cannot use variable as type
 
 [case testGlobalVarRedefinition]
 import typing
@@ -489,20 +490,22 @@ del z     # E: Name 'z' is not defined
 [out]
 
 [case testFunctionTvarScope]
+# flags: --new-semantic-analyzer
 from typing import TypeVar
 t = TypeVar('t')
 def f(x: t) -> t: pass
 x = 0 # type: t
 [out]
-main:4: error: Invalid type "__main__.t"
+main:5: error: Invalid type "__main__.t": Can only use bound type variables as types
 
 [case testClassTvarScope]
+# flags: --new-semantic-analyzer
 from typing import Generic, TypeVar
 t = TypeVar('t')
 class c(Generic[t]): pass
 x = 0 # type: t
 [out]
-main:4: error: Invalid type "__main__.t"
+main:5: error: Invalid type "__main__.t": Can only use bound type variables as types
 
 [case testExpressionRefersToTypeVariable]
 from typing import TypeVar, Generic
@@ -736,14 +739,15 @@ foo = 0 # type: A.x      # E: Name 'A.x' is not defined
 [out]
 
 [case testTvarScopingWithNestedClass]
+# flags: --new-semantic-analyzer
 from typing import TypeVar, Generic
 t = TypeVar('t')
 s = TypeVar('s')
 class A(Generic[t]):
     class B(Generic[s]):
         x = 0 # type: A[s]
-        y = 0 # type: A[t]        # E: Invalid type "__main__.t"
-    z = 0 # type: A[s]            # E: Invalid type "__main__.s"
+        y = 0 # type: A[t]        # E: Invalid type "__main__.t": Can only use bound type variables as types
+    z = 0 # type: A[s]            # E: Invalid type "__main__.s": Can only use bound type variables as types
     a = 0 # type: A[t]
 [out]
 
@@ -784,19 +788,21 @@ A = None # E: Cannot assign to a type
 [out]
 
 [case testInvalidCastTargetSyntax]
+# flags: --new-semantic-analyzer
 from typing import cast, TypeVar, Generic
 t = TypeVar('t')
 class C(Generic[t]): pass
 cast(str + str, None)    # E: Cast target is not a type
 cast(C[str][str], None)  # E: Cast target is not a type
 cast(C[str + str], None) # E: Cast target is not a type
-cast([int, str], None)   # E: Invalid type
+cast([int, str], None)   # E: Invalid type: Did you want to use "List[...]"?
 [out]
 
 [case testInvalidCastTargetType]
+# flags: --new-semantic-analyzer
 from typing import cast
 x = 0
-cast(x, None)        # E: Invalid type "__main__.x"
+cast(x, None)        # E: Invalid type "__main__.x": Cannot use variable as type
 cast(t, None)        # E: Name 't' is not defined
 cast(__builtins__.x, None) # E: Name '__builtins__.x' is not defined
 [out]
@@ -826,7 +832,8 @@ Any(arg=str)  # E: Any(...) is no longer supported. Use cast(Any, ...) instead
 [out]
 
 [case testTypeListAsType]
-def f(x:[int, str]) -> None: # E: Invalid type
+# flags: --new-semantic-analyzer
+def f(x:[int, str]) -> None: # E: Invalid type: Did you want to use "List[...]"?
     pass
 [out]
 
@@ -885,10 +892,11 @@ class A:
 main:4: error: Type cannot be declared in assignment to non-self attribute
 
 [case testInvalidTypeInTypeApplication]
+# flags: --new-semantic-analyzer
 from typing import TypeVar, Generic
 t = TypeVar('t')
 class A(Generic[t]): pass
-A[TypeVar] # E: Invalid type "typing.TypeVar"
+A[TypeVar] # E: Invalid type "typing.TypeVar": Cannot use variable as type
 [out]
 
 [case testInvalidTypeInTypeApplication2]
@@ -1435,6 +1443,7 @@ class A: ... # E: Name 'A' already defined on line 2
 [out]
 
 [case testNoInvalidTypeInDynamicFunctions]
+# flags: --new-semantic-analyzer
 from typing import Dict, TypeVar
 T = TypeVar('T')
 
@@ -1446,7 +1455,7 @@ def f():  # Note no annotation
     t: nested
 
 def g() -> None:
-    x: Dict[str, T] = {}  # E: Invalid type "__main__.T"
+    x: Dict[str, T] = {}  # E: Invalid type "__main__.T": Can only use bound type variables as types
 
 [builtins fixtures/dict.pyi]
 [out]
diff --git a/test-data/unit/semanal-namedtuple.test b/test-data/unit/semanal-namedtuple.test
index 8dba24117726..d362a57db708 100644
--- a/test-data/unit/semanal-namedtuple.test
+++ b/test-data/unit/semanal-namedtuple.test
@@ -170,5 +170,5 @@ class B(A): pass
 class A(NamedTuple('N', [1])): pass
 class B(A): pass
 [out]
-main:2: error: Invalid base class
+main:2: error: Invalid base class "NamedTuple": Unsupported dynamic base class
 main:2: error: Name 'NamedTuple' is not defined
diff --git a/test-data/unit/semanal-typealiases.test b/test-data/unit/semanal-typealiases.test
index edc215c29c40..b0c1efb87a6f 100644
--- a/test-data/unit/semanal-typealiases.test
+++ b/test-data/unit/semanal-typealiases.test
@@ -400,14 +400,16 @@ MypyFile:1(
     Union[builtins.int, builtins.str]))
 
 [case testListTypeDoesNotGenerateAlias]
+# flags: --new-semantic-analyzer
 import typing
 A = [int, str]
-a = 1 # type: A  # E: Invalid type "__main__.A"
+a = 1 # type: A  # E: Invalid type "__main__.A": Cannot use variable as type
 
 [case testCantUseStringLiteralAsTypeAlias]
+# flags: --new-semantic-analyzer
 from typing import Union
 A = 'Union[int, str]'
-a = 1 # type: A  # E: Invalid type "__main__.A"
+a = 1 # type: A  # E: Invalid type "__main__.A": Cannot use variable as type
 
 [case testStringLiteralTypeAsAliasComponent]
 from typing import Union


Code review comments:
diff_hunk: @@ -2195,9 +2196,9 @@ from b import x
 1 + 1
 [out]
 [out2]
-tmp/b.py:2: error: Invalid type "c.C"
+tmp/b.py:2: error: Invalid type "c.C": Cannot use function as type, use Callable[...] or a protocol instead
path: test-data/unit/check-incremental.test
text: @user1:
Again, the first part could be shortened to `Function "c.C" is not valid as a type`. I'd move the final part to a separate note and rephrase it, since it's not clear of `Callable[...]` or a protocol is actually what the user wants. Maybe something like this:

```
b.py:2: error: Function "c.C" is not a valid type
b.py:2: note: Perhaps you need Callable[...] or a callback protocol?
```

diff_hunk: @@ -914,28 +916,31 @@ c: {"a": 1, "b": 2}             # E: Invalid type comment or annotation
 d: {1, 2, 3}                    # E: Invalid type comment or annotation
 
 [case testLiteralDisallowCollections2]
+# flags: --new-semantic-analyzer
 from typing_extensions import Literal
 a: (1, 2, 3)                    # E: Syntax error in type annotation \
                                 # N: Suggestion: Use Tuple[T1, ..., Tn] instead of (T1, ..., Tn)
 b: Literal[[1, 2, 3]]           # E: Parameter 1 of Literal[...] is invalid
-c: [1, 2, 3]                    # E: Invalid type
+c: [1, 2, 3]                    # E: Invalid type: Did you want to use "List[...]"?
path: test-data/unit/check-literal.test
text: @user1:
When we give a suggestion, we typically give it as a separate line. For example:

```
x.py:2: error: Expression "[...]" is not a valid type
x.py:2 note: Did you mean "List[t]"? 
```

We could do the same thing for `x: (int, str)` (`Tuple[t1, ..., tN]`), `x: {int}` (`Set[t]`) and `x: {int: str}` (`Dict[kt, vt]`).

@author:
We already do this for tuples, dict and set are more tricky (and I think more rare) so I would leave this for a separate PR.

diff_hunk: @@ -4833,12 +4833,13 @@ reveal_type(Arc1[MyDestr]())  # N: Revealed type is '__main__.Arc1[__main__.MyDe
 [typing fixtures/typing-full.pyi]
 
 [case testSixMetaclassErrors]
+# flags: --new-semantic-analyzer
 import six
 class M(type): pass
 class A(object): pass
 def f() -> type: return M
-class C1(six.with_metaclass(M), object): pass  # E: Invalid base class
-class C2(C1, six.with_metaclass(M)): pass  # E: Invalid base class
+class C1(six.with_metaclass(M), object): pass  # E: Invalid base class "six.with_metaclass": Unsupported dynamic base class
path: test-data/unit/check-classes.test
text: @user1:
Similar to above, this could be shortened to `Unsupported dynamic base class "six.with_metaclass"`.

diff_hunk: @@ -69,32 +69,34 @@ if int():
     (f(b=object())) # E:6: Unexpected keyword argument "b" for "f"
 
 [case testColumnInvalidType]
+# flags: --new-semantic-analyzer
 from typing import Iterable
 
 bad = 0
 
-def f(x: bad): # E:10: Invalid type "__main__.bad"
-    y: bad # E:8: Invalid type "__main__.bad"
+def f(x: bad): # E:10: Invalid type "__main__.bad": Cannot use variable as type
+    y: bad # E:8: Invalid type "__main__.bad": Cannot use variable as type
path: test-data/unit/check-columns.test
text: @user1:
Like above, this could be shortened to `Variable "__main__.bad" is not valid as a type`.