Repository: python/cpython
Pull request: gh-133960: Improve typing.evaluate_forward_ref
As explained in #133960, this removes most of the behavior differences with ForwardRef.evaluate.
The remaining difference is about recursive evaluation of forwardrefs; this is practically useful
in cases where an annotation refers to a type alias that itself is string-valued.

This also improves several edge cases that were previously not handled optimally. For example,
the function now takes advantage of the partial evaluation behavior of ForwardRef.evaluate() to
evaluate more ForwardRefs in the FORWARDREF format.

This also fixes #133959 as a side effect, because the buggy behavior in #133959 derives from
evaluate_forward_ref().


<!-- gh-issue-number: gh-133960 -->
* Issue: gh-133960
<!-- /gh-issue-number -->


<!-- readthedocs-preview cpython-previews start -->
----
📚 Documentation preview 📚: https://cpython-previews--133961.org.readthedocs.build/

<!-- readthedocs-preview cpython-previews end -->

Problem statement:
`typing.get_type_hints()` raises on invalid types only if they're wrapped in a ForwardRef
```pytb
>>> import typing
>>> def f(x: typing.ClassVar[int]): pass
... 
>>> typing.get_type_hints(f)
{'x': typing.ClassVar[int]}
>>> def f(x: "typing.ClassVar[int]"): pass
... 
>>> typing.get_type_hints(f)
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    typing.get_type_hints(f)
    ~~~~~~~~~~~~~~~~~~~~~^^^
  File "/Users/jelle/py/cpython/Lib/typing.py", line 2390, in get_type_hints
    hints[name] = _eval_type(value, globalns, localns, type_params, format=format, owner=obj)
                  ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jelle/py/cpython/Lib/typing.py", line 448, in _eval_type
    return evaluate_forward_ref(t, globals=globalns, locals=localns,
                                type_params=type_params, owner=owner,
                                _recursive_guard=recursive_guard, format=format)
  File "/Users/jelle/py/cpython/Lib/typing.py", line 993, in evaluate_forward_ref
    type_ = _type_check(
        value,
    ...<2 lines>...
        allow_special_forms=forward_ref.__forward_is_class__,
    )
  File "/Users/jelle/py/cpython/Lib/typing.py", line 204, in _type_check
    raise TypeError(f"{arg} is not valid as type argument")
TypeError: typing.ClassVar[int] is not valid as type argument
```

(On main but also reproduced on 3.13.)

This seems inconsistent and I don't think it's helpful for users. We should either raise in both cases or neither. I prefer to not raise in either case, because it's less disruptive for callers (they already have to deal with `ClassVar` etc. potentially appearing in annotations), and provides more information.

Hints from issue discussion:

Commit message: gh-133960: Improve typing.evaluate_forward_ref

As explained in #133960, this removes most of the behavior differences with ForwardRef.evaluate.
The remaining difference is about recursive evaluation of forwardrefs; this is practically useful
in cases where an annotation refers to a type alias that itself is string-valued.

This also improves several edge cases that were previously not handled optimally. For example,
the function now takes advantage of the partial evaluation behavior of ForwardRef.evaluate() to
evaluate more ForwardRefs in the FORWARDREF format.

This also fixes #133959 as a side effect, because the buggy behavior in #133959 derives from
evaluate_forward_ref().

Patch to review:
diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
index 54cc3ea3311adf..dd8ea3c364f49a 100644
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -3500,20 +3500,11 @@ Introspection helpers
    Evaluate an :class:`annotationlib.ForwardRef` as a :term:`type hint`.
 
    This is similar to calling :meth:`annotationlib.ForwardRef.evaluate`,
-   but unlike that method, :func:`!evaluate_forward_ref` also:
-
-   * Recursively evaluates forward references nested within the type hint.
-   * Raises :exc:`TypeError` when it encounters certain objects that are
-     not valid type hints.
-   * Replaces type hints that evaluate to :const:`!None` with
-     :class:`types.NoneType`.
-   * Supports the :attr:`~annotationlib.Format.FORWARDREF` and
-     :attr:`~annotationlib.Format.STRING` formats.
+   but unlike that method, :func:`!evaluate_forward_ref` also
+   recursively evaluates forward references nested within the type hint.
 
    See the documentation for :meth:`annotationlib.ForwardRef.evaluate` for
-   the meaning of the *owner*, *globals*, *locals*, and *type_params* parameters.
-   *format* specifies the format of the annotation and is a member of
-   the :class:`annotationlib.Format` enum.
+   the meaning of the *owner*, *globals*, *locals*, *type_params*, and *format* parameters.
 
    .. versionadded:: 3.14
 
diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
index 6ef633e4545aef..6c6a162531f984 100644
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -6859,12 +6859,10 @@ def test_forward_ref_and_final(self):
         self.assertEqual(hints, {'value': Final})
 
     def test_top_level_class_var(self):
-        # https://bugs.python.org/issue45166
-        with self.assertRaisesRegex(
-            TypeError,
-            r'typing.ClassVar\[int\] is not valid as type argument',
-        ):
-            get_type_hints(ann_module6)
+        # This is not meaningful but we don't raise for it.
+        # https://github.com/python/cpython/issues/133959
+        hints = get_type_hints(ann_module6)
+        self.assertEqual(hints, {'wrong': ClassVar[int]})
 
     def test_get_type_hints_typeddict(self):
         self.assertEqual(get_type_hints(TotalMovie), {'title': str, 'year': int})
@@ -6967,6 +6965,11 @@ def foo(a: 'Callable[..., T]'):
         self.assertEqual(get_type_hints(foo, globals(), locals()),
                          {'a': Callable[..., T]})
 
+    def test_special_forms_no_forward(self):
+        def f(x: ClassVar[int]):
+            pass
+        self.assertEqual(get_type_hints(f), {'x': ClassVar[int]})
+
     def test_special_forms_forward(self):
 
         class C:
@@ -6982,8 +6985,9 @@ class CF:
         self.assertEqual(get_type_hints(C, globals())['b'], Final[int])
         self.assertEqual(get_type_hints(C, globals())['x'], ClassVar)
         self.assertEqual(get_type_hints(C, globals())['y'], Final)
-        with self.assertRaises(TypeError):
-            get_type_hints(CF, globals()),
+        lfi = get_type_hints(CF, globals())['b']
+        self.assertIs(get_origin(lfi), list)
+        self.assertEqual(get_args(lfi), (Final[int],))
 
     def test_union_forward_recursion(self):
         ValueList = List['Value']
@@ -7216,33 +7220,112 @@ class C(Generic[T]): pass
 class EvaluateForwardRefTests(BaseTestCase):
     def test_evaluate_forward_ref(self):
         int_ref = ForwardRef('int')
-        missing = ForwardRef('missing')
+        self.assertIs(typing.evaluate_forward_ref(int_ref), int)
         self.assertIs(
             typing.evaluate_forward_ref(int_ref, type_params=()),
             int,
         )
+        self.assertIs(
+            typing.evaluate_forward_ref(int_ref, format=annotationlib.Format.VALUE),
+            int,
+        )
         self.assertIs(
             typing.evaluate_forward_ref(
-                int_ref, type_params=(), format=annotationlib.Format.FORWARDREF,
+                int_ref, format=annotationlib.Format.FORWARDREF,
             ),
             int,
         )
+        self.assertEqual(
+            typing.evaluate_forward_ref(
+                int_ref, format=annotationlib.Format.STRING,
+            ),
+            'int',
+        )
+
+    def test_evaluate_forward_ref_undefined(self):
+        missing = ForwardRef('missing')
+        with self.assertRaises(NameError):
+            typing.evaluate_forward_ref(missing)
         self.assertIs(
             typing.evaluate_forward_ref(
-                missing, type_params=(), format=annotationlib.Format.FORWARDREF,
+                missing, format=annotationlib.Format.FORWARDREF,
             ),
             missing,
         )
         self.assertEqual(
             typing.evaluate_forward_ref(
-                int_ref, type_params=(), format=annotationlib.Format.STRING,
+                missing, format=annotationlib.Format.STRING,
             ),
-            'int',
+            "missing",
         )
 
-    def test_evaluate_forward_ref_no_type_params(self):
-        ref = ForwardRef('int')
-        self.assertIs(typing.evaluate_forward_ref(ref), int)
+    def test_evaluate_forward_ref_nested(self):
+        ref = ForwardRef("int | list['str']")
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref),
+            int | list[str],
+        )
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, format=annotationlib.Format.FORWARDREF),
+            int | list[str],
+        )
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, format=annotationlib.Format.STRING),
+            "int | list['str']",
+        )
+
+        why = ForwardRef('"\'str\'"')
+        self.assertIs(typing.evaluate_forward_ref(why), str)
+
+    def test_evaluate_forward_ref_none(self):
+        none_ref = ForwardRef('None')
+        self.assertIs(typing.evaluate_forward_ref(none_ref), None)
+
+    def test_globals(self):
+        A = "str"
+        ref = ForwardRef('list[A]')
+        with self.assertRaises(NameError):
+            typing.evaluate_forward_ref(ref)
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, globals={'A': A}),
+            list[str],
+        )
+
+    def test_owner(self):
+        ref = ForwardRef("A")
+
+        with self.assertRaises(NameError):
+            typing.evaluate_forward_ref(ref)
+
+        # Now should pick up the globals of this module
+        self.assertIs(
+            typing.evaluate_forward_ref(ref, owner=Loop), A
+        )
+
+    def test_inherited_owner(self):
+        # owner passed to evaluate_forward_ref
+        ref = ForwardRef("list['A']")
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, owner=Loop),
+            list[A],
+        )
+
+        # owner set on the ForwardRef
+        ref = ForwardRef("list['A']", owner=Loop)
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref),
+            list[A],
+        )
+
+    def test_partial_evaluation(self):
+        ref = ForwardRef("list[A]")
+        with self.assertRaises(NameError):
+            typing.evaluate_forward_ref(ref)
+
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, format=annotationlib.Format.FORWARDREF),
+            list[EqualToForwardRef('A')],
+        )
 
 
 class CollectionsAbcTests(BaseTestCase):
diff --git a/Lib/typing.py b/Lib/typing.py
index 3d64480e1431c1..ea6554152b2b3d 100644
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -956,12 +956,8 @@ def evaluate_forward_ref(
     """Evaluate a forward reference as a type hint.
 
     This is similar to calling the ForwardRef.evaluate() method,
-    but unlike that method, evaluate_forward_ref() also:
-
-    * Recursively evaluates forward references nested within the type hint.
-    * Rejects certain objects that are not valid type hints.
-    * Replaces type hints that evaluate to None with types.NoneType.
-    * Supports the *FORWARDREF* and *STRING* formats.
+    but unlike that method, evaluate_forward_ref() also
+    recursively evaluates forward references nested within the type hint.
 
     *forward_ref* must be an instance of ForwardRef. *owner*, if given,
     should be the object that holds the annotations that the forward reference
@@ -981,23 +977,24 @@ def evaluate_forward_ref(
     if forward_ref.__forward_arg__ in _recursive_guard:
         return forward_ref
 
-    try:
-        value = forward_ref.evaluate(globals=globals, locals=locals,
-                                     type_params=type_params, owner=owner)
-    except NameError:
-        if format == _lazy_annotationlib.Format.FORWARDREF:
-            return forward_ref
-        else:
-            raise
-
-    type_ = _type_check(
-        value,
-        "Forward references must evaluate to types.",
-        is_argument=forward_ref.__forward_is_argument__,
-        allow_special_forms=forward_ref.__forward_is_class__,
-    )
+    if format is None:
+        format = _lazy_annotationlib.Format.VALUE
+    value = forward_ref.evaluate(globals=globals, locals=locals,
+                                 type_params=type_params, owner=owner, format=format)
+
+    if (isinstance(value, _lazy_annotationlib.ForwardRef)
+            and format == _lazy_annotationlib.Format.FORWARDREF):
+        return value
+
+    if isinstance(value, str):
+        value = _make_forward_ref(value, module=forward_ref.__forward_module__,
+                                  owner=owner or forward_ref.__owner__,
+                                  is_argument=forward_ref.__forward_is_argument__,
+                                  is_class=forward_ref.__forward_is_class__)
+    if owner is None:
+        owner = forward_ref.__owner__
     return _eval_type(
-        type_,
+        value,
         globals,
         locals,
         type_params,
@@ -2338,12 +2335,12 @@ def get_type_hints(obj, globalns=None, localns=None, include_extras=False,
                 # This only affects ForwardRefs.
                 base_globals, base_locals = base_locals, base_globals
             for name, value in ann.items():
-                if value is None:
-                    value = type(None)
                 if isinstance(value, str):
                     value = _make_forward_ref(value, is_argument=False, is_class=True)
                 value = _eval_type(value, base_globals, base_locals, base.__type_params__,
                                    format=format, owner=obj)
+                if value is None:
+                    value = type(None)
                 hints[name] = value
         if include_extras or format == Format.STRING:
             return hints
@@ -2377,8 +2374,6 @@ def get_type_hints(obj, globalns=None, localns=None, include_extras=False,
         localns = globalns
     type_params = getattr(obj, "__type_params__", ())
     for name, value in hints.items():
-        if value is None:
-            value = type(None)
         if isinstance(value, str):
             # class-level forward refs were handled above, this must be either
             # a module-level annotation or a function argument annotation
@@ -2387,7 +2382,10 @@ def get_type_hints(obj, globalns=None, localns=None, include_extras=False,
                 is_argument=not isinstance(obj, types.ModuleType),
                 is_class=False,
             )
-        hints[name] = _eval_type(value, globalns, localns, type_params, format=format, owner=obj)
+        value = _eval_type(value, globalns, localns, type_params, format=format, owner=obj)
+        if value is None:
+            value = type(None)
+        hints[name] = value
     return hints if include_extras else {k: _strip_annotations(t) for k, t in hints.items()}
 
 
diff --git a/Misc/NEWS.d/next/Library/2025-05-12-20-38-57.gh-issue-133960.Aee79f.rst b/Misc/NEWS.d/next/Library/2025-05-12-20-38-57.gh-issue-133960.Aee79f.rst
new file mode 100644
index 00000000000000..66e8483b25bc37
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2025-05-12-20-38-57.gh-issue-133960.Aee79f.rst
@@ -0,0 +1,3 @@
+Simplify and improve :func:`typing.evaluate_forward_ref`. It now no longer
+raises errors on certain invalid types. In several situations, it is now
+able to evaluate forward references that were previously unsupported.


Code review comments:
diff_hunk: @@ -7216,33 +7220,112 @@ class C(Generic[T]): pass
 class EvaluateForwardRefTests(BaseTestCase):
     def test_evaluate_forward_ref(self):
         int_ref = ForwardRef('int')
-        missing = ForwardRef('missing')
+        self.assertIs(typing.evaluate_forward_ref(int_ref), int)
         self.assertIs(
             typing.evaluate_forward_ref(int_ref, type_params=()),
             int,
         )
+        self.assertIs(
+            typing.evaluate_forward_ref(int_ref, format=annotationlib.Format.VALUE),
+            int,
+        )
         self.assertIs(
             typing.evaluate_forward_ref(
-                int_ref, type_params=(), format=annotationlib.Format.FORWARDREF,
+                int_ref, format=annotationlib.Format.FORWARDREF,
             ),
             int,
         )
+        self.assertEqual(
+            typing.evaluate_forward_ref(
+                int_ref, format=annotationlib.Format.STRING,
+            ),
+            'int',
+        )
+
+    def test_evaluate_forward_ref_undefined(self):
+        missing = ForwardRef('missing')
+        with self.assertRaises(NameError):
+            typing.evaluate_forward_ref(missing)
         self.assertIs(
             typing.evaluate_forward_ref(
-                missing, type_params=(), format=annotationlib.Format.FORWARDREF,
+                missing, format=annotationlib.Format.FORWARDREF,
             ),
             missing,
         )
         self.assertEqual(
             typing.evaluate_forward_ref(
-                int_ref, type_params=(), format=annotationlib.Format.STRING,
+                missing, format=annotationlib.Format.STRING,
             ),
-            'int',
+            "missing",
         )
 
-    def test_evaluate_forward_ref_no_type_params(self):
-        ref = ForwardRef('int')
-        self.assertIs(typing.evaluate_forward_ref(ref), int)
+    def test_evaluate_forward_ref_nested(self):
+        ref = ForwardRef("int | list['str']")
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref),
+            int | list[str],
+        )
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, format=annotationlib.Format.FORWARDREF),
+            int | list[str],
+        )
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, format=annotationlib.Format.STRING),
+            "int | list['str']",
+        )
+
+        why = ForwardRef('"\'str\'"')
+        self.assertIs(typing.evaluate_forward_ref(why), str)
+
+    def test_evaluate_forward_ref_none(self):
+        none_ref = ForwardRef('None')
+        self.assertIs(typing.evaluate_forward_ref(none_ref), None)
+
+    def test_globals(self):
+        A = "str"
+        ref = ForwardRef('list[A]')
+        with self.assertRaises(NameError):
+            typing.evaluate_forward_ref(ref)
+        self.assertEqual(
+            typing.evaluate_forward_ref(ref, globals={'A': A}),
+            list[str],
+        )
+
+    def test_owner(self):
+        ref = ForwardRef("A")
+
+        with self.assertRaises(NameError):
+            typing.evaluate_forward_ref(ref)
+
+        # Now should pick up the globals of this module
path: Lib/test/test_typing.py
text: @user1:
```suggestion
        # We default to the globals of `owner`,
        # so it no longer raises `NameError`
```