Repository: pandas-dev/pandas

Issue:
Replace Union annotations with TypeVar
Had a revelation after https://github.com/python/mypy/issues/6847#issuecomment-493715355 and I think we've been using Union too liberally in places where a TypeVar may make more sense (especially in pandas._typing) so would be good to go back and swap those out.

@gwrome @vaibhavhrt 


Discussion:
@WillAyd we are almost done with removing all ignored modules from mypy.ini Once that's done I will take a look where we have used Union and see if we can use TypeVar instead.
@WillAyd I will look for Union in `pandas._typing` and after fixing that will look for Union in other files.
Just to be sure typeVar for `Union[str, float]` would be `Something = TypeVar('Something', str, float)`, right?
Also can please provide some explanation on when to use TypeVar and when to use Union.
TypeVar is going to be more suitable for parametrization and generic functions. In the case of containers I think the difference between:

```python
T = TypeVar('T', str, float)
List[T]
```

and
```python
List[Union[str, float]]
```

Would be that the former could only accept `[1, 2]` and `['a', 'b']` but not `[1, 'a']`. The latter would accept all of those.

Looking at functions:

```python
def foo(something: T) -> T:
    ...
```

Would basically mean that if `foo` accepts a string it also returns a string, and if it accepts a float it returns a float, but that it cannot accept a string and return a float. By contrast:

```python
def foo(Union[str, float]) -> Union[str, float]:
    ...
```

Would allow all of those combinations, so it could accept 'a' and return 1.

We've used Union throughout so far but more often than not I think parametrization of the Type is actually what we want.
Nice, thanks for explaining, I will replace all unions in `pandas._typing` today.

Resolving patch:
diff --git a/pandas/_typing.py b/pandas/_typing.py
--- a/pandas/_typing.py
+++ b/pandas/_typing.py
@@ -1,5 +1,5 @@
 from pathlib import Path
-from typing import IO, AnyStr, Type, Union
+from typing import IO, AnyStr, TypeVar, Union
 
 import numpy as np
 
@@ -11,12 +11,14 @@
 from pandas.core.dtypes.generic import (
     ABCExtensionArray, ABCIndexClass, ABCSeries, ABCSparseSeries)
 
-AnyArrayLike = Union[ABCExtensionArray,
-                     ABCIndexClass,
-                     ABCSeries,
-                     ABCSparseSeries,
-                     np.ndarray]
-ArrayLike = Union[ABCExtensionArray, np.ndarray]
-DatetimeLikeScalar = Type[Union[Period, Timestamp, Timedelta]]
+AnyArrayLike = TypeVar('AnyArrayLike',
+                       ABCExtensionArray,
+                       ABCIndexClass,
+                       ABCSeries,
+                       ABCSparseSeries,
+                       np.ndarray)
+ArrayLike = TypeVar('ArrayLike', ABCExtensionArray, np.ndarray)
+DatetimeLikeScalar = TypeVar('DatetimeLikeScalar', Period, Timestamp,
+                             Timedelta)
 Dtype = Union[str, np.dtype, ExtensionDtype]
 FilePathOrBuffer = Union[str, Path, IO[AnyStr]]
diff --git a/pandas/core/arrays/datetimelike.py b/pandas/core/arrays/datetimelike.py
--- a/pandas/core/arrays/datetimelike.py
+++ b/pandas/core/arrays/datetimelike.py
@@ -1,6 +1,6 @@
 from datetime import datetime, timedelta
 import operator
-from typing import Any, Sequence, Union, cast
+from typing import Any, Sequence, Type, Union, cast
 import warnings
 
 import numpy as np
@@ -58,7 +58,7 @@ def _get_attributes_dict(self):
         return {k: getattr(self, k, None) for k in self._attributes}
 
     @property
-    def _scalar_type(self) -> DatetimeLikeScalar:
+    def _scalar_type(self) -> Type[DatetimeLikeScalar]:
         """The scalar associated with this datelike
 
         * PeriodArray : Period


Test patch: