Type promotion, result_type(), promote_types() and can_cast() in PyTorch
Tags: pytorch, typepromotion, resulttype, promotetypes
Published: 2024-05-11 23:00:06

*Memos:
- [My post](https://dev.to/hyperkai/check-pytorch-version-cpu-and-gpucuda-in-pytorch-6jk) explains how to check PyTorch version, CPU and GPU(CUDA).
- [My post](https://dev.to/hyperkai/create-a-tensor-in-pytorch-127g) explains how to create a tensor.
- [My post](https://dev.to/hyperkai/access-a-tensor-in-pytorch-1f4e) explains how to access a tensor.
- [My post](https://dev.to/hyperkai/istensor-numel-and-device-in-pytorch-2eha) explains [is_tensor()](https://pytorch.org/docs/stable/generated/torch.is_tensor.html), [numel()](https://pytorch.org/docs/stable/generated/torch.numel.html) and [device()](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device).
- [My post](https://dev.to/hyperkai/type-conversion-with-type-to-and-a-tensor-in-pytorch-2a0g) explains type conversion with [type()](https://pytorch.org/docs/stable/generated/torch.Tensor.type.html), [to()](https://pytorch.org/docs/stable/generated/torch.Tensor.to.html) and a tensor.
- [My post](https://dev.to/hyperkai/device-conversion-fromnumpy-and-numpy-in-pytorch-1iih) explains device conversion, [from_numpy()](https://pytorch.org/docs/stable/generated/torch.from_numpy.html) and [numpy()](https://pytorch.org/docs/stable/generated/torch.Tensor.numpy.html).
- [My post](https://dev.to/hyperkai/setdefaultdtype-setdefaultdevice-and-setprintoptions-in-pytorch-55g8) explains [set_default_dtype()](https://pytorch.org/docs/stable/generated/torch.set_default_dtype.html), [set_default_device()](https://pytorch.org/docs/stable/generated/torch.set_default_device.html) and [set_printoptions()](https://pytorch.org/docs/stable/generated/torch.set_printoptions.html).
- [My post](https://dev.to/hyperkai/manualseed-initialseed-and-seed-in-pytorch-5gm8) explains [manual_seed()](https://pytorch.org/docs/stable/generated/torch.manual_seed.html), [initial_seed()](https://pytorch.org/docs/stable/generated/torch.initial_seed.html) and [seed()](https://pytorch.org/docs/stable/generated/torch.seed.html).

Arithmetic operation like (addition, subtraction, multiplication, etc) has type promotion in PyTorch as shown below:

*Memos:
- The priority of type promotion is `complex` <- `float` <- `int` <- `bool`.
- If types' category is the same but the size is different, the higher size's one is taken. (e.g. `int32` and `int64`)

```python
import torch

tensor1 = torch.tensor([True, False, True], dtype=torch.bool)
tensor2 = torch.tensor([3, 4, 5], dtype=torch.int64)

torch.add(input=tensor1, other=tensor2)
# tensor([4, 4, 6])

torch.add(input=tensor1, other=tensor2).dtype
# torch.int64

tensor1 = torch.tensor([0, 1, 2], dtype=torch.int64)
tensor2 = torch.tensor([3., 4., 5.], dtype=torch.float32)

torch.add(input=tensor1, other=tensor2)
# tensor([3., 5., 7.])

torch.add(input=tensor1, other=tensor2).dtype
# torch.float32

tensor1 = torch.tensor([0., 1., 2.], dtype=torch.float32)
tensor2 = torch.tensor([3.+0.j, 4.+0.j, 5.+0.j], 
                       dtype=torch.complex64)
torch.add(input=tensor1, other=tensor2)
# tensor([3.+0.j, 5.+0.j, 7.+0.j])

torch.add(input=tensor1, other=tensor2).dtype
# torch.complex64

tensor1 = torch.tensor([0, 1, 2], dtype=torch.int32)
tensor2 = torch.tensor([3, 4, 5], dtype=torch.int64)

torch.add(input=tensor1, other=tensor2)
# tensor([3, 5, 7])

torch.add(input=tensor1, other=tensor2).dtype
# torch.int64

tensor1 = torch.tensor([0., 1., 2.], dtype=torch.float32)
tensor2 = torch.tensor([3., 4., 5.], dtype=torch.float64)

torch.add(input=tensor1, other=tensor2)
# tensor([3., 5., 7.], dtype=torch.float64)

torch.add(input=tensor1, other=tensor2).dtype
# torch.float64

tensor1 = torch.tensor([0.+0.j, 1.+0.j, 2.+0.j], 
                       dtype=torch.complex32)
tensor2 = torch.tensor([3.+0.j, 4.+0.j, 5.+0.j], 
                       dtype=torch.complex64)
torch.add(input=tensor1, other=tensor2)
# tensor([3.+0.j, 5.+0.j, 7.+0.j])

torch.add(input=tensor1, other=tensor2).dtype
# torch.complex64
```

[result_type()](https://pytorch.org/docs/stable/generated/torch.result_type.html) can check the result type of two of the 0D or more D tersors of zero or more elements or scalars or the 0D or more D tensor of zero or more elements and a scalar, getting a [dtype](https://pytorch.org/docs/stable/tensor_attributes.html#torch-dtype) as shown below:

*Memos:
- `result_type()` can be used with [torch](https://pytorch.org/docs/stable/torch.html) but not with a tensor.
- The 1st argument with `torch` is `tensor`(Required-Type:`tensor` or `scalar` of `int`, `float`, `complex` or `bool`). *A scalar must used without `tensor=`.
- The 2nd argument with `torch` is `other`(Required-Type:`tensor` or `scalar` of `int`, `float`, `complex` or `bool`). 

```python
import torch

tensor1 = torch.tensor([True, False, True], dtype=torch.bool)
tensor2 = torch.tensor([3, 4, 5], dtype=torch.int64)

torch.result_type(tensor=tensor1, other=tensor2)
torch.result_type(tensor=True, other=tensor2)
torch.result_type(tensor=tensor1, other=3)
torch.result_type(True, 3)
# torch.int64

tensor1 = torch.tensor([0, 1, 2], dtype=torch.int64)
tensor2 = torch.tensor([3., 4., 5.], dtype=torch.float32)

torch.result_type(tensor=tensor1, other=tensor2)
# torch.float32

tensor1 = torch.tensor([0., 1., 2.], dtype=torch.float32)
tensor2 = torch.tensor([3.+0.j, 4.+0.j, 5.+0.j], 
                       dtype=torch.complex64)
torch.result_type(tensor=tensor1, other=tensor2)
# torch.complex64

tensor1 = torch.tensor([0, 1, 2], dtype=torch.int32)
tensor2 = torch.tensor([3, 4, 5], dtype=torch.int64)

torch.result_type(tensor=tensor1, other=tensor2)
# torch.int64

tensor1 = torch.tensor([0., 1., 2.], dtype=torch.float32)
tensor2 = torch.tensor([3., 4., 5.], dtype=torch.float64)

torch.result_type(tensor=tensor1, other=tensor2)
# torch.float64

tensor1 = torch.tensor([0.+0.j, 1.+0.j, 2.+0.j], 
                       dtype=torch.complex32)
tensor2 = torch.tensor([3.+0.j, 4.+0.j, 5.+0.j], 
                       dtype=torch.complex64)
torch.result_type(tensor=tensor1, other=tensor2)
# torch.complex64
```

[promote_types()](https://pytorch.org/docs/stable/generated/torch.promote_types.html) can check the result type of two of dtypes, getting a [dtype](https://pytorch.org/docs/stable/tensor_attributes.html#torch-dtype) as shown below:

*Memos:
- `promote_types()` can be used with `torch` but not with a tensor.
- The 1st argument with `torch` is `type1`(Required-Type:[dtype](https://pytorch.org/docs/stable/tensor_attributes.html#torch-dtype)).
- The 2nd argument with `torch` is `type2`(Required-Type:`dtype`). 
- `type1` and `type2` can also accept [int()](https://docs.python.org/3/library/functions.html#int), [float()](https://docs.python.org/3/library/functions.html#float) and [bool()](https://docs.python.org/3/library/functions.html#bool) but not [complex()](https://docs.python.org/3/library/functions.html#complex) which are [python built-in functions](https://docs.python.org/3/library/functions.html).

```python
import torch

torch.promote_types(type1=torch.bool, type2=torch.int64)
torch.promote_types(type1=torch.bool, type2=int)
torch.promote_types(type1=bool, type2=torch.int64)
torch.promote_types(type1=bool, type2=int)
torch.promote_types(type1=torch.int32, type2=torch.int64)
# torch.int64

torch.promote_types(type1=torch.int64, type2=torch.float32)
# torch.float32

torch.promote_types(type1=torch.float32, type2=torch.complex64)
torch.promote_types(type1=torch.complex32, type2=torch.complex64)
# torch.complex64

torch.promote_types(type1=torch.float32, type2=torch.float64)
torch.promote_types(type1=int, type2=float)
# torch.float64
```

[can_cast()](https://pytorch.org/docs/stable/generated/torch.can_cast.html) can check the dtype's category(`from`) is promotable to the dtype'category(`to`), getting a boolean value as shown below:

*Memos:
- `can_cast()` can be used with `torch` but not with a tensor.
- The 1st argument with `torch` is `from`(Required-Type:[dtype](https://pytorch.org/docs/stable/tensor_attributes.html#torch-dtype)). *It must be without `from=`.
- The 2nd argument with `torch` is `to`(Required-Type:`dtype`).
- `from` and `to` can also accept [int()](https://docs.python.org/3/library/functions.html#int), [float()](https://docs.python.org/3/library/functions.html#float) and [bool()](https://docs.python.org/3/library/functions.html#bool) but not [complex()](https://docs.python.org/3/library/functions.html#complex) which are [python built-in functions](https://docs.python.org/3/library/functions.html).

```python
import torch

torch.can_cast(torch.bool, to=torch.int64)
torch.can_cast(bool, to=torch.int64)
torch.can_cast(torch.bool, to=int)
torch.can_cast(bool, to=int)
torch.can_cast(torch.int64, to=torch.float32)
torch.can_cast(torch.float32, to=torch.complex64)
torch.can_cast(torch.int32, to=torch.int64)
torch.can_cast(torch.int64, to=torch.int32)
torch.can_cast(torch.float32, to=torch.float64)
torch.can_cast(torch.float64, to=torch.float32)
torch.can_cast(torch.complex32, to=torch.complex64)
torch.can_cast(torch.complex64, to=torch.complex32)
# True

torch.can_cast(torch.int64, to=torch.bool)
torch.can_cast(int, to=torch.bool)
torch.can_cast(torch.int64, to=bool)
torch.can_cast(int, to=bool)
torch.can_cast(torch.float32, to=torch.int64)
torch.can_cast(torch.complex64, to=torch.float32)
# False
```