Device conversion, from_numpy() and numpy() in PyTorch
Tags: pytorch, device, conversion, numpy
Published: 2024-05-25 15:54:33

*Memos:
- [My post](https://dev.to/hyperkai/check-pytorch-version-cpu-and-gpucuda-in-pytorch-6jk) explains how to check PyTorch version, CPU and GPU(CUDA).
- [My post](https://dev.to/hyperkai/create-a-tensor-in-pytorch-127g) explains how to create a tensor.
- [My post](https://dev.to/hyperkai/access-a-tensor-in-pytorch-1f4e) explains how to access a tensor.
- [My post](https://dev.to/hyperkai/istensor-numel-and-device-in-pytorch-2eha) explains [is_tensor()](https://pytorch.org/docs/stable/generated/torch.is_tensor.html), [numel()](https://pytorch.org/docs/stable/generated/torch.numel.html) and [device()](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device).
- [My post](https://dev.to/hyperkai/type-conversion-with-type-to-and-a-tensor-in-pytorch-2a0g) explains type conversion with [type()](https://pytorch.org/docs/stable/generated/torch.Tensor.type.html), [to()](https://pytorch.org/docs/stable/generated/torch.Tensor.to.html) and a tensor.
- [My post](https://dev.to/hyperkai/type-promotion-resulttype-promotetypes-and-cancast-in-pytorch-33p8) explains type promotion, [result_type()](https://pytorch.org/docs/stable/generated/torch.result_type.html), [promote_types()](https://pytorch.org/docs/stable/generated/torch.promote_types.html) and [can_cast()](https://pytorch.org/docs/stable/generated/torch.can_cast.html).
- [My post](https://dev.to/hyperkai/setdefaultdtype-setdefaultdevice-and-setprintoptions-in-pytorch-55g8) explains [set_default_dtype()](https://pytorch.org/docs/stable/generated/torch.set_default_dtype.html), [set_default_device()](https://pytorch.org/docs/stable/generated/torch.set_default_device.html) and [set_printoptions()](https://pytorch.org/docs/stable/generated/torch.set_printoptions.html).
- [My post](https://dev.to/hyperkai/manualseed-initialseed-and-seed-in-pytorch-5gm8) explains [manual_seed()](https://pytorch.org/docs/stable/generated/torch.manual_seed.html), [initial_seed()](https://pytorch.org/docs/stable/generated/torch.initial_seed.html) and [seed()](https://pytorch.org/docs/stable/generated/torch.seed.html).

[to()](https://pytorch.org/docs/stable/generated/torch.Tensor.to.html) can do device conversion as shown below:

*Memos:
- `to()` can be used with a tensor but not with [torch](https://pytorch.org/docs/stable/torch.html).
- The 1st argument with a tensor is `device`(Optional-Type:`str`, `int` or [device()](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device)).
- If `device` is not given, the device is not converted.
- `cpu`, `cuda`, `ipu`, `xpu`, `mkldnn`, `opengl`, `opencl`, `ideep`, `hip`, `ve`, `fpga`, `ort`, `xla`, `lazy`, `vulkan`, `mps`, `meta`, `hpu`, `mtia` or `privateuseone` can be set to `device`.
- Setting `0` to `device` uses `cuda`(GPU). *The number must be zero or positive.
- [My post](https://dev.to/hyperkai/istensor-numel-and-device-in-pytorch-2eha) explains [device()](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device).
- A copied tensor can be created.

```python
import torch

cpu_tensor = torch.tensor([0, 1, 2])

cpu_tensor.device
# device(type='cpu')

cpu_tensor.to().device
# device(type='cpu')

gpu_tensor = cpu_tensor.to(device='cuda:0')
gpu_tensor = cpu_tensor.to(device='cuda')
gpu_tensor = cpu_tensor.to(device=0)
gpu_tensor = cpu_tensor.to(device=torch.device(device='cuda:0'))
gpu_tensor = cpu_tensor.to(device=torch.device(device='cuda'))
gpu_tensor = cpu_tensor.to(device=torch.device(device=0))
gpu_tensor = cpu_tensor.to(device=torch.device(type='cuda', index=0))
gpu_tensor = cpu_tensor.to(device=torch.device(type='cuda'))

gpu_tensor.device
# device(type='cuda', index=0)

gpu_tensor.to().device
# device(type='cuda', index=0)

cpu_tensor = gpu_tensor.to(device='cpu')

cpu_tensor.device
# device(type='cpu')
```

[cuda()](https://pytorch.org/docs/stable/generated/torch.Tensor.cuda.html) and [cpu()](https://pytorch.org/docs/stable/generated/torch.Tensor.cpu.html) can change the device of a tensor to GPU(CUDA) and CPU respectively as shwon below:

*Memos:
- `cuda()` or `cpu()` can be used with a tensor but not with `torch`.
- For `cuda()`, - the 1st argument with a tensor is `device`(Optional-Type:`str`, `int` or [device()](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device)). 
  *Memos:
  - The default is the current GPU(CUDA).
  - `cpu`, `cuda`, `ipu`, `xpu`, `mkldnn`, `opengl`, `opencl`, `ideep`, `hip`, `ve`, `fpga`, `ort`, `xla`, `lazy`, `vulkan`, `mps`, `meta`, `hpu`, `mtia` or `privateuseone` can be set to `device`.
- Setting `0` to `device` uses `cuda`(GPU). *The number must be zero or positive.
- [My post](https://dev.to/hyperkai/istensor-numel-and-device-in-pytorch-2eha) explains [device()](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device).
- A copied tensor is created.

```python
import torch

cpu_tensor = torch.tensor([0, 1, 2])

cpu_tensor.device
# device(type='cpu')

gpu_tensor = cpu_tensor.cuda()
gpu_tensor = cpu_tensor.cuda(device='cuda:0')
gpu_tensor = cpu_tensor.cuda(device='cuda')
gpu_tensor = cpu_tensor.cuda(device=0)
gpu_tensor = cpu_tensor.cuda(device=torch.device(device='cuda:0'))
gpu_tensor = cpu_tensor.cuda(device=torch.device(device='cuda'))
gpu_tensor = cpu_tensor.cuda(device=torch.device(device=0))
gpu_tensor = cpu_tensor.cuda(device=torch.device(type='cuda', index=0))
gpu_tensor = cpu_tensor.cuda(device=torch.device(type='cuda'))

gpu_tensor.device
# device(type='cuda', index=0)

cpu_tensor = gpu_tensor.cpu()

cpu_tensor.device
# device(type='cpu')
```

[from_numpy()](https://pytorch.org/docs/stable/generated/torch.from_numpy.html) can convert a NumPy array to a PyTorch tensor as shown below:

*Memos:
- `from_numpy()` can be used with `torch` but not with a tensor.
- The 1st argument with `torch`(Required-Type:`ndarray`). *There is no keyword argument.
- The type of a NumPy array is also inherited to a PyTorch tensor. 

```python
import torch

my_array = np.array([0., 1., 2.])

my_array.dtype
# dtype('float64')

my_tensor = torch.from_numpy(my_array)

my_tensor
# tensor([0., 1., 2.], dtype=torch.float64)
```

[numpy()](https://pytorch.org/docs/stable/generated/torch.Tensor.numpy.html) can convert a PyTorch tensor to a NumPy array as shown below:

*Memos:
- `numpy()` can be used with a tensor but not with `torch`.
- There is `force` argument with a tensor(Optional-Default:`False`-Type:`bool`).
  *Memos:
  - If it's `True`, a GPU(CUDA) PyTorch tensor can be converted to a NumPy array which may be a copy.
  - `force=` must be used.
- The type of a PyTorch tensor is also inherited to a NumPy array.

```python
import torch

my_tensor = torch.tensor([0., 1., 2.])

my_tensor.dtype
# torch.float32

my_array = my_tensor.numpy()

my_array
# array([0., 1., 2.], dtype=float32)

my_tensor = torch.tensor([0., 1., 2.], device='cuda:0')

my_tensor.numpy(force=True)
# array([0., 1., 2.], dtype=float32)

my_tensor = torch.tensor([0., 1., 2.], device='cuda:0')

my_tensor.numpy()
my_tensor.numpy(force=False)
# Error
```