empty_strided in PyTorch
Tags: pytorch, emptystrided, empty, stride
Published: 2024-07-17 12:05:45

[Buy Me a Coffee](ko-fi.com/superkai)☕

*[My post](https://dev.to/hyperkai/empty-and-emptylike-in-pytorch-1l1k) explains [empty()](https://pytorch.org/docs/stable/generated/torch.empty.html) and [empty_like()](https://pytorch.org/docs/stable/generated/torch.empty_like.html).

[empty_strided()](https://pytorch.org/docs/stable/generated/torch.empty_strided.html) can create the 1D or more D strided tensor with the zero or more floating-point numbers(Default), integers, complex numbers or boolean values from memory which are called **uninitialized data** as shown below:

*Memos:
- `empty_strided()` can be used with [torch](https://pytorch.org/docs/stable/torch.html) but not with a tensor.
- The 1st argument with `torch` is `size`(Required-Type:`tuple` of `int`, `list` of `int`, or [size()](https://pytorch.org/docs/stable/generated/torch.Tensor.size.html)).
- The 2nd argument with `torch` is `stride`(Required-Type:`tuple` of `int` or `list` of `int`).
- There is `dtype` argument with `torch`(Optional-Type:[dtype](https://pytorch.org/docs/stable/tensor_attributes.html#torch.dtype)):
  *Memos:
  - If `dtype` is not given, [get_default_dtype()](https://pytorch.org/docs/stable/generated/torch.get_default_dtype.html) is used. *[My post](https://dev.to/hyperkai/setdefaultdtype-setdefaultdevice-and-setprintoptions-in-pytorch-55g8) explains `get_default_dtype()` and [set_default_dtype()](https://pytorch.org/docs/stable/generated/torch.set_default_tensor_type.html).
  - `dtype=` must be used.
  - [My post](https://dev.to/hyperkai/set-dtype-with-dtype-argument-functions-and-get-it-in-pytorch-13h2) explains `dtype` argument.
- There is `device` argument with `torch`(Optional-Type:`str`, `int` or [device()](https://pytorch.org/docs/stable/tensor_attributes.html#torch.device)):
  *Memos:
  - If `device` is not given, [get_default_device()](https://pytorch.org/docs/stable/generated/torch.get_default_device.html) is used. *[My post](https://dev.to/hyperkai/setdefaultdtype-setdefaultdevice-and-setprintoptions-in-pytorch-55g8) explains `get_default_device()` and [set_default_device()](https://pytorch.org/docs/stable/generated/torch.set_default_device.html).
  - `device=` must be used.
  - [My post](https://dev.to/hyperkai/set-device-with-device-argument-functions-and-get-it-in-pytorch-1o2p) explains `device` argument.
- There is `requires_grad` argument with `torch`(Optional-Type:`bool`):
  *Memos:
  - `requires_grad=` must be used.
  - [My post](https://dev.to/hyperkai/set-requiresgrad-with-requiresgrad-argument-functions-and-get-it-in-pytorch-39c3) explains `requires_grad` argument.
- The number of `size` and `stride` must be the same.

```python
import torch

torch.empty_strided(size=(), stride=())
torch.empty_strided(size=torch.tensor(8).size(), stride=())
# tensor(1.2770e+19)

torch.empty_strided(size=(0,), stride=(0,))
torch.empty_strided(size=(0,), stride=(1,))
torch.empty_strided(size=(0,), stride=(2,))
torch.empty_strided(size=torch.tensor([]).size(), stride=(0,))
# tensor([])

torch.empty_strided(size=(3,), stride=(0,))
torch.empty_strided(size=torch.tensor([8, 3, 6]).size(), stride=(0,))
# tensor([7.4511e-33, 7.4511e-33, 7.4511e-33])

torch.empty_strided(size=(3,), stride=(1,))
torch.empty_strided(size=torch.tensor([8, 3, 6]).size(), stride=(1,))
# tensor([9.7245e-33, 3.1678e-41, 9.6997e-33])

torch.empty_strided(size=(3,), stride=(2,))
torch.empty_strided(size=torch.tensor([8, 3, 6]).size(), stride=(2,))
# tensor([-5.0667e-38,  4.4842e-44,  8.9683e-44])

torch.empty_strided((3, 2), stride=(0, 0))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(0, 0))
# tensor([[-8.4397e-35, -8.4397e-35],
#         [-8.4397e-35, -8.4397e-35],
#         [-8.4397e-35, -8.4397e-35]])

torch.empty_strided((3, 2), stride=(0, 1))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(0, 1))
# tensor([[9.7001e-33, 3.1678e-41],
#         [9.7001e-33, 3.1678e-41],
#         [9.7001e-33, 3.1678e-41]])

torch.empty_strided((3, 2), stride=(0, 2))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(0, 2))
# tensor([[1.4013e-45, -1.7014e+38],
#         [1.4013e-45, -1.7014e+38],
#         [1.4013e-45, -1.7014e+38]])

torch.empty_strided((3, 2), stride=(1, 0))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(1, 0))
# tensor([[-8.4397e-35, -8.4397e-35],
#         [4.5188e-41, 4.5188e-41],
#         [9.7611e-33, 9.7611e-33]])

torch.empty_strided((3, 2), stride=(1, 1))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(1, 1))
# tensor([[-8.4397e-35, 4.5188e-41],
#         [4.5188e-41, 9.7396e-33],
#         [9.7396e-33, 3.1678e-41]])

torch.empty_strided((3, 2), stride=(1, 2))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(1, 2))
# tensor([[1.7340e-07, 6.8988e-07],
#         [1.6599e-07, 1.2539e+16],
#         [6.8988e-07, 2.1707e-18]])

torch.empty_strided((3, 2), stride=(2, 0))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(2, 0))
# tensor([[-8.4397e-35, -8.4397e-35],
#         [9.7265e-33, 9.7265e-33],
#         [6.6757e-07, 6.6757e-07]])

torch.empty_strided((3, 2), stride=(2, 1))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(2, 1))
# tensor([[-8.4397e-35, 4.5188e-41],
#         [9.6884e-33, 3.1678e-41],
#         [4.4842e-44, 0.0000e+00]])

torch.empty_strided((3, 2), stride=(2, 2))
torch.empty_strided(size=torch.tensor([[8, 3], [6, 0], [2, 9]]).size(),
                    stride=(2, 2))
# tensor([[6.7121e-07, 1.3085e-11],
#         [1.3085e-11, 1.6690e+22],
#         [1.6690e+22, 2.1707e-18]])
etc.

torch.empty_strided(size=(3, 2, 4), stride=(0, 1, 2))
# tensor([[[-8.4397e-35, -8.4397e-35, 4.4842e-44, 1.1210e-43],
#          [4.5188e-41, 4.5188e-41, 0.0000e+00, 0.0000e+00]],
#         [[-8.4397e-35, -8.4397e-35, 4.4842e-44, 1.1210e-43],
#          [4.5188e-41, 4.5188e-41, 0.0000e+00, 0.0000e+00]],
#         [[-8.4397e-35, -8.4397e-35, 4.4842e-44, 1.1210e-43],
#          [4.5188e-41, 4.5188e-41, 0.0000e+00, 0.0000e+00]]])

torch.empty_strided(size=(3, 2, 4), stride=(0, 1, 2), dtype=torch.int64)
# tensor([[[0, 97092179969056, 0, 0],
#          [97092200351808, 138498049810784, 0, 0]],
#         [[0, 97092179969056, 0, 0],
#          [97092200351808, 138498049810784, 0, 0]],
#         [[0, 97092179969056, 0, 0],
#          [97092200351808, 138498049810784, 0, 0]]])

torch.empty_strided(size=(3, 2, 4), stride=(0, 1, 2), dtype=torch.complex64)
# tensor([[[-8.4397e-35+4.5188e-41j, 9.6886e-33+3.1678e-41j,
#           0.0000e+00+0.0000e+00j, 1.3829e-33+3.1678e-41j],
#          [9.6840e-33+3.1678e-41j, 0.0000e+00+0.0000e+00j,
#           9.8336e-33+3.1678e-41j, -3.8910e-25+4.5186e-41j]],
#         [[-8.4397e-35+4.5188e-41j, 9.6886e-33+3.1678e-41j,
#           0.0000e+00+0.0000e+00j, 1.3829e-33+3.1678e-41j],
#          [9.6840e-33+3.1678e-41j, 0.0000e+00+0.0000e+00j,
#           9.8336e-33+3.1678e-41j, -3.8910e-25+4.5186e-41j]],
#         [[-8.4397e-35+4.5188e-41j, 9.6886e-33+3.1678e-41j,
#           0.0000e+00+0.0000e+00j, 1.3829e-33+3.1678e-41j],
#          [9.6840e-33+3.1678e-41j, 0.0000e+00+0.0000e+00j,
#           9.8336e-33+3.1678e-41j, -3.8910e-25+4.5186e-41j]]])

torch.empty_strided(size=(3, 2, 4), stride=(0, 1, 2), dtype=torch.bool)
# tensor([[[True, True, True, False],
#          [True, True, True, False]],
#         [[True, True, True, False],
#          [True, True, True, False]],
#         [[True, True, True, False],
#          [True, True, True, False]]])
```