Tensors in Pytorch

04 Jul, 2021
A Pytorch Tensor is basically the same as a NumPy array. This means it does not know anything about deep learning or computational graphs or gradients and is just a generic n-dimensional array to be used for arbitrary numeric computation. However, the biggest difference between a NumPy array and a PyTorch Tensor is that a PyTorch Tensor can run on either CPU or GPU. To run operations on the GPU, just cast the Tensor to a cuda datatype using:
device = torch.device(“cpu”)
# to create random input and output data ,
# and H is hidden dimension; D_out is output dimension.
N, D_in, H, D_out = 32, 100, 10, 2 
x = torch.randn(N, D_in, device=device, dtype=torch.float)    #where x is a tensor 
In the above example, x can be thought of as a random feature tensor as an input to a model. We will see how to create tensors, different attributes, and operations on a tensor in this article.
You can create a tensor using some simple lines of code as shown below.
Python3
import torchV_data = [1, 2, 3, 4, 5]V = torch.tensor(V_data)print(V)
Output: 
tensor([1, 2, 3, 4, 5])
You can also create a tensor of random data with a given dimensionality like:
Python3
import torch  x = torch.randn((3, 4, 5))print(x)
Output :
tensor([[[ 0.8332, -0.2102,  0.0213,  0.4375, -0.9506],
         [ 0.0877, -1.5845, -0.1520,  0.3944, -0.7282],
         [-0.6923,  0.0332, -0.4628, -0.9127, -1.4349],
         [-0.3641, -0.5880, -0.5963, -1.4126,  0.5308]],

        [[ 0.4492, -1.2030,  2.5985,  0.8966,  0.4876],
         [ 0.5083,  1.4515,  0.6496,  0.3407,  0.0093],
         [ 0.1237,  0.3783, -0.7969,  1.4019,  0.0633],
         [ 0.4399,  0.3827,  1.2231, -0.0674, -1.0158]],

        [[-0.2490, -0.5475,  0.6201, -2.2092,  0.8405],
         [ 0.1684, -1.0118,  0.7414, -3.3518, -0.3209],
         [ 0.6543,  0.1956, -0.2954,  0.1055,  1.6523],
         [-0.9872, -2.0118, -1.6609,  1.4072,  0.0632]]])
You can also create tensors using the following functions:
torch.zeros(): Creates a new Tensor with all elements, initialized as zeros.
Python3
import torch  z= torch.zeros([3,3], dtype=torch.int32)print(z)
Output:  
tensor([[0, 0, 0],
        [0, 0, 0],
        [0, 0, 0]], dtype=torch.int32)
torch.ones():  Creates a new Tensor with all elements, initialized as ones.
Python3
import torch  z = torch.ones([3,3])print(z)
Output:
tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]])
torch.full() and torch.full_like():  These functions return a Tensor of the required size filled with required fill_value provided. The complete prototype for torch.full() is:
Syntax: torch.full(size, fill_value, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
And torch.full_like() is:
Syntax: torch.full_like(input, fill_value, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False, memory_format=torch.preserve_format)
Python3
import torch  # example of torch.full()newTensor= torch.full((4, 3), 3.14,dtype= torch.float32)print(newTensor)
Output:
tensor([[3.1400, 3.1400, 3.1400],
        [3.1400, 3.1400, 3.1400],
        [3.1400, 3.1400, 3.1400],
        [3.1400, 3.1400, 3.1400]])
Python3
import torch  # Example for torch.full_like()x = torch.full_like(newTensor,3.24, dtype=None )print(x)
Output: 
tensor([[3.2400, 3.2400, 3.2400],
        [3.2400, 3.2400, 3.2400],
        [3.2400, 3.2400, 3.2400],
        [3.2400, 3.2400, 3.2400]])
Here a new Tensor is returned, with the same size and dtype as the newTensor which was earlier returned from the torch.full method in the example shown above.
Each tensor( torch.Tensor ) has a torch.dtype, torch.device, and torch.layout attributes. 
torch.dtype: A torch.dtype is an object that represents the data type of torch.Tensor. PyTorch has twelve different data types.
torch.device: A torch.device is an object representing the device on which a torch.Tensor is or will be allocated. The torch.device contains a device type (‘cpu’ or ‘cuda’) and an optional device ordinal for the device type.
Example:
Python3
torch.device('cuda:0')
Output : 
device(type='cuda', index=0)
If the device ordinal is not present, this object will always represent the current device for the device type, even after torch.cuda.set_device() is called.
torch.layout: A torch.layout is an object that represents the memory layout of a torch.Tensor. Currently, the torch supports two types of memory layout.
1. torch.strided: Represents dense Tensors and is the memory layout that is most commonly used. Each stridden tensor has an associated torch.Storage, which holds its data. These tensors provide a multi-dimensional, stridden view of storage. The stride of an array (also referred to as increment, pitch or step size) is the number of locations in memory between the beginnings of successive array elements, measured in bytes or in units of the size of the array’s elements. The stride cannot be smaller than the element size but can be larger, indicating extra space between elements. So basically here Strides are a list of integers: the k-th stride represents the jump in the memory necessary to go from one element to the next one in the kth dimension of the Tensor. This concept makes it possible to perform many tensor operations efficiently.
Let’s run some example snippets:
Python3
x = torch.Tensor([[1, 2, 3, 4], [5, 7, 8, 9]])x.stride()
Output:
(4,1)
2. torch.sparse_coo_tensor: Used to store array in the sparse coordinate list. In COO format, the specified elements are stored as tuples of element indices and the corresponding values. 
Python3
i = [[0, 1, 1],     [2, 0, 2]]  v =  [3, 4, 5]s = torch.sparse_coo_tensor(i, v, (2, 3))Print(s)
Output: 
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3, 4, 5]),
       size=(2, 3), nnz=3, layout=torch.sparse_coo)
You can add two tensors like matrix addition.
Python3
x = torch.tensor([1., 2., 3.])y = torch.tensor([4., 5., 6.])z = x + yprint(z)
Output: 
tensor([5., 7., 9.])
torch.cat() : Concatenates a list of tensors
Python3
x_1 = torch.randn(2, 5)y_1 = torch.randn(3, 5)z_1 = torch.cat([x_1, y_1])print(z_1)
Output: 
tensor([[ 0.5761,  0.6781,  0.1621,  0.4986,  0.3410],
        [-0.8428,  0.2510, -0.2668, -1.1475,  0.5675],
        [-0.2797, -0.0699,  2.8936,  1.8260,  2.1227],
        [ 1.3765, -0.0939, -0.3774, -0.3834,  0.0682],
        [ 2.3666,  0.0904,  0.7956,  1.2281,  0.5561]])
To concatenate columns you can do the following.
Python3
x_2 = torch.randn(2, 3)y_2 = torch.randn(2, 5)  # second argument specifies which axis to concat alongz_2 = torch.cat([x_2, y_2], 1)print(z_2)
Output:
tensor([[ 0.5818,  0.7047,  0.1581,  1.8658,  0.5953, -0.9453, -0.6395, -0.7106],
        [ 1.2197,  0.8110, -1.6072,  0.1463,  0.4895, -0.8226, -0.1889,  0.2668]])
view(): You can reshape tensors using the .view() method as shown below.
Python3
x = torch.randn(2, 3, 4)print(x)  # reshape to 2 rows, 12 columnsprint(x.view(2, 12))
Output:
tensor([[[ 0.4321,  0.2414, -0.4776,  1.6408],
         [ 0.9085,  0.9195,  0.1321,  1.1891],
         [-0.9267, -0.1384,  0.0115, -0.4731]],

        [[ 0.7256,  0.6990, -1.7374,  0.6053],
         [ 0.0224, -1.2108,  0.1974,  0.0655],
         [-0.6182, -0.0797,  0.2603, -1.3280]]])
tensor([[ 0.4321,  0.2414, -0.4776,  1.6408,  0.9085,  0.9195,  0.1321,  1.1891,
         -0.9267, -0.1384,  0.0115, -0.4731],
        [ 0.7256,  0.6990, -1.7374,  0.6053,  0.0224, -1.2108,  0.1974,  0.0655,
         -0.6182, -0.0797,  0.2603, -1.3280]])
torch.argmax(): Returns the index of the maximum value of all elements in the input tensor.
Python3
x = torch.randn(3,3)print((x, torch.argmax(x)))
Output: 
(tensor([[ 1.9610, -0.7683, -2.6080],
        [-0.3659, -0.1731,  0.1061],
        [ 0.8582,  0.6420, -0.2380]]), tensor(0))
torch.argmin(): Similar to argmax(), it returns the minimum value of all elements in the input tensor.
Python3
x = torch.randn(3,3)print((x, torch.argmin(x)))
Output: 
(tensor([[ 0.9838, -1.2761,  0.2257],
        [-0.4754,  1.2677,  1.1973],
        [-1.2298, -0.5710, -1.3635]]), tensor(8))
Picked
Python-PyTorch
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
Python Classes and Objects
Python OOPs Concepts
Introduction To PYTHON
How to drop one or multiple columns in Pandas Dataframe
Python | os.path.join() method
Check if element exists in list in Python
How To Convert Python Dictionary To JSON?
Python | Get unique values from a list
Python | datetime.timedelta() function