PyTorch :: Understanding Tensors (Part 2)
Tags: deeplearning, python, pytorch, neuralnetworks
Published: 2021-01-01 15:46:19

Hello friends, Happy new year

So far you have understood the types and shapes of the tensors. In this I will go more beyond the basics and teach you some more concepts about the tensors.

I have created this as Part-2 as all the information in one post was lengthy :P

**Table of Contents**

1. [Indexing and Slicing of Tensors](#indexing-and-slicing-of-tensors)
2. [Arithmetic Operations on Tensors](#arithmetic-operations-on-tensors)
3. [Basic Functions on the Tensors](#basic-functions-on-the -tensors)
4. [Operations on 2D Tensors](#operations-on-2d-tensors)


## Indexing and Slicing of Tensors

The individual values of the tensors are also a tensor, for instance

```py
import torch

t1 = torch.tensor([1, 2, 3, 4])
print(t1[0])
```

```
tensor(1)
```

Since the original tensor was of 1-D, the value would be of course a 0-D tensor. So to if you have 2-D tensor, and  are doing using only one index, you will get 1-D tensor
```py
import torch

t2 = torch.tensor([[1, 2, 3, 4], [5, 6, 7, 8]])
print(t2[0])
```

```
tensor([1, 2, 3, 4])
```

> **NOTE:** If you want to get the raw python list of 1+ D tensors, you can use [`Tensor.tolist()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.tolist) method

Like normal list, you can also update the tensor value by simply accessing the index. For example
```py
t1[2] = 100
print(t1)
```

```
tensor([1, 2, 100, 4])
```

> **NOTE** For your sake, pytorch returns a new tensor object whenever you perform slicing on the original tensor. But it will overwrite the original tensor when you will change the value of tensor at particular index.

## Arithmetic Operations on Tensors

The very basic operations on tensors are [vector additions and subtractions](https://www.varsitytutors.com/hotmath/hotmath_help/topics/adding-and-subtracting-vector). Visit the link in case you want to study the maths behind these vector operations

Suppose you have two tensors {% katex inline %} u {% endkatex %} and {% katex inline %} u {% endkatex %} defined as
```py
u = torch.tensor([1.0, 2.0])
v = torch.tensor([3.0, 4.0])

print(u)
print(v)
```

```
tensor([1., 2.])
tensor([3., 4.])
```

Then the vector operations _addition_ and _subtraction_ respectively
```py
print(u + v)
print(u - v)
```
```
tensor([4., 6.])
tensor([-2., -2.])
```

Scalar operations are also supported. For example, taking {% katex inline %} 5 {% endkatex %} as our scalar quantity 
```py
ws = torch.tensor(5)
print(ws)
``` 

```
tensor(5)
```

The operations _addition_, _subtraction_, _multiplication_ and _division_ are shown respectively
```py
print(u + v + ws)
print(u + v - ws)
print(u + v * ws)
print(u + (v / ws))
```

```
tensor([ 9., 11.])
tensor([-1.,  1.])
tensor([16., 22.])
tensor([1.6000, 2.8000])
```

In pytorch, there is no need of creating a 0D tensor to perform scalar operations you can simply use the scalar value and perform the action. For example,

```py
print(v * 5)
```

```
tensor([15., 20.])
```

The cross-product is fairly short and easy, by using **`*`** symbol. But to perform dot product, you should use [`torch.dot(vector1, vector2)`](https://pytorch.org/docs/stable/generated/torch.dot.html#torch.dot) or [`Tensor.dot(vector2)`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.dot)

```py
print(u * v)
print(torch.dot(u, v))
print(u.dot(v))
```

```
tensor([3., 8.])
tensor(11.)
tensor(11.)
```

> **NOTE** While performing cross-product or dot-product, _dimensions of both the tensors should be equal_, otherwise you will get `RuntimeError`


## Basic Functions on the Tensors

Torch tensors provide a plethora of functions that you can apply on the tensors for the desired results. The first one of all is [`Tensor.mean()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.mean)

```py
x = torch.tensor([1, 2, 3, 4, 5], dtype=torch.float32)
print(x)
print(x.mean())
```

```
tensor([1., 2., 3., 4., 5.])
x.mean()
```

> **NOTE** Since mean of a tensor will be a floating tensor, you can find the mean of floating tensor only otherwise you will get [`RuntimeError`](https://stackoverflow.com/questions/64358283/runtimeerror-can-only-calculate-the-mean-of-floating-types-got-byte-instead-f)

Finding maximum or minimum value in the tensor can be done by using [`Tensor.max()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.max) and [`Tensor.min()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.min) respectively

```py
print(x.max())
print(x.min())
```

```
tensor(5.)
tensor(1.)
```

You can also apply a function to a tensor element-wise. Suppose you have tensor containing various values of pi and you want to apply the sin and cos function on it. You can use [`torch.sin(tensor)`](https://pytorch.org/docs/stable/generated/torch.sin.html#torch.sin) and [`torch.cos(tensor)`](https://pytorch.org/docs/stable/generated/torch.cos.html#torch.cos)

```
import numpy as np

x = torch.tensor([0, np.pi / 2, np.pi])
print(x)
print(torch.sin(x))
print(torch.cos(x))
```

```
tensor([0.0000, 1.5708, 3.1416])
tensor([ 0.0000e+00,  1.0000e+00, -8.7423e-08])
tensor([ 1.0000e+00, -4.3711e-08, -1.0000e+00])
```

Sometimes you will have to get an evenly spaced list of numbers between a range, you can use [`torch.linspace(start, end, [step])`](https://pytorch.org/docs/stable/generated/torch.linspace.html)

Let's make it more interactive by plotting the {% katex inline %} \sin (- \pi )  {% endkatex %} to {% katex inline %} \sin ( \pi )  {% endkatex %} and {% katex inline %} \cos (- \pi )  {% endkatex %} to {% katex inline %} \cos ( \pi )  {% endkatex %}

Making the dataset first,
```py
pi = torch.linspace(-np.pi/2, np.pi/2, steps=1000)
print(pi[:5])  # lower bound
print(pi[-5:]) # upper bound

sined = torch.sin(pi)
cosed = torch.cos(pi)
print(sined[0:5])
print(cosed[0:5])
```
```
tensor([-1.5708, -1.5677, -1.5645, -1.5614, -1.5582])
tensor([1.5582, 1.5614, 1.5645, 1.5677, 1.5708])
tensor([-1.0000, -1.0000, -1.0000, -1.0000, -0.9999])
tensor([-4.3711e-08,  3.1447e-03,  6.2894e-03,  9.4340e-03,  1.2579e-02])
```

Now using matplotlib to create the graph
```py
import matplotlib.pyplot as plt

plt.plot(sined, label="sined")
plt.plot(cosed, label="cosed")
plt.legend()
plt.show()
```
![](https://i.imgur.com/5MJbmrv.png)

## Operations on 2D Tensors

The greyscale images are the best example of 2D tensors. Each pixel which you see as {% katex inline %}width * height {% endkatex %} are the {% katex inline %}cols * rows{% endkatex %} for the matrix.

![](https://www.researchgate.net/publication/50211012/figure/fig6/AS:669089471160327@1536534769114/a-Grayscale-image-of-character-A-b-Binary-representation-of-character-A-c.ppm)

In the above images, it is demonstrated how a binary image can be represented in a matrix. In case of grayscale image, the value of each element of the matrix would be in range of {% katex inline %}0 - 255{% endkatex %}

Just in case you are wondering how a RGB coloured image is represented, it is basically a 3D tensor where there are 3 matrix for each R, G and B channel. For example

![](https://www.researchgate.net/profile/Bhupendra_Pratap_Singh/publication/282798184/figure/fig15/AS:283989639221249@1444719814399/Color-image-representation-and-RGB-matrix.png)


Creating a random 2D tensor using [`torch.rand(*size)`](https://pytorch.org/docs/stable/generated/torch.rand.html) method. 

>**NOTE:** This is a general method, you can create a random tensor of any dimension using this method

```py
t2 = torch.rand((3, 3))
print(t2)
print(t2.ndim)   # dimension of matrix is also called rank of matrix
print(t2.shape)
print(t2.numel())
```
```
tensor([[0.0376, 0.4297, 0.2987],
        [0.8009, 0.1815, 0.5538],
        [0.2482, 0.7099, 0.3132]])
2
torch.Size([3, 3])
9
```

The [`Tensor.numel()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.numel) method used above returns the total number of the elements in the tensor


The indexing and slicing works same as 1D tensor, but there is a minor change. In 1D you were used to use `[number_start:number_end]` for slicing and `[index]` for getting element at particular index. 

Here now you have rows and columns, so the syntax would be `[index_row, index_col]` to get the particular element at row and column in the matrix and `[number_start_row:number_end_row, number_start_col:number_end_row]`. 

```py
print(t2[0, 0])
print(t2[1, 1])
```
```
tensor(0.0376)
tensor(0.1815)
```

The above code will give you the very first and middle element from the matrix. As shown in the following matrix 

![](https://i.ibb.co/8YLNsrH/image.png)

Getting an element with indexing will always give you a new tensor of 0D
```py
t2[1:2, 1:]
```
```
tensor([[0.1815, 0.5538]])
```

To perform the matrix multiplication you can use either of them: [`Tensor.matmul(tensor2)`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.matmul), [`torch.matmul(tensor1, tensor2)`](https://pytorch.org/docs/stable/generated/torch.matmul.html#torch.matmul), [`Tensor.mm(tensor2)`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.mm) or [`torch.mm(tensor1, tensor2)`](https://pytorch.org/docs/stable/generated/torch.mm.html#torch.mm)

```py
x = torch.rand((3, 4))
y = torch.rand((4, 3))
print(x)
print(y)

print(x.matmul(y))
print(torch.mm(x, y))
```
```
tensor([[0.6413, 0.1338, 0.5066, 0.1618],
        [0.3807, 0.8555, 0.2187, 0.5024],
        [0.2771, 0.6381, 0.5671, 0.6934]])
tensor([[0.0263, 0.9461, 0.6314],
        [0.9180, 0.1586, 0.2589],
        [0.3363, 0.4529, 0.9433],
        [0.6760, 0.8877, 0.1171]])
tensor([[0.4195, 1.0009, 0.9363],
        [1.2086, 1.0409, 0.7270],
        [1.2525, 1.2358, 0.9563]])
tensor([[0.4195, 1.0009, 0.9363],
        [1.2086, 1.0409, 0.7270],
        [1.2525, 1.2358, 0.9563]])
```


I hope you have liked this post. Please share it with your friends and colleagues and help them learn the concepts of PyTorch. If you have doubts or any idea reach me out via following sources
+ Email: tbhaxor@gmail.com (Recommended)
+ Twitter: [@tbhaxor](https://twitter.com/tbhaxor)
+ GitHub: [@tbhaxor](https://github.com/tbhaxor)
+ LinkedIn: [@gurkirat--singh](https://www.linkedin.com/in/gurkirat--singh)
+ Instagram: [@_tbhaxor_](https://instagram.com/_tbhaxor_)