PyTorch :: Understanding Tensors (Part 1)
Tags: deeplearning, python, pytorch, neuralnetworks
Published: 2020-12-23 01:46:39

In this post, you will start from the very basics of deep learning frameworks. **TENSORS**

So basically a neural network is the bunch of **nodes that take one or more input, processes them and returns one or more output**.

**Table of Content**
1. [What are Tensors](#what-are-tensors)
2. [Creating Tensors](#creating-tensors)
3. [Tensor Types and Their Conversions](#tensor-types-and-their-conversions)
4. [Shapes and Dimensions](#shapes-and-dimensions)
   + [Reshaping Tensors](#reshaping-tensors)

## What are Tensors

Tensors are the building blocks of a neural network. A PyTorch tensor is the **generalized form of arrays in  {% katex inline %} n {% endkatex %} dimensions** to run arbitrary computations on GPU. <small>[[READ MORE]](https://pytorch.org/tutorials/beginner/examples_tensor/two_layer_net_tensor.html#pytorch-tensors)</small> 

A tensor can be of any dimension. In this course you will learn about the following dimensions only
+ 0D -> Scalar numbers like `0`, `10`, `20` etc. are 0-Dimensional tensors
+ 1D -> Array of numbers like `[1, 2, 3]` or `[20, 30, 10]` etc.
+ 2D -> Matrix of numbers like `[[1, 2, 3], [2, 3, 4]]`

**Note** During the process of learning, the network applies some tensor methods on these tensor values in order to gain the desired output.

Before creating and using tensors in PyTorch, install it by following the instructions from here: https://pytorch.org/get-started/locally/

## Creating Tensors

You can create the tensors by importing `torch` and using [`torch.tensor(data)`](https://pytorch.org/docs/stable/generated/torch.tensor.html#torch.tensor) method.

```python
import torch

t0 = torch.tensor(10)                     # 0-D tensor
t1 = torch.tensor([1, 2, 3, 4])           # 1-D tensor
t2 = torch.tensor([[1, 2, 3], [2, 3, 4]]) # 2-D tensor

print(t0)
print(t1)
print(t2)
```
```
tensor(10)
tensor([1, 2, 3, 4])
tensor([[1, 2, 3],
        [2, 3, 4]])
```

To convert it into numpy array, simply use [`Tensor.numpy()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.numpy) method and use it as you wish

```python
print(t2.numpy())
```
```
array([[1, 2, 3],
       [2, 3, 4]])
```

## Tensor Types and Their Conversions

PyTorch supports the following data-types of the tensors

![](https://i.ibb.co/X83WyQp/image.png")

To get the data type  use [`Tensor.dtype`](https://pytorch.org/docs/stable/tensor_attributes.html#torch-dtype) property and [`Tensor.type()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.type) to get the PyTorch tensor type

```python
print(t2.dtype)
print(t2.type())
```
```
torch.int64
torch.LongTensor
```

**Note** Difference between `.dtype` and `.type()` is better explained [here](https://discuss.pytorch.org/t/difference-between-torch-dtype-and-torch-tensortype/58948/2)

To explicitly create a tensor of specific type, either pass `dtype` while defining the tensor in [`torch.tensor`](https://pytorch.org/docs/stable/generated/torch.tensor.html#torch.tensor) method or use tensor constructor. 

For example, here I am creating a float tensor by providing _int32_ data

```python
tf0 = torch.tensor([1, 2, 3, 4], dtype=torch.float32)
print(tf0)
print(tf0.dtype)

tf1 = torch.FloatTensor([5, 6, 7, 8])
print(tf1)
print(tf1.dtype)
```
```
tensor([1., 2., 3., 4.])
torch.float32
tensor([5., 6., 7., 8.])
torch.float32
```

You can pass-in the the datatype in the [`Tensor.type([type])`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.type) method, in order to change the data-type of the existing tensor.

For example, changing float tensor to int32

```python
tf0 = tf0.type(torch.int32) # after changing, it returns a new tensor object of that type
print(tf0.dtype)
```
```
torch.int32
```

## Shapes and Dimensions

While training the model, you will deal with higher dimensions and neural network only accept the dimension which is defined at input layer while architecting the model.

The dimension basically tells whether the tensor is 0-D or 1-D or 2-D or even higher than that. **Dimension of tensor is also called the rank of the tensor**.

To get the size you can use [`Tensor.size()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.size) method or `Tensor.shape` property and to get the dimension of the tensor, use [`Tensor.ndimension()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.size) method or `Tensor.ndim` property.

```python
print(t1.size())
print(t1.ndim)

print(t2.shape)
print(t2.ndimension())
```
```
torch.Size([4])
1
torch.Size([2, 3])
2
```

### Reshaping Tensors

Sometimes working with neural network, you will face a situation when you are supposed to change the dimension of the tensor. This can be done by using [`Tensor.view(nrows, ncols)`](https://pytorch.org/docs/stable/tensor_view.html) method of the tensor

For example, I have a tensor `[[1, 2, 3], [2, 3, 4]]` and I want to convert it to 2D with 1 row and 6 columns 

```python
tx = t2.view(1, 6)
print(tx)
```
```
tensor([[1, 2, 3, 2, 3, 4]])
```

**Note** The number of parameters in `.view` method is the total number of dimensions

```python
# Converting 2D to 1D tensor
tx = t2.view(6)
print(tx)
```
```
tensor([1, 2, 3, 2, 3, 4])
```

There are special values for the view method. Suppose you don't know how many rows would be good while resizing your matrix, you can set it to -1 it will automatically adjust the rows for you. Same goes with columns.

```python
# Telling PyTorch to keep the max 2 elements in one column and make as many rows as you want
tr = t2.view(-1, 2)
print(tr)

# Telling PyTorch to keep max 2 rows and adjust columns automatically
tc = t2.view(2, -1)
tc
```
```
tensor([[1, 2],
        [3, 2],
        [3, 4]])
tensor([[1, 2, 3],
        [2, 3, 4]])
```
**Note**  Both the rows and columns can't be set to `-1` at same time.

I hope you have liked this post. Please share it with your friends and colleagues and help them learn the concepts of PyTorch. If you have doubts or any idea reach me out via following sources
+ Email: tbhaxor@gmail.com (Recommended)
+ Twitter: [@tbhaxor](https://twitter.com/tbhaxor)
+ GitHub: [@tbhaxor](https://github.com/tbhaxor)
+ LinkedIn: [@gurkirat--singh](https://www.linkedin.com/in/gurkirat--singh)
+ Instagram: [@_tbhaxor_](https://instagram.com/_tbhaxor_)