Getting started with Pytorch: 5 Interesting functions for Tensor operations
By: Rupsha Kar
Tags: Deep Learning, Tensor, Pytorch, Arrays, Data Science

1. TORCH.NORMAL

This function returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given.

Parameters:

mean(tensor/float): the tensor of per-element means/the mean for all distributions

std(Tensor/float): the tensor of per-element standard deviations/the standard deviation for all distributions

size(int..): a sequence of integers defining the shape of the output tensor.

generator (torch.Generator, optional) — a pseudorandom number generator for sampling

out (Tensor, optional) — the output tensor.

The shapes of mean and std don’t need to match, but the total number of elements in each tensor needs to be the same.

In the above example using parameter mean we gave a pre-defined tensor whose mean will decide the mean of output tensor’s normal distribution.

Using parameter std we gave a pre-defined tensor whose standard deviation will be followed by the output tensor’s normal distribution.

A normally distributed tensor got created whose mean is be same as of tensor (100.,110.) and with standard deviation as derived from a tensor (1,0,-0.1)

In the above example, we get a normally distributed tensor of size 2 rows and 4 columns, with mean 50 and standard deviation 5.

The normal function with mean=0 only works with a standard deviation that is more than 0.0 therefore it won’t take a negative standard deviation.

Summary

The normal function of the torch can be used for various statistical analyses where a normally distributed tensor is required for testing various hypotheses.

2. TORCH.FROM_NUMPY

Torch.from_numpy is the function used to convert a NumPy Array into a Tensor. When working with Machine or deep learning algorithms we mostly have data in Numpy array format that is why this function holds a lot of importance when working with real-life datasets.

It is to be noted that since the returned tensor and ndarray share the same memory any modifications to the tensor will be reflected in the ndarray and vice versa. The returned tensor is not resizable.

In the above example, we take an array of size (2,3) of int64 datatype and changed it to a tensor using torch.from_numpy function.

In the above example, we created 2 NumPy arrays for Prices of 2 products and the sales that got generated for the respective prices then converted both of them into 2 tensors.

From_numpy currently accepts ndarray with dtypes of numpy.float64, numpy.float32, numpy.float16, numpy.complex64, numpy.complex128, numpy.int64, numpy.int32, numpy.int16, numpy.int8, numpy.uint8, and numpy.bool. Therefore, it is not possible for any other datatype like string to get converted to tensor as is visible in above example.

Summary

This function is very useful when working on data science projects where NumPy arrays are a preferred choice of datatype.

3. TORCH.SPLIT

Splits the tensor into chunks where each chunk is a view of the original tensor.

Parameters:

tensor (Tensor) — tensor to split.

split_size_or_sections (int)/(list(int)) — the size of a single chunk or list of sizes for each chunk

dim (int) — dimension along which to split the tensor.

If split_size_or_sections is an integer type, then the tensor will be split into equally sized chunks (if possible) and the last chunk will be smaller if the tensor size along the given dimension dim is not divisible by split_size.

In the above example, we used split_size_or_section of value 2 that is each chunk will be of size 2 to split our original (5,2) tensor into 3 tensor chunks each of size 2 except the last one which due to the absence of an equal number of elements takes only 1 array.

If split_size_or_sections is a list, then tensor will be split into len(split_size_or_sections) chunks with sizes in dim according to split_size_or_sections.

In the above example, we used split the tensor of shape (3,4) into tensor chunks of chunk size 2 (len[1,2]) with the first chunk having size 1 and 2nd chunk with size 2.

The above example gives an error as we tried splitting the tensor of shape 3-row elements and 4 columns into 2 tensors of chunk size of 1 and 3 respectively which is not possible as we have only 3 elements.

Summary

Torch.split is a very useful function when working with functions and modules that require different functions to be performed in different sections of a tensor.

4. TORCH.ARGMAX

Returns the indices of the maximum value of all elements in the input tensor.

Parameters:

input (Tensor) — the input tensor.

dim (int) — the dimension to reduce. If None, the argmax of the flattened input is returned.

keepdim (bool) — whether the output tensor has dim retained or not. Ignored if dim=None.

In the above example, we create a random tensor of shape (3,3) and then used armax() to find the index value of the element with maximum value.

In the above example we created a 3d tensor then found the index of the maximum element along with each dimension/element present in the tensor.

In the above example, we created a random 2-dimensional tensor using randn function of the torch, and then we tried getting the index position of the maximum element present across 3 dimensions which is not possible that's why we get a “Dimension out of range error”

Summary

The argmax function is a very useful function that we can use when dealing with large tensor data and need to perform functions or require only the index position of the maximum value of the element.

5 . TORCH.MATMUL

Its a function to perform the mathematical operation of multiplying 2 tensors and thus gives an output of matrix product of 2 tensors.

Parameters:

input (Tensor) — the first tensor to be multiplied

other (Tensor) — the second tensor to be multiplied.

In the above example, we created 2 tensors of 1 dimension each having 3 elements/columns, and then calculated its product using matmul function and stored it in 1-dimensional tensor t3.

In the above example, we create 1 tensor matrix t1 of shape [(3,2)] and another tensor matrix t2 having shape [(2)], therefore the product of t1 and t2 produced the third tensor t3 which of shape [(3)].

In the above example, we are getting an error as we tried to multiply a matrix of shape [(1,3)] with a matrix of shape [(2,2)] which is not possible.

Summary

We can use the matmul function in different use cases depending on the dimensionality of the tensors as follows:

If both tensors are 1-dimensional, the dot product (scalar) is returned.

If both arguments are 2-dimensional, the matrix-matrix product is returned.

If the first argument is 1-dimensional and the second argument is 2-dimensional, a 1 is prepended to its dimension for the purpose of the matrix multiply. After the matrix multiply, the prepended dimension is removed.

If the first argument is 2-dimensional and the second argument is 1-dimensional, the matrix-vector product is returned.

If both arguments are at least 1-dimensional and at least one argument is N-dimensional (where N > 2), then a batched matrix multiply is returned. If the first argument is 1-dimensional, a 1 is prepended to its dimension for the purpose of the batched matrix multiply and removed after. If the second argument is 1-dimensional, a 1 is appended to its dimension for the purpose of the batched matrix multiple and removed after. The non-matrix (i.e. batch) dimensions are broadcasted (and thus must be broadcastable). For example, if the input is a (j×1×n×n) tensor and the other is a (k×n×n) tensor, out will be a (j×k×n×n) tensor.

Conclusion

In this notebook, we looked at 5 interesting randomly selected tensor.Torch functions that can be used to create, convert, slice, reduce, and mathematical operations. Each of the functions was illustrated with both working and error instances. For a complete list of all available functions, please see the official PyTorch document on tensors.

Link to Jupyter Notebook: https://jovian.ai/rupshakr/01-tensor-operations

Reference Links

Provide links to your references and other interesting articles about tensors