What is PyTorch?. Think about Numpy, but with strong GPU... | by Khuyen Tran | Towards Data Science

PyTorch is a library for Python programs that facilitates building deep learning projects. We like Python because is easy to read and understand. PyTorch emphasizes flexibility and allows deep learning models to be expressed in idiomatic Python.
In a simple sentence, think about Numpy, but with strong GPU acceleration. Better yet, PyTorch supports dynamic computation graphs that allow you to change how the network behaves on the fly, unlike static graphs that are used in frameworks such as Tensorflow.
Why PyTorch?
- NumPy-like arrays on GPU’s
- Dynamic computational graphs
- It’s Pythonic!
PyTorch can be installed and used on macOS. It is recommended, but not required to have NVIDIA GPU in order to harness the full power of PyTorch’s CUDA support.
Installation with Anaconda
conda install pytorch torchvision -c pytorch
Installation with pip
pip3 install torch torchvision
If you have any problem with installation, find out more about different ways to install PyTorch here.
If you don’t have NVIDIA GPU in your computer, use Google Colab instead to harness its free GPU power! Click on New notebook in the top left to get started.
Remember to change runtime type to GPU before running the notebook
Are you familiar with Numpy? Great! You just need to shift the syntax using on Numpy to syntax of PyTorch. If you are not familiar with Numpy, PyTorch is written in such an intuitive way that you can learn in second.
Import the two libraries to compare their results and performance
import torchimport numpy
PyTorch tensors are similar to NumPy ndarrays with the option to operate on GPU.
>>> numpy.array([[0.1, 1.2], [2.2, 3.1], [4.9, 5.2]])array([[0.1, 1.2],       [2.2, 3.1],       [4.9, 5.2]])>>> torch.tensor([[0.1, 1.2], [2.2, 3.1], [4.9, 5.2]])tensor([[0.1000, 1.2000],        [2.2000, 3.1000],        [4.9000, 5.2000]])
What do we see here? tensor is shown in place of array. Remember that np.empty(), np.random.randn(), np.zeros(), np.ones() that we love? The same functions and syntax can be applied with PyTorch
w = torch.empty(3, 3)print(w,'\n', w.shape, '\n')x = torch.randn(3, 3, 7)print(x,'\n', x.shape, '\n')y = torch.zeros(3, 3)print(y,'\n', y.shape, '\n')z = torch.ones(3, 3)print(z,'\n', z.shape, '\n')[0., 0., 3., 0., 0.],                 [4., 0., 5., 0., 0.],                 [0., 0., 0., 0., 0.]
Change the shape with view() method
>>> x = torch.rand(100,50)>>> print(x.shape)torch.Size([100, 50])>>> y=x.view(20,5,50)>>> print(y.shape)torch.Size([20, 5, 50])>>> z=x.view(-1,5,50)>>> print(z.shape)torch.Size([20, 5, 50])
GPU (graphics processing units) composes of hundreds of simpler cores, which makes training deep learning models much faster. Below is the quick comparison between GPU and CPU.
Whether we decide to use GPU or CPU, PyTorch makes it easy for us to switch between the two
cpu=torch.device("cpu")gpu=torch.device("cuda:0") # GPU 0# Create tensor with CPUx=torch.ones(3,3, device=cpu)print("CPU:",x.device)x=torch.ones(3,3, device=gpu)print("GPU:",x.device)x=torch.ones(3,3).cuda(0)print("CPU to GPU:",x.device)x=torch.ones(3,3, device=gpu).cpu()print("GPU to CPU:",x.device)
Compare time between CPU and GPU
Time in CPU
>>> import time>>> x=torch.rand(10000,10000)>>> y=torch.rand(10000,10000)>>> t = time.time()>>> z=x@y>>> t = time.time()-t>>> print(t)6.999474763870239
Time in GPU
>>> yc=y.cuda(0)>>> t = time.time()>>> z=xc@yc>>> t = time.time()-t>>> print(t)0.4787747859954834
It is nearly 15 times faster than Numpy for simple matrix multiplication!
Since NumPy and PyTorch are really similar, is there a method to change NumPy array to PyTorch array and vice versa? Yes!
a = np.ones(5)#From NumPy to Torchb = torch.from_numpy(a)print('a:',a)print('b:',b)
What is Autograd? Remember in your calculus class when you need to calculate the derivatives of a function? The gradient is like derivative but in vector form. It is important to calculate the loss function in neural networks. But it impractical to calculate gradients of such large composite functions by solving mathematical equations because of the high number of dimensions. Luckily, PyTorch can find this gradient numerically in a matter of seconds!
Let’s say we want to find the gradient of the vector below. We expect the gradient of y to be x. Use tensor to find the gradient and check whether we get the right answer.
Awesome! The gradient is x like what we expected. Break down of the codes above:
requires_grad = True allows for efficient gradient computation. If we know we will compute the gradient based on x, we need to set .requires_grad = True for the input. Learn more about how autograd works here.
y.backward() computes the gradient numerically
x.grad() returns the gradient of y at the point x
Let’s understand how to use PyTorch with a classification example. Our task is to find if a point is in the cluster of yellow or purple
Start with constructing a subclass of nn.Module, a PyTorch module for building a neural network.
Split the data into training and test set
Train the data
Predict and evaluation the prediction
Congratulations! You have just learned what is PyTorch and how to use it. This article is just a gentle introduction to PyTorch. What I hope you gain from this article is that deep learning can be easy and efficient with this tool. To understand more about how to use PyTorch for your deep learning project, I recommend checking this great book: Deep Learning with PyTorch, one of the best resources to learn about PyTorch.
Feel free to fork and play with the code for this article in this Github repo.
I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.
Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these: