An easy introduction to Pytorch for Neural Networks | by George Seif | Towards Data Science

Want to be inspired? Come join my Super Quotes newsletter. 😎
Deep Learning has reignited the public interest in AI. The reason is simple: Deep Learning just works. It’s given us the ability to build technologies that we previously weren’t able to. It’s created new business opportunities and improved the technology world as a whole.
To do Deep Learning, you’re going to need to know how to code, especially with Python. From there, there’s a big ever-growing set of Deep Learning libraries to choose from: TensorFlow, Keras, MXNet, MatConvNet and, quite recently, Pytorch!
Very shortly after its release, Pytorch rapidly gained popularity. People were calling it the TensorFlow killer, since it was so much more user-friendly and easier to use. Indeed, you’re about to see just how easy it is to get up and running in Deep Learning with Pytorch.
At its core, the development of Pytorch was aimed at being as similar to Python’s Numpy as possible. Doing so would allow an easy and smooth interaction between regular Python code, Numpy, and Pytorch allowing for faster and easier coding.
To get started, we can install Pytorch via pip:
pip3 install torch torchvision
If you’re interested in looking at specific features, the Pytorch docs are amazing.
The most basic building block of any Deep Learning library is the tensor. Tensors are matrix-like data structures very similar in function and properties to Numpy arrays. In fact, for most purposes you can think of them exactly like Numpy arrays. The most important difference between the two is that the implementation of tensors in modern Deep Learning libraries can run on CPU or GPU (very fast).
In PyTorch, tensors can be declared using the simple Tensor object:
import torch x = torch.Tensor(3, 3)
The above code creates a tensor of size (3, 3) — i.e. 3 rows and 3 columns, filled with floating point zeros:
0.  0.  0.0.  0.  0.0.  0.  0.[torch.FloatTensor of size 3x3]
We can also create tensors filled random floating point values:
x = torch.rand(3, 3)print(x)"""Prints out:tensor([[0.5264, 0.1839, 0.9907],        [0.0343, 0.9839, 0.9294],        [0.6938, 0.6755, 0.2258]])"""
Multiplying tensors, adding them, and other basic math is super easy with Pytorch:
x = torch.ones(3,3)y = torch.ones(3,3) * 4z = x + yprint(z)"""Prints out:tensor([[5., 5., 5.],        [5., 5., 5.],        [5., 5., 5.]])"""
Even Numpy-like slicing functions are available with Pytorch tensors!
x = torch.ones(3,3) * 5y = x[:, :2]print(y)"""Prints out:tensor([[5., 5.],        [5., 5.],        [5., 5.]])"""
So Pytorch tensors can very much be used and worked with in the same way as Numpy arrays. Now we’ll look at how we can build Deep Networks with these easy Pytorch tensors as our building blocks!
With Pytorch, neural networks are defined as Python classes. The class which defines the network extends the torch.nn.Module from the Torch library. Let’s create a class for a Convolutional Neural Network (CNN) which we’ll apply on the MNIST dataset.
Check out the code below which defines our network!
The two most important functions in a Pytorch network class are the __init__() and the forward() functions. The __init__() is used to define any network layers that your model will use. The forward() function is where you actually set up the model by stacking all the layers together.
For our model, we’ve defined 2 convolutional layers in the init function, one of which we’ll re-use a few times (conv2). We have a max-pooling layer and a global average pooling layer to be applied near the end. Finally we have our Full-Connected (FC) layers and a softmax to get the final output probabilities.
In the forward function, we define exactly how our layers stack up together to form the full model. It’s a standard network with stacked conv, pooling, and FC layers. The beauty of Pytorch is that we can print out the shape and result of any tensor within the intermediate layers with just a simple print statement wherever you want in the forward() function!
Time to get our data ready for training! We’ll starting but getting the necessary imports ready, initialise parameters, and making sure Pytorch is setup to use the GPU. The line below which uses torch.device() checks if Pytorch was installed with CUDA support and if so uses the GPU!
We can retrieve the MNIST dataset straight from Pytroch. We’ll download the data and put the train and test sets into separate tensors. Once that data is loaded, we’ll pass it to a torch DataLoader which just gets it ready to pass to the model with a specific batch size and optional shuffling.
It’s training time!
The optimzer (we’ll use Adam) and loss function (we’ll use cross entropy) are defined quite similarly to other deep learning libraries like TensorFlow, Keras, and MXNet.
In Pytorch, all network models and datasets much be explicitly transferred from CPU to GPU. We do this by applying the .to() function to our model below. Later, we’ll do the same for our image data.
Finally, we can write out our training loop. Check out the code below to see how it works!
All Pytorch training loops will go through each epoch and each batch in the training data loader.On each loop iteration, the image data and labels are transferred to the GPU.Each training loop also explicitly applies the forward pass, backward pass, and optimisation steps.The model is applied to the images in the batch and then the loss for that batch is calculated.The gradients are calculated and back-propagated through the network
All Pytorch training loops will go through each epoch and each batch in the training data loader.
On each loop iteration, the image data and labels are transferred to the GPU.
Each training loop also explicitly applies the forward pass, backward pass, and optimisation steps.
The model is applied to the images in the batch and then the loss for that batch is calculated.
The gradients are calculated and back-propagated through the network
Testing a network’s performance in Pytorch sets up a similar loop as in the training phase. The main difference being that we don’t need to do a backward propagation of the gradients. We’ll still do the forward-pass and just get the label with the maximum probability at the output of the network.
In this case, after 10 epochs our network got an accuracy of 99.06% on the test set!
To save the model to disk to use later, just use the torch.save() function and voila!
Follow me on twitter where I post all about the latest and greatest AI, Technology, and Science! Connect with me on LinkedIn too!