Title: Intro to PyTorch: Training your first neural network using PyTorch - PyImageSearch

URL Source: https://pyimagesearch.com/2021/07/12/intro-to-pytorch-training-your-first-neural-network-using-pytorch/

Published Time: 2021-07-12T14:00:00+00:00

Markdown Content:
In this tutorial, you will learn how to train your first neural network using the PyTorch deep learning library.

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20467'%3E%3C/svg%3E)

This tutorial is part two in our five part series on PyTorch deep learning fundamentals:

1.  [_What is PyTorch?_](https://pyimagesearch.com/2021/07/05/what-is-pytorch/)
2.  _Intro to PyTorch: Training your first neural network using PyTorch_ (today’s tutorial)
3.  _PyTorch: Training your first Convolutional Neural Network_ (next week’s tutorial)
4.  _PyTorch image classification with pre-trained networks_
5.  _PyTorch object detection with pre-trained networks_

By the end of this guide, you will have learned:

1.  How to define a basic neural network architecture with PyTorch
2.  How to define your loss function and optimizer
3.  **How to properly zero your gradient, perform backpropagation, and update your model parameters** — most deep learning practitioners new to PyTorch make a mistake in this step

**To learn how to train your first neural network with PyTorch,** **_just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

Inside this guide, you will become familiar with common procedures in PyTorch, including:

1.  Defining your neural network architecture
2.  Initializing your optimizer and loss function
3.  Looping over your number of training epochs
4.  Looping over data batches inside each epoch
5.  Making predictions and computing the loss on the current batch of data
6.  Zeroing out your gradient
7.  Performing backpropagation
8.  Telling your optimizer to update the gradients of your network
9.  Telling PyTorch to train your network with a GPU (if a GPU is available on your machine, of course)

We’ll start by reviewing our project directory structure and then configuring our development environment.

From there, we’ll implement two Python scripts:

1.  The first script will be our simple feedforward neural network architecture, implemented with Python and the PyTorch library
2.  The second script will then load our example dataset and demonstrate how to train the network architecture we just implemented using PyTorch

With our two Python scripts implemented, we’ll move on to training our network. We’ll wrap up the tutorial with a discussion of our results.

Let’s get started!

### **Configuring your development environment**

To follow this guide, you need to have the PyTorch deep learning library and the scikit-machine learning package installed on your system.

Luckily, both PyTorch and scikit-learn are extremely easy to install using pip:

$ pip install torch torchvision
$ pip install scikit-image

**If you need help configuring your development environment for PyTorch, I _highly recommend_ that you** [**read the PyTorch documentation**](https://pytorch.org/get-started/locally/) — PyTorch’s documentation is comprehensive and will have you up and running quickly.

### **Having problems configuring your development environment?**

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20468'%3E%3C/svg%3E)](https://pyimagesearch.com/pyimagesearch-university/)

**Figure 1:** Having trouble configuring your dev environment? Want access to pre-configured Jupyter Notebooks running on Google Colab? Be sure to join [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/) — you’ll be up and running with this tutorial in a matter of minutes.

All that said, are you:

*   Short on time?
*   Learning on your employer’s administratively locked system?
*   Wanting to skip the hassle of fighting with the command line, package managers, and virtual environments?
*   **Ready to run the code** **_right now_** **on your Windows, macOS, or Linux system?**

Then join [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/) today!

**Gain access to Jupyter Notebooks for this tutorial and other PyImageSearch guides that are** **_pre-configured_** **to run on Google Colab’s ecosystem right in your web browser!** No installation required.

And best of all, these Jupyter Notebooks will run on Windows, macOS, and Linux!

### **Project structure**

To follow along with this tutorial, be sure to access the **_“Downloads”_** section of this guide to retrieve the source code.

You’ll then be presented with the following directory structure.

$ tree . --dirsfirst
.
├── pyimagesearch
│   └── mlp.py
└── train.py

1 directory, 2 files

The `mlp.py` file will store our implementation of a basic multi-layer perceptron (MLP).

We’ll then implement `train.py` which will be used to train our MLP on an example dataset.

### **Implementing our neural network with PyTorch**

![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20361'%3E%3C/svg%3E)

**Figure 2:** Implementing a basic multi-layer perceptron with PyTorch.

You are now about ready to implement your first neural network with PyTorch!

This network is a very simple feedforward neural network called a **multi-layer perceptron (MLP)** (meaning that it has one or more hidden layers). You’ll learn how to build more advanced neural network architectures next week’s tutorial.

To get started building our PyTorch neural network, open the `mlp.py` file in the `pyimagesearch` module of your project directory structure, and let’s get to work:

\# import the necessary packages
from collections import OrderedDict
import torch.nn as nn

def get\_training\_model(inFeatures=4, hiddenDim=8, nbClasses=3):
	# construct a shallow, sequential neural network
	mlpModel = nn.Sequential(OrderedDict(\[
		("hidden\_layer\_1", nn.Linear(inFeatures, hiddenDim)),
		("activation\_1", nn.ReLU()),
		("output\_layer", nn.Linear(hiddenDim, nbClasses))
	\]))

	# return the sequential model
	return mlpModel

**Lines 2 and 3** import our required Python packages:

*   `OrderedDict`: A dictionary object that _remembers the order in which objects were added_ — we use this ordered dictionary to provide human-readable names to each layer in the network
*   `nn`: PyTorch’s neural network implementations

We then define the `get_training_model` function (**Line 5**) which accepts three parameters:

1.  The number of input nodes to the neural network
2.  The number of nodes in the hidden layer of the network
3.  The number of output nodes (i.e., dimensionality of the output prediction)

Based on the default values provided, you can see that we are building a 4-8-3 neural network, meaning that the input layer has 4 nodes, the hidden layer 8 nodes, and the output of the neural network will consist of 3 values.

The actual neural network architecture is then constructed on **Lines 7-11** by first initializing a `nn.Sequential` object (very similar to Keras/TensorFlow’s `Sequential` class).

Inside the `Sequential` class we build an `OrderedDict` where each entry in the dictionary consists of two values:

1.  A string containing the human-readable name for the layer (which is _very useful_ when debugging neural network architectures using PyTorch)
2.  The PyTorch layer definition itself

The `Linear` class is our **fully connected** layer definition, meaning that each of the inputs connects to each of the outputs in the layer. The `Linear` class accepts two required arguments:

1.  The number of **inputs** to the layer
2.  The number of **outputs**

On **Line 8**, we define `hidden_layer_1` which consists of a fully connected layer accepting `inFeatures` (4) inputs and then producing an output of `hiddenDim` (8).

From there, we apply a ReLU activation function (**Line 9**) followed by another `Linear` layer which serves as our output (**Line 10**).

**Notice that the second `Linear` definition contains the _same_ number of inputs as the _previous_ `Linear` layer did outputs — this is not by accident!** The output dimensions of the previous layer _must_ match the input dimensions of the next layer, otherwise PyTorch will error out (and then you’ll have the quite tedious task of debugging the layer dimensions yourself).

PyTorch is _not_ as forgiving in this regard (as opposed to Keras/TensorFlow), so _be extra cautious_ when specifying your layer dimensions.

The resulting PyTorch neural network is then returned to the calling function.

### **Creating our PyTorch training script**

With our neural network architecture implemented, we can move on to training the model using PyTorch.

To accomplish this task, we’ll need to implement a training script which:

1.  Creates an instance of our neural network architecture
2.  Builds our dataset
3.  Determines whether or not we are training our model on a GPU
4.  Defines a training loop (the hardest part of our script)

Open `train.py`, and lets get started:

\# import the necessary packages
from pyimagesearch import mlp
from torch.optim import SGD
from sklearn.model\_selection import train\_test\_split
from sklearn.datasets import make\_blobs
import torch.nn as nn
import torch

**Lines 2-7** import our required Python packages, including:

*   `mlp`: Our definition of the multi-layer perceptron architecture, implemented in PyTorch
*   `SGD`: The [Stochastic Gradient Descent](https://pyimagesearch.com/2016/10/17/stochastic-gradient-descent-sgd-with-python/) optimizer that we’ll be using to train our model
*   `make_blobs`: Builds a synthetic dataset of example data
*   `train_test_split`: Splits our dataset into a training and testing split
*   `nn`: PyTorch’s neural network functionality
*   `torch`: The base PyTorch library

When training a neural network, we do so in _batches_ of data (as you’ve previously learned). The following function, `next_batch`, yields such batches to our training loop:

def next\_batch(inputs, targets, batchSize):
	# loop over the dataset
	for i in range(0, inputs.shape\[0\], batchSize):
		# yield a tuple of the current batched data and labels
		yield (inputs\[i:i + batchSize\], targets\[i:i + batchSize\])

The `next_batch` function accepts three arguments:

1.  `inputs`: Our input data to the neural network
2.  `targets`: Our target output values (i.e., what we want our neural network to accurately predict)
3.  `batchSize`: Size of data batch

We then loop over our input data in `batchSize` chunks (**Line 11**) and yield them to the calling function (**Line 13**).

Next, we have some important initializations to take care of:

\# specify our batch size, number of epochs, and learning rate
BATCH\_SIZE = 64
EPOCHS = 10
LR = 1e-2

# determine the device we will be using for training
DEVICE = "cuda" if torch.cuda.is\_available() else "cpu"
print("\[INFO\] training using {}...".format(DEVICE))

When training our neural network with PyTorch we’ll use a batch size of 64, train for 10 epochs, and use a learning rate of 1e-2 (**Lines 16-18**).

We set our training device (either CPU or GPU) on **Line 21.** A GPU will certainly speed up training but is _not required_ for this example.

Next, we need an example dataset to train our neural network on. We’ll learn how to load images from disk and train a neural network on image data in the next tutorial in this series, but for now, let’s use [scikit-learn’s make\_blobs function](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_blobs.html) to create a synthetic dataset for us:

\# generate a 3-class classification problem with 1000 data points,
# where each data point is a 4D feature vector
print("\[INFO\] preparing data...")
(X, y) = make\_blobs(n\_samples=1000, n\_features=4, centers=3,
	cluster\_std=2.5, random\_state=95)

# create training and testing splits, and convert them to PyTorch
# tensors
(trainX, testX, trainY, testY) = train\_test\_split(X, y,
	test\_size=0.15, random\_state=95)
trainX = torch.from\_numpy(trainX).float()
testX = torch.from\_numpy(testX).float()
trainY = torch.from\_numpy(trainY).float()
testY = torch.from\_numpy(testY).float()

**Lines 27 and 28** build our dataset, consisting of:

1.  Three class labels (`centers=3`)
2.  Four total features/inputs to the neural network (`n_features=4`)
3.  A total of 1000 data points (`n_samples=1000`)

Essentially, the `make_blobs` function is generating Gaussian blobs of clustered data points. For 2D data, the `make_blobs` function would create data similar to the following:

![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20650%20430'%3E%3C/svg%3E)

**Figure 3:** An example 2D dataset generated using scikit-learn’s _“make\_blobs”_ function ([image source](https://www.mlguides.com/guides/data-analysis/generate-test-data-machine-learning-python-using-scikit-learn/make_blobs/)).

Notice there are three clusters of data here. We are doing the same thing, but instead of two dimensions we have four dimensions (meaning we cannot easily visualize it).

Once our data is generated, we apply the `train_test_split` function (**Lines 32 and 33**) to create our training split, 85% for training and 15% for evaluation.

From there, the training and testing data is converted to PyTorch tensors from NumPy arrays, and then converted to the floating point data type (**Lines 34-37**).

Let’s now instantiate our PyTorch neural network architecture:

\# initialize our model and display its architecture
mlp = mlp.get\_training\_model().to(DEVICE)
print(mlp)

# initialize optimizer and loss function
opt = SGD(mlp.parameters(), lr=LR)
lossFunc = nn.CrossEntropyLoss()

**Line 40** initializes our MLP and pushes it to whatever `DEVICE` we are using for training (either CPU or GPU).

**Line 44** defines our SGD optimizer, which accepts two arguments:

1.  The MLP model parameters, obtained by simply calling `mlp.parameters()`
2.  The learning rate

Finally, we initialize our categorical cross-entropy loss function, which is the standard loss method you’ll use when performing classification with _\> 2_ classes.

**We now arrive at our most important code block, the training loop.** Unlike Keras/TensorFlow, which allow you to simply call `model.fit` to train your model, PyTorch _requires_ that you implement your training loop _by hand._

There are pros and cons of having to implement the training loop by hand.

On one side of the spectrum, you have _complete and total control_ over the training procedure, which makes it easier to implement custom training loops.

But on the other side of the spectrum, implementing a training loop by hand requires more code, and worst of all, makes it far easier to shoot yourself in the foot (which can be especially true for budding deep learning practitioners).

**My suggestion: You’ll want to read the explanations to the following code blocks** **_multiple times_** **so that you understand the intricacies of the training loop. You’ll** **_especially_** **want to pay close attention to how we zero the gradient, perform backpropagation, and then update the model parameters —** **_failing to do so in that exact order will lead to erroneous results!_**

Let’s review our training loop:

\# create a template to summarize current training progress
trainTemplate = "epoch: {} test loss: {:.3f} test accuracy: {:.3f}"

# loop through the epochs
for epoch in range(0, EPOCHS):
	# initialize tracker variables and set our model to trainable
	print("\[INFO\] epoch: {}...".format(epoch + 1))
	trainLoss = 0
	trainAcc = 0
	samples = 0
	mlp.train()

	# loop over the current batch of data
	for (batchX, batchY) in next\_batch(trainX, trainY, BATCH\_SIZE):
		# flash data to the current device, run it through our
		# model, and calculate loss
		(batchX, batchY) = (batchX.to(DEVICE), batchY.to(DEVICE))
		predictions = mlp(batchX)
		loss = lossFunc(predictions, batchY.long())

		# zero the gradients accumulated from the previous steps,
		# perform backpropagation, and update model parameters
		opt.zero\_grad()
		loss.backward()
		opt.step()

		# update training loss, accuracy, and the number of samples
		# visited
		trainLoss += loss.item() \* batchY.size(0)
		trainAcc += (predictions.max(1)\[1\] == batchY).sum().item()
		samples += batchY.size(0)

	# display model progress on the current training batch
	trainTemplate = "epoch: {} train loss: {:.3f} train accuracy: {:.3f}"
	print(trainTemplate.format(epoch + 1, (trainLoss / samples),
		(trainAcc / samples)))

**Line 48** initializes `trainTemplate`, a string that will allow us to conveniently display the epoch number, along with the loss and accuracy at each step.

We then loop over our number of desired training epochs on **Line 51.** Immediately inside this `for` loop we:

1.  Show the epoch number, which is useful for debugging purposes (**Line 53**)
2.  Initialize our training loss and accuracy (**Lines 54 and 55**)
3.  Initialize the total number of data points used inside the current iteration of the training loop (**Line 56**)
4.  Put the PyTorch model in training mode (**Line 57**)

**Calling the `train()` method of the PyTorch model is required for the model parameters to be updated during backpropagation.**

In our next code block, you’ll see that we put the model into `eval()` mode so that we can evaluate the loss and accuracy on our testing set. If we forgot to then call `train()` at the top of the next training loop, then our model parameters will not be updated.

The outer `for` loop (**Line 51**) loops over our number of epochs. **Line 60** then starts an inner `for` loop that loops over each of our batches in the training set. **Nearly every training procedure you write using PyTorch will consist of an _outer loop_ (over the number of epochs) and an _inner loop_ (over the data batches).**

Within the inner loop (i.e., the batch loop), we proceed to:

1.  Move the `batchX` and `batchY` data to our CPU or GPU (depending on our `DEVICE`)
2.  Pass the `batchX` data through the neural and make predictions on it
3.  Use our loss function to compute our loss by comparing the output `predictions` to our ground-truth class labels

Now that we have our `loss`, we can update our model parameters — **this is the most important step in the PyTorch training procedure and often the one most beginners mess up.**

To update the parameters of our model, we _must_ call **Lines 69-71** in the _exact order_ specified:

1.  `opt.zero_grad()`: Zeros the gradients accumulated from the previous batch/step of the model
2.  `loss.backward()`: Performs backpropagation
3.  `opt.step()`: Updates the weights in our neural network based on the results of backpropagation

**Again, I want to stress that you** **_must_** **apply zeroing the gradients, performing a backward pass, and then updating the model parameters in the** **_exact order_** **that I’ve indicated.**

As I’ve mentioned, PyTorch gives you _a lot_ of control over your training loop … but it also makes it _very easy_ to shoot yourself in the foot. Every single deep learning practitioner, whether brand new to the world of deep learning or a seasoned expert, has at one time or another messed up these steps.

**The most common mistake is forgetting to zero the gradient.** If you don’t zero the gradient then you’ll accumulate gradients across multiple batches and over multiple epochs. That will mess up your backpropagation and lead to erroneous weight updates.

**Seriously, don’t mess up these steps.** Write them on a sticky note and put them on your monitor if you need to.

After we’ve updated the weights to our model, we compute our train loss, train accuracy, and number of samples examined (i.e., number of data points in the batch) on **Lines 75-77.**

We then apply our `trainTemplate` to display our epoch number, training loss, and training accuracy. Note how we divide our loss and accuracy by the total number of samples in the batch to obtain an average.

**At this point, we’ve trained our PyTorch model on all data points in an epoch — now we need to evaluate it on our testing set:**

	# initialize tracker variables for testing, then set our model to
	# evaluation mode
	testLoss = 0
	testAcc = 0
	samples = 0
	mlp.eval()

	# initialize a no-gradient context
	with torch.no\_grad():
		# loop over the current batch of test data
		for (batchX, batchY) in next\_batch(testX, testY, BATCH\_SIZE):
			# flash the data to the current device
			(batchX, batchY) = (batchX.to(DEVICE), batchY.to(DEVICE))

			# run data through our model and calculate loss
			predictions = mlp(batchX)
			loss = lossFunc(predictions, batchY.long())

			# update test loss, accuracy, and the number of
			# samples visited
			testLoss += loss.item() \* batchY.size(0)
			testAcc += (predictions.max(1)\[1\] == batchY).sum().item()
			samples += batchY.size(0)

		# display model progress on the current test batch
		testTemplate = "epoch: {} test loss: {:.3f} test accuracy: {:.3f}"
		print(testTemplate.format(epoch + 1, (testLoss / samples),
			(testAcc / samples)))
		print("")

Similar to how we initialized our training loss, training accuracy, and number of samples in a batch, we do the same thing for our testing set on **Lines 86-88.** Here, we initialize variables to store our testing loss, testing accuracy, and number of samples in the testing set.

We also put our model into `eval()` model on **Line 89.** We are _required_ to put our model in evaluation mode when we need to compute losses/accuracies on the testing or validation set.

**But what does the `eval()` mode actually _do?_** You think of evaluation mode as a switch for turning off specific layer functionality, such as stopping dropout from being applied, or allowing the accumulated states of batch normalization to be applied.

**Secondly, you typically use `eval()` in conjunction with a `torch.no_grad()` context,** meaning that gradient computation is turned _off_ in evaluation mode (**Line 92**).

From there, we loop over all batches in our testing set (**Line 94**), similar to how we looped over our training batches in the previous code block.

For each batch (**Line 96**), we make predictions using our model and then compute the loss (**Lines 99 and 100**).

We then update our `testLoss`, `testAcc`, and number of `samples` (**Lines 104-106**).

Finally, we display our epoch number, testing loss, and testing accuracy on our terminal (**Lines 109-112**).

**In general, the evaluation portion of our training loop is** **_very similar_** **to the training portion, with no minor** **_but very significant_** **changes:**

1.  We put our model into evaluation mode using `eval()`
2.  We use a `torch.no_grad()` context to ensure no graduation computation is performed

From there, we can make predictions using our model and compute the accuracy/loss on the testing set.

### **PyTorch training results**

We are now ready to train our neural network with PyTorch!

Be sure to access the **_“Downloads”_** section of this tutorial to retrieve the source code.

To launch the PyTorch training process, simply execute the `train.py` script:

$ python train.py
\[INFO\] training on cuda...
\[INFO\] preparing data...
Sequential(
  (hidden\_layer\_1): Linear(in\_features=4, out\_features=8, bias=True)
  (activation\_1): ReLU()
  (output\_layer): Linear(in\_features=8, out\_features=3, bias=True)
)
\[INFO\] training in epoch: 1...
epoch: 1 train loss: 0.971 train accuracy: 0.580
epoch: 1 test loss: 0.737 test accuracy: 0.827

\[INFO\] training in epoch: 2...
epoch: 2 train loss: 0.644 train accuracy: 0.861
epoch: 2 test loss: 0.590 test accuracy: 0.893

\[INFO\] training in epoch: 3...
epoch: 3 train loss: 0.511 train accuracy: 0.916
epoch: 3 test loss: 0.495 test accuracy: 0.900

\[INFO\] training in epoch: 4...
epoch: 4 train loss: 0.425 train accuracy: 0.941
epoch: 4 test loss: 0.423 test accuracy: 0.933

\[INFO\] training in epoch: 5...
epoch: 5 train loss: 0.359 train accuracy: 0.961
epoch: 5 test loss: 0.364 test accuracy: 0.953

\[INFO\] training in epoch: 6...
epoch: 6 train loss: 0.302 train accuracy: 0.975
epoch: 6 test loss: 0.310 test accuracy: 0.960

\[INFO\] training in epoch: 7...
epoch: 7 train loss: 0.252 train accuracy: 0.984
epoch: 7 test loss: 0.259 test accuracy: 0.967

\[INFO\] training in epoch: 8...
epoch: 8 train loss: 0.209 train accuracy: 0.987
epoch: 8 test loss: 0.215 test accuracy: 0.980

\[INFO\] training in epoch: 9...
epoch: 9 train loss: 0.174 train accuracy: 0.988
epoch: 9 test loss: 0.180 test accuracy: 0.980

\[INFO\] training in epoch: 10...
epoch: 10 train loss: 0.147 train accuracy: 0.991
epoch: 10 test loss: 0.153 test accuracy: 0.980

Our first few lines of output show the simple 4-8-3 MLP architecture, meaning that there are _four_ inputs to the neural network, a single hidden layer with _eight_ nodes, and a final output layer with _three_ nodes.

We then train our network for a total of ten epochs. By the end of the training process, we are obtaining **99.1% accuracy on our training set** and **98% accuracy on our testing set.**

We can therefore conclude that our neural network is doing a good job making accurate predictions.

Congrats on training your first neural network with PyTorch!

### **How do I train a PyTorch model on my own custom dataset?**

This tutorial showed you how to train a PyTorch neural network on an example dataset generated by scikit-learn’s `make_blobs` function.

While this was a great example to learn the basics of PyTorch, it’s admittedly not very interesting from a real-world scenario perspective.

Next week, you’ll learn how to train a PyTorch model on a dataset of handwritten characters, which has many practical applications, including handwriting recognition, OCR, _and more!_

Stay tuned for next week’s tutorial to learn more about PyTorch and image classification.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

**Summary**
-----------

In this tutorial, you learned how to train your first neural network using the PyTorch deep learning library. This example was admittedly simple, but demonstrated the fundamentals of the PyTorch framework.

**The biggest mistake I see with deep learning practitioners new to the PyTorch library is forgetting and/or mixing up the following steps:**

1.  Zeroing out gradients from the previous steps (`opt.zero_grad()`)
2.  Performing backpropagation (`loss.backward()`)
3.  Updating model parameters (`opt.step()`)

**Failure to perform these steps** **_in this exact order_** **is a surefire way to shoot yourself in the foot when using PyTorch, and worse, PyTorch doesn’t report an error if you mix up these steps,** **_so you may not even know you shot yourself!_**

The PyTorch library is _super powerful,_ but you’ll need to get used to the fact that training a neural network with PyTorch is like taking off your bicycle’s training wheels — there’s no safety net to catch you if you mix up important steps (unlike with Keras/TensorFlow which allow you to encapsulate entire training procedures into a single `model.fit` call).

That’s not to say that Keras/TensorFlow are “better” than PyTorch — it’s just a difference between the two deep learning libraries of which you need to be aware.

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _simply enter your email address in the form below!_**

![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!