A guide to help you learn Backpropagation
By: Marcin Kraszewski
Tags: Machine Learning, Learing Disabilities, Learning And Development, Focus

Stuffing knowledge into your brain. Credit: Author

In my attempts to learn this deceptively simple concept I’ve noticed myself going through various stages of understanding. Let’s explore each of those below. I’ll do my best to offer the most effective approach at each phase. This is just my approach to mastering this specific concept, results may vary, and you might Grok it after watching some random Youtube video on the topic.

Learn to spell the word

Image credit: the author

Yes, this is the very first thing you should do. Get good at spelling `backpropagation`. It may seem like too low of a level at which to start, but nothing is more embarrassing than realizing you have no idea how to spell a word that is so important in the field of machine learning.

Let’s practice (read the letters aloud):

b-a-c-k-p-r-o-p-a-g-a-t-i-o-n

Say the word:

backpropagation

Repeat until it’s hard-wired into your brain.

Read the entire Wikipedia entry

I like to draw out formulas using large pencils, markers, or crayons (sometimes). This is based on the illustration: “error function over n training examples” from Wikipedia.

Yes. Don’t just skim the first section. Don’t jump around. Read this entire article starting from the top. Take it slow. If you run into a term you don’t know, please for your own sake, right-click on the word, open it in a new tab and read the first section. If that gives you enough of an understanding continue reading the backpropagation article where you left off. If it’s didn’t give you enough understanding, read that entire article as well.

Warning this may go on so long that you’ll reach the entry on Philosophy, but that’s something else entirely.

Write your definition of backpropagation

Do your best to make it short and precise. Leave it alone for 5 minutes. If after 5 minutes read your definition. If it sounds off, continue — and even if it sounds ok, you should probably keep learning, to make sure you get this.

Watch this video at 1x

Watch this video as you would a show on Netflix. Relax sit back and let the soothing voice of the instructor lull you into a sense of understanding backpropagation.

Watch this video and take copious notes

That was fun. Now let’s be more deliberate. As you watch this video:

take notes about important concepts the instructor mentions

write out the formulas along with the instructor

with the video paused, try to explain the formula you’ve just written down and how it relates to backpropagation.

Andrew Ng of Coursera explains backpropagation

Accept that you may not have learned much!

Ok. You’ve done all of this work and you just don’t get backpropagation. You’ve felt your heart rate increase at the sight of various math formulas and you may have even heard certain words for the first time in your life.

It may be the case that you don’t have enough of the foundational knowledge mastered to be able to honestly say to yourself that you understand this topic.

Get ready to learn a lot more. :)

If you want to get proficient at machine learning I’m going to strongly suggest you make a determined effort to learn the math behind machine learning. The most important areas of math to understand machine learning concepts are Linear Algebra and Calculus.

Khan will help you step by step

Sign-up for a free account. Complete all the Linear Algebra modules. Complete all the Calculus modules.

Congratulations!

If at this point you still don’t understand backpropagation, that’s ok. Machine Learning is difficult. Yes, anyone can “learn Tensorflow in 5 minutes” but what does that even mean? Anyone can learn how to use a hammer in a few seconds. But, what are you going to do next? If all you have is a hammer, you’ll bang up the whole world without getting anywhere. Heck, you might even lose your new job for “faking it until you burn-out”.

When faced with your greatest fears, be deliberate, and work your buns off.

Why are so many people in such a hurry to learn things? They may have a belief that if they just learn this one thing, their lives will be better. But the most important skill in the 21st century may be embracing one’s lack of knowledge and accepting it as a challenge. Not as a chore to be done with as soon as possible, but as something that should be done with one breath in and one breath out.

Life (which includes learning) is a journey

Stop wanting results now. The process of learning is just as stimulating and engaging as playing StarCraft or downing a bottle of Cola, as long as you let yourself be immersed in the experience. Put away your phone, close your 35 tabs, and just FOCUS on the topic at hand.

Your homework

Write an article about a difficult topic you are trying to learn. Publish it right here on medium as Part 1. When you’ve mastered the topic, write a follow-up as Part 2. People will thank you. Learning is a challenge, that is much greater for some of us than for others. By writing about your personal experience, and success you may help that person sitting in a corner hating themselves for being worthless to get just enough motivation to nudge them in the right direction.

Share your struggles! Share your triumphs!