How To Train Your Neural Network: The Hidden Code | by Murto Hilali | Towards Data Science

On the flip side, you’re far more likely to have encountered some form of artificial intelligence in your life than you have a dragon — or maybe you have, who knows? I don’t know your life.
Needless to say, AI has become an integral part of our lives, from the Netflix algorithm magically making your weekend disappear to Google Duplex making phone calls for you so you never have to deal with the anxiety of booking an appointment over the phone ever again (human interaction? *gasp*).
But AI is on the path to solving bigger problems than ever before, ranging from tumour detection to self-driving cars. The world is changing rapidly, so it’s important you know how the future is going to work. After all, how the heck did AI get to be so smart in the first place?
The dragons in How to Train Your Dragon and the humans of Earth (that’s us) have a lot in common. We’re both pretty smart, but we don’t start out like that.
Growing up, we’re fed information about the world around us — from our parents, friends, teachers, (in the dragons’ case, trainers) etc. Using this information, we try to make some models for making good choices for ourselves and others (unless you’re some sort of anarchist, in which case Viva la Revolution, I guess?)
Sometimes, the choices we make don’t turn out so well. For instance, not buying Facebook stock in May 2012. Or buying Facebook stock in July 2018. But we learn from our mistakes and adjust our decision-making process accordingly, so we make better decisions in the future. Good for us!
Using some not-too-complex mathematics, computer scientists have developed a process called machine learning that does basically the same thing. Machine learning describes the method of feeding a neural network lots of data, letting it find patterns in it, then having it make choices and predictions based on them. But much like dragons, we have to train these systems first.
So what is a neural network, and how exactly does it learn?
Neural networks are systems of neurons or fancy perceptrons. A perceptron takes in multiple input values and produces a singular output, through a whole process called forward propagation — on a high level, think Power Rangers coming together to form a Megazord.
Let’s say It’s Morphin Time is the function performed on the input values, the Power Rangers, and each of them has a specific value that just corresponds to them, for example:
x1 = 0
x2 = 1
x3 = 0
Now let’s say the Black Power Ranger is more significant than the others. His input will have more importance given to it, or weight. In fact, all of the Rangers have their own respective weights:
w1 = 1
w2 = 2
w3 = 3
Then the neuron is assigned a bias which gives it the ability to change after some iterations to better fit your data. By adding together all the input values multiplied by their respective weight, as well as accounting for b = bias, we get:
(w1 × x1) + (w2 × x2) + (w3 × x3) + (1 × b )
What makes this neuron really useful is an activation function. This basically takes the sum of what we got up there and uses it as the argument (or input) for a transformation function, making it non-linear. This is done with functions like Sigmoid or other complex models of polynomial regression.
Since a lot of data doesn’t always fit into a linear relationship, this helps our neural network model to fit the data better and is the special sauce that turns the Rangers into a Megazord. Or maybe it’s alien technology. I can’t remember.
One neuron on its own isn’t really all that useful. But when we use a whole bunch of them together, we can form a network of neurons, almost like a neural network... :)
These networks can be thought of in layers, with an input layer where all the data comes, an output layer that gives you your prediction, and some hidden layers in between.
All the neurons (or nodes) in a layer connect back to all the neurons in the previous layer. They eventually produce an output through this fancy forward propagation we’ve been talking about. The real magic, however, comes from backpropagation.
Backpropagation is the process of determining the error or loss at the output, then going back into the network to adjust weights and biases based on the error; common algorithms used to achieve this include gradient descent, which involves finding derivatives of your parameters with respect to the weights, and multiplying it by your learning rate, which decides how fast you want to optimize.
In essence, we find the derivative of the loss function and try to minimize it, thereby minimizing error. Make sense?
Every round of this process, or training iteration, is called an epoch. Sounds badass, I know. For more difficult datasets, we would require numerous epochs, like all the different versions of Power Rangers (there’s like what, a million varieties?)
We can build a very simple neural network in Python right now, one that does binary classification, or prediction of 1 or 0. Note: I’m using Python 3.6
Of course, we can’t start without our good friend Numpy:
import numpy as np
Let’s set X as our input array and y as our output array:
X=np.array([[0,1,1,0],[0,1,1,1],[1,0,0,1]])y=np.array([[0],[1],[1]])
Let’s add our Sigmoid function (this will be part of forward propagation) and its derivative (this will be part of backpropagation):
def sigmoid ():    return 1/(1 + np.exp(-x))def derivatives_sigmoid ():    return x * (1-x)
Now let’s set how many epochs we want, what our learning rate lrwill be, the number of features in our dataset (1), the number of hidden layer neurons (3) and the number of output neurons (1):
epoch=10000lr=0.1 inputlayer_neurons = X.shape[1] hiddenlayer_neurons = 3 output_neurons = 1
Now we can set our initial weights and biases. Let’s let wh and bh be the weights and biases for the hidden layer neuron, and wout and bout be the same weights and biases for the output neuron:
wh=np.random.uniform(size=(inputlayer_neurons,hiddenlayer_neurons))bh=np.random.uniform(size=(1,hiddenlayer_neurons))wout=np.random.uniform(size=(hiddenlayer_neurons,output_neurons))bout=np.random.uniform(size=(1,output_neurons))
Alright, the stage is set: we have our data, our initial weights and biases are set, and our forward and backpropagation functions are ready to calculate like bosses. Let’s set the rest of the pieces into place:
Forward Propagation:
First, we’ll get the product of the weight and the input values.
Next, we’ll add the bias to the products.
Then, well use our sigmoid function to transform/activate it.
We’ll repeat this with the result in the output layer neuron.
hidden_layer_input1=np.dot(X,wh)hidden_layer_input=hidden_layer_input1 + bhhiddenlayer_activations = sigmoid(hidden_layer_input)output_layer_input1=np.dot(hiddenlayer_activations,wout)output_layer_input= output_layer_input1+ boutoutput = sigmoid(output_layer_input)
Back Propagation:
We’ll calculate the error, the difference between y and the output.
Next, we’ll find gradients of the output and transformation/activation.
Now let’s find the delta, or the change factor at the output layer by multiplying it by the error.
Now the error will propagate back into the network, so we have to get the dot product of the delta and weight parameters between the second and third layer.
We’ll do the same for the hidden layer, followed by updating the weights and biases.
E = y-outputslope_output_layer = derivatives_sigmoid(output)slope_hidden_layer = derivatives_sigmoid(hiddenlayer_activations)d_output = E * slope_output_layerError_at_hidden_layer = d_output.dot(wout.T)d_hiddenlayer = Error_at_hidden_layer * slope_hidden_layerwout += hiddenlayer_activations.T.dot(d_output) *lrbout += np.sum(d_output, axis=0,keepdims=True) *lrwh += X.T.dot(d_hiddenlayer) *lrbh += np.sum(d_hiddenlayer, axis=0,keepdims=True) *lr
We can finish off by printing our output:
print (output)
It should return something like this:
[[0.03391414] [0.97065091] [0.9895072 ]]
Our original outputs were 0, 1, 1, and these outputs are super close. It’s good they’re not exact because it allows for flexibility with other data. You can mess around with the number of epochs to how it makes the model less accurate.
We did it! There are, of course, different ways to code a neural network, and you can find plenty of articles that teach you, this is just one of the ways I’ve learned. Now while everyone else is busy tapping away at their screenplay at Starbucks, you can be happy knowing you’re riding the dragon of the future.
(That’s a lot cheesier in writing than it was in my head.)
You’ve trained your dragon. It took 10 000 epochs, but you did it. But this was just the beginning. You’ve got a whole franchise ahead of you.
Today machine learning is being applied in some really exciting ways. Companies like Cogito are using it to help call center workers gauge client interest over the phone just by analyzing their voice. The medical world is using AI to help diagnose malignant blood cells and tumours. Computer vision is helping to put self-driving cars on the road. There’s even AI writing entire books!
In How to Train Your Dragon, partnering with dragons lets the riders of Berk improve life socially and agriculturally. AI is already improving life for us as we know it; it’s disrupting every single industry in the world right now, so it’s important you get onboard ASAP. Understanding how neural networks work, even slightly, puts you ahead of the curve.
So understanding artificial intelligence may not make you as cool as someone who has a dragon, but it’s revolutionary, exciting, and bound to secure your place in the future all the same.
A neuron takes in multiple inputs and produces an output. Inputs have associated weights, and neurons apply their own biases.
Neurons also include an activation function, which makes the data graph non-linear. When we layer a bunch of these together, we have a neural network.
Backpropagation is the process of adjusting weights and biases to minimize error at the output, often using training algorithms like gradient descent.
Data requires numerous rounds of training iterations, or epochs, to be accurate.
Artificial intelligence is changing how the world works, from the medical industry to literature. It’s a disruptive train you want to be on.
Check out my other article on how we’re going to make emotional AI...Connect with me on LinkedIn! If you want to talk about AI/ML or dragons, I’d love to connect!
Check out my other article on how we’re going to make emotional AI...
Connect with me on LinkedIn! If you want to talk about AI/ML or dragons, I’d love to connect!