A Simple Neural Network. This is the simplest possible neural... | by Doug Wright | Towards Data Science

Though I recently completed Udacity’s Deep Learning NanoDegree program, I am fairly new to this subject. In an effort to further my studies, I am currently reading an excellent book on the subject, Andrew Trask’s Grokking Deep Learning. This blog post is directly inspired by this fantastic book.
So, what is a neural network?
Neural Networks are incredibly powerful parametric models that transform your input data into output data using a combination of matrices and differentiable functions. — Andrew Trask
In other words, a neural network is a way of modeling your input data so that math functions performed on this data produce meaningful results. Let’s dig in and build one to learn more.
def neural_network(input, weight):    prediction = input * weight    return prediction
This is the simplest possible neural network. We take an input (real world data) and a weight to multiply against the input, then we return the result. So for example, let’s say that Denmark’s soccer team scored 3 goals in their last game. Given a weight of 0.2, we can use our neural network to predict that Denmark has a 60% chance to win their next game.
$ neural_network(3, 0.2)$ => 0.6
But how do we find our weight?
With supervised deep learning, we can train our model with input data and actual results to determine a weight. For example, if we had the following historical data for Denmark’s soccer team.
GOALS  |  WIN/LOSS3      |  11      |  04      |  1
We can then tell our network to start with a random weight and compute the likelihood of winning. Differing the computed prediction with the actual win/loss number, we can modify our weight over and over again until we get a number that, when multiplied by our inputs (goals scored), gives us a number closest to the actual win/loss result.
So, if the random number at first was 0.5, our prediction would be
3 * 0.5 = 1.51 * 0.5 = 0.54 * 0.5 = 2
With this starting point, our network is 50% over estimating for the first game, 50% over for the second, and 100% over estimating for the last game. Now with this information, we can go back and modify our weight to reduce our error rate in predicting our outcome. This is the essence of linear regression, which I won’t discuss in detail in this article. But this should give you a sense of how we can determine weights with supervised learning, by reducing our error rate each time we modify our weights and compare our predictions with our outcomes.
Before I leave you scratching your head with how the above code can possibly be used to drive autonomous cars, translate language or win a Go game, let’s dive into a multiple input neural net.
What if we had more data than just number of goals scored to predict if Denmark’s soccer team will win their next game. Let’s say we knew the number of goals, their current win/loss ratio, and the number of fans they have.
Goals Scored  |  Win/Loss Ratio  |  No. Fans3             |  0.6             |  30,000
With more data, we can combine multiple inputs to compute a weighted sum for our prediction.
def neural_network(inputs, weights):    output = 0    for i in range(len(inputs)):      output += (inputs[i] * weights[i])    return output
Taking our 3 points of input data with the following weights, we can determine that Denmark is 78% likely to win their next game.
weights = [0.2, 0.3, 0]inputs  = [  3, 0.6, 30000]$ neural_network(inputs, weights)$ => 0.78
Notice that our corresponding weight for number of fans is zero. You can imagine that given a set of historical game data, number of fans may prove to have little outcome on the actual game, thus a computed weight would be close to zero.
Congratulations! You have just learned one of the essential building blocks of Deep Learning neural networks. We take data in and perform math functions (in our case, multiplication) on our data to determine an output. With larger data sets we can train our networks to compute more accurate weights. We can also use hidden layers to perform more functions before determining our outputs.
For more information, I highly recommend Andrew Trask’s Grokking Deep Learning and watching Siraj Raval’s YouTube channel.