Introduction to Deep Learning Using Keras and Tensorflow — Part1
Tags: Keras, Neural Networks, Deep Learning

We’ll cover some key concepts of deep learning with a hands-on example:

What is Deep Learning?

1) Single Neuron — Linear Unit

So let’s begin with the fundamental component of a neural network: the individual neuron. As a diagram, a neuron (or unit) with one input looks like

The input is x. Its connection to the neuron has a weight which is w. Whenever a value flows through a connection, you multiply the value by the connection’s weight. For the input x, what reaches the neuron is w * x. A neural network “learns” by modifying its weights.

The b is a special kind of weight we call the bias. The bias doesn’t have any input data associated with it; instead, we put a 1 in the diagram so that the value that reaches the neuron is just b (since 1 * b = b). The bias enables the neuron to modify the output independently of its inputs.

The y is the value the neuron ultimately outputs. To get the output, the neuron sums up all the values it receives through its connections. This neuron’s activation is y = w * x + b, or as a formula y=wx+b .

It’s an equation of a line! It’s the slope-intercept equation, where w is the slope, and b is the y-intercept.

Example — The Linear Unit as a Model

Though individual neurons will usually only function as part of a larger network, it’s often useful to start with a single neuron model as a baseline. Single neuron models are linear models.

Let’s think about how this might work on a dataset like 80 Cereals. Training a model with ‘sugars’ (grams of sugars per serving) as input and ‘calories’ (calories per serving) as output, we might find the bias is b=90 and the weight is w=2.5. We could estimate the calorie content of cereal with 5 grams of sugar per serving like this:

Multiple Inputs

The 80 Cereals dataset has many more features than just ‘sugars’. What if we wanted to expand our model to include things like fiber or protein content? That’s easy enough. We can just add more input connections to the neuron, one for each additional feature. To find the output, we would multiply each input to its connection weight and then add them all together.

The formula for this neuron would be y=w0x0+w1x1+w2x2+b. A linear unit with two inputs will fit a plane, and a unit with more inputs than that will fit a hyperplane.

Define a linear model in Keras

The easiest way to create a model in Keras is through keras.Sequential, which creates a neural network as a stack of layers. We can create models like those above using a dense layer (which we’ll learn more about in the next lesson).

We could define a linear model accepting three input features (‘sugars’, ‘fiber’, and ‘protein’) and producing a single output (‘calories’) like so:

With the first argument, units, we define how many outputs we want. In this case, we are just predicting ‘calories’, so we’ll use units=1.

With the second argument, input_shape, we tell Keras the dimensions of the inputs. Setting input_shape=[3] ensures the model will accept three features as input (‘sugars’, ‘fiber’, and ‘protein’).

Weights

Internally, Keras represents the weights of a neural network with tensors. Tensors are basically TensorFlow’s version of a Numpy array with a few differences that make them better suited to deep learning. One of the most important is that tensors are compatible with GPU and TPU) accelerators. TPUs, in fact, are designed specifically for tensor computations.

A model’s weights are kept in its weights attribute as a list of tensors. Get the weights of the model you defined above.

(If you want, you could display the weights with something like: print(“Weights

{}



Bias

{}”.format(w, b))).

Do you see how there’s one weight for each input (and a bias)? Notice though that there doesn’t seem to be any pattern to the values the weights have. Before the model is trained, the weights are set to random numbers (and the bias to 0.0). A neural network learns by finding better values for its weights.

(By the way, Keras represents weights as tensors but also uses tensors to represent data. When you set the input_shape argument, you are telling Keras the dimensions of the array it should expect for each example in the training data. Setting input_shape=[3] would create a network accepting vectors of length 3, like [0.2, 0.4, 0.6].)