RNN Numerical Walkthrough + Code. Intro to Recurrent Neural Networks…
By: Kat He
Tags: Machine Learning, Beginners Guide, Recurrent Neural Network, Neural Networks, Backpropagation

Table of Contents

Overview Model Forward Propagation Backpropagation Through Time (BPTT) Weight Updates Truncated Backpropagation Through Time (TBPTT) Conclusion

Overview

In this article, we take a look at the mathematical calculations behind a recurrent neural network. We’ll explore the outputs of the forward pass and the gradients of the backward pass. If you’re unfamiliar with forward and backward propagation in a vanilla neural network, take a look at this article: Step-by-Step Neural Network Calculations of Forward and Backpropagation

In contrast to feed forward networks, RNNs are able to capture temporal dependencies (dependencies through time) through a feedback loop that feeds its output in as input. Recurrent neural networks derive their name from the term recurrent which means occurring often or repeatedly because the same task is performed repeatedly for each input.

Some popular uses of recurrent neural networks arespeech recognition, time series prediction, NLP (machine translation, question answering), and gesture recognition. There are different variations of RNNs, depending on the number of inputs and outputs of the model. The following are the 4 principal structures:

Source: Andrej Karpathy

Examples

one-to-one : Binary Classification [single images/words → single output]

: Binary Classification [single images/words → single output] one-to-many : Image Captioning [image → Sequence of words](variability sized)

: Image Captioning [image → Sequence of words](variability sized) many-to-one : Sentiment Classification [sequence of words (variable sized) → single sentiment]

: Sentiment Classification [sequence of words (variable sized) → single sentiment] many-to-many (non-matching) : Machine Translation [sequence of words → sequence of words], where both sequences have variable length and these lenghts may be difference

: Machine Translation [sequence of words → sequence of words], where both sequences have variable length and these lenghts may be difference many-to-many (matching): Video classification on a frame-by-frame level [frame → some decision], where input and output have the same length because a decision is made at each time step

The remainder of the article will focus on a many-to-many (matching) RNN.

Model

At each timestep t:

x_t : raw input into the network

: raw input into the network h_t : hidden state that is passed through time from h_{t-1} to h_t

: hidden state that is passed through time from to y_t: output at each time step

Weights are shared between all time steps

W_x : input weights

: input weights W_h : hidden weights

: hidden weights W_y: weights of the output between the hidden and input

Initialize weight matrices

def __init__(self, input_dim, hidden_dim, output_dim, alpha=1e-3):



# Weights

self.W_x = np.random.uniform(0, 1, (hidden_dim, input_dim))

self.W_h = np.random.uniform(0, 1, (hidden_dim, hidden_dim))

self.W_y = np.random.uniform(0, 1, (output_dim, hidden_dim))



# Biases

self.b_h = np.random.uniform(0, 1, (hidden_dim, hidden_dim))

self.b_y = np.random.uniform(0, 1, (output_dim, hidden_dim)) self.hidden_dim = hidden_dim

self.output_dim = output_dim self.learning_rate = alpha

Forward Propagation

The forward propagation step is similar to forward propagation for a vanilla neural network. If you’re not familiar with the process, check out this article which runs through the math behind it step-by-step.

Once again, there are three sets of weights, W_x, W_h, and W_y, in an RNN. These weights remain the same throughout all time steps for a given sequence.

W_x is the weight matrix for the input that is fed into the network at each timestep (depending on the structure)

is the weight matrix for the input that is fed into the network at each timestep (depending on the structure) W_h is the weight matrix for the hidden input that is fed back into the model (due to the recurrence structure)

is the weight matrix for the hidden input that is fed back into the model (due to the recurrence structure) W_y is the weight matrix for the output of the network.

At each time step, we will compute the hidden state h_t and the output y_t. Essentially, forward propagation consists of the following steps:

1. Transform and combine input and hidden state, then apply an activation function.

2. Transform hidden state, then apply an activation function to get output.

Activation Functions

Activation functions are applied to the transformed inputs. In this recurrent neural network, applying the Softmax function results in the output y_t and applying the Hyperbolic Tangent results in the hidden states h_t.

Why Tanh?

Since RNNs have the vanishing/exploding gradient problem, we need to keep the gradient in linear region of the function. Therefore, we need a function whose second derivative can sustain for a long range before going to zero. Notice that that Tanh is close to linear on both sides of the y-axis. The gradient in these regions is close to 1. Essentially, we choose tanh as the hidden activation because it is has these properties.

Hyperbolic Tangent Function

def tanh(self, x):

return np.tanh(x)

Why Softmax?

The softmax function is essentially a normalizing function that takes a vector of logits and transforms it into a categorical distribution. The values z_i of the vector Z will be squished to 0 ≤ z_i ≤ 1 and the sum of all z_i from i to K classes will total to 1.

For example, our many-to-many RNN could be a character-to-character model that attempts to predict the next letter at each time step, such as in Simple RNN Deciphers Code. Thus, the output vector could be a vector of size K=26, with each class representing a letter.

Alternatively, we could also add spaces to the output vector or an empty string to pad sequences. After the Softmax function squishes the logits in the vector, each value will be between 0 to 1, representing the probability of being a certain letter. We could choose the argmax of the output vector to be our prediction where args 0 to 25 represent the letters a to z.

def softmax(self, x):

return np.exp(x) / np.sum(np.exp(x), axis=0)

Hidden State

The current hidden state depends on the previous hidden state and the input vector at that current timestep. Essentially, the hidden state can be seen as the memory in the network.

Hidden state is a function of previously computed hidden state and raw input x

By unrolling the function, notice that W_h transforms the previous hidden state and W_x transforms the input. Then, the bias term is added and the Hyperbolic Tangent activation function is applied.

Hidden State Unrolled

We will initialized the hidden state h_0 to all zeros. The hidden state should be reset with each new batch as advised here. In this article, we will reset the hidden state at each forward pass.

Output

Output at each time step

An output value is computed at each time step by transforming the hidden state with the weight W_y and adding a bias term b_y. Then, the Softmax function transforms the logits into a categorical distribution.

Notice that the output y_t is a vector of size N for N classes and that each element in the vector can be represented as i, where i is in the set from 1 to N.

In other words, this function is the normalized term for a single class i. The normalizing term is in the denominator. This denominator simply gathers all the terms from 1 to N in the output vector, sums them up, then divides the ith term in the output vector by the normalizing value.

def forward_propagation(self, x):

T = len(x) # for backpropagation calculation

## notice the hidden state gets reset each forward pass

hidden_states = np.zeros(T+1, self.hidden_dim)

outputs = np.zeros((T, self.output_dim))



# process sequence individually

for t in range(T):

hidden_states[t] = self.tanh(np.dot(self.W_h, hidden_states[t-1]) + np.dot(self.W_x, x[t]) + self.b_h)



y_t = self.softmax(np.dot(self.W_y, hidden_states[t]) + self.b_y)



return outputs, hidden_states

Backpropagation Through Time (BPTT)

Due to the recurrent nature of RNNs, backpropagation becomes more challenging because the weight gradients need to span further back through all the previous timesteps. This gives rise to the issue of vanishing gradients which LSTMS and GRUs address.

Cross-entropy loss

When training neural networks, we must define a loss function that measures the accuracy of the network predictions. Since the output at each time step is a vector of size K, for K classes, we will use the multi-class Cross-Entropy loss function.

Loss for a single timestep

Note that the cross-entropy loss is not a distance function since L(y, yhat) is not equal to L(yhat, y). This loss function computes the loss for a single timestep t, more specifically, it computes the loss between y_{t} and yhat_{t}

def cross_entropy_loss(self, y, yhat):

return np.sum(y * np.log(yhat))

Since the network gives an input at each time step, the loss L_t is calculated at each time step. Therefore, the total Loss for a forward pass is the sum of the losses.

def calculate_total_loss(self, x, y):



L_total = 0



for t in range(len(y)): # from t=0 to T

outputs, hidden_states = self.forward_propagation(x[t])

yhat = outputs[np.arange(len(y[i])), y[i]]

L += self.cross_entropy_loss(y, yhat)

Now, we need to calculate the gradients of the weights to see how their changes affect the loss, then we can update the weights by pushing them slightly towards their target values.

Why Cross-Entropy loss instead of MSE or some other loss?

Sometimes people use softmax loss interchangeably with cross-entropy loss which is technically incorrect. The softmax function when applied as an activation function outputs a categorical distribution over the outputs where the sum of these output probabilities equals to 1.

The cross-entropy loss allows you to compute the loss when the softmax function is applied to the output layer. It allows us to measure the error at the softmax layer. We can measure how well we are doing by simply comparing the target vector to the output vector using the cross-entropy loss function.

Predictions (black); Targets(red)

In this vector, assuming non-zero based indexing, the correct class is i = 3, or the third term. The RNN predicted the class i=3 with a certainty of 0.343.

Each i represents a distinct class over N classes. Notice that for classes that have a target of 0, the entire term for the ith class is 0 and for target_i = 1, y_i*log(yhat_i) = 1 * log(yhat_i). Since the values in the target are one-hot encoded, they contain a lot of zeros. Make sure that the targets are in the right position and that you are only taking the log of predictions since for any target_i =0, log(0) = undefined.

In practice, we always add some very small epsilon value to each of the logits in order to avoid an infinite loss, which also implies that we can never get an exact zero-valued loss.

This epsilon term prevents division by 0.

Gradients

To prevent redundancy, I will forego the details of the gradient computations. If you’d like to see the math, check this out. It goes through all the steps to derive the gradients of an RNN.

def backward_propagation_through_time(self, x, y): # from forward pass

outputs = self.outputs

hidden_states = self.hidden_states # accumulate gradients here

dLdX = np.zeros(self.W_x.shape)

dLdH = np.zeros(self.W_h.shape)

dLdY = np.zeros(self.W_y.shape) d_out = outputs

d_out[np.arange(len(y)), y] -= 1 # y_hat - y # backward propagation starting from T = len(y)

for t in range(len(y)-1, -1, -1): # T-1 to 0

dLdY += np.outer(d_out[t], hidden_states[t].T)

dt = np.dot(self.W_y, d_out[t])*(1-hidden_states[t]**2) # from current timestep to 0

for t_i in range(t, -1, -1):

dLdX += np.outer(dt, hidden_states[t_i-1])

dldX[:, x[t_i]] += dt # update for next step t_i-1

dt = np.dot(self.W_h, delta_t)*(1-hidden_states[t_i-1]**2)



return dLdX, dLdH, dLdY # gradients

Weight Updates

Given a learning rate of alpha, we’ll update the weights using this formula.

def update_weights(self, dLdX, dLdH, dLdY):

self.W_x -= self.learning_rate * dLdX

self.W_h -= self.learning_rate * dLdH

self.W_y -= self.learning_rate * dLdY

Truncated Backpropagation Through Time (TBPTT)

Truncated BPTT keeps the computational benefits of Backpropagation Through Time (BPTT) while relieving the need for a complete backtrack through the whole data sequence at every step.

BPTT can be computationally expensive as the number of timesteps increases. Performing backpropagation requires saving all inputs and outputs during the forward pass. In language models, especially character-level models, the input sequences can grow exceptionally long, since a single character instead of a single word is considered one input at each timestep.

If input sequences are comprised of thousands of timesteps, then the number of derivatives required for a single update weight update will be greater than a thousand. Performing a thousand multiplication operations can cause weights to vanish or explode (go to zero or overflow), resulting in slow learning and a noisy model.

In practice, a naive solution is to separate lengthy sequences into batches, then perform backpropagation normally, dropping the gradients after we traverse all the way back. During the forward pass, we bypass resetting the hidden state until the entire sequence across all batches has been processed. In other words, we keep the same hidden state for entire original sequence.

Truncated Backpropagation Through Time modifies BPTT by considering two main parameters: k1 and k2.

k1: the number of timesteps for the forward pass between weight updates, not including the backward pass. This parameter influences the training time.

the number of timesteps for the forward pass between weight updates, not including the backward pass. This parameter influences the training time. k2: the number of timesteps for the backward pass. This parameter can be adjusted smaller to address vanishing gradients so that the gradients don’t need to span as far back. However, it should be large enough so that it can capture the temporal structure of the sequence.

Algorithm

k1 timesteps to predict→ k2 timesteps for error and gradient calculations → update weights → rinse and repeat

Vanishing Gradients

Training RNNs depends on the chaining of derivatives, resulting in difficulties learning long term dependencies. If we have a long sentence such as “The brown and black dog, which was playing with the cat and the mouse, was a German Shepherd”, the network will have difficulty learning to predict the word ‘Shepherd’, which depends on brown and black dog. This is due to the gradient calculation of W_x.

For this 18-word sentence, ‘Shepherd’ is the 18th word. The gradient dL_18/dy_18 needs to chain all the way back to W_x. If we simply unroll the hidden gradient dh_15/ dh_2, which is the word brown (word 2) and the word Shepherd (word 15), we’ll notice the many multiplication operations between terms.

Multiplications of values between 0 and 1 quickly decrease to 0 so the gradient will approach 0 quickly or will “vanish”. This makes the network not learn because if the gradient is 0, then the weights will never update. This is a major disadvantage of RNNs. However, GRUs and LSTMs address these issues.

Conclusion

In this article, we introduced the RNN along with code in numpy. We looked into the mathematicals behind training an RNN in both the forward and backward propagation phases. Issues surrounding RNNs such as vanishing and exploding gradients were discussed. We also looked at how RNNs are trained in practice using Truncated BPTT and the advantages of this algorithm.

In the next article, we use an RNN to decode a cipher and explore the entire pipeline from start to finish, including the preprocessing and postprocessing steps for the sequence.

Next: Simple RNN Deciphers Code

I hope this article was informative. Please let me know if there are any errors or if there are any improvements that can be made.

References