# Throughput Prediction of 5G Network Based on Trace Similarity for Adaptive Video
## 3 Theoretical Background
### Long Short-Term Memory Networks  
Artificial neural networks (ANNs) are a popular machine learning technique due to their ability to learn complex non-linear relationships between multiple variables, making them accurate and versatile. They can serve as universal function approximators, allowing them to learn and solve complex problems. Initially, single-layered ANNs were more popular than deep ANNs due to training difficulties. However, deep ANNs overcame these issues and gradually replaced single-layered ANNs as a solution to more complex problems. Deep ANNs added more complexity and non-linearity to network traffic models, making them useful for time series prediction, including network traffic. As a result, ANNs have become a universal tool for predicting time series data in various fields.  
However, most ANNs have a feed-forward architecture that does not store a trace of previous inputs. They process every input independently and do not memorize information, which is sufficient in many areas, such as computer vision. Nevertheless, as they process the whole input sequence at once, they miss the temporal dependency present in the time series. Hence, Recurrent Neural Networks (RNNs) are better suited for processing time series data. Contrary to feed-forward ANNs, RNNs' connections form an internal loop. Such an architecture allows one to treat sequences by iterating through their elements and to store information about previously processed data. Training Recurrent Neural Networks (RNNs) with a vanilla recurrent unit can be inefficient due to the vanishing gradient problem, which hinders the ability to retain data from earlier computations. This is because RNNs miss information about long-memory dependencies. However, an advanced type of RNN called a Long Short-Term Memory (LSTM) ANN can overcome this shortcoming. LSTMs allow for long-term memory storage in cells with self-connections, which helps to retain information from earlier computations. LSTM has multiplicative gates in each cell. These gates--the input, output, and forget gates--control information flow and can learn to open and close as needed; see Figure 6. This allows memory cells to store data over long periods of time, making them particularly useful in time series prediction with long temporal dependency. Multiple memory blocks form a hidden layer, and an LSTM model typically consists of an input layer, at least one hidden layer, and an output layer. Deep LSTM architectures with multiple hidden layers can create increasingly higher levels of data representations due to their greater capacity. In this architecture, multiple hidden layers are stacked in such a way that the subsequent layer takes the output of the previous LSTM layer as its input; see Figure 7. In this work, we applied such a multilayer architecture.  
Figure 6: Structure of LSTM memory block: \(x_{t}\) is an input signal at the time \(t\); \(c_{t}\) and \(c_{t+1}\) denote cell states at the actual \(t\) and future \(t+1\) times, respectively; \(h_{t}\) and \(h_{t+1}\) are output signals of the cell at \(t\) and \(t+1\) times, respectively; and \(b\) denotes the bias. The symbol \(+\) denotes the summation and \(\times\) the multiplication of signals. The signals \(x_{t}\) and \(h_{t}\) follow the same path but do not interact when entering the cell. The cell output \(z_{t}\) equals the state signal \(h_{t+1}\).