# Network Traffic Prediction Incorporating Prior Knowledge for an Intelligent Network
## 4 Predictive Models
### Integration of Prior Knowledge Modules  
The LSTM model consists of an input layer, a recursive layer (with memory blocks, instead of traditional neuron nodes, as the basic units), and an output layer. A memory block is a set of cyclically connected subnets, each containing one or more self-connected memory cells and three multiplication units--input gates \(i_{t}\), output gates \(o_{t}\), and forget gates \(f_{t}\)--which perform continuous simulation of write, read, and reset operations for the cells. The main purpose of the LSTM is to model long-term dependencies and to determine the optimal input length through the use of the three multiplication units. The implicit state of the LSTM is a tuple consisting of two states \((c_{i-1},h_{i-1})\). In the initialization state, the tuple is an all-zero tensor, the input tensor at each time is \(x_{t}\), and the output tensor is \(y_{t}\). As shown in Figure 4, the forget gate \(f_{t}\) controls what information needs to be discarded from state \(c_{t-1}\) of the previous moment. Thus, it ignores irrelevant features and automatically determines the best input. The input gate \(i_{t}\) determines the state that the unit needs to update and, therefore, has a long-term memory capability. The output gate \(i_{t}\) filters the output based on the state of the unit. In Figure 4, the LSTM network forward propagation is calculated as follows:  
\[f_{t}=\sigma(W_{if}x_{t}+b_{if}+W_{hf}h_{t-1}+b_{hf}) \tag{6}\]  
\[i_{t}=\sigma(W_{ii}x_{t}+b_{ii}+W_{hi}h_{t-1}+b_{hi}) \tag{7}\]  
\[g_{t}=\tanh(W_{ig}x_{t}+b_{ig}+W_{hg}h_{t-1}+b_{hg}) \tag{8}\]  
\[o_{t}=\sigma(W_{io}x_{t}+b_{io}+W_{ho}h_{t-1}+b_{ho}) \tag{9}\]  
where \(h_{t}\) denotes the output vector; \(w\) denotes the weight matrix before the linear transformation; \(g_{t}\) updates the new information; the symbols \(\otimes\) and \(\oplus\) denote element-level multiplication and element-level concatenation, respectively; \(\sigma\) is the _sigmoid_ function; and \(tanh\) is the hyperbolic tangent function. Figure 3: Schematic diagram of one-dimensional convolution. The traditional LSTM requires the value \(f_{t}\) calculated by the sigmoid function to control the flow of the implied state from the previous step \(c_{i-1}\) into the next step before calculating the next implied state \(c_{i}\). FPK-Net adds the Hurst exponent of the input sequence before the function transformation to improve the forget and input gates of the LSTM. The specific forget and retention values of the LSTM are controlled by the current input and the previous implicit state through the sigmoid function, such that this improved gating mechanism can be considered reasonable. From Equation (6) above, we can see that both the forget gate and input gate finally need to go through the sigmoid function output, and the output takes values between 0 and 1. When the output tends to 1, it indicates a memory state and, vice versa, when it tends to 0, it indicates a forget state. The Hurst exponent reflects the local and overall scale invariance of the traffic sequence; that is, when \(0.5<H<1\), the trend of the traffic sequence in the future time per-iod can be predicted. The larger the Hurst exponent, the higher the degree of self-similarity, which is positively correlated with the operation mechanism of the forget and input gates. Therefore, if the Hurst exponent is added before the linear transformation, it can meaningfully learn the linear transformation weights and, thus, guide traffic sequence prediction.