# A Framework for User Traffic Prediction and Resource Allocation in 5G Networks
## 4 Methodology
### User Traffic Prediction Module
#### 4.1.1 Long Short-Term Memory Variant  
LSTM networks can effectively capture temporal relationships in time-series data, which is essential for prediction problems, such as the one explored in this work. As depicted in Figure 2, the LSTM neural network comprises two LSTM layers, one with 256 units that performs the initial hierarchical feature extraction and a second with 128 units that captures the higher-level temporal patterns. These layers are followed by a dense (fully-connected) layer of 128 units with an activation Rectified Linear Unit (ReLU) activation layer. The final layer is again a fully connected output layer of 60 units representing the next 60 predicted time steps. The first LSTM layer is used to identify immediate sequential relationships such as traffic fluctuations and seasonal variations. The second LSTM layer then operates on those relationships, identifying patterns and higher-order temporal relations [28]. This hierarchical process allows the architecture to capture more than one order of temporal patterns, which is particularly useful in traffic prediction where both immediate changes and long-term patterns can be found in the data. The input is a set of 120 time steps with 5 features each: (i) the actual value of user traffic of that time step, (ii) the rolling mean of the last 10 time steps, (iii) a rolling standard deviation of the last 10 time steps, (iv) a first-order difference that represents the momentum of the user traffic, and finally (v) the percentage change of the current time step compared to the first of the rolling window. \begin{table}
\begin{tabular}{c c} \hline \hline
**Input Feature** & **Value** \\ \hline Second\_Derivative & \(-0.000301\pm 0.000045\) \\ First\_Derivative & \(-0.000205\pm 0.000027\) \\ Original\_Values & \(-0.000045\pm 0.000003\) \\ Rolling\_Mean & \(-0.000003\pm 0.000001\) \\ Rolling\_Std & \(-0.000002\pm 0.000001\) \\ \hline \hline \end{tabular} The ranking of the input features of our user traffic prediction models based on their importance. The mean values for the two architectures, namely, LSTM and Transformer-TCN, are reported.  
\end{table} 
Table 1: Feature Importance Ranking. A custom solution that combines the loss from mean squared error, a trend direction loss calculation to assist the consistency of the directionality of the time series, and a volatility-based loss to reduce patterns of statistical dispersion was implemented as the loss function. This combined loss strategy improves consistency by taking into account the volatility of the data as well as the magnitude and penalizes model parameters that would just optimize for one or the other. This way we overcome the natural limitation of optimizing for magnitude prediction by just using mean squared error. Adam [29] with decay rate and early stopping was used as the optimizer to exploit the adaptive learning mechanism in the root mean squared error propagation method and the momentum mechanism employed in the gradient descent process. The equation for the custom loss is shown below. With \(L_{t}\) we symbolize total loss, \(L_{m}\) is the loss component from the mean squared error calculation, \(L_{d}\) is the loss component from trend direction loss, and \(L_{v}\) is the loss component from volatility-based loss. With \(w_{x}\) we symbolize the weighting of each loss component, from 0 to 1, and \(x\) is that component, e.g., \(w_{m}\) is the weight of the mean squared error component. In the models demonstrated in this work, we used \(w_{m}=0.3,\ w_{d}=0.3,\ w_{v}=0.4\). \[L_{t}=w_{m}*L_{m}+w_{d}*L_{d}+w_{v}*L_{v}\]  
The mean squared error loss is calculated normally. The trend direction loss is calculated as shown in the equation below. With \(D_{a}\) we symbolize the actual direction of the series and with \(D_{p}\) the predicted direction. With \(M(x,y)\) we symbolize an operator that is 1 if \(x\) and \(y\) are equal and 0 if not.