# Optimizing Weighted Fair Queuing with Deep Reinforcement Learning for Dynamic Bandwidth Allocation
## 5 Results and Analysis
### Experiment 1: Eight Queues
Eventually, the reward stabilizes during training, indicating that the controller consistently produces optimal performance rewards, thereby balancing exploration with exploitation while preserving past knowledge. A comparison of the SAC and SAC with EWC rewards in Table 2, the standard SAC algorithm exhibits unsteady and fluctuating performance during training episodes from 50 to 1000. Initial rewards decline from \(-8.891\) to \(-10.699\) by episode 150, with only slight and inconsistent through episode 300. In middle training, rewards initially improve--reaching \(-7.381\) at episode 650--but remain erratic. In the next steps, SAC achieves its best reward of \(-6.999\) at episode 800; however, overall, SAC struggles to maintain stable and high rewards over time, reflecting challenges in long-term policy retention. The SAC with EWC consistently exceeds standard SAC across all training steps. At the same time, SAC shows unstable and suboptimal reward trends. The SAC with EWC maintains stable and significantly higher rewards, ranging nearly from \(-2.266\) to \(-2\). The EWC-enhanced model exhibits better learning and improved convergence, especially during the middle to late training steps, where SAC rewards remain inconsistent; in contrast, SAC with EWC approach near-optimal values. This highlights the effectiveness of EWC in supporting stable and high-quality policy learning in DRL for the WFQ system over time. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Episode Number** & **SAC Rewards** & **SAC with EWC Rewards** \\ \hline
50 & \(-8.891\) & \(-3.048\) \\ \hline
100 & \(-10.579\) & \(-2.659\) \\ \hline
150 & \(-10.699\) & \(-2.216\) \\ \hline
200 & \(-10.029\) & \(-2.242\) \\ \hline
250 & \(-9.929\) & \(-2.642\) \\ \hline
300 & \(-10.286\) & \(-2.234\) \\ \hline
350 & \(-8.322\) & \(-2.189\) \\ \hline
400 & \(-7.872\) & \(-2.286\) \\ \hline
450 & \(-9.479\) & \(-2.294\) \\ \hline
500 & \(-8.709\) & \(-2.093\) \\ \hline
550 & \(-10.896\) & \(-2.255\) \\ \hline
600 & \(-9.008\) & \(-2.303\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison table of SAC and SAC with EWC rewards. Figure 7: The average reward for eight queues in the WFQ DRL algorithm and the proposed WFQ continual-DRL algorithm. Table 3 presents the MSE and MEA records for every 50 episodes during training for SAC with EWC in the WFQ system, using the WFQ continual-DRL algorithm, which enables an assessment of how well the environment supports adaptive training and learning. The MSE value is 0.612 at episode 50, which rises slightly to 0.639 and exhibits more fluctuations. At the start of training, the agent typically explores different assignment procedures as actions (bandwidth weights) for the queues, resulting in increased fluctuations in the system. That point to the agent explores all possible processes of the action it is learning to reach an action that yields benefits. Then, it decreases to 0.597 at episode 150, indicates that the model was successfully adjusted and improved its predictions. These fluctuations are common in reinforcement learning and reflect the balance between exploration and convergence. It can be observed that the MSE values steadily decrease from 0.547 at episode 200 to 0.445 at episode 600, suggesting steady learning and improvement in the model's predictions. The results demonstrate successful training with continuous improvement in MSE between episodes 650 and 1000, and the model achieves its best performance while maintaining stable operation within these values. The slight increase in the MSE values after episode 900 remains within an acceptable range. The models reflect more complex or bursty traffic patterns that appear in training. Yet, the values of MSE remain within an acceptable range (near 0.47), meaning that the model has reached stability and effectively minimizes prediction error, although it has increased variability. The training process is well optimized, resulting in high accuracy.