# Optimizing Weighted Fair Queuing with Deep Reinforcement Learning for Dynamic Bandwidth Allocation
## 5 Results and Analysis
### Experiment 1: Eight Queues
After 200 episodes, the curve of SAC decreased to a stable range between 0.07 and 0.09. According to the curve for SAC, the EWC declines after 200 episodes, stabilizing between 0.05 and 0.07. The enhanced SAC with EWC exhibits higher learning performance, achieving a lower and more stable RMSE compared to the SAC. This suggests that EWC helps mitigate catastrophic forgetting learning, especially in complex or dynamic environments of the WFQ system. The curve of enhancement appeared to decrease in RMSE, reflecting improved steadiness and adaptability in decision-making. Lower RMSE indicates better accuracy in predicting actions in dynamic queue management (resource allocation) with respect to optimal resource allocation computed in the Fairness function. As shown in Figure 11, the comparison of the [PERSON]'s index between SAC and SAC with EWC reveals that the results show the curve of SAC after a 200-episode rise and steady between 0.6 and 0.7, while the curve of SAC with EWC remains stable after a 200-episode rise and remains steady between 0.7 and 0.8. Whenever the [PERSON]'s Index value is close to 1, it indicates excellent fairness, meaning better exploitation of resources. Complexity Analysis  
The computation and comparison of the time complexity of SAC were implemented in the WFQ-DRL algorithm [22], and SAC with EWC was implemented in the WFQ continual  
Figure 11: The comparison table of Jain’s Index for eight queues. Figure 10: The comparison of the Root Mean Square Error (RMSE) of eight queues. DRL algorithm in this paper. Analysis of the execution time for both average step and overhead represented by periodic FIM in DRL as a continual process, providing insight into the tradeoff between learning stability of performance model and computational cost. For policy (\(\pi\)) network structure:  
Input layer 8 \(\rightarrow\) 64  
Hidden layer 64 \(\rightarrow\) 64  
Two output layers (\(\mu\), \(\sigma\)) 64 \(\rightarrow\) 8 for each layer  
Calculating the parameter (P) for the policy (\(\pi\)) structure:  
For the first fully connected layer O (8 \(\times\) 64) \(\rightarrow\) O (512)  
Second fully connected layer O (64 \(\times\) 64) \(\rightarrow\) O (4096)  
Output fully connected layer 2 \(\times\) (64 \(\times\) 8) \(\rightarrow\) O (1024)  
Total P for policy network (\(\pi\)): O (512 + 4096 + 1024) = O (5632)  
Complexity model:  
For policy network (\(\pi\)) is O (1) because the dimension is constant. For the critic network (Q) structure:  
Input layer (8 + 8) \(\rightarrow\) 16  
Hidden layer 64 \(\rightarrow\) 64  
The output layer is a 64 \(\rightarrow\) 1 output layer in a neural network  
Calculating the parameter (P) for the policy (\(\pi\)) structure:  
For the first fully connected layer O (16 \(\times\) 64) \(\rightarrow\) O (1024)  
Second fully connected layer O (64 \(\times\) 64) \(\rightarrow\) O (4096)  
Output fully connected layer (64 \(\times\) 1) \(\rightarrow\) O (64)  
Total (P) for one critic network (\(\pi\)): O (1024 + 4096 + 64) = O (5184)  
Total P for both critic networks (Q1, Q2) and policy network (\(\pi\))  
5632 + 2 \(\times\) 5148= P (16,000)  
Since the network dimension remains constant, the per-step computational complexity remains O (1). Over the course of N training steps (100 steps per episode), the total complexity linearity is O(N); although, for WFQ continual-DRL using SAC with EWC step cost remains O (1), an additional overhead of O(P) arises for 30 episodes due to FIM, highlighting the computational trade-off between standard DRL algorithm and enhancement algorithm introduced by continual learning. For the complexity model:  
For critic network (Q): O (1), because the dimension is constant. Overall complexity for N training network steps:  
Total time complexity O (N \(\times\) 1) = O(N). One episode has 100 steps, which is O (100), where \(N\) = 100. For 1000 episodes (100,000) where N = 100,000, 100 steps for each episode. Where FIM computes every 30 episodes, 100 steps per episode, P = 16,000 parameters.