# Optimizing Weighted Fair Queuing with Deep Reinforcement Learning for Dynamic Bandwidth Allocation
## 5 Results and Analysis
### Experiment 1: Eight Queues  
In this section, we describe an experiment involving eight queues in the form of an arrival flow with arrival rates \(\lambda_{i}=\) (0.03, 0.16, 0.2, 0.1, 0.08, 0.15, 0.06, 0.05) packets/s, where each arriving packet is classified and placed into its respective queue. The WFQ continual-DRL algorithm services queues in a manner that reflects their assigned weights. The interarrival times and service times were generated using a gamma distribution with a Squared Coefficient of Variation (SCV) of 0.8. The average packet size was 1 byte/packet, the buffers of queues were \(K_{1}=K_{2}=\ldots K_{8}=32\)  packets, and the total finite bandwidth was 1 byte per second. The control policy allocated bandwidth weights satisfying \(w_{1}+w_{2}+\ldots\ldots+w_{8}=1\). The number of episodes was 1000, with a time length of 2000 s for each episode. The agent took action every 20 s.  
When the random function had been implemented, \(\lambda_{EWC}\)  was obtained, as explained in Section 2, based on six ranges, each of which was operated over 1000 episodes. Next, we set the six values in the grid to search for optimal values from them. The optimal value of \(\lambda_{EWC}\)  was then obtained from the grid approach process, the FIM was computed every 30 episodes, and then the parameters \(\theta_{i}^{*}\)  and \(F_{i}\) were stored. At the initial implementation step, a multi-queueing system consisting of eight queues was experimentally designed to evaluate the WFQ-DRL algorithm, as presented in [22]. This research aims to employ deep reinforcement learning for the allocation of dynamic bandwidth in router environments using a WFQ mechanism; each queue was modeled with identical arrival rates (\(\lambda\)) and buffer sizes, as detailed in the previous section. The implementation structure adopts a neural network containing policy and critic networks, as illustrated in Figure 3. The key hyperparameters of the network are summarized in Table 1.  
After this, improving the WFQ-DRL algorithm by integrating continual learning allows the model to adapt over time without forgetting previously acquired knowledge. It primarily utilizes the EWC mechanism to preserve important network parameters as the model learns new tasks. The adaptive algorithm of the WFQ continual-DRL enables ongoing learning and adaptation to varying network conditions and objectives while maintaining the same structural configuration and parameter settings across the eight queues. This technique effectively maximizes the cumulative reward over time, ensuring robust performance in dynamic networking environments. The average reward is shown in Figure 7 for the curve of SAC, and it can be seen that the curve exponentially converges after a 200-episode run, and the fluctuation range for average reward is between \(-10\) and \(-6.5\), according to SAC with EWC curve of average reward for enhance algorithm resulting in consistent performance, and it can be seen that the curve exponentially converges after a 100-episode run. This indicates that the model was stable, with a steady range for the average rewards of between \(-2\) and \(-1.5\), resulting in consistent performance. At the beginning of training, the controller typically explores different assignments as action strategies (bandwidth weights) through queues, leading to increased fluctuation. In this step, the controller has the potential to explore all the probabilities of processes to achieve an action that yields benefits. During the learning process, the controller exploits actions, resulting in a rise in the reward curve, and the policy gradually converges toward the optimal allocation bandwidth, reducing the variability in the reward curve. The rise indicates that the policy used in the WFQ continual-DRL algorithm improves performance and provides a suitable understanding of the environment dynamics.