# Optimizing Weighted Fair Queuing with Deep Reinforcement Learning for Dynamic Bandwidth Allocation
## 4 Proposed Model
### WFQ Continual-DRL Algorithm
Each range is run over 1000 episodes of network training, and random values for the new \(\lambda_{EWC}\) are sampled and evaluated. The performance of each sample is estimated based on the measure of the loss function and reward performance. When the random function had been implemented, \(\lambda_{EWC}\) was obtained based on six ranges, for each range created five values (random value), which were operated over 1000 episodes. The model then iteratively trains the SAC with EWC agent for each \(\lambda_{EWC}\), which evaluates the agent's performance. The loop stores tracking of the best-parameter \(\lambda_{EWC}\), then updates the best \(\lambda_{EWC}\) by comparing with performance whenever a higher-performing result is found. This procedure enables observed tuning of the \(\lambda_{EWC}\) parameter to optimize learning stability and policy performance. Then, from each range, pick the best value of \(\lambda_{EWC}\). In the second step, after all six ranges have been explored, obtain the optimal six \(\lambda_{EWC}\) value by conducting a structural evaluation in which a grid search process is applied to systematically compare the performance of the six optimal \(\lambda_{EWC}\) values that resulted from the random function used in training for 1000 episodes. The best-performing \(\lambda_{EWC}\) is selected based on the impact of the MSE value and the reward performance, ensuring optimal performance in continual learning. The model reached the best performance with a rather high \(\lambda_{EWC}\) value of 2271 for eight queues, 2275 for twelve queues, which led to significantly improved reward results and more stability in MSE and MAE. The high value of \(\lambda_{EWC}\) demonstrates that the model strongly penalizes changes to significant parameters, thus preserving previously learned knowledge more effectively. It reflects the model's dependance on stability over adaptivity, which is valuable in dynamic environments where continual learning is critical. To cover the experimental characteristics of a random search, a combination of these two methods represented by the random function and grid search can control a wide range of possible values; this is followed by a refined grid-method evaluation to determine the best-performing \(\lambda_{EWC}\). The WFQ Continual-DRL Framework: Policy-Critic Update with EWC  
The WFQ environment is designed to model multi-queue scheduling principles. It simulates multiple input traffic flows; each represented as a separate queue that shares a limited total bandwidth. Inter-arrival times and service times use gamma-distributed values that reflect the bursty nature of real network traffic. At every decision step (20 s), the state defined as queue length \(s=\{q=(q_{1},q_{2},\ldots,q_{n})\in N_{n}:q_{i}\leq K_{i}\}\) takes as input to actor network to stabilize learning, and queue states normalize as \(\sum_{i=1}^{n}\left[log_{2}(K_{i}+1)\right]\), ensuring the agent perceives state change on a normalized scale and queues receive an action vector from an actor network \(a_{t}\sim\pi_{\varnothing}(a_{t}|s_{t})\), describing the ratio of total bandwidth allocated to each queue. This allocation determines the service rate for each queue \(\mathbf{\mu}_{i}=w_{i}\times b\), and the number of packets served is calculated based on gamma-distributed service times noting action defined as \(\{a=(w_{1},w_{2},\ldots,\ldots,w_{n})\in\mathbb{R}_{+}^{n}:\sum_{t=1}^{n}w_{i}=1\}\), which account for variability in processing or transmission delays. The served packets are removed from the queue, while newly arrived packets arrive, ensuring a fluid and realistic evolution of the queue states. The reward function contains two forms of penalties: firstly, holding cost \(h_{i}\) for unserved packets remaining in the queue, representing delay, and \(l_{i}\) represents packet lost due to buffer overflow. Two critics \(Q_{\theta_{1}}(s_{k}\,|\,a_{k})\), \(Q_{\theta_{2}}(s_{k}\,|\,a_{k})\) functions are maintained, which are trained to minimize the error between the predicted Q-values and the target Q-value.