# Machine-Learning-Based Optimization for Multiple-IRS-Aided Communication System
## 5 Simulation Results
### Performance Analysis for the Dynamic UE Scenario  
In this section, we concentrate on the dynamic UE scenario. Under this case, the random walk of UEs is adopted to imitate the change in locations for UEs. Each UE is assumed to move at speed \(v_{k}\) (\(0\leq v_{k}\leq 1\)) [m/s] and moving angle \(g_{k}\) (\(0\leq g_{k}\leq 2\pi\)), where the UEs are confined in the region defined at the right half circle in front of the IRSs, as in Figure 4.  
The proposed DDP  
hyper-parameters, i.e., the actor and critic learning rates and target smoothing factors, on our designed model in the following. Figure 8 plots the average reward versus the episode steps under different learning rates. The figure shows that the curve with a learning rate of 0.001 can achieve the highest average rewards. The performance gain obtained from the smaller learning rate value comes at the expense of a long time for convergence. In addition, it is worth noting that the learning rate of 0.1 has the worst performance, as, for too high rates, the oscillations increase and degrade the performance. This implies that the learning rate cannot be too low or too high as to compromise performance and convergence. Additionally, Figure 9 plots the optimized transmit power at the source BS versus the episode steps under the different learning rates. The figure shows that the model has the best-optimized transmitted power under the learning rate of 0.001, where the transmitted power decreases over steps and reaches optimal values. Additionally, applying high learning rate values leads to high transmit power values or increases the oscillation values, as in the case of 0.1 rates. Accordingly, we set the learning rates for both the actor and the critic in our model to be 0.001, which ensures long-term rewards and satisfies our objective of minimum transmit power. Figure 8: Average rewards versus episode steps under different learning rates. Figure 7: Received rewards versus episode steps. Another important parameter that should be tuned for the model is the smoothing factor for the target actor and critic parameters updates. Generally, these values need to be set to less than 1 to guarantee smoothness. Figure 10 plots the average rewards versus the episode steps under different smoothing factors. We consider both low- and high-value ranges. It is observed that applying 0.001 as a smoothing value can achieve the best performance compared to other values. Furthermore, we observe that increasing this value reduces the average rewards and worsens the performance. As shown, applying 0.5 as a smoothing value degrades the model's performance and decreases the received rewards. This implies that applying low smoothing factor values to the model is well suited for updating the target values and converging to optimal active and passive beamforming for the designed system. Accordingly, we apply 0.001 as a smoothing factor update for both the actor and the critic. Finally, Figure 11 plots the average total transmit power during algorithm learning episodes with tuned parameters. The average here is calculated as the sum of cumulative values during the total learning episodes. Note that during training, in each episode, the environment is reset to new positions for UEs with new channel gains. It can be shown that the average total transmit power of the BS is decreased with the increasing episodes of  
Figure 10: Average rewards versus episode steps under different target smoothing factors. Figure 9: Total transmit power of BS versus episode steps under different learning rates. training until convergence occurs. This means that the algorithm updates its policy after each episode towards achieving its objective with the minimum transmit power under different positions for UEs with different channel gains. We then compare our proposed DDPG-based algorithm with the considered benchmark schemes in terms of the optimized total transmit power.