# Vibration Control with Reinforcement Learning Based on Multi-Reward Lightweight Networks
## 4 Experimental Results and Discussion
### Relevant Comparative Experiments
#### 4.6.2 Relevant Comparative Experiments for Prioritized Experience Replay  
In Section 4.2, we emphasized the superiority of prioritized experience replay under the DRSN and TD3 reinforcement learning frameworks. To further demonstrate its generalizability, we conducted multiple experiments. Figure 13a-f demonstrate the enhanced data utilization efficiency by prioritized experience replay under both fully connected and DRSN conditions. Both (a) and (b) adopt the DDPG framework under the DRSN. In (a), the number of rounds taken to reach convergence under prioritized experience replay is around 350, while in (b), convergence is achieved at approximately 700 rounds. Both (c) and (d) utilize the TD3 framework under the fully connected network. In (c), the number of rounds taken to achieve convergence under prioritized experience replay is approximately 350, while in (f), convergence is not reached even after 1750 rounds. Both (e) and (f) adopt the DDPG framework under the fully connected network. In (e), the number of rounds taken to reach convergence under prioritized experience replay is approximately 400, with a vibration reduction of 9.25 dB, while in (f), convergence is achieved at approximately 600 rounds, with a vibration reduction of 3.53 dB. Both the DDPG and TD3 reinforcement learning algorithms exhibit effectiveness, highlighting that prioritized experience replay is not constrained by specific neural network architectures or reinforcement learning frameworks. It fundamentally enhances data utilization efficiency, proving its generalizability. While Figure 13a-f differ in the network structures, Figure 13 showcases more pronounced control effects, indirectly validating the feasibility of the DRSN method.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Actor** & **Critic** & **RL** & **Parameter (N)** & **Reduction (dB)** \\ \hline FC\({}_{256}\) & FC\({}_{256}\) & DDPG & 338437 & 12.722 \\ DRSN & FC\({}_{256}\) & DDPG & 191545 & 13.102 \\ FC\({}_{256}\) & DRSN & DDPG & 191961 & 14.727 \\ DRSN & DRSN & DDPG & 45069 & 12.966 \\ FC\({}_{256}\) & FC\({}_{256}\) & TD3 & 507398 & 14.276 \\ DRSN & FC\({}_{256}\) & TD3 & 360506 & 16.196 \\ FC\({}_{256}\) & DRSN & TD3 & 214446 & 13.014 \\ DRSN & DRSN & TD3 & 67554 & 20.240 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The impact of Actor and Critic networks as well as the reinforcement learning framework in the lightweighting comparative experiments. By controlling the Actor network, Critic network, RL framework, and parameter count, it was observed that in both the DDPG and TD3 frameworks, the influence of the Critic network on control effectiveness outweighs that of the Actor network.  
Figure 13: **(a) Prioritized experience replay with the DRSN under the DDPG framework. (b) Non-prioritized experience replay with the DRSN under the DDPG framework. (c) Prioritized experience replay with the fully connected network under the TD3 framework. (d) Non-prioritized experience replay with fully connected network under the TD3 framework. (e) Prioritized experience replay with the fully connected network under the DDPG framework. (f) Non-prioritized experience replay with the fully connected network under the DDPG framework.**