# Vibration Control with Reinforcement Learning Based on Multi-Reward Lightweight Networks
## 4 Experimental Results and Discussion
### Comparison between Prioritized Experience Replaying and Non-Prioritized Experience Replaying  
To clearly present the experimental results, we visualized the time-domain signals before and after the control, the frequency-domain signals obtained through Fourier transformation, the action-value signals, and the reward values. Similar experimental results are also summarized here for subsequent experiments. As shown in Figure 9, the first row (a)-(d) displays the time-domain control signals, where the x-axis represents time and the y-axis represents the control signal in volts (V); the second row (e)-(f) illustrates the frequency-domain control signals, with the x-axis representing the frequency range in Hz and the y-axis indicating the values after Fourier transformation (FFT); the third row (i)-(l) represents the action values of the agent; and the fourth row (m) represents the reward values. In the latter two rows, the x-axis represents time and the y-axis represents the magnitude of the action values or the number of rounds for the reward values. Although reward values may be negative, higher values indicate better control effectiveness, albeit not absolutely. These visualizations help demonstrate the performance of the algorithm during the control process and the variations in signals.  
Figure 10 illustrates how the utilization of prioritized experience replay enhances the convergence speed and control effectiveness under both DRSN and TD3 reinforcement learning frameworks. Under narrowband white-noise excitation ranging from 0 to 100 Hz, Figure 10a shows that optimal performance is reached after approximately 2500 iterations, while Figure 10b shows that the same is achieved after only 1300 iterations, showcasing a convergence speed close to twice that of non-experience replay. To validate the effectiveness of prioritized experience replay, we conducted training with and without prioritized experience replay for 1000 episodes, as shown in Figure 10c,d, respectively. Through calculations, Figure 10c demonstrates a vibration reduction of 7.9 dB without prioritized experience replay, while Figure 10d indicates a reduction of up to 12.5 dB with prioritized experience replay. These contrasting experiments reveal that training with prioritized experience replay is more stable and exhibits higher data efficiency.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & **Uncontrolled (V)** & **Controlled (V)** & **Parameter (N)** & **Reduction (dB)** \\ \hline FC & 0.3311 & 0.0762 & 507398 & 12.750 \\ CNN & 0.3311 & 0.0870 & 3102254 & 11.604 \\ GoogleNet & 0.3311 & 0.0812 & 15774938 & 11.894 \\ LSTM & 0.3311 & 0.0875 & 375326 & 11.516 \\ DRSL-MPER & 0.3311 & 0.0328 & 67554 & 20.240 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Lightweight comparison experiment. Under the same experimental conditions, the method proposed in this paper achieves better control effectiveness with fewer parameters. The first row represents the algorithm, the second row displays the vibration signal values before being controlled, the third row shows the vibration signal values after being controlled, the fourth row indicates the parameter quantity, and the fifth row represents the reduction in vibration control, measured in dB.