## 5 Control-Performance Verification  
In order to further verify the effectiveness of the simulator established with the FIR filter, we carried out experiments based on another simulator that was established with dynamics equations. We set up the dynamics equations in the simulation software and trained the proposed RL-based controller, which interacted with the dynamics equations to find the optimal control policy, which was then used in the experiments.  
The vibration system shown in Figure 22 is simplified as a spring-mass-damper system, which can be formulated by ordinary differential equations. The spring-mass-damper system has three degrees of freedom, including vertical motion along the Z-axis, rotation motion along the X-axis, and rotation motion along the Y-axis. The symbol \(F_{cl}(i=1,2,34)\) stands for the control force of the actuator, and \(F_{bi}(i=1,2,34)\) stands for the force of the rubber isolator. The symbol \(F\) denotes the force of the exciter, and its location is determined by the coordinates \((x,y)\). The dynamics equations are obtained at the equilibrium position:  
\[M\ddot{z}=F+\sum_{i=1}^{4}(F_{cl}+F_{bi}) \tag{21}\]  
\[I_{xx}\ddot{\theta}=(F_{c2}+F_{c3})L_{1}-(F_{c1}+F_{c4})L_{2}+F\cdot y \tag{22}\]  
\[I_{yy}\ddot{\gamma}=(F_{c1}+F_{c2})L_{3}-(F_{c3}+F_{c4})L_{4}-F\cdot x \tag{23}\]  
where \(z\) denotes the vertical displacement of the mass center; \(\theta\) denotes the rotation angle along the X-axis; \(\gamma\) denotes the rotation angle along the Y-axis; \(M\) denotes the mass; \(I_{xx}\) and \(I_{yy}\) denote the inertias.  
Next, the proposed RL-based controller that interacts with the dynamics equations was trained to find the optimal control policy. Then, the control policy was compiled into binary code and run on the controller hardware in the experiment. We performed the experiments by giving different signals to the vibration exciter. The controlled sensor signals were recorded when the controller was on, and the uncontrolled sensor signals were recorded when the controller was off.  
Figure 22: Dynamic model.  
Figure 23 shows the sensor signals when a simple 55 Hz sine signal was given to the exciter. The sampling frequency was 1 kHz. The amount of the total sampled data was more than 30 s. For clarity, a small amount of data is shown. Table 9 shows the RMS values of the uncontrolled and controlled sensor signals. It was found that the RMS values were increased by more than 53%. The vibration enhancement can also be seen from the PSD curves of the sensor signals, as shown in Figure 24. Table 10 shows the peak values of the PSD curves of the sensor signals. It can be seen that the peak values of the PSD curves at specific frequencies were increased by more than 140%.  
Figure 23: Sensor signals obtained by experiments (1 mV represents 0.1 m/s2).  
Figure 24: PSDs of sensor signals obtained by experiments.  
It was found that the control policy obtained by using the dynamics equations led to the vibration enhancement. In comparison with the control policy obtained by using the FIR filter, which effectively reduces the vibration, the effectiveness of the FIR filter could be verified.  
The main reason for the vibration enhancement may be that the dynamics equations can only be used to effectively solve vibration-control problems of low dimension and simple dynamics. The parameters of the dynamic model were carefully verified, such as the mass, stiffness, damping, size, and so on, but it is still much less accurate than the FIR filter. Thus, it is difficult for the RL-based controller to successfully learn an effective control policy from the dynamics equations. To solve this problem, we present an RL-based controller that interacts with the accurate parametric model (FIR filter) instead of dynamics equations. Therefore, the RL-based controller converges more easily, and the vibration can be effectively reduced.