# Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests
## 3 Modeling and Parameter Optimization of the Training Model
### Quarter-Vehicle Modeling
These simulations were repeated for a minimal damper current of 0.4 A, a medium damper current of 0.8 A, and a maximum damper current of 1.6 A. MATLAB's _testimate_ function was used to calculate the frequency response from road displacement to the acceleration of the vehicle body depicted in Figure 5 and from the road displacement to the dynamic wheel load shown in Figure 6. Even though transfer functions were only well defined for pure linear systems, we assumed only limited errors because of the close-to-linear system behavior for constant damper currents. For constant damper currents, the input-to-force dynamics described in Section 3.2 did not come into effect. Nevertheless, the conducted investigation was only qualitative and, in contrast to pure linear systems, may have resulted in slightly different results for other excitations. It has to be noted that Figures 5 and 6 only show one excitation, namely, the exponential sweep, 1-30 Hz with a 100 mm/s zero-crossing velocity. Figure 5a shows that the body acceleration on the front left vehicle side had one peak at around 1.5 Hz for the 0.4 A damper current case, which represents the body mass eigenfrequency. Increasing the damper current suppressed this first peak. A second broad peak appeared approximately between 10 and 16 Hz. This peak was only slightly influenced by the different damper currents. Inspecting the second peak in Figure 5b at around 14 Hz in the transfer function from the road profile to wheel load suggests that one part of the broad peak in Figure 5a resulted from the wheel eigenfrequency. Additionally, the inspection of the simulation data showed that, at around 10 Hz, the body and engine mass oscillated in an anti-phased way. Hence, introducing the engine mass in the QVM enhanced the capability of the model to approximate the magnitude of the body acceleration, especially at higher frequencies. Even though the simple QVM approximated the height of the first peak for the 0.4 A damper current better, the engine QVM also met the eigenfrequency of the body mass. Looking at the 0.8 A and 1.6 A cases from Figure 5a exhibits a similar approximation capability for both the simple and the engine QVM model structure in the low-frequency range. For frequencies above 5 Hz, the engine QVM approximated the magnitude much better. Despite minor exceptions, Figure 5b shows that the engine QVM approximated the wheel load more accurately than the simple QVM. Figure 6a reveals that the broad peak between 10 and 16 Hz, which can be observed in Figure 5a, was less pronounced on the rear left side of the vehicle. Since the vehicle's engine was in the front, one cause for the broad peak was not present at the rear of the vehicle. Similar to the front left side, the eigenfrequency of the body mass could be observed as a peak in the magnitude of the body acceleration for the 0.4 A damper current in Figure 6a at around 1.6 Hz. The wheel eigenfrequency of the 0.4 A damper current in Figure 6b can be observed at around 15 Hz. Since the simple QVM model and the topmount QVM only differed in the additional topmount elasticity, the transfer functions of both fitted models were pretty close. Nevertheless, the more complex model structure can approximate the magnitude of the body mass acceleration and the wheel load better than the simple QVM. The advantage is evident for higher frequencies and the magnitude of wheel load depicted in Figure 6b. Figure 5: Comparison of AFM’s frequency response from (**a**) road displacement \(z_{road}\) to the acceleration of the vehicle body \(a_{body}\) for different constant damper currents and (**b**) from road displacement \(z_{road}\) to the dynamic wheel load \(F_{z}\) for different constant damper currents for the front left side of the vehicle. Each subplot visualizes the measurement data, the data obtained from an optimized simple QVM model, as well as the resulting data obtained from the optimized best QVM model structure.