# Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests
## 3 Modeling and Parameter Optimization of the Training Model
### Quarter-Vehicle Modeling  
The damper model developed in Section 3.2 as well as additional measurements of the full vehicle on the four-post test rig (see Figure 1) built the basis for deriving the training \begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{1}{c}{**Front Axle**} & \multicolumn{2}{c}{**Rear Axle**} \\ \hline \(T_{i,\text{rising}}\) & 3.915 ms & \(T_{r,\text{rising}}\) & 9.654 ms \\ \hline \(d_{i,\text{rising}}\) & 4.5 ms & \(d_{r,\text{rising}}\) & 4.0 ms \\ \hline \(T_{i,\text{falling}}\)* & 2.615 ms & \(T_{r,\text{falling}}\) & 3.459 ms \\ \hline \(d_{i,\text{falling}}\)* & 1.5 ms & \(d_{r,\text{falling}}\) & 1.5 ms \\ \hline \hline \end{tabular}  
* Second best parameter option selected after visual inspection.  
\end{table} 
Table 1: Optimized damper input-to-force transfer function parameters. Figure 4: Input-to-force dynamics for different current steps and different damper velocities with (**a**) a rising current step and (**b**) a falling current step. The variables are depicted as fraction of their start or end value over time. In addition to the measurement data, the input signal and the result of the fit are plotted. model. This section describes the derivation of the QVM structures, the optimization-based parametrization, and the evaluation of the training models. The four-post test rig was used to generate an extensive vehicle motion dataset subject to vertical excitations. In contrast to measuring while driving on a real road, the vehicle can be excited with predefined vertical excitations. Using the four-post test rig also yielded the advantage that additional measurements, such as the dynamic wheel loads, were available. Moreover, the measurements were reproducible and were not affected by external environmental factors, such as the weather. The vehicle was excited with three different types of sine sweeps to cover a wide range of frequencies: sine sweeps with linear increasing frequency from 0.5 Hz to 5 Hz, from 1 Hz to 20 Hz, and an exponentially increasing frequency sweep from 1 Hz to 30 Hz. Each frequency range was applied with different zero-crossing velocities of 50 mm/s, 100 mm/s, 150 mm/s, 200 mm/s, and 250 mm/s. Additionally, each sweep excitation was performed with different constant damper currents of 0.4 A, 0.6 A, 0.8 A, 1.0A, and 1.6 A. Since the main focus of the training model was to approximate the vehicle's vertical dynamics and not the pitch or roll motion of the vehicle, only synchronous post excitations were selected for the modeling process. We used the following sensor signals for the optimization-based parametrization of the vehicle: position of each post, dynamic wheel load of each wheel, acceleration of each wheel carrier, and acceleration of the chassis at the front right, front left, and rear left sides of the vehicle. The chassis acceleration on the rear right side was not included in the standard sensor setup on the four-post test rig and, thus, not recorded. Different quarter-vehicle model structures were considered during the modeling process to synthesize an accurate model of the vehicle (see also [20] and Table 2). All QVM structures shared the following elements: a linear spring/damper element as approximation of the tire vertical dynamics, the damper model from Section 3.2, and two mass elements for both the wheel carrier and the body, respectively. Apart from these shared elements, the QVM structures differed in the elements listed in Table 2.  
Both the vehicle spring and damper exhibited slightly nonlinear kinematics with respect to the vertical deflection between the body and wheel. This implies that a vertical deflection between the body and wheel \(l_{\text{bw}}\) resulted in a nonlinear deflection at the spring \(l_{\text{s}}\) or damper \(l_{\text{d}}\). In the model variants, which included a nonlinear spring or damper transmission, the transmission ratio was defined as  
\[i_{j}=\frac{\partial l_{j}}{\partial l_{\text{bw}}} \tag{2}\]  
with \(j\in\{\text{s},\text{d}\}\).