# Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests
## 3 Modeling and Parameter Optimization of the Training Model
### Damper Identification and Modeling  
The test vehicle was equipped with hydraulic triple-tube dampers with an external valve. The damper characteristics could be changed by the positioning of the valve, which was controlled by the magnetic field induced by a current flow through the valve coil. A low-level current controller ensured the tracking of the current setpoint. In this work, we selected the current setpoint of the low-level current controller as the input variable. Since the front axle of the vehicle was constructed as a McPherson strut and the rear axle was constructed as an integral link suspension, different damper configurations were used for the front axle and the rear axle. This implied different characteristics at the front and rear axle and different model parametrizations were needed. In a dedicated damper test rig, several measurements were conducted on one damper from the front axle and one from the rear axle. The damper test rig could apply defined velocity profiles to the damper and measure the induced force by the damper. The following measurements were conducted to obtain the whole damper characteristics and parameters: First of all, a standardized identification process with different constant currents was conducted to obtain the force-velocity map, which is depicted in Figure 3.  
Figure 2(a),b depict the velocity-force maps of the front and rear axle dampers, respectively. The damper characteristics of the front and rear dampers differed significantly. The atypical characteristics of the front axle damper resulted from the conversion of the uncontrolled standard damper to a semi-active one. A quasi-static profile with different breakpoints was applied to measure the friction force as well as the air spring stiffness \(k_{i}\) with \(i\in\{t,r\}\) for the front axle and the rear axle, respectively. This identification resulted in \(k_{\text{f}}=895\) N/m and \(k_{\text{r}}=680\) N/m. To identify the input-to-force dynamics of the dampers, different steps in the current setpoints with constant damper velocity were applied and the induced damper forces were recorded. With this test, the input-to-force dynamics could be identified, which included the dynamics of the low-level current controller, the valve dynamics, and additional dynamics within the damper. The results of these tests for the front axle damper are depicted in Figure 3(a) for a rising current step and in Figure 3(b) for a falling current step. In this measurement setup, we were only interested in the dynamical behavior of the damper. Thus, the depicted measurement data as well as the input signal were scaled between 0 and 1. As can be seen in a comparison of Figures 3(a) and 3(b), the dynamical behavior differed between a rising current step and a falling current step. The dynamics were more responsive for a falling current step. However, only small differences in the dynamical behavior between rebound and compression could be determined. Therefore, our model distinguished between positive and negative current steps but did not represent rebound and compression in the input-to-force dynamics. Figure 3: Damper force–velocity characteristics for different damper currents for AFM’s (**a**) front axle and (**b**) rear axle (compare [20]). Based on this measurement data, the input-to-force dynamics were approximated by a first-order transfer function with an additional delay:  
\[G(s)=\frac{1}{1+T_{i,j}s}\cdot e^{-s\ d_{i,j}} \tag{1}\]  
with \(i\in\{\text{f},\text{r}\}\) for front axle and rear axle and \(j\in\{\text{falling},\text{rising}\}\). The transfer function parameters were obtained through optimization using MATLAB's _tfest_ function in MATLAB 2022b. We performed a grid search over the delay time \(d_{i,j}\) with a spacing of 0.5 ms. Thus, we obtained a time constant \(T_{i,j}\) for each grid point \(d_{i,j}\) by running the optimization. The parameters \(d_{i,j}\) and \(T_{i,j}\) that resulted in the lowest cost function values and were considered plausible were selected.