# Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests
## 5 Control System Verification  
After the training and selection of the RL agent described in Section 4, the obtained agent then needed to be deployed on the real vehicle. Nevertheless, an in-depth verification of the trained RL agent and the whole control system was favorable to support a smooth transition to the real vehicle. The aim of the verification process comprised three objectives: (1) check the time domain behavior of the ANN-based RL agent, (2) evaluate the interaction of the agent with the remaining part of the control system, and (3) assess the agent's behavior on a full vehicle.  
The policy obtained during the RL training was, in our case, an ANN-based mapping from the observation space to the input space. In contrast to analytical control laws, it was hard to derive a priori properties for the trained ANN-based agent. Even though the selected agent performed well on the scalar metrics defined within the agent performance assessment described in Section 4.2.4, it was still possible that the agent exhibited undesired behavior in the time domain. If the assessment metrics defined in Section 4.2.4. did not cover all possible undesired behaviors, the agent might have learned a policy that was able to maximize the reward and perform well across all performance assessment metrics but still show undesirable behavior. Often, such undesirable policy patterns can be identified by evaluating the time domain signals generated by the agent.  
The second reason to validate the agent within a full-vehicle model (FVM) setting is that the agent will be deployed in an entire control system, in which the RL policy is only one component. In addition to the agent, the control system was comprised of different modules, of which some will be described in Section 5.2. During the verification process, the whole control system was integrated into the FVM simulation. Thus, the interaction between different subcomponents as well as the implementation of their interfaces could be checked.  
The last cause to perform the evaluation simulation with the whole control system integrated into an FVM is the approach from Section 4 to train the controller on QVM models. To check the behavior of the agent applied to the FVM, the following controller verification simulations were conducted.  
It has to be noted that we did not conduct a quantitative performance analysis on the FVM. Since we were interested in the performance of the trained controller on the real vehicle, the verification simulations were just a preliminary investigation to enable real-world tests. Even though the FVM described in Section 5.1 was developed to fit the real-world dynamics as well as possible, each model was a just a replica of the real world and might not have been able to fully capture the underlying real-world dynamics. Therefore, we will demonstrate the performance analysis on the data obtained from the real-world experiments described in Section 6.  
The remainder of this section is organized as follows: The full-vehicle model of the AFM demonstrator is presented in Section 5.1. The entire vertical dynamics control system composed of different submodules is then discussed in Section 5.2. The design and optimization of the benchmark controller is briefly discussed in Section 5.3. The integration of the developed control system into the simulation environment, which was later executed on the RCP platform in the vehicle, is described in Section 5.4. Verification simulations conclude this section in Section 5.5.