## 3 LB-MPC in Path Tracking of Mobile Platforms
### Learning and Optimizing for Controller Output under Uncertain Disturbances
#### 3.3.1 Learning and Optimizing Controller Output Based on Reinforcement Learning  
When the mobile platform is driving at a high speed in a known environment or driving in an unknown complex terrain, the MPC controller is unable to slow down the mobile platforms considerably during steering. It is necessary to optimize the output of the MPC controller. The reinforcement learning algorithm evaluates the feedback signal of the environment to improve the action plan and adapt the environment in order to achieve the intended goals. The adaptive MPC path tracking controller based on reinforcement learning is designed to correct the predicted output of the model by interacting with the real environment. The reinforcement learning algorithm is used to adaptively adjust the MPC controller online to realize path tracking in mobile platforms with high robustness on complex terrains.  
As compared with the traditional MPC controller, the adaptive MPC path tracking controller based on reinforcement learning further reduces the system overshoot and oscillations. It optimizes the performance of system dynamics and steady-state error. The hierarchical reinforcement learning method improves the generalization ability of reinforcement learning for optimal path tracking in wheeled mobile robots [117]. The behavior-reward scoring mechanism of reinforcement learning is used to learn the behavior rules so that the unmanned surface ships are able to estimate the best path tracking control behavior [118]. The proximal policy optimization (PPO) method is used as a deep reinforcement learning algorithm and combined with the traditional pure pursuit method to construct the vehicle controller architecture [119]. The blend of such controllers makes the overall system operations more robust, adaptive, and effective in terms of path tracking. Similarly, the PPO method is also used to achieve the trade-off between smooth control and path errors by designing the reward function to consider the smoothness and tracking error [120]. Figure 9 presents the flowchart of PPO control framework. The trained model can be nested in the combined controller to improve the accuracy of path tracking control on the basis of the interactions with the environment.  
On the basis of the aforementioned works, we propose a reinforcement learning-based model predictive control (RLB-MPC) path tracking control framework. Additionally, the output of the controller is optimized on the basis of the reinforcement learning model. First, the predictive path is obtained by controlling the data-driven prediction model in the path-tracking controller. The target path is obtained by using visual and radar sensors. Second, the path deviation generated by the comparison between the predicted path and the target path is considered as the optimization objective. Finally, the path-tracking controller interacts with the real environment on the basis of the reinforcement learning model and corrects the path deviation generated by the comparison between the actual path and the optimized predicted path. The state information that conforms to the internal reward and punishment function of the reinforcement learning model is returned to the reference output and output directly. The other state information is returned to the MPC internal optimization until it meets the requirements of direct output. The RLB-MPC control framework is shown in Figure 10.  
Figure 9: The control framework of PPO [120].