## 3 Results
### Analysis and Synthesis
#### 3.1.5 LB EMSs  
Reinforcement learning (RL) mimics the learning process in the human brain. It maximizes the reward by interacting with and exploring the environment. RL algorithms are model-free and learn the optimal EMS solution in a trial-and-error manner. In many cases, human expertise can provide optimal training samples or preferences for the learning agent to guide exploration in the training process. There also exist large amounts of prior knowledge in the energy management of HEVs, some of which can be applied to learning-based EMSs to improve their sampling efficiency. These methods are concluded as the main future trend in the EMSs [10]. In [47], RL was used as EMS for super-mild HEVs. The proposed RL-based EMS was adopted to obtain the optimum control policy that takes the vehicle speed, driver's power demand and state of charge (SOC) as the input and the engine power as the output. The results showed that, compared with DP, this method could not only be adapted to random driving cycles and reduce fuel consumption, but it could also be implemented online because of its small calculation volume. Learning control algorithms can also be combined with other control strategies, as was proposed in [48]. The proposed EMS combined batch-wise iterative learning control (ILC) and time-wise model predictive control (MPC). ILC theory is a powerful tool for dynamical systems with repetitive operation due to its ability to adjust the control input from batch to batch. The ILC technique can be combined with a predictive control technique: an MPC utilizes the repetitive nature of the batch operation and performs batch-wise feedback together with real-time predictive control. A combination of MPC and ILC was proposed to not only speed up the response time but also effectively reduce the speed ripples. The proposed EMS was validated by comparing it with DP and an RB EMS, showing a performance that approached that of DP. Moreover, it was verified for robustness and real-time processing capacity. In [46], a novel model-free algorithm, named Dyna-H, was constructed to overcome the difficulties related to the easy online application of a previously studied RL-based control based on Q-learning. To validate the effectiveness of the proposed Dyna-H-based EMS, the DP-based results were used as a benchmark. Compared with DP, although the proposed EMS results were suboptimal, the computational time indicated that they can be applied in real-time. Following a similar premise, that the optimization and training processes of RL can be very slow and resource-intensive, a new approach to accelerating the learning process is proposed in [49]. The EMS embedded expert knowledge into a deep deterministic policy gradient. By incorporating this prior knowledge, the proposed framework not only accelerated the learning process but also achieved a better fuel economy, thus making the energy management system relatively stable. DP served as the benchmark to validate its performance through simulations. Finally, [PERSON] et al. [50] presented a parametric study on several factors during the RL-based EMS development. The main results showed that learning experience selection could effectively reduce the vehicle fuel consumption; the vehicle fuel consumption reduced as action discretization increased; and increasing the states discretization was detrimental to the fuel consumption.