## 3 Optimization-Based Strategies
### Global Information-Driven Optimization
#### 3.2.3 Stochastic Dynamic Programming (SDP)
The value iteration is implemented in the first few steps to obtain a better estimated value and updates the value function based on the Bellman function, and then implements the policy improvement to acquire the optimal strategy. Therefore, the key points of the SDP-based strategy mainly include the speed prediction and the optimal solver, which are shown in Figure 8. In addition, the main steps of the SDP-based strategy in HEVs are as follows:  
1. Establish the discrete-time dynamic systems and determine state variables, the control variables, the optimal objectives and the constraints of the powertrain components (engine, motors, and battery). 2. Establish the Markov chain model of the driver's torque demand. Implement the statistical analysis of the driver's torque demand based on multiple standard driving cycles or historical driving data, and then use the maximum likelihood estimation method to establish a transition probability matrix. 3. Calculate the matrix of the one step reward function and solve the optimization of SDP based on value iteration, policy iteration or modified policy iteration [86]. If the converged condition is satisfied, the iteration will stop. 4. Obtain the optimal operation mode and the power split between engine and motors. The future power demand can be formulated as a discrete-time Markov decision process and can be modeled as a stochastic model established by Markov chain or Monte Carlo method, which reflects the probability distribution of the future power demand and the variation of the future driving cycles. In [87], a novel cost function (incorporating the square of battery charge) with a penalty on high-powered systems was used to lessen the affliction of real-world concerns such as battery health and motor temperature. Furthermore, a Markov chain was augmented with the information regarding SOC transitions to complete a full-state transition probability matrix, and the interpolation was used to distribute each state transition between multiple transition probabilities. Figure 8: Depiction of the main research areas of the SDP-based strategy. Due to the time-invariant characteristics of system dynamics, the optimal EMS can be formulated from the finite horizon and infinite horizon respectively. As for the finite horizon problem, the cost is accumulated in the finite stage, while the infinite horizon problem provides a reasonable approximation of a class of problems, which have a finite range but a very large number [88]. To make the decision-making process independent of driving time, the energy management strategy is solved from the perspective of an infinite horizon in [89]. By considering the statistics of traffic speed profiles, the SDP method based on a modified policy iteration can be adopted to generate a time-invariant state-dependent power split strategy to optimize fuel consumption and charge sustenance of HEVs in a general sense [90]. In [26], a pre-optimization based on the basic operating modes was carried out to obtain the optimal decisions in advance, followed by obtaining the optimal combination of the operating modes and the power split between engine and motor based on SDP. In the SDP algorithm, a homogenous Markov chain in a finite horizon is used to model the driver demand, whose stochastic transition probability matrix is constructed from 15 driving cycles. In the SDP algorithm, the discount factor can reduce the contribution of the exponentially increasing cost function in the future, and has a significant influence on ensuring the convergence of the cost function. Therefore, it is necessary to consider the impact of different penalty factors on the performance of the optimal control strategy, which is mainly reflected in the SOC domain, the deviation from initial SOC, and the fuel economy. According to the effects of difficult penalty factors on the optimal results, the most suitable value was chosen as 8 to balance fuel economy and battery performance in [26].