## 3 Optimization-Based Strategies
### Data-Driven Strategies
#### 3.3.3 Adaptive Dynamic Programming (ADP)
Similarly, the action network error is defined as the quadratic form of the error between the estimated control and the expected value, which is obtained when the partial derivative of the cost function to the control variable is equal to 0, that is,  
\[\frac{\partial J(x_{t})}{\partial u_{t}}=2B\cdot\hat{v}_{t}+\beta\frac{ \partial\hat{J}(\hat{x}_{t+1})}{\partial\hat{x}_{t+1}}\frac{\partial\hat{x}_ {t+1}}{\partial\hat{u}_{t}}=0 \tag{30}\]  
The gradient-based descent approach is used for weight updating during the back-propagation process, which is:  
\[W_{m}(t+1)=W_{m}(t)-\eta_{m}\cdot\frac{\partial E_{m}}{\partial e_{m}}\frac{ \partial e_{m}}{\partial\hat{x}_{t+1}}\frac{\partial\hat{x}_{t+1}}{\partial W _{m}} \tag{31}\]  
When the cost function error of the adjacent iteration is extremely small, the iteration stops, and the optimal control sequence and optimal performance index function are obtained [110]. Based on the above analysis, the training process of HDP can be briefly summarized as in Algorithm 1. (\(I_{max}\) is the maximum number of iterations for ADP, \(N_{crit}\) is the maximum number of iterations for CNN training, and \(N_{act}\) is the maximum number of iterations for ANN training). To achieve more effective approximation of nonlinear systems, the multi-resolution wavelet neural network (MRWNN) based on multi-resolution theory, Meyer scaling, and wavelet functions can be introduced to construct the critic network, whose activation function is orthogonal in order to achieve faster convergence and higher approximate accuracy. A conventional wavelet neural network (CWNN) using a wavelet function as the activation function in the hidden layer does not need clustering preprocessing on the input data, which can be utilized to construct the action network. Similar to BP-NN or RBF-NN, the adaptation of weights and parameters of both networks can be achieved by error back propagation and a gradient descent algorithm. A neuro-dynamic programming (NDP) method using MRWNN as the critic network and the CWNN as the action network was detailed in [111], and it was able to optimize fuel economy for online application and optimize SOC without previewing future traffic information. In addition, a correction model for the output of the action network was used to discretize the continuous gear ratio to the real vehicle gear ratio of the vehicle. For multi-source hybrid electric vehicles (HEVs), the novel concepts of adaptive dynamic programming (ADP) and the progressive optimal search (POS) were proposed in [112]. From the perspectives of trip cost minimization, charge sustenance, and real-time applicability, the advantages of each method were compared with previously developed methods of rule-based strategy and adaptive rule-based strategy. In [113], real-time optimal control of multisource HEVs by ADP was proposed, which was based on drive state recognition according to physics-based parameters. This included the implementation of ADP to obtain optimal EMS and the optimization of vehicle operating conditions for each state, which is optimized offline using the NSGA-II optimization tool. As a consequence, the total cost function decreases in terms of fuel consumption and on-board charge sustaining. Since the ADP algorithm is still being developed, the ADP-based energy management strategy is currently in a stage of development and practical application. Some organizations have realized its experimental application based on a hardware-in-the-loop (HIL) test-rig, and the next step is to realize application in real vehicles. Therefore, the method of the ADP-based strategy for energy management in HEVs has been a research hot spot in universities and the automotive industry. With the rapid development of big data technology, artificial intelligence algorithms and the parallel computing technology of supercomputers, the data-driven energy management control strategy is able to be realized in real-vehicle applications to achieve the lowest fuel consumption in a global scope.