## 4 Discussing Correlation between Bus Driving Cycles
To avoid this issue, the base policy provided by the DP should be adapted to actual working conditions. In the present work, the rollout algorithm [31] is used to this aim. Consider the optimal control problem proposed in Equation (1), where the trip duration and disturbance (vehicle speed sequence) is that of one loop of the studied bus line registered beforehand. Call the registered cycle \(ref\). The DP solution for this particular problem can be obtained offline, as explained in Section 3, and provides the optimal cost-to-go from any state \(x_{j}\) at time-step \(k\) denoted by \(\mathcal{J}^{*}(x_{j},k)_{ref}\) for this specific cycle. Since the magnitude conserved between loops is the distance, a space-based optimal cost-to-go can be built by transforming time to space basis by integration of the vehicle speed registered  
Figure 6: Representation of the \(19\) modes obtained by SVD of the SAPDs shown in Figure 5 and amount of the variance or energy in the measured SAPDs captured depending on the number of modes considered. during cycle \(ref\). In this sense, a matrix \((\mathcal{J}^{*}(x_{j},s_{i})_{ref})\) mapping the optimal cost-to-go in the reference cycle at a given vehicle position in the loop (\(s_{i}\)), depending on its state (\(SoE\), \(ICE_{on}\)), can be built from the DP solution. Taking into account that the complete driving cycle consists of a sequence of similar loops, \(\mathcal{J}^{*}(x_{j},s_{i})_{ref}\) is an approximation of the optimal cost-to-go from any state \(x_{j}\) in a position \(s_{i}\) in any of the loops that can be used as a base policy. Recalling the HJB Equation (16), the one-step look-ahead rollout algorithm proposes to replace the optimal cost-to-go from the next state by the base policy, then:  
\[u_{k}=argmin_{u}\Big{\{}L(x_{k},d_{k},u)+\mathcal{J}^{*}(f_{k}(x_{k},u),s_{k+1 })_{ref}\Big{\}} \tag{20}\]  
where \(f_{k}\) is the discrete version of the state function \(f\) in Equation (15) and \(\mathcal{J}^{*}(f_{k}(x_{k},u),s_{k+1})_{ref}\) is an approximation of the optimal cost-to-go from the state in the next time step, resulting after applying action \(u\), that is obtained from the DP solution of the reference cycle. Even though the proposed algorithm provides a suboptimal solution due to the fact that \(\mathcal{J}^{*}(x_{j},s_{i})_{ref}\) is only an approximation of \(\mathcal{J}^{*}(x_{j},k)\), their difference tends to vanish as the estimated driving cycle (\(ref\)) approaches the real one. The inspection of Equation (20) shows that the proposed rollout algorithm has two main properties:  
* The obtained policy always improves the base policy since it uses information from the actual cycle in the next time-step (\(d_{k}\)) to choose \(u_{k}\). * The algorithm is able to consider the feedback on the current system state to adapt the control policy, for instance avoiding battery SoE excursions outside the desired range by the evaluation of \(L\). Figure 7 shows the diagram of the proposed algorithm particularized for the case study. The left side shows the offline part of the algorithm, consisting of solving by DP a previous driving cycle of the bus in the same route. In the case at hand, a single loop previously recorded is considered as the reference cycle. The cost-to-go of the reference cycle is obtained by DP. Then, the cost-to-go is stored in a map \((\mathcal{J}^{*}(SoE,I\tilde{C}E_{on},\tilde{s})_{ref})\) that provides the minimum fuel that the bus consumes in the reference cycle depending on the SoE, state of the ICE (on or off) and vehicle distance. In this sense, \(SoE\), \(I\tilde{C}E_{on}\) and \(\tilde{s}\) are vectors containing the possible values of SoE, state of ICE and vehicle distance. On the right side, the algorithm that runs while the vehicle is covering its actual route is described.