## 1 Introduction
Two of the leading implementations of a Q-learning for real-time control of a HEV have been presented in [20] and [21], in which a transition probability matrix (TPM) has been used to provide the agent with some statistics about the driving mission (DM) to be experienced with a series and a parallel HEV, respectively. In [22], a Q-learning algorithm is applied to the real-time control problem of a range-extended HEV (REHEV) without any a priori information about the DM. Its capability of learning a control policy that reduces the cumulative FC is demonstrated. Similar considerations about Q-learning are made in [23] about the performance of the control agent when applied to a power-split HEV architecture. In this case, a comparison between the best performances achieved at the end of the learning process and those produced by a benchmark DP is carried out. In [24], a wider set of experiments has been carried out testing a Q-learning for a parallel HEV with ICE and MG mounted on two different axles. Specifically, a change in the states and actions is performed about the number and the type of considered variables; moreover, the learning experience of the agent is discussed; finally, a sensitivity analysis over a constant-through-episodes value of the agent exploration rate. Regardless of the different applications and tests conducted over the Q-learning, [20, 21, 22, 23, 24] do not involve a detailed testing plan of the controller aimed at stressing its performances when applied to the optimal control of a hybrid powertrain on real-world driving environments. In fact, a reliable utilization of ML-based techniques has to be achieved by means of detailed analyses focused on the assessment of the agent response when modifications are introduced into its most influencing parameters. Unexpected rather than counter-intuitive behaviors could be theoretically realized, and the results obtained over different driving environments could be overturned. The aim of the present paper is to show the results produced by an extensive test plan of a Q-learning agent for the real-time control of a parallel HEV based on experimental data. Here, the environment and the vehicle model (referred to as \"simulator\") have not been nested inside a single framework, whereas they have been separated from the agent. In the literature, the software architecture developed for implementing RL-basedis not specified. On the contrary, an integrated modular software framework has been developed in the present research to separately manage the dynamics of the agent, the environment and the simulator. Thanks to this approach, a particular analysis of a single module can be performed without including modification to other portions of the entire software. According to this framework, several considerations have been made about the development and application of a Q-learning agent for hybrid vehicles real-time control. The most relevant clarifications made about the theory of a RL tabular approach when applied to the control of hybrid powertrains are hereafter listed:  
* The distinction between state and observation space in case of application to (P)HEVs;
* The distinction between training and testing episodes;
* The definition of the most meaningful metrics and signals employed for analyzing the behavior of a Q-learning agent when applied to (P)HEVs;
* The identification of a test set of for the interpretation of the results obtained with the RL agent. Once the theoretical framework has been discussed, the main results obtained when modifications are applied to the configuration of the Q-learning agent when tested on multiple experimentally derived DMs are reported.