## 1 Introduction  
Hybrid electric vehicles (HEVs) are, nowadays, representing one of the most exploited technologies to be adopted for reducing the emissions produced by on-road vehicles [1]. In HEVs, the conventional powertrains featured by an internal combustion engine (ICE) are integrated with one or more motor-generators (MGs) and batteries. Thanks to this configuration, the propulsion system can rely upon both an electric (MGs and battery) and a conventional/thermal (ICE) path. Multiple classifications can be accounted for by referring to the possible architectural solutions of hybrid powertrains [2, 3]. First, a vehicle with the capability of being recharged from the external grid is typically referred to as a plug-in HEV (PHEV); on the contrary, a full HEV (FHEV) is considered as a HEV without the possibility of exploiting a recharge from the grid [3]. Then, micro (mHEVs), mild (MHEVs) and FHEVs can be classified according to the progressively increasing degree-of-hybridization (DoH) [4], which is representative of the share of power produced by the electric components over the maximum power of the powertrain as a whole. Finally, the number of MGs, their locations as well as the mechanical connections realized in the powertrain are used to define the driveline architecture (i.e., series, parallel, power-split, multi-mode) [5]. Within the present paper, (P)HEV will be used to identify the entire family of hybrid electric powertrains, including each of the abovementioned versions. As far as the control of the main power and energy components involved in hybrid powertrains is concerned, an energy management system (EMS) is used to define the sequence of operating modes to be realized during a specific trip [6,7]. The EMS is typically representative of a control logic which aims at achieving a specific optimization goal, such as minimizing the fuel consumption (FC) of the ICE, constraining the battery state of charge (\(So\)C), maximizing the components efficiencies, etc. Such an operation can turn into a very complex control problem when it deals with real-time driving conditions. To this end, several research activities in the literature have focused on the development and application of approaches for the real-time control of (P)HEV. In [8], an exhaustive chronological review of the most famous real-time capable control methods for (P)HEVs is reported. Starting from the 1990s, more and more sophisticated techniques have been developed, from very simple rule-based EMSs [8], passing through equivalent consumption minimization strategies (ECMS) [9,10] and dynamic programming (DP) [11] related optimizers, until the latest findings of model predictive control (MPC) [12] and learning algorithms [13,14]. Among the latter, Reinforcement Learning (RL) has started to be effectively exploited for control problems from 2015 onwards [8,15]; hence, it can be considered as an up-to-date approach. RL algorithms are one of the possible Machine Learning (ML) approaches which can be trained to learn a specific task. According to RL, an agent is trained to learn an optimal control policy for a specific environment by means of an iterative smart trial-and-error method [16]. Among the possible RL agents, Q-learning is considered one of the benchmark off-policy tabular algorithms to be employed for solving control problems in discretized environments. Other techniques are also present in the literature and typically refer to different environments (i.e., miscellaneous discrete-continuous or completely continuous), such as deep Q-network (DQN) [17], double DQN (DDQN) [18] or deep deterministic policy gradient (DDPG) [19]. The analysis of these advanced methods is beyond the scope of the present paper and is considered a future research step.