Health-aware smart charging strategies based on
Pontryagin's Minimum Principle and reinforcement
learning

Xuelu Wang, Toufik Azib, Jianwen Meng, Khelil Sidi-Brahim

<figure><img src="image_1.png" /></figure>

To cite this version:

Xuelu Wang, Toufik Azib, Jianwen Meng, Khelil Sidi-Brahim. Health-aware smart charging strategies based on Pontryagin's Minimum Principle and reinforcement learning. Journal of Power Sources, 2026, 663, pp.238846. 10.1016/j.jpowsour.2025.238846. hal-05465656

HAL Id: hal-05465656

https://hal.science/hal-05465656v1

Submitted on 19 Jan 2026

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Health-aware smart charging strategies based on Pontryagin's Minimum Principle and reinforcement learning

Xuelu Wang<sup>ID*</sup>, Toufik Azib<sup>ID</sup>, Jianwen Meng<sup>ID</sup>, Khelil Sidi-Brahim

ESTACA'Lab, 12 Avenue Paul Délouvrier, Montiny-le-Bretonneux, 78180, Ile-de-France, France

<figure><img src="image_3.png" /></figure>

## HIGHLIGHTS

* A novel health-aware charging framework combining PMP and Q-Learning.

* The strategy minimizes heat generation and adapts over battery lifespan.

* A second-order ECM simulates charging effects and predicts degradation.

* Results show effective life extension under real-world fast-charging limits.

## ARTICLE INFO

**Keywords:**

Fast-charging strategy

Battery state of health

Pontryagin's Minimum Principle

Reinforcement learning

## ABSTRACT

Fast charging of Li-ion batteries in electric vehicles often results in excessive heat generation, accelerates degradation and compromising safety. This work proposes two health-aware charging strategies based on Pontryagin's Minimum Principle (PMP) and Q-Learning, aiming to minimize cumulative heat generation while achieving the target state of charge (SOC) within specified constraints. A second-order equivalent circuit model (ECM) integrated with a thermal model is used to simulate battery behavior in a Simulink environment. Both strategies are evaluated using reference fast-charging protocols from the Toyota Research Institute (TRI) dataset. Under equal charging time constraints, the proposed strategies reduce total heat generation relative to the TRI two-step CC-CV protocols (e.g., "8C(35%)–3.6C"). The adaptive PMP achieves up to 27.3% reduction and Q-Learning achieves up to 26.8%, while keeping the final SOC error typically within ≤10%. A correlation analysis between cumulative heat and end-of-life (EOL) motivates the use of heat minimization as the health-aware objective, while the strategy design and evaluations focus on charging performance and thermal stress under practical constraints.

## 1. Introduction

With the rapid adoption of electric vehicles (EVs) and hybrid electric vehicles (HEVs), the demand for smart and efficient battery charging strategies to support sustainable transportation is growing. Among these, fast-charging plays a pivotal role in enabling practical EV deployment. However, current fast-charging systems face multiple challenges, including infrastructure constraints [1,2], grid stress [3,4], and safety risks such as thermal runaway [5,6]. Furthermore, effective real-time charging management is needed to ensure coordination with smart grids and user demand, as well as to enhance the adaptability of power systems under increasing EV penetration [7,8]. As battery health and safety remain critical concerns, recent studies have focused on the degradation induced by fast charging and on the development of health-aware strategies to preserve long-term battery performance [9–12].

Lithium-ion batteries (LIBs), widely adopted in EVs for their high energy and power density, are particularly susceptible to accelerated degradation under fast-charging conditions. High current charging can trigger solid-electrolyte interphase (SEI) growth and lithium plating, impairing battery performance and shortening lifespan. Additionally, fast charging causes substantial heat accumulation, requiring robust thermal management systems to prevent thermal runaway [13,14]. It also leads to cell imbalance, non-uniform aging, and chemical stress on cathode materials that are not inherently optimized for high-rate charging [15,16]. These challenges highlight the urgent need for intelligent charging strategies that balance speed, safety, and longevity in diverse operating conditions.

The constant-current/constant-voltage (CC-CV) protocol remains the most commonly used charging method due to its simplicity and reliability [17]. However, its conservative design limits the charging speed

<footnote>* Corresponding author.
E-mail address: xuelu.wang@estaca.fr (X. Wang).</footnote>
and responsiveness to battery health, which has led to the development of alternative strategies [18,19]. In general, charging strategies can be divided into three categories: rule-based, model-based, and model-free approaches, each with distinct advantages and limitations [5,17,20].

Charging strategies can be categorized into rule-based, model-based and model free strategies. Rule-based strategies such as CC-CV and multi-step charging protocols follow predefined rules or sequences to manage the charging process. These methods are designed to be simple, safe, and reliable under a broad range of conditions, making them industry standards [21]. However, they lack adaptability and often rely on conservative constraints that limit charging speed to avoid potential safety risks, such as overheating. This results in reduced charging efficiency and prolonged charging times, making them less effective for applications that demand faster and more adaptable charging solutions [22,23]. Model-based charging strategies leverage mathematical or physical models, such as Dynamic Programming (DP), Pontryagin's Minimum Principle (PMP), and Equivalent Consumption Minimization, to optimize the charging profile by predicting battery behavior. These strategies provide precise control over charging parameters by accounting for detailed battery dynamics, making them particularly effective in scenarios that require close management of battery health [24, 25]. However, the accuracy of model-based strategies depends on the quality of the underlying battery model, which can be challenging to calibrate and may require significant computational resources. Despite their advantages in precision, these strategies can be complex and resource-intensive to implement in real-time systems [26,27].

Model-free strategies, which include data-driven, machine learning-based methods like reinforcement learning (RL), fuzzy logic, and heuristic-based strategies, operate independently of detailed battery models [28–30]. Instead, these methods adapt dynamically by leveraging real-time data, historical usage patterns, or learned behaviors to optimize the charging process. RL, for instance, allows charging algorithms to learn from previous interactions with the battery to achieve an optimal charging profile, even in complex environments. Fuzzy logic and heuristic approaches apply rule-based logic that simulates human decision-making, adjusting charging based on observable variables like temperature and the state-of-health (SOH) [31,32]. While model-free strategies offer high adaptability and are well-suited for dynamic conditions, their performance can vary, and they may require extensive training data to achieve the desired results [33,34].

Beyond these categories, recent health-aware charging studies have introduced advanced control-oriented approaches, including nonlinear model-inversion control [35,36] and physics-constraint-triggered proportional–integral (PI) schemes [37,38], which explicitly limit thermal and SOH stress during fast charging. We position our PMP/RL framework as complementary to these control-oriented methods under the same health-aware objective, providing a balance between theoretical optimality and data-driven adaptability.

Accurate battery modeling is still fundamental to developing effective charging strategies. Two main approaches, Electrochemical Models (EM) and Equivalent Circuit Models (ECM), serve as the basis for simulating battery dynamics. EMs provide detailed insights into internal processes such as ion diffusion and reaction kinetics but are computationally intensive and unsuitable for real-time control [39, 40]. In contrast, ECMs simplify battery behavior into electrical components (e.g., resistors and capacitors) to capture essential dynamic characteristics like voltage response and internal resistance [41,42]. The second-order ECM, in particular, provides a practical trade-off between accuracy and computational efficiency. By including two parallel resistance-capacitor (RC) networks, it reliably models battery dynamics while avoiding the computational overhead associated with EMs [43]. For these reasons, this study adopts the second-order ECM to support real-time simulation and strategy development.

Despite recent advances in fast-charging protocols, existing strategies often overlook the cumulative thermal and aging effects, which are crucial for long-term battery health. Most prior works either assume

fixed thermal limits or use conservative cutoffs, without dynamically adapting to the battery's degradation state. Moreover, many model-based optimization strategies (e.g., DP, PMP) require accurate battery models that may not reflect real-time aging, while model-free approaches (e.g., RL) are seldom benchmarked under health-centric objectives and realistic thermal constraints. A unified framework that directly compares these two paradigms from a health-aware perspective under consistent conditions remains missing.

To address this gap, we propose a comparative framework that integrates a model-based strategy (Pontryagin's Minimum Principle) and a model-free reinforcement learning (Q-Learning) approach, both explicitly targeting cumulative heat minimization as a degradation proxy. Using a second-order ECM for real-time simulation and real-world aging data from the TRI dataset, we quantify the nonlinear relationship between heat generation and end-of-life (EOL) to validate our thermal-aware objective. Our results demonstrate that both strategies reduce cumulative heat generation by over 27%, offering insights into designing adaptive, health-oriented fast-charging policies.

In summary, we integrate a model-based (PMP) and a model-free (Q-Learning) approach to offer complementary benefits: the PMP strategy provides an interpretable, near-optimal baseline with explicit optimality conditions, while the Q-Learning method adapts to modeling errors and non-stationary operating conditions (e.g., temperature and SOH), enabling deployment under imperfect model fidelity. First, an adaptive PMP-based optimal control strategy is developed to explicitly minimize the cumulative heat generation while ensuring that the target SOC is achieved within the prescribed charging period. In parallel, a model-free Q-Learning algorithm is designed to learn health-aware charging policies by interacting with a second-order electro-thermal equivalent circuit model of the battery. Furthermore, using real-world aging data from the TRI battery database, a comprehensive statistical analysis confirms a strong nonlinear negative correlation between cumulative heat generation and battery EOL. Simulation results under various fast-charging protocols demonstrate that both strategies substantially reduce thermal stress while preserving charging efficiency: the PMP-based control achieves a 27.3% reduction in cumulative heat generation with explicit optimality guarantees, and the Q-Learning strategy achieves a comparable reduction of up to 26.8% through model-free learning.

## 2. Methodology

Fig. 1 presents the overall workflow of this study. The process begins with a second-order electro-thermal ECM simulating the battery's response under fast-charging protocols from the TRI database. Heat generation profiles are extracted for each case. A correlation analysis is then conducted to explore the relationship between cumulative heat and EOL. Based on these insights, two health-aware strategies, PMP and Q-Learning, are designed to minimize heat generation while achieving SOC targets. Finally, simulation results are compared to evaluate each strategy's effectiveness in reducing total heat generation (HG).

### 2.1. ECM for battery modeling

The ECM is an important battery modeling method that can reliably capture battery dynamic characteristics and is widely applied in battery management systems. Several studies in the literature provide valuable references for further research on battery modeling. [44] proposed 12 different lithium-ion battery ECMs and conducted comparative studies on them. In [45] authors proposed an improved ECM and applied it to battery power state prediction. In [46], a second-order ECM to describe the behavior of lithium titanium oxide batteries was developed. In [47], authors investigated the influence of temperature and state of charge on the parameters of the ECM. In [48], authors proposed a new method that combines the ECM with electrochemical characteristics for battery parameter identification and update. [49] examined the
<figure><img src="image_4.png" /><figcaption>Fig. 1. Overview of the proposed methodology. Step 1: Simulate generated heat HG(t) using a 2nd-order electro-thermal ECM under various charging protocols. Step 2: Analyze the correlation between cumulative heat Σ HG and battery EOL using Pearson and Spearman methods. Step 3: Design optimal charging strategies using PMP and Q-Learning approaches with ECM-in-the-loop optimization. Step 4: Compare the strategies based on total generated heat to evaluate health-aware performance.</figcaption></figure>

performance of first-order and second-order RC ECM in electric vehicle applications. [50] reviewed the ECM used for real-time estimation of LIB states.

The second-order ECM used in this study provides a practical and computationally efficient approach to simulate battery behavior, capturing both electrical and thermal dynamics with adequate accuracy for real-time applications. This model was proposed by Forgez et al. [43]. and widely used in battery research. It simplifies the complex electrochemical processes inside the battery into electrical components, effectively representing the battery's transient response and thermal effects during charging and discharging cycles. The second-order ECM incorporates two RC branches, representing both the fast and slow dynamic responses of the battery, as is shown in Fig. 2(a). This structure allows the model to simulate critical characteristics, such as voltage relaxation and transient behavior, with high accuracy [43].

In order to describe the battery behavior, the adopted model proposed a double-layer structure that simulates both thermal and electrical domains.

### 2.1.1. Electric model

The electrical part of the second-order ECM can be represented by two parallel RC branches, each capturing different aspects of the battery's dynamic response:

**SOC Dynamics:** The SOC is updated based on Coulomb counting, which relates the SOC change to the current flowing through the battery:

$$SOC = -\frac{I}{C_{Bat}} \qquad (1)$$

where I is the current and $C_{Bat}$ is battery capacity.

**Voltage Dynamics:** The voltage across each RC branch ($V_1$ and $V_2$) is governed by differential equations representing the fast and slow relaxation processes:

$$\dot{V}_1 = -\frac{V_1}{R_1 C_1} + \frac{I}{C_1} \qquad (2)$$

$$\dot{V}_2 = -\frac{V_2}{R_2 C_2} + \frac{I}{C_2} \qquad (3)$$

where $R_1$, $C_1$ are associated with fast dynamics, and $R_2$, $C_2$ with slower dynamics.

**Terminal Voltage:** The battery's terminal voltage $V_T$ is computed by combining the open-circuit voltage (OCV), the voltages across the RC pairs, and the series resistance $R_s$:

$$V_T = V_{OCV}(SOC) - V_1 - V_2 - IR_s \qquad (4)$$

where $V_{OCV}(SOC)$ represents the open-circuit voltage as a function of the SOC.

### 2.1.2. Thermal model

Fig. 2(b) represents an equivalent resistance model for thermal transmission. The following parameters characterize the thermal behavior of a cell [43].

**HG:** Generated heat Q in the battery arises from both Joule heating and entropic effects:

$$Q = I(V_{OCV} - V_T) - I T_m \frac{dU}{dT} \qquad (5)$$

where $T_m$ is the average temperature, and $\frac{dU}{dT}$ is the entropic heat coefficient.

**Core Temperature Dynamics:** The dynamics of the core temperature $T_c$ are given by:

$$C_c \dot{T}_c = Q + \frac{T_s - T_c}{R_c} \qquad (6)$$

where $C_c$ is the core's heat capacity, Q is the generated heat, and $R_c$ is the thermal resistance between the core and surface.

**Surface Temperature Dynamics:** The surface temperature $T_s$ is governed by:

$$C_s \dot{T}_s = \frac{T_c - T_s}{R_c} + \frac{T_f - T_s}{R_u} \qquad (7)$$
<figure><img src="image_5.png" /><figcaption>Fig. 2. (a) Circuit for a 2nd ECM and (b) Thermal equivalent resistance model for thermal transmission. [43]. (c) 2nd order ECM Simulink model used in this research with charging rate I and environmental temperature $T_f$ as inputs and cell core temperature $T_c$, cell surface temperature $T_s$, battery SOC SOC, cell terminal voltage $V_T$ and generated heat HG as outputs.</figcaption></figure>

where $C_s$ is the surface's heat capacity, $R_u$ is the convection resistance to the coolant, and $T_f$ is the coolant flow temperature.

**Average Temperature:** The average temperature $T_m$ is used to account for temperature-dependent parameters in the electrical model:

$$T_m = \frac{T_c + T_s}{2} \qquad (8)$$

This second-order ECM, which integrates both electrical and thermal submodels, provides a detailed yet computationally efficient framework for simulating lithium-ion battery behavior. It effectively captures terminal-voltage dynamics and heat generation, thereby serving as a robust foundation for designing health-aware charging strategies that balance fast-charging performance and battery longevity. All model parameters, electrical ($R_s, R_{1,2}, C_{1,2}$) and thermal ($R_c, R_u, C_c, C_s$) together with the open-circuit-voltage (OCV-SOC) relationship and the entropic heat coefficient $\frac{dU}{dT}$, are adopted from the validated work of Forgez et al. (2010) [43]. The parameter values were originally identified from cylindrical LiFePO$_4$/graphite cells under isothermal conditions and have been widely reused in subsequent electro-thermal simulations due to their physical consistency and stability. It is worth noting that these parameters were originally identified from cylindrical 26 × 80 mm (2680-type) LiFePO$_4$/graphite cells rather than the 18 × 65 mm (18650-type) cells employed in the TRI reference dataset. Both cell formats share the same LFP-based chemistry and exhibit similar thermal and electrochemical characteristics, ensuring that the present model remains representative for comparative analysis. Table 1 summarizes the adopted parameter set and corresponding units. Fig. 2(c) illustrates the model inputs and outputs, enabling accurate simulation of the electro-thermal response of the battery under various operating conditions.

## 2.2. Problem formulation

### 2.2.1. Problem definition

In this section, we formulate the supervisory optimal control as a finite-horizon problem, where the objective function (e.g., instantaneous heat or an aging coefficient) is minimized subject to battery dynamics and operational constraints.

**State variable ($x(t)$) and control variable ($u(t)$):**

$$x(t) = \mathrm{SOC}(t) \qquad (9)$$

$$u(t) = I_{\mathrm{charge}}(t) \qquad (10)$$

**System Dynamics:**

$$\dot{x}(t) = -\frac{I(t)}{C_{\mathrm{Bat}}} \qquad (11)$$

**Objective Function:** The goal of the smart charging strategy is to determine the optimal control input $u(t)$ that minimizes the instantaneous cost function $\mathcal{L}(u(t), t)$ over the charging horizon $[t_0, t_f]$. This can be expressed as the minimization of the finite-time performance index:

$$J = \int_{t_0}^{t_f} \mathcal{L}(u(t), t) dt \qquad (12)$$

The minimization of $J$ is subject to physical constraints of the charging system, including actuator limitations, energy capacity bounds, and the requirement that the battery SOC remains within prescribed limits. This defines a constrained finite-time optimal control problem, where both global and local constraints are imposed on the state and control variables. In this work, a strong nonlinear negative correlation between the cumulative generated heat during charging and the EOL of the battery is observed (explained in Section 3). Accordingly, a health-aware optimal control objective is formulated to minimize the total generated heat throughout the charging process by determining the
<table><caption>**Table 1**

Electrical and thermal parameters of the adopted second-order ECM during charging process [43].</caption><thead><tr><th>Parameter</th><th>Symbol</th><th>Range/Value</th><th>Unit</th></tr></thead><tbody><tr><td>Series resistance (SOC-dependent)</td><td>R<sub>s</sub>(SOC)</td><td>0.0082–0.0772</td><td>Ω</td></tr><tr><td>RC 1 resistance (SOC-dependent)</td><td>R<sub>1</sub>(SOC)</td><td>0.0073–0.1749</td><td>Ω</td></tr><tr><td>RC 1 capacitance (SOC-dependent)</td><td>C<sub>1</sub>(SOC)</td><td>321.56–2984.80</td><td>F</td></tr><tr><td>RC 2 resistance (SOC-dependent)</td><td>R<sub>2</sub>(SOC)</td><td>0.2262–0.0108</td><td>Ω</td></tr><tr><td>RC 2 capacitance (SOC-dependent)</td><td>C<sub>2</sub>(SOC)</td><td>6151.35–129076.52</td><td>F</td></tr><tr><td>Core heat capacity</td><td>C<sub>c</sub></td><td>62.7</td><td>J/K</td></tr><tr><td>Surface heat capacity</td><td>C<sub>s</sub></td><td>4.5</td><td>J/K</td></tr><tr><td>Core–surface thermal resistance</td><td>R<sub>c</sub></td><td>1.94</td><td>K/W</td></tr><tr><td>Surface–ambient thermal resistance</td><td>R<sub>s</sub></td><td>15</td><td>K/W</td></tr><tr><td>Entropic heat coefficient (SOC-dependent)</td><td>dU/dT (SOC)</td><td>-0.3038–0.1302</td><td>V/K</td></tr></tbody></table>

optimal charging current profile $I_{charge}(t)$. The specific cost function is therefore defined as:

$$J = \int_{t_0}^{t_f} HG(I_{charge}(t), T(t)) dt \quad (13)$$

where $HG(\cdot)$ denotes the instantaneous heat generation rate (J/s), computed from the electro-thermal model as $HG = I_{charge}(V_{OCV} - V_T) - I_{charge}T_m \frac{dU}{dT}$.

### Global and Local Constraints:

Global constraint indicates the fundamental requirements which cannot be violated during the optimal control process. In the scenario of charging strategy, the final SOC value $x(t_f)$ should reach a predefined value $SOC_{target}$:

$$SOC(t_f) = SOC_{target} \quad (14)$$

Local constraints are applied to both the state and control variables. For the state variable, the constraint ensures that the state of charge (SOC) remains within predefined upper and lower bounds. As for the control variables, constraints are enforced to respect the physical limitations of the system's operation:

$$ \begin{aligned} & SOC_{min} \le SOC(t) \le SOC_{max}, \\ & T_{c,min} \le T_c(t) \le T_{c,max}, \\ & I_{C,min} \le I_C(t) \le I_{C,max}, \\ & T_{s,min} \le T_s(t) \le T_{s,max} \end{aligned} \quad (15) $$

where $(\cdot)_{min}$, $(\cdot)_{max}$ are the minimum and maximum value of SOC, charging rate and temperatures at each instant $t$.

### 2.2.2. PMP based smart charging strategy

PMP is used to solve an optimal control problem and this problem consists of finding the optimal control sequence $I_{charge}^*(t)$ that minimizes the cost function while meeting the dynamic state constraint, the global state constraint and local state and control constraints. The charge-sustaining constraint has been taken into account as a hard constraint by requiring that the SOC at the end of the mission equals the target final value of SOC. Alternatively, this constraint can be enforced as a soft constraint, i.e., by penalizing deviations from the initial value of the energy stored at the end of the mission. This is done by means of a penalty function $\phi(x(t_f))$ (generally, a function of the difference $x(t_f) - x(t_0)$) to be added to the performance index (13) to obtain a battery charge-sustaining performance index:

$$J = \phi(SOC(t_f)) + \int_{t_0}^{t_f} HG(I_C(t), t) dt \quad (16)$$

In this research, a soft constraint is also represented by a quadratic function of the difference $SOC(t_f) - SOC(t_0)$, namely $\phi = \alpha ||SOC(t_f) - SOC(t_0)||^2$ where $\alpha$ is a positive weighting factor in order to close the gap between $SOC(t)$ and $SOC(t_f)$ faster when the charging process is nearing the end. In summary, a more general expression is presented by:

$$J = \phi(SOC(t_f)) + \int_{t_0}^{t_f} HG(SOC(t), I_C(t), t) dt \quad (17)$$

The explicit dependence of the Hamiltonian $H$ on time $t$ in the smart charging context is focused solely on the control input $I_{charge}$. Therefore, the Hamiltonian function for the smart charging problem becomes:

$$H(SOC(t), I(t), \lambda(t)) = HG(t) + \lambda(t) \cdot \frac{dSOC(t)}{dt} \quad (18)$$

where $HG(t)$ is the instantaneous heat generation, $\lambda(t)$ is the co-state associated with the state variable $SOC(t)$, $\frac{dSOC(t)}{dt}$ is the SOC dynamics. The optimal charging strategy $I_{charge}^*(t)$ is given by:

$$I_{charge}^*(t) = \arg \min_{I_{charge} \in U_{I_c}} H(I_{charge}(t), SOC(t), \lambda(t)) \quad (19)$$

Then, as boundary conditions, the optimal SOC $SOC^*(t)$ and optimal co-state $\lambda^*(t)$ at optimal charging current $I_{charge}^*(t)$ are given by:

$$SOC^*(t) = f(SOC^*(t), I_{charge}^*(t)), \quad (20)$$

$$ \begin{aligned} \dot{\lambda}^*(t) = & -(\lambda^*(t) + w(SOC, t)) \frac{\partial f}{\partial SOC} (SOC^*, I_{charge}^*) \\ & = h(SOC^*(t), I_{charge}^*(t), \lambda^*(t)), \end{aligned} \quad (21) $$

$$SOC^*(t_0) = SOC_0, \quad (22)$$

$$SOC^*(t_f) = SOC_{target}. \quad (23)$$

Eqs. (20) and (21) represent a system of two first-order differential equations in the variables $SOC$ and $\lambda$. Despite being completely defined, this two-point boundary value problem can be solved numerically only using an iterative procedure, since one of the boundary conditions is defined at the final time.

The solution of the PMP's necessary conditions is obtained via shooting method. Starting with an initial guess of $\lambda_0$, at each iteration of the shooting method, the minimum principle conditions are solved throughout the length of the optimization horizon, $[t_0, t_f]$, typically corresponding to the duration of a driving cycle. At the end of the simulation, the obtained value of the $SOC(t_f)$ is compared to the desired state of charge, $SOC_{target}$. Depending on the difference $SOC(t_f) - SOC_{target}$, the value of $\lambda_0$ is either adjusted and the simulation repeated, or the algorithm terminates if the difference reaches the desired target (i.e., it is close to zero within a pre-defined tolerance of 5% to 15%, depending on different testing conditions and computation burdens).

Different from traditional PMP strategy that requires knowledge of the entire charging process, an adaptive PMP does not require prior knowledge of the charging trajectory by introducing a mechanism to adjust co-state value based on multi-objectives. This adaptive mechanism allows PMP to be deployed to optimize the fast charging strategy in real time. As we can see, Fig. 3 represents the flow chart of the adaptive PMP strategy. In order to realize such a mechanism, we defined the cost function in the PMP strategy as the generated heat calculated by the 2nd order ECM of each time step plus a penalty on the quadratic difference between $SOC_f$ and $SOC(t)$:

$$L = \alpha \times \sum HG(t) + \beta \times (SOC_{final} - SOC(t))^2 \quad (24)$$
<figure><img src="image_6.png" /><figcaption>Fig. 3. Adaptive PMP Strategy flow chart.</figcaption></figure>

with α and β the importance of each factor. Based on (18), the Hamil-
tonian is calculated by:

$$
H(I(t)) = L + \lambda(t) \frac{-I(t)}{C_{bat}} \qquad (25)
$$

The co-state function $\lambda(t)$ is then updated via the following expression
by adapting to a time-dependent penalty $w_{time}$:

$$
\lambda(t+1) = \lambda(t) + w_{time} \qquad (26)
$$

where $w_{time} = K \frac{1}{T_f - t} \frac{-I(t)}{C_{bat}}$ with $K$ a constant and $T_f$ the terminal time of the charging horizon. By providing a proper initial co-state value $\lambda_0$, the mechanism will adapt to an optimal charging curve that minimizes the objective function.

2.2.3. RL based smart charging strategy

RL has been widely used to solve optimal control problems. For
example, RL has shown promising potential in the energy management
strategies of HEVs, which can effectively improve the fuel economy
and power performance of vehicles. A comparative analysis of DP,
stochastic DP, and RL in the energy management strategy of HEV was
comparatively reported in [51]. [52] proposed a data-driven energy
management strategy for HEVs based on deep RL. [53] proposed an
energy management strategy for HEVs based on hierarchical RL. RL was
applied using batch-learning methods to solve the energy management
problem of HEVs in [54]. Researchers have also proposed an improved
RL-based EV swarm intelligent charging management system that can
effectively prevent grid overload. The system achieves this by learning
the optimal charging control strategy [55].

In this study, a Q-Learning-based framework is developed to derive
an optimal battery charging policy that satisfies target SOC require-
ments while minimizing heat generation, thereby contributing to im-
proved battery longevity. The Q-Learning algorithm enables an agent
to incrementally learn the most effective charging currents through
interaction with a simulated battery environment. The algorithm main-
tains a Q-table in which each entry represents the expected cumulative
reward for executing a specific action (charging current) in a given
state (SOC). Over time, the agent updates this table to converge toward
a policy that maximizes the expected return, balancing charging effi-
ciency and thermal management. To balance exploration of unvisited
state-action pairs with exploitation of learned knowledge, an ε-greedy
strategy is employed during training. At each decision step, the agent
selects a random action with probability ε, encouraging exploration
of alternative strategies. Conversely, with probability 1 − ε, the agent
chooses the action with the highest Q-value, favoring exploitation.

This trade-off is particularly important in the early learning stages to ensure sufficient coverage of the state-action space and avoid premature convergence. The agent interacts with a Simulink battery model that illustrates the system's electro-thermal dynamics. For each action taken, the 2nd order ECM simulates the battery's response and outputs key parameters shown in Fig. 2(c). These outputs are then utilized to compute the reward, which guides subsequent updates to the Q-values. This real-time interaction between the Q-Learning agent and the physical simulation environment ensures that the learned policy is well aligned with the actual battery behavior. The reward function is designed to encourage timely achievement of the target SOC while penalizing excessive heat generation and significant SOC deviations, particularly near the end of the charging period. To ensure tractability of the Q-table, the continuous SOC space is discretized into 60 uniformly distributed bins. A mapping function converts the actual SOC value into a corresponding discrete state index, enabling efficient Q-value updates. Through this RL approach, the agent progressively acquires a charging strategy that adheres to both energy efficiency and thermal safety constraints. The resulting policy effectively balances fast charging and temperature control, contributing to enhanced battery performance and extended operational lifespan.

2.3. Adopted referential fast charging protocol database - Toyota Research Institute (TRI)

The database used as a reference in this research is provided by
Severson, et al. and contains capacity degradation data of 124 com-
mercial lithium iron phosphate/graphite (LFP) cells (APR18650M1 A)
under fast charging conditions [56]. The charging rates vary from
3.6C CC-CV to 6C in order to observe the performance of current-
generation LFP cells under different extreme fast-charging conditions
(about 10 min charging). By varying the charging conditions, a dataset
that captures a wide range of cycle lives is generated. The EOL varies
from approximately 150 to 2300 cycles. These cells were cycled in an
isothermal chamber under varied fast-charging conditions but identical
discharging conditions. All cells in this database are charged with a
one-step or two-step fast-charging policy with a format of “C1(Q1)-C2”.

3. Results and discussions

3.1. TRI fast charging protocols under 2nd order ECM simulation

In order to study the battery behavior simulated by the mentioned
electrical-thermal 2nd order ECM model, a preliminary test using fast
<figure><img src="image_7.png" /><figcaption>Fig. 4. Simulation framework for correlation analysis between cumulative heat generation and battery EOL. Fast charging protocols extracted from the TRI battery aging database are applied to a 2nd order ECM implemented in Simulink. The cumulative heat sum(HG) is computed from HG and used to analyze its statistical correlation with EOL under different charging profiles and cycles. Metrics such as Pearson Correlation, Spearman correlation, MI, dCor and Poly R² are used to evaluate.</figcaption></figure>

charging protocols selected in TRI database is conducted, as is pre-
sented in Fig. 4. Due to different LFP battery model and specifications
(such as nominal capacity), only charging process was adopted and the
final SOC in the simulation is not 100% as is tested in the database, and
the final SOC in the simulation is directly obtained by the 2nd order
ECM.

To investigate the relationship between battery aging and heat
generation, we conducted Pearson and Spearman correlation analyses
between the cumulative heat generation (Σ HG) and cell EOL across
different degradation cycles (Cycle 1, 50, 100, and 150). As shown in
Fig. 5, all test cycles exhibit negative correlation trends, indicating that
increased heat generation is generally associated with faster battery
degradation and a reduced EOL. In early degradation stages (e.g., Cycle
1), the Pearson correlation coefficient reaches –0.372, while the Spear-
man correlation is even stronger at –0.478, both with p-values < 0.001,
indicating statistically significant relationships. Similar patterns are
observed at Cycle 50 and Cycle 100, where the Spearman coefficient re-
mains consistently lower (more negative) than the Pearson coefficient,
suggesting the presence of monotonic yet nonlinear dependencies. At
later degradation stages (Cycle 150), the Pearson correlation weakens
to –0.106 (p = 0.242), while the Spearman coefficient remains rela-
tively stronger at –0.204 (p = 0.024), revealing that although the linear
association diminishes, a nonlinear negative correlation still persists.

To further validate and quantify these nonlinear dependencies, we
incorporated Mutual Information (MI) and Distance Correlation (dCor)
analyses. MI captures the general statistical dependence between two
variables without assuming any specific functional form, while dCor
quantifies how strongly two variables are associated across any rela-
tionship structure. The results show that MI remains consistently above
1.0 bit across all cycles, peaking at 1.162 bits at Cycle 50, indicating
that Σ HG contains substantial predictive information regarding bat-
tery EOL, even when linear trends weaken. Similarly, dCor stays above
0.34 throughout all cycles, reaching 0.484 at Cycle 50, confirming
the existence of statistical dependency beyond simple correlation. In
addition, second-order polynomial regression was applied to model the
nonlinear relationship between Σ HG and EOL. The corresponding R²
values reflect the explanatory power of this simple nonlinear model,
with the highest R² observed at Cycle 50 (0.280) and Cycle 5 (0.260),

gradually decreasing in later stages (0.223 at Cycle 100 and 0.153 at
Cycle 150).

These findings confirm a nonlinear negative correlation between
cumulative heat generation and battery EOL, especially in the earlier
cycles. This indicates that as more heat is generated during the charging
process, the battery tends to degrade faster, resulting in a shorter
overall lifespan and a quicker decline in its SOH. The nonlinearity
suggests that even small increases in heat can lead to disproportionately
large reductions in battery health, particularly during early degradation
stages.

## 3.2. Optimal smart charging strategy based on adaptive PMP and Q-learning

3.2.1. Adaptive PMP based optimal charging strategy

In this part, an adaptive PMP strategy is developed and tuned to
compute in real time the optimal charging strategy that minimizes total
generated heat, thereby extending the EOL as we discussed in the previous
section.

Fig. 6 shows the comparison between the simulated results based on TRI fast charging protocol and results based on proposed adaptive PMP optimal charging strategy by using parameters shown in Table 2. The TRI reference charging protocol (blue) represents a two-step CC-CV charging mode with 5.4C to 60% SOC and then with 3.6C till around 80% SOC. The orange curve represents the calculated adaptive optimal charging curve via PMP. In order to stabilize the charging protocol and to reduce the computation burden, a discrete mode is adopted with a time step of 10 s, which means that each 10 s, the charging rate is updated by computing the optimal charging current that minimizes the Hamiltonian at this instant.

The detailed result is presented in Table 3. As a result, while
maintaining the same charging time, the reached final SOC has a little
difference due to algorithm parameter calibration precision (1.9%), the
total generated heat of PMP optimal charging strategy during charging
process is reduced by approximately 5.35%.

The experimental simulation was conducted across all 124 fast-
charging protocols provided in the TRI database. Table 4 summarizes
the final SOC error and the reduction in total heat generation achieved
<figure><img src="image_8.png" /><figcaption>Fig. 5. Comprehensive correlation analysis between cumulative heat generation and cell EOL at different degradation cycles (Cycle 1, 50, 100, and 150). Each subplot reports the Pearson and Spearman correlation coefficients, MI, dCor, and the R² value of a second-order polynomial fit. Results indicate progressively weakening correlations as the battery ages, with stronger nonlinear dependencies observed in earlier cycles.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 6. Proposed adaptive PMP result for one charging cycle.</figcaption></figure>

by the proposed adaptive PMP strategy compared to the original TRI protocols.

Additionally, Fig. 7 illustrates the distribution of these two metrics across all protocols. The results clearly demonstrate that the adaptive PMP strategy offers improved health-aware performance by consistently lowering total heat generation under fast-charging conditions,

even while maintaining a comparable SOC performance. Fig. 7 presents a comprehensive statistical evaluation of the proposed adaptive PMP strategy. The upper subplots illustrate the frequency distributions of the relative reduction in total heat generation (left) and the final SOC error (right), in comparison with the conventional TRI protocol. Notably, in
<figure><img src="image_10.png" /><figcaption>**Fig. 7.** (a) Histogram of experimental results under the proposed PMP strategy compared to the TRI protocols. The left panel shows the distribution of reduction percentages in total heat generation and the right panel shows the final SOC error distribution across all experiments. (b) Normalized relative error of HG and final SOC on each episode of the charging protocols.</figcaption></figure>

<table><caption>**Table 2**

Adaptive PMP simulation parameters, identified under the ECM parameter set reported in Table 1. All values fall within physically plausible ranges and exhibit narrow confidence intervals across multiple runs, ensuring model validity for on-policy optimization.</caption><thead><tr><th>Param.</th><th>λ<sub>0</sub></th><th>T<sub>env</sub></th><th>K</th><th>α</th><th>β</th></tr></thead><tbody><tr><td>Value</td><td>22</td><td>30</td><td>-5</td><td>145</td><td>1</td></tr></tbody></table>

the majority of test cases, the PMP strategy yields a significant reduction in heat generation, commonly exceeding 20%, while maintaining the final SOC error within an acceptable margin, typically below 10%. The bottom subplot displays the normalized relative error of heat generation and final SOC across individual episodes. The results highlight the stability and effectiveness of the strategy, with most episodes achieving high consistency in thermal control and SOC accuracy. Importantly, performance variation across episodes reveals that the strategy performs particularly well under extended charging durations. In contrast, suboptimal performance tends to occur in scenarios characterized by one-step or two-step high-rate charging protocols, where the short duration offers limited opportunity for the agent to adapt and optimize the charging trajectory. These observations underscore the strength of the proposed method in dynamically adjusting charging currents over time, particularly when sufficient temporal resolution is available to refine the control policy.

### 3.2.2. Q-Learning based optimal charging strategy

In this part, a Q-Learning-based optimal charging strategy is developed to optimize the battery charging process by minimizing total heat generation while achieving a target SOC. The Q-Learning agent interacts with a 2nd order ECM to calculate an optimal control policy over multiple charging episodes. First of all, the environment state is

defined as the battery's SOC, denoted by $s_t = SOC_t$. To make the Q-table tractable, the continuous SOC space is discretized into N uniform states using:

$$s^{\text{index}} = \min \left( N, \max \left( 1, \left[ \frac{\text{SOC}_t - \text{SOC}_0}{\text{SOC}_{\text{final}} - \text{SOC}_0} \cdot (N-1) \right] + 1 \right) \right) \quad (27)$$

The action space consists of a finite set of charging current values:

$$A = \{I_{\min}, I_{\min} + \Delta I, \dots, I_{\max}\} \quad (28)$$

with $I_{\min} = 1C$, $I_{\max} = 8C$, $\Delta I = C_{\text{bat}}$. At each time step t, the agent selects an action $a_t \in A$ using the $\epsilon$-greedy policy:

$$a_t = \begin{cases} \text{random action, with probability } \epsilon \\ \arg \max_a Q(s_t, a), \text{ with probability } 1-\epsilon \end{cases} \quad (29)$$

The agent receives a scalar reward $r_t$, and updates its Q-value using the standard Bellman equation:

$$Q(s_t, a_t) \leftarrow Q(s_t, a_t) + \alpha \left[ r_t + \gamma \cdot \max_{a'} Q(s_{t+1}, a') - Q(s_t, a_t) \right] \quad (30)$$

where $\alpha$ is the learning rate and $\gamma$ is the discount factor. The reward function is crafted to encourage efficient charging with minimal thermal stress. At each time step, the reward is computed as:

$$r_t = 200 \cdot SOC_t - 0.8 \cdot HG_t \quad (31)$$

The relative scales of SOC and HG differ significantly. To ensure both components contribute comparably to the reward signal, their coefficients were scaled accordingly. Specifically, since SOC $\in [0, 1]$, a multiplier of 200 ensures that SOC progression meaningfully influences the reward. Meanwhile, HG(t) values typically range between 0 to 2 per step, so a smaller penalty weight (0.8) was empirically effective in discouraging thermal spikes without overwhelming the SOC gain signal. Additionally, a bonus reward is given when:

$$\text{if } SOC_t < 0.5 \text{ and } I_t > 4 \Rightarrow r_t \leftarrow r_t + 20 \quad (32)$$
<table><caption>**Table 3**

Result comparison between TRI Reference and adaptive PMP Strategy for $i_{cell}$ = 17.</caption><thead><tr><td>Method</td><td>Charge time (s)</td><td>Final SOC</td><td>Total heat (sum(HG))</td></tr></thead><tbody><tr><td>TRI Protocol</td><td>605</td><td>72.93%</td><td>303.89</td></tr><tr><td>PMP Strategy</td><td>605</td><td>71.52%</td><td>287.63</td></tr></tbody></table>

<table><caption>**Table 4**

Results of reduced total generated heat by PMP strategy during charging process comparing to different TRI fast charging protocols.</caption><thead><tr><td>Ref. Protocol</td><td>Error on final SOC</td><td>Reduction on sum(HG)</td></tr></thead><tbody><tr><td>5.4C(60%)3.6C</td><td>1.91%</td><td>5.35%</td></tr><tr><td>5.4C(40%)3.6C</td><td>3.21%</td><td>7.35%</td></tr><tr><td>5.4C(50%)3C</td><td>0.00%</td><td>2.84%</td></tr><tr><td>6C(30%)3.6C</td><td>1.74%</td><td>4.82%</td></tr><tr><td>6C(50%)3C</td><td>4.03%</td><td>14.58%</td></tr><tr><td>7C(40%)3C</td><td>0.92%</td><td>11.42%</td></tr><tr><td>8C(35%)3.6C</td><td>9.07%</td><td>27.25%</td></tr><tr><td>2C(10%)6C</td><td>7.68%</td><td>25.73%</td></tr><tr><td>5.2C(66%)3.5C</td><td>9.97%</td><td>22.92%</td></tr><tr><td>5C(67%)4C</td><td>10.03%</td><td>21.87%</td></tr></tbody></table>

<table><caption>**Table 5**

Q-Learning hyperparameters and system settings used in the training process.</caption><thead><tr><td>Parameters</td><td>Value/Description</td></tr></thead><tbody><tr><td>SOC discretization (N)</td><td>60 states</td></tr><tr><td>Action space (charging current)</td><td>{1, 2, ..., 8} C</td></tr><tr><td>Time step (Δt)</td><td>1 s</td></tr><tr><td>Exploration rate (ε)</td><td>0.5 (initial), decays by 0.98 per episode</td></tr><tr><td>Learning rate (α)</td><td>0.15</td></tr><tr><td>Discount factor (γ)</td><td>0.7</td></tr><tr><td>Final reward (R<sub>final</sub>)</td><td>100</td></tr><tr><td>Number of training episodes</td><td>30</td></tr><tr><td>Reward formula (r<sub>t</sub>)</td><td>200 · SOC<sub>t</sub> − 0.8 · HG<sub>t</sub></td></tr><tr><td>Additional reward</td><td>+20 if SOC<sub>t</sub> < 0.5 and I<sub>t</sub> > 4 A</td></tr></tbody></table>

Once the target SOC is reached, a final reward is assigned:

$$ \text{if } SOC_{t} \geq SOC_{\text{final}} \Rightarrow r_{t} = R_{\text{final}} = 100 \quad (33) $$

The selected action $a_t$ is interpreted as the charging current and passed into a validated Simulink battery model. The model outputs the real-time SOC, surface temperature $T_s$, and heat generation $HG_t$, which are then used to compute the reward and guide the learning update. The key hyperparameters and their assigned values are summarized in Table 5.

The agent is trained over 30 episodes, each simulating a full charging cycle from an initial SOC of 0 to a target SOC of around 80%. After each episode, the exploration rate $\epsilon$ is decayed by a factor of 0.98. Performance metrics such as the final SOC, cumulative reward, and total generated heat are recorded at the end of each episode to evaluate learning progress. The best-performing policy is selected based on the highest ratio of final SOC to total heat generated. For this episode, detailed temporal trajectories of SOC, charging current, surface temperature, and heat generation are analyzed to assess the quality and efficiency of the learned strategy. These hyperparameters were refined through iterative empirical testing over a range of values. The final set of parameters yielded stable learning performance and balanced convergence between fast charging and thermal minimization across training episodes.

Fig. 8 shows the result of Q-Learning based optimal charging strategy across three aspects: learning progress, action-value policy, and real-time control outcomes. The learning curve (a) demonstrates the evolution of total rewards, accumulated heat generated, and final SOC over 30 episodes. The total reward increases steadily, indicating effective policy learning. Meanwhile, the cumulative heat generated decreases over time, suggesting that the agent successfully learns to minimize thermal impact during charging. The final SOC remains consistently above 0.7, confirming that the charging goal is achieved

<table><caption>**Table 6**

Result comparison between TRI Reference and Q-Learning Strategy for $i_{cell}$ = 17.</caption><thead><tr><td>Method</td><td>Charge time (s)</td><td>Final SOC</td><td>Total heat (sum(HG))</td></tr></thead><tbody><tr><td>TRI Protocol</td><td>605</td><td>72.93%</td><td>303.89</td></tr><tr><td>Q-Learning</td><td>605</td><td>66.32%</td><td>243.08</td></tr></tbody></table>

while balancing heat constraints. Fig. 8(b) presents the final Q-table mapping between SOC states and charging current actions. Brighter regions represent higher Q-values, it shows the agent's learned preference for moderate charging currents at low SOC levels and more conservative actions as SOC increases. This distribution supports the strategy's health-aware nature, avoiding aggressive currents when the battery is more vulnerable. Fig. 8(c) provide a comparative analysis between the Q-Learning strategy and a traditional TRI reference under the same charging task. The Q-Learning approach shows more dynamic current adjustments, resulting in significantly lower battery temperatures and reduced heat generation throughout the charging period. In particular, the temperature and heat curves clearly indicate that the Q-Learning strategy effectively delays thermal buildup compared to the TRI baseline. SOC trajectories in both cases are similar, confirming that the learning-based method achieves comparable charging performance while better preserving battery thermal conditions.

Overall, the Q-Learning-based strategy demonstrates a clear advantage in controlling heat generation and temperature rise without compromising charging efficiency. This supports its potential as a viable, health-aware alternative to conventional fast-charging. Table 6 presents the detailed results. Within the same charging duration, the Q-Learning-based strategy achieved a final SOC that was 9.06% lower than the target value. However, this was accompanied by a notable reduction of 20.01% in total heat generation, highlighting a clear trade-off between charging completeness and thermal performance.

The experimental simulation was conducted across a selected subset of 10 representative fast charging protocols from the TRI database, as shown in Table 7. The table summarizes the final SOC error and the corresponding reduction in total heat generation when applying the Q-Learning-based charging strategy in comparison to each reference protocol.

The results reveal that the Q-Learning strategy consistently reduces the total heat generated during charging, achieving up to 26.78% reduction, while maintaining a relatively low deviation in final SOC, typically within 10% of the target value. This demonstrates a favorable trade-off between energy efficiency and thermal management. In several cases, such as the “5.4C(50%)3C” protocol, the Q-Learning strategy achieves near-zero SOC error while still delivering tangible reductions in heat generation. The findings provide qualitative evidence that the Q-Learning-based charging policy enhances health-aware by effectively lowering thermal stress under various fast charging conditions, thereby offering a promising alternative to conventional rule-based approaches for battery longevity and safety.

### 3.3. Discussion

The simulation results reveal that both the PMP-based and Q-Learning-based charging strategies can effectively minimize battery heat generation while fulfilling the target SOC requirements. However, their respective strengths and limitations warrant further analysis.

The PMP strategy offers a mathematically optimal control solution under the assumption of complete model knowledge and deterministic
<figure><img src="image_11.png" /><figcaption>Fig. 8. (a) Q-Learning results: training reward, final SOC, and heat generation; (b) Q-table visualization; (c) optimal strategy performance.</figcaption></figure>

<caption>**Table 7**</caption>

Results of reduced total generated heat by Q-Learning strategy during charging process compared to different TRI fast charging protocols.

<table><thead><tr><td>Ref. Protocol</td><td>Error on final SOC</td><td>Reduction on sum(HG)</td></tr></thead><tbody><tr><td>5.4C(60%)3.6C</td><td>9.06%</td><td>20.01%</td></tr><tr><td>5.4C(40%)3.6C</td><td>3.38%</td><td>7.48%</td></tr><tr><td>5.4C(50%)3C</td><td>0.11%</td><td>2.97%</td></tr><tr><td>6C(30%)3.6C</td><td>1.61%</td><td>4.73%</td></tr><tr><td>6C(50%)3C</td><td>4.18%</td><td>14.34%</td></tr><tr><td>7C(40%)3C</td><td>1.03%</td><td>11.26%</td></tr><tr><td>8C(35%)3.6C</td><td>9.22%</td><td>26.78%</td></tr><tr><td>2C(10%)6C</td><td>7.55%</td><td>25.61%</td></tr><tr><td>5.2C(66%)3.5C</td><td>9.88%</td><td>23.09%</td></tr><tr><td>5C(67%)4C</td><td>9.96%</td><td>21.93%</td></tr></tbody></table>

behavior. Its performance is particularly notable in terms of smooth current trajectories and consistent convergence toward minimum heat generation. Nevertheless, PMP's reliance on predefined system parameters and forward-backward numerical solutions restricts its adaptability in environments with high model uncertainty or real-time variations, such as fluctuating ambient temperature or aging-induced parameter drift.

On the other hand, the RL-based strategy exhibits strong adaptability, particularly in dynamic environments or under partial knowledge of the battery system. Through reward-guided exploration, the Q-Learning agent gradually learns effective charging patterns that balance heat suppression and charging efficiency. Although slight oscillations in current were observed due to discrete action spaces, the RL strategy achieved competitive or even superior performance in late-stage heat reduction, especially when temperature sensitivity was integrated into the reward function. Moreover, the closed-loop nature of RL suggests potential for real-world implementation with onboard feedback.

While the learned policy occasionally exhibits approximately 5 A step variations, such oscillations could exacerbate specific aging modes. In practice, the control command can be post-shaped with a first-order digital low-pass filter or move-blocking mechanism (e.g., 5–10

s hold) and quantized to the charger's current resolution. Both methods effectively reduce current ripple with negligible impact on heat minimization. In addition, reward shaping with ΔI penalties further suppresses oscillations without sacrificing SOC targets. Comparative results also highlight that heat minimization is strongly correlated with the shape of the charging current curve-stepwise, early-stage aggressive charging often results in a substantial increase in cumulative heat. Thus, both methods converge toward strategies that defer high current pulses to later stages where the temperature gain per unit current is lower. Our ECM does not explicitly capture lithium plating or SEI growth kinetics; although heat is used as a qualitative health proxy, these mechanisms may impose additional constraints for extreme fast charging. Future extensions will incorporate plating/SEI indicators or physics-informed penalties in the cost/reward to better reflect such degradation pathways.

For deployment, the proposed PMP-based strategy can run online with adaptive co-state updates and periodic parameter identification, whereas the RL policy supports continual learning and on-policy fine-tuning using onboard telemetry to track temperature and SOH drift. Both strategies therefore admit real-time adaptation under varying ambient temperatures and progressive aging, which is essential for practical EV fast-charging applications.

In summary, while PMP offers theoretical optimality under ideal conditions, RL demonstrates promising adaptability and scalability. A hybrid framework that initializes charging with PMP-derived profiles and gradually transitions to RL-based control under feedback conditions could be a promising direction for future work.

## 4. Conclusion

This study proposed and evaluated two health-aware charging strategies (PMP and Q-Learning) for Li-ion battery fast charging, aiming to minimize heat generation while ensuring charging completeness. A second-order ECM, coupled with a thermal model, was implemented in Simulink to simulate electro-thermal behavior during charging. The proposed strategies were benchmarked against conventional fast-charging protocols used in the TRI database.
Simulation results show that both PMP and Q-Learning strategies effectively reduce total heat generation with minimal SOC deviation. In particular, the Q-Learning strategy achieved over 20% average reduction in HG across multiple TRI protocols, while maintaining SOC error within 10%. Q-table visualization revealed the learned charging patterns adapt to varying SOC conditions, emphasizing safety and efficiency. These findings support the use of cumulative HG as a practical qualitative proxy for battery degradation, without requiring an explicit EOL-HG model.

We use the TRI fast-charging protocols as a benchmark baseline for comparative evaluation under equal time budgets. The ECM is calibrated for LFP and thermally validated, yet it is not parameterized from the exact TRI cells; thus, results quantify policy improvements relative to the TRI protocols rather than exact replication of TRI aging. This limitation is revisited in Section 4 with plans for protocol-specific recalibration. Future work will involve calibrating a more accurate ECM based on TRI cell parameters to improve model fidelity and experimental consistency. Additionally, this work does not attempt to construct a mathematical model between EOL and cumulative HG, due to the limited strength of the correlation and the influence of other contributing features. In future work, we plan to integrate additional degradation-related features and continue leveraging database-simulation hybrid approaches to simulate realistic battery aging mechanisms. These efforts aim to develop more sophisticated and robust health-aware charging strategies tailored to real-world fast-charging scenarios.

The proposed strategies are compatible with real-time applications and adaptable to diverse operating scenarios. This research lays the groundwork for the development of safer, more efficient, and durable fast-charging solutions for next-generation EVs.

## CRediT authorship contribution statement

**Xuelu Wang:** Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization.
**Toufik Azib:** Writing – review & editing, Visualization, Validation, Supervision, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization.
**Jianwen Meng:** Writing – review & editing, Visualization, Validation, Supervision, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization.
**Khelil Sidi-Brahim:** Writing – review & editing, Visualization, Validation, Supervision, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgments

This work was supported by the China Scholarship Council (CSC) under Grant No. 202208070104.

## Data availability

Data used is already in open source which is mentioned in the article and the reference.

## References

[1] M. Shafiei, A. Ghasemi-Marzbali, Fast-charging station for electric vehicles, challenges and issues: A comprehensive review, J. Energy Storage 49 (2022) 104136.

[2] D. Sbordone, I. Bertini, B. Di Pietra, M.C. Falvo, A. Genovese, L. Martirano, EV fast charging stations and energy storage technologies: A real implementation in the smart micro grid paradigm, Electr. Power Syst. Res. 120 (2015) 96–108.

[3] O. Elma, A dynamic charging strategy with hybrid fast charging station for electric vehicles, Energy 202 (2020) 117680.

[4] A. Ahmad, M. Khalid, Z. Ullah, N. Ahmad, M. Aljaidi, F.A. Malik, U. Manzoor, Electric vehicle charging modes, technologies and applications of smart charging, Energies 15 (2022) 9471, http://dx.doi.org/10.3390/en15249471.

[5] B. Bose, A. Garg, B.K. Panigrahi, J. Kim, Study on Li-ion battery fast charging strategies: Review, challenges and proposed charging framework, J. Energy Storage 55 (Part B) (2022) 105507.

[6] S. Geng, Y. Zhang, B. Xie, A. Shi, Y. Ning, S. Lou, G. Yin, Challenges and opportunities for fast-charging batteries, J. Phys. Chem. C 127 (2023) http://dx.doi.org/10.1021/acs.jpcc.3c01927.

[7] S.M. Shariff, M.S. Alam, S. Hameed, M. Khalid, A. Ahmad, E. Al-Ammar, I. Alsaidan, H. Alrajhi, A state-of-the-art review on the impact of fast EV charging on the utility sector, Energy Storage 4 (2021) http://dx.doi.org/10.1002/est2.300.

[8] P. Franzese, D.D. Patel, A. Mohamed, D. Iannuzzi, B. Fahimi, M. Risso, J. Miller, Fast DC charging infrastructures for electric vehicles: Overview of technologies, standards, and challenges, IEEE Trans. Transp. Electrific. (2023) http://dx.doi.org/10.1109/TTE.2023.3239224, 1–1.

[9] X. Huang, S. Tao, C. Liang, R. Ma, X. Wang, B. Xia, X. Zhang, Robust and generalizable lithium-ion battery health estimation using multi-scale field data decomposition and fusion, J. Power Sources 642 (2025) 236939, http://dx.doi.org/10.1016/j.jpowsour.2025.236939.

[10] T.S. Tao, M. Zhang, Z. Zhao, H. Li, R. Ma, Y. Che, X. Sun, L. Su, C. Sun, X. Chen, H. Chang, S. Zhou, Z. Li, H. Lin, Y. Liu, W. Yu, Z. Xu, H. Hao, S. Moura, G. Zhou, Non-destructive degradation pattern decoupling for early battery trajectory prediction via physics-informed learning, Energy Env. Sci. 18 (2025) http://dx.doi.org/10.1039/D4EE03839H.

[11] S. Tao, R. Ma, Z. Zhao, et al., Generative learning assisted state-of-health estimation for sustainable battery recycling with random retirement conditions, *Nat. Commun.* 15 (2024) 10154, http://dx.doi.org/10.1038/s41467-024-54454-0.

[12] S. Tao, R. Ma, Y. Chen, Z. Liang, H. Ji, Z. Han, G. Wei, X. Zhang, G. Zhou, Rapid and sustainable battery health diagnosis for recycling pretreatment using fast pulse test and random forest machine learning, *J. Power Sources* 597 (2024) 234156, http://dx.doi.org/10.1016/j.jpowsour.2024.234156.

[13] W. Xie, X. Liu, R. He, Y. Li, X. Gao, X. Li, Z. Peng, S. Feng, X. Feng, S. Yang, Challenges and opportunities toward fast-charging of lithium-ion batteries, J. Energy Storage 32 (2020) 101837.

[14] A. Tomaszewska, Z. Chu, X. Feng, S. O’Kane, X. Liu, J. Chen, C. Ji, E. Endler, R. Li, L. Liu, Y. Li, S. Zheng, S. Vetterlein, M. Gao, J. Du, M. Parkes, M. Ouyang, M. Marinescu, G. Offer, B. Wu, Lithium-ion battery fast charging: A review, eTransportation 1 (2019) 100011.

[15] T. Tanim, E. Dufek, M. Evans, C. Dickerson, A. Jansen, B. Polzin, A. Dunlop, S. Trask, R. Jackman, I. Bloom, Z. Yang, E. Lee, Extreme fast charge challenges for lithium-ion battery: Variability and positive electrode issues, J. Electrochem. Soc. 166 (2019) A1926–A1938, http://dx.doi.org/10.1149/2.0731910jes.

[16] C.-Y. Wang, T. Liu, X.-G. Yang, S. Ge, N.V. Stanley, E.S. Rountree, Y. Leng, B.D. McCarthy, Fast charging of energy-dense lithium-ion batteries, Nature 611 (7936) (2022) 485–490.

[17] Y. Gao, X. Zhang, Q. Cheng, B. Guo, J. Yang, Classification and review of the charging strategies for commercial lithium-ion batteries, IEEE Access 7 (2019) 43511–43524, http://dx.doi.org/10.1109/ACCESS.2019.2906117.

[18] N. Wassiliadis, J. Schneider, A. Frank, L. Wildfeuer, X. Lin, A. Jossen, M. Lienkamp, Review of fast charging strategies for lithium-ion battery systems and their applicability for battery electric vehicles, J. Energy Storage 44 (Part B) (2021) 103306.

[19] Y. Li, K. Li, Y. Xie, J. Liu, C. Fu, B. Liu, Optimized charging of lithium-ion battery for electric vehicles: Adaptive multistage constant current–constant voltage charging strategy, Renew. Energy 146 (2020) 2688–2699, http://dx.doi.org/10.1016/j.renene.2019.08.077.

[20] N. Ghaeminezhad, M. Monfared, Charging control strategies for lithium-ion battery packs: Review and recent developments, IET Power Electron. 15 (2021) http://dx.doi.org/10.1049/pel2.12219.

[21] Y. Wang, Z. Sun, Z. Chen, Development of energy management system based on a rule-based power distribution strategy for hybrid power sources, Energy 175 (2019) 1055–1066, http://dx.doi.org/10.1016/j.energy.2019.03.155.

[22] H.E. Oliveira Farias, C.A. Sepulveda Rangel, L. Weber Stringini, L. Neves Canha, D. Pegoraro Bertineti, W. da Silva Brignol, Z. Iensen Nadal, Combined framework with heuristic programming and rule-based strategies for scheduling and real-time operation in electric vehicle charging stations, Energies 14 (2021) 1370, http://dx.doi.org/10.3390/en14051370.
[23] B. Zou, J. Peng, S. Li, Y. Li, J. Yan, H. Yang, Comparative study of the dynamic programming-based and rule-based operation strategies for grid-connected PV-battery systems of office buildings, Appl. Energy 305 (2022) 117875.

[24] A. Bäumler, A. Benterki, J. Meng, T. Azib, M. Boukhnifer, Energy management strategies based on soft actor critic reinforcement learning with a proper reward function design based on battery state of charge constraints, J. Energy Storage 90 (Part A) (2024) 111797, http://dx.doi.org/10.1016/j.est.2024.111797.

[25] S. Pramanik, S. Anwar, Electrochemical model based charge optimization for lithium-ion batteries, J. Power Sources 313 (2016) 164-177, http://dx.doi.org/10.1016/j.jpowsour.2016.01.096.

[26] K. Liu, C. Zou, K. Li, T. Wik, Charging pattern optimization for lithium-ion batteries with an electrothermal-aging model, IEEE Trans. Ind. Informat. PP (2018) http://dx.doi.org/10.1109/TII.2018.2866493, 1-1.

[27] J. Yan, G. Xu, H. Qian, Y. Xu, Battery fast charging strategy based on model predictive control, in: Proc. IEEE Veh. Technol. Conf, 2010, pp. 1-8, http://dx.doi.org/10.1109/VETECF.2010.5594382.

[28] B. Shuai, Q. Zhou, J. Li, Y. He, Z. Li, H. Williams, H. Xu, S. Shuai, Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning, Appl. Energy 267 (2020) 114900.

[29] A. Salari, M. Zeinali, M. Marzband, Model-free reinforcement learning-based energy management for plug-in electric vehicles in a cooperative multi-agent home microgrid with consideration of travel behavior, Energy 288 (2023) 129725, http://dx.doi.org/10.1016/j.energy.2023.129725.

[30] T. Liu, X. Hu, W. Hu, Y. Zou, A heuristic planning reinforcement learning-based energy management for power-split plug-in hybrid electric vehicles, IEEE Trans. Ind. Informat. 15 (12) (2019) 6436-6445, http://dx.doi.org/10.1109/TII.2019.2903098.

[31] B. She, F. Li, H. Cui, J. Zhang, R. Bo, Fusion of microgrid control with model-free reinforcement learning: Review and vision, IEEE Trans. Smart Grid PP (2022) http://dx.doi.org/10.1109/TSG.2022.3222323, 1-1.

[32] M. Gheisarnejad, A. Fathollahi, M. Sharifzadeh, E. Laurendeau, K. Al-Haddad, Data-driven switching control technique based on deep reinforcement learning for packed E-cell as smart EV charger, IEEE Trans. Transp. Electrific. (2024) http://dx.doi.org/10.1109/TTE.2024.3435763.

[33] A. Vishnu, R.M.L. George, P. Gopakumar, Reinforcement learning based energy management system for hybrid electric vehicles, 2023, pp. 202-208, http://dx.doi.org/10.1109/ACCESS57397.2023.10200701.

[34] J. Gan, S. Li, C. Wei, L. Deng, X. Tang, Intelligent learning algorithm and intelligent transportation-based energy management strategies for hybrid electric vehicles: A review, IEEE Trans. Intell. Transp. Syst. (2023) 1-17, http://dx.doi.org/10.1109/TITS.2023.3283010.

[35] Y. Li, T. Wik, Y. Huang, C. Zou, Nonlinear model inversion-based output tracking control for battery fast charging, IEEE Trans. Control Syst. Technol. 32 (2024) 225-240, http://dx.doi.org/10.1109/TCST.2023.3306240.

[36] Y. Li, D.M. Vilathagamuwa, E. Wikner, Z. Wei, X. Zhang, T. Thiringer, T. Wik, C. Zou, Electrochemical model-based fast charging: Physical constraint-triggered PI control, IEEE Trans. Energy Convers. 36 (4) (2021) 3208-3220, http://dx.doi.org/10.1109/TEC.2021.3065983.

[37] N. Wassiliadis, J. Kriegler, K.A. Gamra, M. Lienkamp, Model-based health-aware fast charging to mitigate the risk of lithium plating and prolong the cycle life of lithium-ion batteries in electric vehicles, J. Power Sources 561 (2023) 232586, http://dx.doi.org/10.1016/j.jpowsour.2022.232586.

[38] S. Park, A. Pozzi, M. Whitmeyer, H. Perez, A. Kandel, G. Kim, Y. Choi, W.T. Joe, D.M. Raimondo, S. Moura, A deep reinforcement learning framework for fast charging of Li-ion batteries, IEEE Trans. Transp. Electr. 8 (2) (2022) 2770-2784, http://dx.doi.org/10.1109/TTE.2022.3140316.

[39] N. Nicodemo, R. Di Rienzo, M. Lagnoni, A. Bertei, F. Baronti, Estimation of lithium-ion battery electrochemical properties from equivalent circuit model parameters using machine learning, J. Energy Storage 99 (2024) 113257, http://dx.doi.org/10.1016/j.est.2024.113257.

[40] L. Barzacchi, M. Lagnoni, R. Di Rienzo, A. Bertei, F. Baronti, Enabling early detection of lithium-ion battery degradation by linking electrochemical properties to equivalent circuit model parameters, J. Energy Storage 50 (2022) 104213, http://dx.doi.org/10.1016/j.est.2022.104213.

[41] X. Zhang, W. Zhang, G. Lei, A review of li-ion battery equivalent circuit models, Trans. Electr. Electron. Mater. 17 (2016) 311-316, http://dx.doi.org/10.4313/TEEM.2016.17.6.311.

[42] T.K. Dong, A. Kirchev, F. Mattera, J. Kowal, Y. Bultel, Dynamic modeling of Li-ion batteries using an equivalent electrical circuit, J. Electrochem. Soc. 158 (3) (2011) A326, http://dx.doi.org/10.1149/1.3543710.

[43] C. Forgez, D.V. Do, G. Friedrich, M. Morcrette, C. Delacourt, Thermal modeling of a cylindrical LiFePO4/graphite lithium-ion battery, J. Power Sources 195 (9) (2010) 2961-2968, http://dx.doi.org/10.1016/j.jpowsour.2009.10.105.

[44] X. Hu, S. Li, H. Peng, A comparative study of equivalent circuit models for Li-ion batteries, J. Power Sources 198 (2012) 359-367, http://dx.doi.org/10.1016/j.jpowsour.2011.10.013.

[45] T. Feng, L. Yang, X. Zhao, H. Zhang, J. Qiang, Online identification of lithium-ion battery parameters based on an improved equivalent-circuit model and its implementation on battery state-of-power prediction, J. Power Sources 281 (2015) http://dx.doi.org/10.1016/j.jpowsour.2015.01.154.

[46] S.S. Madani, E. Schaltz, S. Knudsen Kær, An electrical equivalent circuit model of a lithium titanate oxide battery, Batteries 5 (2019) 31, http://dx.doi.org/10.3390/batteries5010031.

[47] J. Gomez, R. Nelson, E. Kalu, M.H. Weatherspoon, J. Zheng, Equivalent circuit model parameters of a high-power Li-ion battery: Thermal and state of charge effects, J. Power Sources 196 (2011) 4826-4831, http://dx.doi.org/10.1016/j.jpowsour.2010.12.107.

[48] X. Zhang, J. Lu, S. Yuan, J. Yang, X. Zhou, A novel method for identification of lithium-ion battery equivalent circuit model parameters considering electrochemical properties, J. Power Sources 345 (2017) 21-29, http://dx.doi.org/10.1016/j.jpowsour.2017.01.126.

[49] L. Zhang, H. Peng, Z. Ning, Z. Mu, C.-Y. Sun, Comparative research on RC equivalent circuit models for lithium-ion batteries of electric vehicles, Appl. Sci. 7 (2017) 1002, http://dx.doi.org/10.3390/app7101002.

[50] S. Nejad, A systematic review of lumped-parameter equivalent circuit models for real-time estimation of lithium-ion battery states, J. Power Sources 316 (2016) 183-196, http://dx.doi.org/10.1016/j.jpowsour.2016.03.042.

[51] H. Lee, C. Song, N. Kim, Comparative analysis of energy management strategies for HEV: Dynamic programming and reinforcement learning, IEEE Access 8 (2020) 67112-67123, http://dx.doi.org/10.1109/ACCESS.2020.2986205.

[52] Y. Hu, W. Li, K. Xu, Energy management strategy for a hybrid electric vehicle based on deep reinforcement learning, Appl. Sci. 8 (2) (2018) 187, http://dx.doi.org/10.3390/app8020187.

[53] C. Qi, Y. Zhu, C. Song, Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle, Energy 238 (Part A) (2022) 121703, http://dx.doi.org/10.1016/j.energy.2021.121703.

[54] S. Inuzuka, F. Xu, B. Zhang, Reinforcement learning based on energy management strategy for HEVs, in: Proc. 2019 IEEE Vehicle Power and Propulsion Conf, VPPC, Hanoi, Vietnam, 2019, pp. 1-6, http://dx.doi.org/10.1109/VPPC46532.2019.8952511.

[55] A. Bäumler, A. Benterki, J. Meng, T. Azib, M. Boukhnifer, Energy management strategy based on reinforcement learning for fuel cell hybrid vehicle with a new reward function approach, in: Proc. 2023 IEEE Vehicle Power and Propulsion Conference, VPPC, San Diego, CA, USA, 2023, pp. 01-06, http://dx.doi.org/10.1109/VPPC58343.2023.10378413.

[56] K.A. Severson, P.M. Attia, N. Jin, et al., Data-driven prediction of battery cycle life before capacity degradation, Nat Energy 4 (2019) 383-391, http://dx.doi.org/10.1038/s41560-019-0356-8.