# Comparative of control strategies on electrical vehicle fleet charging management strategies under uncertainties

Zhewei Zhang, Rémy Rigo-Mariani, Nouredine Hadjsaid

► To cite this version:

Zhewei Zhang, Rémy Rigo-Mariani, Nouredine Hadjsaid. Comparative of control strategies on electrical vehicle fleet charging management strategies under uncertainties. Energy and AI, 2025, 21, 10.1016/j.egyai.2025.100522. hal-05065763

HAL Id: hal-05065763

https://hal.science/hal-05065763v1

Submitted on 13 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

# Comparative of control strategies on electrical vehicle fleet charging management strategies under uncertainties

Zhewei Zhang<sup>a,b,c</sup>, Rémy Rigo-Mariani<sup>a,*</sup>, Nouredine Hadjsaid<sup>a</sup>

## HIGHLIGHTS

* Mitigation of the uncertainty of users' inputs in EV fleet charging management by diffusion model.

* Mitigation of future arrivals uncertainties in EV fleet charging management by Gaussian mixture model.

* Comparison of three controls strategies based on model predictive control and/or data-driven techniques.

## ARTICLE INFO

### Keywords:

Electrical vehicles fleet

Uncertainty mitigation

Data driven, Model predictive control

Deep reinforcement learning

## GRAPHICAL ABSTRACT

<figure><img src="image_4.png" /></figure>

## ABSTRACT

The growing penetration of Electric Vehicles (EVs) in transportation brings challenges to power distribution systems due to uncertain usage patterns and increased peak loads. Effective EV fleet charging management strategies are needed to minimize network impacts, such as peak charging power. While existing studies have addressed uncertainties in future arrivals, they often overlook the uncertainties in user-provided inputs of current ongoing charging EVs, such as estimated departure time and energy demand. This paper analyzes the impact of these uncertainties and evaluates three management strategies: a baseline Model Predictive Control (MPC), a data-hybrid MPC, and a fully data-driven Deep Reinforcement Learning (DRL) approach. For data-hybrid MPC, we adopted a diffusion model to handle user input uncertainties and a Gaussian Mixture Model for modeling arrival/departure scenarios. Additionally, the DRL method is based on a Partially Observable Markov Decision Process (POMDP) to manage uncertainty and employs a Convolutional Neural Network (CNN) for feature extraction. Robustness tests under different user uncertainty levels show that the data hybrid MPC performs better on the baseline MPC by 20 %, while the DRL-based method achieves around 10 % improvement.

### Abbreviations

EV Electrical Vehicles

CCE Current Charging Events

FCE Future Charging Events

MPC Model Predictive Control

MLP Multi-Layer Perceptions

GMM Gaussian Mixture Model

DRL Deep Reinforcement Learning

CNN Convolutional Neural Network

MDP Markov Decision Problem

POMDP Partially Observable Markov Decision Problem

<footnote><sup>a</sup> Univ. Grenoble Alpes, CNRS, Grenoble INP, G2Elab, 38000 Grenoble, France</footnote>

<footnote><sup>b</sup> CNRS@CREATE, 1 Create Way, #08-01 Create Tower 138602, Singapore</footnote>

<footnote><sup>c</sup> Nanyang Technological University, 639798, Singapore</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: remy.rigo-mariani@g2elab.grenoble-inp.fr (R. Rigo-Mariani).</footnote>
# 1. Introduction

## 1.1. Motivation & backgrounds

The share of total sales of electrical vehicles (EVs) has increased significantly in recent years. In Europe, the registration of EVs increased by nearly 20% in 2023 compared with 2022[1]. According to the global EV Outlook, >25% of vehicles on the road worldwide will be EVs by 2035, and the total European electricity demand for EV charging will account for 13.7%~14.5% of the total load (compared with 1.1% only in 2023) [1]. This growth challenges electricity generation in terms of volume and power and network constraints with more volatile load profiles and potentially greater peak load levels [2,3]. Therefore, controlled charging for the EV fleet is necessary, with one typical objective being to minimize the aggregated charging power (i.e. lower peak load) to reduce the impact of EVs on the power system while supplying as much energy as possible to the users. For example, Liu et al. considered a joint planning of charging infrastructure and charging strategies [4], and with the controlled charging, the charging cost and peak demand are reduced. In [5], Das and Kayal considered controlled charging in distribution grid and reduce the peak-to-average ratio of demand curve.

From the operational perspective, one major concern is that the control of the EV fleet suffers from various uncertainties. The first source of uncertainties lies in the impossibility of predicting accurately the numbers of future EV arrivals at the charging ports - i.e. EV arrival time, departure time and energy demand of each EV [6]. In this paper, this uncertainty is denoted as "future uncertainty". Many studies focus on that specific uncertainty from the control perspective, and various methods, such as cluster model [7,8], recurrent neural network [9,10], statistic model [11,12] and generative model [13,14], are implemented to forecast the future arrivals to mitigate this uncertainty. In actual operation, another source of uncertainty arises from the EV already connected to the charging ports. That source of uncertainty is oftentimes neglected in the literature that assumes perfect knowledge of those EVs' departure time and energy demand, which is far from realistic. In [15], the researchers attempted to consider "user uncertainty" with EV owners unwilling to provide their departure time and energy demand when they connect to the ports. In [16], the researchers found that EV users might provide incorrect departure time and energy demand, which are different from real charging behavior, as Fig. 1 shows. From the figure, it is obvious that the user input does not always equal the actual value.

In summary, the increasing EV charging demand brings the necessity of management strategy, while the control strategy is suffered from the user and future uncertainties. Our research aims to propose and compare

management strategies that can minimize the uncertainty impacts.

## 1.2. Control strategies

### 1.2.1. MPC-based methods

One main family of methods typically considered for EV charging relates to Model Predictive Control (MPC). With the forecast of future EVs arrivals along a given horizon, MPC typically solves an optimization problem to compute a sequence of control signals sent to the charging ports. The procedure can be run on a rolling windows basis with successive forecast updates throughout the day. For example, [17] and [18] proposed an approach for EV fleet charging management to reduce peak charging power by valley-filling. In these works, the EVs are divided into two categories: current and future charging events. Any charging event starts at the EV's "arrival time" and ends at the EV's "departure time" with a certain amount of energy delivered to the EV. The current charging events include all EVs connected at a given time when the MPC is run (i.e. relating to the "user uncertainty"). The future charging events include all forecast arrivals – i.e. those whose arrival times are later than the current time. They can be represented through scenario generations fed to the MPC optimization. For example, C. Wu et al. sampled scenarios from assumed Gaussian distributions [19] and Engel et al. sampled scenarios from Gaussian Mixture Models (GMMs), which are fitted from real-world charging records [20]. Machine learning is another way to mitigate future uncertainty with predictions. McClone et al. ensembled the TreeBagger function, the Long Short Term Memory (LSTM) model and the K-Nearest Neighbor (KNN) to forecast future charging events and then forward the forecast results to the MPC [21]. Li et al. used a denoising diffusion model to forecast future charging events, and they tested their result with MPC [22]. Only a few existing studies on the MPC method have considered the influence of user uncertainty. For example, in [23], researchers considered a robust multi-objective optimization considering the user to minimize the peak power of the charging station and maximize the overall revenue of the charging station. However, in this work, the complexity of the optimization is increased by adding peak power as a penalty and adding a robust auxiliary variable in the objective function. Moreover, future charging events are neglected in this work. However, the research into the user uncertainty is lacking.

### 1.2.2. DRL-based methods

With the rise of data-driven for control applications, Deep Reinforcement Learning (DRL) is another approach typically used for EV fleet charging management. In DRL-based methods, a control agent learns a policy from successive exchanges with the controlled system by

<figure><img src="image_5.png" /><figcaption>Fig. 1. Difference of user inputs and actual charging event at (a) Departure time and (b) Energy demand.</figcaption></figure>
maximizing a reward function based on observations. The training can be performed offline with historical data. During the real-time deployment, the agent generates actions from the policy learnt online while considering actual system observations. Most existing literature focuses on using Markov Decision Problem (MDP) or Constrained MDP to deal with future uncertainty. For example, [24] adopted Offline DRL to maximize profit of charging station; [25] adopted Transfer Learning with DRL to enable using small dataset to achieve better performance, and [26] used Lagrangian-based DRL with safe filter to ensure constraints of charging station operation. Some literature considered user uncertainty in the DRL problem formulation. Tuchnitz et al. proposed a DRL method for peak power minimizing and charging power curve smoothing for EV fleet charging management [27]. They used quartile deviation to address the user uncertainty while defining the lower and upper limits of uncertain departure time and energy demand for current charging events. The agent is trained with a Deep Q-Network (DQN) algorithm. In [15], the researchers considered the actual departure time and State of Charge (SOC) of each EV as unobservable variables and formulated a Partially Observable Markov Decision Problem (POMDP) for DRL. The agent is trained with the Proximal Policy Optimization (PPO) algorithm. In [28], Claessens et al. proposed a Batch-RL algorithm to shave the charging peak. In their work, they did not require departure time but required the exact SOC for each EV at the problem formulation. And the summary of MPC and DRL methods are shown in Table 1, where the scale is the average hosted EV per day or total charging ports.

### 1.3. Contributions

Both MPC and DRL methods have their pros and cons. For the MPC-based methods, we can get deterministic results from a deterministic problem while it suffers from the optimization problem formulation, e.g., convexity, linearity, and scaling problems. The DRL-based methods do not rely on convexity or linearity of the model but will return to a stochastic result, depending on the reward function design [29]. Moreover, due to the nature of DRL, which requires a fixed shape of the state (or observation) space, it is hard to include the forecast of future charging events in the MDP or POMDP because of the unknown number of future charging events. Therefore, considering both user and future uncertainty, one interesting question is whether MPC or DRL performs better. This paper proposes and compares three strategies for controlling an EV fleet to mitigate user and future uncertainty. In particular, hybrid approaches will be investigated to tackle the disadvantages of MPC and data-driven methods for an EV fleet charging management problem based on real-world data [30]. The management objective for MPC and DRL is to minimize peak charging power while supplying as much energy as possible to the EV fleet and mitigating both “user” and “future” uncertainties. The main contributions are:

* We analyzed the influence of the user and future uncertainties. We mitigated the user uncertainty with a diffusion model and the future

uncertainty by generating scenarios from GMMs, and we proposed a data hybrid MPC that combines both.

* We implemented a fully data-driven approach for the EV fleet management problem by DRL. We used POMDP to address user uncertainty. We implemented a Convolutional Neural Network (CNN) to extract features from a large observation space.

* We compared the baseline MPC approach, data hybrid MPC approach, and fully data-driven performance under different user uncertainty levels. The data hybrid MPC has the best performance, delivering around 95 % energy to the EV fleet while reducing >20 % peak charging power. The fully data-driven method comes in second, delivering 85 % energy to the EV fleet while reducing around 20 % peak charging power.

The paper is organized as follows: Section 2 introduces our problem definition and user uncertainty assumptions. In Section 3, we introduce the control strategies, which include the baseline MPC approach, the data hybrid MPC approach and the fully data-driven approach by DRL. In Section 4, we introduce our experiment setup and provide the numerical results for comparison of three approaches before conclusions are drawn in Section 5.

## 2. EV fleet charging management problem definition

### 2.1. Case study and charging events mathematical definition

The case study in this paper is illustrated in Fig. 2. We consider a centralized controller that manages the charging power for a set of charging ports Π, with the size of $N_p$. The controller’s objective is to minimize the peak of the aggregated power $P_t^{\text{CSE}}$ at the charging station over a day (temporal set $t \in T$ with $\Delta t=15$ min). We adopt similar formulation with [18], where EVs are classified into current and future charging events. At each timestep, the control takes information about uncertainties of Current Charging Events (CCEs) and forecast of Future Charging Events (FCEs) to calculate the control command. Every CCE is characterized by the arrival/departure times and the energy demand – i. e. the tuple $(T_c^{\text{arr}}, T_c^{\text{dep}}, E_c)$. The user uncertainty for current events arises from the difference between the departure time and energy demand declare by the EV users, and their actual values. Therefore, at the controller level, the decision is made with user inputs, the uncertain tuples $(T_c^{\text{arr}}, \tilde{T}_c^{\text{dep}}, \tilde{E}_c)$, and $N_c$ is the total number of CCEs. One example of the user uncertainty is shown in Fig. 3. When an EV connects to the corresponding charging port, the controller can only get $\tilde{T}_c^{\text{dep}}, \tilde{E}_c$ which are provided by the EV user. Then the controller while make decision based on these inputs rather than $T_c^{\text{dep}}, E_c$. Therefore, the controller is influenced by the user uncertainty.

Similar to the CCE, every FCE is characterized by the tuple $(T_f^{\text{arr}}, T_f^{\text{dep}},$

<table><caption>**Table 1**
Summary of literature review.</caption><thead><tr><th>Ref.</th><th>Objective</th><th>Methods</th><th>User uncertainty</th><th>Future uncertainty</th><th>Scale</th><th>EV dataset source</th></tr></thead><tbody><tr><td>[17,18]</td><td>Valley-filling</td><td>MPC</td><td>No</td><td>Yes</td><td>20 EVs</td><td>Real</td></tr><tr><td>[19]</td><td>Power exchange</td><td>MPC</td><td>No</td><td>Yes</td><td>300 EVs</td><td>Generated</td></tr><tr><td>[20]</td><td>Cost, comfort, degradation</td><td>MPC</td><td>Yes</td><td>Yes</td><td>52 Ports</td><td>Real</td></tr><tr><td>[21]</td><td>Cost</td><td>MPC</td><td>No</td><td>Yes</td><td>48 Ports</td><td>Real</td></tr><tr><td>[22]</td><td>Cost</td><td>MPC</td><td>No</td><td>Yes</td><td>28 EVs</td><td>Real</td></tr><tr><td>[23]</td><td>Cost, peak power</td><td>MPC</td><td>Yes</td><td>No</td><td>~30 EVs</td><td>Real</td></tr><tr><td>[15]</td><td>Cost, demand</td><td>DRL</td><td>Yes</td><td>Yes</td><td>~3.4 EVs</td><td>Real</td></tr><tr><td>[24]</td><td>Cost</td><td>DRL</td><td>No</td><td>Yes</td><td>100 Ports</td><td>Generated</td></tr><tr><td>[25]</td><td>Cost, demand</td><td>DRL</td><td>No</td><td>Yes</td><td>~58 EVs</td><td>Real</td></tr><tr><td>[26]</td><td>Cost</td><td>DRL</td><td>No</td><td>Yes</td><td>100 Ports</td><td>Generated</td></tr><tr><td>[27]</td><td>Peak power</td><td>DRL</td><td>Yes</td><td>Yes</td><td>50 EVs</td><td>Generated</td></tr><tr><td>[28]</td><td>Peak power</td><td>DRL</td><td>Yes</td><td>Yes</td><td>100/160 EVs</td><td>Generated</td></tr><tr><td>Our work</td><td>Peak power, energy</td><td>Data hybrid MPC</td><td>Yes</td><td>Yes</td><td>52 Ports (~120 EVs)</td><td>Real</td></tr></tbody></table>
<figure><img src="image_6.png" /><figcaption>Fig. 2. General case study of EV fleet charging management problem.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 3. User uncertainty illustration.</figcaption></figure>

$E_f$). In that case, all the event parameters are subjected to uncertainty at the controller level as the arrival time cannot be known accurately, which is noted with the uncertain tuple $(\tilde{T}_f^{\text{arr}}, \tilde{T}_f^{\text{dep}}, \tilde{E}_f)$. Also, the number of FCEs $\tilde{N}_F$ is uncertain.

The controls ultimately take the form of power setpoints $P_{p,t}$ to the charging ports, where $p$ is the charging port and $p \in \Pi$. The charging ports deliver the same amount of power to the connecting EVs, which is noted as $P_{c,t}$, where $c \in \tilde{C}$. In this work, we considered public charging stations. Control signals are sent to the charging ports displaying EV connection regardless of EV ownership. In other words, we do not consider controlling a dedicated EV fleet but any EV that arrives at the charging station. The charging port occupancy is implicitly considered through the “charging events” communicated to the central controller every time a new EV connects.

## 2.2. User uncertainty assumptions

In order to account for CCE uncertainty, we assume that $\tilde{T}_c^{\text{dep}}$ and $\tilde{E}_c$ follows the Gaussian distribution around the actual value with Eqs. (1) and (2), where $q(\cdot)$ is the probability distribution function, $\sigma_T$ is the departure time error (in h) and $\sigma_E$ is the energy demand error (in kWh). By adjusting the errors values of $\sigma_T$ and $\sigma_E$, we can get different levels of user uncertainty quantified with the error metric $e_C$ (in %) by (3) when comparing the user's inputs and actual values. For example, $e_C$ for the dataset [16] is around 54.08 %.

$$q(\tilde{T}_c^{\text{dep}} | T_c^{\text{dep}}) = N(\tilde{T}_c^{\text{dep}}; T_c^{\text{dep}}, \sigma_T), \forall c \in \tilde{C} \quad (1)$$

$$q(\tilde{E}_c | E_c) = N(\tilde{E}_c; E_c, \sigma_E), \forall c \in \tilde{C} \quad (2)$$

$$e_C = \frac{1}{N_C} \sum_{i=1}^{N_C} \left[ \max \left( \left| 1 - \frac{\tilde{T}_c^{\text{dep}}}{T_c^{\text{dep}}} \right| \times 100, \left| 1 - \frac{\tilde{E}_c}{E_c} \right| \times 100 \right) \right] \quad (3)$$

# 3. Control strategies

## 3.1. Generic model predictive control (MPC) formulation

A generic MPC approach is formulated as an optimization problem following Eqs. (4) ~ (10). The objective (4) is to minimize the peak power of the charging station along the prediction horizon, which in our case is a shrinking horizon – i.e. from the current time $t_s$ until the end of the day $T_{end}$. The MPC is then run on a daily basis with a resolution $\Delta t$ of 15 min, successively on the shrinking horizon mentioned above. Constraints (5) and (6) are charging power constraints and energy demand constraint for the CCEs. $\Delta E_{c,t_s}$ is an input parameter of the generic MPC and keeps tracks of the delivered energy to CCE $c$ from their arrival $T_c^{\text{arr}}$ until $t_s$. As we consider a homogenous EV fleet, the maximum charging power of all EVs and charging ports is set to a single value $\bar{P}^{\text{ev}}$. Similarly, we define the charging power and energy demand constraint for the FCEs with (7) and (8). We match the charging power of port to the CCE by Constraint (9), with a connection matrix $M$. If CCE $c$ is connected to the port $p$ then $m_{cp}$ is set to 1, otherwise $m_{cp}$ is 0. By summing up the charging power of all charging ports and all FCE, we can get the sequence of $P_t^{\text{CS}}$ for the prediction horizon, which is calculated by (10).

$$\min \left( \max_{t \in [t_s, T_{end}]} P_t^{\text{CS}} \right) \quad (4)$$

$$\begin{cases} 0 \le P_{c,t} \le \bar{P}^{\text{ev}}, \forall t \in [T_c^{\text{arr}}, \tilde{T}_c^{\text{dep}}], \forall c \in \tilde{C} \\ P_{c,t} = 0, \forall t \notin [T_c^{\text{arr}}, \tilde{T}_c^{\text{dep}}] \end{cases} \quad (5)$$
$$ \sum_{t=t_s}^{\tilde{T}_c^{\text{dep}}} P_{c,t} \times \Delta t = \tilde{E}_c - \Delta E_{c,t_s}, \forall c \in \tilde{C}, t \in [t_s, T_{\text{end}}] \quad (6) $$

$$ \left\{ \begin{array}{l} 0 \le P_{f,t} \le \bar{P}^{\text{ev}}, \forall t \in [\tilde{T}_f^{\text{arr}}, \tilde{T}_f^{\text{dep}}] \\ P_{f,t} = 0, \forall t \notin [\tilde{T}_f^{\text{arr}}, \tilde{T}_f^{\text{dep}}] \end{array} \right. , \forall f \in \tilde{F} \quad (7) $$

$$ \sum_{t=t_s}^{\tilde{T}_f^{\text{dep}}} P_{f,t} \times \Delta t = \tilde{E}_f, \forall f \in \tilde{F}, t \in [t_s, T_{\text{end}}] \quad (8) $$

$$ P_{p,t} = P_{c,t} \times m_{cp}, \forall p \in \Pi, c \in \tilde{C}, t \in [t_s, T_{\text{end}}] \quad (9) $$

$$ P_t^{\text{CS}} = \sum_{p=1}^{N_p} P_{p,t} + \sum_{f=1}^{N_f} P_{f,t}, \forall t \in [t_s, T_{\text{end}}] \quad (10) $$

## 3.2. Baseline MPC

The diagram of the baseline MPC approach is shown in Fig. 4. For every timestep $t_s$, the generic MPC is run with the users information of all CCEs at the charging station $\tilde{C}$, delivered energy of each CCE $\Delta E_{c,t_s}$ and the connection matrix of CCEs to ports $M$, and forecast of FCES $\tilde{F}$. $\Delta E_{c,t_s}$ is calculated by (11) in each timestep. We adopt results of the deterministic forecast of $\tilde{F}$ from the EDF Lab France [11] for FCES. Baseline MPC approach calculates the control signals $P_{p,t}$ to each port within prediction horizon by solving (4) ~ (10). As the baseline MPC runs on a rolling basis with a shrinking horizon, only the first element of control signal sequences $P_{1:N_p, t_s}$ is actually sent to each charging port.

$$ \Delta E_{c,t_s} = \Delta E_{c,t_s-1} + m_{cp} \times P_{p,t_s} \times \Delta t \quad (11) $$

## 3.3. Data hybrid MPC

This paper adopts a data-driven approach to mitigate both “user” and “future” uncertainties in the EV fleet management problem. We mitigate the user uncertainty by a diffusion model while the future uncertainty is accounted for by a scenario-based approach realized by fitting Gaussian Mixture Models (GMMs) from historical data.

### 3.3.1. Diffusion model for CCE user uncertainty reduction

The diffusion model is one emerging kind of generative model. The idea is to reach a Gaussian distribution by gradually adding noise to the original data and then to learn to sample back from noisy data (which follow the Gaussian distribution) to recover the original data. The noise-adding process is called the “forward process”, and the noise-reduction process is called the “reverse process”. The diffusion model has been widely deployed in image recovery from noise for computer vision [31,

[32], geographical sensing [33] and medical sensing [34].

This paper adopt the method from [31] and [32]. The reason why we adopt the diffusion model is that the diffusion model constructs a Markov chain with a series of uncertain values and tries to retrieve the original values based on these uncertain values. Fig. 5 shows the Markov chain and brief training idea of the proposed diffusion model, where $x^{[k]}$ is the k-th state of the Markov chain and the total diffusion step K is a hyperparameter. The difference between the proposed and the conventional diffusion model is that we do not add noise until we get a Gaussian distribution at the final state $x^{[K]}$. Instead, we set the user input departure and energy demand to the final state $x^{[K]}$ and actual departure and energy demand to the initial state $x^{[0]}$. In short, we trained a diffusion model that included the uncertain values in the middle of the Markov chain and the actual values at the start of the Markov chain.

The detailed training process is shown in Alg. 1. The training target is to train a model that can estimate $x^{[0]}$ based on $x^{[k]}$ and k. Then, the training process can be divided into two parts. The first is to sample and calculate a random intermediate state $x^{[k]}$ shown in Fig. 5, the second is to tune the model parameters to estimate the value of $x^{[0]}$. During training process, we first sample a batch of actual values C and user values $\tilde{C}$ from historical charging events records. Then we assign C to $x^{[0]}$ and $\tilde{C}$ to $x^{[K]}$ as mentioned above. To calculate $x^{[k]}$, we assume that for each diffusion step k, the amount of noise $\epsilon^{[k]}$ added at each state are roughly equal (12). Based on the k and $\epsilon^{[k]}$, we can calculate any $x^{[k]}$ by (13), where $\beta^{[k]}$ is the diffusion coefficient for simple notification. The details of derivations of (13) are shown in Appendix 7.1. As we worked on data with two dimensions (departure and energy demand by user input) rather than image, instead of using U-Net in [35], we use a Multi-Layer-Perceptron (MLP) with one hidden layer of 16 neurons as denoising network $f_0$ to estimate the $x_0$ based on $x_k$ and k, namely (14), where $\theta$ is parameters of the denoising neural network. We also include the $T_c^{\text{arr}}$ as one input feature of $f_0$. Our learning target is to minimize the Sum Square Error (SSE) between the estimated and actual value by gradient descending (15).

$$ \epsilon^{[k]} \approx \frac{x_K - x_0}{K} \quad (12) $$

$$ \left\{ \begin{array}{l} x^{[k]} = (1 - \beta^{[k]}) \times x^{[0]} + \beta^{[k]} \times x^{[k]} \\ \beta^{[k]} = \frac{k}{K} \end{array} \right. \quad (13) $$

$$ x^{[k]} = (1 - \beta^{[k]}) \times f_0(x^{[k]}, T_c^{\text{arr}}, k) + \beta^{[k]} \times x^{[k]} \quad (14) $$

$$ \theta^* = \underset{\theta}{\operatorname{argmin}} \| f_0(x^{[k]}, T_c^{\text{arr}}, k) - x^{[0]} \|_2 \quad (15) $$

Alg. 1 Training process of diffusion model for user uncertainty reduction

<figure><img src="image_8.png" /><figcaption>Fig. 4. Diagram of baseline MPC approach.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Fig. 5. Markov chain of EV fleet user uncertainty mitigation.</figcaption></figure>

**Algorithm 1**

Training process of diffusion model.

Input: $\tilde{C}, f_{\theta}, K$

Output: $f_{\theta}$

While $\theta$ is not converged:

Sample $\tilde{C}$ and $\tilde{C}$ with given batch size $B$ from historical charging records;

$x^{(0)} \leftarrow \tilde{C}, x^{(K)} \leftarrow \tilde{C}$;

Sample $k = \text{Uniform}(k; \{1, 2, \dots, K\})$;

$\beta^{(k)} = k/K$;

Calculate $x^{(k)}$ by (13);

Perform the gradient descending by $\theta_{i+1} = \nabla_{\theta} \| f_{\theta}(x^{(k)}, T_c^{\text{arr}}, k) - x^{(0)} \|_2$;

If $\| \theta_{i+1} - \theta_i \|_2 \le e: \theta^* \leftarrow \theta_{i+1}, \text{break}$;

Else: $i \leftarrow i + 1$;

The reverse process is the estimation of the actual departure time and energy demand $\tilde{C}$ based on the user inputs $\tilde{C}$, and the process is shown in Alg. 2. During the online deployment, we forward $\tilde{C}$ to the trained model $f_{\theta}$. In each diffusion step $k$, we estimate noise added at each step $\epsilon^{(k)}$ by $f_{\theta}$, then calculate the previous state $x^{(k-1)}$ by subtracting the $\epsilon^{(k)}$ from $x^{(k)}$. $x^{(0)}$ is then saved as the estimation of departure time and energy demand $\tilde{C}$.

Alg. 2 Reverse Process of Diffusion Model for User Uncertainty Reduction

### 3.3.2. Gaussian mixture model (GMM) for FCE scenarios generation

To account for the “future” uncertainty, historical data are used to estimate a probability distribution of EV charging events by fitting GMMs. The GMM is a combination of multiple Gaussian distributions (elements) and is used to fit the probabilistic distributions of the uncertain variables which are the FCE parameters in our case. Each Gaussian distribution is called an element of GMM. Here, we then fit three probability distributions by GMMs: the number of events $q(N_F)$, the arrival time of each event $q(T_f^{\text{arr}})$ and joint distribution of energy demand with charging duration for each event $q(E_f, T_f^{\text{dur}})$. The motivation to couple energy and duration is that they are strongly correlated,

**Algorithm 2**

Reverse process (online deployment).

Input: $\tilde{C}, f_{\theta}$

Output: $\tilde{C}$

$x^{(K)} \leftarrow \tilde{C}, t \leftarrow K$

While $k > 0$:

$\epsilon^{(k)} = \frac{1}{K} \times (x^{(K)} - f_{\theta}(x^{(k)}, T_c^{\text{arr}}, k))$

$x^{(k-1)} = x^{(k)} - \epsilon^{(k)}$

$k \leftarrow k - 1$

$\tilde{C} \leftarrow x^{(0)}$

as high $E_f$ will lead to high $T_f^{\text{dur}}$. We fit $q(N_F)$ and $q(T_f^{\text{arr}})$ with four elements and fit $q(E_f, T_f^{\text{dur}})$ with two elements.

We use scenario approach to address the future uncertainty based on fitted GMMs. Each scenario is a set of possible future arrivals. The scenario generation process is shown in Alg. 3. For each future charging events scenario $\tilde{F}_i$, we first sample the number of events $\tilde{N}_{F,i}$. Then for each event $j$ in $\tilde{F}_i$, we generate its arrival time and energy-charging duration from the fitted GMMs. The departure time of each event is its arrival time plus its charging duration. We only select arrivals that are later than the current time $t_s$ as we consider future events. We iterate this process until we generate enough scenarios $\tilde{F}_{1:N_S}$. In this paper, the total scenarios $N_S$ is set to 10.

Alg. 3 Future charging events scenario generation

### 3.3.3. Data hybrid MPC

Both the diffusion model and GMMs scenario generation are integrated into a data hybrid MPC whose overall flowchart is displayed in Fig. 6. Unlike the baseline MPC approach, when there is a new arrival at the charging station, the corresponding user input $\tilde{T}_c^{\text{dep}}$ and $\tilde{E}_c$ is passed thought the diffusion model and get estimations $\tilde{T}_c^{\text{dep}}$ and $\tilde{E}_c$, with “reduced” uncertainty. Based on the formulation of generic MPC, we run the scenario MPC where each MPC is run accounting for those estimated input and the different FCE scenarios generated with takes the $\tilde{F}_{1:N_S}$ follows Alg. 4 and construct $N_S$ MPCs. Each MPC is formulated with (4) ~ (10), calculates one control command $P_{p,sc,t}$ based on $\tilde{C}, \Delta E_{t,s}$, Mand one scenario $\tilde{F}_{sc}$ from $\tilde{F}_{1:N_S}$. The final control $P_{p,t}$ is the average of all MPCs' output, as (16). Then we send $P_{p,t}$ to the charging station and the EV fleet is updated with (11). The diagram of the data hybrid MPC

**Algorithm 3**

Future charging events scenario generation.

Input: $q(N_F), q(T_f^{\text{arr}}), q(E_f, T_f^{\text{dur}}), N_S, t_s$

Output: $\tilde{F}_{1:N_S}$

For $i$ in $[1, 2, \dots, N_S]$:

Sample $\tilde{N}_{F,i} = q(\tilde{N}_F; N_F)$;

For $j$ in $[1, 2, \dots, \tilde{N}_{F,i}]$:

Sample $\tilde{T}_j^{\text{arr}} = q(\tilde{T}_j^{\text{arr}}, T_j^{\text{arr}})$;

Sample $\tilde{E}_j, \tilde{T}_j^{\text{dur}} = q(\tilde{E}_j, \tilde{T}_j^{\text{dur}}; E_f, T_f^{\text{dur}})$;

$\tilde{T}_j^{\text{dep}} \leftarrow \tilde{T}_j^{\text{arr}} + \tilde{T}_j^{\text{dur}}$;

Save $[\tilde{T}_j^{\text{arr}}, \tilde{T}_j^{\text{dep}}, \tilde{E}_j]$ to $\tilde{F}_i$

$j \leftarrow j + 1$

Slice $\tilde{F}_i$ to save only future events: $\tilde{F}_i = (\tilde{F}_i)_{\tilde{T}_j^{\text{arr}} \ge t_s, j \in N_F}$, and record sliced $\tilde{F}_i$ as one scenario

$i \leftarrow i + 1$
<figure><img src="image_10.png" /><figcaption>Fig. 6. Diagram of data hybrid MPC approach.</figcaption></figure>

approach is shown as Fig. 6.

$$P_{p,t_s} = \frac{1}{N_S} \sum_{sc=1}^{N_S} P_{p,sc,t_s}, \forall p \in \Pi \quad (16)$$

### 3.4. Fully data driven approach: deep reinforcement learning (DRL)

The fully data-driven approach for EV fleet charging management is based on the DRL. The agent learns the control policy $\pi_{\theta}$ from historical charging data by optimizing rewards then deploy $\pi_{\theta}$ into real-time operation.

#### 3.4.1. Partially-observable Markov decision problem (POMDP) formulation

A POMDP problem is defined as an five elements tuple $(S, O, A, T, r, \gamma)$, where $S$ is the set of states, $O$ is the set of observations with $O \subseteq S$ in case the system is not fully observable. $T$ denotes the states' transition function $T: S \times A \to S, r: S \times A \to \mathbb{R}$ is the reward function and $\gamma \in (0, 1]$ is the discount factor. We will elaborate the details of POMDP as follows.

**States:** The entire state vector contains two parts. The first part $s_{p,t} \in \mathbb{R}^{7 \times 1}$ contains information of CCE at the connecting charging port, charging power and delivered energy. The second part is the peak charging power until $t$, for the peak charging power is a main component as we want to minimize it. $s_{p,t}$ is defined as (17) and the entire states of POMDP is defined as (18).

$$s_{p,t} = (t, T_c^{\text{dep}}, \tilde{T}_c^{\text{dep}}, E_c, \tilde{E}_c, P_{p,t}, \Delta E_{c,t}), \forall p \in \Pi; c \in C, \tilde{C}; m_{cp} = 1 \quad (17)$$

$$s_t = \left( s_{1:N_p,t}, \max_{[1,t]}(P_t^{CS}) \right), s_t \in \mathbb{R}^{(7N_p+1) \times 1} \quad (18)$$

**Observations:** As mentioned in above, the agent cannot make full observation of the state space due to uncertainties. As we consider the user uncertainties, the actual values $T_c^{\text{dep}}$ and $E_c$ are not observable for the agent. Only the information inputted by users and information from charging ports are observable. The observation $o_t$ is a subset of $s_t$. The observation of charging port $p$ at time $t$ is defined as (19), $o_{p,t} \in \mathbb{R}^{5 \times 1}$. And the entire observations for POMDP is defined as (20).

$$o_{p,t} = (t, \tilde{T}_c^{\text{dep}}, \tilde{E}_c, P_{p,t}, \Delta E_{c,t}), \forall p \in \Pi, c \in \tilde{C}, m_{cp} = 1 \quad (19)$$

$$o_t = \left( o_{1:N_p,t}^p, \max_{[1,t]}(P_t^{CS}) \right), o_t \in \mathbb{R}^{(5N_p+1) \times 1} \quad (20)$$

**Actions:** The actions are charging power set point sent to each charging port. The actions are defined as (21).

$$a_t = (P_{1:N_p,t}), a_t \in \mathbb{R}^{N_p \times 1} \quad (21)$$

**State Transitions:** The state transition function $T$ contain three parts. First, the state will be updated if there is a new arrival or departure. If there is a new EV arriving at charging port $p$, the information of $\tilde{C}$ and $C$ are appended to the state vector $s_{p,t}$. Also, in case an EV leaves from port $p$, then corresponding $\tilde{C}$ and $C$ at port $p$ are set to 0. Second, the state will be updated with the charging. The $\Delta E_{c,t}$ is updated with (11). Third, at each timestep $t$, we update the $\max_{1,t}(P_t^{CS})$.

**Reward Function:** The reward function corresponds to the objective targeted by the control policy. We choose to compose the reward into three parts. The first part is to reduce the charging power of each time step, as (22) shows. Second, when an EV departs from the charging station, we define a penalty of difference between delivered energy and actual energy demand, as (23) shows. Third, at $T_{end}$, we define a penalty for the peak power of entire day, as (24) shows. Remind that we control the EV fleet charging on a daily basis. Each episode of POMDP is then terminated at the end of the day $T_{end}$. The total reward of one episode (one day) is ultimately defined as (25).

$$r_{1,t} = \begin{cases} 0, & 0 \le P_t^{CS} < P_1 \\ -\left(\frac{P_t^{CS}}{100}\right)^2, & P_1 \le P_t^{CS} < P_2 \\ -5, & P_t^{CS} \ge P_3 \end{cases} \quad (22)$$

$$r_{2,t} = \sum_{c=1}^{N_C} \left( \frac{\Delta E_{c,t}}{E_c} - 1 \right), \forall \{c \in N_C | T_c^{\text{dep}} = t\} \quad (23)$$

$$r_3 = - \left( \frac{\max_{[1,T_{\text{end}}]} (P_t^{CS})}{100} \right)^2 \quad (24)$$

$$r = \sum_{t=1}^{T_{\text{end}}} (r_{1,t} + r_{2,t}) + r_3 \quad (25)$$

When the shape of the observation is large, the learning efficiency of the agent can be decreased. One way to improve the DRL agent performance in such cases is to extract meaningful features from the overall input vector – e.g., using a Convolutional Neural Network (CNN) to project the original long vector to a short vector [36]. The architecture of the CNN implemented in this work is shown as Fig. 7 with a structure adopted from [37]. The size of the CNN’s input layer is the same as the size of $o_t$. The parameters of CNN are not updated during the training of
<figure><img src="image_11.png" /><figcaption>Fig. 7. Diagram of CNN architecture.</figcaption></figure>

the DRL agent since the CNN is used to reduce the observation space by using a lower dimension vector to represent a higher dimension vector. We note the CNN feature extraction process as (26).

$$o'_{t} = \text{CNN}(o_{t}), \text{CNN} : \mathbb{R}^{(5N_{p}+1) \times 1} \rightarrow \mathbb{R}^{64 \times 1} \quad (26)$$

### 3.4.2. Training and deployment of the fully data-driven approach

We train the DRL agent with PPO-clip algorithm [38]. The brief diagram for fully data-driven approach for EV Fleet charging management is shown as Fig. 8. During the deployment phase, the $o'_t$ is forwarded to the trained policy network and the policy network generate control command. The policy and value networks are two MLPs with the same structure of two hidden layers and 128 neurons in each hidden layer. The activation functions of policy and value networks are Tanh. The details of the PPO-clip are not emphasized here as it is a well-known algorithm.

### 3.5. Baseline and reference control strategies

The three control strategies are compared with a reference control strategy denoted as “uncontrolled charging”. In that baseline, the EVs are charged at a constant power $\bar{P}^{ev}$ when connecting to the charging port until the leave or are fully charged.

To assess the performances, the control methods are also compared with a reference corresponding to an “offline optimal” as an ideal case with perfect system knowledge (i.e., there are no uncertainties on the departure time and energy demand for the current charging events and no uncertainties on the arrival, departure, and energy demand for the future charging events). As no uncertainties exist, the results obtained are theoretically the best. This theoretical ideal control is run using the optimal problem formulated by (4) ~ (10).

### 3.6. Evaluation metrics

From the control aspect, the objective is to deliver as much energy as

<figure><img src="image_12.png" /><figcaption>Fig. 8. Deployment of fully data-driven approach.</figcaption></figure>

possible to the EV fleet while minimizing the peak charging power. Therefore, all the control strategies are assessed with regards to the total daily energy delivery $\sum_{c \in C} \Delta E_{c, T_{c}^{dep}}$ in kWh and peak power of charging station of each day $\max_{[1,t]} (P_{t}^{CS})$ in kW. For easier comparison of the methods, we introduce a precision metric $d$ that estimate the errors with the performances of reference ideal control, as in (27) shows, where $E^{ref}$ and $P^{ref}$ is the total delivered energy and peak power of each day of offline optimal. The value of $E^{nom}$ and $P^{nom}$ is chose to resize energy and power difference to [0,1].

$$d = \sqrt{\left(\frac{\sum_{c \in C} \Delta E_{c, T_{c}^{dep}} - E^{ref}}{E^{nom}}\right)^2 + \left(\frac{\max(P_{1:T_{end}}^{CS}) - P^{ref}}{P^{nom}}\right)^2} \quad (27)$$

## 4. Numerical results

### 4.1. Simulation setup

#### 4.1.1. User uncertainties

As mentioned in Section 2.2, we set different $\sigma_E$ (kWh) and $\sigma_T$ (h) to emulate different levels of “user” uncertainty. Table 2 displays the values of the uncertainties metrics in the different cases regard to $e_C$, which is calculated based on all CCEs of one month (July 2019).

#### 4.1.2. MPC configurations

We run both MPC strategies with 15 min time resolution and with shrinking horizon ($t_k \rightarrow T_{end}$). Therefore, in each day there are 96 time-steps ($T_{end}$). We have in total 52 charging ports ($N_p$). The maximum charging power $P^{ev}$ for all EVs are set to 7.7 kW. The $E^{nom}$ is set to 1100 kWh and $P^{nom}$ is set to 150 kW. The implementation of the MPC is based on Pyomo 6.6.2 and Gurobi 9.5. We run the MPCs and following training process on our own PC. The configuration of our CPU is 13th Gen Intel (R) Core(TM) i7-13700H 2.40 GHz with 32GB RAM.

#### 4.1.3. Training process configuration and hyperparameters of diffusion model

The implementation of the diffusion model is based on Pytorch 2.2. We train the diffusion model with the historical charging events and their users' inputs from 01/01/2019 to 06/30/2019. The training process configuration is shown in appendix on Table 9.

<table><caption>Table 2
User uncertainty setups.</caption><thead><tr><td>Configuration</td><td>σ<sub>E</sub>/kWh</td><td>σ<sub>T</sub>/h</td><td>e<sub>C</sub>/%</td></tr></thead><tbody><tr><td>Low user uncertainty</td><td>2</td><td>1</td><td>26.11</td></tr><tr><td>Medium user uncertainty</td><td>4</td><td>1.5</td><td>46.61</td></tr><tr><td>High user uncertainty</td><td>6</td><td>2</td><td>65.27</td></tr></tbody></table>
4.1.4. Training process configuration and of DRL

The implementation of CNN feature extractor and PPO-clip algo-
rithm is based on Pytorch 2.2 and Stable Baseline3 [39]. We trained the
DRL agent with historical charging events and their users' inputs from
01/01/2019 to 06/30/2019 (and later tested with charging record from
10/01/2019 ~ 10/31/2019, the ratio between training and testing is
around 6:1). The training configuration and hyperparameters are shown
in appendix on Table 10. The hyperparameters are defined based on
some preliminary tests to optimize the DRL performance.

4.2. *Uncertainties impact comparison*

Before assessing the performances of the different control strategies,
we first analyze the impact of "user" and "future" uncertainties by
running the baseline MPC approach for a month. To better illustrate the
meaning of the "user" and "future" uncertainty, we select one day (10/
16/2019) from the testing horizon and plot the value uncertainty vari-
ables in Fig. 9. Each dot corresponds to a charging event. In the Fig. 9(a),
we show the difference in departure time and energy demand for CCE
between the actual value and the user input and in Fig. 9(b), we show
the difference in arrival, departure time and energy demand for FCE
between actual value and prediction at the beginning of the day. From

Fig. 9, it is obvious that the orange dots (charging event with uncertainty
value) do not overlap with the blue dots (actual charging event). This
means that the controller cannot obtain the actual value due to the ex-
istence of the uncertainty. From the quantitative aspect, the RMSE
corresponding to actual and users' input values for the departure and
energy demand of CCE in Fig. 9(a) are respectively 1.8 h and 5.1 kWh. As
the number of events N<sub>f</sub> is also uncertainty for the FCE, the size of actual
FCE and uncertain FCE are not same, so it is hard to describe RMSE. The
difference is that uncertain FCE has 11 more predicted FCE than actual
FCE, and the arrival, departure and energy demand will be different.

The results are shown in Fig. 10. The x-axis relates to different un-
certainty configurations, and the y-axis is precision to the offline optimal
(27) along the month tested. This figure shows the distribution of daily
precision for each configuration during the tested month. In Fig. 10,
from the left to the left, the value of d increases from 0.12 to 0.28, which
means the controller's performance is decreasing. The performance of
the baseline MPC significantly degraded from 58 % to 133 % in the case
of only future uncertainty.

A more detailed analysis of the results is displayed in Table 3. The
lower energy can be delivered to the EV fleet with a higher user un-
certainty level. This is due to the early departure and underestimation of
energy demand by the EV users, which makes the controller fail to

<figure><img src="image_13.png" /><figcaption>Fig. 9. Example of (a) user uncertainty (b) future uncertainty.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 10. Uncertainty impact of user and future uncertainty over one month real-time deployment.</figcaption></figure>
<caption>**Table 3**</caption>

Uncertainty impact of user and future uncertainty on energy delivery over one month real-time deployment.

<table><thead><tr><th>Uncertainty configuration</th><th>Energy delivery/ kWh</th><th>Peak power/ kW</th></tr></thead><tbody><tr><td>Offline optimal</td><td>1174.49 ± 218.45</td><td>88.82 ± 13.77</td></tr><tr><td>No user uncertainty,<br>Only future uncertainty</td><td>1174.49 ± 218.45</td><td>107.41 ± 15.10</td></tr><tr><td>No future uncertainty,<br>Low user uncertainty</td><td>978.90 ± 183.81</td><td>88.71 ± 13.61</td></tr><tr><td>No future uncertainty,<br>Medium user uncertainty</td><td>913.93 ± 166.14</td><td>87.92 ± 12.76</td></tr><tr><td>No future uncertainty,<br>High user uncertainty</td><td>872.72 ± 165.86</td><td>87.66 ± 13.33</td></tr></tbody></table>

provide enough energy as the actual energy demand. Even if the user provides an over-estimated energy demand, they will not charge more energy than the actual demand. Thus, user uncertainty always leads to less energy delivery. From the results, the user uncertainty has a higher impact on the controller. Therefore, to effectively manage the EV fleet charging, we need to consider user uncertainties.

### 4.3. User uncertainty reduction

A second run of preliminary tests investigates the reduction of “user” uncertainty with the diffusion model proposed over a test month. From the results in Table 4, the diffusion model can reduce the user uncertainty of all user uncertainty setups. It is noticeable that improvement is greater with higher user uncertainty level.

Table 5 shows the RMSE of the departure time and energy demand after tuned by proposed model. From the results, the uncertainty on the departure time and energy demand are all reduced.

We also analyzed the impact of the diffusion model hyperparameter K in order to select the most appropriate value and the results are shown in Fig. 11. The x-axis is the value of K and the y-axis shows the tuned $e_C$ by the diffusion model. We run the test with K from 5 to 40 with a step size 5. From the results, the hyperparameter K slightly impacts the diffusion model performance, and the optimal K differs with different user uncertainty configurations. Generally speaking, we can choose K between 15 and 35. In this paper, we choose K equals to 25.

### 4.4. Control strategies performances

The three control strategies are then tested along a month, Oct. 2019. Fig. 12 show charging power one day of October 2019. From the Fig. 12,

<caption>**Table 4**</caption>

Diffusion model user uncertainty tuner performance under different user uncertainty level.

<table><caption>**Table 5**</caption><thead><tr><th>Configuration</th><th>Original e<sub>C</sub> /%</th><th>Tuned e<sub>C</sub> /%</th></tr></thead><tbody><tr><td>Low</td><td>26.11</td><td>22.63</td></tr><tr><td>Medium</td><td>46.61</td><td>36.19</td></tr><tr><td>High</td><td>65.27</td><td>43.76</td></tr></tbody></table>

RMSE of departure time and energy demand.

<table><thead><tr><th rowspan="2">Configuration</th><th colspan="2">Original</th><th colspan="2">Tuned</th></tr><tr><th>Departure time (h)</th><th>Energy demand (kWh)</th><th>Departure time (h)</th><th>Energy demand (kWh)</th></tr></thead><tbody><tr><td>Low</td><td>0.84</td><td>1.89</td><td>0.56</td><td>1.79</td></tr><tr><td>Medium</td><td>1.28</td><td>3.55</td><td>0.71</td><td>3.12</td></tr><tr><td>High</td><td>1.71</td><td>4.92</td><td>0.79</td><td>3.99</td></tr></tbody></table>

<figure><img src="image_15.png" /><figcaption>**Fig. 11.** The influence of diffusion steps on the diffusion model performance under different user uncertainty.</figcaption></figure>

all three control strategies can reduce the peak charging power of $P_t^{CS}$ compared to uncontrolled charging. Meanwhile, it seems that the baseline MPC returns lowest charging power. However, this low charging power is paid with the cost of very low energy delivery. This can be found in the numerical details.

The performance metrics along the test month are shown in Fig. 13 and Table 6. In Fig. 13, the x-axis displays different control strategies, and the y-axis shows the distance of each strategy to the offline optimal – distribution of daily values along the test month. From Fig. 13 all three strategies are better than uncontrolled charging. Compared to the un-controlled charging, the baseline MPC is 37.0 %~53.4 % closer to the offline optimal, the data hybrid MPC is 58.7 %~69.6 % closer, and the entirely data-driven approach is 45.6 % ~ 47.8 % closer. Among the three strategies, the data hybrid MPC is the closest to the offline optimal reference for every user uncertainty level, and the fully data-driven strategy is better than the baseline MPC for medium and high user uncertainty levels.

From the numerical results in Table 6, the baseline MPC approach has lowest energy delivery, mainly due to the untuned user uncertainty. The baseline MPC reduces the peak power at the very high cost of energy delivery. The baseline MPC reduces peak power by 34.5 % ~ 36.0 % compared to the uncontrolled charging, while only delivering 74.0 % ~ 83.2 % energy to the EV fleet. With the higher user uncertainty level, the baseline MPC approach delivers less energy to the EV fleet. For the data hybrid MPC, it can reduce the peak charging power by 20.2 % ~ 23.8 % compared with the uncontrolled charging while delivering 94.6 % ~ 95.8 % energy to EV, which is much higher than the baseline MPC. This is due to the mitigation of the user uncertainty by the diffusion model and the mitigation of the future uncertainty by scenario generations from GMMs rather than a single forecast, which also boosts the performance of data hybrid MPC. With the increasing user uncertainty, the data hybrid MPC’s performance degrades with lower energy delivery, yet the data hybrid MPC approach has better performance than baseline MPC under every level of user uncertainty. The fully data-driven approach can reduce the peak charging power by 20.4 % ~ 23.7 %
<figure><img src="image_16.png" /><figcaption>Fig. 12. Charging power at the charging station on Oct. 16.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 13. Performance of different control strategies under a) low b) medium c) high user uncertainty.</figcaption></figure>

<table><caption>**Table 6**

Numerical results control strategies performance.</caption><thead><tr><th>Strategy</th><th>User uncertainty</th><th>Peak power/kW</th><th>Delivered energy/kWh</th></tr></thead><tbody><tr><td>Offline optimal</td><td>N/A</td><td>88.82 ± 13.77</td><td>1174.49 ± 214.9</td></tr><tr><td>Uncontrolled</td><td>N/A</td><td>148.28 ± 22.38</td><td>1174.49 ± 214.9</td></tr><tr><td rowspan="3">Baseline MPC</td><td>Low</td><td>97.1 ± 26.04</td><td>976.91 ± 175.58</td></tr><tr><td>Medium</td><td>96.28 ± 13.84</td><td>910.06 ± 160.37</td></tr><tr><td>High</td><td>94.84 ± 14.16</td><td>869.25 ± 158.2</td></tr><tr><td rowspan="3">Data hybrid MPC</td><td>Low</td><td>118.37 ± 19.94</td><td>1124.14 ± 201.56</td></tr><tr><td>Medium</td><td>116.2 ± 19.7</td><td>1124.47 ± 206.84</td></tr><tr><td>High</td><td>113.06 ± 16.05</td><td>1110.85 ± 200.37</td></tr><tr><td rowspan="3">Fully data-driven</td><td>Low</td><td>112.88 ± 11.28</td><td>995.21 ± 148.6</td></tr><tr><td>Medium</td><td>113.06 ± 10.42</td><td>1001.8 ± 148.92</td></tr><tr><td>High</td><td>118.02 ± 10.69</td><td>1034.4 ± 153.15</td></tr></tbody></table>

while delivering 84.7% ~ 88.1% energy to the EV fleet. The fully data-driven approach cannot explicitly take advantage of FCE's information to make decisions, as FCEs are hard to include in the DRL state formulation due to the unknown $\tilde{N}_F$. A visualization of daily energy delivery is shown in Fig. 14. From the results, the data-hybrid MPC has the highest energy delivery compared with two others, and it is the closest to the actual demand. We also tested the fully data-driven approach: CNN-DRL and simple DRL to test the effectiveness of the CNN feature extractor. The results are shown in appendix, in Fig. 15 and Table 11. The results show that the CNN feature extractor can improve the DRL performance.

Therefore, the data hybrid MPC has the best performance that can reduce the peak charging power while maintaining high energy delivery to the EV fleet.

### 4.5. Time consumption

The training and runtime of each strategy is displayed in Table 7. The fully data-driven requires huge training time because the DRL agent needs to run many episodes to learn an efficient control policy. For the aspect of runtime, the data hybrid MPC takes the longest runtime because, in each time, 10 MPCs are run for the scenario-MPC. However, the max runtime is still much smaller than the time resolution, which is 900 s (15 min). Therefore, the runtime is not an obstacle for the data hybrid MPC.

### 4.6. Scaling performance of data-hybrid MPC

We test the data-hybrid MPC with different number of charging ports from the Palo Alto dataset. We run tests of different numbers of charging ports selected from the Palo Alto Dataset, and the user uncertainty is set to a low level. The evaluation metrics are distance to offline optimal (defined by (27)) and average runtime per step. The results are shown in Table 8. With the increasing of the $N_p$, the distance of the data-hybrid MPC to the offline optimal increase, because there are more CCE and FCE and, consequently, more uncertainty sources. Nevertheless, the results show that data-hybrid MPC performs better than the baseline MPC of any charging port size. Also, with the increasing of $N_p$, the runtime per step increase because the optimization problem is bigger. However, the runtime per step for the largest configuration is smaller than control time resolution. It is noteworthy to mention that this runtime only measured the time consumption of controller to compute control signal. In real-world, the runtime can be higher due to the communication delay and calculation resources limitation.
<figure><img src="image_18.png" /><figcaption>Fig. 14. Daily energy delivery of each strategies under different user uncertainty.</figcaption></figure>

<table><caption>**Table 7**
Training and runtime for three strategies.</caption><thead><tr><td>Strategies</td><td>Total training time/s</td><td>Max. runtime/s</td></tr></thead><tbody><tr><td>Baseline MPC</td><td>N/A</td><td>1.43</td></tr><tr><td>Data hybrid MPC</td><td>195.79</td><td>10.97</td></tr><tr><td>Fully data-driven</td><td>3457.59</td><td>0.01</td></tr></tbody></table>

<table><caption>**Table 8**
Scaling test performance of data-hybrid MPC.</caption><thead><tr><th rowspan="2">Number of charging ports N<sub>p</sub></th><th colspan="2">Distance to offline optimal</th><th rowspan="2">Average runtime per step (s) (96 steps per day)</th></tr><tr><th>Baseline MPC</th><th>Data hybrid MPC</th></tr></thead><tbody><tr><td>13</td><td>0.09</td><td><strong>0.06</strong></td><td>1.66</td></tr><tr><td>26</td><td>0.13</td><td><strong>0.10</strong></td><td>2.35</td></tr><tr><td>39</td><td>0.17</td><td><strong>0.12</strong></td><td>3.03</td></tr><tr><td>52</td><td>0.21</td><td><strong>0.15</strong></td><td>3.46</td></tr></tbody></table>

## 5. Conclusion

In this paper, we considered and mitigated the user and future uncertainties in the EV fleet charging management problem. We found that user uncertainty has a 1.3 times higher impact on the controller, so we proposed mitigation strategies for both user and future uncertainties using a diffusion model and GMMs. We proposed and compared the performance of the baseline MPC, data hybrid MPC and fully data-driven approaches with the reference of offline optimal and uncontrolled charging. For the fully data-driven approach, we implemented a CNN to improve the DRL performance. From the results, the data hybrid MPC has the best performance. The data hybrid MPC can increase energy delivery by 12.6% ~ 20.6% and is 22.3% to 28.4% closer to the offline optimal compared with baseline MPC, depending on the user uncertainty level. The fully data-driven approach performs better than the

baseline MPC when the user uncertainty is high but is always worse than the data hybrid MPC.

In practical application, the proposed control strategy can be beneficial to both the charging station manager (more energy delivered to EVs), and distribution grid operator (reduced aggregated peak power). Further study can be related to 1) considering the emerging EV charging station, such as battery-swap and bi-directional charging, 2) considering charging station integration with power grid with renewable energy generation and 3) considering other control objectives, such as economical operation of charging station, environmental impact minimization of charging station, then analysis and mitigate the impact of user and future uncertainties. The limitation of this paper is that the user uncertainties are sampled based on gaussian distribution. Therefore, another valuable research potential is investigation of other mitigation strategy for other kinds of user uncertainties.

## CRediT authorship contribution statement

Zhewei Zhang: Writing – review & editing, Writing – original draft, Visualization, Validation, Methodology, Investigation, Data curation, Conceptualization. Rémy Rigo-Mariani: Writing – review & editing, Visualization, Supervision, Methodology, Conceptualization. Nour-edine Hadjsaid: Supervision, Funding acquisition.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgement

This research is supported by the National Research Foundation, Prime Minister's Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme.

## Appendix

### Derivation of Eq. (15)

We can write the forward process in a recursive format, as (i) shows. Then based on assumption (14), we can rewrite (i) as (ii). Then we note $\beta^{(k)} = k/k_0$, so we can (15).
$$ 
\begin{aligned}
\mathbf{x}^{\{k\}} &= \mathbf{x}^{\{k-1\}} + \varepsilon^{\{k\}} \\
&= \mathbf{x}^{\{k-2\}} + \varepsilon^{\{k-1\}} + \varepsilon^{\{k-2\}} \\
&\vdots \\
&= \mathbf{x}^{\{0\}} + \sum_{k} \varepsilon^{\{k\}}
\end{aligned} 
\qquad (i) $$

$$ 
\begin{aligned}
\mathbf{x}^{\{k\}} &\approx \mathbf{x}^{\{0\}} + k \frac{\mathbf{x}^{\{K\}} - \mathbf{x}^{\{0\}}}{K} \\
&\approx \left(1 - \frac{k}{K}\right) \mathbf{x}^{\{0\}} + \frac{k}{K} \mathbf{x}^{\{K\}}
\end{aligned} 
\qquad (ii) $$

### Parameters of diffusion model

## Table 9

<table><caption>**Table 9**
Diffusion model configuration.</caption><thead><tr><td>Parameters</td><td>Value</td></tr></thead><tbody><tr><td>Batch Size <i>B</i></td><td>128</td></tr><tr><td>Learning Rate <i>η</i></td><td>0.001</td></tr><tr><td>Diffusion Processes <i>T</i></td><td>25</td></tr><tr><td>Optimizer</td><td>Adam</td></tr><tr><td>Maximum Training Epochs</td><td>1000</td></tr></tbody></table>

### Parameters of deep reinforcement learning

## Table 10

<table><caption>**Table 10**
Deep reinforcement learning configuration.</caption><thead><tr><td>Parameters</td><td>Value</td></tr></thead><tbody><tr><td>Batch size <i>B</i></td><td>96</td></tr><tr><td>Replay buffer size</td><td>1e6</td></tr><tr><td>Learning rate <i>η</i></td><td>0.0003</td></tr><tr><td>Total episodes</td><td>1e4</td></tr><tr><td>Total timesteps (Numbers of Steps per Episode × Total Episodes)</td><td>9.6e5</td></tr><tr><td>Clip coefficient <i>ε</i></td><td>0.2</td></tr><tr><td>Discount factor <i>γ</i></td><td>1</td></tr><tr><td>Value function loss coefficient</td><td>0.5</td></tr><tr><td>Entropy loss coefficient</td><td>0</td></tr><tr><td><i>P</i><sub>1</sub></td><td>100 kW</td></tr><tr><td><i>P</i><sub>2</sub></td><td>130 kW</td></tr></tbody></table>

### Performance of DRL with/without CNN assist feature extraction

Here, we show the test results of DRL with/without CNN assisted feature extraction. We ran the test of DRL for entire October 2019. The metric is the agent's daily reward based on (28) ~ (29). The results are shown in Fig. 15 and Table 11. From the results, the CNN assisted DRL can improve the DRL performance from 3.68 % to 8.91 %.
<figure><img src="image_19.png" /><figcaption>Fig. 15. Daily reward of DRL with/without CNN feature extraction.</figcaption></figure>

<table><caption>**Table 11**
Average daily reward of DRL with/without CNN feature extraction.</caption><thead><tr><td>User uncertainty level</td><td>CNN-DRL</td><td>DRL</td><td>Improvement</td></tr></thead><tbody><tr><td>Low</td><td>-13.29</td><td>-13.85</td><td>+3.68 %</td></tr><tr><td>Medium</td><td>-13.76</td><td>-14.40</td><td>+4.44 %</td></tr><tr><td>High</td><td>-13.19</td><td>-14.48</td><td>+8.91 %</td></tr></tbody></table>

## Data availability

The data that has been used is confidential.

## References

[1] Trends in electric cars – Global EV outlook 2024 – Analysis. IEA 2024. Accessed: Sep. 23,[Online]. Available: https://www.iea.org/reports/global-ev-outlook-2024/trends-in-electric-cars.

[2] Sadeghian O, Oshnoei A, Mohammadi-ivatloo B, Vahidinasab V, Anvari-Moghaddam A. A comprehensive review on electric vehicles smart charging: solutions, strategies, technologies, and challenges. J Energy Storage 2022;54: 105241. https://doi.org/10.1016/j.est.2022.105241.

[3] Nour M, Chaves-Ávila JP, Magdy G, Sánchez-Miralles Á. Review of positive and negative impacts of electric vehicles charging on electric power systems. Energies 2020;13(18). https://doi.org/10.3390/en13184675. Art. no. 18.

[4] Liu J, Yang X, Zhuge C. A joint model of infrastructure planning and smart charging strategies for shared electric vehicles. Green Energy Intell Transp 2024;3(4): 100168. https://doi.org/10.1016/j.geits.2024.100168.

[5] Das P, Kayal P. An advantageous charging/discharging scheduling of electric vehicles in a PV energy enhanced power distribution grid. Green Energy Intell Transp 2024;3(2):100170. https://doi.org/10.1016/j.geits.2024.100170.

[6] Brinkel N, Visser L, van Sark W, AlSaif T. A novel forecasting approach to schedule aggregated electric vehicle charging. Energy AI Oct. 2023;14:100297. https://doi.org/10.1016/j.egyai.2023.100297.

[7] Shahriar S, Al-Ali AR, Osman AH, Dhou S, Nijim M. Prediction of EV charging behavior using machine learning. IEEE Access 2021;9:111576–86. https://doi.org/10.1109/ACCESS.2021.3103119.

[8] Nespoli A, Ogliari E, Leva S. User behavior clustering based method for EV charging forecast. IEEE Access 2023;11:6273–83. https://doi.org/10.1109/ACCESS.2023.3235952.

[9] Wang S, Zhuge C, Shao C, Wang P, Yang X, Wang S. Short-term electric vehicle charging demand prediction: a deep learning approach. Appl Energy 2023;340: 121032. https://doi.org/10.1016/j.apenergy.2023.121032.

[10] Mohammad F, Kang D-K, Ahmed MA, Kim Y-C. Energy demand load forecasting for electric vehicle charging stations network based on ConvLSTM and BiConvLSTM architectures. IEEE Access 2023;11:67350–69. https://doi.org/10.1109/ACCESS.2023.3274657.

[11] Amara-Ouali Y, Goude Y, Hamroube C, Bishara M. A benchmark of electric vehicle load and occupancy models for day-ahead forecasting on open charging session data. In: Proceedings of the Thirteenth ACM International Conference on Future Energy Systems, in e-Energy '22. New York, NY, USA: Association for Computing Machinery; 2022. p. 193–207. https://doi.org/10.1145/3538637.3538850.

[12] Huber J, Dann D, Weinhardt C. Probabilistic forecasts of time and energy flexibility in battery electric vehicle charging. Appl Energy 2020;262:114525. https://doi.org/10.1016/j.apenergy.2020.114525.

[13] Li S, Xiong H, Chen Y. DiffPLF: a conditional diffusion model for probabilistic forecasting of EV charging load. Electr Power Syst Res 2024;235:110723. https://doi.org/10.1016/j.epsr.2024.110723.

[14] Forootani A, Rastegar M, Zareipour H. Transfer learning-based framework enhanced by deep generative model for cold-start forecasting of residential EV charging behavior. IEEE Trans Intell Veh 2024;9(1):190–8. https://doi.org/10.1109/TIV.2023.3328458.

[15] Mbuwir BV, Vanmunster L, Thoelen K, Deconinck G. A hybrid policy gradient and rule-based control framework for electric vehicle charging. Energy AI Jun. 2021;4: 100059. https://doi.org/10.1016/j.egyai.2021.100059.

[16] Lee ZJ, Li T, Low SH. ACN-data: analysis and applications of an open EV charging dataset. In: Proceedings of the Tenth ACM International Conference on Future Energy Systems. Phoenix AZ USA: ACM; 2019. p. 139–49. https://doi.org/10.1145/3307772.3328313.

[17] Gan L, Topcu U, Low SH. Optimal decentralized protocol for electric vehicle charging. IEEE Trans Power Syst 2013;28(2):940–51. https://doi.org/10.1109/TPWRS.2012.2210288.

[18] Li Z, Guo Q, Sun H, Xin S, Wang J. A new real-time smart-charging method considering expected electric vehicle fleet connections. IEEE Trans Power Syst 2014;29(6):3114–5. https://doi.org/10.1109/TPWRS.2014.2311954.
[19] Wu C, Gao S, Liu Y, Song TE, Han H. A model predictive control approach in microgrid considering multi-uncertainty of electric vehicles. Renew Energy 2021; 163:1385-96. https://doi.org/10.1016/j.renene.2020.08.137.

[20] Engel J, Schmitt T, Rodemann T, Adamy J. Hierarchical economic model predictive control approach for a building energy management system with scenario-driven EV charging. IEEE Trans Smart Grid 2022;13(4):3082-93. https://doi.org/10.1109/TSG.2022.3160390.

[21] McClone G, Ghosh A, Khurram A, Washom B, Kleissl J. Hybrid machine learning forecasting for online MPC of work place electric vehicle charging. IEEE Trans Smart Grid 2023:1. https://doi.org/10.1109/TSG.2023.3296014.

[22] S. Li, H. Xiong, and Y. Chen, "DiffCharge: generating EV charging scenarios via a denoising diffusion model," 2024, arXiv: 2308.09857. Accessed: Sep. 24, 2024. [Online]. Available: http://arxiv.org/abs/2308.09857.

[23] Kim J, Oh H. Robust operation scheme of EV charging facility with uncertain user behavior. IEEE Trans Ind Inform Oct. 2023;19(10):10624-34. https://doi.org/10.1109/TII.2023.3240752.

[24] Jia R, Pan H, Zhang S, Hu Y. Charging scheduling strategy for electric vehicles in residential areas based on offline reinforcement learning. J Energy Storage Dec. 2024;103:114319. https://doi.org/10.1016/j.est.2024.114319.

[25] Wang K, et al. A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning. Appl Energy Aug. 2023;343:121186. https://doi.org/10.1016/j.apenergy.2023.121186.

[26] Zhang S, Jia R, Pan H, Cao Y. A safe reinforcement learning-based charging strategy for electric vehicles in residential microgrid. Appl Energy Oct. 2023;348: 121490. https://doi.org/10.1016/j.apenergy.2023.121490.

[27] Tuchnitz F, Ebell N, Schlund J, Pruckner M. Development and evaluation of a smart charging strategy for an electric vehicle fleet based on reinforcement learning. Appl Energy Mar. 2021;285:116382. https://doi.org/10.1016/j.apenergy.2020.116382.

[28] Claessens BJ, Vandael S, Ruelens F, De Craemer K, Beusen B. Peak shaving of a heterogeneous cluster of residential flexibility carriers using reinforcement learning. In: IEEE PES ISGT Europe 2013; Oct. 2013. p. 1-5. https://doi.org/10.1109/ISGTEurope.2013.6695254.

[29] M. Calvo-Fullana, S. Paternain, L.F.O. Chamon, and A. Ribeiro, "State augmented constrained reinforcement learning: overcoming the limitations of learning with rewards," 2023, arXiv: 2102.11941. Accessed: Apr. 29, 2024. [Online]. Available: http://arxiv.org/abs/2102.11941.

[30] E. Wong, "Electric vehicle charging station usage (July 2011 - Dec 2020)." Feb. 08, 2021. [Online]. Available: https://data.cityofpaloalto.org/dataviews/257812/ele ctric-vehicle-charging-station-usage-july-2011-dec-2020/.

[31] C. Yang, L. Liang, and Z. Su, "Real-world denoising via diffusion model," 2023, arXiv: arXiv:2305.04457. doi: 10.48550/arXiv.2305.04457.

[32] Y. Xie, M. Yuan, B. Dong, and Q. Li, "Diffusion model for generative image denoising," 2023, arXiv: arXiv:2302.02398. doi: 10.48550/arXiv.2302.02398.

[33] "Diffusion model for DAS-VSP data denoising." Accessed: Sep. 25, 2024. [Online]. Available: https://www.mdpi.com/1424-8220/23/20/8619.

[34] Kazerouni A, et al. Diffusion models in medical imaging: a comprehensive survey. Med Image Anal 2023;88:102846. https://doi.org/10.1016/j.media.2023.102846.

[35] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems. Curran Associates, Inc.; 2020. p. 6840-51. Accessed: Sep. 25, 2024. [Online]. Available: https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html.

[36] Hakenes S, Glasmachers T. Boosting reinforcement learning with unsupervised feature extraction. In: Tetko IV, Kürková V, Karpov P, Theis F, editors. Artificial Neural Networks and Machine Learning – ICANN2019: Theoretical Neural Computation. Cham: Springer International Publishing; 2019. p. 555-66. https://doi.org/10.1007/978-3-030-30487-4_43.

[37] Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc. IEEE 1998;86(11):2278-324. https://doi.org/10.1109/5.726791.

[38] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," 2017, arXiv: arXiv:1707.06347. doi: 10.48550/arXiv.1707.06347.

[39] Raffin A, Hill A, Gleave A, Kanervisto A, Ernestus M, Dormann N. Stable-Baselines3: reliable reinforcement learning implementations. J Mach Learn Res 2021;22(268):1-8.