## 2 Related Works
### Charging Coordination for EVs and Home Energy Management  
Most existing research on charging coordination and optimization in smart grids has primarily focused on EVs and home energy management systems [14; 15; 16; 17; 18; 19; 20]. Coordinating and optimizing charging strategies is crucial for demand response, enabling utilities to shift charging to off-peak hours, which reduces grid strain and consumer costs. Additionally, it enhances stability by efficiently harnessing excess renewable energy to support sustainable energy systems [13; 21; 22]. In [14], [PERSON] et al. investigated the coordinated charging of multiple EVs to reduce charging costs, maintain a high SoC, and prevent transformer overloads. They proposed a multi-agent deep RL that utilizes a communication neural network for distributed computation of global information and incorporates long short-term memory (LSTM) for flexible price prediction. The given experimental results verify the effectiveness and practicality of the proposed method in a residential charging area. Similarly, the authors in [15] proposed an enhanced RL charging management system for EVs that accounts for essential parameters, such as departure times and energy needs upon arrival at charging stations. They utilized RL using a Double Deep Q-learning (DQL) to offer a flexible and scalable strategy for managing EV fleets while addressing overestimation in action values. The proposed RL technique significantly reduces load variance by 68% compared to uncontrolled charging strategies  
In [16], the authors proposed an RL framework to intelligently recommend publicly accessible charging stations. They utilized a multi-objective, multi-agent RL approach using a multi-agent actor-critic (A2C) algorithm with centralized training and decentralized execution. The proposed framework aims to optimize multiple learning objectives by employing multiple critics combined with a dynamic gradient re-weighting strategy. It also incorporates an adaptive imitation learning scheme to improve policy convergence. The evaluation results showed that the proposed framework outperformed 10 baseline approaches when tested on real-world datasets from major cities, including Beijing and Shanghai. Similarly, in [17], the authors proposed a multi-agent RL approach to tackle the challenges of energy scheduling at charging stations, with the goal of stabilizing the electricity market and meeting the charging demands of EVs. Their approach integrates a purchasing strategy based on the multi-agent deep deterministic policy gradient and an LSTM for predicting EV charging demand, creating a unified energy scheduling strategy that functions effectively across both temporal and spatial dimensions. The results demonstrated that this approach outperforms existing energy scheduling strategies regarding economic profits and user satisfaction, based on tests conducted with real electricity market data from Chicago. Additionally, [PERSON] et al. [18] proposed a multi-agent charge scheduling control framework integrated with a cooperative vehicle infrastructure system. They introduced a novel multi-agent A2C algorithm based on a non-cooperative game framework. This algorithm employs a K-nearest neighbor multi-head attention mechanism for state representation, an adaptive service price adjustment strategy for action definition, and a spatio-temporal discount joint reward system to ensure stable learning convergence. The results demonstrate that the proposed method effectively improves the efficiency of EV charging stations while reducing charging costs. It also shows strong extensibility and effectiveness in complex urban environments when compared to various baseline strategies, including fixed and greedy strategies, independent Q-learning, multi-agent Q-learning, and multi-agent deep deterministic policy gradient. [PERSON] et al. [19] investigated the complexities of energy management in multi-home installations of solar photovoltaics (PV) combined with an EV charging schedule.