## 1 Introduction  
A smart power grid is an advanced electrical grid that integrates digital technologies to monitor, control, and optimize the generation, distribution, and consumption of electricity [1, 2]. Unlike traditional power grids, smart grids utilize sensors, communication networks, and automation systems to enable real-time data exchange between utilities and consumers, enhancing the efficiency, reliability, and sustainability of power distribution [3, 4]. These capabilities also support the integration of renewable energy sources, which further contributes to sustainable energy management. A key component of smart grids is the Advanced Metering Infrastructure (AMI), which facilitates the automatic collection of energy consumption data from smart meters and transmits it to utility providers [5; 6]. Smart meters (SMs) at the consumer end of the system replace traditional meters, providing real-time consumption data to both the utility and consumers. This enables better energy management, dynamic pricing, and more accurate billing [7]. SMs also support remote monitoring and control, allowing for automated load balancing and seamless integration of distributed energy resources. Managing energy in smart grids requires the coordinated control of generation, distribution, and consumption to ensure grid stability, reliability, and cost-efficiency [8]. However, the integration of distributed energy resources (DERs), such as solar panels and wind turbines, introduces variability due to weather conditions. Additionally, the growing adoption of electric vehicles (EVs) and home energy storage systems increases demand and alters consumption patterns, further complicating grid management. Effective energy management strategies must balance supply and demand, minimize operational costs, and optimize the use of renewable energy while maintaining grid stability. Advanced techniques, such as optimization algorithms and machine learning, play a crucial role in achieving these goals by enabling real-time decision-making and efficient energy distribution. While several optimization techniques--such as Linear, Integer, and Binary Programming--have been employed in the literature [9; 10; 11], they often encounter challenges. These methods can struggle with scalability in large systems, have difficulty adapting to real-time fluctuations in demand and pricing, and depend heavily on accurate system models and parameters, which may not always be available [12]. To address these limitations, our previous work proposed a reinforcement learning (RL) approach using an actor-critic algorithm to optimize home battery charging coordination [13]. The RL-based charging coordination system has an agent and an environment, where the agent is the charging coordinator and the environment is the home batteries of a number of homes connected to the same electrical feeder. Home meters can communicate with the control center, which operates a charging coordination system, using various technologies such as cellular networks, the Internet, and wired connections. At each time step \(t\), the environment provides the agent with the current state \(S_{t}\) which includes the State of Charge (SoC) of each battery and the maximum amount of power allocated for charging (C\({}_{max}\)). The agent then selects an action \(a_{t}\) which includes the amount of charging each battery can charge. This decision is made to balance three key objectives: avoiding the consumption of more than the energy allocated for charging (\(C_{max}\)) to ensure the proper operation of the grid, distributing energy fairly across all consumers, and ensuring good service by maintaining the SoC values within predefined thresholds to ensure uninterrupted service for consumers. Simultaneously, the agent seeks to minimize overall power consumption from the grid to reduce operational costs and delay infrastructure upgrades. In response, the environment returns the next state \(S_{t+1}\) and a reward \(r_{t+1}\), which reflects the agent's success in achieving the objectives.