## 2 Related Works
### RL-Based Charging Coordination for Home Batteries
This normalization ensures consistency across all reward components by mapping their values to the [0, 1] range, where 0 represents the minimum reward and 1 the maximum. * Consumer satisfaction reward (\(r_{c}\)): This component of the reward motivates the agent to allocate an appropriate amount of charging power to each consumer to
keep their SoC within limits, i.e., SoC\({}_{I}\leq\) SoC\({}_{m}\leq\) SoC\({}_{h}\). The satisfaction reward is calculated using different functions that correspond to various segments of the SoC values, as follows. \[r_{c,i}= \begin{cases}n_{1}&\text{if SoC}_{i}>\text{SoC}_{h}\\ \left(\frac{n_{2}-n_{1}}{\text{SoC}_{m}-\text{SoC}_{h}}\right)(\text{SoC}_{i}- \text{SoC}_{h})+n_{1}&\text{if SoC}_{h}\leq\text{SoC}_{i}<\text{SoC}_{m}\\ \left(n_{2}\cdot e\left(\frac{\ln(1/n_{2})}{\text{SoC}_{l}-\text{SoC}_{m}}\cdot (\text{SoC}_{i}-\text{SoC}_{m})\right)\right)-1&\text{if SoC}_{m}\geq\text{SoC} _{i}\geq\text{SoC}_{l}\\ -n_{1}&\text{if SoC}_{i}<\text{SoC}_{l}\end{cases}  \tag{7}\]  
\[r_{c}=\frac{1}{n_{1}}\sum_{i=1}^{N}r_{c,i} \tag{8}\]  
where \(r_{c}\) represents the total customer satisfaction reward, constrained to the range (0, 1). The individual reward \(r_{c,i}\) for each customer depends on their battery's SoC, following a piecewise function composed of distinct segments. These segments are defined as:  
* Fixed Penalty (SoC < SoC\({}_{I}\)): When the SoC is below a low threshold (SoC\({}_{I}\)), the reward is a fixed penalty of \(-n_{1}\). This penalty discourages low battery levels, ensuring that customer satisfaction remains a priority by avoiding situations where batteries are critically low. * Exponential Equation (SoC\({}_{I}\leq\) SoC < SoC\({}_{m}\)): As the SoC increases from SoC\({}_{I}\) to a medium threshold (SoC\({}_{m}\)), the reward follows an exponential curve, starting from 0 at SoC\({}_{I}\) and rising to \(n_{2}\) at SoC\({}_{m}\). This segment of the reward function aims to provide a smooth and increasing incentive as the battery charge increases, reflecting an improved customer experience. * Linear Equation (SoC\({}_{m}\leq\) SoC < SoC\({}_{h}\)): When the SoC is between SoC\({}_{m}\) and SoC\({}_{h}\), the reward increases linearly from \(n_{2}\) to \(n_{1}\). This linear increment incentivizes maintaining battery levels within this range, which is considered optimal for balancing power supply and customer satisfaction. * Fixed Reward (SoC \(\geq\) SoC\({}_{h}\)): Once the SoC exceeds SoC\({}_{h}\), the reward is fixed at \(n_{1}\). This fixed reward ensures that there is no incentive to overcharge the batteries above SoC\({}_{h}\). We want the agent to keep the SoC between SoC\({}_{I}\) and SoC\({}_{h}\). The system uses an actor-critic algorithm framed as a Markov Decision Process (MDP), which is particularly useful to deal with continuous action spaces. This algorithm optimizes policies by training separate models for the policy (actor) and the value function (critic), allowing for better control and exploration in complex environments. The experimental results using real-world data underscore the benefits, including reduced power consumption, improved consumer satisfaction by minimizing critical SoC instances, and enhanced fairness, with the highest fair reward increasing by 173.7%. The total reward also rose by 94.1%, demonstrating the efficiency of the RL method. Unlike the work in [13], which assumes all consumers are honest, this paper aims to investigate the vulnerability of the system to FDI attacks where consumers behave maliciously to increase their chances of charging and propose a countermeasure.