## 3 False Data Injection Attacks on RL-Based Charging Coordination
### Evaluations of FDI Attacks
#### 3.2.1 Training a RL-Based Charging Coordination System  
To train the RL-based charging coordination system proposed in [13] to assess its vulnerability to the FDI attacks discussed earlier, we utilized the publicly available Ausgrid dataset from an Australian electricity provider [28]. This dataset contains real-world data on power consumption and solar energy generation from households equipped with solar panels in Sydney and New South Wales. It includes 30-min interval readings collected over three years, from 1 July 2010, to 30 June 2013. Each home had two meters: one to measure grid electricity consumption and another to track solar power generation.  
Several preprocessing steps were applied to the raw data, such as outlier removal, net metering calculations (which account for the difference between grid consumption and solar generation for each household), and resampling to generate 24 hourly readings per day. The refined dataset included net meter readings for 10 houses over 1096 days. This data was used to calculate \(C_{max}\), the maximum grid capacity, a critical parameter in the model's state representation. We considered two scenarios for \(C_{max}\): one where it equals the average consumption of all 10 consumers (\(C_{max}=6\)), and another where it exceeds the average (\(C_{max}=10\)). Then, we simulated a battery for each house and based on the amount of power consumption and generation, we computed an SoC value for a time slot of 30 min. Finally, we ran the RL-based charging coordination system, and the SoC value of each battery was dynamically updated based on actions taken in the previous state. For each consumer, each sample in the dataset contains 24 SoC values for one day.  
The hyperparameters used in the RL-based charging coordination system are given in Tables 4 and 5. Key elements such as gradient clipping, replay buffer size, and target network update rate were included to ensure stable learning.  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Learning Rate (1r) & 0.01 \\ \hline Discount Factor (\(\gamma\)) & 0.98 \\ \hline Clip Ratio (\(\epsilon\)) & 0.2 \\ \hline GAE Lambda (\(\lambda\)) & 0.95 \\ \hline Batch Size & 64 \\ \hline Entropy Coefficient (\(c_{2}\)) & 0.01 \\ \hline Max Gradient Norm & 0.5 \\ \hline Number of Steps per Update & 2048 \\ \hline Policy Update Frequency & Every 4 steps \\ \hline \hline \end{tabular}
\end{table}
Table 4: Hyperparameters used for the Actor-Critic RL-based charging coordinator.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & **Reward** & **Fairness** & **Power Usage** & **Avg. Honest SoC** & **Avg. Malicious SoC** \\ \hline Baseline (No Attack) & 0\% & 0\% & 0\% & 57\% & 57\% \\ \hline Attack \#1 & \(-\)15.5\% & \(-\)10.5\% & +5.0\% & 55\% & 60\% \\ \hline Attack \#2 & \(-\)18.0\% & \(-\)12.5\% & +6.5\% & 53\% & 62\% \\ \hline Attack \#3 & \(-\)21.0\% & \(-\)15.5\% & +9.0\% & 51\% & 65\% \\ \hline Attack \#4 & \(-\)24.5\% & \(-\)18.0\% & +11.0\% & 49\% & 67\% \\ \hline Attack \#5 & \(-\)27.5\% & \(-\)20.0\% & +13.0\% & 47\% & 69\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: Impact of Different Attacks on RL Agent’s Performance in case of \(C_{\max}=10\).