# A Data-Driven Approach for Generating Synthetic Load Profiles with GANs
## 2 Materials and Methods
### Benchmark Models
Table 2 shows the architecture of the vanilla GAN used as a benchmark model. The networks' structures adhere to the recommendations outlined in [21]. The Wasserstein GAN with gradient penalty (WGAN-GP) [28; 37; 38] improves upon the limitations of vanilla GANs by addressing core training instabilities and offering a more meaningful loss function--the Wasserstein loss function [29], believed to help with mode collapse [39], as follows:  
\[L_{D}=E_{\widetilde{x}\sim\mathbb{P}_{g}}\big{[}D\left(\widetilde{x}\right) \big{]}-E_{x\sim\mathbb{P}_{g}}[D(x)]+\lambda\cdot E_{\tilde{x}\sim\mathbb{P}_ {\tilde{x}}}(\mid\
abla_{\tilde{x}}D(\tilde{x})\mid_{2}-1)^{2}, \tag{7}\]  
where \(D(x)\) is the critic output on the real sample, \(D\left(\widetilde{x}\right)\) is the critic output on a fake sample generated by the generator, \(\mathbb{P}_{r}\) is the distribution of real data, \(\mathbb{P}_{g}\) is the distribution of generated data, \(\mathbb{P}_{\tilde{x}}\) is the distribution of interpolated samples between real and fake, and \(\lambda\) is the gradient penalty coefficient. Instead of relying on Jensen-Shannon divergence (as vanilla GANs do), WGAN-GP minimizes the Wasserstein distance between the real and generated data distributions [29]. This distance metric provides smoother gradients and better reflects the similarity between two distributions, which helps guide the generator more effectively during training. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Characteristic** & **Value** \\ \hline \multirow{8}{*}{Generator} & 1 st layer: & Densely connected: 128 units; \\  & 2 nd layer: & activation—LeakyReLU \\  & 3 rd layer: & Batch normalization \\  & 4 th layer: & Batch normalization \\  & Output: & Densely connected: 24 units; \\  & Loss function: & activation—sigmoid \\  & Optimizer & Binary cross-entropy \\  & & Adam (Adaptive moment estimation): \\  & & learning rate = 0.0002; \\ \hline \multirow{8}{*}{Discriminator} & 1 st layer: & Densely connected: 256 units; \\  & 2 nd layer: & Activation—LeakyReLU \\  & 3 rd layer: & Dropout: dropout rate = 0.3 \\ \cline{1-1}  & 4 th layer: & Densely connected: 128 units; \\ \cline{1-1}  & & activation—LeakyReLU \\ \cline{1-1}  & 4 th layer: & Dropout: dropout rate = 0.3 \\ \cline{1-1}  & & Densely connected: 1 unit; \\ \cline{1-1}  & & activation—sigmoid \\ \cline{1-1}  & Loss function: & Binary cross-entropy \\ \cline{1-1}  & Optimizer & Adam (adaptive moment estimation): \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Architecture of the vanilla GAN used in the benchmark process. Another key innovation in the WGAN-GP is the introduction of a gradient penalty (called the \"critic\" in most cases), as follows:  
\[GP=E_{\hat{x}\sim P_{\hat{x}}}(\mid\
abla_{\hat{x}}D(\hat{x})\mid_{2}-1)^{2}, \tag{8}\]  
where \(\hat{x}=ax+(1-\alpha)\overset{\sim}{x}\) is a random interpolation between the real and fake samples, and \(\
abla_{\hat{x}}D(\hat{x})\) is the gradient of the critic's output. This penalty replaces the less effective method of weight clipping used in the original WGAN [40] and leads to much more stable training, especially for complex data types. The critic no longer needs to classify samples as real or fake with hard probabilities but, instead, learns to assign real-valued scores indicating how close a sample is to the true data distribution. For electrical load profiles, this architecture is especially useful, as it can better model subtle distributional differences in energy consumption. The advantages of WGAN-GP are especially evident in the domain of synthetic data generation. Compared to vanilla GANs, the WGAN-GP provides a more stable training process, reduces the likelihood of mode collapse, and facilitates the generation of diverse and high-fidelity samples. These characteristics make WGAN-GP suitable for generating synthetic data.