# WGAN-Based Realization Process of Gravel Soil for Hydraulic Property Simulation
## 3 Model Training
### Loss Function of the WGAN with Gradient Penalty (GP)
They focus on minimizing differences between distributions, which are measured by the Wasserstein distance. Weight clipping: WGANs enforce a Lipschitz constraint using weight clipping. After each discriminator parameter update, weights are clipped to prevent them from exceeding a fixed constant \"c\". This constraint stabilizes training and prevents the discriminator from overpowering the generator. Optimization algorithms: Momentum-based optimizers like Momentum and Adam, which are commonly used in traditional neural networks, are not ideal for WGANs. Instead, RMSProp and SGD are recommended due to their stability during training, especially in conjunction with weight clipping and the absence of sigmoid activation functions. These changes collectively address the instability issues commonly encountered in training original GANs, such as mode collapse (where the generator fails to explore the entire data distribution) and discriminator overpowering (where the discriminator becomes too effective, making it difficult for the generator to improve). By focusing on the Wasserstein distance and implementing these modifications, WGANs have shown improved stability and generated image quality in various applications of generative modeling. The proposed Wasserstein loss function based on the Wasserstein distance between \(p_{data}\) and \(p_{G}\) is shown in Equation (4):  
\[W\big{(}G_{\theta},D_{\varphi}\big{)}=E_{x_{r}\sim p_{data}}D_{\varphi}(x_{r} )-E_{z\sim p_{z}}D_{\varphi}(G_{\theta}(z)) \tag{4}\]  
where \(D_{\varphi}\) does not have the sigmoid function in the last layer and constrains the backpropagate training parameters in the range of (\(-\)c, c) by weight clipping. [PERSON] et al. (2017) [8] improved weight clipping further using an alternative method to penalize the norm of the gradient of the discriminator with respect to its input. The proposed method performs better than standard WGANs and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning. This improved loss function is shown in Equation (5):  
\[W\big{(}G_{\theta},D_{\varphi}\big{)}=E_{x_{r}\sim p_{data}}D_{\varphi}(x_{r} )-E_{z\sim p_{z}}D_{\varphi}(G_{\theta}(z))-\lambda E_{\hat{x}\sim p_{\hat{x} }}\Big{[}\big{(}||\
abla_{\hat{x}}D_{\varphi}(\hat{x})||_{2}-1\big{)}^{2}\Big{]} \tag{5}\]  
where \(\lambda\) is the penalty coefficient and the default value is 10, and \(p_{\hat{x}}\) is the distribution of the sampled \(\hat{x}\) between \(x_{G}\sim p_{G}\) and \(x_{r}\sim p_{data}\) by the _\(t\)-uniform_ (0,1) function, i.e., \(\hat{x}=t\alpha_{r}+(1-t)x_{G}\). In our study, we took the loss function of the WGAN with the gradient penalty as the minimizing objective for backward propagation to ensure more stable training performance. We utilized PyTorch to train and test a model for digital realization reconstruction. It begins with preparing datasets by placing them in a designated folder. Ensuring the PyTorch environment, users configure key variables and hyperparameters such as learning rate and batch size. Training is initiated by executing main.py, while pre-execution checks include verifying dataset formats and adjusting any hardcoded paths in the code. For testing, users modify the path in test.py to load the generator model, facilitating the reconstruction of digital realizations when executed within the PyTorch environment. Throughout training, essential data such as loss functions, MDS data, and model states are automatically saved, facilitating subsequent analysis and utilization. This structured approach ensures systematic setup, execution, and validation of the digital realization reconstruction process using PyTorch. The code and plotting data for WGAN-GP in the study are freely available at GitHub via [[https://github.com/Zhu-Bi/Gravelsoil_GAN](https://github.com/Zhu-Bi/Gravelsoil_GAN)]([https://github.com/Zhu-Bi/Gravelsoil_GAN](https://github.com/Zhu-Bi/Gravelsoil_GAN))[13], accessed on 1 May 2024.