# Welding Image Data Augmentation Method Based on LRGAN Model
## 2 LRGAN Model
### The Discriminator Part of the LRGAN Model
#### 2.1.2 Loss Function  
The least squares method is an optimization technique used to find the best function fit for the data by minimizing the sum of the squared errors. By applying the least squares method, we can easily estimate unknown data while minimizing the sum of squared errors between the estimated data and the actual data. The instability of the original GAN training process largely arises from using the cross-entropy loss function as the discriminator's loss function. Replacing the cross-entropy loss function with the least squares loss function can effectively address this issue. According to [PERSON]'s principle of best approximation, the least squares method is represented by the following Equation (1). \[E=\sum_{i=1}^{n}\epsilon_{i}^{2}=\sum_{i=1}^{n}\left(y_{i}-y\right)^{2} \tag{1}\]  
where \(y_{i}\) represents the observed multiple sample data, y is the hypothesized fitting model function, and E is the loss function (the sum of squared residuals). The goal of the least squares method is to minimize the loss function. After introducing the least squares method into the improved LRGAN model, the final objective functions for the generator and discriminator networks are as follows:  
\[\min V_{LRGAN}(D)=\frac{1}{2}E_{x-p_{\text{data}}(x)}\Big{[}\big{(}D(x)-b)^{2 }\Big{]}+\frac{1}{2}E_{z-p_{z}(z)}\Big{[}\big{(}D(G(z,y))-a\big{)}^{2}\Big{]} \tag{2}\]  
\[\min V_{LRGAN}(G)=\frac{1}{2}E_{z-p_{z}(z)}\Big{[}\big{(}D(G(z,y))-c\big{)}^{2 }\Big{]}+\frac{1}{2}E_{x-p_{\text{data}}(x)}\Big{[}\big{(}D(x)-b\big{)}^{2}\Big{]} \tag{3}\]  
Figure 3 illustrates the difference between the cross-entropy loss function and the least squares loss function. Even when fake samples are correctly classified, the least squares loss function still imposes a penalty on them, pulling them toward the decision boundary. The cross-entropy loss function has a large saturated region on the right side where the gradient is zero. When samples fall within this region, the generator's update becomes significantly weakened. In contrast, the least squares loss function only saturates at one point, allowing the LRGAN training process to continue without interruption. Figure 2: LRGAN discriminator structure diagram. The goal of the least squares method is to minimize the loss function. In traditional GAN, when the discriminator network is trained optimally, the objective function of the generative adversarial network \(V(D_{G}^{*},G)\) becomes a Jensen-Shannon divergence problem, where \(D_{G}^{*}=\frac{P_{data}(x)}{P_{data}(x)+P_{g}(x)}\) is obtained:  
\[C(G) =V(D_{G},G)=-\log 4+KL\left(p_{data}(x)\mid\mid\frac{P_{data}(x)+p_{g} (x)}{2}\right)+KL\left(p_{g}(x)\mid\mid\frac{P_{data}(x)+p_{g}(x)}{2}\right) \tag{4}\] \[=-\log 4+2 JSD\left(p_{data}(x)\mid\mid p_{g}(x)\right)\]  
Here, D is a binary classifier that distinguishes between real and fake, x is the input real data, p(x) is the distribution of the real training samples, G is the generative network, and C(G) represents the global minimum of the objective function of the generative network G. The formulas for the Kullback-Leibler (KL) divergence and the Jensen-Shannon (JS) divergence are as follows:  
\[KL(p\mid q)=-\int p(x)\ \log\frac{q(x)}{p(x)}\ dx \tag{5}\]  
\[JSD(p\mid q)=\frac{1}{2}KL\left(p\parallel\frac{p+q}{2}\right)+\frac{1}{2}KL \left(q\parallel\frac{p+q}{2}\right) \tag{6}\]  
For the KL divergence, if the two distributions p and q are far apart and have no overlap, the JS divergence in the objective function of the generative network will tend to a constant value of log4, causing the objective function value to approach 0 and resulting in a vanishing gradient problem. LRGAN addresses this issue by using the least squares loss function.