## 4 SoftGAN
In total, for our proposed SoftGAN, the value function \(V(D,G)\) is defined as follows:  
\[V(D,G)=\mathop{\mathbb{E}}_{x\sim P_{d}}[\mu_{D}(x)+\beta H_{D}(x)]+\mathop{ \mathbb{E}}_{z\sim P_{z}}[\mu_{\
eg D}(G(z))+\beta H_{\
eg D}(G(z))] \tag{4}\]  
When incorporating both the goals of the discriminator \(D\) and the generator \(G\), the minimax game between \(D\) and \(G\) is as follows:  
\[\min_{G}\max_{D}V(D,G) \tag{5}\]  
The \(\beta\) in Equation (4) is the weight of the fuzzy entropy term (we also can call it the encouragement term), which is used to control the uncertainty of the borderline between real samples and fake samples. The contribution of the \(\beta\) to regulating the entropy is three-fold:  
1. In the early training stage, the larger \(\beta\) can make a global search, which pushes the generated distribution \(P_{g}\) to have multiple modes, since it forces the support set of \(P_{g}\) to be the same as the support set of \(P_{d}\): \(\text{supp}(P_{g})=\text{supp}(P_{d})\) (see the theoretical analysis in Proposition 3 of Section 5). 2. In the early training stage, the larger \(\beta\) can also be helpful to improve the tolerance of the discriminator, which will be effective to avoid gradient vanishing and to accelerate the training process. 3. As the capability of the generator increases, the discriminator also becomes very rigorous. The smaller \(\beta\) can make a local search, and the generator is forced to produce samples similar to the real ones, which means the generated quality will be enhanced. In brief, we call this process the **dynamic borderline softening mechanism**. To better illustrate the dynamic borderline softening mechanism, let us use the analogy of a tutor and a student instead of an art forger and an art expert. The generator, acting as the student, aspires to learn effectively, while the discriminator, acting as the tutor, aims to guide the student in the best possible way. Initially, the student struggles to grasp the key points and barely completes the task. At this stage, the tutor appropriately lowers the standards, allowing the student to pass exams. This approach provides the student with more encouragement and confidence, fostering improvement. As the student progresses, the tutor gradually raises the requirements to motivate the student to develop in a more advanced direction. Through continuous encouragement, the student steadily grows and meets the qualifications even as the standards improve. Ultimately, the student becomes capable of excelling even when faced with challenging problems. Similarly, let us examine the issue of mode collapse from the perspectives of the tutor and the student. The reason behind the existence of mode collapse lies in the generator discovering that certain modes it generates receive high evaluations from the discriminator. Consequently, it tends to solely focus on generating those modes and avoids generating modes where its performance is lacking. This can be compared to a student who excessively focuses on or enjoys certain subjects while disliking others where they struggle to achieve good grades. In order to receive praise from their tutor, the student consistently works on improving their skills in subjects they excel at while avoiding the challenges posed by subjects they struggle in. This phenomenon results in them leaning towards their strengths rather than a balanced overall learning experience. To address this problem, the tutor should encourage the student to maintain a balanced approach to their studies. The tutor can achieve this by initially setting lower standards when the student does not display a clear preference towards any subject. Subsequently, the tutor can gradually raise the requirements in a balanced manner to motivate the student towards overall progress. We can also reconsider the dynamic borderline softening mechanism from an optimization viewpoint. The weighted fuzzy entropy term (encouragement term) can be seen as a penalty term that helps in smoothing the curve of the objective function.