## 4 SoftGAN  
In this section, we propose the SoftGAN, which solves the training instability and mode collapse problem of the GAN simultaneously. It is a new discriminator mechanism that can dynamically adjust the tolerance of the discriminator to samples. We will explain the details of our approach below. The training failure in the original GAN can be attributed to the discriminator completely differentiating between real and generated samples. It creates a hard boundary (or cliff) between the discriminant outcomes, preventing the generator from receiving practicable gradients and impeding its improvement, as depicted in Figure 1. Building upon this observation, we introduce a novel training objective for the discriminator. In our SoftGAN, the discriminator aims to implicitly learn a fuzzy concept of samples from distribution. And the output of the discriminator for each real/generated sample is the degree of membership of the sample to the fuzzy concept. During the early stages of training, the boundary of the fuzzy concept is uncertain. So, the discriminator will not completely negate the generator to lead to the problem of gradient vanishing. As the training progresses, the boundary of the fuzzy concept shrinks and the uncertainty decreases. Correspondingly, the ability of the discriminator increases with the decrease in uncertainty, and the generated distribution learned by the generator gradually approaches the distribution of real data. Formally speaking, the discriminator tries to learn a fuzzy concept \(D\) and its negation \(\
eg D\) to describe real samples from \(P_{d}\) and fake samples from \(P_{g}\), respectively. And the borderline between \(D\) and \(\
eg D\) should be as soft as possible. In the following, we use \(\mu_{D}(x)\in[0,1]\) to represent the degree of sample \(x\) belonging to the fuzzy concept \(D\). In other words, \(\mu_{D}(x)\) represents the degree of the concept coverage of \(D\) to the sample \(x\). Then, the entropy  
\[H_{D}(x)=-\mu_{D}(x)\ln\mu_{D}(x)-\mu_{\
eg D}(x)\ln\mu_{\
eg D}(x) \tag{3}\]  
reflects the degree of sample \(x\) being a borderline case between \(D\) and \(\
eg D\), where \(\mu_{\
eg D}(x)=1-\mu_{D}(x)\). We name this entropy fuzzy entropy of \(D\). In particular, \(H_{\
eg D}(x)=H_{D}(x)\), which means that a fuzzy concept has the same uncertainty of borderline with its negation. According to the above definition, we have the following:  
* \(\underset{x\sim P_{d}}{\mathbb{E}}[\mu_{D}(x)]\) reflects the concept coverage of \(D\) with respect to the true distribution \(P_{d}\). * \(\underset{x\sim P_{g}}{\mathbb{E}}[\mu_{\
eg D}(x)]\) reflects the concept coverage of \(\
eg D\) with respect to the generated distribution \(P_{g}\)
* \(\underset{x\sim P_{d}}{\mathbb{E}}[H_{D}(x)]\) reflects the uncertainty of the borderline between \(D\) and \(\
eg D\) with respect to distribution \(P_{d}\). * \(\underset{x\sim P_{g}}{\mathbb{E}}[H_{\
eg D}(x)]\) reflects the uncertainty of the borderline between \(\
eg D\) and \(D\) with respect to distribution \(P_{g}\). According to the work by [PERSON] and [PERSON] [12], when learning a fuzzy concept \(D\) from samples we should adopt two learning principles of **maximum concept coverage** and **maximum fuzzy entropy**:  
\[\underset{D}{\max}[\mu_{D}(x)+\beta H_{D}(x)]\]  
where \(\beta(>0)\) is a factor compromising the concept coverage and the fuzzy entropy. So, in order to learn a fuzzy concept \(D\) and its negation \(\
eg D\) for distributions \(P_{d}\) and \(P_{g}\), respectively, the discriminator \(D\) has the following two subobjective functions:  
\[\underset{D}{\max}\underset{x\sim P_{d}}{\mathbb{E}}[\mu_{D}(x)+\beta H_{D}( x)]\]  
Figure 1: The discriminator of the original GAN is responsible for distinguishing between two distributions. At the beginning of training, there exists a distinct boundary (or cliff) between the discriminant outcomes, which prevents the generator from receiving the gradient information it needs to enhance its performance.