## 5 Theoretical Analyses
### Optimality of the Generator  
Now, let us focus on the optimal generator \(G\). We discuss this issue in both \(\beta\to 0\) and \(\beta>0\) cases.  
**Proposition 2**.: _When \(\beta\to 0\) and \(\mu_{D}(x)=\mu_{D}^{*}(x)\) for any \(x\), the global optimum of Equation (5) is achieved if and only if \(P_{g}=P_{d}\) almost everywhere._  
Proof.: Let \(\Omega_{+}=\{x:p_{d}(x)>p_{g}(x)\}\), \(\Omega_{-}=\{x:p_{d}(x)<p_{g}(x)\}\) and \(\Omega_{0}=\{x:p_{d}(x)=p_{g}(x)\}\). According to Equation (8), we have  
\[\lim_{\beta\to 0}\mu_{D}^{*}(x)=\begin{cases}1&\Omega_{+}\\ \frac{1}{2}&\Omega_{0}\\ 0&\Omega_{-}\end{cases}\]  
When given \(\mu_{D}^{*}(x)\) and \(\beta\to 0\), the training criterion for the generator \(G\) is to minimize the following function \(Q\) according to Equation (5):  
\[Q=\int_{\Omega}[p_{d}(x)\mu_{D}^{*}(x)+p_{g}(x)(1-\mu_{D}^{*}(x))]\mathrm{d}x\]  
Then, using the trick of adding and subtracting the same object, we obtain the following:  
\[Q =\int_{\Omega}[p_{d}(x)\mu_{D}^{*}(x)+p_{g}(x)(1-\mu_{D}^{*}(x))] \mathrm{d}x\] \[=\int_{\Omega_{+}}p_{d}(x)\mathrm{d}x+\int_{\Omega_{-}}p_{g}(x) \mathrm{d}x+\int_{\Omega_{0}}p_{g}(x)\mathrm{d}x\;-\int_{\Omega}p_{g}(x) \mathrm{d}x+\int_{\Omega}p_{g}(x)\mathrm{d}x\] \[=\int_{\Omega_{+}}(p_{d}(x)-p_{g}(x))\mathrm{d}x+\int_{\Omega_{- }}(p_{g}(x)-p_{g}(x))\mathrm{d}x+\int_{\Omega_{0}}(p_{g}(x)-p_{g}(x))\mathrm{d }x+\int_{\Omega}p_{g}(x)\mathrm{d}x\] \[=\int_{\Omega_{+}}(p_{d}(x)-p_{g}(x))\mathrm{d}x+1\]  
Figure 2: The schematic diagram of the training process for SoftGAN. The y-axis represents \(\mu_{D}(x)\).  
Consequently, \(Q\) reaches the minimum value if and only if \(P_{g}=P_{d}\) almost everywhere.  
When \(\beta>0\), the algorithm will force the support set of \(P_{g}\) to approximate to the support set of \(P_{d}\): \(supp(P_{g})=supp(P_{d})\).  
For the training of \(P_{g}\), according to our optimizing trick as shown in Equation (7), we have the following:  
\[\min_{p_{g}}\int_{\Omega}p_{g}(x)\mu_{\
eg D}^{*}(x)dx=\max_{p_{g}}\int_{ \Omega}p_{g}(x)\mu_{D}^{*}(x)\mathrm{d}x\]  
According to Equation (8), we have the following:  
\[\mu_{D}^{*}(x)\in\begin{cases}(\frac{1}{2},1]&\Omega_{+}\\ &\Omega_{0}\\ &\Omega_{-}\end{cases}\]  
Thus, we have the following form:  
\[\int_{\Omega}p_{g}(x)\mu_{D}^{*}(x)\mathrm{d}x\] \[=\int_{\Omega_{+}\cup\Omega_{0}\cup\Omega_{-}}p_{g}(x)\mu_{D}^{* }(x)\mathrm{d}x\] \[\geq\frac{1}{2}\int_{\Omega_{+}\cup\Omega_{0}}p_{g}(x)\mathrm{d }x+\int_{\Omega_{-}}p_{g}(x)\mu_{D}^{*}(x)\mathrm{d}x\] \[=\frac{1}{2}\int_{\Omega_{+}\cup\Omega_{0}\cup\Omega_{-}}p_{g}(x )\mathrm{d}x-\frac{1}{2}\int_{\Omega_{-}}p_{g}(x)\mathrm{d}x+\int_{\Omega_{-} }p_{g}(x)\mu_{D}^{*}(x)\mathrm{d}x\] \[=\frac{1}{2}+\int_{\Omega_{-}}p_{g}(x)(\mu_{D}^{*}(x)-\frac{1}{2} )\mathrm{d}x\]  
When \(x\in\Omega_{-}\), we have \(\mu_{D}^{*}(x)-\frac{1}{2}<0\). So, in order to maximize the lower bound, we have either \(p_{g}\to p_{d}\) for \(x\in\Omega_{-}\) or \(\Omega_{-}\to\emptyset\). In other words, when \(\beta>0\), for any \(x\in\Omega\), either \(p_{d}(x)>p_{g}(x)\) or \(p_{g}(x)=p_{d}(x)\).  
In addition, we also need to avoid the situation \(p_{g}(x)=0\) for the goal of the generator. In summary, we have the above Proposition 3.  
According to this proposition, we can conclude that in the SoftGAN \(\beta\) not only makes a global search of distribution comprehensively but also assigns the non-zero density of the generated distribution to the support set of the true distribution. This kind of open-minded strategy naturally avoids the mode collapse problem. So, in the very early training stage, the generated distribution \(P_{g}\) will have enough multiple modes.