Stein Bridging: Enabling Mutual Reinforcement between Explicit and Implicit Generative Models
Authors: Qitian Wu
Rui Gao
Hongyuan Zha
Venue:  (2020)
Keywords: generative models
generative adversarial networks
energy models

Abstract: Deep generative models are generally categorized into explicit models and implicit models. The former assumes an explicit density form whose normalizing constant is often unknown; while the latter, including generative adversarial networks (GANs), generates samples using a push-forward mapping. In spite of substantial recent advances demonstrating the power of the two classes of generative models in many applications, both of them, when used alone, suffer from respective limitations and drawbacks. To mitigate these issues, we propose Stein Bridging, a novel joint training framework that connects an explicit density estimator and an implicit sample generator with Stein discrepancy. We show that the Stein Bridge induces new regularization schemes for both explicit and implicit models. Convergence analysis and extensive experiments demonstrate that the Stein Bridging i) improves the stability and sample quality of the GAN training, and ii) facilitates the density estimator to seek more modes in data and alleviate the mode-collapse issue. Additionally, we discuss several applications of Stein Bridging and useful tricks in practical implementation used in our experiments.

Under review as a conference paper at ICLR 2020
STEIN BRIDGING: ENABLING MUTUAL REINFORCE- MENT BETWEEN EXPLICIT AND IMPLICIT GENERA- TIVE MODELS
Anonymous authors Paper under double-blee blind review
ABSTRACT
Deep generative models are generally categorized into explicit models and im- plicit models. The former defines an explicit density form, whose normalizing constant is often unknown; while the latter, including generative adversarial net- works (GANs), generates samples without explicitly defining density function. In spite of substantial recent advances demonstrating the power the two classes of generative models in many applications, both of them, when used alone, suf- fer from respective limitations and drawbacks. To mitigate these issues, we pro- pose Stein Bridging, a novel joint training framework that connects explicit (un-normalized) density estimator and an implicit sample generator with Stein discrepancy. We show that the Stein Bridge induces new regularization schemes for both explicit and implicit models. Convergence analysis and extensive exper- iments demonstrate that the Stein Bridging improves the stability and sample quality of the GAN training, and ii) facilitates the density estimator to seek more modes in data and alleviate the mode-collapse issue. Additionally, we discuss sev- eral applications of Stein Bridging and useful tricks in practical implementation used in our experiments.
INTRODUCTION
Deep generative model, as a powerful unsupervised framework for learning the distribution of high- dimensional multi-modal data, has been extensively studied in recent literature. Typically, there are two types of generative models (Goodfellow al., 2014). Explicit models define an explicit (unnormalized) density function, while implicit models learn to sample from the distribution without explicitly defining density function.
Explicit models have wide applications in undirected graphical models (LeCun et al. 2006 Salakhutdinov Hinton| 2009; Hinton et al. 2006 Ngiam et al.] (01T). random graph theory (Robins et al. (2007), mnergy-based reinforcement learning (Haarnoj al. 2017). etc. However, the unknown normalizing constant makes the model hard to train and sample from, and the ex- plicit models might not be able to capture the complex structure of true samples while maintaining tractability. In contrast, implicit models are more flexible in training and easy to sample from, and in pariticular, generative adverarial networks (GANs) have shown great power learning repre- sentations of images. natural languages, graphs, etc. (Goodfellow et al. 2014 Radford et al. 2016 Arjovsky et al. 2017; Brock al. 2019), Nevertheless, due the minimax game between generator and discriminator/critic in GANs, the training process often suffers from instability, and produces undesirable samples often associated with missing modes in data or generating extra modes out of data. Therefore, it motivates us to consider jointly learning of two models that can presumably com- pensate and reinforce each other the training process. (More discussions about related works are in Appendix (A)
Furthermore, there are many of situations where we do need both an explicit (unnormalized) density and flexible implicit sampler. For sample evaluation, it is not enough to merely distinguish samples between real and faked one, and one may also expect to provide fine-graine grained evaluation on generated samples, where the energy values given by the explicit models can be a good metric (Dai et al. 2017). Another situation outlier detection. Implicit models often leverage all true samples (possibly mixed with corrupted samples) true examples for training. To make up for the issue, explicit

---

Under review as a conference paper at ICLR 2020
models could help detect cut-oo-distributtie samples via the estimated densities (hai et al. 2016). Also, when given insufficient observed samples, explicit models may fail to capture an accurate distribution, in which case implicit model may help with data augmentation and facilitate training for density estimation. These situations inspire us to combine both of the worlds so as to take advantage of two models in effective way.
In this work, we aim at jointly learning explicit and implicit generative models. In our framework, an explicit energy model is used estimate the unnormalized densities of true samples via mini- mizing a Stein discrepancy; in the meantime, an implicit generator model is exploited to minimize the Wasserstein metric (or Censen-Shannon divergence) between distributions of true and generated samples. On top of these, another Stein discrepancy, acting as bridge between implicit generated samples and explicit estimated densities, introduced and pushes the two models to achieve a con- sensus. We show that the Stein bridge allows the two generative models to reinforce each other by imposing new regularizations on both models, which help the generator to output high-quality samples and facilitate the energy model to avoid mode-collapse. Moreover, we show that the joint training helps to stabilize GAN training via a convergence analysis. Extensive experiments on vari- ous tasks verify theoretical findings well as demonstrate the superiority of proposed methods compared with existing deep energy models and GAN- -ased models.
BACKGROUND
In this section, we briefly provide some technical background used in our model.
Energy Model. The energy model assigns each data x E Rdd with a scalar energy value Fg where (-)-) is called the energy function and is parameterized by ø. The model is expected to assign low energy to true samples according to a Gibbs distribution px(x) = where Ze is normalizing constant dependent on ø. The normalizing term Ze is often hard to compute, making the training intractable, and various methods are proposed to detour such term (see Appendix A))
Stein Discrepancy. Stein discrepancy (Gorham & Mackey 2015 Liu et al. 2016 Chwialkowski
let al. 2016) Oates et al. 2017) is measure of closeness between two probability distributions that does not require the knowledge for the normalizing constant of one of the compared distributions. Let P and Q be two probability distributions on X Rdd and assume Q has (unnormalized) density g. The Stein discrepancy S(P, Q) is defined as
S Q == sup Ex-x[Acf(xx] == sup (Ø(Ex>P(V log (x) + vxf(x)))) feF
(I)
where F is often chosen to be a Stein class (see, e.g.. Definition 2.1 in Liu et al. (2016). f Rd Rdd a vector-valued function called Stein critic and & is an operation that transforms a x d' matrix into scalar value. One common choice" of é is trace operation when d' = d. If F is unit ball in some reproducing kernel Hilbert space (RKHS) with a positive definite kernel k it induces Kernel Stein Discrepancy (KSD). We provide more details in Appendix
Wasserstein Metric. Wasserstein metric is suitable for measuring distances between two distri- butions with non-overlapping supports (Arjovsky et al. 2017). The Wasserstein-1 metric between distributions IP and Q is defined as W(P, Q) := min. Exx.y>~>llxx - yl, where the minimization is over all joint distributions with marginals P and Q. By Kantorovich-Rubinsten duality, it has dual representation
W(P, Q) = max (Exxr(D(x) - Ey-o(D( D
where the maximization is over all I-Lipschitz continuous functions.
Sobolev space and Sobolev dual norm. Use I,2 to denote the canonical Hilbert space on Bd equipped with an inner product z) == Sgs uedx. The Sobolev space H1 is defined as the closure of C the set of smooth functions on Rd with compact support, with respect to the norm
Jull == (u
)xll))xx) For u E I.2, its Sobolev dual norm ||||| defined by
Evans & Society 2010)
(1, / Jz
[ivel2 dx L, / u(x)dx 0
|| --1 == sup NEM! m!
One can also use other forms for Φ. like matrix norm when d' * d dLiiiue et al. (0066).
2

---

Under review as a conference paper at ICLR 2020
The constraint SRa u(x)dx = 0 is necessary in ensuring the finiteness of the supremum. 11-1 can be viewed measure of smoothness, which measures the similarity (in terms of largest I.²-norm) between v and a subset of smooth functions in H
3
PROPOSED MODEL
In this section, we formulate our model, Stein Bridging, and highlight its regularization effects.
3.1 MODEL FORMULATION
We denote by Preat the underlying real distribution from which the data {x} are sampled. We simultaneously learn two generative models one explicit and one implicit that represent esti- mates of IP real- The explicit generative model has an explicit probability density PE proportional to exp(-E(x)), where is referred as an energy function. The implicit generative model trans- forms an easy-to- to-y-ple random noise z with distribution Po via generator Gi to generated sample (zz) with distribution Pg. We use the Stein discrepancy as measure of closeness between the explicit unnormalized density PE and the real distribution real and use the Wasserstein metric as a measure of closeness between the implicit distribution Pc and P Preat.
To jointly learn the two generative models Pa and Pr, arguably the most straightforward approach is to minimize the sum of the Stein discrepancy and the Wasserstein metric:
min
real, Pc) + (I IF), E,G
where 2 O is weight coefficient. However, this approach appears no different than learning the two generative models separately. To better train the model, we incorporate the objective an- other term S(PG, PÆ) called the Stein bridge that measures the closeness between the explicit unnormalized density Pg and the implicit distribution Po:
min W(Preat, PG) (Freatet S2 F.G
where \1, A2 2 are weight coefficients. Although the Stein bridge might seem redundant mathe- matically, we show that helps regularize the models in Section 3.2
The Wasserstein term in 3 is implemented using its equivalent dual representation (2). The two Stein terms in (3 can be implemented using ① with either Stein critic parameterized by neural network, or the Kernel Stein Discrepancy. To reduce the computational cost, the two Stein critics share their parameters, namely, kernels or neural networks. A scheme of our framework is presented in Fig. [ We also discuss some related works that attempt to combine both of the worlds (such as energy-based GAN, contrastive learning and cooperative learning) in Appendix A.3 and highlight the difference between our method and theirs in terms of the objective in Table
Remark. In general, we can also choose other statistical distances (3) measure closeness be- tween probability distributions. For example, the Wasserstein metric [[rreat,,PPP can be replaced by other common choices for implicit generative models, such as -hanenn divergence used in the original GAN paper (Goodfellow al. [014]. If the normalizing constant of [P is known or easy to calculate, one can replace the Stein discrepancy by the Kullback-Leibh divergence, which equivalent to the maximum likelihood estimation. We present details for model specifications in various forms and training algorithm in Appendix E.2
3.2 REGULARIZATION EFFECTS BY VIRTUE OF THE STEIN BRIDGE
The intuitive motivation of the Stein bridge term in is to push the two models achieve consensus. In this subsection, we theoretically show that the Stein bridge allows the two models to reinforce each other by imposing regularizations the critics.
3.2.1 KERNEL SOBOLEV DUAL NORM REGULARIZATION ON THE WASSERSTEIN CRITIC
We show the regularization effect of the Stein bridge on the Wasserstein critic. Define the kernel Sobolev dual norm as
D 8- H-1(P.)) sup ((Dp)) Exxer (exxx=e(Vu(x)) M(x, 1, Ep[h] = 0} wEO
3

---

Under review as a conference paper at ICLR 2020
Model
Objective
Implicit Model Fc
Explicit Pi
D
(x)
Pa(X]
GAN
D;
Wasserstein
Critic
True Sample
Cnttcc
Stein
:P(()
True Density
Enerpy-hused GAN
Zhuo et al 2017
Energy Model
D;
Da
Density Estimation
Contrastive Leamiing
Kim Bengio 2
D2
(G(())
X
Cooperative Learning
Stein
2018g)
Ds
Sample
Dencaty
Generator
[
Critc
P₂(X)
Stein Bridging (ours)
21 + Da + D2
Stain Bridge
Table 1: Comparison of objectives between different generative mod- els, where D1 D2 = D2(Preat, and D3 = (T Pr) denote general statistical distances between two distributions.
With
Update Impact
Parameter
P. W/2 real: Pa)
D3 S(Frrat Ps.)
D2 S(Fo.Fs)
Dj (IP real. Pc).
Figure 1: Model framework for Stein Bridging which jointly train implicit sample generator and explicit (unnormalized) density estimator via Stein bridge.
which can be viewed as kernel generalization of the Sobolev dual norm defined in Section 2 which reduces to the Sobolev dual norm when l(x, x') = I(x = x') and TP being the Lebesgue measure. We have the following result.
Theorem 1. Assume that {Pa}a exhausts all continuous probability distributions and chosen as kernel Stein discrepancy. Then problem 3 is equivalent to
min max Ey-rr (ywrn(((y)) Ex [D(x)) E
| ||il==1(enss)
D
4A2
According Section 2 the regularization stein critic D, which is in the same spirit of gradient- penalty (e.g.. Gulrajani et al. (2017); Roth al. (2017)), but with new way to encouraging smoothness.
would penalize the non-smoothness of the Wasser-
Another way to interpret the Sobolev dual norm penalty is by observing that if A(x x') = T(xx = x') and and E= [D] (iillani 2008), then
D lim
where W2 denotes the 2- 2-Wasserstein metric. Therefore, the regularization ensures that D would not change suddenly on the high-density region of IPg, and the explicit model reinforces the learning of the Wasserstein critic.
3.2.2 LIPSCIITZ REGULARIZATION ON THE STEIN CRITIC
We next investigate how the Stein bridge helps to regularize the Stein critic. Recall that the two Stein terms in (3) share the same Stein critic. We have the following result.
Theorem 2. Assume {Pata exhausts all continuous probability distributions, and the Stein class
defining the Stein discrepancy is compact (in some linear topological space). Then problem B is equivalent to
min (AISmaaa(XSXPPIA PF) >2Bxx [MaaAr (xx)]]
where f(:) denotes the (generalized) Moreau- Moreau-Yosida regularization of the function Arg f with parameter X i.e., MxaAsgr(x) minyex(AA f(y) + 2/ yll).
Theorem 2 shows that the Stein bridge, together with the Wasserstein metric W(P Pa), plays as a smoothness regularization on the Stein critic via Moreau- Yosida regularization, which smoothens the Stein operator A and further encourages the energy model seek more modes in data instead of focusing on some dominated modes, thus helps alleviate the mode-collapse issue. To the best of knowledge, this suggests novel regularization scheme for Stein-based GAN.
4

---

Under review as a conference paper at ICLR 2020
4 CONVERGENCE ANALYSIS
In Section 3.2 we justify Stein Bridging by showing the regularization effects. In this section, we further show that it could help to stabilize GAN training with local convergence guarantee. To this end, we first compare the behaviors of WGAN, likelihood- and entropy-regularized WGAN, and our Stein Bridging under SGD via an easy to comprehend toy example. Then we give a formal result that interprets why the introduction of (unnormalized) density estimator could stablize GAN training and help for convergence.
4.1 ANALYSIS OF LINEAR SYSTEM
The training for minimax game in GAN is difficult. When using traditional gradient methods, the training would suffer from some oscillatory behaviors (Goodfellow 2017): [iing & Stokes (2019)- In order to better understand the optimization behaviors, we first study one-dimension linear sys- tem that provides some insights on this problem. Note that such toy example (or a similar one) is also utilized Gidel et al. 2019); Nagarajan & Kolter (007)) shed lights the instability of WGAN training Consider a linear critic (z) = रंz and generator Gø(z) = Oz. Then the Wasserstein GAN objective can be written as constrained bilinear problem: ming ma e (0E which could be further simplified as an unconstrained version (the behaviors could be generalized to multi-dimensional cases (Gidel et al. (2019))
min max J 2 0.
(4)
Unfortunately, such simple objective cannot guarantee convergence by traditional gradient methods like SGD with alternate updating" 0x+1 = 0% + 1///,, t 1/k + 1 0x+1)- Such optimiza- tion would suffer from an oscillatory behavior, i.e., the updated parameters go around the optimum point ((vb", 0*] = [0, 1) forming circle without converging to the centrality, which is shown in Fig. 2 a). A recent study in Liang & Stokes (2019) theoretically show that such oscillation is due to
the interaction term in (4)
One solution to the instability of GAN training is to add (likelihood) regularization, which has been widely studied by recent literatures (Warde-Farley & Bengio (2017): Li & Turner (2088). With regularization term, the objective changes into ming max maaxv [ where (u-)) denotes the likelihood function and À is a hyperparameter. A recent study (Tao et al. (2019) proves that when ing sample evidence, helping to stabilize GAN training; when À > 0
+0E|= XEllog / (0=),,
Mikeelihoodregullarzztiioo
the extra term is equivalent to maximiz-
the extra term maximizes sample entropy, which encourages diversity of generator. Here we consider a Gaussian likelihood function for generated sample x'. =u(x)) exp(- b) which is up to
constant, and then the objective becomes (see Appendix D.I for details):
min max 1/ v/ 0 x(0 0).
(5)
V
The above system would converge with
0 and diverge with > O in gradient-based optimiza- tion, shown in Fig. a Another issue of rrelihood-regularizatii that the extra term changes the op- timum point and makes the model converge biased distribution, as proved by Tao et al. (219)). In this case, one can verify that the optimum point becomes [v',,0' = I-A, re. sulting a bias. To avoid this issue. Tao et al. (2019) proposes to temporally decrease [X] through training. How- ever, such method would also be stuck in oscillation when |X] gets close to zero is shown in Fig. aa.).
12
is
(b) (a) Numerical iterations for SGD training
Figure 2: of WGAN, likelihood-regularized WGAN (WGAN+LR), variational annealing for WGAN+LR (WGAN+LR+VA), entropy-regularized WGAN (WGAN+ER) and our Stein Bridging. (b) Stein Bridging with different λ and X2.
OOr theoretical discussions focus on WGAN, and we also compare with original GAN the experiments. Here, adopt the most widely used altemnate updating strategy. The simultaneous updating, i.c., 0--1
0g + 777 and the+1 = thx + 7711 0z), would diverge in this case.
5

---

Under review as a conference paper at ICLR 2020
Finally, let consider our proposed model. We also simplify the density estimator as a basic energy model Ps (x) = exp( Φx)) whose score function Vz log logPo(z) - Φ. Then if we specify the two Stein discrepancies in as KSD, we have the objective (see Appendix D. for details),
22 min max & 0+ (1+
X1
(0+
(6)
2
Interestingly, one can verify that for AY1, Ag. the optimum point remains the same [6b* 0", 66) [0, -1]. Then we show that the optimization can guarantee convergence to [v* 0* 66).
Proposition 1. Using alternate SGD for (6) geometrically decreases the square norm N1 1 [0 112 + 10 + 112, for any 0 < 1 < with X1 = X2 = 1,
N = (1 - 1f(11 N,
In Fig. 2a), we can see that Stein Bridging achieves a good convergence to the right optimum. Compared with (4), the objective (6) adds a new bilinear term c- 0, which acts like a connection between the two generator and estimator, and other quadratic terms, which help to push the
values to decrease through training. The added terms and the original terms in (6) cooperate to guarantee convergence to a unique optimum. (More discussions in Appendix [..]]
We further generalize the analysis to multi-dimensionn bilinear system F(bb, 0) = A b c Tub which is extensively used by researches for analysis of GAN stability (Goodfellow (017)); Gemp & Mahadevan (2018): Liang & Stokes (2019); Gidel et al. 2019).. For any bilinear system, with the added term H( 0) 1 Φ) B x) where B = (AAT)a to the objective, we
can prove that i) the optimum point remains the same as the original system (Proposition ( and ii)
using alternate SGD algorithm for the new objective guarantee convergence (Theorem 4. The
results are given in Appendix D.3
4.2 LOCAL CONVERGENCE FOR GENERAL MODEL.
To study the convergence for Stein Bridging, we proceed to consider general optimization objective
min max min L(O0, 1b, Φ),
where L(0, th, =( + Φ), and wnd [0, T/} and w/ 10, Φl (0 a shared parameter set). Use w' = -0' "0""""" o6] to denote the optimum point of L and w = [0", "0, w = [0", ø²1 represent the optimum points of and H respectively. Define Q2 = Gg x S2 and 2g = Gg x Ωg for sb, Φ respectively. Function H j-strongly convex,
where Ωg. Sy. Sig denote constraint and F' is ju-ttronlyy convex for 0 and u-strongly concave for eù (see Appendix [.4] for definition of strongly convex condition). Here we define h(w,) VoH VaH. f(w)) VoF F. and then we have the following theorem.
Theorem 3. If F is j--strongly convex-concave and H is p-stroogly convex, we can leverage the alternate SGD algorithm, i.e.
w 1+1 = (w) 2+1/2 n/((w// (+1/21), /1((w,+1/2)))
(8)
w/t1 = Pay (w) 1+1/2 nf nf(ww(t(/// (+1122))
(9)
where w/ 2+1/2 [0' *b h
10 1bt 2/12 | 121 w/
and Pa(w) = arg min
|| w112 denotes the projection mapping to Ω. Then we can achieve the
En convergence by using
<
Theorem 3 shows that Stein Bridging could converge to at least a local optimum. Due to the un- known and intricate landscape of deep neural networks, the global optimization and convergence analysis for GAN has remained as an unexplored problem. Despite the fact that strong convexity assumption cannot be guaranteed with deep neural networks, the optimization could converge to stable point once there exists local region that satisfies the strongly convex conditions. In the experiments, we will empirically compare the training stability of each method on various datasets to validate our theoretical discussions.

---

Under review as a conference paper at ICLR 2020
Figure 3: (a) True sam- ples and (b)~(f) gener- ated samples produced by the generators of different methods on Two-Circle (upper line) and Two-Spiral (bottom line) datasets.
S
(a) True
(b) GAN
(c) WGAN
(d) DGM
(e) Joint-JS
(f) Joint-W
Figure 4: (a) True den- sities and (b)~(f) es- timated densities given by the estimators of different methods on Two-Circle (upper line) and Two-Spiral (bottom line) datasets.
S
S
S
S
(a) True
(b) DEM
(c) EGAN
(d) DGM
(c) Joint-JS
(f) Joint-W
5 EXPERIMENTS
In this section, we conduct experiments to verify the effectiveness of proposed method from multi- faceted views. First, we select three tasks with different evaluation metrics in Section 5.1, 5.2 and 5.3. Then we further discuss some applications of joint training as well as some useful tricks in Section 5.4, 5.5 and 5.6. The codes will be released later.
We consider two synthetic datasets with mixtures of Gaussian distributions: Two-Circle and Two- Spiral. The first one is composed of 24 Gaussian mixtures that lie in two circles. Such dataset is extended from the caauussian-mixture scenario which is widely used previous GAN papers and is more difficult, that of learned energy. The second synthetic dataset consists of 100 Gaussian mixtures whose centers densely arranged on two centrally symmetrical spiral-shaped curves. This dataset be used to examine the power of generative model on complicated data distributions. The ground-truth distributions and samples are shown in Fig. (a) and Fig. 4 (a). Furthermore, we also apply the methods to MNIST and CIFAR datasets which require the model to deal with high-dimensional data. In each dataset, use observed samples as input of the model and leverage them train the generators and the estimators. The details for each dataset are reported in Appendix E.
use it to test the quality of generated samples and mode coverage
In our experiments, we also replace the Wasserstein metric in 3 by JS divergence. To well distin- guish different specifications, we term the model Joint-W W if using Wasserstein metric and Joint-JS using JS divergence this section. We consider several competitors. First, for implicit genera- tive models, we consider valina GAN, WGAN-GP (Gulrajani et al.) (017)). likelihood-regularizeed GANWGAN-GG (short as GAN+LR/WGAN+LBR cntropy-regularized GANWGAN--G (short as GAN+ERWGAN+ERR and recently proposed variational annealing regularization (Tao et al. (019 ) for GAN (short as
to compare the quality of generated samples. We employ the denoising auto-encoder to estimate the gradient for regularization penalty, which is pro- posed by Alain & Bengio (2014) and utilized by Tao et al. (2019). Second, for explicit models, we consider Deep Energy Model (DEM) which is optimized based Stein discrepancy, and energy- based GAN (EGAN) (Dai et al. (207)). Besides, we also compare with Deep Directed Generative (DGM) Model (Kim & Bengio (017)) which adopts contrastive divergence to unite sample gener- ator and energy estimator. See Appendix A for brief introduction of these methods and Appendix for implementation details for each method.
5.1 SAMPLE QUALITY OF IMPLICIT MODEL
Calibrating explicit (unnormalized) density model with implicit generator is expected to improve the quality of generated samples. In Fig. 3 and Fig. 4 we show the results of different generators in Two-CCrcle and Two-Spiral datasets. As see, Two-Circle, there large number of gen-

---

Under review as a conference paper at ICLR 2020
MNIST (Conditional)
MNIST (Unconditional)
CIFAR-I0 (Unconditional)
Method
Score
CEPC
Method
Score
CEPC
Method
Score
CEPC
DCGAN
8.43
0.168
WGAN GGGNN-GP
7.71
0.256
WGAN-GI GP
6.80
0.153
DCGAN+LR
8.40
0.171
WGAN+LR
7.82
0.243
WGAN+LR
6.89
0.154
DCGAN+ER
8.33
0.179
WGAN+ER
7.75
0.252
WGAN+ER
6.99
0.156
DCGAN+VA
8.40
0.172
WGAN+VA
7.74
0.254
WGAN+VA
6.95f
0.154
DGM
8.15
0.201
DGM
6.87
0.372
DGM
4.79
0.146
Joint-JS(ours)
8.53
0.156
loirt-W(ours)
7.90
0.231
Joint- (ours)()urs)
7.11
0.151
Table 2: Inception scores (higher is better) and conditional entropies (short as CEPC and lower is better) on MNIST and CIFAR-10, i We directly use the best result reported in their paper.
Emcatio
Wig 2006
Eeeetion
Deeration
Beritlon
(a) Two-Cirele
(b) Two-Spiral
(c) MNIST (Condition) (d) MNIST (Uncondition)
Figure 5: Learning curves of Joint-W (resp. Joint-JS) compared with WGAN (resp. GAN DC-
GAN) and its regularization bibbiattti----ee variants.
erated samples given by GAN, WGAN-GP and DGM (the worst one in this case) locating between two Gaussian components, and the boundary for each component is not distinguishable. Since the ground-truth densities of regions between two components are very low, such generated samples possess low-quality, which depicts that these models capture the combinations of two dominated features (i.e., modes) in the data but such combination does make sense in practice. By con- trast, Joint-JS and Joint-W W could alleviate such issue, reduce the low-quality samples and produce more distinguishable boundaries for components. In Two-Spiral, similarly, the generated samples given by GAN and WGAN-GP form a circle instead of two spirals while the samples of DGM 'link' two spirals. Joint-IS manages to focus more on true high densities compared GAN and Joint- W provides the best results. To quantitatively measure the sample quality, we adopt two metrics: Maxi- mum Mean Discrepancy (MMD) and High-quality Sample Rate (HSR). The detailed definitions are given in Appendix E.4 and we report the results in Table
We visualize the generated digits/images on MNIST/CIFAR-II datasets in Fig. and Fig. 10 and use Inception Score and conditional entropy of predicted classes (CEPC) to measure the sample quality (See Appendix [E.4 for details). As shown in Table [ Joint-W (resp. Joint-JS) is superior than WGAN-GP (resp. DCGAN), regularized WGAN (resp. DCGAN) and DGM. The CEPC char- acterizes how well the picture can be distinguished by a pre-trained classifier, i.e., the quality of picture, the results depict that proposed method could give higher-quality generated pictures.
5.2 DENSITY ESTIMATION OF EXPLICIT MODEL
Another advantage of joint learning is that the generator could help the density estimator to capture more accurate distribution. As shown in Fig 3 both Joint-JS and Joint-W W manage to capture all Gaussian components while other methods miss some of modes. In Fig 4 Joint-JS and Joint-W exactly fit the ground-truth distribution. By contrast, DEM misses one spiral while EGAN degrades uniform-like distribution. DGM manages to fit two spirals but allocate high densities to regions that have low densities in the groung- distribution. To quantitatively measure the performance, we introduce three evaluation metrics: KI. & JS divergence between the ground-truth and estimated densities and Area Under the Curve (AUC) for false-positive rate v.s. truc-positive rate where select points with true high (resp. low) densities as positive (resp. negative) examples. The detailed information and results given in Appendix E.4 and Table 5 respectively. The values show that Joint- W and Joint-JS could provide more accurate (unnormalized) density estimation than other competitors.
We also rank the generated digits (and true digits) on MNIST w.nt the densities given by the energy model in Fig. 11. Fig. 12 and Fig. 13 As depicted in the figures, the digits with high densities (or
8

---

Under review as a conference paper at ICLR 2020
410ch.40
Epoch
Epoch
Ludd 11
Epoch
Epoch 150
Salo Max
(a) Noised Data
(b) Insufficient Data
WGAN-GP
WGAN-VA
Joint-W
Epoch
4
3
1
3
?
5
7
q
3
4 0
1
7
5
Epoch
4
3
7
6
8
9
7
7
5
Epoch
4
2
1
5
8
D
7
7
7
5 5
Epoch
7
2
1
0
8
9
7
6
3
4
7
Epach
4
3
7
5
8
D
7
0
7
200
Rchie 203
(c) Warm up on CIFAR-10
WGAN-GP
WG
Joint-W
Figure 6: Generated digits (resp. images) given by the same noise in adjacent training epoches on MNIST (reps. CIFAR) dataset.
Figure 7: Joint-W W with (a) noised data, (b) insufficient data and (c) "warm up" iterations before joint training.
low densities) given by Joint-JS possess enough diversity (the thickmess, the inclination angles as well as the shapes of digits diverses). By constrast, all the digits with high densities given by DGM tend be thin and digits with low densities are very thick. Also, for EGAN, digits with high (or low) densities appear to have the same inclination angle (for high densities, '1' keeps straight and '9' 'leans' to the left while for low densities, just the opposite). Such phenomenon indicates that DGM and EGAN tend to allocate high (or low) densities to data with certain modes and would miss some modes that possibly possess high densities in ground-truth distributions. Fortunately, our method overcomes the issue and manages to capture complicated distributions.
5.3 ENHANCING THE STABILITY OF GAN
Our discussions and analysis show that joint training helps to stabilize GAN training. In Fig. we present the learning curves of Joint-W (resp. Joint-JS) compared with WGAN (resp. GAN or DCGAN) and its regularization- bassed variants on different datasets. One can clearly see from the curves that joint training could reduce the variance of metric values especially during the second half of training. Furthermore, we visualize the generated pictures given by the same noise z adjacent epoches in Fig. 6 The results show that Joint-W outputs more stable generation in adjacent epoches while the generated samples given by WGAN-GP and WGAN+VA exhibit an obvious variation. Especially, some digits generated by WGAN-GP and WGAN+ VA change from one class to another. Such phenomenon is quite similar to the oscillatory behavior with non-convergence in optimization that we discuss in Section 4.1.
Another issue discussed in Section 4.1 is the bias of model distribution for regularized GAN meth- ods. To quantify this evaluation, we calculate / and l2 distances between the means of 50000 generated digits (resp. images) and 50000 true digits (resp. images) in MNIST (reps. CIFAR-10). The results are shown in Table 3 The smaller distances given by Joint-W indicate that it converges to better local optimum with smaller bias from the original data distribution. Also, in Table Table 7
we report the distances for digits (resp. images) in each class on MNIST (resp. CIFAR).
5.4 DETECTING OUT-OF-DISSE DISTRIBUTION SAMPLES
The explicit model estimates densities for each sample and one of its applications is to detect outliers in the imput data. Here, we adopt CIFAR- 10 to measure the ability of our estimator to distinguish the in-distribution samples and (true/false) out-of -istrirbint samples. We consider four situations and in each case, we consider the test images of CIFAR- 10 as positive set (expected to allocate high densities) and construct negative set (expected to allocate low densities). We let the model output densities for images in two sets, rank them according to the densities and plot the ROC curve for false-positive rate v.s. true-positive rate in Fig. g In the first case, we flip each image in the positive set as negative set. Note that such flipped images are not out-of-distribution samples, so the model is expected to allocate high densities to them, i.e., the ROC curve should be close to a straight line from (0, 0) to (1, 1). The results show that Joint- W. EGAN and DEM give the exact results while DGM assigns all flipped images with lower densities, which means that it fails to capture the semantics

---

Under review as a conference paper at ICLR 2020
Joint-W
DEM
DGM
EGAN
MNIST
CIFAR
1.0
140
Method
4 Dis
12 Dis
11 Dis
Dis
WGAN-GP
13,80
0.93
80.98
1.72
625
0.2
WGAN+LR
12.91
0.86
82.96
1.81
WGAN+ER
12.26
0.77
72.28
1.59
Random
WGAN+VA
12.38
0.78
69.01
1,53
DGM
12.12
0.79
179.30
3.95
Joint-W
11,82
0.73
64.23
1.41
66
22 64 03 16 3.
30
0.2
0.4
85
00
Table 3: Distances between means of generated digits (resp. images) and ground-truth digits (resp. images) on MNIST (resp. CIFAR-10).
(c) CIFAR10 Overlay
(d) Lsun
Figure 8: ROC curves for evaluation of out- lier detection on CIFAR-10.
in images. the following three cases, we i) generate random noise, average two images with different CIFAR classes, and iii) adopt Lsun Bedroom dataset as the negative set, respectively. In these situations, the model is expected to distinguish the images two sets. The results in Fig. show that DGM provides the best results while the performance of Joint-W W is quite close to DGM and much better than DEM and EGAN.
5.5 ADDRESSING DATA INSUFFICIENCY AND NOISY DATA
We proceed to test the model performance in some extreme situations where the observed samples are mixed with noises or the observed samples are quite insufficient. The results are presented in Fig. ((a where we add different ratios of random noise to the true samples in Two-Circle dataset and Fig. 7b) where we only sample insufficient data for training in Two-Spiral dataset. The details are in Appendix E. The noise in data impacts the performance of WGAN and Joint-W, but com- paratively, the performance decline for Joint-W W is less insignificant than WGAN, which indicates better robustness of joint training w.r. noised data. In Fig. (bb). when the sample size decreases from 2000 to 100, the AUC value of DEM declines dramatically, showing its dependency on sufficient training samples. By contrast, the AUC of Joint-W exhibits small decline when the sample size is more than 500 and suffers from an obvious decline when it is less than 300. Such phenomenon demonstrates lower sensitivity of joint training observed sample size.
5.6
WHEN TO START JOINT LEARNING
In our experiment, we also observe an interesting phenomenon: the performance achieved at con- startjjoint joint training after some iterations with independent training for the generator and the estimator. In other words, at the beginning, we could set X2 0 (or some very small values)
vergence would be better if we start
and after some iterations on MNIST with different numbers of iterations for independent training in Fig. (7cc) where we can see that the score firstly goes up and then goes down when we increase iterations for independent training. Such phenomenon is quite similar to the 'warm up" trick used for training deep networks where one can use small learning rates at iterations in the begining and amplify its value for further training. One intuitive reason behind this phenomenon is that at the beginning, both the genera- tor and estimator are weak and if we minimize the discrepancy between them at this point, they would possibly constrain each other and get limited in some bad local optima. When they become strong enough after some training iterations, uniting them through joint training would help them compensate and reinforce each other our discussions.
it as a normal level. We report the inception scores
CONCLUSIONS
In this paper, we aim at uniting the training for implicit generative model (represented by GAN) and explicit generative model (represented by a deep energy-based model). Besides two loss terms for GAN and energy-based model, we introduce the third loss characterized via Stein discrepancy be- tween the generator in GAN and the energy-based model. Theoretically, we show that joint training could i) help to stablize GAN training and facilitate its convergence, and ii) enforcing dual regu- larization effects on both models and help to escape from local optima in optimization. We also conduct extensive experiments with different tasks and application senarios to verify our theoretical findings as well as demonstrate the superiority of our method compared with various GAN models and deep energy-based models.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Guillaume Alain and Yoshua Bengio. What regularized auto-encoders learn from the data-generating distribution. J. Mach. Learn. Res., 15(1):3563-335) 2014.
Martín Arjovsky. Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In ICMI., pp. 214-223, 2017.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In ICLR, 2019.
Kacper Chwialkowski, Heiko Strathmann, and Arthur Gretton. A kernel test of goodness of fit. In ICML, pp. 2606-2615, 2016.
Zihang Dai, Amjad Almahairi, Philip Bachman, Eduard H. Hovy, and Aaron C. Courville. Calibrat- ing energy-based generative adversarial networks. ICLR, 2017.
Chao Du, Kun Xu, Chongxuan Li, Jun Zhu, and Bo Zhang. Learning implicit generative models by teaching explicit ones. CoRR, abs/1807. 807.00077,, 2018.
Yilun Du and Igor Mordatch. Implicit generation and generalization in energy-based models. CoRR, abs/1903.08689. 2019.
L.C. Evans and American Mathematical Society, Partial Disserential Equations. Graduate studies in mathematics. American Mathematical Society, 2010. ISBN 9781470411442. URL https:
7/ 7/books. gwogle..oooooooooooooooooooo.. com woooks:1d-0LL::AAAAAA..
Rui Gao and Anton J Kleywegt. Distributionally robust stochastic optimization with wasserstein distance, arXiv preprint 1604v:160 2016.
Rui Gao, Xi Chen, and Anton Kleywegt. Wasserstein distributional robustness and regularization in statistical learning. arXiv preprint arXiv: 2017.
Stuart Geman and Donald Geman Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell., 6(6):721-741, 1984.
Ian Gemp and Sridhar Mahadevan. Global convergence to the equilibrium of gans using variational inequalities. CoRR, 00/11008.01311 2018
Gauthier Gidel, Hugo Berard, Gaëtan Vignoud, Pascal Vincent, and Simon Lacoste-Julien. A varia- tional inequality perspective generative adversarial networks. In ICLR, 2019.
Ian J. Goodfellow. NIPS 2016 tutorial: Generative adversarial networks. CoRR, abs/ 700.00010, 2017.
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, pp. 2672-2680, 2014.
Jackson Gorham and Lester Mackey. Measuring sample quality with stein's method. In Advances in Neural Information Processing Systems, pp. 226-234, 2015.
Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, and Aaron C. Courville, Im- proved training of wasserstein gans. In NIPS, pp. 5767-5777, 2017.
Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement learning with deep ensrey-based policies. In ICML, pp. 1352-1361, 2017.
Geoffrey E. Hinton. Product of experts. In /CANN99 Artificial Neural Networks, 1999.
Geoffrey E. Hinton, Simon Osindero, and Yee Whye Teh. A fast learning algorithm for deep belief' nets. Neural Computation, 18(7):122-1154, 2006.
Tianyang Hu, Zixiang Chen. Hanxi Sun, Jincheng Bai, Mao Ye, and Guang Cheng. Stein neural sampler. CoRR, abs/1810.03545. 2018.
11

---

Under review as a conference paper at ICLR 2020
Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res., 6:695-709, 2005.
Taesup Kim and Yoshua Bengio. Deep directed generative models with energy-based probability estimation. In ICLR, 2017.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
Yann LeCun, Sumit Chopra, Raia Hadsell, MarcAurelio Ranzato, and Fu Jie Huang. A tutorial energy-based learning. Predicting Structured Data, MIT Press, 2006.
Yingzhen Li and Richard E. Turner. Gradient estimators for implicit models. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018.
Tengyuan Liang and James Stokes, Interaction matters: A note on non-asymptotic local convergence of generative adversarial networks. In AISTATS, pp. 907-919 2019.
Qiang Liu and Dilin Wang. Stein variational gradient descent: general purpose bayesian inference algorithm. In NIPS, pp. 2370-2378, 2016.
Qiang Liu and Dilin Wang. Learning deep energy models: Contrastive divergence vs. amortized MLE. CoRR, 0on//707.00799, 2017.
Qiang Liu, Jason D. Lee, and Michael Jordan. A kernelized stein discrepancy for goodness-of-fi tests. In ICML, pp. 284, 2016.
Vaishnavh Nagarajan and J. Zico Kolter. Gradient descent GAN optimization is locally stable. In NIPS, pp. 5585-5595, 2017.
Radford M. Neal. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. Handbook of Markov Chain Monte Carlo, 2, 2011.
Jiquan Ngiam, Zhenghao Chen, Pang Wei Koh, and Andrew Y. Ng. Learning deep energy models. In ICML, pp. 1105-1112, 2011.
Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski. Plug & play generative networks: Conditional iterative generation of images in latent space. In CVPR, pp. 3510-3520, 2017.
Erik Nijkamp, Mitch Hill, Song-Chun Zhu, and Yingnian Wu. On learning non-convergent non- persistent short-run meme toward energy-based model. CoRR, 0e4/1944477700 2019.
Chris J Oates, Mark Girolami. and Nicolas Chopin. Control functionals for monte carlo integration. Journal of the Royal Statistical Society, Series B, 2017.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR, 2016.
Garry Robins, Pip Pattison, Yuval Kalish, and Dean Lusher. An introduction to exponential random graph (p" models for social networks. Social Networks, 27<2): 2007.
Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann. Stabilizing training of generative adversarial networks through regularization. Advances in neural information pro- cessing systems, pp. 2018-2020 2017.
Ruslan Salakhutdinov and Geoffrey E. Hinton. Deep boltzmann machines. In AISTATS, pp. 448-
455, 2009.
Alexander Shapiro, Darinka Dentcheva, and Andrzej Ruszczyúski. Lectures on stochastic program- ming: modeling and theory. SIAM, 2009.
Chenyang Tao, Shuyang Dai. Liqun Chen, Ke Bai. Junya Chen, Chang Liu, Ruiyi Zhang, Georgiy V. Bobashev, and Lawrence Carin. Variational annealing of gans: A langevin perspective. In ICML, pp. 6176-6185, 2019.
12

---

Under review as a conference paper at ICLR 2020
Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
David Warde-Farley and Yoshua Bengio. Improving generative adversarial networks with denoising feature matching. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. 2017.
Nik Weaver. Lipschitz algebras. World Scientific, 1999.
Ying Nian Wu, Song Chun Zhu, and Xiuwen Liu. Equivalence of julesz ensembles and FRAME models. International Journal of Computer Vision, 38(3):247-265, 2000.
Jianwen Xie, Yang Lu, Song-Chun Zhu, and Ying Nian Wu. Cooperative training of descriptor and generator networks. CoRR, a 609.09408, 2016a.
Jianwen Xie, Yang Lu, Song- Chun Zhu, and Ying Nian Wu. theory of generative convnet. In ICML, pp. 2635-2644, 2016b.
Jianwen Xie, Yang Lu, Ruiqi Gao, and Ying Nian Wu. Cooperative learning of energy-based model and latent variable model via MCMC teaching. In AAAL, pp. 4292-4301, 2018.
Shuangfei Zhai, Yu Cheng, Weining Lu, and Zhongfei Zhang. Deep structured energy based models for anomaly detection. In ICML. pp. 1100-1109, 2016.
Junbo Jake Zhao, Michaël Mathieu, and Yann LeCun. Energy-based generative adversarial net- works. In ICLR. 2017.
Song Chun Zhu Ying Nian Wu, and David Mumford. Minimax entropy principle and its application to texture modeling. Neural Computation, 127, 1997.
13

---

Under review as a conference paper at ICLR 2020
A LITERATURE REVIEWS
We discuss some of related literatures and shed lights on the relationship between our work with others.
A.1 EXPLICIT GENERATIVE MODELS
Explicit generative models are interested in fitting each instance with a scaler (unnormalized) den- sity expected to explicitly capture the distribution behind data. Such densities are often up to a constant and called as energy functions which are common in undirected graphical models (LeCun) itt al. (2006)) Hence, explicit generative models are also termed as energy- based models. An early version of energy energy-based models is the FRAME (Filters, Random field, And Maximum Entropy) model (Zhu et al. (197)); Wu et al. 2000)) Later on, some works leverage deep neural networks to model the energy function (Ngiam et al. (011 Xie al. 2016b)) and pave the way for researches on deep energy model (DEM) (e.g.. Liu & Wang (2017) Kim & Bengio (2017): Zhai et al.] (2016); Haarnoja al. (2017); Du & Mordatch| (2019); ; Nijkamp et al. (2019). Apart from DEM, there are also some other forms of deep explicit models based on restricted Boltzmann machines like
deep belief networks (Hinton et al. 2006)) and deep Boltzmann machines (Salakhutdinov Hinton
(2009).
The normalized constant under the energy function requires an intractable integral over all possible instances, which makes the model hard to learn via Maximum Likelihood Estimation (MLE). To solve this issue, some works propose to approximate the constant by MCMC methods (Geman & Geman (1984); Neal (2011)). However, MCMC requires inner-loop samples in each training, which induces high computational costs. Another solution to optimize an alternate surrogate loss function. For example, contrastive divergence (CD) (Liu & Wang (0177) is proposed to measure how much KL divergence can be improved by running a small numbers of Markov chain steps to- wards the intractable likelihood, while score matching (SM) (Hyvärinen 2005 ) detours the constant by minimizing the distance for gradients of log-likelihoods. Moreover, the intractable normalized constant makes it hard sample from. To obtain an accurate samples from unnormalized densities, many studies propose to approximate the generation by diffusion- based processes, like generative flow (Nguyen al. (2017)) and variational gradient descent (Liu & Wang (2016)). Also, a recent work (Hu et al. (2018) leverages Stein discrepancy to design a neural sampler from unnormalized densities. The fundamental disadvantage of explicit model that the energy- based learning is dif- ficult to accurately capture the distribution of true samples due to the low manifold of real-world instances (Liu Wang (2017).
A.2 IMPLICIT GENERATIVE MODELS
Implicit generative models focus on a generation mapping from random noises to generated sam- ples. Such mapping function is often called as generator and possesses better flexibility compared with explicit models. Two typical implicit models are Variational Auto-Encoder (VAE) (Kingmal & Welling (2144) and Generative Adversarial Networks (GAN) (Goodfellow et al. (0144)) VAE introduces a latent variable and attempts to maximize the variational lower bound for likelihood of joint distribution of latent variable and observable variable, while GAN targets adversarial game between the generator and a discriminator (or critic in WGAN) that aims at discriminating the gen- erated and true samples. In this paper, we focus on GAN and its variants (e.g., WGAN (Arjovsky eet al. (2017)), WGAN-GP (Gulrajani et al.) (2017)) DCGAN (Radford et al. (2016)), etc.) as the implicit generative model and we leave the discussions VAE as futture work.
Two important issues concerning GAN and its variants are instability of training and local optima. The typical local optima for GAN can be divided into two categories: mode-collapse (the model fails to capture all the modes in data) and mode-redundance (the model generates modes that do not exist in data). Recently there are many attempts to solve these issues from various perspectives. One perspective is from regularization. Two typical regularization methods are likelihood- based and entropy-based regularization with the prominent examples WWrde-Farley Bengio (0117) and Li & Turner (2018) that respectively leverage denoising feature matching and implicit gradient approxi- mation to enforce the regularization constraints. The likelihood and entropy regularizations could respectively help the generator to focus data distribution and encourage more diverse samples, and a recent work (Tao et al. (2019) uses Langevin dynamics to indicate that i) the entropy and likelihood regularizations equivalent and share an opposite relationship in mathematics, and ii)
14

---

Under review as a conference paper at ICLR 2020
both regularizations would make the model converge to surrogate point with bias from original data distribution. Then Tao et al. (2019) proposes a variational annealing strategy to empirically unite two regularizations and tackle the biased distributions.
To deal with the instability issue, there are also some recent literatures from optimization perspec- tives and proposes different algorithms address the non-convergence of minimax game optimiza- tion (for instance, [Gemp& & Mahadevan (2018); Liang & Stokes (2019); Gidel et al. (2019)). More- over, the disadvantage of implicit models is the lack of explicit densities over instances, which disables the black-box generator to characterize the distributions behind data.
A.3 ATTEMPTS TO COMBINE BOTH OF THE WORLDS
Recently, there are several studies that attempt combine explicit and implicit generative models from different ways. For instance, iZhao et al.] (2017) proposes energy-based GAN that leverages energy model as discriminator
distinguish the generated and true samples. The similar idea is also used by Kim & Bengio (0177) and Dai et al. (017) which let the discriminator estimate a scaler energy value for each sample. Such discriminator is optimized to give high energy to gener- ated samples and low energy to true samples while the generator aims at generating samples with low energy. The fundamental difference that Zhao et al. (2017) and Dai et al.) (2017) both aim at minimizing the discrepancy between distributions of generated and true samples while the moti- vation of Kim & Bengio (2017) to minimize the KI. divergence between estimated densities and true samples. Kim & Bengio (2017) adopts contrastive divergence (CD) to link MLE for energy model over true data with the adversarial training of energy-based GAN. However, both CD-based method and mergy-based GAN have limited power for both generator and discriminator. Firstly, if the generated samples resemble true samples, then the gradients for discriminator given by true and generated samples are just the opposite and will counteract each other, and the training will stop before the discriminitor captures accurate data distribution. Second, since the objective boils down to minimizing the KL divergence (for Kim & Bengio (017)) Wasserstein distance (for Dai al.) (217)) between model and true distributions, the issues concerning GAN (or WGAN) like training instability and mode-collapse would also bother these methods.
Another way for combination is by cooperative training. Xie et al. (2016a) (and its improved version Xie et al. (0018) leverages the samples of generator as the MCMC initialization for energy-based model. The synthesized samples produced from finite-step MCMC are closer to the energy model and the generator is optimized to make the finite-step MCMC revise its initial samples. Also, a recent work et al. (2018) proposes to regard the explicit model as teacher net who guides the training of implicit generator as a student net to produce samples that could overcome the mode-collapse issue. The main drawback of cooperative training is that they indirectly optimize the discrepancy between the generator and data distribution via the energy model as a 'mediator', which leads to a fact that once the energy model gets stuck in a local optimum (e.g., mode-collapse training for the generator would be affected. In other words, the training for two models would constrain rather than exactly compensate each other. In Table [ do high-level comparison among the above-mentioned generative models w.r.t the objectives. Different from other methods, our model considers three discrepancies simultaneously as a triangle to jointly train the generator and the estimator, enabling them to compensate and reinforce each other.
mode-redundance) the
B BACKGROUND FOR STEIN DISCREPANCY
Assume qxx) to be continuously differentiable density supported on X' c Rd and f R Rd smooth vector function. Define A, [[()] = Vx log q(x) (x) + Vxf(x) as a Stein operator. If f is
a Stein class (satisfying some mild boundary conditions) then we have the following Stein identity property:
Exzq/ [x))]] = Ex-uq . V log (x) + Vxf(x)] = 0.
Such property induces the Stein discrepancy between distributions P: p(x) and Q g(x), x E X:
S(Q.P) (app FEF
suplo feF (V logp(x)f(x)) T + vxf(x))))
(10)
where f is what we call Stein critic that exploits over function space F and if F is large enough then S(Q, IP) = 0 if and only if Q = P. Note that in (I). we do not need the normalized constant for p(x) which enables Stein discrepancy deal with unnormalized density.
15

---

Under review as a conference paper at ICLR 2020
J is unit ball in Reproducing Kernel Hilbert Space (RKHS) with positive definite kemnel func- tion k(..-), then the supremum in (1 would have a close form (see Liu et al. (2016); Chwialkowski al. (2016); Oates et al. (2017) for more details):
P) Ex,x' (oppgglppxxp)m x'),
(11)
where "x(X,x))
Vx (x) 1(x, V p(x'
Vx logp(x) V Vx x')
Vxh(x,x)) x') Vx log p(x') x x))) The 11 gives the Kernel Stein Discrepancy (KSD).
C PROOFS OF RESULTS IN SECTION 3.2
C.I PROOF OF THEOREM !
Proof. Using Kantorovich's duality, we rewrite the problem as
min max
B.P (Br())
Fr
[D] (ePra, PE) + (PP(P,
where the minimization with respect to E is over all energy functions, the minimization with respect P' is over all probability distributions with continuous density, and the maximization over all 1-Lipschitz continuous functions. From the definition of kernel Stein discrepancy
S(P, IP) Ex log (x) V log (egFF(x)) (x)) k(x, (V)(VVV(VVVV - log IP(x') V log (x())))),
S(P, IP)) is infinite if IP is not absolutely continuous with respect to Pg. Hence, to minimize the objective, it suffices to consider those P's that are absolutely continuous with respect to Pg- We
set h(x) == dP/ (xx) 1, where dIP/dl HP/PPP is the Radon-Nikodyno derivative. It follows that the
problem becomes
min max E, D
EFe + Erg [hD] EE [D] + S(P real, Pæ)
+ A2 Ex V log(1 + h(x)) Txx X') log(1 + h(xx)))
where the minimization with respect to is over all L (IP)) functions with Pe-expectatioo zero.
Fixing E. we claim that can swap ming and max Indeed, without loss of generality, we can restrict D to be such that (xx)) = 0 for some element Xo, as a constant shift does not change the value of Epg [[1 + h) Ee.... [D]. The of Lipschitz functions that vanish at X is a Banach space, and the subset of L-Lipschtiz functions is compact Weaver 1999). Moreover, L (PP is also a Banach space. The above verifies the condition of Sion's minimax theorem, and thus the claim is proved.
Swapping ming and maxp, we consider
min max
min (Er, [hD] + A2 Exx'N V log(1 + h(x)) k(x, x') Vu log(1 + h(x)))l)
D
[b]=0 [b]=0
Vah(()
vah(x')
min max
min
Ep [hD] + Exxx'>
k((x x') T l((xx)
AeF [b]
1 + li(x)
min max
min M =0
[Epg [hD] + X2 Ex
l(x, V2h((x)
where the second equality follows from the chain rule of the derivative. For the inner minimum, have that
min (Erg [hD] d2 Ex,x' [vh(x) k(x, x' X(xxx'VV,xxx))
hr [M]=
min
min [Ee [hD] + Xar Ex Vzl(() ki X( S 5t²}
r20 hile -
min
min { [hD] D- + 2ar2 ExxxPE (xx,x)))xxxxxxxVxxx $1}
r20 hil =0]=0
min { (xar² - 1 (P(())))
r20
1
- 41g
16

---

Under review as a conference paper at ICLR 2020
where the first equality follows the introduction of the auxiliary variable r, the second equality follows a change of variable from h to rh; and the second equality the third equality follows from the definition of the kernel Sobolev dual norm. Plugging in the previous equation yields the ideal result.
C.2 PROOF FOR THEOREM 2
Proof. Essentially, the result a consequence of distributionally robust optimization with Wasser-
stein metric (Gao Kleywegt 2016 Gao et al. 2017). Here we provide simplified version for completeness.
Using the definition of Stein discrepancy, rewrite the problem
min max ( S1 IPE) + ( W( PP)
Using the compactness of the Stein class and similar argument to the proof of Theorem we can swap the minimization over IP and the maximization over f. Fixing consider
min De2yy-e(Ar f(y)] + W([Freal: P)).
P
Using the definition of Wasserstein metric
W(F P) min Exxy)>> yll).
where the minimization over all joint distributions (x, y) with x-marginal Prea and y-marginal P. we write the problem above as
min (x.)) (A2 Az f(y) + X -ylll,
P.T
where y has marginals F real and P.. Since is unconstrained, the above problem is further equivalent to
min (x.)) Ar f(y) + ||x yll),
where the minimization is over all joint distributions of (x, y) with x-marginal being Pras). Using the law of total expectation, the problem above is equivalent to
min Ex-Prn treb
[By~z
Ar fly) ||x yll x|]
ExxPProot
min A2 f(y) + yll x
Poot yEX
min f(y) + ||-xXl1
where the minimization in the first equation is over 7xx all conditional distributions of given x, and the exchanging of min and E follows from the interchangebability principle (Shapiro et al.) 2009); the second equality holds because the infimum can be restricted the of point masses. We finally have the original problem is equivalent to
min/ ( A2, f(y) + ||x +||x->||| [|lx-xl11]] b/1b}
min max f
X S1 real: IP) + Ex~Preul
yee
Hence the proof is completed using the definition of Moreau- Moreau-Yosida regularization.
D PROOFS AND MORE DISCUSSIONS IN SECTION 4
D.1 DETAILS FOR ONE-DIMENSIONAL CASE
For the analysis of I-dim regularized WGAN in section 3.1.1, we assume a Gaussian likelihood function for sample μ(x) exp(-A(z - 6)²) which is up to a constant. Its parameter can be estimated by [[xx). Then for generated sample x' 0z, have E(oog p(0z)) 2 +
17

---

Under review as a conference paper at ICLR 2020
[x2] [x]? Like the case in WGAN, consider E[x] E[z] 1. Assume Var[z] 1 and we have E[=²] 1 + E[a]. Hence, for the analysis likelihood- (and entropy-) regularized WGAN, we can study the following system:
min max V 2. X(02 -).
v
When A = 1, the above objective degrades to 44): when A < O (likelihood (likelihood regularization) the the gradient of regularization term pushes 9 to shrink, which helps for convergence; when X > the added term forms amplifiying strength on 0 and leads to divergence.
We proceed to consider 1-dim case of Stein Bridging with energy model Po(x)
exp(- 1=2 Φx). If using KSD with kernel k(z1, I[(x; x2). then S(Preat,Pae
E Em [V x1 (xg)
Vn, log (z)))))(( (V.r. log pa(p)) V log [oz))(
[ 2 lo - Vz log [(x)) = (Ø + B(x])² Similarly, one can obtain ((PG,PP) =
(6 + #E Therefore we arrive at the objective in 6
min max min v v 0
(1 1)+
(0 +))
(12)
Interestingly. the added terms (1+ o) (0 6) in (6) and the original terms 3b b in WGAN play both necessary roles to guarantee the convergence to the unique optimum points 16",00" "6"" =
If we remove the critic and optimize and & with the remaining loss terms, we would find that the training would converge but not necessarily to 1 0*] [0, (since the optimum points are not unique in this case). On the other hand, if we remove the estimator, the
system degrades to (4) and would not converge to the unique optimum point [ut
00, 1]. If
we consider both of the world and optimize three terms together, the training would converge to a unique global optimum [vl", 0*, o*] [0, 1,1].
D.2 PROOF FOR PROPOSITION
Proof. Instead of directly studying the optimization for 66 we first prove the following problem will converge to the unique optimum,
min max min Gy + 0o 000 0² + 0
(13)
V +
Applying alternate SGD we have the following iterations:
Vi+1 = 1/4 + 7) 0r,
Φ1+1 0. 1 (0: 2 (1 2: 1/0,,
O.+1 0, (1/++1 + Φ1+1 + 0.) ( 21) + (1 7)00 1ph1-
Then we obtain the relationship between adjacent iterations:
+/++1]
1
0
T d
@++1
=
1-27 27
1
Øc
0
-1
-n(1
2n)
m
00
0,
We further calculate the cigenvalues for matrix M and have the following equations (assume the eigenvalue A):
()-133 3 1)2 + 21 - 7) 1) + 21y² = 0.
One can verify that the solutions the above equation satisfy |V] /(1 Then we have the following relationship
(11 n7))
d
= [ehe Φe M.
M M
S
X2 m
Φt
0
0,
[0:
where Am denotes the eigenvalue with the maximum absolute value of matrix M. Hence, we have
2 + of 8..1 $ (1 + 71 (1 - ? + 0, + +
We proceed to replace o and 0 in (13) by w. o' and Ø' respectively and conduct change of
variable: let 0' = 1 - 0 and ø' = -1--0
1-- Φ. Then we get the conclusion in the proposition.
18

---

Under review as a conference paper at ICLR 2020
D.3 GENERALIZATION TO BILINEAR SYSTEMS
Our analysis in the one-dimension case inspires us that we can add affiliated variable to modify the objective and stabilize the training for general bilinear system. The bilinear system is of wide interest for researchers focusing on stability of GAN training (Goodfellow (2017); Liang & Stokes (2019);
Gidel et al. 2019); Gemp & Mahadevan ).18)). The general bilinear function can be written as
F( 0) 0 A - -
(14)
where tb, are both r-dimensional vectors and the objective is min max F(tb, 9) which can be + seen as a basic form of various GAN objectives. Unfortunately, if we directly use simultancous (resp. alternate) SGD optimize such objectives, one can obtain divergence (resp. fluctuation). To solve the issue, some recent papers propose several optimization algorithms, like extrapolation from the past (Gidel et al. (2019)), crossing the curl (Gemp & Mahadevan (2018)) and consensus optimization (Liang & Stokes term which generates non-zero values for Vew and V s0 that leads to such instability of training. Different from previous works that focused on algorithmic perspective, we propose to add new affiliated variables which modify the objective function and allow the SGD algorithm achieve convergence without changing the optimum points.
(to1e) (2019)). Also, Also.Liang & Stokes (019) shows that it is the interaction
Based on the minimax objective of (14 we add affiliated r-dimensional variable & (corresponding to the estimator in our model) the original system and tackle the following problem:
min max min F F(ab, 0) + (HH(
(15)
where H 0) - ( + Φ) Theoretically, the new problem keeps the optimum points of (14 unchanged. Let L( cb F(W. 0) + aG(ø, 0)
②) B(O + Φ). B = (AAT) and a is non-negative constant.
Proposition 2. Assume the optimum point of min max F(bb, 0) are (b), then the optimum
0
points of would be [zb", 0 Φ¹) where o² 0
Proof. The condition tells us that VoF(D, 0) = and VuF(b, 0*) = 0. Then derive the gradients for L( Φ, 0). VuL(E, 0) = FoF(v"00)=
(16)
1
VoL 0- = (vb + VoH (2,0")
B B ( + +
(17)
V V Ttb, 0, = V (7 (B ( B ( +)+ (B+B)+++))
(18)
Combining
17 and 18 we get o² = 0" Hence, the optimum point of
(15) is [ub" 0", o8) where
Φ²
The advantage of the problem is that it can be solved by SGD algorithm and guarantees conver- gence theoretically. We formulate the results in the following theorem.
Theorem 4, For problem min max min L(b, Φ 0) using alternate SGD algorithm, i.e.,
9
dbt = th + V/ L(O,, 1h Φ1), Φ Φ: VOL(,, thtt1, Φ2),
(19)
0.+1 = 0, - VoL 14 Q1+1).
we can achieve convergence to (ab", , 0", .5) where 0² O+ with least linear rate of (1 m + (1 + +12 - 'it) where 1)1 = mim = 110 and omin (resp. Tmaz) denotes the maximum
(resp. minimum) singular value of matrix A.
To prove Theorem 3, can prove more general argument.
19

---

Under review as a conference paper at ICLR 2020
Lemma 1. If we consider any first-onder optimization method on 11 i.e.
1i+1 E vo + span(L((ba Φ. 0), F Φ, 0)), Vt E N,
Φ++1 E 20 + span ((t), 0o 0), L(b, Φr, 0)), VI E,,
1++1 E 40 span span Φ, 0o), L(b, Φ. 0:)), Vt E N,
Then we have
hb V 24)), कe U (Ø: -'), - (0. -²),
where U and V are the singular vectors decomposed by matrix A using SVD decomposition, i.e., A UDV and the triple ([pbe] ( ())))) sagr follows the update rule with step size on as the same optimization method on unidimensional problem
min max min 0y + 0 02 0 2
(20)
V $
with step size where a, denotes the i-th singular value on the diagonal of D.
Proof. The proof is extended from the proof of Lemma in Gidel et al. 2019). The general class of first-order optimization methods derive the following updations:
1+1
+1+ 40
Pat A 0, c) vo Σ
PaA S0,,T00000 0"),
1 1+1
Φ141 = Φu +
Σ
( (B + B ( + b.),
2 ==0
1+1
O
0o
+
Pstt ( - か³) (B + B (0, + +.)).
==0
where Pst, Est. Mst E R depend on specific optimization method (for example, in SGD, Pu 8
(Ase remain as a non-zero constant for Vt and other coefficients are zero).
Using SVD A UDV and the fact 0- = B (UDD UT) D, we have
(ihtu - V (+b V - 4b)
Pa PD
D T 8*)
[+1
U (0111 -U - -0))+
Σ 8=0
Sgt U D(O, -0 +U) - x²),
t+1
UT (0.+1- UT (T, -0)
DV T
#=0
and equivalently,
1+1
1h+1 vo
Σ P DTO 0,,
3. = Φ0 S..
t+1
0 0 0. +
Pst D
0, Tz).
8=0
Note that D is rectangular matrix with non-zero elements diagonal block of size Hence, the above y-dimensional problem can be reduced to r unidimensional problems:
1+1
[5441k (ole +
Σ
Paro Oels:
[ = [Zo] +
Σ
Sua ((Oz): + (T))))
==0
==0
0 = (0),
Σ Psio (1 + (0.). . + (..1.).
The above iterations can be conducted independently in each dimension where the optimization in i-th dimension follows the same updating rule with step size σæz) problem in 20
20

---

Under review as a conference paper at ICLR 2020
Furthermore, since problem (20 can achieve convergence with linear rate of (1 +1++ n²
using alternate SGD (the proof is similar to that of ((33) D. the multi-dimmenionnl problem in (55
can achieve convergence by SGD with at least rate of (1-m + (1+772 -nn) where 771 n1 maas
722 = 1j0 and maz (resp. min denotes the maximum (resp. minimum) singular value of matrix A. We conclude the proof for Theorem 4.
Theorem 4 suggests that the added term (Ø, O) with affiliated variables & could help the SGD algorithm achieve convergence the the same optimum points directly optimizing F( 0). Our method is related to consensus optimization algorithm (Liang & Stokes (2019) which adds a regularization term ( 0) + (lV O)] to (14) resulting extra quadratic terms for and 1 The disadvantage of such method is the requirement of Hessian matrix of F( 0) which computational expensive for high-dimensional data. By contrast, our solution only requires the first-order derivatives.
D.4 STRONGLY CONVEXITY
In section 3.1.2, we assume H H(0, Φ) as a j-strongly convex function which indicates that satisfies the conditions:
(Vel((0,)) VoHl(0',))) (0 (0-00 -00 0¹) 2 u/10 0'12, v0, 0' E Ωo.
(v VoH( (')) ( (Ø  2 p||{b 2 Vob ø' St.
Bedises, F(O, 1 is p-strongly convex for 0 and p-strongly concave for e it satisfies:
(VoF(e.-) VoF(0' T) (0 0') e|10 0 10, 40, 0' Qo.
(V+F(,)))) - v.F(: bb) (1b w¹) 2 مااه - A 11.. Veb, es E Es
In section 3.1.2, we also define h(w))) = VoH + V H and f(w,) = VoF F so the above condition be written in compact form,
(I(w)) - h(wh)) (wo - wh) 2 1 wn - w/113 Ywhi w/ Sn.
(f(wy) - f(w(;)) (wS - w'y) 2 pm|lr - w/yl2 kwoy wsy E ely.
D.5 PROOF FOR THEOREM 2
The proof relies two lemmas,
Lemma 2. For any w E Ω and w = Palw + u). then we have
l| - w112 S u (w+ w). Proof. Since w is a projection of w on a convex set Ω have
(w+ + (w + u)) (wt w) O. Rearranging the above inequality one can easily get the lemma.
(21)
Lemma 3. If function (ww) is p--stronly convex, we have
k||l - w012 < VDlww) (w-
Similarly, if (w)) p-strongly concave, we have pll w S V - (w ww).
Proof. By optimality of w*, we have
Vo(W") (w - w²) 20.
Since p-convex, we can further derive
س||l w" w0 s VD(W")T(W> w²) + pl|l w (w-
21

---

Under review as a conference paper at ICLR 2020
Proof. (Proof for Theorem 3 We apply Lemma
2
to
with
(w, u, w.)
(w) -nf (1/3) ( w and we have
|w/9t1 w/tt//11 - (1 (wo)t) - w
(22)
Then we have
|1o0t1 w6 112 = | wj + wott w 11
S / 4+122 will2 + 21 2|w w
w
(by l| + b11 s 2||a|12 + +2]6613) (23)
s 2 , 1+//2 will2 2n f (w) /t113) (w))
w w
According to Lemma
we have
2nf(wy 1/2) (w/tt//- ww}) 2 2 2//////////2 -w
(24)
Plug
24
into
23 and we get
| ++ w 11 - (2- 2 , ww712
(25)
The above inequality is equivalent to
| 1 + ||
1/ < (2 21/14) (110 -0112 + 1|/b" 11))
(26)
Similarly, one can obtain
|| w/ 112 S (2 2mm1)|lw, /t//1// will.
(27)
i.e.,
| 1² + |1 - s (2 2n/ 00 + || 10 (2)).
(28)
Combining (66 and (88 we have
|| w w01 = |100+1
0 0 + 1 - -/0 + 110 - 00112
I (1 - 2n/ - 0" 112 + (2 - 2n/ii [/pp)///-011 u5--1 2 + (2 - 21 001112 (29)
< (2 - |l w0
Hence, if 1 < n < "
1 we have 0- < 2 2np < 1 and l|w w 113 I (2 21 w011
2p
E DETAILS FOR EXPERIMENT SETUP
E.I SYNTHETIC DATASETS
We provide the details for two synthetic datasets. The Two-Cirele dataset consists of 24 Gaus- sian mixtures where 8 of them are located in an inner circle with radius rı 4 and 16 of them lie in an outer circle with radius r2 = 8. For each Gaussian component, the covari-
ance matrix 0.2 0 0.2
I,I and the mean value is [rı cost, 71 sin sin where t 2m-k
25.k A = 1,.... 16 for the = I,...,,,,, ,,, for the inner circle, and [r2 cost, f, r2 sin f). where = 16
outer circle. We sample N₁ 2000 points true observed samples for model training. In sec- tion 5.5) we consider noised data scenario. In this case, we randomly add n noise points sampled from Gaussian distribution AO, I) where on 2 to the original true samples. Here we set n [40, 100, 160, 300, 400, 600, 800, 1000].
The Two-Spiral dataset contains 100 Gaussian mixtures whose centers locate on two spiral-shaped
curves. For each Gaussian component, the covariance matrix is (0.5 0
0.5| = I.II and the mean
value is |-c cos C1, C1 sin 41), where C₁ = 2T linspace (0, 0.5, 50) 2Tr, for one spiral, and [oz cos C22 C2 sin o2], where C2- 2 + linspace (0, 0.5, 50) 2n for another spiral. We sample N₂ = 5000 points as true observed samples. In section 5.5 we consider insufficient data scenario. In this case, the sample size N₂ is reduced to [100, 200, 300, 500, 700, 1000, 2000).
22

---

Under review as a conference paper at ICLR 2020
D1
D2
D3
Objective
ming ming maxę max, Ex (x)] Exapo d (Go(z)) +X1lExxPaaae (Ap. [fr (x)]l + X212 (pp [ (G
W
S
S
ming ming maxp Pelce [d (x)] - Ex-ro dd (z))
W
S
S
+X1 Exxx Lup (Ir, x )] + A2 B Bx,x''pu [up (G ()), Ga
ming ming max, max, Ex-p, Dlog( dg. (x)) + E Bg log [1 dey (Ga(z)))
JS
8
s
+11 App [r (x)] + X21Ea-1o Ap fr (Gg
ming ming max Ex~P, [log dv (x)]] + Exupo log(1 11 (Gp(z)))
JS
Sk
S
+\1 Ex [upg (Ir, x ) + A2 "p. (Go(z), Gz(z)))
Table 4: Objectives for different specifications of D (P PG). D2 (IP real. Fạ) and Ds(Po.PE) We specify Dj as Wasserstein distance or JS divergence in our paper and for D2 and D3 we consider the general Stein discrepancy or kernel Stein discrepancy. Here we use W, JS to denote Wasser- stein distance and JS divergence respectively, and S, Sz to represent general Stein discrepancy and kernel Stein discrepancy respectively. We omit the gradient penalty term for Wasserstein distance here but use it in experiments.
E.2 MODEL SPECIFICATIONS AND TRAINING ALGORITHM
In different tasks, we consider different model specifications in order to meet the demand of capacify as well as test the effectiveness under various settings. Our proposed framework (3) adopts Wasser- stein distance for the first term and two Stein discrepancies for the second and the third terms. We can write (3) as a more general form
min Di (Preat, Pa) + A1 reat,, PF) + DPP, 0,6
(30)
where D1, D2, D3 denote three general discrepancy measures for distributions. As stated in our remark, Dj can be specified as arbitrary discrepancy measures for implicit generative models. Here also use JS divergence, the objective for valina GAN. To well distinguish them, we call the model using Wasserstein distance (resp. JS divergence) as Joint-W (resp. Joint-JS) in our experiments. On the other hand, the two Stein discrepancies in (3) can be specified by KSD (as defined by S₁ in (IT) or general Stein discrepancy with an extra critic (as defined by S in (T). Hence, the two specifications for Dj and the two for D2 (Dg) compose four different combinations in total, and we organize the objectives in each case in Table
In our experiments, we use KSD with RBF kernels for D2 and D3 in Joint-W W and Joint-JS on two synthetic datasets. For MNIST with conditional training (given the digit class as model input), we also use KSD with RBF kernels. For MNIST and CIFAR with unconditional training (the class is not given as known information), we find that KSD cannot provide desirable results so adopt general Stein discrepancy for higher model capacity.
The objectives in Table 4 appear to be comutationally expensive. In the worst case (using general Stein discrepancy), there are two minimax operations where one is from GAN or WGAN and one is from Stein discrepancy estimation. To guarantee training efficiency, we alternatively update the generator, estimator, Wasserstein critic and Stein critic over the parameters 0, Φ, to and æ respec- tively. Specifically, in one iteration, we optimize the generator over 0 and the estimator over & with one step respectively, and then optimize the Wasserstein critic over & with na steps and the Stein critic over π with ne steps. Such training approach guarantees the same time complexity order of proposed method as that of GAN or WGAN, and the training time for our model can be bounded within constant times the time for training GAN model. In our experiment, we set nl = nr. = 5 and empirically find that our model Stein Bridging would be two times slower than WGAN on average. We present the training algorithm for Stein Bridging in Algorithm
E.3 IMPLEMENTATION DETAILS
We give the information of network architectures and as each competitor in our experiments.
settings for our model as well
23

---

Under review as a conference paper at ICLR 2020
Algorithm 1: Training Algorithm for Stein Bridging
REQUIRE:
observed training samples (x) ~ Preat-
REQUIRE:
Oo. Φo. tho, To, initial parameters for generator, estimator, Wasserstein critic and Stein
critic models respectively. af 0.0002, 8 0.9, 8 0.999, Adam hyper-parameters for explicit models. a' = 0.0002, 31 = 0.5, 3/2 = 0.999, Adam hyper-parameters for implicit models. \1 1. A2, weights for D22 and D3 (we suggest increasing A from 0 to I through training). nd 5,
5 number of iterations for Wasserstein critic and Stein critic, respectively, before one iteration for generator and estimator. B 100, batch size.
while not converged do for n = -, 1, ; do Sample B true samples {x:} from {x]:
Sample B random noise {z Po and obtain generated samples x; Gp(z.)
Liis 1 EE= (x:) (x,) X(I dx - 1) // the last term is for gradient B
penalty in WGAN-GP where & exX; + (1 e, ) es ~ (0, 1);
k & Adam(-Laie 1k. a² ,31, |1.,33 // update the Wasserstein critic;
for n = 1, , do
Sample B true samples {x from {x):
10
Sample B random noise { ~ Po and obtain generated samples X; = Gz(z))
11
Loritse E.= A1 (x)] + A2-Aps (x;)];
12
Tk+1 f- (olam(-Lorittee Th, E 3.8 ( )// update the Stein critic:
13
Sample B randomm noise {z: ) ~ Po and obtain generated samples Go (z.):
14
Lest 1 E/= Apo [Ex (x)] + A2. [xx
15
B
Φu+1 Adam Cests Øx, a . ,3F,,3 ,3 /// update the density estimator;
Egen 1 Ei (xa) + A2 [fx (R))]:
17
B
Oc+1 < Adamm(ggen Oz, at, $1, 3/ )M/ update the sample generator;
18
19
OUTPUT: trained sample generator (z) and density estimator (x)...
The energy function is often parametrized as a sum of multiple experts (Hinton (1999)) and each expert can have various function forms depending on the distributions. If using sigmoid distribution, the energy function becomes (see section 2.1 in Kim & Bengio (2017) for details)
Ex(x)= = Σ log(1 + +WWnnt))++)))
(31)
where n(x) maps input x to a feature vector and could be specified as a deep neural network, which corresponds to deep energy model (Ngiam et al. (2011))
For synthetic datasets, we set the noise dimension as 4. All the generators are specified as a three- layer fully-connected (FC) neural network with neuron size 4- 128 122 2, and all the Wasserstein critics (or the discriminators in IS-divergence- based GAN) are also a three-layer FC network with neuron size 2 128 128 1. For the estimators, we the expert number as 4 and the feature function nxx is a FC network with neuron size 2 - 128 - 128 - 4. Then in the last layer we sum the outputs from each expert as the energy value E(x). The activation units are searched within [eeaky.ReLU, tanh, sigmoid, softplus). The learning rate [le - 6, le 5, le 4, le 3, 1e and the batch size [50, 100, 150, 200). The gradient penalty weight for WGAN is searched in [0, 0.1, 10, 100).
For MNIST dataset, we set the noise dimension as 100. All the crities/diseriminatoss are im- plemented as a four-layer network where the first two layers adopt convolution operations with filter size 5 and stride [2. 2] and the last two layers are FC layers. The size for each layer is 1- 64 128 256 1. All the generators are implemented as four-layer networks where the first two layers are FC and the last two adopt deconvolution operations with filter size 5 and stride [2, The size for each layer is 100- 256 128 64 1. For the estimators, we sider the expert number as 128 and the feature function is the same as the Wasserstein critic ex- cept that the size of last layer is 128. Then we sum the outputs from each expert as the energy
24

---

Under review as a conference paper at ICLR 2020
Two-Cirlce
Two-Spiral
Method
MMD
HSR
KLD
JSD
AUC
MMD
HSR
KLD
JSD
AUC
GAN
0.0033
0.772
0.0082
0.583
GAN+LR
0.0106
0.391
0.0068
0.821
GAN+ER
0.0103
0.428
0.0071
0.780
VA
0.0118
0.295
0.0085
0.761
WGAN-CI
0.0010
0,841
0.0090
0.697
WGAN+LR
0.0013
0,840
0.0095
0.607
WGAN+ER
0.0008
0.830
0.0182
0.730
WGAN+VA
0.0016
0.835
0.0159
0.618
DEM
2.036
0.431
0.683
1,206
0.315
0.640
EGAN
3.350
0.474
0.616
1.916
0.445
0.499
DGM
0.0040
0.774
2.272
0.445
0.600
0.0019
0.833
1.725
0.414
0.589
Joint-JS
0.0037
0.883
1.104
0.297
0.962
0.0031
0.717
0.655
0.193
0.808
Joint-W
0.0007
0.844
1.030
0.281
0.961
0.0003
0.909
0.364
0.110
0.810
Table 5: Quantitative results including MMD (lower better), HSR (higher is better) as the metrics for quality of generated samples and KLD (lower is better), JSD (lower is better), AUC (higher is better) as the metrics for accuracy of estimated densities on Two-Cirele and Two-Spiral datasets.
value. The activation units are searched within [ReLU,
tanh). The learning rate [2e -e-33 5, 2e - 4, 2e 3, 2e - 2] and the batch size [32, 64, 100, 128]. The gradient penalty weight for WGAN is searched in [1, 10, 100, 1000].
For CIFAR dataset, we adopt the same architecture as DCGAN for critics and generators. As for the estimator, the architecture of feature function is the same the crities except the last year where set the expert number as 128 and sum each output as the output energy value. The architectures for Stein critic are the same Wasserstein critic for both MNIST and CIFAR datasets. In other words, we consider d' = 1 in (1) and further simply & as an average of each dimension of Ex Empirically we found this setting can provide efficient computation and decent performance.
E.4 EVALUATION METRICS
We adopt some quantitative metrics to evaluate the performance of each method on different tasks. In section 4.1, we use two metrics to test the sample quality: Maximum Mean Discrepancy (MMD) and High-quality Sample Rate (HSR). MMD measures the discrepancy between two distributions X and Y, .M MD(XX Y) = | Σ (<x;) X and Y respectively and & maps each sample to a RKHS. Here we use RBF kernel and calculate MMD between generated samples and true samples. HSR statistics the rate of high-quality samples over generated samples. For Two-Cirice dataset, we define the generated points whose distance from the nearest Gaussian component is less than points total and statistic HSR. For Swo-Spiral dataset, we set the distance threshold as 50g and generate 5000 points to calculate HSR.
Σ)= (fya))l where z; and yj denote samples from
high-quality samples. We generate 2000
As for Inception Score and CEPC. For MNIST, we pre-train a classifier for 10 digits which can pro- vide the test accuracy up to 99% for calculation of scores. The conditional entropy of predicted classes (CEPC) for given samples is defined as H)y) 2 Σ E (())/2))) log p(ys. (a;)) where x is generated instance and y denotes the predicted class given x from pre-trained classi- fier. CEPC measures how well a given sample can be classfied into right class, i.e. the quality of such sample. For CIFAR, we use the Inception V3 Network in Tensorflow as pre-trained classifier.
In section 4.2, we use three metrics characterize the performance for density estimation: KL divergence, JS divergence and AUC. We divide the map into 300 meshgrid. calculate the unnor- malized density values of each point given by the estimators and compute the KL and JS divergences between estimated density and ground -ruth density. Besides, we select the centers of each Gaussian components as positive examples (expected to have high densities) and randomly sample 10 points within circle around each center as negative examples (expected to have relatively low densities) and rank them according to the densities given by the model. Then we obtain the area under the curve (AUC) for false-positive rate v.s. true-positive rate.
25

---

Under review as a conference paper at ICLR 2020
Class
'0"
'1'
2
'3'
'4'
'5'
'6'
'7'
'8'
9
4
20.3
11.4
14.3
14.8
13.5
13.3
13.8
11.0
13.0
12.3
WGAN-GP
lz
1.74
1.07
0.82
0.98
0.83
0.68
0.95
0.62
0.82
0.75
4
13.8
5.9
13.6
19.1
11.8
18.3
10.7
11.5
14.0
9.9
WGAN+LR
l2
0.80
0.34
0.84
1.81
0.65
1.37
0.62
0.70
0.90
0.57
4
16.1
11.7
14.2
12.3
10.8
13.9
11.4
12.1
10.9
WGAN+ER
lz
1.20
0.74
0.54
0.86
0.73
0.54
0.97
0.69
0.72
0.63
4
16.3
7.1
13.7
13.7
11.9
13.2
13.6
11.2
12.1
10.6
WGAN+VA
l2
1.12
0.35
0.81
0.85
0.69
0.76
1.04
0.71
0.74
0.71
4
22.2
10.9
12.7
10.2
10.8
9.0
9.5
10.9
12.7
11.7
DGM
l2
1.41
0.83
0.81
0.65
0.67
0.56
0.66
0.67
0.88
0.76
4
14.1
7.5
14.3
12.9
11.1
11.0
13.7
9.7
12.0
11.5
Joint-W
12
0.89
0.47
0.93
0.73
0.55
0.51
1.06
0.53
0.70
0.97
Table 6:
11 and 12 distances between means of true digits and generated digits in each class on
MNIST.
Class
'0'
'1'
'2'
'3'
"4'
'5'
'6'
'7'
'8'
*9"
80.8
82.7
40.2
69.3
44.7
59.2
77.6
107.7
50.81
89.3
WGAN-GP
12
1.75
1.84
0.92
1.57
1.04
1,40
1.78
2.32
1.78
1.92
78.4
79.2
73.8
86.0
75.8
77.2
106.7
103.0
56.5
92.3
WGAN+LR
12
1.63
1.76
1.59
1.88
1.68
1.74
2.36
2.23
1.24
2.00
75.5
64.0
100.0
65.0
58.5
69.1
74.5
81.8
62.5
71.3
WGAN+ER
12
1.56
1.45
2.04
1.43
1.35
1.57
1.67
1.82
1.40
1.58
60.9
70.0
79.4
62.7
63.0
73.9
76.2
77.2
59.8
66.4
WGAN+VA
la
1.32
1.55
1.68
1.39
1.42
1.63
1.70
1.77
1.33
1.48
167.8
185.0
149.4
250.1
105.3
134.0
223.8
197.3
148.3
231.7
DGM
12
3.67
4.14
3.15
5.41
2.39
3.04
4.68
4.51
3.24
5.25
59.3
58.1
77.3
54.8
58.1
65.1
63.9
82.8
59.1
63.2
Joint-W
12
1.26
1.30
1.60
1.23
1.28
1.44
1.44
1.80
1.27
1.43
Table and 12 distances between means of true images and generated images in each class on CIFAR. (Class '0', '1', '2', '3', '4', '5', '6', '7', '8' and '9" stand for 'airplane', 'automobile'. 'bird', 'cat', 'deer', 'dog', 'frog', 'horse", 'ship' and "truck' respectively.)
26

---

Under review as a conference paper at ICLR 2020
8
5
5
6
8
4
3
3
5
9
5 0
7
6
8
4
2
5
3
1
3
3
O
2
3
6
9
7
1
6 6 1
1
6
2
8
6
7
8
7
8
6
6
3
8
0
9 4
7 9
6
0
5
6
5 9 3
4 8 7 1
8
3
3
5
8
3
3
$
/ 6
6
6
9
4
6
3
9
0
5
7
7
3
3
/ 0
4
6
5
b
3
6 5
2 201
4
9
5
0
8
8
4
8
9 8
5
/ 6
0 4
9
4
0
6
0 8
8
0 4 /
6
2
5
3
4
4
6
9
5
2 3
1/6831
8
9
0 9
7
(
0 6
0
8
6
8
7
2
9 6 / 5
8
$ /
0
3
6
5
0 4
8
9
0
4
7
9
8
8 1
03
1
3
3
5
4
2
8
6 / 3
5
9
0
8
4
8
9 9
4 1
8
| 2
6
7
7
6
1
4
4
3 6
4
8
6
2
8
4
3
/
0
/
8
2
7
4
8
5
7
8
O
5
6
(a) Randomly sampled over all digits
(b) Randomly sampled over digits with 50% densi-
ties
Figure 9: Generated digits given by Joint-W MNIST.
(a) Randomly sampled over all images
(b) Randomly sampled over images with top 50% densi-
ties
Figure 10: Generated images given by Joint-W W on CIFAR.
S
5
5
Q
9
9
6
9
9
3
9
5
(a) Generated digits with highest densities
(b) Generated digits with lowest densities
/
1 /
1 /
5
5 5
5 5
5
5 5 5 5
9
9
9
99999999
(c) Real digits with highest densities
(d) Real digits with lowest densities
Figure 11: The generated digits (and real digits) with the highest densities and the lowest densities given by Joint-W. W.
27

---

Under review as a conference paper at ICLR 2020
5
5
5 5
99
Generated digits with highest densities
(h) Generated digits with lowest densities
1
/ 1
1
$ 5
5
S 999999
9
9
9
9
(c) Real digits with highest densitics
(d) Real digits with lowest densities
Figure 12: The generated digits (and real digits) with the highest densities and the lowest densities given by DGM.
5
S
9
(a) Generated digits with highest densities
(b) Generated digits with lowest densities
5 5
5
9 Q 9
9 9
9
(c) Real digits with highest densitics
(d) Real digits with lowest densities
Figure 13: The generated digits (and real digits) with the highest densities and the lowest densities
given by EGAN.
28


Reviews:
[{"review_id": "rklud-o3tH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to train a GAN and an EBM jointly, and bridge them using a Stein discrepancy. The paper claims it leads to novel regularization effect on both models, and stablizes the optimization process. Experiments on MNIST and CIFAR-10 show improvement in sample quality and outlier detection.\n\nBoth the idea and the experiment results are interesting. However, the derivations contain too many typos and are in general confusing, and I cannot confirm their correctness. Therefore I cannot recommend acceptance.\n\nSpecifically the proof of Theorem 1 seems problematic:\n1. In the proof you claim (15) equals $\\frac{-1}{4\\lambda_2}\\lVert D-t\\rVert_{H^{-1}}$. But (15) could only simplify to \n$\\frac{1}{\\lambda_2} ( E[D\\cdot(\\lambda_2 h)] + E_{x,x'}[\\nabla(\\lambda_2 h(x))^T k(x,x') \\nabla(\\lambda_2h(x'))],$\nwhere h is unconstrained. Compare this with the definition of the $H^{-1}$ norm,\n$sup_h \\{E[D\\cdot h]: E_{x,x'}[\\nabla h(x)^T k(x,x') \\nabla h(x')] \\le 1\\},$\nhow did you drop the inequality constraint on h?\n2. The transformation from the original objective (14) to (15) is strange as well. In the proof you claim the minimization problem below \"invoking Lagrangian duality gives\" could only turn to (15) after \"applying the approximation log(1+a)=a+O(a^2)\" and \"a further approximation\". But you can turn it into\nE[(D-t)h]+λ E_{x,x'~P_E}[∇h(x)^T k(x,x') ∇h(x')]\nsimply by simplifying the gradient terms. Also, why did the $t$ disappear in (15)?\n\nThere are also typos and issues elsewhere. To list a few:\n3. Energy-based models are not generally referred to as \"explicit models\", since the normalization constant is intractable. I would suggest to replace the occurrences of (log) \"density\" with \"energy\" to avoid confusion.\n4. The GD update rule of (6) is incorrect; the optima should also be (0, 1), instead of (1, 0).\n5. On the second line on Page 8, the unnormalized log density cannot be x^2+\\phi x, as the normalization constant would then be infinity.\n\nFor these reasons, I believe this paper needs a thorough proofreading before it can be reviewed efficiently.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SklGwvQpKH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Thank you for your rebuttal. The paper improved after the rebuttal but I still think point 5 in the rebuttal is problematic since using d'=d, may not guarantee that we have a proper metric as claimed by the authors. I m updating  my score as a weak reject for this paper.\n\nSummary of the paper: \n\nThe paper proposes to train implicit  model such as gan and an explicit model (Energy Based ) jointly . The GAN is trained using WGAN-GP objective or the original JS objective (we have a discriminator D and Generator G). The energy based model (E) is trained using Stein Divergence with a fixed kernel k or a learned critic who's parameters are denoted pi in the paper. Note that the critic of the stein divergence is vector valued. This paper propose to add a regularization loss on the stein divergence between the generator G (implicit model ) and the explicit model (E). This gives a training objective \n\n$\\min_{G,E} W(P_r, G) + \\lambda_1 S(P_r, P_E)+ \\lambda_2 S(P_{G}, P_{E})$\n\nIn the paper the stein critic is shared between the two stein divergence which means that the authors are rather considering : $S(\\lambda_1 P_r + \\lambda_ 2P_{G}, P_{E})$\n\nPaper shows the effect of this additional coupling between the two models as a regularization on the Discriminator D and on the critic of the stein divergence. \n\nThen  the effect of the regularization is also show in terms of convergence in the optimization on a bilinear game, and in the convex concave case. \n\nExperiments are given showing the benefits of the joint training. \n\nReview: \n\nThe paper has a lot of typos and needs a lot of proofreading and is not in shape for being reviewed. \nThere are too many concerns with this papers:\n\n1- The first one was mentioned above if the critic is shared then you better be considering :  $S(\\lambda_1 P_r + \\lambda_ 2P_{G}, P_{E})$\n\n2- In equation 4,  the problem is $\\min_{G} \\max_{D}$ it is swapped.\n\n3- There a lot of gaps in the proofs of Theorems 1 and 2. The transition from equation 14 to the $\\inf_{\\mathbb{P}}...$ is not explained and seems flawed. In theorem 2 , the proof is too short and swapping of $\\min$ and $\\mathbb{E}$ is not backed rigoursly. \n\n4- Again in Equation 8, it is not clear how the Stein terms were computed , the appendix does not give the derivations either.\n\n5 - Authors say that the Stein critic have similar architecture to the GAN critic , which indicates an error in the implementation in the neural case for stein critic. Stein critic has to be vector valued, after checking the code of this paper on GitHub, stein critic maps to a real value in the code , which is flawed. The critic of stein needs to map the image to an image , which actually quite expensive. \n\nTypos: \nabstract : without explicitly defines -> defining \nmultimodal data . has been -> have \nwithout explicit defines -> defining \n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "SJe9euyhFS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a training objective that combines three terms:\n* A Stein discrepancy for learning a energy model with intractable normalizing constant\n* A Wasserstein GAN objective for learning an implicit neural sampler.\n* A Stein discrepancy for minimizing the distance between distributions defined by the energy model and the GAN.\n\nThe third term is called \"Stein bridging\" by the authors. It seems pretty difficult to motivate such a bridging term because from the first glance this term does not add anything to learning the two models from data. So I'm wondering how the authors motivate themselves to study this modification. This is my main concern about the paper if this term appears simply because that energy model has an unnormalized density while from GANs we can sample, and Stein discrepancy is best applicable to such pair of distributions.\n\nApart from the concern on motivations, I tried to follow the arguments in Section 3, as the bridging term is justified as regularization to both models. However, I think the proof of Theorem 1 is incorrect:\n\n* I don't think it makes sense from \\nabla \\log (1 + h(x)) to (1 + h(x))\\nabla h(x), even if the taylor expansion suggested by the authors is applied.\n* In the next step, \"Consider a further approximation\", this approximation basically sets 1 + h(x) to 1, if h(x) is approximately 0, then P=P_E..\n\nMinor:\n* IN proof of Theorem 1, the expectation should be always over x,x'~P_E instead of x~P, right?\n\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a generative model that jointly trains an implicit generative model and an explicit energy based model using Stein's method. There are concerns about technical correctness of the proofs and the authors are advised to look carefully into the points raised by the reviewers. 
Decision: Reject