Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Thank you for your rebuttal. The paper improved after the rebuttal but I still think point 5 in the rebuttal is problematic since using d'=d, may not guarantee that we have a proper metric as claimed by the authors. I m updating  my score as a weak reject for this paper.

Summary of the paper: 

The paper proposes to train implicit  model such as gan and an explicit model (Energy Based ) jointly . The GAN is trained using WGAN-GP objective or the original JS objective (we have a discriminator D and Generator G). The energy based model (E) is trained using Stein Divergence with a fixed kernel k or a learned critic who's parameters are denoted pi in the paper. Note that the critic of the stein divergence is vector valued. This paper propose to add a regularization loss on the stein divergence between the generator G (implicit model ) and the explicit model (E). This gives a training objective 

$\min_{G,E} W(P_r, G) + \lambda_1 S(P_r, P_E)+ \lambda_2 S(P_{G}, P_{E})$

In the paper the stein critic is shared between the two stein divergence which means that the authors are rather considering : $S(\lambda_1 P_r + \lambda_ 2P_{G}, P_{E})$

Paper shows the effect of this additional coupling between the two models as a regularization on the Discriminator D and on the critic of the stein divergence. 

Then  the effect of the regularization is also show in terms of convergence in the optimization on a bilinear game, and in the convex concave case. 

Experiments are given showing the benefits of the joint training. 

Review: 

The paper has a lot of typos and needs a lot of proofreading and is not in shape for being reviewed. 
There are too many concerns with this papers:

1- The first one was mentioned above if the critic is shared then you better be considering :  $S(\lambda_1 P_r + \lambda_ 2P_{G}, P_{E})$

2- In equation 4,  the problem is $\min_{G} \max_{D}$ it is swapped.

3- There a lot of gaps in the proofs of Theorems 1 and 2. The transition from equation 14 to the $\inf_{\mathbb{P}}...$ is not explained and seems flawed. In theorem 2 , the proof is too short and swapping of $\min$ and $\mathbb{E}$ is not backed rigoursly. 

4- Again in Equation 8, it is not clear how the Stein terms were computed , the appendix does not give the derivations either.

5 - Authors say that the Stein critic have similar architecture to the GAN critic , which indicates an error in the implementation in the neural case for stein critic. Stein critic has to be vector valued, after checking the code of this paper on GitHub, stein critic maps to a real value in the code , which is flawed. The critic of stein needs to map the image to an image , which actually quite expensive. 

Typos: 
abstract : without explicitly defines -> defining 
multimodal data . has been -> have 
without explicit defines -> defining 

review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper proposes to train a GAN and an EBM jointly, and bridge them using a Stein discrepancy. The paper claims it leads to novel regularization effect on both models, and stablizes the optimization process. Experiments on MNIST and CIFAR-10 show improvement in sample quality and outlier detection.

Both the idea and the experiment results are interesting. However, the derivations contain too many typos and are in general confusing, and I cannot confirm their correctness. Therefore I cannot recommend acceptance.

Specifically the proof of Theorem 1 seems problematic:
1. In the proof you claim (15) equals $\frac{-1}{4\lambda_2}\lVert D-t\rVert_{H^{-1}}$. But (15) could only simplify to 
$\frac{1}{\lambda_2} ( E[D\cdot(\lambda_2 h)] + E_{x,x'}[\nabla(\lambda_2 h(x))^T k(x,x') \nabla(\lambda_2h(x'))],$
where h is unconstrained. Compare this with the definition of the $H^{-1}$ norm,
$sup_h \{E[D\cdot h]: E_{x,x'}[\nabla h(x)^T k(x,x') \nabla h(x')] \le 1\},$
how did you drop the inequality constraint on h?
2. The transformation from the original objective (14) to (15) is strange as well. In the proof you claim the minimization problem below "invoking Lagrangian duality gives" could only turn to (15) after "applying the approximation log(1+a)=a+O(a^2)" and "a further approximation". But you can turn it into
E[(D-t)h]+λ E_{x,x'~P_E}[∇h(x)^T k(x,x') ∇h(x')]
simply by simplifying the gradient terms. Also, why did the $t$ disappear in (15)?

There are also typos and issues elsewhere. To list a few:
3. Energy-based models are not generally referred to as "explicit models", since the normalization constant is intractable. I would suggest to replace the occurrences of (log) "density" with "energy" to avoid confusion.
4. The GD update rule of (6) is incorrect; the optima should also be (0, 1), instead of (1, 0).
5. On the second line on Page 8, the unnormalized log density cannot be x^2+\phi x, as the normalization constant would then be infinity.

For these reasons, I believe this paper needs a thorough proofreading before it can be reviewed efficiently.
review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: This paper proposes a training objective that combines three terms:
* A Stein discrepancy for learning a energy model with intractable normalizing constant
* A Wasserstein GAN objective for learning an implicit neural sampler.
* A Stein discrepancy for minimizing the distance between distributions defined by the energy model and the GAN.

The third term is called "Stein bridging" by the authors. It seems pretty difficult to motivate such a bridging term because from the first glance this term does not add anything to learning the two models from data. So I'm wondering how the authors motivate themselves to study this modification. This is my main concern about the paper if this term appears simply because that energy model has an unnormalized density while from GANs we can sample, and Stein discrepancy is best applicable to such pair of distributions.

Apart from the concern on motivations, I tried to follow the arguments in Section 3, as the bridging term is justified as regularization to both models. However, I think the proof of Theorem 1 is incorrect:

* I don't think it makes sense from \nabla \log (1 + h(x)) to (1 + h(x))\nabla h(x), even if the taylor expansion suggested by the authors is applied.
* In the next step, "Consider a further approximation", this approximation basically sets 1 + h(x) to 1, if h(x) is approximately 0, then P=P_E..

Minor:
* IN proof of Theorem 1, the expectation should be always over x,x'~P_E instead of x~P, right?



review_title: Official Blind Review #2

Meta-review: The paper proposes a generative model that jointly trains an implicit generative model and an explicit energy based model using Stein's method. There are concerns about technical correctness of the proofs and the authors are advised to look carefully into the points raised by the reviewers. 
Decision: Reject