Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper presents non-asymptotic error analysis for bidirectional GANs. The distinctive feature of the error analysis is relaxing the assumption that the data and latent variable should have the same dimension.  The constants in the error bound is also explicit in dimension.  
main_review: The paper is well written and discusses the related literature well. I could not go through the details of error analysis, but the statement of the results is rigorous. I have some question/comments about the assumptions and results.  


1- What is the motivation to study bi directional GAN as opposed to GAN? Can these results be extended to GAN, or is there a special property of GAN that allows relaxing the assumptions?

2- What are the limitations of Assumption 2? Does it exclude data distributions that lie in low-dimensional manifolds? How necessary is this assumption? 

3- For Theorem 3.2., the constant for the unbounded case does not seem to be explicit as the paper suggests. It seems to better distinguish this in the introduction

4- It is surprising that the dimension k does not appear in result in Theorem 3.3. Is there any explanations for this?  

5- I did not understand how the results in Theorem 4.3 is extended to multi-dimensional case for $\nu$.  

Overall, the paper is well written and comparison is given with the related literature. I only suggest including an outline to navigate the theoretical results 
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies the statistical properties of bidirectional GANs. It studies the sample complexity of the bidirectional GAN estimate under the Dudley metric. They relax a couple of assumptions usually made in GAN theory literature; they do not assume that the dimension of the latent distribution and the data distribution are the same or that the true data distribution has bounded support. 
main_review: This is an interesting paper that provides useful insight for understanding bidirectional GANs. It is clearly written and well organized. This work shows to theoretically handle the bidirectionality in the particular GAN framework and unboundedness of the support of the data. They also show via a decomposition of the generalization error how the different dimensions of the latent and the data distribution affect this error. Moreover, they provide explicit prefactors for the upper bound of the generalization error. 

It is important to note that explicit prefactors are only provided in the case of bounded support of the data. In this case the Dudley metric and the Wasserstein distance is the same. Since the paper by Lu and Lu (2020) cited in the related work study the Wasserstein distance, does their proof imply the same dependence of the pre factor on the dimension? Moreover, given that the generalization bound has exponential dependence on the dimension when is a better estimate of the prefactor important?

My other concerns/suggestions with this work are as follows: 
- It would be nice to have lower bounds for statistical complexity under the setting considered in this work to better understand the results. 
- Often the number of samples from the latent distribution used to train GANs far exceeds that of the true data. It would provide useful perspective to incorporate this into the results. 
- Some discussion on the computational complexity of computing the GAN estimator proposed would be very beneficial even if only under the setting where the discriminator is replaced by an oracle that provides the loss. 
limitations_and_societal_impact: The authors have discussed some limitations of this work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents non-asymptotic error bounds for bidirectional GAN estimators. Their primary contribution is a dependence on the square root of dimension under certain assumptions on the data distribution and the latent distribution.
main_review: 1. Originality

The task is not new, but the results are new. One minor comment for highlighting differences with previous contributions is to write out previously known results in terms of equations, rather than only in plain words as is the case of the current version. Related work is adequately cited.

2. Quality

The submission seems technically sound. It might be a good idea to add a bit more discussion into the dependence on n and C_d as well---the current paper focuses on the square root dependence on d, but the main theorems (3.1., 3.2., 3.3.) all depend on specifying generator and discriminator class structure in terms of n, d., and C_d "depends on d". I think it would be helpful to have some more discussion in terms of these dependencies. Following the above point, it would be nice to address whether general results would also hold without a specification of the class structures; otherwise, the claims seem to be fairly specific and only holds for these particular specifications.  

3. Clarity

The paper is fairly well-written and organized. Some comments on typos are addressed in Section 5. of the main review. Some comments regarding clarity:

a. The notation $\hat{\mathbb{E}}$ is not clearly defined in Line 188-189.

b. There is no notion of specifying network structures in the abstract, and it leaves an impression that the results are general and hold true for a wide range of bidrectional GANs. This may need to be addressed in further revisions of this paper.

c. Following the above point, it leads to the question of whether the assumptions and conditions are suitable for this particular specification, and what would happen under looser assumptions: even in Section 2.2 the authors pointed out that "Assumptions 1 and 2 can be easily satisfied by specifying $\nu$ as some common distribution with easy-to-sample density such as Gaussian or uniform...". It would be interesting if the necessity of these assumptions are discussed in a clearer sense.

d. The use of the $\implies$ signs in Lines 137-139 are slightly ambiguous.

e. Perhaps a minor point, but in the abstract and Section 1 the term "target distribution" is used but is not clearly defined. It may be helpful to clarify in further revisions the difference/similarities of the "target distribution" compared with the "data distribution" and "latent distribution".


4. Significance

The results are interesting and bring insight into understanding bidirectional GANs. From this perspective, the results are important. However, I think the significance would be clearer if some of the above points are addressed.

5. Typos

Some typos:

a. In Theorems 3.2. and 3.3., "Conditions 1" may be "Condition 1".

b. In Line 145: "defined" may be "define".

Update after reviewing comments: Increased score to 6.
limitations_and_societal_impact: The authors discuss some societal impact, but there are no societal issues due to the theoretical nature of this work.
review_title: null

Meta-review: This paper presents non-asymptotic error analysis for the bidirectional GANs.
In the analysis the authors have succeeded in relaxing those assumptions usually adopted in the GAN literature, such that the latent distribution and the data distribution has the same dimension and that the true data distribution has a bounded support. They have also succeeded in making some of the coefficients in the error bounds explicit. These constitute the main contributions of this paper.

The reviewers raised several questions, and the authors have addressed them adequately, as evidenced by two of the reviewers having raised their scores from below the threshold to above. Now all the review scores are above the threshold, I am happy to recommend acceptance of this paper for presentation at the NeurIPS conference.

Minor points:
- A learnability issue: Lemma 4.2 is based on existence of a neural network $\\psi$, and Theorem 4.3 is based on existence of neural networks $g$ and $e$, with desired properties. These lemma and theorem do not care about the learnability of these neural networks. It would therefore be nicer if some empirical results be presented to at least suggest that the learnability issue would be minor in practice, so that the stated error bounds are indeed relevant.
- Line 34: join(t)
- Line 59: the data joint distribution.( )To the best
- Lines 93, 94: $(\sim)\nu$, $(\sim)\mu$
- Lines 121, 124: vector(s)
- Line 138: metri(ci)zing
- Equation between lines 186 and 187: $h\in(\mathcal{F}\to\mathcal{H})$
- Definitions of $\mathcal{E}_3$ and $\mathcal{E}_4$: In the proof of Lemma 4.1, it seems that the authors used slightly different definitions of these quantities, where $g^*$ and $e^*$ are replaced with $\hat{g}$ and $\hat{e}$, respectively.
- Line 267: cover(ing) numbers
- SM, line 26: $|z_0|\le\log n$ $\to$ $\\\|z_0\\\|\le\log n$
Decision: Accept (Poster)