Paper: A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data (ICLR-2024)

Reviewer comment (Theory):
Though the paper provides the analysis all the way to containing both sample complexity and misspecification, the single question of sample complexity seems to be an interesting question on its own, i.e. population v.s. empirical over the same Holder class, not neural networks. In this setting, optimization is out of the picture, thus questions such as the convergence rate of $\|id - G\circ E\|$ or $\|\hat{G}-G\|$ can be asked without ambiguity. The reviewer understands that this is additional work, but encourages the authors to provide a discussion, rather than stuffing all ingredients all at once. This could possibly be related to the tightness (minimaxity) of the proposed bound.

Author response:
Regarding the weaknesses you pointed out, we agree that the single question of sample complexity is interesting on its own. The convergence of $\hat{G}$ to $G$ is guaranteed in Theorem 15, under additional assumptions of uniform equicontinuity. For uniformly Lipshitz generators, the convergence rate is upper bounded by the rates derived in Theorem 8, as stated in Remark 16. Similarly, Proposition 13 deals with $id - G \circ E$. We agree with the reviewer that a more focused discussion on decoding (Proposition 13) and data generation guarantee (Theorem 15) will benefit the readability of the paper. In the final version of the manuscript, we will formally state Remark 16 as a corollary for further clarification and streamline Section 5.5 to predominantly cover these two aspects. In response to your thoughtful feedback, we will omit Remark 11 as it diverges from the primary theme of the paper.

Reviewer comment (Theory):
In Remark 11, the construction of $E^*$ does not seem correct, as repeating $E_1$ several times will only result in a graph supported on the diagonal, thus $E^{*}_\sharp\mu$ is not the uniform distribution over the higher dimensional cube. Similarly, the next claim in the remark over the use of $\nu$ with independent marginals seems unjustified.

Author response:
In response to your thoughtful feedback, we will omit Remark 11 as it diverges from the primary theme of the paper.