Paper:
Banach Wasserstein GAN
Wasserstein Generative Adversarial Networks (WGANs) can be used to generate realistic samples from complicated image distributions. The Wasserstein metric used in WGANs is based on a notion of distance between individual images, which induces a notion of distance between probability distributions of images. So far the community has considered 2 as the underlying distance. We generalize the theory of WGAN with gradient penalty to Banach spaces, allowing practitioners to select the features to emphasize in the generator. We further discuss the effect of some particular choices of underlying norms, focusing on Sobolev norms. Finally, we demonstrate a boost in performance for an appropriate choice of norm on CIFAR-10 and CelebA.

Review:
Update: After reading the authors' response, my opinion remains that this is a strong paper and I argue for its acceptance. The addition of FID and emphasis of the L^p norms further improve it. I do still suggest that the authors run experiments on at least one other dataset: I agree that more datasets would "contribute relatively little to the reader’s understanding of the proposed method", but they do serve a purpose of convincing readers of the practical effectiveness and versatility of that method. I also don't think the authors need to present the same number of results for each dataset; one experiment per dataset demonstrating improvement seems sufficient. --- The popular WGAN-GP method of GAN training minimizes a loss which is defined in terms of an L^2 ground metric. This paper generalizes that method to arbitrary Banach spaces and in particular explores training WGAN-GPs with Sobolev norms, which are perhaps a more sensible choice for image modeling. Experimentally, the proposed BWGAN achieves very good Inception scores on CIFAR-10.  This work directly addresses an important but under-explored aspect of WGANs, namely the importance of the ground metric. The choice of L^2 was made originally entirely because it was convenient, and prior to now nobody had carefully studied (a) whether the choice of ground metric matters or (b) how to implement WGANs with other ground metrics. In my opinion this paper is a solid contribution toward both.   The theoretical contribution seemed sound to me, but I don't have a very extensive theory background so I defer to the other reviewers' opinions there.  The experiments are generally carefully carried out and convincing. The authors achieve very competitive Inception scores with just a faithful (and careful, judging by the description in the supplementary) reimplementation of the WGAN-GP CIFAR-10 model, plus the method in the paper. An important caveat is that Inception scores are known to vary substantially between different random seeds of the same model hyperparameters [3], but the authors do re-train the model several times from scratch and plot the distribution in Fig 2. The paper could use additional experiments on different datasets and architectures. In particular I'd be very interested in a 128x128 image synthesis result, since it's plausible that the choice of norm is even more important in higher dimensions.  Regarding figure 1, I'm not totally convinced by the visual assessment of samples or Fréchet derivatives. In particular, it's hard to tell whether the derivatives for s=-2 are have relatively greater low-frequency components than s=2, or whether the magnitudes for s=2 are just lower overall. Were the gradients rescaled to the same interval before visualization?  The paper relies heavily on Inception scores for evaluation. This is understandable (they're a de-facto standard) but somewhat fraught: the experiments in this paper amount to evaluating different objectives (the BWGAN losses induced by different metrics) on the basis of their correlation with another objective (Inception score), which seems risky when the ``evaluation'' objective is heuristic. Unlike most other GAN papers, which argue (explicitly or implicitly) that improved scores result from more stable training dynamics, here the improvements are attributed mostly to the perceptual alignment of the loss. As such, it would be good to see at least one other evaluation metric reported; I suggest also reporting FID [1] scores.  As a side result, the authors derive a heuristic for choosing hyperparameters which applies to both their method and WGAN-GP. Their baseline WGAN-GP outperforms the one in the original paper (which was state of the art at the time). It would be ideal to validate the heuristic empirically, for examply by sweeping over hyperparameters and showing that the heuristic achieves scores close to the optimal found by the sweep. In particular, in order for Figure 2 to support the conclusion that the choice of s matters, the other hyperparameters must be set to optimal or near-optimal values, which might themselves vary with s.  The result (in the supplemental) that performance varies substantially between different L^p norms is significant and I feel should be included in the main paper, especially given that they basically matched the Sobolev norms in Inception score. Seeing that result changed my conclusion that Sobolev norms are an especially good choice for image modeling.  The writing is generally clear and thorough. The math is easy to follow and the experiments are described in enough detail that reimplementation shouldn't be hard. It wasn't obvious before fully reading the text that the computation of the Fréchet derivative could be done ``implicitly'' by the trick on L172; the authors might want to highlight this in an algorithm box somewhere. The authors do a good job of providing background and placing their work within a larger context.  Overall this paper explores an important topic, presents sound theory and a simple method, shows convincing (if slightly limited) experimental results, and is written clearly. Therefore I recommend acceptance.