Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper provides a notion of generalisation for sliced Wasserstein distances, that allows to explore nonlinear projections in arbitrary subspaces in a suitable and efficient way. The paper is well-written and provides a novel solution to the problem of exploiting nonlinear subspaces in computing distances. I feel this idea, although simple can be quite powerful, and can be extended to wide domains especially identifying objects based on arbitrary feature selections although I do not think the authors have explored that direction in this piece of work. 
Comments:
(1) Numerical results show the distance computed by ASWD is smaller than other measures. This can, however, be misleading in the sense that it might also be obtained by insufficient exploration of non-linear subspaces. For eg, if the sliced Wasserstein distances are computed along orthogonal directions to the primary features the Wasserstein distance obtained in that regard will also be very small, although this does not in anyway validate the superiority of the distance metric. I feel this argument therefore needs some clarification.
(2) Can I use this method to construct arbitrary nonlinear projections of choice instead of the projection that gives the best distinction? For eg, suppose two pictures have several objects among which cars in the two pictures are the most distinguishing features. However, I also want to see what other features can help distinguish these two pictures from other pictures which do not have cars. How do I do that? In that broader sense I suppose my question deals with trying to identify barycenters of objects via sliced Wasserstein distances. Any thoughts in that regard could be useful for the reader.

Overall I find the paper an interesting read although it requires some clarifications as outlined above.
review_title: Clarification of numerical results necessary

final_score: rating: 7: Good paper, accept
review_content: This paper introduces the augmented sliced Wasserstein distances (ASWD) to capture non-linear projections, as opposed to sliced Wasserstein distances. To this end, the architecture first maps inputs to higher dimensional hypersurfaces, before projecting. The designed ASWD is shown to be a metric as long as the initial mapping is injective. 

Numerically, the initial mapping is parameterized as the concatenation of the input and its output using a neural network. The loss aims at capturing a hypersurface that differentiates measures the most, and is regularized to control the initial mapping. The first task considers the minimization of sliced Wasserstein flows for different settings of synthetic data. The other task consists of generating images with GANs, using a sliced Wasserstein loss to learn the generator network.

Strong points of the paper include:
1.	The paper is well written. The literature review is very thorough and comparison with the proposed method is well explained.
2.	Both theoretically and numerically, the method is well described, well compared to existing notions and yields convincing results.

Drawbacks / questions:
1.	It seems that a lot of projections are needed to retrieve visually satisfactory generated examples.
2.	Have you tried other types of injective maps than Eq 15?
3.	Can you comment on the impact of regularization strength lambda in practice?

I recommend an accept for this paper, which, to the best of my knowledge, brings both theoretical and numerical valuable contributions to the literature.
review_title: Valuable contributions

final_score: rating: 4: Ok but not good enough - rejection
review_content: Augmented Sliced Wasserstein distance between random vectors X&Y is defined as SWD between g(X) and g(Y) for an injective g. 

It is claimed that this generalization solves the problem with SWD, i.e. a small distance for most of the projection directions.  Since the paper is purely experimental, then it is better to demonstrate that this issue is resolved (it is not obvious why the problem disappears).

Since g_omega is also searched for by maximizing (16), then it becomes hard to interpret the value of metrics (also taking into account the role of lambda). What it measures now?

The code is given, results seem reproducible. Reported results with generative modeling compare ASWD with SWD, GSWD, DSWD. Since the paper is experimental, maybe it is natural to expect a comparison with SOTA generators that do not deal with generalizations of Wasserstein distance. Also, fake images generated by SWD, GSWD, DSWD are not given (only FID is shown).
review_title: The metrics is hard to interpret

Meta-review: Given two data measures in R^d, this paper proposes to use a NN to augment the representation of each data point found in these measures with additional coordinates. The measures are then compared using the sliced Wasserstein distance on these augmented representations. Because this augmentation is injective by design (the original vectors are part of the new representation) simple metric properties are kept. The authors propose to learn in a robust/adversarial way these augmentations. They propose simple experiments to illustrate that idea.

Although I found the idea interesting, I think it falls short of acceptance at ICLR. I agree with the sentiment of other reviewers 1 and 2 that defining another variant of robust/NN inspired variant of the W distance is interesting, but at this point the readership of the conference expects more than simple experiments on toy data and hard to interpret GAN results. I think there is value in the draft as it stands now, but that more efforts are needed to convince this variant is scalable / useful for other downstream tasks (e.g. W barycenters, or other easier to interpret W problems in lower dimensions).

minor comments
- as it stands, equation 2 is wrong if you do not add more conditions on the cost function d(.,.). 
- " the idea of SWD by projecting distributions onto hypersurfaces rather than hyperplane" -> this is wrong, the projection is done onto lines or curves, not hyperplanes or hypersurfaces.

Decision: Reject