Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper introduces a new formulation to compute the distance between probabilities distribution on the hyper-sphere, namely, the Spherical Sliced-Wasserstein (SSW). The proposed formulation involves a novel spherical Radon transform (to project to great circles) and the computation of the WD on the circle.
The method is validated on two tasks:  variational inference (matching a target distribution up to a scale) and autoencoders with a prior distribution on the spherical latent space.
strengths_and_weaknesses: The paper is quite difficult to follow in the mathematical formulation without the proper background. If possible, it would be helpful to add some intuitive explanation or figure to better understand the main steps of the proposed formulation. Nevertheless, the paper tackles an interesting problem, and being able to compare density distributions on a hyperspherical domain could be useful for some machine learning tasks, as shown with the (toy) applications variational inference and autoencoders.

On this latter point, I found the experimental setup not properly explained. Most of the details are demanded of the supplementary material, which makes it difficult to understand what was done without jumping forth and back. It would also be nice to see more examples and analyses on the spherical latent space of the autoencoders (for instance, the different behaviour between datasets for which the data manifold is known to be spherical (e.g. rotations) and datasets the data manifold is presumed to be more complex or hierarchical). 
questions: I would like the authors to address my previous two concerns:
- trying to add an intuitive layer to the mathematical formulation (possibly with supporting figures)
- expanding the analysis on the benefits of using a spherical distribution prior with autoencoder on datasets with different characteristics.
limitations: yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposed a new version of Sliced Wasserstein Distance on the sphere as a first step to deal with manifold data. Section 2 is to recall the definition of Wasserstein distance and Sliced Wasserstein distance. In Section 3, the authors introduced the spherical sliced Wasserstein distance, mainly based on the property that the sphere could be described by spherical coordinate system. They also defined the Radon transform for the SSWD and show some properties of Wasserstein distance on circle. The implementation is shown in Section 4. Section 5 is devoted for applications which include variational inference, generative modeling and density estimation.
strengths_and_weaknesses: Strengths: The idea of Spherical Sliced Wasserstein Distance is nice and interesting. The SSWD is well-defined and some of its properties are explored.

Weaknesses: The applications of SSWD shown in the paper are limited. In those experiments, the SSWD does not really outperform the SWD. The authors need to find applications with spherical data, where the advantages of SSWD could be shown clearly. 
questions: No
limitations: It is fine.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes spherical sliced Wasserstein discrepancy, a variant of sliced Wasserstein on the sphere. To construct the discrepancy, the authors develop a new variant of Radon Transform which is spherical Radon Transform, and utilize the closed-form solutions of the Wasserstein distance on the circle. In more detail, the spherical sliced Wasserstein is defined as the Wasserstein distance between geodesic projected measures on the great circles measured by the uniform distribution over all the projections or Stiefel manifold. On the application side, the authors apply the new discrepancy to hyperspherical auto-encoders and density estimation on the sphere.
strengths_and_weaknesses: # Strengths
## Originality 
* The paper proposes the first variant of sliced Wasserstein on the hypersphere.
* Spherical Radon Transform is new.

## Quality
* The authors derive the metricity of spherical sliced Wasserstein, the connection of spherical Radon Transform and geodesic projection, and the kernel of spherical Radon Transform. 

## Clarity
* The paper is well-written and easy to follow.

## Significance
* The contribution of the paper is significant for estimating densities and modeling on the hypersphere and for the community of sliced Wasserstein and optimal transport. 
* Spherical sliced Wasserstein is better than sliced Wasserstein on variational inference task on the sphere.

# Weaknesses
* The injectivity of spherical Radon Transform has not been established yet hence spherical sliced Wasserstein is not a metric.
* The experiments are not intensive. The only experiment on real data in the main paper is hyper-spherical AE on MNIST. However, the improvement is not very significant compared to the conventional sliced Wasserstein. Other datasets such as CIFAR10 and CelebA should be considered. 
* The experiments on self-supervised learning should be moved to the main text.
questions: * One benefit of sliced Wasserstein is a good sample complexity which escapes the curse of dimensionality of the Wasserstein distance. Is this still hold for spherical sliced Wasserstein?
* It seems that both spherical sliced Wasserstein and sliced Wasserstein need to use empirical samples when dealing with continuous measures in hyperspherical autoencoder. This can be seen as the usage of mini-batch OT [1,2,3,4,5]. Can we derive some special cases of spherical sliced Wasserstein where the Wasserstein distance between projected continuous measures has closed-form e.g, von Mises Fisher distribution?
* One paper might be related [6].

I will raise my score if the authors can add additional experiments on real datasets e.g., hyperspherical autoencoder on CIFAR10, CelebA, estimating density on Tiff dataset [7], and so on.

[1] "Learning with minibatch Wasserstein : asymptotic and gradient properties" 
[2] "Minibatch optimal transport distances; analysis and applications"
[3] "Improving Mini-batch Optimal Transport via Partial Transportation"
[4] "On Transportation of Mini-batches: A Hierarchical Approach"
[5] "Unbalanced minibatch Optimal Transport; applications to Domain Adaptation"
[6] "Improving Relational Regularized Autoencoders with Spherical Sliced Fused Gromov Wasserstein"
[7] "https://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-density-adjusted-to-2015-unwpp-country-totals-rev11/data-download#"
limitations: The paper develops a fundamental tool hence there is no foreseen negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper introduces sliced Wasserstein discrepancies defined on the hypersphere $S^{d-1}$. 

From a computational perspective, this corresponds to computing an expected Wasserstein distance on great circles. 

From a theoretical perspective, the authors introduce a new spherical Radon transform that allows to formally define the proposed discrepancy. Further, this formal treatment allows to show that the proposed discrepancy is at least a pseudo-distance on the set of distributions on $S^{d-1}$ with finite $L_p$-norm.

Finally, the authors conclude with various experiments ranging from simple gradient-flow examples to more sophisticated variational inference or auto encoder experiments.
strengths_and_weaknesses: The paper is overall well-written and easy to follow. The theoretical claims sound reasonable to me, although I am not a mathematician and not particularly specialized in the theory of Radon transforms. Finally, the experiments are insightful and demonstrate that the discrepancy measure behaves as expected.

While the experiments do not show a strong performance increase over "naively" using the standard sliced Wasserstein distance, I still believe that the paper makes an important contribution as the introduced discrepancy is clearly more suited to the space $S^{d-1}$. Further, Figure 6 seems to show an improved class separation in the auto encoder experiment.

I see the following points for further improving the paper:
 * The authors could more clearly highlight the reason why their paper is not just a particular instance of the generalized sliced Wasserstein distance (GSW) by Kolouri et al. If I understand correctly, GSW treats the "projected particles" as members of the real line, and hence cannot take the particular geometry of a circle into account. Or would it be possible to define the proposed discrepancy as a particular instance of a GSW distance?
 * Following up on the previous point, the authors could additionally compare to GSW in their experiments. If the proposed method can indeed not be defined as a GSW instance, the GSW approach would I guess correspond to an intermediate version between regular SW and SSW, which uses the projection onto the great circles but does not respect the circular nature of the projected space.
questions: * Was the improved separation of the latent space for SSAWE consistent among seeds? (Figure 6)
* What is the measure of variability in Figure 5? Standard deviation? Standard error? And how many intervals are visualized? Two times?
limitations: In general, the authors do not make exaggerated claims about applicability or potential of the method. In my opinion, only the statement in line 289/290 regarding improvement in the context of VI may be a bit exaggerated/unclear, as it is not indicated which variability measure is visualized in Figure 5 and further the measures overlap. Consequently, the authors could simply say that SSWVI performs just as well as SWVI.

Finally, a common problem of sliced Wasserstein distances is that the required number of projections potentially grow exponentially with dimension (as e.g. argued for in the "Generalized Sliced Wasserstein Distances" paper by Kolouri et al.). The authors could mention this shortcoming and further link it to Figure 8b) in the appendix, as the reduction of $SSW_2^2$ distance with increasing dimension may indeed by explained by the aforementioned dimensionality issue of any sliced Wasserstein distance.
review_title: null

Meta-review: metareview: This paper has generated a long discussion and although it has strong theoretical merits, we all concord that the paper lacks of empirical motivations as well as a strong empirical evaluations with respect to distance distributions not exploiting manifold sructure and thosed define on a manifold.  Hence, we believe that at this point it would be preferable to have such empirical evidence (ideally with quantitative results on real-world problems) before accepting the paper.  Given that, we are sure that the paper will be much stronger and of broader interest to the ML community.
confidence: Certain
Decision: Reject