Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper introduces a general geometric-aware pooling function. The key idea is based on sliced Wasserstein distance. The input is transformed to a set of points no matter what the original format is. Then, the Wasserstein distance between this input set and a trainable reference set is computed. The 1 sliced Wasserstein distance can be computed by sorting the input feature in a straightforward fashion. This paper demonstrates the performance on set classification, graph classification, and image classification tasks. In some cases, the proposed method outperforms its counterparts.  
main_review: This paper introduces a general geometric-aware pooling function. The key idea is based on sliced Wasserstein distance. The input is transformed to a set of points no matter what the original format is. Then, the Wasserstein distance between this input set and a trainable reference set is computed. The sliced Wasserstein distance can be computed by sorting the input feature in a straightforward fashion. This paper demonstrates the performance on set classification, graph classification, and image classification tasks. In some cases, the proposed method outperforms its counterparts.  

Pros:
1. This paper presents a great motivation to design a geometric-aware pooling function. Compared to others, this method treats the input features as  a set of samples from a probability distribution. This novel interpretation makes it possible to incorporate tools in geometric optimization. 
2. The idea of using a sliced Wasserstein distance is technically sound. In the 1D case, the Wasserstein distance can be easily computed by sorting. So this significantly reduces the complexity of Wasserstein distance computation. 
3. This paper introduces the idea of using a trainable set, which can be viewed as  a dataset prior. 

Cons:

1. The proposed method seems only applicable to the global level and it is not able to replace the a general pooling (e.g., strided average pooling) with the proposed method. As a consequence, this pooling module can only work on classification tasks. 
2. My another concern is the practical value of the proposed method. Although this method has a good motivation, it’s still quite challenging to incorporate it into a general deep learning architecture and achieve comparable performance to state-of-the-arts. 
3. In the point cloud classification case, the performance of the proposed method is much lower than existing point-based methods. E.g., PointNet without the spatial transformer can achieve more than 87% on this dataset. In addition, ModelNet40 is a small scale dataset, so testing with the SOTA encoders (DGCNN or KPConv) will be more convincing. 
4. In the graph classification case, the proposed method does not show consistent improvements over its counterparts, either. 
5. Similarly, in the image classification case, the method is only comparable to FSPool. Also, I am confused why the method does not evaluate on the most common datasets such as CIFAR and/or ImangeNet. 

limitations_and_societal_impact: See above.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This is an extremely well-written paper on a topic well-suited for presentation at NeurIPS. The material is highly technical and the authors do an excellent job at providing a clear exposition and necessary background material (see below for minor caveats). While empirical results are not compelling across all experiments, the results do show promise in this line of research, especially given its strong theoretical underpinnings.
main_review: I have very little to add beyond the summary above. Some small questions for the authors:

1. Can the sets $X_i$ be multisets, which would give rise to different empirical distributions (i.e., weighted delta functions) and how would this affect the method?
2. Why use ${\cal X}$ and ${\cal X'}$ in Eqn (1) instead of ${\cal X}_i$ and ${\cal X}_j$?
3. Are the regularity conditions presented in [39] (L.131) satisfied when the inputs are learned from a neural network?
4. Exactly what is learned in the set $X_0$ (L.171)? My understanding is that it's the $x_n^0$ that are learned/perturbed during learning, so it's unclear why the underlying distribution $p_0$ is needed. These points are never resampled.
5. The paper discusses computational complexity in terms of computing PSWE. Can the authors provide running times as a function of L for the experiments?
 


limitations_and_societal_impact: Limitations are discussed briefly throughout the paper but there is no single place where they are summarized. There is no discussion of societal impact and I don't see any negative consequences of this work to society.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper presents a new pooling operation that is based on computing the sliced-wasserstein distance between empirical distributions of input samples. Specific modifications to the conventional sliced-wasserstein distance by the authors include the introduction of the trainable slicer and the trainable reference sample. Experiments on point cloud classification, graph classification, and image classification show improvements when a large number of learnable slicers are used.
main_review: Pros:
* Paper is relatively clear written and easy to follow. The content is pretty much self-contained with sufficient background information about wasserstein distance in Section 3.
* A straightforward modification of the sliced-wasserstein distance by introducing two trainable components: trainable slicers and trainable reference sets.
* The problem being addressed here is important and has many practical applications.

Cons:
* Motivation of the introduced modifications is not clear. In line 58, the authors motivate the proposed changes by claiming that “PSWE provides an exact Euclidean embedding for the (generalized) sliced-Wasserstein (SW) distance”. However, without the two proposed modifications, the conventional generalized sliced-Wasserstein distance also provides Euclidean embedding during distance computation.
* Section 4.3 “On projection complexity of sliced distances” is not clear in its current form. The authors claimed that samples often live on a low-dimensional manifold and therefore a large number of projections L should be used (line 191). However the exact projection complexity (compute/memory) analysis is not presented.
* Also in Section 4.3, experiments with different numbers of projections have been conducted in [A1]. What is the main message the authors try to convey here? Are trainable slicers more efficient than random projection slicers? From my point of view, random projection in practice is a very strong baseline and it is quite critical to compare the proposed trainable slicer and random projection slicer in terms of classification accuracy and computational cost.
* [A2] is one of the early works that proposed the concept of sliced-wasserstein distance and should be discussed.

[A1] Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation. CVPR 2019\
[A2] Wasserstein barycenter and its application to texture mixing. SSVM, 2011.

limitations_and_societal_impact: An immediate baseline would be replacing the trainable slicers with random projections and comparing the performance on various classification tasks listed in this work and the corresponding computational overhead.
review_title: null

Meta-review: Reviewers have strong disagreements about this paper. After discussions, I concur with several
points adressed by the reviewers. Most importantly, we believe that the experimental studies need
to be made stronger, in term of comparison with SOTA methods (even if they do not reach SOTA), 
potential way for local pooling... 
In addition, I would like to mention the missing stronger discussion with the paper of Mialon 2021 et al.
on "trainable OT embedding ... ". Indeed, From my point of view, the proposed method is just a variant of 
the one of Mialon where the Wasserstein distance is replaced by a 1D Wasserstein and you consider multiple references
where one reference is obtained from each slicing (multiple references  is described in Section 3.5 in Mialon et al). 
An averaging over the row would give an embedding similar to those of Mialon. Main difference is that the input distribution is 
somehow pertubed as it is also projected through the slicing. Hence, I would have expected a comparison also
with this method.
Decision: Accept (Poster)