Pooling by Sliced-Wasserstein Embedding
Authors: Navid Naderializadeh
Joseph F. Comer
Reed W Andrews
Heiko Hoffmann
Soheil Kolouri
Venue: NeurIPS 2021 Poster (2021)
Keywords: Pooling
feature aggregation
permutation invariance
optimal transport
set learning
representation learning

Abstract: Learning representations from sets has become increasingly important with many applications in point cloud processing, graph learning, image/video recognition, and object detection. We introduce a geometrically-interpretable and generic pooling mechanism for aggregating a set of features into a fixed-dimensional representation. In particular, we treat elements of a set as samples from a probability distribution and propose an end-to-end trainable Euclidean embedding for sliced-Wasserstein distance to learn from set-structured data effectively. We evaluate our proposed pooling method on a wide variety of set-structured data, including point-cloud, graph, and image classification tasks, and demonstrate that our proposed method provides superior performance over existing set representation learning approaches. Our code is available at https://github.com/navid-naderi/PSWE.

Pooling by Sliced- Embedding
Navid Naderializadeh"
Department of Electrical and Systems Engineering
University of Pennsylvania Philadelphia, PA 19104
nnaderi@seas. wue.nneees...eeee....
Joseph F. Comer, Reed W. Andrews, Heiko Hoffmann
HRL Laboratories, LLC.
Malibu, CA 90265 ( (jfcomer, rwandrews, hhof fmann J 1@rrh com
Soheil Kolouri
Computer Science Department
Vanderbilt University Nashville, TN 37235
Abstract
Learning representations from sets has become increasingly important with many applications in point cloud processing, graph learning, image/video recognition, and object detection. We introduce mechanism for aggregating set of features into fixeddimenssonal representation. particular, we treat elements of a set as samples from probability distribution and propose an end-to-end trainable Euclidean embedding for sliced- Sosdd--sssrstein distance to learn from set-structured data effectively. We evaluate our proposed pooling method graph, and image classification tasks, and demonstrate that our proposed method provides superior performance over existing set representation learning approaches. Our code is available at hattps://github com/ navid swwavidndderiii naderi/PSMi
cameetrically-inerrreeete
and generic pooling
a wide variety of set-structured data, including point-cloud,
Introduction
Many modern machine learning (ML) tasks deal with learning from set-structured data. In some cases, the input object itself a set, as in point cloud clasiffcation/reggrssion, and in other cases, the complex input object is described as set of features after being processed through backbone, i.e., feature extractor. For instance, in graph mining, graph is represented as a set of node embeddings, and in computer vision, image is represented as set of local features extracted from its different regions (i.e., fields of view). There are unique challenges namely: i) the set cardinalities could differ from one instance another, and ii) the elements of the set do not necessarily have an inherent ordering. These challenges call for MI. models that can both handle varied input sizes and invariant permutations, i.e., the model output does not change under any permutation of the input set elements.
dealing with such set-structured data,
Prior work on learning from set-structured data can be broadly categorized methods based on either implicit or explicil embedding of sets into Hilbert space. Implicit embedding approaches
Work done while at HRL Laboratories, LLC.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

(i.e.. kernel methods) rely on defining a distance/similarity measure (i.e., kernel) between two sets 2 7 9 These methods involve one of the two strategies of 1) solving a correspondence problem between elements of the input sets and measuring the similarity between corresponding elements, or 2) comparing all pairs of elements between the two sets based on a similarity measure (e.g., approaches based on Maximum Mean Discrepancy). On the other hand, explicit embedding methods learn a -nniatatiinnnn mapping into a Hilbert space and provide bxedddiienniooal representation for given input set that classic ML approaches could further process [T More recently, algorithms based on a composition of perruutation-equivarae pooling mechanisms have been proposed to
neural network backbones and cermutation-invariia define a parametric permutation invariant mapping [II 13
14 15 [6]. Notably, Zaheer et al. [ proved that such composition provides a universal approximator for any set function. Lee et al. [4] further showed that utilizing permutation-equiaa equivvarint backbones that do not process set elements independently but model the interactions between the set elements (e.g., using self-attention) is theoretically and numerically advantageous. Similar observations have been made in the field of
graph learning using various graph neural network (GNN) architectures [17 18| 19). In parallel,
several works have studied the importance of permutation-invaaian pooling mechanisms to go beyond
the commonly used mean, sum, max, or similar operators [12 13 15 [6].
convenient interpretation in dealing with sets is considering their elements as samples from an these probability distributions
unknown underlying probability distribution and comparing/embeddee to perform set learning. Due to this interpretation, optimal transport has played a prominent role in learning from sets. For instance, Kusner et al. 20) and later Huang al. (2I] represented document as a set of words. They leveraged the /- 1-Wasserstein distance (i.e., the earth mover's distance) to compare these sets with one another and define a measure of document similarity. Various researchers have devised similarly flavored approaches in computer vision by comparing images via calculating the Wasserstein distance between their sets of local features. For instance, Zhou et al. [22] use this distance to learn prototypes for image classes and perform few-shot inference, while Lin et al. [23 leverage it for designing diverse adversarial examples. More recently, similar ideas were used for image enhancement [24]. Finally, comparing sets via Wasserstein distances has also been proven to
be useful in other applications including graph learning 9 12 domain adaptation 25 26 and
transfer learning (27].
In this work, we propose novel cheoretically-gronnde. and simple compute permutation-invaaie pooling mechanism for embedding sets of various sizes into a fixed-size representation. Our proposed method, which refer to as Pooling by Sliced- Wasserstein Embedding (PSWE), provides an exact Euclidean embedding for the (generalized) sliced-Wassersee Wasserstein (SW) distance. We start by defining a similarity measure between sets of samples based on the SW distance. We then propose an explicit set embedding for which the Euclidean distance between embedded sets equals the SW distance between them. In our experiments, we follow the recent work on set learning [ [4 -et.tttttiioeuuvviiiiiiina backbone followed by our permutation- invariint pooling method
and use to perform end-to-end learning on different data modalities. We demonstrate the scalability and effectiveness of our approach on various learning tasks, including point cloud classification, graph classification, and image recognition. Aside from introducing a novel pooling mechanism, one of the key numerical insights of our work is that basic pooling mechanisms, such as mean meen-poolingg provide competitive performance when the permutation-cen -qquvvvrrnn backbone is complex. However, for plain backbones (e.g.. shared multi-layer perceptron (MLP) among the set elements), more sophisticated pooling mechanisms, including our proposed PSWE method as well other recently-proposed pooling mechanisms in the literature (e.g.. Pooling by Multi-Head Attention (PMA) [4] and Featurewise Sort Pool (FSPool) [5) significantly boost the performance compared to basic pooling mechanisms.
2 Related Work
Permutation-invarte functions are crucial components in learning from sets and are often used as pooling layers to aggregate features from a set and provide constant-size representation regardless of the set cardinality. Max, sum, and mean pooling are simple, yet very widely used, examples of such functions. Recently, various works have shown the effectiveness of more sophisticated and often learnable pooling operators in improving the performance of learning from set-structured data [3 14 28
Murphy et al. [13] introduced a pooling mechanism based on the average of permutation snnivarriantfuncttion applied to all re-orderings of the set elements. Summing over all
2

---

re-orderings of an input set is, of course, computationally prohibitive. Hence, one can use a canonical ordering of set elements (e.g.. via sorting [5].. or learn to predict the optimal permutation for an input set [29 30 28
Another prevalent idea is to perform pooling based comparing input set with trainable and fixed-size reference sets. For instance, Skianis et al. [10] proposed a pooling that consists of the distances between an input set and trainable reference sets, where the distance was calculated by solving the correspondence problems between the input and each reference set. More interestingly. this idea is analogous to pooling by multi-head attention (PMA), an important building block in the Set Transformer and Perceiver architectures reference sets and an input set is used as a permutation-invarr
[14 311, where the cross- apsss-tttntionn between trainable function. Attention-baser pooling has been shown to perform really well in practice on wide range of applications.
14 32
We introduce novel pooling mechanism by treating sets as empirical probability measures and calculating an embedding for these probability measures in which the Euclidean distance between two embedded sets is equal to the Wassersten distance between their empirical distributions. Our work is closely related to the work by Mialon et al. [12] and Kolouri et al. [16]. In short, [12] proposes an approximate Euclidean embedding for the Wasserstein distance, similar [6], reproducing kernel Hilbert space (RKHS), while our proposed framework is based on devising an exact Euclidean embedding for the (generalized) sliced- Wasserstein distance. Interestingly, our proposed pooling by sliced- Wasserstein embedding (PSWE) can also be viewed as theoretically-groune of the sorting-basse based FSPool mechanism proposed in [15), where we show that the introduction of trainable slicers, as well as trainable reference sets, further boost the end-to-end performance in wide spectrum of classification tasks.
generalization
3 Background
Let X, (arh Rd) denote an input with N; elements living in Rd. We assume that the set elements are samples from unknown underlying probability measure, f4, defined in X' R with probability density d1)(() = (r))dr, and what we have observed is the empirical distribution (x())) N. No 8(x= xr)), where d(-) is the Dirac delta function.
3.1 2- Wasserstein Distance
Let μ4 and μU denote two Borel probability measures with finite 2nd moment defined on N;, X2 C Rd. with corresponding probability density functions and Py, respectively. The 2- Wasseresttiin distance between μ4 and μ4 the solution to the optimal mass transportation problem with &z transport cost
33
inf / (EI)())))))) XXXX, ( ((Tr)
1/4
||=; 1,11dd7
where 74,4, the set of all transportation plans y E [( (4y) such that 7(4 x X,) = (A) and (&& x BB) μ, (B) for any Borel subsets A C X; and B C Ay. Due to Brenier's theorem B4] for absolutely continuous probability measures μi; and μy (with respect to the Lebesgue measure), the 2- Wasserstein distance can be equivalently obtained from the Monge formulation [33]
We(mi, ( (f inf (EMPP(EN))) J / ||= ((0 (()))(())()))))
where MP(44, = ( X 1 = (1} and f414 represents the pushforward of measure Ms. characterized as f= (B) Hf(f (BB) for any Borel subset B C Xy. The mapping f is referred to as a transport map [35. and the optimal transport map is called the Monge map. For discrete probability measures, when the transport plan y is deterministic optimal coupling, such a transport plan is referred to as a Monge coupling 33 In case of non-deterministic transport plan 7, one can obtain an approximation of the Monge coupling via barycenteric projection, e.g., see [16] 12] In this paper, we mainly use the 2- Wasserstein distance and hereafter, for brevity, we refer as the Wasserstein distance.
3

---

For one-dimensional probability measures, the Wasserstein distance has closed-fore form solution and can be calculated
We(ue, = - (1
(3)
where F is the quantile function of μ4. The simplicity of calculating Wasserstein distances between one-dimensional probability measures has led to the idea of sliced- Wasserstein 336 and generalized sliced- Wasserstein [40] distances, which we will review next.
3.2 (Generalized) Sliced- Wasserseessr Distances
Let go R2 + R be parametric function with parameters 0 E Ωg C satisffing the regularity conditions in both inputs and parameters as presented in [40] For -aaeed-tessrtteinn distance, go(x) 8" where 9 gd- is a unit vector in Rd, and S denotes the unit d-dimensional
hypersphere. The generalized slice of probability measure μ4 with respect to ge is the one-dimensional
probability measure g0 H4, which has the following density for all R.
/ 8(t go(x))dp,(x).
(t)
(4)
The generalized Wasssesessr distance is then defined as
(SWa(uisH) = (1 Wa (go# (0
1
(5)
Note that for go(x) = 0 and Sw = Sd-1, the generalized sliced- Wasserstein distance is equivalent the Wasserseenn distance. Equation (5) the expected value of the Wasserstein distances
between slices of distributions and jíj-
distance include max (generalized) sliced- Wasserstein distance [38 40), in which the expected value in (5) is substituted with a maximum over Og i.e.
Extensions of the (generalized)
max-gSW2(14, p2)
max Wa W2 Pi; (1)) BE
subspace-robust Wasserstein distance 41 which generalizes the notion of slicing to projection onto subspaces, and the distributional sliced- Wasserstein distance [42] that proposes to replace the expectation with respect to the uniform distribution on Pg with non-uniform and learnable distribution.
From an algorithmic point of view, the expectation in 5 is approximated using Monte-Carlo integration, which results in an average of set of Wasserstein distances between random slices of d-dimensional measures. In practice, however, GSW distances only output good Monte-Carlo approximation using a large number slices, while max-GSW distances achieve similar results with only single slice, although at the cost of an optimization over 0.
PSWE: Pooling by Sliced Wasser-Wassestteinn Embedding
4.1 (Generalized) Sliced- Wasser-aassr Embedding
We propose a Euclidean embedding for probability measures, such that the weighted Euclidean distance between two embedded measures equivalent to the GSW distance between them. Consider set of probability measures J.M. with densities {p: bp For simplicity of notation, let n4f := go#t4 14 denote the slice of measure μ4 with respect to ge. Also, let μi denote a reference measure, with o representing its corresponding slice. Then, it is straightforward to show that the optimal transport map (i.e., Monge map) between μ and pf can be written as
- = F-1 F a

---

where as mentioned before, F. and FT respectively denote the quantile functions of μ and 12 o M Now, letting id denote the identity function, can write the so-called cumulative distribution transform (CDT) 433 of μ as
v =1 id.
(8)
For fixed 0, of satisfies the following conditions, the proof of which can found in the Supple- mentary Material:
Cl: The weighted 2-norm of of equals the Wasserstein distance between uf and M i.e..
|| 11 = (1 11
W2(
hence implying that || lug.2 = O. C2: the weighted & distance between us and of equals the Wasserstein distance between n4 and u., i...
|| 191,, We(m 1g).
Finally, the GSW distance between two measures, Jı; and pay. can be obtained
(SW2(p))))) (1 1|, ~/ 22 ( 10000
Based on (9). for probability measure μ4, the mapping to the embedding space obtained via (11)) = ( 0 E lt.).
4.2
Algorithmic Considerations
In this section, we introduce novel pooling algorithm, termed pooling by sliced- Wasserstein embedding (PSWE). Let X; (ar pi) denote input set with N; elements, and Xa {x" Po J.y denote the set of N samples from trainable reference set. Let θ, {0: ~ Un } denote set of I parameters sampled uniformly from Sz. Then, the empirical distribution of the 7th slice of p; be written ph Monge coupling between P. 20
Em= 5(t go, (=).)) To obtain vit need to calculate the and :th In what follows, we consider two scenarios:
1. When the input set and the reference set have the same cardinalities, i.e., N, = N. the Monge coupling (i.e., the discrete counterpart of the Monge map shown in 7 obtained by sorting X. == (go, brr)))= and X Let Ti denote the permutation indices (i.e., a argsort) obtained by sorting X Then, letting To denote the ordering that permutes the sorted set back to the original ordering based on sorting of elements in X the Monge coupling is obtained via Tı ma(mo [-]]] and the per-slice embedding is calculated as
10.0 = ge: (Ii,,r,
*(m1) -ge, (x
(10)
2. When the set cardinalities vary, the Monge coupling can be obtained via interpolation using 77. In our experiments, we use the Py PoTccc implementation of linear interpolation to evaluate F F-1 The per-slice embedding is calculated as
[ufi]
F-1 15;
Too
[1]
go, (4h)),
(11)
N
where F F, (x..) N4
To "" +1 assuming that the indices start from 0.
Note that, regardless of the cardinality of the input set, the per-slice embedding N-dimensional, i.e., tith N. The final embedding then defined as Φ(m4) = "/ ..... L4ll E N which satisfies GSW2(u4,14 ~ [[ي(())) o(4,) ||,,
(12)
where || 1|| denotes the Frobenius norm, and the approximation is due to the Monte-Carlo integral approximation with the slices.
httpp:://gii htt.:///w...nnalaaattttttrrrrena
5

---

Input: v (+6 M
Sliers Permutation Equivariant)
Ourputs sizens los, (r))
Mapa/Csupings Regalo Rayal
aud wird
Suub
M
M
Muni
R
Halt Set, fot Pollint
(ef + m200) fre [0.11
R
Trainable
Pooliing Sliced- -Wasserstein Embedding (PSWE) characterized
Permutation--mvariant Output
BOCKJ ~ (PTOree))
[e0x] - aZ,1, (aW(ma))
@ 4." (mx,6:3)) (
An overview of the proposed PSWE method. Each d-dimensional element in given input set X,, as well as each element in the trainable reference set X, is passed through multiple trainable then perform interpolation
Figure 1:
slicers {ge, 1 For each slicer, optimal transport maps that push the slicer output distributions of the reference set to the slicer output distributions of given set via 77. (10). and (TT). The resultant transport maps are then concatenated across all slices to derive the final set embedding.
the slicer outputs and derive the
4.3
On Projection Complexity of Sliced Distances
Given the high-dimensional nature of the problems of interest in machine learning, and the fact that samples often live on low-dimensional manifold, one requires large number of random projections, L, to obtain a good approximation of the GSW distance. This issue is related to the projection complexity of the sliced distances [38 40 Given the dependence
our pooling dimensionality on the number of slices, L, we would like to avoid using very large numbers of slices. Here, we devise a unique approach that ties our proposed embedding to metric learning. First, we note that ideas like max- max-GSW 440 38) or subspace-robust Wasserstein distance [4]] would not be practical in this setting, as the slicing parameters, I, are fixed for all probability measures and not chosen separately for each pairs of probability measures (/i, M45).
Given the training input sets, i.e., (x,) JMM and a reference set, Xo. we seek an optimal set of slices O that could be learned from the data alongside the other parameters in an end-to-end manner. This idea is related to [42] as it is similar to learning distribution over the unit hypersphere from which are sampling our L slices. The optimization O ties the PSWE framework to the field of metric learning, allowing us to find slices or, in other words, an embedding with a specific statistical characterization.
To put it all together, our pooling requires identifying: 1) the type of slicer ge R + R (e.g.. go(x) = 0T x), 2) the number of slices, I,, and 3) the number of elements in the reference set, N. Then, for an input set X; with N; elements, PSWE first slices the elements of the input and reference sets with respect to slicers ge, for E {1...., L}. Then, it sorts the sliced values (go, (xr) ) (x 1 and calculates or approximates the corresponding Monge couplings according to (10) or II respectively. Finally, PSWE calculates the per-slice embedding v." and returns ((x)) = 1 ..... v E x.. This procedure depicted in Figure [ as well as Algorithm Note that in proposed framework, the slicer parameters and the reference set elements all trainable parameters that are updated using backpropagation of gradients due to the objective function of interest.
and {ga,
5 Experimental Evaluation
We evaluate the proposed PSWE method variety of point cloud, graph, and image datasets as depicted Figure For comparison, we consider four different pooling methods: Global average pooling (GAP), global max pooling (MAX-evaluate on image classification only), Pooling by

---

Algorithm Pooling by Sliced Wasserstein Embedding
procedure PSWE(X, = {ar E )==))
Trainable parameters: Slicer parameters O E x. Reference elements Xg R Nxd
for 1orl=1
1=10 to L do
Calculate ge, (X,) == (go, (=)))== and go, (Xo) {90, [=)))==1
Calculate n; argsort(go, (X,)), m0 = argsort(go, (Xo)), and =u
N; N then
Calculate v according to (T0)
else
Calculate oht according to
return ((x)) [0? 04 E R X
(c)
Figure 2: We evaluate the performance of PSWE and other baseline pooling methods (a) 3D point
cloud classification from ModelNer40 dataset [44], (b) TUD graph classification datasets [45]. and
(c) image recognition NWPU-RESISCA, 46 and Places-Extra69 (77 datasets.
Multi-head Attention (PMA) 14] and Featurewise Sort Pooling (FSPool) [1]. In all the PSWE experiments, to ease the optimization process of reference elements, we optimize the reference clements at the output of the slicers rather than in the input space of the slicers. Morcover, for both PMA and PSWE, to reduce the embedding size, we use a similar weighting approach to that of FSPool, where the output embedding (which is of size N L and N x d for PSWE and PMA, respectively) is multiplied elementwise by a learnable weight matrix W of the same size, and the result is summed over the rows to derive final L-dim and d-dim embedding for PSWE and PMA, respectively. Further details on the experiments can be found in the Supplementary Material.
5.1 Point Cloud Processing
We consider the ModelNet40 dataset [44. consisting of 3-dimensional point clouds derived from triangular meshes of 12.31 CAD models belonging to 40 object categories. We sample 1024 points uniformly at random from each object as in [8] [9] and use the official split, with 9,843 training samples and 2,468 test samples. We consider two different backbones, namely multi-layer perceptron (MLP) and induced set attention block (ISAB) from the Set Transformer architecture [14]
Table shows the accuracy achieved by the proposed PSWE method using different numbers of slices (7. E (1,4, 4, 16, 64, 256, 1024]) and the baseline pooling methods GAP, PMA, and FSPool. As the table shows, for both backbone types, PSWE is able outperform other pooling methods for high-enough numbers of slices, Furthermore, it is noteworthy that while mean-pon mean-pooling does not perform well when using MLP backbone, performing message passing among the set elements using the ISAB backbone significantly boosts GAP's performance, suggesting that simple averaging of the per-element embeddings suffices to achieve a high performance level. This implies that there an inherent trade-off between the backbone and pooling complexity, and to maintain a high accuracy level, at least one
two components should complex enough. comparison the wall-clock training and testing times of PSWE and the baseline pooling methods on the ModelNer40 dataset, as well as experimental results on visualizing the closest and farthest samples to/from the trained reference for PSWE, can be found in the Supplementary Material.
7

---

PSWE
Backbone
PMA
MLP
57.8 + 0.5
86.6
0.6
8588 +
149 10
$2.9 2.1
774 =
0.4
83.9 + 0.6
86.3 0.5
86.9 1: 0.3
ISAB
86.6
87.3
324 144-34
86.2
86.9 4
0.4
Table I: Test classification accuracy (%) of the proposed PSWE method and the baseline pooling mechanisms on the ModelNet40 point cloud dataset using multi-layer perceptron (MLP) and induced set attention block (ISAB) [14] backbones.
Backbome
GAP
PMA
FSPool
L
L 16
L 64
206
1024
IMDB- M IMDB BB
GCN
69.5 d: 3.9
74.1 +: 5.3
73.5 3.7
72,6 i: 7.6
74.6 d: 5.8
17.3 d: 5.1
73.0 à:
72.7 +:
73.5 4: 6.1
73.4 = 3,5
+
72.4 6.9
71.3
74.0 +
70.9 +: 7.6
73.0 366
73.4 +
GIN
73,0 1:
70.0 8.0
73,4 6.1
73.8
72.5 1. 6.0
72.0 d:
72.3 4:
74.6 4: 7.0
68.8 1:
GCN
51.8 =: 4.2
50.1 + 2.8
51.1 =:: 3.4
44.2 4.9
90.7 + 4.6
30,8 =: 3.8
49.6 + 3.6
$1.4 4.2
50.2 1: 4.7
50.2 +
GIN
49.7 = 2.9
50. 3.0
50.8 5.3
44.6 2 4.4
49.1 + 2.7
48.0 + 6.4
50.6 + 2.6
$0.2 3.4
49.5 +
GCN
81.9 = 2.6
X2.2 + 2.5
84.0 2.8
77.5 4.5
$0.4 + 3.3
81.5 = 2.1
82.1 + 3.2
81.9 - 3.2
81.7 + 3.0
RDT
:3.0
GIN
81.2 d: 3.1
77.6 7.9
84.4 = 2.9
83.2 5.3
83.1 + 2.8
83.8 #: 2.8
84.6 +: 2.3
K3.9 -
83.7 + 1.7
ENZYMES PROTEINS
GCN
69.1 1 5.2
72.4 + 5.9
74.9 5.4
72.5 +: 3.9
73.3 1:: 5.3
73.3 ::
73.2 +: 6.1
72.8 1 6.0
73.9 1:
GAT
69.7 4.4
72.4 + 6.1
5.1
72.8
73.9 +: 4.6
74.4 + 4.4
13.7
GIN
69.8 =:: 6.5
72.3 3333-:44
72.6 22.6:2244 4.5
71.3 1: 4.0
72.4 + 6.0
73.4 - 4.8
73.5 +
73.0 0: 5.0
74.9 +: 3.9
-:
$2.1
+
31.8 -
32.5 +: 3.0
4.9
33.7 +
GAT
24.2 1:
28.8 1: 3.9
34.2
22.3 4.0
26.3 5.56
30.6 1:
34.6 03::::3
38.1 :1 5.5
34,9 1:
GIN
29.6 do 6.3
50.1 4:488
43.6 de 6.1
19.1 5.5
25.9 +
4.6
36.5 + 3.1
37.2 5.7
45.4 + 7.0
40.0 +: 6.0
Table 2: Cross-validation accuracy (%) of PSWE with different numbers of slices, as well as baseline pooling methods on different TUD graph classification tasks (45) using three backbones of GCN (17) GAT [18, and GIN [19]. The best performing pooling method in each row (i.e., (dataset, backbone) pair) is highlighted in bold.
5.2 Graph Classification
Next, we consider the prominent TUD benchmark 455 and evaluate the performance of the proposed method on five graph classification datasets, consisting of social network (IMDB-B, IMDB-M. REDDIT-B) and bio-informatics (ENZYMES, PROTEINS) datasets. For the former group of datasets, we use one-hot encoded degrees as initial node features, while for the latter group, we use the provided node labels as initial node features. We then pass the features, alongside the adjacency matrices, three popular graph neural network (GNN) backbones, namely Graph Convolutional Network (GCN) 17,, Graph Attention Network (GAT) [8]. and Graph Isomorphism Network (GIN) 19. Upon deriving the final node embeddings we treat them elements of a set and apply PSWE and the baseline pooling methods to derive fixed-size graph-level representation that is fed to a linear classifier.
given graph from GNN backbone,
Table 2 shows the resulting 10-fold cross-validation accuracies on different datasets, and using different backbone/pooling pairings, following the evaluation methodology used in the literature [19
50 [1]. As the table demonstrates, PSWE is able to perform similarly to or better than other pooling
methods on all datasets. Furthermore, the results show that the commonly used mean-pooling for
GNNs might not be the best choice, and more complex backbones might be needed to enhance the
classification performance. is important to note that in some scenarios, especially with smaller datasets (e.g., IMDB-B / ENZYMES) and more complex backbones (e.g., GIN), the performance of PSWE does not monotonically improve with the number of slices, L., which due the model becoming more complex and overfitting the training data.

---

Dataset
Backbone
MAX
GAP
FSPool
PSWE
16 Patches + MLP
71.0 +:
65.0
77.2
75.0
NWPU-RESISC45
ResNet18
89.3 + 0.5
91.4
90.7 0.3
90.6 0.5
16 16 Patches MLP
18.5 +: 2.2
29.1 + 0.1
35.4 1 1.8
35.2 0.2
Places-Extra69
ResNet18
57.3 + 0.1
58.6 0.0
57.3 + 1.8
58.3 + 0.0
Table 3: Image classification results (% test accuracy) the NWPU-RESISC45 and Places-Extra69 datasets using two backbone types coupled with MAX, GAP, FSPool, and PSWE pooling methods.
5.3 Image Recognition
Finally, we evaluate PSWE in the context of image recognition on two large-scale image datasets:
NWPU-RESISC45 [46,, which is remote sensing image scene classification dataset comprising a total of 31,500 images belonging to 45 different aerial scene classes, and Places-Extra69 [47]. which contains 98,721 training and 6,600 test images, belonging to 69 different scene categories. For processing the images, we consider two different backbone types:
16 16 Patches MLP: Inspired by the architecture used in the Vision Transformer (ViT) framework [52], we break the image into 256 patches, each flattened into a 16x 16x3 = 768- dimensional vector, pass each patch through shared multi-layer perceptron (MLP), add positional encoding to the MLP outputs, and treat the outputs as a set of 256 elements, each with 256 features.
ResNet18 [53] As an alternative, we pass the image through ResNet18, which is convolu- tional neural network backbone, mapping the input image into a 7 -i2-dimensional tensor. We treat this tensor set of 49 elements, each containing 512 features.
Table shows the test classification accuracy of PSWE, as compared to GAP, MAX, and FSPool on the two datasets using the two aforementioned backbones. For PSWE, we set the number of slices to T. 1024 for the 16 x 16 Patches + MI.P backhone, and T. = 1000 for the ResNet18 backbone. We did not include PMA results here as it performed significantly worse than other pooling types. As the table shows, PSWE performs on par with FSPool using both backbones, and significantly better than GAP with the simpler MLP-based backbone. This is consistent with our observation that more sophisticated pooling mechanisms can compensate the performance drop caused by simpler backbone architectures.
Conclusion
We introduced a novel method for permutation-invariant feature aggregation from set-structured data, called pooling sliced-Wassersser Wasserstein embedding (PSWE). Our method treats the elements of each input set as samples from a distribution, and derives a constant-size representation for the entire set based on the (generalized) sliced-Waa Wasserstein distance between the set elements and a reference set, whose elements are learned in an end-to-end fashion, alongside with the slicer parameters. We showed for
that our method derives an exact Euclidean embedding which is coometrically-interpeeaaa structured data. Moreover, we demonstrated, through experimental results, that our set embedding approach outperforms baseline pooling mechanisms on variety of supervised classification tasks on point cloud, graph, and image datasets. While our focus in this work was on deriving global representations for input samples (such as point clouds, graphs, and images), our method is not necessarily limited to global pooling. Indeed, our approach is generic mechanism for embedding an input set into a fixed-dimensional representation and, therefore, may also used for local pooling, which is an interesting direction for future work.

---

Acknowledgments and Disclosure of Funding
This material is based upon work supported by the United States Air Force under Contract No. FA8750-19 19-0--99-- Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the United States Air Force and DARPA.
References
[1] Tony Jebara, Risi Kondor, and Andrew Howard. Probability product kernels. The Journal of Machine Learning Research, 5:819-844, 2004.
[2] Arthur Gretton, Karsten Borgwardt, Malte Rasch, Berhhard Schölkopf, and Alex Smola. A kernel method for the two-sample-ae prooble. Advances in neural information processing systems, 19:513 520, 2006.
[3] Oren Boiman, Eli Shechtman, and Michal Trani. In defense of nearest-neighhor based image classification. In 2008 IEEE Conference on Computer Vision and Puttern Recognition, pages 1-8, IEEE, 2008.
Barnabás Póczos, Liang Xiong, and Jeff Schneider. Nonparametric divergence estimation with applications to machine learning on distributions. Proceedings of the Twenty-Sevent Conference on Uncertainty in Artificial Intelligence, pages 599-608, 2011.
[5] Barnaáss Póczos and Jeff Schncider. Nonparametric estimation of conditional information and divergences. In Artificial Intelligence and Statistics, pages 914 923. PMLR, 2012.
[6] Krikamol Muandet, Kenji Fukumizu, Francesco Dinuzzo, and Berhhard Schölkopf. Learning from distributions via support measure machines, In Proceedings of the 25th International Conference on Neural Information Processing Systems- Volume 1, pages 10-18, 2012.
[7] Liang Xiong and Jeff Schneider. Learning from point sets with observational bias. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, pages 898 906, 2014.
[8] Soheil Kolouri, Yang Zou, and Gustavo K Rohde. Sliced- Wasserstein kernels for probability distributions. In Proceedings the IEEE Conference on Computer Vision and Pattern Recognition, pages 4876-4884, 2016.
[9] Matteo Togninalli, M Elisahetta Ghisu, Felipe Llinares-Lépez, Bastian Rieck, and Karsten M Borgwardt. Wasserstein weisfeiler-lehman graph kernels. NeurlPS, 2019.
[10] Konstantinos Skianis, Giannis Nikolentzos, Stratis Limnios, and Michalis Vazirgiannis. Rep the set: Neural networks for learning set representations. In International conference on artificial intelligence and statistics, pages 1410-1420. PMLR, 2020.
[11] Manzil Zaheer, Satwik Kottur, Siamak Ravanbhakhsh. Barnabás Póczos. Ruslan Salakhutdinov. and Alexander J Smola. Deep sets. In Proceedings of the 31 International Conference on Neural Information Processing Systems, pages 340444444, 2017.
[12] Grégoire Mialon, Dexiong Chen, Alexandre d' Aspremont, and Julien Mairal. A trainable optimal transport embedding for feature aggregation and its relationship attention. In International Conference Learning Representations, 2021.
[13] Ryan Murphy, Balasubramaniam Srinivasan. Vinayak Rao, and Bruno Ribeiro. Janossy pooling: Learning deep permutation invariant functions variable-size inputs. In International Conference on Learning Representations, 2019.
[14] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, Whye Teh. Set transformer: sermutation-invarian neural networks. In International Conference on
A framework for attention Machine Learning, pages 3744-3753, PMLR, 2019.
[15] Yan Zhang, Jonathon Hare, and Adam Prügel-Bennett. Fspool: Learning set representations with feature- wise sort pooling, In International Conference on Learning Representations, 2020.
[16] Soheil Kolouri, Navid Naderializadeh, Gustavo K. Rohde, and Heiko Hoffmann. Wasserstein embodding for graph learning, In International Conference on Learning Representations, 2021.
[17] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference Learning Representations (ICLR), 2017.
10

---

[18] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. International Conference on Learning Representations, 2018.
[19] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations. 2019.
[20] Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. From word embeddings document distances. In International conference on machine learning, pages 957-966. PMLR, 2015.
[21] Gao Huang, Chuan Guo, Matt J Kusner, Sun, Fei Sha, and Kilian Q Weinherger. Supervised word mover's distance. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29, pages 1962--870 Curran Associates, Inc., 2016.
[22] Chi Zhang, Yujun Cai. Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classification with differentiable earth mover's distance and structured classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12203-12213 2020.
[23] Jincheng Li, Jiezhang Cao, Shuhai Zhang, Yanwu Xu, Jian Chen, and Mingkui Tan. Internal wasserstein distance for adversarial attack and defense, arXiv preprint arXiv:2/03.0759, 2021.
[24] Mauricio Delbracio, Hossein Talebi, and Peyman Milanfar. Projected distribution loss for image enhance- ment. arXiv preprint 20XV:22012.09289, 2020.
[25] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamoniy. Optimal transport for domain adaptation. IEEE transactions on pattern analysis and machine intelligence, 19(9):1853-1888 2017.
[26] Bharath Bhushan Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, and Nicolas Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European Conference Computer Vision (ECCV), pages 447-463, 2018
[27] David Alvarez Melis and Nicolo Fusi. Geometric dataset distances via optimal transport. Advances Neural Information Processing Systems, 33, 2020.
[28] Yan Zhang, Jonathon Hare, and Adam Prügel-Benneth Learning representations sets through optimized permutations. In International Conference on Learning Representations, 2019.
[29] Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: Sequence to sequence for sets, arXiv preprint X5v
1..3911 1,0391,, 2015.
[30] S Hamid Rezatofighi, Roman Kaskman, Farbod T Motlagh, Qinfeng Shi, Daniel Cremers, Laura Leal- Taixé, and Ian Reid. Deep perm-set net: Learn to predict sets with unknown permutation and cardinality using deep neural networks. arXiv preprint ftti.:v80000000 2018.
[31] Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, and Joao Carreira. Perceiver: General perception with iterative attention. arXiv preprint arXiv:2103.03206. 2021.
[32] Tricu Trinh, Minh Minh-Thang Luong, and Quoc Le. Selfie: supe-supe pretraining for image embedding, arXiv preprint arXiv:1906.0244 2019.
[33] Cédrie Villani. Optimal transport: old and new, volume 338. Springer Science Business Media, 2008
[34] Yann Brenier. Polar factorization and monotone rearrangement vector-valued functions. Communications on pure and applied mathematics, 44(4):375-417, 1991
[35] Soheil Kolouri, Se Rim Park, Matthew Thorpe, Dejan Slepcev, and Gustavo K Rohde. Optimal mass transport: Signal processing and machine-learning applications. IEEE Signal Processing Magazine, 34(4):43-59, 2017.
[36] Julien Rabin, Gabriel Peyré, Julie Delon, and Mare Bernot. Wasserstein barycenter and its application texture mixing. In International Conference on Scale Space and Variational Methods in Computer Vision, pages 435-446. Springer, 2011.
[37] Nicolas Bonnotte, Unidimensional and evolution methods for optimal transportation. PhD thesis, Université Paris 11, France, 2013.
[38] Ishan Deshpande, Yuan- Yuan-Ting Hu, Ruoyu Sun, Ayis Pyrros, Nasir Siddiqui, Sanmi Koyejo, Zhizhen Zhao, David Forsyth, and Alexander Schwing. Max-sliced wasserstein distance and its use for gans. In IEEE Conference Computer Vision and Pattern Recognition, 2019.
11

---

[39] Julien Rabin, Gabriel Peyré, Julie Delon, and Mare Bernot. Wasserstein barycenter and its application to texture mixing. In Scale Space and Variational Methods in Computer Vision, pages 435-446. Springer. 2012.
[40] Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, and Gustavo Rohde. Generalized sliced wasserstein distances. Advances in Neural Information Processing Systems, pages 261-272. 2019.
[41] François-Pierre Paty and Marco Cuturi. Subspace robust wasserstein distances. In International Conference on Machine Learning, 2019.
[42] Khai Nguyen, Nhat Ho. Tung Pham, and Hung Bui. Distributional sliced-wasserstiin and applications generative modeling. International Conference on Learning Reprexentations, 2021.
[43] Se Rim Park, Soheil Kolouri, Shinjini Kundu, and Gustavo K Rohde. The cumulative distribution transform and linear pattern classification. Applied and Computational Harmonic Analysis, 45(3);616 641, 2018.
[44] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang. and Jianxiong Xiao. shapenets: A deep representation volumetric shapes. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912-1920, 2015.
[45] Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop on Gruph Representation Learning and Beyond (GRL+ 2020). 2020.
[46] Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, (85 1883, 2017.
[47] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
[48] Charles R Qi. Hao Su. Kaichun Mo. and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recogmition, pages 652-660 660. 2017.
[49] Meng- Hao Guo, Jun- Xiong Cai, Zheng-Ning Liu, Tai- Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pet: Point cloud transformer. arXiv preprint arXiv:2012.00 096888, 2020.
[50] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1365-1374, 2015.
[51] Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. International conference on machine learning, pages 2014-2023 PMLR, 2016.
[52] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[53] Kaiming He, Xiangyu Zhang, Shaoqing Rea, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
12


Reviews:
[{"review_id": "HVRWv_O8gxn", "reviewer": "Reviewer_wTMH", "summary": "This paper presents a new pooling operation that is based on computing the sliced-wasserstein distance between empirical distributions of input samples. Specific modifications to the conventional sliced-wasserstein distance by the authors include the introduction of the trainable slicer and the trainable reference sample. Experiments on point cloud classification, graph classification, and image classification show improvements when a large number of learnable slicers are used.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n* Paper is relatively clear written and easy to follow. The content is pretty much self-contained with sufficient background information about wasserstein distance in Section 3.\n* A straightforward modification of the sliced-wasserstein distance by introducing two trainable components: trainable slicers and trainable reference sets.\n* The problem being addressed here is important and has many practical applications.\n\nCons:\n* Motivation of the introduced modifications is not clear. In line 58, the authors motivate the proposed changes by claiming that “PSWE provides an exact Euclidean embedding for the (generalized) sliced-Wasserstein (SW) distance”. However, without the two proposed modifications, the conventional generalized sliced-Wasserstein distance also provides Euclidean embedding during distance computation.\n* Section 4.3 “On projection complexity of sliced distances” is not clear in its current form. The authors claimed that samples often live on a low-dimensional manifold and therefore a large number of projections L should be used (line 191). However the exact projection complexity (compute/memory) analysis is not presented.\n* Also in Section 4.3, experiments with different numbers of projections have been conducted in [A1]. What is the main message the authors try to convey here? Are trainable slicers more efficient than random projection slicers? From my point of view, random projection in practice is a very strong baseline and it is quite critical to compare the proposed trainable slicer and random projection slicer in terms of classification accuracy and computational cost.\n* [A2] is one of the early works that proposed the concept of sliced-wasserstein distance and should be discussed.\n\n[A1] Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation. CVPR 2019\\\n[A2] Wasserstein barycenter and its application to texture mixing. SSVM, 2011.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "An immediate baseline would be replacing the trainable slicers with random projections and comparing the performance on various classification tasks listed in this work and the corresponding computational overhead.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "3SgOISbrJ4", "reviewer": "Reviewer_ABo1", "summary": "This paper introduces a general geometric-aware pooling function. The key idea is based on sliced Wasserstein distance. The input is transformed to a set of points no matter what the original format is. Then, the Wasserstein distance between this input set and a trainable reference set is computed. The 1 sliced Wasserstein distance can be computed by sorting the input feature in a straightforward fashion. This paper demonstrates the performance on set classification, graph classification, and image classification tasks. In some cases, the proposed method outperforms its counterparts.  ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper introduces a general geometric-aware pooling function. The key idea is based on sliced Wasserstein distance. The input is transformed to a set of points no matter what the original format is. Then, the Wasserstein distance between this input set and a trainable reference set is computed. The sliced Wasserstein distance can be computed by sorting the input feature in a straightforward fashion. This paper demonstrates the performance on set classification, graph classification, and image classification tasks. In some cases, the proposed method outperforms its counterparts.  \n\nPros:\n1. This paper presents a great motivation to design a geometric-aware pooling function. Compared to others, this method treats the input features as  a set of samples from a probability distribution. This novel interpretation makes it possible to incorporate tools in geometric optimization. \n2. The idea of using a sliced Wasserstein distance is technically sound. In the 1D case, the Wasserstein distance can be easily computed by sorting. So this significantly reduces the complexity of Wasserstein distance computation. \n3. This paper introduces the idea of using a trainable set, which can be viewed as  a dataset prior. \n\nCons:\n\n1. The proposed method seems only applicable to the global level and it is not able to replace the a general pooling (e.g., strided average pooling) with the proposed method. As a consequence, this pooling module can only work on classification tasks. \n2. My another concern is the practical value of the proposed method. Although this method has a good motivation, it’s still quite challenging to incorporate it into a general deep learning architecture and achieve comparable performance to state-of-the-arts. \n3. In the point cloud classification case, the performance of the proposed method is much lower than existing point-based methods. E.g., PointNet without the spatial transformer can achieve more than 87% on this dataset. In addition, ModelNet40 is a small scale dataset, so testing with the SOTA encoders (DGCNN or KPConv) will be more convincing. \n4. In the graph classification case, the proposed method does not show consistent improvements over its counterparts, either. \n5. Similarly, in the image classification case, the method is only comparable to FSPool. Also, I am confused why the method does not evaluate on the most common datasets such as CIFAR and/or ImangeNet. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See above.", "review_text": "", "extra_scores": {"ethical_concerns": "I don't have ethical concern on this paper. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "0_NBnXRBIb-", "reviewer": "Reviewer_qqRn", "summary": "This is an extremely well-written paper on a topic well-suited for presentation at NeurIPS. The material is highly technical and the authors do an excellent job at providing a clear exposition and necessary background material (see below for minor caveats). While empirical results are not compelling across all experiments, the results do show promise in this line of research, especially given its strong theoretical underpinnings.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I have very little to add beyond the summary above. Some small questions for the authors:\n\n1. Can the sets $X_i$ be multisets, which would give rise to different empirical distributions (i.e., weighted delta functions) and how would this affect the method?\n2. Why use ${\\cal X}$ and ${\\cal X'}$ in Eqn (1) instead of ${\\cal X}_i$ and ${\\cal X}_j$?\n3. Are the regularity conditions presented in [39] (L.131) satisfied when the inputs are learned from a neural network?\n4. Exactly what is learned in the set $X_0$ (L.171)? My understanding is that it's the $x_n^0$ that are learned/perturbed during learning, so it's unclear why the underlying distribution $p_0$ is needed. These points are never resampled.\n5. The paper discusses computational complexity in terms of computing PSWE. Can the authors provide running times as a function of L for the experiments?\n \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are discussed briefly throughout the paper but there is no single place where they are summarized. There is no discussion of societal impact and I don't see any negative consequences of this work to society.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers have strong disagreements about this paper. After discussions, I concur with several
points adressed by the reviewers. Most importantly, we believe that the experimental studies need
to be made stronger, in term of comparison with SOTA methods (even if they do not reach SOTA), 
potential way for local pooling... 
In addition, I would like to mention the missing stronger discussion with the paper of Mialon 2021 et al.
on "trainable OT embedding ... ". Indeed, From my point of view, the proposed method is just a variant of 
the one of Mialon where the Wasserstein distance is replaced by a 1D Wasserstein and you consider multiple references
where one reference is obtained from each slicing (multiple references  is described in Section 3.5 in Mialon et al). 
An averaging over the row would give an embedding similar to those of Mialon. Main difference is that the input distribution is 
somehow pertubed as it is also projected through the slicing. Hence, I would have expected a comparison also
with this method.
Decision: Accept (Poster)