Revisiting Sliced Wasserstein on Images: From Vectorization to Convolution
Authors: Khai Nguyen
Nhat Ho
Venue: NeurIPS 2022 Accept (2022)
Keywords: Sliced Wasserstein
Optimal Transport
Generative Models
Convolutional Operators

Abstract: The conventional sliced Wasserstein is defined between two probability measures that have realizations as \textit{vectors}. When comparing two probability measures over images, practitioners first need to vectorize images and then project them to one-dimensional space by using matrix multiplication between the sample matrix and the projection matrix. After that, the sliced Wasserstein is evaluated by averaging the two corresponding one-dimensional projected probability measures. However, this approach has two limitations. The first limitation is that the spatial structure of images is not captured efficiently by the vectorization step; therefore, the later slicing process becomes harder to gather the discrepancy information. The second limitation is memory inefficiency since each slicing direction is a vector that has the same dimension as the images. To address these limitations, we propose novel slicing methods for sliced Wasserstein between probability measures over images that are based on the convolution operators. We derive \emph{convolution sliced Wasserstein} (CSW) and its variants via incorporating stride, dilation, and non-linear activation function into the convolution operators. We investigate the metricity of CSW as well as its sample complexity, its computational complexity, and its connection to conventional sliced Wasserstein distances. Finally, we demonstrate the favorable performance of CSW over the conventional sliced Wasserstein in comparing probability measures over images and in training deep generative modeling on images.

Revisiting Sliced Wasserstein on Images: From Vectorization to Convolution
Khai Nguyen
Nhat Ho
Department of Statistics and Data Sciences
Department of Statistics and Data Sciences The University of Texas at Austin Austin, TX 78712 minhnhat@utexas. edu
The University of Texas Austin Austin, TX 78712
khainb@utexas.ert
Abstract
The conventional sliced Wasserstein defined between two probability measures that have realizations as vectors. When comparing two probability measures over images, practitioners first need to vectorize images and then project them to one-dimensional space by using matrix multiplication between the sample matrix and the projection matrix. After that, the sliced Wasserstein is evaluated by averaging the two corresponding one-dimensional projected probability measures. However, this approach has two limitations. The first limitation is that the spatial structure of images is not captured efficiently by the vectorization step; therefore, the later slicing process becomes harder to gather the discrepancy information. The second limitation is memory inefficiency since each slicing direction is vector that has the same dimension as the images. To address these limitations, we propose novel slicing methods for sliced Wasserstein between probability measures over images that are based on the convolution operators. We derive convolution sliced Wasserstein (CSW) and its variants via incorporating stride, dilation, and non-linear activation function into the convolution operators. We investigate the metricity of CSW as well its sample complexity, computational complexity, and its connection to conventional sliced Wasserstein distances. Finally, we demonstrate the favorable performance of CSW over the conventional sliced Wasserstein in comparing probability measures over images and in training deep generative modeling on images
Introduction
Optimal transport and Wasserstein distance 559 51 have become popular tools in machine learning and data science. For example, optimal transport has been utilized generative modeling tasks to generate realistic images ?
53 in domain adaptation applications to transfer knowledge from in clustering applications to capture the heterogeneity of data (21). Despite having appealing performance, Wasserstein distance has been known to suffer from high computational complexity, namely, its computational complexity is at the order of O(m² log m))
source to target domains
3
and in other applications
PV
62
63
when the probability measures have at most supports. In addition, Wasserstein distance also suffers from the curse of dimensionality. namely, its sample complexity is at the order of O(n -1/d) [S where n is the sample size. A popular line of work to improve the speed of computation and the sample complexity of the Wasserstein distance is by adding an entropic regularization term to the Wasserstein distance [[.. This variant known as entropic regularized optimal transport (or equivalently entropic regularized Wasserstein). By using the entropic version, we can approximate the value of Wasserstein distance with the computational complexities
49
Cooe for the paper is published at wttppp///ggthhub.coo//tttiiiiiii Sacceaa--eueataa---a-aaeeeeeeee -Scinnc--rrrerp/SSSS
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

being at the order of (m²)) Furthermore, the sample complexity of the entropic version had also been shown to be at the order of which indicates that it does not suffer from the curse of dimensionality.
I B51
RG
(up to some polynomial orders of approximation errors).
O(n-1/2) 1/2)
B
Another useful line of work to improve both the computational and sample complexities of the Wasserstein distance is based on the closed-form solution of optimal transport in one dimension. A notable distance along this direction is sliced Wasserstein (SW) distance [6]. Due to the fast compu- tational complexity (m log2 m) and no curse of dimensionality (1-11/), the sliced Wasserstein has been applied successfully in several applications, such generative modeling 61 1 25 47
domain adaptation [II and clustering [26]. The sliced Wasserstein is defined between two prob- ability measures that have supports belonging to vector space, e.g. F.. As defined in K6 the sliced Wasserstein is written as the expectation of one-dimensional Wasserstein distance between two projected measures over the uniform distribution on the unit sphere. Due to the intractability of the expectation, Monte Carlo samples from the uniform distribution over the unit sphere are used to approximate the sliced Wasserstein distance. The number of samples is often called the number of projections and it is denoted as L. On the computational side, the computation of sliced Wasserstein can be decomposed into two steps. In the first step, L projecting directions are first sampled and then stacked as a matrix (the projection matrix). After that, the projection matrix is multiplied by the two data matrices resulting in two matrices that represent L one-dimensional projected probability measures. In the second step, L one-dimensional Wasserstein distances are computed between the two corresponding projected measures with the same projecting direction. Finally, the average of those distances is yielded as the value of the sliced Wasserstein.
Despite being applied widely in tasks that deal with probability measures over images 61 [3], the conventional formulation of sliced Wasserstein is not well-defined to the nature of images. In particular, an image is not vector but is tensor. Therefore, probability measure over images should be defined over the space of tensors instead of vectors. The conventional formulation leads to an extra step in using the sliced Wasserstein on the domain of images which is vectorization. Namely, all images (supports of two probability measures) are transformed into vectors by deterministic one-one mapping which is the "reshape" operator. This extra step does not keep the spatial structures of the supports, which are crucial information of images. Furthermore, the vectorization step also poses certain challenges to design efficient ways of projecting (slicing) samples to one dimension based on prior knowledge about the domain of samples. Finally, prior empirical investigations indicate that there are several slices in the conventional Wasserstein collapsing the two probability measures to the Dirac Delta at zero [13 12 24 Therefore, these slices do not contribute to the overall discrepancy. These works suggest that the space of projecting directions in the conventional sliced Wasserstein (the unit hyper-sphere) is potentially not optimal, at least for images.
Contribution. To address these issues of the sliced Wasserstein over images, we propose to replace the conventional formulation of the sliced Wasserstein with a new formulation that is defined on the space of probability measures over tensors. Moreover, we also propose novel slicing process by changing the conventional matrix multiplication to the convolution operators 16 18 In summary, our main contributions are two-fold:
1. We leverage the benefits of the convolution operators on images, including their efficient parameter sharing and memory saving as well their superior performance in several tasks on images 28 19). to introduce efficient slicing methods on sliced Wasserstein, named convolution slicers. With those slicers, we derive a novel variant of sliced Wasserstein, named convolution sliced Wasserstein (CSW). We investigate the metricity of CSW, its sample and computational complexities, and its connection to other variants of SW.
2. We illustrate the favorable performance of CSW in comparing probability measures over images. In particular, we show that CSW provides an almost identical discrepancy between MNIST's digits compared to that of the SW while having much less slicing memory. Furthermore, we compare SW and CSW in training deep generative models on standard benchmark image datasets, including C0, CclebA, STL10, and CelebA-HO By considering the quality of the trained models, training speed, and training memory of CSW and SW. we observe that CSW has more favorable performance than the vanilla SW.
Organization. The remainder of the paper is organized as follows. We first provide background about Wasserstein distance, the conventional slicing process in the sliced Wasserstein distance, and the convolution operator in Section 2 In Section 3 we propose the convolution slicing and the

---

convolution sliced Wasserstein, and analyze some of its theoretical properties. Section 4 contains the application of CSW to generative models, qualitative experimental results, and quantitative experimental results on standard benchmarks. We conclude the paper In Section S the proofs of key results and extra materials in the Appendices.
Finally, we defer
For any d 2 2, g == {0 E d 1 } denotes the d dimensional unit hyper-sphere in C2 norm, and (() is the uniform measure over s Moreover, denotes the Dirac delta function. For p 2 1, (TR is the set of all probability measures on IRd that have finite p-moments. For u.v E (R [1,, v) == {iT E P x Rd)) fg= (ax, y)dr = v, Jge x (2z y)dy = μ* the set of transportation plans between y and v. For m 2 1, we denotes y as the product measure which has the supports are the joint vector of m random variables that follows μr. For vector X E RRd X == (f1,...,Tml) Px denotes the empirical measures 1 - 6zg- For any two sequences an and bn. the notation an Ob,,) means that On S Cb, for all where C is some universal constant.
Notation.
2 Background
In this section, we first review the definitions of the Wasserstein distance, the conventional slicing, and the sliced Wasserstein distance, and discuss its limitation. We then review the convolution and the padding operators on images.
Sliced Wasserstein: For any p 2 and dimension d' 2 1, we first define the Wasserstein-p distance [39 150 between two probability measures μ E P Pp (RR) and E Pp(R)), ) which is given
When d' 1, the Wasserstein distance has closed form which is Wp(u, = ( |F. (2) F. (z) Pdz) where Fp. and F, are the cumulative distribution function (CDF) of j and respectively.
by W (p p) (infzen(an)) Jae' x ||0- =1l (I, v)
Given this closed-fome form property of Wasserstein distance in one dimension, the sliced Wasser- stein distance (6] between μ. and had been introduced and admitted the following formulation: SW2 D) = Jga-1 (f 02v)d0, where 0in is the push-forward probability measure of through the funetion Tg : Rdd + R with (x 0T... For each 0 E g W2,(0u,, 0tv) can be computed in linear time Om log, m) where m the number of supports of μ and v. However, the integration over the unit sphere in the sliced Wasserstein distance is intractable to compute. Therefore, Monte Carlo scheme is employed to approximate the integration. namely, oI..... 02 ~ 21($' are drawn uniformly from the unit sphere and the approximation of the sliced Wasserstein distance P SW (p, v) 2 + Σ W2(0,,tu 0cti). In practice, L should be chosen to be sufficiently
given by: SW large compared to the dimension d', which can be undesirable.
Sliced Wasserstein on Images: Now, focus two probability measures over images: μl, (Pexxxd) for number of channels c 2 and dimension d 2 1. In this case, the sliced Wasserstein between μ and y is defined as:
SW (W,(,, v) = SW, (Riu, R4v),
where R: Rexdxd -> Bee is deterministic one-to-one "reshape" mapping.
The slicing process: The slicing of sliced Wasserstein distance on probability measures over images consists of two steps: vectorization and projection. Suppose that the probability measure E (((Rexdxd) has supports. Then the supports of fı are transformed into vectors in Redd and are stacked as a matrix of size n x ed2. A projection matrix of size cd2 is then sampled and has each column as random vector following the uniform measure over the unit hyper-sphere. Finally, the multiplication of those two matrices returns projected probability measures of n supports in one dimension. We illustrate this process in Figure
Limitation of the conventional slicing: First of all, images contain spatial relations across channels and local information. Therefore, transforming images into vectors makes it challenging to obtain that information. Second, vectorization leads to the usage of projecting directions from the unit hyper-sphere, which can have several directions that do not have good discriminative power: Finally, sampling projecting directions in high-dimension also time-consuming and memory-consumin As consequence, avoiding the vectorization step improve the efficiency of the whole process.
3

---

By 3
Flatten
2200
0 hp
Sampling
0g
0g Ri
Dot
0
U(BaF
0,
Or 31u
conventional slicing process of the sliced Wasserstein
Figure 1: The conventional slicing process of sliced Wasserstein distance. The images X Xn, E Rexddx are first flattened into vectors in uged then the Radon transform applied these vectors to to sliced Wasserstein [ on images.
Convolution operator: We now define the convolution operator on tensors 16 which will be used as an alternative way of projecting images to one dimension in the sliced Wasserstein. The definition of the convolution operator with stride and dilation is as follows.
Definition (Convolution) Given the number of channels c 2 1, the dimension 2 1. the stride size 1, the dilation size b 2 1, the size of kernel k 2 1. the convolution of tensor X E Rexdxd with kernel size K E Rexkxk is X a.b K Y, Y E Rixd'xd xd where d' = d-b(k-1)-1 + 1. For I,...,d' d' and j = 1, I,...,d', Yig. Y1,i. Ei= DE Ly'=0 Xn, +1o0-)) K1.4+1444
is defined as:
From its definition, we can check that the computational complexity the convolution operator is
2 k2 O d-D(--1)-- + 1)
Convolution Sliced Wasserstein
In this section, we will define a convolution slicer that maps a tensor to scalar by convolution operators. Moreover, we discuss the convolution slicer and some of its specific forms including the convolution-base slicer, the convolution-strice slicer, the convoluiion-dilation slicer, and their non-linear extensions. After that, we derive the convolution sliced Wasserstein (CSW), a family of variants of sliced Wasserstein, that utilizes convolution slicer the projecting method. Finally, discuss some theoretical propertics of CSW, namely, its metricity, its computational complexity, its sample complexity, and its connection other variants of sliced Wasserstein.
3.1 Convolution Slicer
We first start with the definition of the convolution slicer, which plays an important role in defining convolution sliced Wasserstein.
Definition 2 (Comvolution Slicer) For N > 1. given sequence of kernels K() e
Te xd(l) xd(1)
(N) E ( xa xa(N) convolution slicer S(:(K()) ....., K(N)
Rexdxd is composition of N convolution functions with kernels K(1) ...., K(N) (with stride or dilation if needed) such that ((X)K(1) (K),...... K(N)) E AX E Rexdxd""
As indicated in Definition 2 the idea of the convolution slicer is to progressively map given data X to a one-dimensional subspace through a sequence of convolution kernels, which capture spatial relations across channels as well as local information of the data. It is starkly different from the vectorization step standard sliced Wasserstein on images T The illustration of the convolution
slicer is given in Figure 2

---

Samplin
Sampling
Sampling
x
X
K
x 19
xy
E ge
x
*
"...,,?") 12
The process comvllaiio slicad Wasserstela
n
Figure 2: The convolution slicing process (using the convolution slicer). The images XI...,, Xn E Rexddx directly mapped scalar sequence of convolution functions which have kernels as random tensors. This slicing process leads to convolution sliced Wasserstein on images.
consider three particular types of convolution slicers based on using linear function on convolution operator, named convolution-base convolution-stride, and convolution-dilation slicers. We defer the definition of corooouttondiilationn slicers to Definition [ of the convolution- base slicer.
We first start with the definition
Definition (Convolution-bus base Slicer) Given X Rexdxd (d 2).
1. When d is even, N is the biggest integer that satisfies d = 2 with a is also an integer, sliced kernels defined as (())
(+)))22 '+1) and K(h) RIx(2 [Ix(2~"++1)xx) (2++))+))))))))) d+
for h 2...., . N and (NN) E R xxax where a = 2 Then, the convolution base slicer CS-b(X)K()) K(NN) is defined as:
[X ((h # k(h) 1 h S,
h=0
CS-b(X|K(), ....., K(N,) X X
111
2. When d is odd, the convolution-base slicer (S-b(X]K()) K(,)) takes the form: CS-b(X]K(), ..... K(N)) CS-b(X *
,..., (..,,K....,KKK
where (1)) E <Rxx2x2 and ((2) ..... K(N) are the corresponding sliced kernels that are defined on the dimension d 1.
The idea of the convolution-ban slicer in Definition is to reduce the width and the height of the image by half after each convolution operator. If the width and the height of the image are odd, the first convolution operator is to reduce the size of the image by one via convolution with kernels of size 2 x 2, and then the same procedure as that of the even case is applied. We would like to remark that the conventional slicing of sliced Wasserstein in Section is equivalent to convolution-ban E Ei K
slicer S (()) where (1) E Rexdxd that satisfies the constraint Σi dd
h,isj
We now discuss the second variant of the convolution slicer, named convolution stride slicer, where we further incorporate stride into the convolution operators. Its definition as follows.
Definition 4 (Convolution-stin stride Slicer) Given X E exxxxd (d 2).
I. When d is even, N is the biggest integer that satisfies d N- with is also integer, sliced kernels are defined as K() E Rex2x2 and (hh) E R1x2x2 for 2...., 1, and KNN E Ilxxax where -w-T. Then, the convolution-stride slicer CS-s K()) K k()) is defined as:
X
h = 0
(S-s(X ,...,
...., ((N)) X(M), x
()-
R(h)
1 S - 1,
(h-1) 1.1 (b))
h N,
5

---

2. When d is odd, the convolution-stricc slicer CS-s(X K(N)) takes the form:
CS-s(X|K() ...... ....,,KNN) CS-s(X 11 k where K() E RRxx2x and K(2), ,.... K(N) , the dimension 1.
..... ......,(())) K(N)),
are the corresponding sliced kernels that are defined on
Similar to the convolution-base slicer in Definition 3 the convolution-stride slicer reduces the width and the height of the image by half after each convolution operator. We use the same procedure of reducing the height and the width of the image by one when the height and the width of the image are odd. The benefit the convolution-strid slicer is that the size its kernels does not depend on slicer. This difference improves
the width and the height of images as that of the convolution- the computational complexity and time complexity of the convolution-stie slicer over those of the convolution-bel base slicer (ef. Proposition T..
Definition 5 (Comvolation-dilation Slicer) Given X E RRxddx (d 2 2),
1. When is even, N is the biggest integer that satisfies d = 2N a with a is also an integer, sliced kernels are defined K(1) Rex2x2 and K(h) R1x2x2 for h शύ--. ,,...,N 1, and K(N) E Rxxaxx where a = TN-T. Then, the convolution-dilation (.MKK),...,...))))
slicer CS-d(X)K() ..... K(N)) is defined as:
X
h
CS-d(X]n[)) ,.... K(N)= x(00 x())_
((h 1,4/2 kth)
1s -chsN-
x(h-1) 111 K()))
h= N,
2. When d is odd, the convolution-dilatin slicer CX- ..... K(N) takes the form:
CS-d(X ,.., K(M) = CS-d(X 1.1 ((12) K(N)) where K()) E Rex2x2 and ((2), defined on the dimension d
KNN) are the corresponding sliced kernels that are
As with the previous slicers, the convolution-dilatti slicer also reduces the width and the height of the image by half after each convolution operator and it uses the same procedure for the odd dimension cases. The design of kernels' size of the convolution-dilatin slicer the same as that of the convolution-stride slicer. However, the convolution-dilaan each convolution operator which might be appealing when the information of the image is presented by big block of pixels.
slicer has bigger receptive field in
Computational and projection memories complexities of the convolution slicers: We now establish the computational and projection memory complexities of convolution-base. convolution- stride, and convolution-dilatinn slicers in the following proposition. We would like to recall that the projection memory complexity is the memory that is needed to store slice (convolution kernels).
Proposition 1 (a) When is even, is the biggest integer that satisfies d = 2 a with a also an integer, and N [logz d). the computational and projection memory complexities of convolution-bass slicer are respectively at the order of (cd')) and (ccd)). When is odd, these complexities are at the order of Ocd² d and O d²).
(b) The computational and projection memory complexities of convolution slicer are respec- tively at the order of O(cd)) and O(c + [log2 d).
(c) The computational and projection memory complexities of convolution convolutionddilation slicer are respec- tively at the order of Occd²) and O(c [log, d)).
Proof of Proposition is Appendix B.4 We recall that the computational complexity and the projection memory complexity of the conventional slicing in sliced Wasserstein are O(cd) and Olod). We can observe that the convolution-bas slicer has a worse computational complexity than the conventional slicing while having the same projection memory complexity. Since the size of

---

kernels does not depend on the size of images, the convolution-stride slicer and the convolution- dilation slicer have the same computational complexity as the conventional slicing ( However, their projection memory complexities are cheaper than conventional slicing, namely, (c + [lgg2 d)) compared to (cd.).
Non-linear convolution-base slicer: The composition of convolution functions in the linear convolution slicer and its linear variants still a linear function, which may not be effective when the data lie in complex and highly non-linear iow-dimensional subspace. A natural generalization of linear convolution slicers to enhance the ability of the slicers to capture the non-linearity of the data is to apply non-linear activation function after convolution operators. This enables us to define a non-lineat slicer in Definition in Appendix [ The non- non-linear slicer can be seen as a defining function in generalized Radon Transform [52] which was used in generalized sliced Wasserstein 24
3.2 Convolution Sliced Wasserstein
Given the definition of convolution slicers, we now state general definition of convolution sliced Wasserstein. An illustration of the convolution sliced Wasserstein is given in Figure 2
Definition 6 For any 2 1, the convolution sliced Wasserstein (CSW) of order p > 0 between two given probability measures μ, v E Pp (R)xdx)) given by:
CSWp(u, = E WB (S(1KK() K(N) Stn, S(IKK)) ..... K( )EV) )1)
where the expectation is laken with respect to K(I)) ~ U(K()), K(N) ~ U(K(N)). Here, S(|K()) ...., ((.., is convolution slicer with K() R xx xk0) for any I [N] and U(K()) is the uniform distribution with the realizations being in the set K0) which is defined as K()) ==
K(O) E Ied xX xx(0) Eh
k (2) h,3 E
= 1). namely, the set 1C(1) consists of tensors
K()) whose squared lg norm 1.
The constraint that & norms K(O) is for guaranteeing the distances between projected supports are bounded. When we specifically consider the convolution slicer as convolution-bal slicer (CS-b), convolution-stride slicer (CS-s), and convolution-dilation slicer (CS-d), we have the corresponding notions of convolution-bial sliced Wasserstein (CSW-b), convolution-stride sliced Wasserstein (CSW-s), and cnvolution-dilation sliced Wasserstein (CSW-d).
Monte Carlo estimation and implementation: Similar to the conventional sliced Wasserstein, the expectation with respect to kernels K')),.... ....,, KN) uniformly drawn from the sets KC(), KC() in the convolution sliced Wasserstein is intractable to compute. Therefore, we also make use of Monte Carlo method to approximate the expectation, which leads to the following approximation of the convolution sliced Wasserstein:
1 CS
L S WP (S (S ,...,
k(N))
~0
(2)
where K/O are uniform samples from the sets k(()) (which is equivalent to sample uni- formly from g000 then applying the one-to-one reshape mapping) for any E [N] and i E [Z]. Since each of the convolution slicer S(-IK!1)) ..... K. k(()) is in one dimension, we can utilize the closed-form expression of Wasserstein metric in one dimension to compute Wp (S(IK(2) ....., k((")) )1,, S(-Ix() ......,!!))))) " )tv)
with a complexity of O(m log, m)) for each
i e [L] where is the maximum number of supports of μ and Therefore, the total computational complexity of computing the Monte Carlo approximation (2) is ( log, m) when the probability measures μ and have at most m supports. It is comparable to the computational complexity of sliced Wasserstein on images h where we directly vectorize the images and apply the Radon transform to these flatten images. Finally, for the implementation, we would like to remark that L convolution slicers in equation @ can be computed independently and parallelly using the group convolution implementation which is supported in almost all libraries.
Properties of convolution sliced Wasserstein: We first have the following result for the metricity of the convolution sliced Wasserstein.
7

---

Theorem For any p 2 1, the convolution sliced Wasserstein cS.,(...) is pseudo-ment mettic on the space of probability measures on Rexdxd, namely, it is symmetric, and satisfies the triangle inequality.
Proof of Theorem T in Appendix We would like mention that CSW can might still be metric since the convolution slicer might be injective. Our next result establishes the connection between the convolution sliced Wasserstein and max-sliced Wasserstein and Wasserstein distances.
Proposition 2 For any 2 1, find that (S,,,, S Max-SW,(M,.u v) S Wp(u, where Max-SW, v) == max 11 Wy 0iv) is max-sliced Wasserstein of order p.
Given the bounds in Proposition 2 we demonstrate that the convolution sliced Wasserstein does not suffer from the curse of dimensionality for the inference purpose, namely, the sample complexity for the empirical distribution from i.i.d. samples to approximate their underlying distribution is at the order O(n 112)).
Proof of Proposition 2 is in Appendix B.2
Proposition 3 Assume that P is a probability measure supported on compact set of Rexdxd, Let X1, X2....... X, be i..dd. samples from P and denote P. = = Σ 8x, the empirical measure of these data. Then, for any Z 1, there exists a universal constant C >> such that
E ICCS, P)] S (cd + log n/n,
where the outer expectation is taken with respect to the data X2,X2,.... X
The result of Proposition 3 indicates that the sample complexity of the convolution sliced Wasserstein is comparable to that of the sliced Wasserstein on 4 and better than that of the Wasserstein metric,
Proof of Proposition 3 Appendix B.3
images which is at the order of (nn
which is at the order of O(
15
Extension to non-linear convolution sliced Wasserstein: Appendix C we provide a non-linear version of the convolution sliced Wasserstein, named non-linear convolution sliced Wasserstein. The high-level idea of the non-linear version is to incorporate non-linear activation functions to the convolution- convolution-stride, and convolution-dilatio slicers. The inclusion of non-linear activation functions is enhance the ability of slicers to capture the non-linearity of the data. By plugging these non-linear convolution slicers into the general definition of the convolution sliced Wasserstein Definition we obtain the non-lines variants of convolution sliced Wasserstein.
Experiments
In this section, we focus on comparing the sliced Wasserstein (SW) (with the conventional slicing), the convolution-base sliced Wasserstein (CSW-b), the convolution sliced Wasserstein with stride (CSW-s), and the convolution sliced Wassersstein with dilation (CSW-d) in training generative models on standard benchmark image datasets such as CIFARIO (32x32) 127. STL10 (96x96) [8) CelebA (64x64), and CelebA-HOQ 128) 37 We recall that the number of projections in SW and CSW's variants is denoted as L. Finally, we also show the values of the SW and the CSW variants between probability measures over digits of the MNIST dataset 60 in Appendix [.I] From experiments on MNIST, we observe that values of CSW variants are similar to values of SW while having better projection complexities.
In generative modeling, we follow the framework of the sliced Wasserstein generator in K with some modifications of neural network architectures. The details of the training are given in Appendix D.2
We train the above model on standard benchmarks such as CIFARIO (32x32) 27. STL10 (96x96) S
CelebA (64x64), and CelebAHQ (128x 128) [37]. To compare models, we use the FID score
and the Inception score (IS) [35 The detailed settings about architectures, hyperparameters, and evaluation of FID and IS are given in Appendix E We first show the FID scores and IS scores of generative models trained by SW and CSW's variants with the number of projections L e {1, 100, 1000) in Table I In the table, we report the performance models the last training epoch. We do not report the IS scores on CelebA and CelebA-HQ since the IS scores are not suitable for face images. We then demonstrate the FID scores and IS scores across training epochs in Figure 3 for investigating the convergence of generative models trained by SW and CSW's variants. After

---

CFARTO
CPAR10
Calend
5th 300
SW-100
Sth 3003
5W 2003
CSNA
CSNMAb-1500
Store
C509-6-1000 0-0000
-e---d-ie $-100
CSW00:100
1100
65e 5-101
CS 5005-1000
800
209
100
SW-3002
5W-2002
C5RAb-100
(509-b-100
CSN--b110
CSW-b- C5W-bb11000
-5RP--300
C59 300
1D Spare
1933
5W-330
CSP 1005
C5NW4-1100
SWI 1000
CSWN-6-100
CS0:b-102
CSW:b-1920
CSW:0-1020
FID scores and IS scores over epochs different training losses on datasets. observe CSW's variants usually help the generative models converge faster.
Figure 3:
Table 1:
Summary of FID and IS scores of methods CIFARIO (32x32), CelebA (64x64), STL10 (96x96),
and CelebA-HO HQ (288.128). Some results CIFARIO reported from different
CIFARTO (32x32)
CelehA (64x64)
STL.0 (96x96)
CelehA-HQ (128x128)
Method
(I)
(f)
FID (3)
FID (L)
(t)
FID (
SW (L=1)
87,97
3,59
128.81
170.96
3.68
275.44
(l=+)((=l)
84,38
4.28
85,83
173.33
3.89
315.91
CSW:s (L=1)
80.10
4.31
66.52
168,93
3.75
303.57
(L=I)
63,94
4,89
89.37
212.61
2.48
321.06
SW (L=100)
52.36:20.76
5.79:1.116
20.08
100.35
8.14
$1,80
CSW-b (L=100)
49.67-22.00
5.87:0.15
18.96
91.75
53.05
CSW-s (L=100)
43.73:2.09
6.17=10.06
13.76
97.08
8.20
32.94
CSW-d (L=100)
1..23=1.12
5.9720011
14.96
102.58
7.53
41.01
SW (L=000000
44.2541.22
6.02-:0.03
14.92
84.78
9.06
28.19
CSW-b (L=1000)
42.88-10.88
6.11100100
14.75
86.98
29.69
CSW-s (1=1000))
16.80+1.44
6.55++0.12
12.55
77.24
9.31
22.25
CSW-d (L=1000)
1.04
6.38:10.14
13.24
83.36
9.42
25.93
that, we report the training time and training memory of SW and CSW variants in Table 5 Finally, show randomly generated images from SW's models and CSW-s' models CelebA dataset in Generated images of all models on all datasets are given in Figures in Appendix D.2
Figure 4
Summary of FID scores and IS scores: According to Table CIFARIO, CSW-d gives the lowest while CSW-s gives the lowest FID scores when == 100 and L 1000. Compared to CSW-s, CSW-d and CSW-b yield higher FID scores and lower IS scores. However, CSW-d and CSW-b are still better than SW. On CelebA, CSW-s performs the best in all settings. On STL10, CSW's variants also better than the vanilla SW; however, it is unclear which is the best variant. On CelebA-HQ, SW gives the lowest FID score when L In contrast, when Since the FID scores L = are very high on CelebA-HQ and STL.10, the scores are not very meaningful for comparing SW and CSW's variants. For all models, increasing L leads better quality. Overall, we observe that CSW's variants enhance the performance of generative models.
values of FID scores and IS scores when
100 and I 1000, CSW-s the best choice for training the generative model.
FID scores and IS scores across epochs: From Figure we observe that CSW's variants help the generative models converge faster than SW when 100 and L 1000. Increasing the number of projections from 100 to 1000, the generative models from both SW and CSW's 's variants become better. Overall, CSW-s is the best option for training generative models among CSW's variants since its FID curves the lowest and its IS curves are the highest.
Generated images: We show randomly generated images on CelebA dataset in Figure 4 and Figure (Appendix D) and generated images CIFARIO, CelebA, STL10, and CelebA-HQ in Figures
N

---

SW (I =
SW (Z 100)
SW (Z 1000)
CSW-s (L 1)
CSW-s (Z = 100)
CSW-s (L = 1000)
Figure 4: Random generated images of SW and CSW-s on CelebA
qualitative comparison between SW and CSW variants. From the figures, can that generated images of CSW-s is more realistic than ones of SW. The difference is visually clear when the number of projections L is small e.g., L and L 100. When CSW-s is better than SW by looking at the sharpness of the generated images. Also, we can visually observe the improvement of SW and CSW-s when increasing the number of projections. In summary. the qualitative results are consistent with the quantitative results (FID scores and IS scores) in Table [] For the generated images of CSW-b and CSW-d, we also observe the improvement compared SW which is consistent with the improvement of FID scores and IS scores.
1000, we can still figure that
Non-linear convolution sliced Wasserstein: We also compare non-linear extensions of SW and CSW variants in training generative models on CIFAR 10 in Appendix D For details of non-lined non-linear extensions, we refer Appendix From experiments, we observe that convolution can also improve the performance of sliced Wasserstein in non-linear projecting cases. Compared to linear versions, non-linear versions can enhance the quality the generative model yield comparable results.
Conclusion
We have addressed the issue of the conventional slicing process of sliced Wasserstein when working with probability measures images. In particular, sliced Wasserstein is defined probability measures over vectors which leads to the step of vectorization for images. As a result, the conventional slicing process cannot exploit the spatial structure of data for designing the space of projecting directions and projecting operators. To address the issue, we propose a new slicing process by using the convolution operator which has been shown to be efficient on images. Moreover, investigate the computational complexity and projection memory complexity of the new slicing technique. We show that convolution slicing is comparable computational complexity while being better in terms of projection memory complexity. By utilizing the new slicing technique, we derive a novel family of sliced Wassersein variants, named convolution sliced Wasserstein. We investigate the properties of the convolution sliced Wasserstein including its metricity, its computational and sample complexities, and its connection other variants of sliced Wasserstein in literature. Finally, we carry out extensive experiments in comparing digits images and training generative models on standard benchmark datasets to demonstrate the favorable performance of the convolution sliced Wasserstein.
conventional slicing in terms of
Acknowledgements
NH acknowledges support from the NSF IFMI. 2019844 and the NSF Institute for Foundations of Machine Learning.
10

---

References
[1] J. Altschuler, J. Niles-Weed, and Rigollet. Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration. In Advances in Neural Information Processing Systems, pages 1964-1974, 2017.
[2] M. Arjovsky, S. Chintala, and L.. Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, pages 214-223, 2017.
[3] B. Bhushan Damodaran, B. Kellenberger, R. Flamary, Tuia, and N. Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 447-4634 2018.
[4] S. Bobkov and M. Ledoux. 'One-dimensional empirical measures, order statistics, and Kan- torovich transport distances. Memoirs of the American Mathematical Society, 261, 2019.
[5] C. Bonet, N. Courty, F. Septier, and L. Drumetz. Sliced-Wasserstein gradient flows. arXiv preprint arXiv:21 10. 10972, 2021.
[6] N. Bonneel, J. Rabin, G. Peyré, and H. Pfister. Sliced and Radon Wasserstein barycenters of measures. Journal of Mathematical Imaging and Vision, 1(51):22 45, 2015.
[7] X. Chen, Y. Yang, and Li. Augmented sliced Wasserstein distances. International Conference on Learning Representations, 2022.
[8] A. Coates, A. Ng, and H. Lec. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international Conference on Artificial Intelligence and Statistics, pages 215-223. JMLR Workshop and Conference Proceedings, 2011.
[9] N. Courty, Flamary, Habrard, and A. Rakotomamoniy. Joint distribution optimal trans- portation for domain adaptation. In Advances in Neural Information Processing Systems, pages 2017.
[10] M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in Neural Information Processing Systems, pages 2292-2300, 2013.
[11] B. Dai and U. Seljak. Sliced iterative normalizing flows. In International Conference Machine Learning, pages 2352-23622 2364. PMLR, 2021.
[12] I. Deshpande, Y.-T. Hu, R. Sun, A. Pyrros, N. Siddiqui, S. Koyejo, 7. Zhao D. Forsyth, and A. G. Schwing. Max-sliced Wasserstein distance and its use for GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10648-10656, 2019.
[13] I. Deshpande, Z. Zhang, and A. G. Schwing. Generative modeling using the sliced Wasserstein distance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3483 3483-3491, 2018.
[14] K. Fatras, Y. Zine, R. Flamary, R. Gribonval, and N. Courty. Learning with minibatch Wasser- stein: asymptotic and gradient properties. In AISTATS 2020-23nd International Conference on Artificial Intelligence and Statisties, volume 108, pages 1 1-20, 2020.
[15] N. Fournier and A. Guillin. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162:707-738, 2015.
[16] K. Fukushima and S. Miyake. Neocognitron: A self-organizing neural network model for mechanism of visual pattern recognition. In Competition and cooperation neural nets, pages 267-285. Springer, 1982.
[17] Z. Goldfeld and K. Greenewald. Sliced mutual information: A scalable measure of statistical dependence. Advances in Neural Information Processing Systems, 34, 2021.
[18] I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. MIT press, 2016.
[19] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
[20] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neurul Information Processing Systems, pages 6626-6637, 2017.

---

[21] N. Ho, X. Nguyen, M. Yurochkin, H. H. Bui, V. Huynh, and D. Phung. Multilevel clustering via Wasserstein means. In International Conference on Machine Learning, pages 1501-1509, 2017.
[22] M. Huang, S. Ma, and L. Lai. A Riemannian block coordinate descent method for computing the projection robust Wasserstein distance. In International Conference on Machine Learning. pages 4446-4455. PMLR, 2021.
[23] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:14/2.6980, 2014.
[24] S. Kolouri, K. Nadjahi, U. Simsckli, R. Badcau, and G. Rohde. Generalized sliced W Wasserstein
distances. In Advances in Neural Information Processing Systems, pages 261-272, 2019.
[25] S. Kolouri, P. F. Pope, C. E. Martin, and G. K. Rohde. Sliced Wasserstein auto-encoders. In International Conference on Learning Representations, 2018.
[26] S. Kolouri, G. K. Rohde, and H. Hoffmann. Sliced Wasserstein distance for learning Gaussian mixture models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3427-3436, 2018.
[27] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. Master's s thesis, Department of Computer Science, University of Toronto, 2009.
[28] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 2012.
[29] Le, T. Nguyen, N. Ho, Bui, and D. Phung. Lamda: Label matching deep domain adaptation. PMLR, 2021.
In International Conference on Machine Learning, pages
[30] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2322 1998.
[31] C.-Y. Lee, T. Batra, M. H. Baig, and D. Ulbricht. Sliced Wasserstein discrepancy for unsuper- vised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2019.
[32] J. Lezama, W. Chen, and Q Qiu. Run-sort-rerun: Escaping batch size limitations in sliced Wasserstein generative models. In International Conference on Machine Learning, pages 6275-622 (155-628..o6 PMLR, 2021.
[33] T. Lin, C. Fan, N. Ho, M. Cuturi, and M. Jordan. Projection robust Wasserstein distance and Riemannian optimization. Advances in Neural Information Processing Systems, 33:9383 2020.
[34] T. Lin, N. Ho, X. Chen, M. Cuturi, and M. I. Jordan. Fixed-support Wasserstein barycenters: Computational hardness and fast algorithm. In NeurIPS, pages 5368-5380, 2020.
[35] T. Lin, N. Ho, and M. Jordan. On efficient optimal transport: An analysis of greedy and accelerated mirror descent algorithms. In International Conference on Machine Learning, pages 3982-3991, 2019.
[36] Lin, N. Ho, and M. I. Jordan. On the efficiency of the Sinkhorn and Greenkhom algorithms and their acceleration for optimal transport. ArXiv Preprint: 0906.01437, 2019.
[37] Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
[38] A. Liutkus, U. Simsekli, S. Majewski, A. Durmus, and F.-R. Stöter. Nonparametric generative modeling via optimal transport and diffusions. In International Conference on Machine Learning, pages 4 4104-4113, PMLR, 2019.
flows:
[39] G. Mena and J. Weed. Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. In Advances in Neural Information Processing Systems, 2019.
[40] N. Naderializadeh, J. Comer, R. Andrews, H. Hoffmaan,, and S. Kolouri. Pooling by sliced- Wasserstein embedding. Advances in Neural Information Processing Systems, 34, 2021.
[41] K. Nadjahi, V. De Bortoli, A. Durmus, R. Badeau, and U. Şimşekli. Approximate Bayesian computation with the sliced-Wasserstein Wasserstein distance. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5470-5474. IEEE, 2020.
12

---

[42] K. Nadjahi, A. Durmus, P. E. Jacob, R. Badeau, and U. Simsekli. Fast approximation of the Warree--assrsttein distance using concentration of random projections. Advances in Neural Information Processing Systems, 34, 2021.
[43] K. Nadjahi, A. Durmus, U. Simsekli, and R. Badeau. Asymptotic guarantees for learning generative models with the sliced-Wassense Wasserstein distance. In Advances in Neural Information Processing Systems, pages 250 26002200,400 2019.
[44] K. Nguyen, N. Ho, T. Pham, and H. Bui. Distributional generative modeling. In International Conference on Learning Representations, 2021.
and applications to
[45] K. Nguyen. D. Nguyen, Q. Nguyen, T. Pham, H. Bui, D. Phung, T. Le, and N. Ho. On transportation of mini- batches: A hierarchical approach. In Proceedings of the 39th International Conference Machine Learning. 2022.
[46] K. Nguyen, D. Nguyen, T. Pham, and N. Ho. Improving mini-batch optimal transport via partial transportation. In Proceedings of the 39th International Conference on Machine Learning, 2022.
[47] K. Nguyen, S. Nguyen, N. Ho, T. Pham, and H. Bui. Improving relational regularized au- toencoders with spherical sliced fused Gromov-Wasserster Wasserstein. In International Conference Learning Representations, 2021.
[48] F.-P. Paty and M. Cuturi. Subspace robust Wasserstein distances. In International Conference on Machine Learning, pages 5072-5081, 2019.
[49] O. Pele and M. Werman. Fast and robust earth mover's s distances. In 2009 IEEE 12th Interna- tional Conference on Computer Vision, pages 460-467. IEEE, September 2009.
[50] G. Peyré and M. Cuturi. Computational optimal transport: With applications to data science. Foundations and Trends@ in Machine Learning, 11(5-6):355-607, 2019,,
[51] G. Peyré and M. Cuturi. Computational optimal transport, 2020.
[52] J. Radon. 1.1 über die bestimmung von funktionen durch ihre integralwerte längs gewisser
mannigfaltigkeiten. Classic papers in modern diagnostic radiology, 5:21, 2005.
[53] A. Rakotomamonjy and R. Liva. Differentially private sliced Wasserstein distance. In Interna- tional Conference on Machine Learning, pages 8810-8820 8820. PMLR, 2021.
[54] M. Rowland, J. Hron, Y. Tang, K. Choromanski, T. Sarlos, and A. Weller. Orthogonal estimation of Wasserstein distances. In The 22nd International Conference Artificial Intelligence and Statistics, pages 186-195. PMLR, 2019.
[55] T. Salimans, 1. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training GANs. Advances in Neural Information Processing Systems, 29, 2016.
[56] Solomon, F. De Goes, G. Peyré, M. Cuturi, A. Butscher, A. Nguyen, T. Du, and L. Guibas. Convolutional wasserstein distances: Efficient optimal transportation geometric domains. ACM Transactions on Graphics (ToG), 3414) 2015.
[57] V. Titouan, R. Flamary, N. Courty, R. Tavenard, and L. Chapel. Sliced Gromov- Wasserstein. Advances in Neural Information Processing Systems, 32, 2019.
[58] I. Tolstikhin, O. Bousquet, S. Gelly, and B. Schoelkopf. Wasserstein auto-encoders. In International Conference Learning Representations, 2018.
[59] C. Villani. Optimal transport: Old and New. Springer, 2008.
[60] M. J. Wainwright. High-dimensional statistics: non-asymptotic viewpoint. Cambridge
University Press, 2019.
[61] J. Wu, Z. Huang, D. Acharya, W. Li, J. Thoma, D. P. Paudel, and L. V. Gool. Sliced Wasserstein generative models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3713-3722, 2019.
[62] Xu, H. Zhou, C. Gan, Zheng, and L. Li. Vocabulary learning via optimal transport for neural machine translation. In Proceedings of the 59th Annual Meeling of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 7361-7373, 2021.
13

---

[63] K. D. Yang. K. Damodaran, S. Venkatachalapathy, A. C. Saylemezoglu, G. Shivashankar, and C. Uhler. Predicting cell lineages using autoencoders and optimal transport. PLoS computational biology, 16(4))e1007828, 2020.
[64] M. Yi and S. Liu. Sliced Wasserstein variational inference. In Fourth Symposium on Advances in Approximale Bayesian Inference, 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(C) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material or as URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "ms2h1f7_Zp_", "reviewer": "Reviewer_s9sz", "summary": "While the Sliced-Wasserstein distance is widely used to compare distributions on images (which can be seen as tensors), it requires to perform a vectorization step using a reshape operation which is not suited to images and loses spatial structure, while being memory inefficient. In this work, authors propose to define a Convolutional Sliced-Wasserstein on the set of probabilities over tensors using convolution operations which are better suited to images. Some properties are derived such as the sample complexity, pseudo-distance. And many experiments are performed on classical image datasets.", "questions": "I have first a question on the theoretical results. In Theorem 1, it is stated that it is a pseudo-distance and that $CSW_p(\\mu,\\nu)=0 \\nLeftrightarrow \\mu=\\nu$. I do not find the last statement very clear. I believe that it is meant that $CSW_p(\\mu,\\nu)=0$ does not imply that $\\mu=\\nu$. I did not find in the proof of the theorem any argument supporting this statement. Is there any counter example of $\\mu\\neq \\nu$ such that $CSW_p(\\mu,\\nu)=0$?\n\nA second question is on the experiments. While the results seem quite convincing, it seems like the experiments (to get e.g. the FID and the IS) were run only once. I believe it would be more robust and convincing to take a mean over several trainings for example.\n\n\n\n\n\n\nTypos:\n- Line 126: $\\mu\\mathbb{R}^{c\\times \\times d}$", "limitations": "Yes", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper proposes to use convolutions to project images on the real line in order to use a SW distance between distributions over images. While the convolution idea is not novel since it has been widely used in neural networks for some time, the idea of using it in the context of optimal transport is new and very interesting. Moreover, the results seem quite convincing.\n\nStrengths:\n- Well written and very clear\n- Using convolution to define a new SW distance.\n- Application on different datasets which give good results\n- Theoretical results\n\nWeaknesses:\n- Only a pseudo-distance?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "at4FObFbYmS", "reviewer": "Reviewer_GwEg", "summary": "This work attempted to extend the sliced Wasserstein distance to be the convolution sliced Wasserstein distance by replacing vectors with tensors and Radon transforms with convolution operators, respectively. Then, this work compared its proposed methods with different convolutions (usual convolution, convolution with stride, convolution with dilation) with the original sliced Wasserstein on image generation task with various datasets (CIFAR10, CelebA, STL10, CelebA-HQ), claiming to show favorable results. Its theorem showed that this new convolution sliced Wasserstein distance is a pseudo-metric, it is always less than or equal to the max sliced Wasserstein distance, and its expectation is upper bounded with a specific value.", "questions": "- Feel free to argue back to the weakness points above.\n- This work argued the needs for using tensors instead of vectors in the sliced Wasserstein distance. While it might be making sense, it was not easy for me to be convinced about these arguments. Conventional image processing works have widely used vector representations in their models and tensors (i.e., reshapes) in their implementations. While this manuscript mentioned \"a probability measure over images should be defined over the space of tensors instead of images\" and \"this extra step does not keep the spatial structures of the supports, which are crucial information of images\", I need more explanation on why these sentences are justified. Even though images are represented as vectors, spatial structures can be considered in the operations on them. In that sense, these arguments seem relatively weak.", "limitations": "Even though the authors indicated that the limitations and potential negative societal impact of their work, I was not able to find them.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Even though this manuscript claimed that \"We derive convolution sliced Wasserstein (CSW) and its variants\" in the abstract, CSW was not derived, but was defined in this work (see Definition 5). While this work seems to assume that the distance between \"images\" are compared with the Wasserstein distance, there are a number of cases where \"vectorized feature tensors\" of two images are compared with the (sliced) Wasserstein distance (e.g., see [35]) and in that case, this work can be seen as a simplified version of these works. In that sense, it can be seen that there is no novelty in the definitions of this manuscript.\n- While this defined CSW was shown to be a pseudo-metric in Theorem 1, it may be a serious weakness as a loss to be used in the optimization. Will the different image with the distance 0 be good to represent Unfortunately, there is no discussion on this issue. Non-linear extension of CSW was mentioned in lines 328-332, it does not seem to be meaningful as a metric considering that linear version of it is a pseudo-metric.\n-  While its experiments were showing results on image generation task for various datasets, it was only compared with a conventional SW method, not a recent max SW or generalized sliced Wasserstein [23]. Moreover, there were many different convolutions defined in the main text and compared in the experiments, the results seems to suggest no clear winner among these convolutions. Thus, the experiments in its current form seem weak. Comparing with recent Wasserstein variants and investigating proper (or optimal) convolution operators could improve this manuscript.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DKLz2q7Txwt", "reviewer": "Reviewer_nLU5", "summary": "This paper proposed convolutional sliced Wasserstein distance and compared it with the conventional sliced Wasserstein distance. In addition, the authors also introduced a convolution base slicer, a convolution stride slicer, and its variant with dilation. The paper provided the details about how the convolutional sliced Wasserstein is calculated, and analyzed several properties. The experiment results on the multiple datasets demonstrate that for the generative tasks, the proposed CSW achieved better performance than SW while using similar computational resources.", "questions": "See the weakness section.\n\nOverall this paper proposed an interesting idea to better compute the Wasserstein distance between two image sets. However I lack the related background knowledge and some experiment details are missing, I cannot judge the merit of this paper.", "limitations": "The authors did not provide the discussions or analysis about the limitations and potential negative societal impact (if I did not miss anything). Overall I think it is fine as the paper focuses more on the fundamental SW distance computation. See the weakness section for potential technical limitations, for example, the choice of kernels.", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well written and provides a lot of details about SW, which is very helpful for readers like me with little SW background.\nThe idea is well motivated and the math seems convincing to me.\n\nI must apologize I don't have enough expertise to judge the merit of this paper. Below are some of my questions and concerns.\n1. In L201, the authors did notice that convolution is a linear operation. Thus consecutive convolution operations can be replaced with a single convolution. Except for the non-linear convolution slicer, what is the purpose of using multiple linear convolution operations? Moreover, convolution is often implemented as matrix multiplication, which also flattens the image and converts the kernel into a Toeplitz matrix. So the proposed convolutional SW is similar to SW with a special R in Eq. 1?\n\n2. I wonder how the kernel is determined? From L228-230, it seems the kernel is uniformly sampled from the set $K^{(l)}$. Do we sample once for all the images (that is, using the same set of kernels for all the images) or we repetitively sample kernels for each image?\n\n3. Some training details are missing. What is the loss function? Based on the main script, it seems the network does not need a discriminator as CSW directly computes the distance between the generated images and real images and the network is trained to minimize the distance. However, in the supplementary, discriminators are still used (L732-739). If discriminators are used, I am a little confused as SW GAN [13] used discriminators to map images into 1D vector and compute SW, which is similar to nonlinear CSW introduced in this submission, except that kernels in [13] are learnable while the kernels in this submission is uniformly sampled.\n\n4. The authors might miss one reference about introducing convolution in Wasserstein to reduce the computational cost:\nJustin Solomon, Fernando de Goes, Gabriel Peyré, Marco Cuturi, Adrian Butscher, Andy Nguyen, Tao Du, Leonidas Guibas, Convolutional Wasserstein Distances: Efficient Optimal Transportation on Geometric Domains, Proc. SIGGRAPH 2015.\n\nMinor: L126 $R^{c\\times \\times d}$ should be $R^{c\\times d\\times d}$", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "B6zcRMxZ8TI", "reviewer": "Reviewer_V4XA", "summary": "This paper presents a new methodology for comparing two probability measures over images.\nThe key idea of the paper is to apply convolution operators on probability measures over images.\nThe proposed method, named convolution sliced Wasserstein(CSW), makes use of the spatial structure of images and needs less slicing memory. \nThe authors provide the metricity of CSW as well as its sample complexity, its computational complexity, and connection to sliced Wasserstein distance.\nBeyond this theoretical contribution, the authors discuss numerical considerations and perform a thorough real data analysis, showing the favorable performance of their method.", "questions": "**Major Comments**\n\n* I am a little confused about the definition of the convolution-base slicer and its invariants.\nIn definition 3, when $d$ is even, the last kernel $K^{(N)}$ has dimension $1\\times a \\times a$.\nHowever, $a=\\frac{d}{2^{N-1}}$ and by definition $a$ is an integer.\nIs there any guarantee about this? \nFor example, what is the definition of these kernels when $d=18$?\nIn real data, the dimension $d=32,64,96,128$,\nwhose prime factor decomposition has at most one number which is not 2.\nBut $a=\\frac{d}{2^{N-1}}$ won't be an integer when $d=18=2\\times 3\\times 3$.\nThis question also lies in the definition of other variants of convolution sliced-Wasserstein distances.\n\n* The authors fix the stride size $s=2$ in definition of convolution-stride slicer.\nThere may be some trade-off between time consumption and estimation efficiency for different stride sizes $s$.\nSome simple analysis may help us recognize it intuitively.\nI am wondering how does the stride size affect the performance of the proposed method? Some discussion and empirical results are needed.\n\n* This paper mainly considers probability measures over images. It would be convincing if the authors can provide other tasks in real data applications.\n\n", "limitations": "  ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The idea of applying the convolution operators on probability measures over images is a very natural idea. \nThe proposed method is novel and simple to describe.\nAlso, the authors show the efficiency numerically. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept