SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition 4.3 (Weak Convergence). min-SWGG metricizes the weak convergence in  $\mathcal{P}_2^n (\mathbb{R}^d)$ . In other words, let  $(\mu_k)_{k\in \mathbb{N}}$  be a sequence of measures in  $\mathcal{P}_2^n (\mathbb{R}^d)$  and  $\mu \in \mathcal{P}_2^n (\mathbb{R}^d)$ . We have:
$$
\mu_ {k} \xrightarrow [ k ]{\mathcal {L} , 2} \mu \iff \min  _ {\mathrm {S W G G} _ {2} ^ {2} (\mu_ {k}, \mu)} \xrightarrow [ k ]{} 0,
$$
where  $\xrightarrow{\mathcal{L},2}$  stands for the weak convergence of measure i.e.  $\int_{\mathbb{R}^d}fd\mu_k\to \int_{\mathbb{R}^d}fd\mu$  for all continuous bounded functions  $f$
Beyond the weak convergence, min-SWGG possesses the translation property, i.e. the translations can be factored out as the Wasserstein distance does (see [53, remark 2.19] for a recall).
Proposition 4.4 (Translation). Let  $T^u$  (resp.  $T^v$ ) be the map  $\pmb{x} \mapsto \pmb{x} - \pmb{u}$  (resp.  $\pmb{x} \mapsto \pmb{x} - \pmb{v}$ ), with  $\pmb{u}, \pmb{v}$  vectors of  $\mathbb{R}^d$ . We have:
$$
\min  - \operatorname {S W G G} _ {2} ^ {2} \left(T _ {\#} ^ {u} \mu_ {1}, T _ {\#} ^ {v} \mu_ {2}\right) = \min  - \operatorname {S W G G} _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) + \| \boldsymbol {u} - \boldsymbol {v} \| _ {2} ^ {2} - 2 \langle \boldsymbol {u} - \boldsymbol {v}, \boldsymbol {m} _ {1} - \boldsymbol {m} _ {2} \rangle
$$
where  $\pmb{m}_{\mathbf{1}} = \int_{\mathbb{R}^d}\pmb {x}d\mu_1(\pmb {x})$  and  $\pmb {m}_2 = \int_{\mathbb{R}^d}\pmb {x}d\mu_2(\pmb {x})$  are the means of  $\mu_{1},\mu_{2}$
This property is useful in some applications such as shape matching, in which translation invariances are sought.
The proofs of the two Propositions are deferred to Supp. 10.2 and 10.3.
Remark 4.5 (Equality). min-SWGG and  $W_2^2$  are equal in different cases. First, [43] showed that it is the case whenever  $\mu_1$  is the shift and scaling of  $\mu_2$  (see Supp. 9.1 for a full discussion). In Lemma 4.6, we will state that it is also the case if one of the two distributions is supported on a line.
4.3 Efficient computation of SWGG
SWGG defined in Eq. (14) involves computing three WDs that are fast to compute, with an overall  $\mathcal{O}(dn + n\log n)$  complexity, as detailed below. Building on this result, we provide an optimization scheme that allows optimizing over  $\theta$  with  $\mathcal{O}(sdn + sn\log sn)$  operations at each iteration, with  $s$  a (small) integer. We first start by giving a new closed form expression of the WD whenever one distribution is supported on a line, that proves useful for deriving an efficient computation scheme.
New closed form of the WD. The following lemma states that  $W_2^2(\mu_1, \mu_2)$  admits a closed form whenever  $\mu_2$  is supported on a line.
This lemma leverages the computation of the WD between  $\mu_{2}$  and the orthogonal projection of  $\mu_{1}$  onto the linear subspace defined by the line. Additionally, it provides an explicit formulation for the optimal transport map  $T^{1\rightarrow 2}$ .
Lemma 4.6. Let  $\mu_1, \mu_2$  in  $\mathcal{P}_2^n(\mathbb{R}^d)$  with  $\mu_2$  supported on a line of direction  $\theta \in \mathbb{S}^{d-1}$ . We have:
$$
W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) = W _ {2} ^ {2} \left(\mu_ {1}, Q _ {\#} ^ {\theta} \mu_ {1}\right) + W _ {2} ^ {2} \left(Q _ {\#} ^ {\theta} \mu_ {1}, \mu_ {2}\right) \tag {15}
$$
with  $Q^{\theta}$  as in Def. 4.1. Note that  $W_{2}^{2}(\mu_{1},Q_{\#}^{\theta}\mu_{1}) = \frac{1}{n}\sum \| \pmb{x}_{i} - Q^{\theta}(\pmb{x}_{i})\|_{2}^{2}$  and  $W_{2}^{2}(Q_{\#}^{\theta}\mu_{1},\mu_{2}) = W_{2}^{2}(P_{\#}^{\theta}\mu_{1},P_{\#}^{\theta}\mu_{2})$  are the WD between 1D distributions. Additionally, the optimal transport map is given by  $T^{1\to 2} = T^{Q_{\#}^{\theta}\mu_{1}\to \mu_{2}}\circ T^{\mu_{1}\to Q_{\#}^{\theta}\mu_{1}} = T^{Q_{\#}^{\theta}\mu_{1}\to \mu_{2}}\circ Q^{\theta}$ . In particular, the map  $T^{1\to 2}$  can be obtained via the permutations of the 1D distributions  $P_{\#}^{\theta}\mu_{1}$  and  $P_{\#}^{\theta}\mu_{2}$ . The proof is provided in Supp. 10.4.
Efficient computation of SWGG. Eq. (14) is defined as the Wasserstein distance between a distribution (either  $\mu_{1}$  or  $\mu_{2}$  or  $\mu_{g,\theta}^{1\rightarrow 2}$ ) and a distribution supported on a line ( $\mu_{\theta}^{1\rightarrow 2}$ ). As detailed in Supp. 10.5, computation of Eq. (14) involves three Wasserstein distances between distributions and their projections: i)  $W_2^2 (\mu_1,Q_\#^\theta \mu_1)$ , ii)  $W_2^2 (\mu_2,Q_\#^\theta \mu_2)$ , iii)  $W_2^2 (\mu_{g,\theta}^{1\rightarrow 2},\mu_\theta^{1\rightarrow 2})$ , and a one-dimensional Wasserstein distance  $W_2^2 (P_\#^\theta \mu_1,P_\#^\theta \mu_2)$ , resulting in a  $\mathcal{O}(dn + n\log n)$  complexity.
Optimization scheme for min-SWGG. The term  $W_2^2 (\mu_{g,\theta}^{1\rightarrow 2},\mu_\theta^{1\rightarrow 2})$  in Eq. (14) is not continuous w.r.t.  $\theta$ . Indeed, the generalized mean  $\mu_{g,\theta}^{1\rightarrow 2}$  depends only on the transport maps  $T^{\mu_{\theta}^{1\rightarrow 2}\rightarrow \mu_{1}}$  and  $T^{\mu_{\theta}^{1\rightarrow 2}\rightarrow \mu_{2}}$ , which remain constant as long as different projection directions  $\theta$  lead to the same permutations  $\sigma_{\theta}$  and  $\tau_{\theta}$ . Hence, we rely on a smooth surrogate  $\widetilde{\mu_{g,\theta}^{1\rightarrow 2}}$  of the generalized mean and we aim to minimize the following objective function:
$$
\widetilde {\mathrm {S W G G}} _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}, \theta\right) \stackrel {\text {d e f}} {=} 2 W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) + 2 W _ {2} ^ {2} \left(\mu_ {\theta} ^ {1 \rightarrow 2}, \mu_ {2}\right) - 4 W _ {2} ^ {2} \left(\widetilde {\mu_ {g , \theta} ^ {1 \rightarrow 2}}, \mu_ {\theta} ^ {1 \rightarrow 2}\right). \tag {16}
$$
To define  $\widetilde{\mu_{g,\theta}^{1\rightarrow 2}}$ , one option would be to use entropic maps in Eq. (11) but at the price of a quadratic time complexity. We rather build upon the blurred Wasserstein distance [26] to define  $\widetilde{\mu_{g,\theta}^{1\rightarrow 2}}$  as it can be seen as an efficient surrogate of entropic transport plans in 1D. In one dimensional setting,  $\widetilde{\mu_{g,\theta}^{1\rightarrow 2}}$  can be approximated efficiently by adding an empirical Gaussian noise followed by a sorting pass. In our case, it resorts in making  $s$  copies of each sorted projection  $P^{\theta}(\pmb{x}_{\sigma(i)})$  and  $P^{\theta}(\pmb{y}_{\tau(i)})$  respectively, to add an empirical Gaussian noise of deviation  $\sqrt{\epsilon}/2$  and to compute averages of sorted blurred copies  $\pmb{x}_{\sigma^s}^s, \pmb{y}_{\tau^s}^s$ . We finally have  $(\widetilde{\mu_{g,\theta}^{1\rightarrow 2}})_i = \frac{1}{2s}\sum_{k=(i-1)s+1}^{is}\pmb{x}_{\sigma^s(k)}^s + \pmb{y}_{\tau^s(k)}^s$ . [26] showed that this blurred WD has the same asymptotic properties as the Sinkhorn divergence.
The surrogate  $\widetilde{\mathrm{SWGG}} (\mu_1,\mu_2,\theta)$  is smoother w.r.t.  $\theta$  and can thus be optimized using gradient descent, converging towards a local minima. Once the optimal direction  $\theta^{*}$  is found, min-SWGG resorts to be the solution provided by  $\mathrm{SWGG}(\mu_1,\mu_2,\theta^*)$ . Fig. 2 illustrates the effect of the smoothing on a toy example and more details are given in Supp. 10.6. The computation of  $\widetilde{\mathrm{SWGG}} (\mu_1,\mu_2,\theta)$  is summarized in Alg. 1.
Algorithm 1 Computing  $\widetilde{\mathrm{SWGG}}_2^2 (\mu_1,\mu_2,\theta)$
Require:  $\mu_{1} = \frac{1}{n}\sum \delta_{\pmb{x}_{i}}$ $\mu_{2} = \frac{1}{n}\sum \delta_{\pmb{y}_{i}}$ $\theta \in \mathbb{S}^{d - 1}$ $s\in \mathbb{N}_+$  and  $\epsilon \in \mathbb{R}_+$
$\sigma, \tau \gets$  ascending ordering of  $(P^{\theta}(\pmb{x}_i))_i, (Q^{\theta}(\pmb{y}_i))_i$
$\pmb{x}^s \gets s$  copies of  $(\pmb{x}_{\sigma(i)})_i, \pmb{y}^s \gets s$  copies of  $(\pmb{y}_{\tau(i)})_i$
$\sigma^s, \tau^s \gets$  ascending ordering of  $\langle \pmb{x}^s, \theta \rangle + \pmb{\xi}, \langle \pmb{y}^s, \theta \rangle + \pmb{\xi}$  for  $\xi_i \sim \mathcal{N}(0, \epsilon/2), \forall i \leq sn$
$$
a \leftarrow \frac {2}{n} \sum_ {i} \left(\left\| \boldsymbol {x} _ {i} - Q ^ {\theta} (\boldsymbol {x} _ {i}) \right\| _ {2} ^ {2} + \left\| \boldsymbol {y} _ {i} - Q ^ {\theta} (\boldsymbol {y} _ {i}) \right\| _ {2} ^ {2}\right) \quad \triangleright 2 W _ {2} ^ {2} \left(\mu_ {1}, Q _ {\#} ^ {\theta} \mu_ {1}\right) + 2 W _ {2} ^ {2} \left(\mu_ {2}, Q _ {\#} ^ {\theta} \mu_ {2}\right)
$$
$$
b \leftarrow \frac {2}{n} \sum_ {i} \left\| P ^ {\theta} \left(\boldsymbol {x} _ {\sigma (i)}\right) + P ^ {\theta} \left(\boldsymbol {x} _ {\tau (i)}\right) \right\| _ {2} ^ {2} \quad \triangleright 2 W _ {2} ^ {2} \left(P _ {\#} ^ {\theta} \mu_ {1}, P _ {\#} ^ {\theta} \mu_ {2}\right)
$$
$$
c \leftarrow \frac {4}{n} \sum_ {i} \left\| \frac {1}{2} \left(Q ^ {\theta} \left(\boldsymbol {x} _ {\sigma (i)}\right) + Q ^ {\theta} \left(\boldsymbol {y} _ {\tau (i)}\right)\right) - \frac {1}{2 s} \sum_ {k = (i - 1) s + 1} ^ {i s} \left(\boldsymbol {x} _ {\sigma^ {s} (k)} ^ {s} + \boldsymbol {y} _ {\tau^ {s} (k)} ^ {s}\right)\right\| _ {2} ^ {2} \triangleright 4 W _ {2} ^ {2} \left(\widetilde {\mu_ {g , \theta} ^ {1 \rightarrow 2}}, \mu_ {\theta} ^ {1 \rightarrow 2}\right)
$$
Output  $a + b - c$
Figure 2: Illustration of the smoothing effect in the same setting as in Fig. 1. (Left) Two sets of generalized Wasserstein means are possible, depending on the direction of the sampled line w.r.t.  $\theta_{1}$  and  $\theta_{2}$ , giving rise to 2 different values for SWGG. (Middle) The surrogate provides a smooth transition between the two sets of generalized Wasserstein means as the direction  $\theta$  changes, (Right) providing a smooth approximation of SWGG that is amenable to optimization.
We highlight that min-SWGG is fast to compute, gives an approximation of the WD and the associated transport plan. We start by comparing the random search and the gradient descent schemes for finding the optimal direction in subsection 5.1. Subsection 5.2 illustrates the weak convergence property of min-SWGG through a gradient flow application to match distributions. We then implement an efficient algorithm for colorization of gray scale images in 5.3, thanks to the new closed form expression of the WD. We finally evaluate min-SWGG in a shape matching context in subsection 5.4. When possible from the context, we compare min-SWGG with the main methods for approximating the WD namely SW, max-SW, Sinkhorn [23], factored coupling [29] and subspace robust WD (SRW) [52]. Supp. 11 provides additional results on the behavior of min-SWGG and experiments on other tasks such as color transfer or on data sets distance computation. All the code is available at
5.1 Computing min-SWGG
Let consider Gaussian distributions in dimensions  $d \in \{2, 20, 200\}$ . We first sample  $n = 1000$  points from each distribution to define  $\mu_1$  and  $\mu_2$ . We then compute  $\min - \mathrm{SWGG}_2^2(\mu_1, \mu_2)$  computed using different schemes, either by random search, by simulated annealing [54] or by gradient descent. We report the obtained results in Fig. 3 (left). For the random search scheme, we repeat each experiment 20 times and we plot the average value of  $\min - \mathrm{SWGG} \pm 2$  times the standard deviation.
For the gradient descent, we select a random initial  $\theta$ . We observe that, in low dimension, all schemes provide similar values of min-SWGG. When the dimension increases, optimizing the direction  $\theta$  yields a more accurate approximation of the true Wasserstein distance (see plots' title in Fig. 3). On Fig. 3 (right), we compare the empirical runtime evaluation for min-SWGG with different competitors for  $d = 3$  and using  $n$  samples from Gaussian distributions, with  $n \in \{10^2, 10^3, 10^4, 5 \times 10^4, 10^5\}$ . We observe that, as expected, min-SWGG with random search is as fast as SW with a super linear time complexity. With the optimization process, it is faster than SRW for a given number of samples. We also note that SRW is more demanding in memory and hence does not scale as well as min-SWGG. We give more details on this experimentation and a comparison with competitors in Supp. 11.2.
Figure 3: (Left) evolution of min-SWGG with different numbers of projections and with the dimension  $d$  in  $\{2,20,200\}$ . (Right) Runtimes.
<https://github.com/MaheyG/SWGG>
5.2 Gradient Flows
We highlight the weak convergence property of min-SWGG. Initiating from a random initial distribution, we aim to move the particles of a source distribution  $\mu_{1}$  towards a target one  $\mu_{2}$  by reducing the objective min-SWGG $_2^2(\mu_1, \mu_2)$  at each step. We compare both variants of min-SWGG against SW, max-SW and PWD, relying on the code provided in [37] for running the experiment; we report the results on Fig. 4. We consider several target distributions, representing diverse scenarios and fix  $n = 100$ . We run each experiment 10 times and report the mean  $\pm$  the standard deviation. In every case, one can see that  $\mu_{1}$  moves towards  $\mu_{2}$  and that all methods tend to have similar behavior. One can notice though that, for the distributions in  $d = 500$  dimensional space, min-SWGG computed with the optimization scheme leads to the best alignment of the distributions.
Figure 4: Log of the WD between different source and target distributions as a function of the number of iterations.
5.3 Gray scale image colorization
Lemma 4.6 states that the WD has a closed form when one of the 2 distributions is supported on a line, allowing us to compute the WD and the OT map with a complexity of  $\mathcal{O}(dn + n\log n)$ . This particular situation arises for instance with RBG images  $(\mu_1,\mu_2\in \mathcal{P}_2^n (\mathbb{R}^3))$ , where black and white images are supported on a line (the line of grays). One can address the problem of image colorization through color transfer [25], where a black and white image is the source and a colorful image the target. Our fast procedure allows considering large images without sub-sampling with a reasonable computation time. Fig. 5 gives an example of colorization of an image of size  $1280\times 1024$  that was computed in less than 0.2 second, while being totally untractable for the  $\mathcal{O}(n^3\log n)$  solver of WD.
Figure 5: Cloud point source and target (left) colorization of image (right).
This procedure can be lifted to pan-sharpening [64] where one aims to construct a super-resolution multi-chromatic satellite image with the help of a super-resolution mono-chromatic image (source) and a low-resolution multi-chromatic image (target). Obtained results are given in the Supp. 11.4.
5.4 Point clouds registration
Iterative Closest Point (ICP) is an algorithm for aligning point clouds based on their geometries [7]. Roughly, its most popular version defines a one-to-one correspondence between point clouds, computes a rigid transformation (namely translation, rotation or reflection), moves the source point clouds using the transformation, and iterates the procedure until convergence. The rigid transformation is the solution of the Procrustes problem i.e.  $\arg \min_{(\Omega ,t)\in O(d)\times \mathbb{R}^d}\| \Omega (\pmb {X} - t) - \pmb {Y}\| _2^2$ , where  $\pmb {X},\pmb{Y}$  are the source and the target cloud points and  $O(d)$  the space of orthogonal matrices of dimension  $d$ . This Procrustes problem can be solved using a SVD [59] for instance.
We perform the ICP algorithm with different variants to compute the one-to-one correspondence: nearest neighbor (NN) correspondence, OT transport map (for small size datasets) and min-SWGG
transport map. Note that SW, PWD, SRW, factored coupling and Sinkhorn cannot be run in this context where a one-to-one correspondence is mandatory; subspace detours [44] are irrelevant in this context (see Supp. 11.5). We evaluate the results of the ICP algorithm in terms of: i) the quality of the final alignment, measured by the Sinkhorn divergence between the re-aligned and target point cloud; ii) the speed of the algorithm given by the running time until convergence. We consider 3 datasets of different sizes. The results are shown in Table 1 and more details about the setup, can be found in Supp. 11.5. In Supp. 11.5 we give a deeper analysis of the results, notably with different criteria for the final assignment, namely the Chamfer and the Frobenius distance. One can see that the assignment provided by OT-based methods is better than NN. min-SWGG allows working with large datasets, while OT fails to provide a solution for  $n = 150000$ .
n	500	3000	150 000
NN	3.54 (0.02)	96.9 (0.30)	23.3 (59.37)
OT	0.32 (0.18)	48.4 (58.46)	-
min-SWGG	0.05 (0.04)	37.6 (0.90)	6.7 (105.75)
Table 1: Sinkhorn Divergence between final transformation on the source and the target. Timings in seconds are into parenthesis. Best values are boldfaced. An example of a point clouds ( $n = 3000$ ) is provided on the left.
In this paper, we hinge on the properties of sliced Wasserstein distance and on the Wasserstein generalized geodesics to define min-SWGG, a new upper bound of the Wasserstein distance that comes with an associated transport map. Topological properties of SWGG are provided, showing that it defines a metric and that min-SWGG metrizes the weak convergence of measure. We also propose two algorithms for computing min-SWGG, either through a random search scheme or a gradient descent procedure after smoothing the generalized geodesics definition of min-SWGG. We illustrate its behavior in several experimental setups, notably showcasing its interest in applications where a transport map is needed.
The set of permutations covered by min-SWGG is the one induced by projections and permutations on the line. It is a subset of the original Birkhoff polytope and it would be interesting to characterize how these two sets relates. In particular, in the case of empirical realizations of continuous distributions, the behavior of min-SWGG, when  $n$  grows, needs to be investigated. In addition, the fact that min-SWGG and WD coincide when  $d > 2n$  calls for embedding the distributions in higher dimensional spaces to benefit from the greater expressive power of projection onto the line. Another important consideration is to establish a theoretical upper bound for min-SWGG.
The authors gratefully acknowledge the financial support of the French Agence Nationale de la Recherche (ANR), under grant ANR-20-CHIA-0021-01 (project RAIMO $^2$ ), grant ANR-20-CHIA-0030 (project OTTOPIA) and grant ANR-18-CE23-0022-01 (project MULTISCALE). Clément Bonet is supported by the project DynaLearn from Labex CominLabs and Région Bretagne ARED DLearnMe.
<sup>2</sup>https://chaire-raimo.github.io/
[56] Julien Rabin, Gabriel Peyré, Julie Delon, and Marc Bernot. Wasserstein barycenter and its application to texture mixing. In Alfred M. Bruckstein, Bart M. ter Haar Romeny, Alexander M. Bronstein, and Michael M. Bronstein, editors, Scale Space and Variational Methods in Computer Vision, pages 435-446, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg.
[57] Mark Rowland, Jiri Hron, Yunhao Tang, Krzysztof Choromanski, Tamas Sarlos, and Adrian Weller. Orthogonal estimation of Wasserstein distances. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 186-195. PMLR, 2019.
[58] Filippo Santambrogio. Optimal transport for applied mathematicians. *Birkhäuser*, NY, 55(58-63):94, 2015.
[59] Peter H Schonemann. A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1):1-10, 1966.
[60] Matteo Togninalli, Elisabetta Ghisu, Felipe Llinares-López, Bastian Rieck, and Karsten Borgwardt. Wasserstein weisfeiler-lehman graph kernels. Advances in Neural Information Processing Systems, 32, 2019.
[61] Stefan Van der Walt, Johannes L Schonberger, Juan Nunez-Iglesias, François Boulogne, Joshua D Warner, Neil Yager, Emmanuelle Gouillart, and Tony Yu. scikit-image: image processing in python. PeerJ, 2:e453, 2014.
[62] Peter JM Van Laarhoven, Emile HL Aarts, Peter JM van Laarhoven, and Emile HL Aarts. Simulated annealing. Springer, 1987.
[63] Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009.
[64] Gemine Vivone, Mauro Dalla Mura, Andrea Garzelli, and Fabio Pacifici. A benchmarking protocol for pansharpening: Dataset, preprocessing, and quality assessment. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:6102-6118, 2021.
[65] Wei Wang, Dejan Slepčev, Saurav Basu, John A Ozolek, and Gustavo K Rohde. A linear optimal transportation framework for quantifying and visualizing variations in sets of images. International journal of computer vision, 101(2):254-269, 2013.
[66] Jonathan Weed and Francis Bach. Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance. Bernoulli, 25(4A):2620-2648, 2019.
[67] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
[68] Jingyi Zhang, Ping Ma, Wenxuan Zhong, and Cheng Meng. Projection-based techniques for high-dimensional optimal transport problems. Wiley Interdisciplinary Reviews: Computational Statistics, 15(2):e1587, 2023.
7 Proofs and supplementary results related to Section 3
7.1 Overestimation of WD by PWD
As stated in Section 2, the projected Wasserstein distance PWD (see Eq. 7) tends to overestimate the Wasserstein distance. This is due to the fact that some permutations  $\sigma_{\theta}$  and  $\tau_{\theta}$  (with  $\theta \in \mathbb{S}^{d - 1}$ ) involved in PWD computation may be irrelevant. Such situation occurs when the distributions are in high dimension but supported on a low dimensional manifold or when the distributions are multi-modal.
Let consider the distributions  $\mu_{1}$  and  $\mu_{2}$  lying on a low dimensional manifold. In high dimension, randomly sampled vectors  $\theta$  tend to be orthogonal. Moreover, vectors orthogonal to the low dimensional manifold lead to "collapsed" projected distributions  $P_{\#}^{\theta}\mu_{1}$  and  $P_{\#}^{\theta}\mu_{2}$  onto  $\theta$ . Hence, such projection directions lead to permutations that can be randomly chosen. To empirically illustrate this behavior of PWD, we consider  $\mu_{1}$  and  $\mu_{2}$  as Gaussian distributions in  $\mathbb{R}^{d}$ ,  $d = 10$  but supported on the first two coordinates and we sample 200 points per distribution. Table 2 summarizes the obtained corresponding distances and shows that PWD overestimates the WD.
Now, let us consider two multimodal distributions  $\mu_1, \mu_2$  with  $K$  clusters such that each cluster of  $\mu_1$  has a close cluster from  $\mu_2$  (cyclical monotonicity assumption). Also we assume the same number of points in each cluster. OT plan will match the corresponding clusters and will lead to a relatively low value for  $W_2^2$  (since cluster from  $\mu_1$  has a closely related cluster in  $\mu_2$ ). However as PWD may allow permutations that make correspondences between points from different clusters (since a source cluster and a target cluster can be far in the original space but very close when projected on 1D), the resulting distance will be much more larger, leading to an overestimation of the Wasserstein distance. Table 2 provides an illustration for  $K = 10$  clusters and  $d = 2$ .
Table 2: Values of  $W_2^2$ , PWD and min-SWGG on two toy examples. PWD samples  $\theta$  uniformly over  $\mathbb{S}^{d-1}$ ; PWD Orthogonal Projections seek orthogonal vectors (see [57] for more details)
Distributions	Multi-modal	Low dimensional manifold
W22	12	12
PWD22Monte-Carlo	54	29
PWD22Orthogonal Projections	54	37
min-SWGG22	13	13
7.2 Quantile version of SWGG
The main body of the paper expresses SWGG for empirical distributions  $\mu_{1}$  and  $\mu_{2}$  with the same number of points and uniform probability masses. In this section we derived SWGG in a more general setting of discrete distributions.