SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let remark that min-SWGG relies on solving a 1D optimal transport (OT) problem. So far, the 1D OT problem was derived for  $\mu_1, \mu_2 \in \mathcal{P}_2^n(\mathbb{R})$  and thus was expressed using the permutation operators  $\tau$  and  $\sigma$ . In the general setting of distributions  $\mu_1 \in \mathcal{P}_2^n(\mathbb{R})$  and  $\mu_2 \in \mathcal{P}_2^m(\mathbb{R})$  with  $n \neq m$ , the 1D optimal transport is computed based on quantile functions. Hence, the expression of SWGG in the general setting of  $\mu_1 \in \mathcal{P}_2^n(\mathbb{R})$  and  $\mu_2 \in \mathcal{P}_2^m(\mathbb{R})$  hinges on quantile functions instead of permutations.
More formally, let  $\mu \in \mathcal{P}_2^n (\mathbb{R})$ ; its cumulative function is defined as:
$$
F _ {\mu}: \mathbb {R} \rightarrow [ 0, 1 ], x \mapsto \int_ {- \infty} ^ {x} d \mu \tag {17}
$$
and its quantile function (or pseudo inverse), is given by:
$$
q _ {\mu}: [ 0, 1 ] \rightarrow \mathbb {R}, r \mapsto \min  \{x \in \mathbb {R} \cup \{- \infty \} \text {s . t .} F _ {\mu} (x) \geq r \} \tag {18}
$$
An important remark is that the quantile function is a step function with  $n$  (the number of atoms) discontinuities. Thus, it can be stored efficiently using two vectors of size  $n$  (one for the locations of the discontinuities and the other for the values of the discontinuities).
For  $\mu_1\in \mathcal{P}_2^n (\mathbb{R})$  and  $\mu_{2}\in \mathcal{P}_{2}^{m}(\mathbb{R})$ , we recover the Wasserstein distance through quantiles with:
$$
W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) = \int_ {0} ^ {1} \left| q _ {\mu_ {1}} (r) - q _ {\mu_ {2}} (r) \right| ^ {2} d r \tag {19}
$$
Moreover, the optimal transport plan is given by:
$$
\pi = \left(q _ {\mu_ {1}}, q _ {\mu_ {2}}\right) _ {\#} \lambda_ {[ 0, 1 ]} \tag {20}
$$
where  $\lambda_{[0,1]}$  is the Lebesgue measure on  $[0,1]$ . The transport plan can be stored efficiently using two vectors of size  $(n + m - 1)$  (see [53] Prop 3.4).
Following [53, Remark 9.6], one can define the quantile function related to the Wasserstein mean by:
$$
q _ {\mu^ {1} \rightarrow 2} = \frac {1}{2} q _ {\mu_ {1}} + \frac {1}{2} q _ {\mu_ {2}}. \tag {21}
$$
Now, let  $\mu_1 \in \mathcal{P}_2^n(\mathbb{R}^d)$  and  $\mu_2 \in \mathcal{P}_2^m(\mathbb{R}^d)$ . Let  $\mu_\theta^{1 \to 2}$  be the Wasserstein mean of the projected distributions on  $\theta$ . Finally let  $\pi^{\theta \to 1}$  denote the transport plan from  $\mu_\theta^{1 \to 2}$  to  $\mu_1$  and  $\pi^{\theta \to 2}$  be the transport plan from  $\mu_\theta^{1 \to 2}$  to  $\mu_2$ . Following the construction of [4, Sec. 9.2], we shall introduce a multi marginal plan defined as:
$$
\pi \in \mathcal {P} _ {2} \left(\mathbb {R} ^ {d} \times \mathbb {R} ^ {d} \times \mathbb {R} ^ {d}\right) \text {s . t .} P _ {\#} ^ {1 2} \pi = \pi^ {\theta \rightarrow 1}, P _ {\#} ^ {1 3} \pi = \pi^ {\theta \rightarrow 2} \text {a n d} \pi \in \Pi \left(\mu_ {\theta} ^ {1 \rightarrow 2}, \mu_ {1}, \mu_ {2}\right) \tag {22}
$$
where  $P^{12}:(\mathbb{R}^d)^3\to (\mathbb{R}^d)^2$  projects to the first two coordinates and  $P^{13}$  projects to the coordinates 1 and 3. In particular,  $P_{\#}^{12}\pi$  is the projection of  $\pi$  on its 2 first marginals and  $P_{\#}^{13}\pi$  on the first and 3rd marginal. Similarly to the 2-marginal transport plan we defined  $\Pi (\mu_{\theta}^{1\rightarrow 2},\mu_1,\mu_2) = \{\pi \in \mathcal{P}_2(\mathbb{R}^d\times$ $\mathbb{R}^d\times \mathbb{R}^d)$  s.t.  $\pi (A\times \mathbb{R}^d\times \mathbb{R}^d) = \mu_\theta^{1\to 2}(A),\pi (\mathbb{R}^d\times A,\times \mathbb{R}^d) = \mu_1(A)$  and  $\pi (\mathbb{R}^d\times \mathbb{R}^d\times A) = \mu_2(A)$ $\forall A$  measurable set of  $\mathbb{R}^d\}$ :
The generalized barycenter  $\mu_{g,\theta}^{1\rightarrow 2}$  is then defined as:
$$
\mu_ {g, \theta} ^ {1 \rightarrow 2} = \left(\frac {1}{2} P ^ {2} + \frac {1}{2} P ^ {3}\right) _ {\#} \pi \tag {23}
$$
where  $P^i$  is the projection on the  $i$ -th coordinate.
We finally have all the building blocks to compute SWGG in the general case. Let remark that the complexity goes from  $\mathcal{O}(dn + n\log n)$  in the  $\mathcal{P}_2^n (\mathbb{R}^d)$  case to  $\mathcal{O}(d(n + m) + (n + m)\log (n + m))$  in the general case.
7.3 Proof of Proposition 3.2
We aim to prove that  $\mathrm{SWGG}_2^2 (\mu_1,\mu_2,\theta)$  is an upper bound of  $W_{2}^{2}(\mu_{1},\mu_{2})$  and that  $\mathrm{SWGG}(\mu_1,\mu_2,\theta)$  is a distance  $\forall \theta \in \mathbb{S}^{d - 1},\mu_i\in \mathcal{P}_2^n (\mathbb{R}^d),i = 1,2.$
Distance. Note that this proof will be derived for the alternative definition of SWGG in supp. 10.8.
Let  $\mu_{1} = \frac{1}{n}\sum \delta_{\pmb{x}_{i}},\mu_{2} = \frac{1}{n}\sum \delta_{\pmb{y}_{i}},\mu_{3} = \frac{1}{n}\sum \delta_{\pmb{z}_{i}}$  be in  $\mathcal{P}_2(\mathbb{R}^d)$ , let  $\theta \in \mathbb{S}^{d - 1}$ . We note  $\sigma$  (resp.  $\tau$  and  $\pi$ ) the permutation such that  $\langle \pmb{x}_{\sigma (1)},\theta \rangle \leq \ldots \leq \ldots \langle \pmb{x}_{\sigma (n)},\theta \rangle$  (resp.  $\langle \pmb{y}_{\tau (1)},\theta \rangle \leq \ldots \leq \ldots \langle \pmb{y}_{\tau (n)},\theta \rangle$  and  $\langle z_{\pi (1)},\theta \rangle \leq \ldots \leq \ldots \langle z_{\pi (n)},\theta \rangle$ ).
Non-negativity and finite value. From the  $\ell_2$  norm, it is derived
Symmetry.  $\mathbf{SWGG}_2^2 (\mu_1,\mu_2,\theta) = \frac{1}{n}\sum_i\| \pmb{x}_{\sigma (i)} - \pmb{y}_{\tau (i)}\| _2^2 = \frac{1}{n}\sum_i\| \pmb{y}_{\tau (i)} - \pmb{x}_{\sigma (i)}\| _2^2 =$  SWGG2(μ,μ,θ)
Identity property. From one side,  $\mu_1 = \mu_2$  implies that  $\langle \pmb{x}_i,\theta \rangle = \langle \pmb{y}_i,\theta \rangle ,\forall 1\leq i\leq n$  and that  $\sigma = \tau$  which implies  $\mathrm{SWGG}_2^2 (\mu_1,\mu_2,\theta) = 0$
From the other side,  $\mathrm{SWGG}_2^2 (\mu_1,\mu_2,\theta) = 0\Rightarrow \frac{1}{n}\sum \| \pmb{x}_{\sigma (i)} - \pmb{y}_{\tau (i)}\| _2^2 = 0\Rightarrow \pmb{x}_{\sigma (i)} = \pmb{y}_{\tau (i)},$ $\forall 1\leq i\leq n\Rightarrow \mu_1 = \mu_2.$
Triangle Inequality. We have  $\mathrm{SWGG}_2(\mu_1, \mu_2, \theta) = \left(\frac{1}{n} \sum_i \| \boldsymbol{x}_{\sigma(i)} - \boldsymbol{y}_{\tau(i)}\|_2^2\right)^{1/2}$
$$
\begin{array}{l} \leq \left(\sum_ {i} \| \boldsymbol {x} _ {\sigma (i)} - \boldsymbol {z} _ {\pi (i)} \| _ {2} ^ {2} + \sum_ {i} \| \boldsymbol {z} _ {\pi (i)} + \boldsymbol {y} _ {\tau (i)} \| _ {2} ^ {2}\right) ^ {1 / 2} \leq \left(\sum_ {i} \| \boldsymbol {x} _ {\sigma (i)} - \boldsymbol {z} _ {\pi (i)} \| _ {2} ^ {2}\right) ^ {1 / 2} + \\ \left(\sum_ {i} \left\| \boldsymbol {z} _ {\pi (i)} + \boldsymbol {y} _ {\tau (i)} \right\| _ {2} ^ {2}\right) ^ {1 / 2} = \operatorname {S W G G} _ {2} \left(\mu_ {1}, \mu_ {3}, \theta\right) + \operatorname {S W G G} _ {2} \left(\mu_ {3}, \mu_ {2}, \theta\right) \\ \end{array}
$$
Upper Bound The fact that  $\min\text{-SWGG}_2^2$  in an upper bound of  $W_2^2$  comes from the sub-optimality of the permutations  $\sigma_{\theta}, \tau_{\theta}$ . Indeed, they induce a one-to-one correspondence  $\boldsymbol{x}_{\sigma_{\theta}(i)} \to \boldsymbol{y}_{\tau_{\theta}(i)} \forall 1 \leq i \leq n$ . This correspondence corresponds to a transport map  $T^\theta$  such that  $T_\#^\theta \mu_1 = \mu_2$ . Since  $W_2^2 = \inf_{T \text{ s.t. } T_\# \mu_1 = \mu_2} \frac{1}{n} \sum \| \boldsymbol{x} - T(\boldsymbol{x}) \|_2^2$  we necessarily have  $W_2^2 \leq \min\text{-SWGG}_2^2$ .
Equality The equality  $W_2^2 = \min - \text{SWGG}_2^2$  whenever  $d > 2n$  comes from the fact that all the permutations are within the range of SWGG. In particular minimizing SWGG is equivalent to solve the Monge problem. We refer to Supp. 11.1 for more details.
7.4 Difference between max-SW and min-SWGG
Herein, we give an example where the selected vectors  $\theta$  for max-SW and min-SWGG differ.
Let  $\mu_1, \mu_2 \in \mathcal{P}(\mathbb{R}^2)$  be an empirical sampling of  $\mathcal{N}(m_1, \Sigma_1)$  and of  $\mathcal{N}(m_2, \Sigma_2)$  with  $m_1 = \begin{pmatrix} -10 \\ 0 \end{pmatrix}$ ,
$$
m _ {2} = \left( \begin{array}{c} 1 0 \\ 0 \end{array} \right), \Sigma_ {1} = \left( \begin{array}{c c} 1 & 0 \\ 0 & 1 1 \end{array} \right) \text {a n d} \Sigma_ {2} = \left( \begin{array}{c c} 2 & 0 \\ 0 & 2 \end{array} \right).
$$
Since these two distributions are far away on the  $x$ -coordinate, max-SW will catch this difference between the means by selecting  $\theta \approx \binom{1}{0}$ . Indeed, the projection on the  $x$ -coordinate represents the largest 1D WD.
Conversely, min-SWGG selects the pivot measure to be supported on  $\theta \approx \left( \begin{array}{c}1\\ 0 \end{array} \right)$  that separates the two distributions. Indeed, this direction better captures the geometry of the 2 distributions, delivering permutations that are well grounded to minimize the transport cost.
Fig. 6 illustrates that difference between max-SW and min-SWGG.
Figure 6: Optimal  $\theta$  for max-SW and min-SWGG
7.5 From permutations to transport map
In this section we provide the way of having a transport map from permutations.
Let  $\mu_1, \mu_2 \in \mathcal{P}_2^n(\mathbb{R}^d)$ , let  $\theta^* \in \arg \min \mathrm{SWGG}$  and let  $\sigma_{\theta^*}, \tau_{\theta^*}$  the associated permutations. The associated map must be  $T(\boldsymbol{x}_{\sigma(i)}) = \boldsymbol{y}_{\tau(i)} \forall 1 \leq i \leq n$ . In the paper, we formulate the associated transport map as:
$$
T \left(\boldsymbol {x} _ {i}\right) = \boldsymbol {y} _ {\tau_ {\theta^ {*}} ^ {- 1} \left(\sigma_ {\theta^ {*}} (i)\right)}, \quad \forall 1 \leq i \leq n. \tag {24}
$$
Moreover, the matrix representation of  $T$  is given by:
$$
T _ {i j} = \left\{ \begin{array}{l l} \frac {1}{n} & \text {i f} \sigma (i) = \tau (j) \\ 0 & \text {o t h e r w i s e} \end{array} \right. \tag {25}
$$
7.6 Examples of Transport Plan
Fig. 7 illustrates two instances of the transport plan obtained via min-SWGG. Even though these transport plans are not optimal, they were able to capture the overall structure of the true optimal transport plans.
Figure 7: Example of transports plan given by Wasserstein (left and middle-right) and min-SWGG (middle left and right). Transport plan distribution (top) and transport matrix (bottom). The relative distances between source and target are given in the title.
The first example shows that the OT plan by min-SWGG exhibits a "block" structure, and thus approximates well the true Wasserstein distance. The second example shows that even in a context of superimposed distribution the "general transport direction" in min-SWGG is representative of that of the optimal transport map.
8 Background on Wasserstein Generalized Geodesics
We introduce some concepts related the Wasserstein generalized geodesics in Sec. 4.1. In this section, we provide more details about these geodesics in order to provide a wider view on this theory.
In the following definitions, we do not address the issue of uniqueness of the geodesics. However this is not a problem in our setup since we focus our study on pivot measure with  $n$ -atoms  $\nu \in \mathcal{P}_2^n (\mathbb{R}^d)$ . In this case, we have uniqueness of the  $\nu$ -based Wasserstein distance [47].
Wasserstein generalized geodesics As mentioned in Sec. 4.1, Wasserstein generalized geodesics rely on a pivot measure  $\nu \in \mathcal{P}_2^n (\mathbb{R}^d)$  to transport  $\mu_{1}$  to  $\mu_{2}$ . Indeed, one can leverage the optimal transport maps  $T^{\nu \to \mu_1}$  and  $T^{\nu \to \mu_2}$  to construct a curve linking  $\mu_{1}$  to  $\mu_{2}$ . The generalized geodesic with pivot measure  $\nu$  is defined as:
$$
\mu_ {g} ^ {1 \rightarrow 2} (t) \stackrel {\text {d e f}} {=} ((1 - t) T ^ {\nu \rightarrow \mu_ {1}} + t T ^ {\nu \rightarrow \mu_ {2}}) _ {\#} \nu \quad \forall t \in [ 0, 1 ]. \tag {26}
$$
The generalized Wasserstein mean refers to the middle of the geodesic, i.e. when  $t = 0.5$  and has been denoted  $\mu_g^{1\to 2}$ .
Intuitively, the optimal transport maps between  $\nu$  and  $\mu_{i}, i = 1,2$  give rise to a sub-optimal transport map between  $\mu_{1}$  and  $\mu_{2}$  through:
$$
T _ {\nu} ^ {1 \rightarrow 2} \stackrel {\text {d e f}} {=} T ^ {\nu \rightarrow \mu_ {2}} \circ T ^ {\mu_ {1} \rightarrow \nu} \quad \text {w i t h} \quad \left(T _ {\nu} ^ {1 \rightarrow 2}\right) _ {\#} \mu_ {1} = \mu_ {2}. \tag {27}
$$
$T_{\nu}^{1 \to 2}$  links  $\mu_1$  to  $\mu_2$  via the generalized geodesic:
$$
\mu_ {g} ^ {1 \rightarrow 2} (t) = ((1 - t) I d + t T _ {\nu} ^ {1 \rightarrow 2}) _ {\#} \mu_ {1}. \tag {28}
$$
We recall here the  $\nu$ -based Wasserstein distance induced by  $T_{\nu}^{1\rightarrow 2}$  and introduced in Eq. (13).
Definition 8.1. The  $\nu$ -based Wasserstein distance [21, 47] is defined as:
$$
\begin{array}{l} W _ {\nu} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) \stackrel {\text {d e f}} {=} \int_ {\mathbb {R} ^ {d}} \| \boldsymbol {x} - T _ {\nu} ^ {1 \rightarrow 2} (\boldsymbol {x}) \| _ {2} ^ {2} d \mu_ {1} (\boldsymbol {x}) (29) \\ = \int_ {\mathbb {R} ^ {d}} \| T ^ {\nu \rightarrow \mu_ {1}} (z) - T ^ {\nu \rightarrow \mu_ {2}} (z) \| _ {2} ^ {2} d \nu (z). (30) \\ \end{array}
$$
Moreover, this new notion of geodesics comes with an inequality, which is of the opposite side to Eq. (3):
$$
W _ {2} ^ {2} \left(\mu_ {g} ^ {1 \rightarrow 2} (t), \nu\right) \leq (1 - t) W _ {2} ^ {2} \left(\mu_ {1}, \nu\right) + t W _ {2} ^ {2} (\nu , \mu_ {2}) - t (1 - t) W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right). \tag {31}
$$
The parallelogram law is not respected but straddles with eq. (3) and eq. (31). We refer to Figure 8 for an intuition behind positive curvature [51], parallelogram law and generalized geodesics.
Figure 8: Geodesic  $(tId + (1 - t)T^{1\rightarrow 2})_{\#}\mu_1$  and generalized geodesic  $(tId + (1 - t)T_{\nu}^{1\rightarrow 2})_{\#}\mu_1$  in Wasserstein space (Left and Right) in dashed line and parallelogram law in  $\mathbb{R}^d$  (middle).
Setting  $t = 0.5$  in Eq. (31) and reordering the term gives:
$$
W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) \leq 2 W _ {2} ^ {2} \left(\mu_ {1}, \nu\right) + 2 W _ {2} ^ {2} \left(\nu , \mu_ {2}\right) - 4 W _ {2} ^ {2} \left(\mu_ {g} ^ {1 \rightarrow 2}, \nu\right). \tag {32}
$$
Moreover one can remark that:
$$
W _ {\nu} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) = 2 W _ {2} ^ {2} \left(\mu_ {1}, \nu\right) + 2 W _ {2} ^ {2} \left(\nu , \mu_ {2}\right) - 4 W _ {2} ^ {2} \left(\mu_ {g} ^ {1 \rightarrow 2}, \nu\right) \tag {33}
$$
In particular situations  $W_{\nu}^{2}$  and  $W_{2}^{2}$  coincide. It is the case for 1D distributions where the Wasserstein space is known to be flat [4]. In that case, the Wasserstein mean and the generalized Wasserstein mean are the same.
Multi-marginal Another formulation of the  $\nu$ -based Wasserstein distance is possible through the perspective of multi-marginal OT [4]. Let  $\Pi(\mu_1, \mu_2, \nu) = \{\pi \text{ s.t. } P_{\#}^{12}\pi = \pi^{1 \to 2}, P_{\#}^{13}\pi = \pi^{1 \to \nu}\text{ and } P_{\#}^{23}\pi = \pi^{2 \to \nu}\}$ , where  $P^{ij}$  is the projection onto the coordinates  $i, j$ . Let also  $\Pi^*(\mu_i, \nu)$  be the space of optimal transport maps between  $\mu_i$  and  $\nu$ . We have:
$$
W _ {\nu} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) = \inf  _ {\pi \in \Pi \left(\mu_ {1}, \mu_ {2}, \nu\right) \text {s . t .} P _ {\#} ^ {i 3} \pi \in \Pi^ {*} \left(\mu_ {i}, \nu\right) i = 1, 2} \int_ {\mathbb {R} ^ {d}} \| \boldsymbol {x} - \boldsymbol {y} \| _ {2} ^ {2} d \pi (\boldsymbol {x}, \boldsymbol {y}) \tag {34}
$$
Equation (34) expresses the fact that we select the optimal plan from  $\Pi (\mu_1,\mu_2,\nu)$  which is already optimal for  $\Pi (\mu_i,\nu)$ . Mathematically, this minimization is not a multi-marginal problem, since the optimal plan is supposed to be already optimal for some coordinate.
The set  $\{\pi \in \Pi (\mu_1,\mu_2,\nu)$  s.t.  $P_{\#}^{i3}\pi \in \Pi^{*}(\mu_{i},\nu) i = 1,2\}$  is never empty, i.e. there is always existence of  $\pi_{\nu}^{1\to 2}$  (thanks to the gluing lemma [63], page 23). Moreover, in situations where it is a singleton, there is uniqueness of  $\pi_{\nu}^{1\to 2}$ . Uniqueness is an ingredient which overpasses the selection of a final coupling and comes with additional result.
Lemma 8.2 (Lemma 6 [47]). Whenever  $\{\pi \in \Pi (\mu_1,\mu_2,\nu)$  s.t.  $P_{\#}^{i3}\pi \in \Pi^{*}(\mu_{i},\nu)$ $i = 1,2\}$  is a singleton,  $W_{\nu}^{2}$  is a proper distance. It is a semi-distance otherwise.
Notably, 1D pivot measure was studied in [35] to ensure a dendritic structure of the distributions along the geodesic.
9 Related Works
In this section we highlight the fact that several upper approximations of  $W_2^2$  are in the framework of generalized geodesics. The differences lay in the choice of the pivot measure  $\nu$ .
Factored Coupling. In [29], the authors impose a low rank structure on the transport plan by factorizing the couplings through a pivot measure  $\nu$  expressed as the  $k$ -Wasserstein mean between  $\mu_{1}$  and  $\mu_{2}$  ( $k \leq n$ ). It is of particular interest since whenever the pivot distribution is the Wasserstein mean between  $\mu_{1}$  and  $\mu_{2}$ ,  $W_{\nu}^{2}$  and  $W_{2}^{2}$  coincide.
Factored coupling results in a problem of computing the  $k$ -Wasserstein mean  $(\mu^{1\rightarrow 2})$  followed by solving two OT problems between the clustered Wasserstein mean and the two input distributions  $(W_2^2 (\mu_1,\mu^{1\rightarrow 2})$  and  $W_{2}^{2}(\mu^{1\rightarrow 2},\mu_{2}))$ . Even though the OT problems are smaller, they are still expensive in practice.
Moreover, in this scenario, the uniqueness of the OT plan  $T_{\nu}^{1 \to 2}$  is not ensured. It appears that [29] chooses the most entropic transport plan, i.e. simply  $T_{\nu}^{1 \to 2} = T^{\mu^{1 \to 2} \to \mu_{2}} \circ T^{\mu_{1} \to \mu^{1 \to 2}}$ .
Subspace Detours. From a statistical point of view, it is beneficial to consider optimal transport on a lower dimensional manifold [66]. In [44], authors compute an optimal transport plan  $T^{\mu_1^E \to \mu_2^E}$  between projections on a lower linear subspace  $E$  of  $\mu_1$  and  $\mu_2$ , i.e.  $\mu_i^E = P_E \# \mu_i$ , where  $P_E$  is the linear projection on  $E$ . They aimed at leveraging  $T^{\mu_1^E \to \mu_2^E}$  to construct a sub-optimal map  $T_E^{1 \to 2}$  between  $\mu_1$  and  $\mu_2$ .
The problem can be recast as a generalized geodesic problem with  $\nu$  being the Wasserstein mean of  $\mu_1^E$  and  $\mu_2^E$  embedded in  $\mathbb{R}^d$ . Once again, uniqueness of  $T_{\nu}^{\mu_1 \to \mu_2}$  is not guaranteed, authors provide two ways of selecting the map, namely Monge-Knothe and Monge-Independent lifting.
Subspace detours result in a problem where one needs to select a linear subspace  $E$  (which is a non-convex procedure), compute an optimal transport between  $\mu_1$  and  $\mu_2$  (in  $\mathcal{O}(n^3\log n)$  whenever  $\dim(E) > 1$ ) and reconstruct  $T_E^{\mu_1 \to \mu_2}$ .
Linear Optimal Transport (LOT). Given a set of distributions  $(\mu_i)_{i=1}^m \in \mathcal{P}_2(\mathbb{R}^d)^m$ , LOT [65] embeds the set of distributions into the  $L^2(\nu)$ -space by computing the OT of each distribution to the pivot distribution. Mathematically, it computes  $T^{\nu \to \mu_i} \forall 1 \leq i \leq m$  and lies on estimating  $W_2^2(\mu_i, \mu_j)$  with  $W_\nu^2(\mu_i, \mu_j)$  through eq. (13).
In LOT, the pivot measure  $\nu$  was chosen to be the average of the input measures [65], the Lebesgue measure on  $\mathbb{R}^d$  [41] or an isotropic Gaussian distribution [43].
Instead of computing  $\binom{m}{2}$  expensive Wasserstein distances, it resorts only on  $m$  Wasserstein distances between  $(\mu_i)_i^m$  and  $\nu$ . While significantly reducing the computational cost when several distributions are at stake, it does not allow speeding up the computation when only two distributions are involved.
9.1 Linear Optimal Transport with shift and scaling
In this section, we recall the result from [43]. The theorem states that the  $\nu$ -based approximation is very close to WD whenever  $\mu_1, \mu_2$  are continuous distributions which are very close to be shift and scaling of each other. It can apply to a continuous version of SWGG, however it works with discrete measures in the particular case of equality between  $W_{\nu}^{2}$  and  $W_{2}^{2}$ .
Theorem 9.1 (Theorem 4.1 [43]). Let  $\Lambda = \{S_{a} (\text{shift}), a \in \mathbb{R}^{d}\} \cup \{R_{c} (\text{scaling}), c \in \mathbb{R}\}$ ,  $\Lambda_{\mu,R} = \{h \in \Lambda \text{s.t.} \| h \|_{\mu} \geq R\}$  and  $G_{\mu,R,\epsilon} = \{g \in L^{2}(\mathbb{R}^{d}, \mu) \text{s.t.} \exists h \in \Lambda_{\mu,R} \text{s.t.} \| g - h \|_{\mu} \leq \epsilon\}$
Let  $\nu, \mu \in \mathcal{P}_2(\mathbb{R}^d)$ , with  $\mu, \nu \ll \lambda$  (the Lebesgue measure). Let  $R > 0$ ,  $\epsilon > 0$
- For  $g_1, g_2 \in G_{\mu, R, \epsilon}$  and  $\nu = \lambda$  on a convex compact subset of  $\mathbb{R}^d$ , we have:
$$
W _ {\nu} \left(g _ {1 \#} \mu , g _ {2 \#} \mu\right) - W _ {2} \left(g _ {1 \#} \mu , g _ {2 \#} \mu\right) \leq C \epsilon^ {\frac {2}{1 5}} + 2 \epsilon \tag {35}
$$
- If  $\mu$  and  $\nu$  satisfy the assumption of Caffarelli's regularity theorem [14], then for  $g_{1}, g_{2} \in G_{\mu, R, \epsilon}$ , we have:
$$
W _ {\nu} \left(g _ {1 \#} \mu , g _ {2 \#} \mu\right) - W _ {2} \left(g _ {1 \#} \mu , g _ {2 \#} \mu\right) \leq \bar {C} \epsilon^ {1 / 2} + C \epsilon \tag {36}
$$
where  $C,\overline{C}$  depends on  $\nu ,\mu$  and  $R$
10 Proofs and other results related to Section 4
10.1 Proof of Proposition 4.2: equivalence between the two formulations of SWGG
In this section, we prove that the two definitions of SWGG in Def. 3.1 and Prop. 4.2 are equivalent. Let  $\theta \in \mathbb{S}^{d - 1}$  be fixed.
From one side in Def. 3.1, we have:
$$
\operatorname {S W G G} _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}, \theta\right) \stackrel {\text {d e f}} {=} \frac {1}{n} \sum_ {i} \| \boldsymbol {x} _ {\sigma_ {\theta} (i)} - \boldsymbol {y} _ {\tau_ {\theta} (i)} \| _ {2} ^ {2} \tag {37}
$$
where  $\sigma_{\theta}$  and  $\tau_{\theta}$  are the permutations obtained by sorting  $P_{\#}^{\theta}\mu_{1}$  and  $P_{\#}^{\theta}\mu_{2}$ .
From the other side we note  $D(\mu_1,\mu_2,\theta)$  the quantity:
$$
D \left(\mu_ {1}, \mu_ {2}, \theta\right) \stackrel {\text {d e f}} {=} 2 W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) + 2 W _ {2} ^ {2} \left(\mu_ {\theta} ^ {1 \rightarrow 2}, \mu_ {2}\right) - 4 W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {1 \rightarrow 2}, \mu_ {\theta} ^ {1 \rightarrow 2}\right). \tag {38}
$$
We want to prove that  $\mathrm{SWGG}_2^2 (\mu_1,\mu_2,\theta) = D(\mu_1,\mu_2,\theta),\quad \forall \mu_1,\mu_2\in \mathcal{P}_2^n (\mathbb{R}^d)$  and  $\theta \in \mathbb{S}^{d - 1}$ .
Eq. (13) in the main paper states that  $D(\mu_1, \mu_2, \theta)$  is equivalent to  $\int_{\mathbb{R}^d} \| \pmb{x} - T_{\mu_0^{1 \to 2}}^{1 \to 2}(\pmb{x}) \|_2^2 d\mu_1(\pmb{x})$ .
Finally, Lemma 4.6 states that the transport map  $T_{\mu_{\theta}^{1 \to 2}}^{1 \to 2}$  is fully determined by the permutations on the line: the projections part is a one-to-one correspondence between  $\mathbf{x}$  and  $\theta \langle \mathbf{x}, \theta \rangle$  (resp. between  $\mathbf{y}$  and  $\theta \langle \mathbf{y}, \theta \rangle$ ). More formally  $T_{\mu_{\theta}^{1 \to 2}}^{1 \to 2}(\mathbf{x}_{\sigma_{\theta}(i)}) = \mathbf{y}_{\tau_{\theta}(i)} \quad \forall 1 \leq i \leq n$ . And thus we recover:
$$
\int_ {\mathbb {R} ^ {d}} \| \boldsymbol {x} - T _ {\mu_ {\theta} ^ {1 \rightarrow 2}} ^ {1 \rightarrow 2} (\boldsymbol {x}) \| _ {2} ^ {2} d \mu_ {1} (\boldsymbol {x}) = \frac {1}{n} \sum_ {i} \| \boldsymbol {x} _ {\sigma_ {\theta} (i)} - \boldsymbol {y} _ {\tau_ {\theta} (i)} \| _ {2} ^ {2} \tag {39}
$$
10.2 Proof of Weak Convergence (Proposition 4.3)
We want to prove that, for a sequence of measures  $(\mu_k)_{k\in \mathbb{N}}\in \mathcal{P}_2^n (\mathbb{R}^d)$ , we have:
$$
\mu_ {k} \xrightarrow {\mathcal {L} , 2} \mu \in \mathcal {P} _ {2} ^ {n} (\mathbb {R} ^ {d})) \Longleftrightarrow \min  - S W G G _ {2} ^ {2} (\mu_ {k}, \mu) \underset {k} {\longrightarrow} 0 \tag {40}
$$