SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The notation  $\mu_k \xrightarrow{\mathcal{L}, 2} \mu$  stands for the weak convergence in  $\mathcal{P}_2^n(\mathbb{R}^d)$  i.e.  $\int_{\mathbb{R}^d} f(\pmb{x}) d\mu_k(\pmb{x}) \to \int_{\mathbb{R}^d} f(\pmb{x}) d\mu(\pmb{x})$  for all continuous bounded functions  $f$  and for the Euclidean distance  $f(\pmb{x}) = \| \pmb{x}_0 - \pmb{x} \|_2^2$  for all  $x_0 \in \mathbb{R}^d$ .
From one side, if  $\min\text{-SWGG}_2^2(\mu_k, \mu) \to 0 \implies W_2^2(\mu_k, \mu) \to 0 \implies \mu_k \xrightarrow{\mathcal{L}, 2} \mu$ . The first implication is due to the fact that  $\min\text{-SWGG}_2^2$  is an upper-bounds of  $W_2^2$ , the Wasserstein distance, and that WD metrizes the weak convergence.
From another side, assume  $\mu_k \xrightarrow{\mathcal{L}, 2} \mu$ ; we have for any  $\theta$ :
1. Let  $\mu_{\theta}^{\mu_k \to \mu} \in \mathcal{P}_2^n(\mathbb{R}^d)$  stands for the Wasserstein mean of the projections  $Q_{\#}^{\theta} \mu_k$  and  $Q_{\#}^{\theta} \mu_k$  and let  $\mu_{\theta}^{\mu \to \mu} = Q_{\#}^{\theta} \mu$ . We have  $\mu_{\theta}^{\mu_k \to \mu}$  converges towards (in law) to  $\mu_{\theta}^{\mu \to \mu}$ , which implies that:
$$
W _ {2} ^ {2} \left(\mu_ {k}, \mu_ {\theta} ^ {\mu_ {k} \rightarrow \mu}\right) \underset {k} {\longrightarrow} W _ {2} ^ {2} \left(\mu , \mu_ {\theta} ^ {\mu \rightarrow \mu}\right). \tag {41}
$$
2. Since  $\mu \in \mathcal{P}_2^n (\mathbb{R}^d)$ , we have  $T^{\mu_{\theta}^{\mu_k\to \mu}\to \mu_k}\xrightarrow[k]{\longrightarrow}T^{\mu_{\theta}^{\mu_k\to \mu}\to \mu}$  (see [22], theorem 3.2). It implies that  $\mu_{g,\theta}^{\mu_k\to \mu}\xrightarrow{\mathcal{L}}\mu$  and particularly:
$$
W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {\mu_ {k} \rightarrow \mu}, \mu_ {\theta} ^ {\mu_ {k} \rightarrow \mu}\right) \underset {k} {\longrightarrow} W _ {2} ^ {2} (\mu , \mu_ {\theta} ^ {\mu \rightarrow \mu}) \tag {42}
$$
By combining the previous elements, we get:
$$
\begin{array}{l} 2 W _ {2} ^ {2} \left(\mu_ {k}, \mu_ {\theta} ^ {\mu_ {k} \rightarrow \mu}\right) + 2 W _ {2} ^ {2} \left(\mu_ {\theta} ^ {\mu_ {k} \rightarrow \mu}, \mu_ {k}\right) - 4 W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {\mu_ {k} \rightarrow \mu}, \mu_ {\theta} ^ {\mu_ {k} \rightarrow \mu}\right) \xrightarrow [ k ]{} 2 W _ {2} ^ {2} \left(\mu , \mu_ {\theta} ^ {\mu \rightarrow \mu}\right) \\ + 2 W _ {2} ^ {2} \left(\mu_ {\theta} ^ {\mu \rightarrow \mu}, \mu\right) \\ - 4 W _ {2} ^ {2} \left(\mu , \mu_ {\theta} ^ {\mu \rightarrow \mu}\right) = 0 \tag {43} \\ \end{array}
$$
The previous relation shows that  $\mu_k \xrightarrow{\mathcal{L}, 2} \mu$  implies  $\mathrm{SWGG}_2^2(\mu_k, \mu, \theta) \xrightarrow[k]{0}$  for any  $\theta$ . Hence, we can conclude that:
$$
\mu_ {k} \xrightarrow {\mathcal {L} , 2} \mu \Longrightarrow \min  - \mathrm {S W G G} _ {2} ^ {2} (\mu_ {k}, \mu) \rightarrow 0 \tag {44}
$$
Note that when  $\mu_{1}$  and  $\mu_{2}$  are continuous, [41] proved that when the distributions are smooth enough (i.e. respecting the Cafarelli theorem [14]), there is a bi-Holder equivalence between the  $\nu$ -based Wasserstein distance and  $W_{2}^{2}$ . Hence, it still holds for SWGG for any  $\theta \in S^{d - 1}$ :
$$
W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) \leq \operatorname {S W G G} _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}, \theta\right) \leq B \times W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) ^ {2 / 1 5} \quad \forall \mu_ {i} \in \mathcal {P} _ {2} \left(\mathbb {R} ^ {d}\right) \tag {45}
$$
where  $B$  depends on  $\mu_i, i \in \{1,2\}, \theta$  and the dimension  $d$ . This bound is sufficient to prove that SWGG metrizes the weak convergence in this context. We refer to [41] for more details.
10.3 Proof of Translation property (Proposition 4.4)
We prove that  $\mathrm{min - SWGG}_2^2$  has the same behavior w.r.t. the translation as  $W_2^2$ . This property is well known for Wasserstein and useful in applications such as shape matching.
Let  $\mu_1, \mu_2 \in \mathcal{P}_2^n(\mathbb{R}^d)$ , and let  $T^u$  (resp.  $T^v$ ) be the map  $\pmb{x} \mapsto \pmb{x} - \pmb{u}$  (resp.  $\pmb{x} \mapsto \pmb{x} - \pmb{v}$ ), with  $\pmb{u}, \pmb{v}$  vectors of  $\mathbb{R}^d$ .
To ease the notations, let define  $\tilde{\mu}_1 = T_{\#}^u\mu_1$  and  $\tilde{\mu}_2 = T_{\#}^v\mu_2$
Let remind that in the case of Wasserstein distance we have [53](Remark 2.19):
$$
W _ {2} ^ {2} \left(\tilde {\mu} _ {1}, \tilde {\mu} _ {2}\right) \stackrel {\text {d e f}} {=} W _ {2} ^ {2} \left(T _ {\#} ^ {u} \mu_ {1}, T _ {\#} ^ {v} \mu_ {2}\right) = W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) - 2 \langle \boldsymbol {u} - \boldsymbol {v}, \boldsymbol {m} _ {1} - \boldsymbol {m} _ {2} \rangle + \| \boldsymbol {u} - \boldsymbol {v} \| _ {2} ^ {2} \tag {46}
$$
with  $\pmb{m}_1 = \int_{\mathbb{R}^d}\pmb {x}d\mu_1(\pmb {x})$  and  $\pmb {m}_2 = \int_{\mathbb{R}^d}\pmb {x}d\mu_2(\pmb {x})$
We aim to compute  $\min\text{-SWGG}_2^2(\tilde{\mu}_1, \tilde{\mu}_2) \stackrel{\text{def}}{=} \min\text{-SWGG}_2^2(T_\#^u \mu_1, T_\#^v \mu_2)$ . Let express first
$$
\mathrm {S W G G} _ {2} ^ {2} \left(\tilde {\mu} _ {1}, \tilde {\mu} _ {2}\right) = 2 W _ {2} ^ {2} \left(\tilde {\mu} _ {1}, \tilde {\mu} _ {\theta} ^ {1 \rightarrow 2}\right) + 2 W _ {2} ^ {2} \left(\tilde {\mu} _ {2}, \tilde {\mu} _ {\theta} ^ {1 \rightarrow 2}\right) - 4 W _ {2} ^ {2} \left(\tilde {\mu} _ {g, \theta} ^ {1 \rightarrow 2}, \tilde {\mu} _ {\theta} ^ {1 \rightarrow 2}\right) \tag {47}
$$
where  $\tilde{\mu}_{\theta}^{1\rightarrow 2}$  is the Wasserstein mean of the projections along  $\theta$  of the shifted measures  $\tilde{\mu}_1 = T_\#^u\mu_1$  and  $\tilde{\mu}_2 = T_\#^v\mu_2$  as in Proposition 2. The generalized Wasserstein mean  $\tilde{\mu}_{g,\theta}^{1\rightarrow 2}$  is defined accordingly (see also Proposition 11).
$$
W _ {2} ^ {2} \left(\tilde {\mu} _ {1}, \tilde {\mu} _ {\theta} ^ {1 \rightarrow 2}\right) = W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) - 2 \langle \boldsymbol {u}, \boldsymbol {m} _ {1} - \boldsymbol {m} _ {3} \rangle + \| \boldsymbol {u} \| _ {2} ^ {2} \tag {48}
$$
where  $\pmb{m}_3 = \int_{\mathbb{R}^d}\pmb {x}d\tilde{\mu}_\theta^{1\to 2}(\pmb {x})$
Similarly  $W_2^2 (\tilde{\mu}_2,\tilde{\mu}_\theta^{1\to 2}) = W_2^2 (\mu_2,\mu_\theta^{1\to 2}) - 2\langle \pmb {v},\pmb {m}_2 - \pmb {m}_3\rangle +\| \pmb {v}\| _2^2.$
Let express now the third term in eq. (47). For that we require to define the generalized Wasserstein mean  $\tilde{\mu}_{g,\theta}^{1\rightarrow 2}$  with pivot measure  $\tilde{\mu}_{\theta}^{1\rightarrow 2}$ . By the virtue of eq. (11) in the main paper, we have:
$$
\begin{array}{l} \tilde {\mu} _ {g, \theta} ^ {1 \rightarrow 2} = \left(\frac {1}{2} T ^ {\tilde {\mu} _ {\theta} ^ {1 \rightarrow 2} \rightarrow \tilde {\mu} _ {1}} + \frac {1}{2} T ^ {\tilde {\mu} _ {\theta} ^ {1 \rightarrow 2} \rightarrow \tilde {\mu} _ {2}}\right) _ {\#} \tilde {\mu} _ {\theta} ^ {1 \rightarrow 2} (49) \\ = \left(\frac {1}{2} T ^ {\mu_ {\theta} ^ {1 \rightarrow 2} \rightarrow \mu_ {1}} + \frac {1}{2} T ^ {\mu_ {\theta} ^ {1 \rightarrow 2} \rightarrow \mu_ {2}} - T ^ {\frac {u + v}{2}}\right) _ {\#} \tilde {\mu} _ {\theta} ^ {1 \rightarrow 2} (50) \\ = T _ {\#} ^ {\frac {u + v}{2}} \left(\left(\frac {1}{2} T ^ {\mu_ {\theta} ^ {1} \rightarrow 2 \rightarrow \mu_ {1}} + \frac {1}{2} T ^ {\mu_ {\theta} ^ {1} \rightarrow 2 \rightarrow \mu_ {2}}\right) _ {\#} \mu_ {\theta} ^ {1 \rightarrow 2}\right) (51) \\ \end{array}
$$
Hence, the third term in (47) is:
$$
W _ {2} ^ {2} \left(\tilde {\mu} _ {g, \theta} ^ {1 \rightarrow 2}, \tilde {\mu} _ {\theta} ^ {1 \rightarrow 2}\right) = W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {1 \rightarrow 2}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) - 2 \left\langle \frac {\boldsymbol {u} + \boldsymbol {v}}{2}, \frac {\boldsymbol {m} _ {1} + \boldsymbol {m} _ {2}}{2} - \boldsymbol {m} _ {3} \right\rangle + \left\| \frac {\boldsymbol {u} + \boldsymbol {v}}{2} \right\| _ {2} ^ {2} \tag {52}
$$
since the mean of a Wasserstein mean is the mean of  $m_{1}, m_{2}$ .
Putting all together, we have:
$$
\min  - \operatorname {S W G G} _ {2} ^ {2} \left(T _ {\#} ^ {u} \mu_ {1}, T _ {\#} ^ {v} \mu_ {2}\right) = \min  - \operatorname {S W G G} _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) - 4 \langle \boldsymbol {u}, \boldsymbol {m} _ {1} - \boldsymbol {m} _ {3} \rangle - 4 \langle \boldsymbol {v}, \boldsymbol {m} _ {2} - \boldsymbol {m} _ {3} \rangle \tag {53}
$$
$$
+ 8 \left\langle \frac {\boldsymbol {u} + \boldsymbol {v}}{2}, \frac {\boldsymbol {m} _ {1} + \boldsymbol {m} _ {2}}{2} - \boldsymbol {m} _ {3} \right\rangle
$$
$$
+ 2 \| \boldsymbol {u} \| _ {2} ^ {2} + 2 \| \boldsymbol {v} \| _ {2} ^ {2} - 4 \left\| \frac {\boldsymbol {u} + \boldsymbol {v}}{2} \right\| _ {2} ^ {2}
$$
$$
= \min  - S W G G _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) + 4 \langle \boldsymbol {u} + \boldsymbol {v}, \boldsymbol {m} _ {3} \rangle \tag {54}
$$
$$
- 4 \langle \boldsymbol {u} + \boldsymbol {v}, \boldsymbol {m} _ {3} \rangle - 4 \langle \boldsymbol {u}, \boldsymbol {m} _ {1} \rangle - 4 \langle \boldsymbol {v}, \boldsymbol {m} _ {2} \rangle
$$
$$
+ 4 \langle \boldsymbol {u} + \boldsymbol {v}, \boldsymbol {m} _ {1} + \boldsymbol {m} _ {2} \rangle + \| \boldsymbol {u} - \boldsymbol {v} \| _ {2} ^ {2}
$$
(Parallelogram law)
$$
= \min  - S W G G _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) - 2 \langle \boldsymbol {u}, \boldsymbol {m} _ {1} \rangle - 2 \langle \boldsymbol {v}, \boldsymbol {m} _ {2} \rangle + 2 \langle \boldsymbol {u}, \boldsymbol {m} _ {2} \rangle + 2 \langle \boldsymbol {v}, \boldsymbol {m} _ {1} \rangle \tag {55}
$$
$$
+ \| \boldsymbol {u} - \boldsymbol {v} \| _ {2} ^ {2}
$$
$$
= \min  - S W G G _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) - 2 \langle \boldsymbol {u} - \boldsymbol {v}, \boldsymbol {m} _ {1} - \boldsymbol {m} _ {2} \rangle + \| \boldsymbol {u} - \boldsymbol {v} \| _ {2} ^ {2} \tag {56}
$$
10.4 Proof of the new closed form of the Wasserstein distance (Lemma 4.6)
We recall and prove the lemma that makes explicit a new closed form for WD. Let  $\mu_1, \mu_2$  be in  $\mathcal{P}_2^n(\mathbb{R}^d)$  with  $\mu_2$  a distribution supported on a line whose direction is  $\theta \in \mathbb{S}^{d-1}$ . We have:
$$
W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) = W _ {2} ^ {2} \left(\mu_ {1}, Q _ {\#} ^ {\theta} \mu_ {1}\right) + W _ {2} ^ {2} \left(Q _ {\#} ^ {\theta} \mu_ {1}, \mu_ {2}\right). \tag {57}
$$
Moreover, the optimal map is given by  $T^{1\to 2} = T^{Q_{\#}^{\theta}\mu_{1}\to \mu_{2}}\circ T^{\mu_{1}\to Q_{\#}^{\theta}\mu_{1}} = T^{Q_{\#}^{\theta}\mu_{1}\to \mu_{2}}\circ Q^{\theta}$
Let  $\mu_1, \mu_2$  be in  $\mathcal{P}_2^n(\mathbb{R}^d)$  with  $\mu_2$  a distribution supported on a line of direction  $\theta$ . We have:
$$
W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) = W _ {2} ^ {2} \left(\mu_ {1}, Q _ {\#} ^ {\theta} \mu_ {1}\right) + W _ {2} ^ {2} \left(Q _ {\#} ^ {\theta} \mu_ {1}, \mu_ {2}\right) \tag {58}
$$
Moreover, the optimal map is given by:
$$
T ^ {1 \rightarrow 2} = T ^ {Q _ {\#} ^ {\theta} \mu_ {1} \rightarrow 2} \circ T ^ {1 \rightarrow Q _ {\#} ^ {\theta} \mu_ {1}} = T ^ {Q _ {\#} ^ {\theta} \mu_ {1} \rightarrow 2} \circ Q ^ {\theta} \tag {59}
$$
Here  $Q^{\theta}$  is given in Def. 4.1 of the paper.
The proof of the Lemma was first inspired by [13](Proposition 2.3), where authors show that  $W_{C}^{2}(\mu_{1},\mu_{2}) = W_{C^{1}}^{2}(\mu_{1},\mu) + W_{C^{2}}^{2}(\mu ,\mu_{2})$ , with  $C^1,C^2$  and  $C$  some cost matrices with the constraints  $C_{ij} = \min_s C_{is}^1 +C_{sj}^2$ .
Let  $\mu_{1} = \frac{1}{n}\sum \delta_{\pmb{x}_{i}}$  and  $\mu_{2} = \frac{1}{n}\delta_{\overline{\mathbf{y}}_i}$  be in  $\mathcal{P}_2^n (\mathbb{R}^d)$  with  $\mu_{2}$  a distribution supported on a line with direction  $\theta$ . Let  $Q_{\#}^{\theta}\mu_{1} = \overline{\mu}_{1} = \frac{1}{n}\sum \delta_{\overline{\mathbf{x}}_{i}}\in \mathcal{P}_{2}^{n}(\mathbb{R}^{d})$ . We emphasize here the fact that the atoms of  $\overline{\mu}_{1}$  and  $\mu_{2}$  are supported on a line are denoted by the overline symbol.
From one side, we have:
$$
\begin{array}{l} W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) = \inf  _ {T ^ {1} \text {s . t .} T _ {\#} ^ {1} \mu_ {1} = \mu_ {2}} \int_ {\mathbb {R} ^ {d}} \| \boldsymbol {x} - T ^ {1} (\boldsymbol {x}) \| _ {2} ^ {2} d \mu_ {1} (\boldsymbol {x}) (60) \\ = \inf  _ {T ^ {1} \text {s . t .} T _ {\#} ^ {1} \mu_ {1} = \mu_ {2}} \int_ {\mathbb {R} ^ {d}} \left(\| \boldsymbol {x} - Q ^ {\theta} (\boldsymbol {x}) \| _ {2} ^ {2} + \| Q ^ {\theta} (\boldsymbol {x}) - T ^ {1} (\boldsymbol {x}) \| _ {2} ^ {2}\right) d \mu_ {1} (\boldsymbol {x}) (61) \\ = \int_ {\mathbb {R} ^ {d}} \| \boldsymbol {x} - Q ^ {\theta} (\boldsymbol {x}) \| _ {2} ^ {2} d \mu_ {1} (\boldsymbol {x}) + \inf  _ {T ^ {1} \text {s . t .} T _ {\#} ^ {1} \mu_ {1} = \mu_ {2}} \int_ {\mathbb {R} ^ {d}} \| Q ^ {\theta} (\boldsymbol {x}) - T ^ {1} (\boldsymbol {x}) \| _ {2} ^ {2} d \mu_ {1} (\boldsymbol {x}) (62) \\ \geq \inf  _ {T ^ {2} \text {s . t .} T _ {\#} ^ {2} \mu_ {1} = \bar {\mu} _ {1}} \int_ {\mathbb {R} ^ {d}} \| \boldsymbol {x} - T ^ {2} (\boldsymbol {x}) \| _ {2} ^ {2} d \mu_ {1} (\boldsymbol {x}) + \inf  _ {T ^ {3} \text {s . t .} T _ {\#} ^ {3} \bar {\mu} _ {1} = \mu_ {2}} \int_ {\mathbb {R} ^ {d}} \| \bar {\boldsymbol {x}} - T ^ {3} (\bar {\boldsymbol {x}}) \| _ {2} ^ {2} d \bar {\mu} _ {1} (\bar {\boldsymbol {x}}) (63) \\ \geq W _ {2} ^ {2} \left(\mu_ {1}, \bar {\mu} _ {1}\right) + W _ {2} ^ {2} \left(\bar {\mu} _ {1}, \mu_ {2}\right) (64) \\ \end{array}
$$
Equation (61) is obtained thanks to the Pythagorean theorem since  $\langle \pmb{x}_i, Q^\theta(\pmb{x}_i), \overline{\pmb{y}_i} \rangle$  is a right triangle  $\forall 1 \leq i \leq n$ . The equation (64) is obtained by taking the inf of the previous first term of the previous equation.
From the other side:
$$
\begin{array}{l} W _ {2} ^ {2} \left(\mu_ {1}, \bar {\mu} _ {1}\right) + W _ {2} ^ {2} \left(\bar {\mu} _ {1}, \mu_ {2}\right) = \int_ {\mathbb {R} ^ {d}} \| \overline {{\boldsymbol {x}}} - T ^ {3} (\overline {{\boldsymbol {x}}}) \| _ {2} ^ {2} d \bar {\mu} _ {1} (\overline {{\boldsymbol {x}}}) + \int_ {\mathbb {R} ^ {d}} \| \overline {{\boldsymbol {x}}} - T ^ {4} (\overline {{\boldsymbol {x}}}) \| _ {2} ^ {2} d \bar {\mu} _ {1} (\overline {{\boldsymbol {x}}}) (65) \\ = \int_ {\mathbb {R} ^ {d}} \| T ^ {3} (\bar {\boldsymbol {x}}) - T ^ {4} (\bar {\boldsymbol {x}}) \| _ {2} ^ {2} d \bar {\mu} _ {1} (\bar {\boldsymbol {x}}) (66) \\ = W _ {\bar {\mu} _ {1}} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) \geq W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}\right) (67) \\ \end{array}
$$
Where  $T^3$  and  $T^4$  are the optimal plan of  $W_2^2(\mu_1, \overline{\mu}_1)$  and  $+W_2^2(\overline{\mu}_1, \mu_2)$ . Similarly, (65) is obtained via the Pythagorean theorem. This concludes the proof.
We plot an illustration of the lemma in Figure 9.
Figure 9: Closed form for Wasserstein with Pythagorus theorem
10.5 Details on the efficient computation of SWGG
We decompose the second formulation of SWGG. Let first remind that  $Q^{\theta}:\mathbb{R}^{d}\to \mathbb{R}^{d},\pmb {x}\mapsto \theta \langle \pmb {x},\theta \rangle$  and  $P^\theta :\mathbb{R}^d\to \mathbb{R},\pmb {x}\mapsto \langle \pmb {x},\theta \rangle$  are the projections on the subspace generated by  $\theta$
$$
\mathrm {S W G G} _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}, \theta\right) = 2 W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) + 2 W _ {2} ^ {2} \left(\mu_ {\theta} ^ {1 \rightarrow 2}, \mu_ {2}\right) - 4 W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {1 \rightarrow 2}, \mu_ {\theta} ^ {1 \rightarrow 2}\right). \tag {68}
$$
First, by lemma 4.6,
$$
2 W _ {2} ^ {2} \left(\mu_ {1}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) = 2 W _ {2} ^ {2} \left(\mu_ {1}, Q _ {\#} ^ {\theta} \mu_ {1}\right) + 2 W _ {2} ^ {2} \left(P _ {\#} ^ {\theta} \mu_ {1}, P _ {\#} ^ {\theta} \mu_ {\theta} ^ {1 \rightarrow 2}\right) \tag {69}
$$
as  $\mu_{\theta}^{1\rightarrow 2}$  s support is on a line. Similarly,
$$
2 W _ {2} ^ {2} \left(\mu_ {2}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) = 2 W _ {2} ^ {2} \left(\mu_ {2}, Q _ {\#} ^ {\theta} \mu_ {2}\right) + 2 W _ {2} ^ {2} \left(P _ {\#} ^ {\theta} \mu_ {2}, P _ {\#} ^ {\theta} \mu_ {\theta} ^ {1 \rightarrow 2}\right). \tag {70}
$$
$$
- 4 W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {1 \rightarrow 2}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) = - 4 W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {1 \rightarrow 2}, Q _ {\#} ^ {\theta} \mu_ {g, \theta} ^ {1 \rightarrow 2}\right) - 4 W _ {2} ^ {2} \left(P _ {\#} ^ {\theta} \mu_ {g, \theta} ^ {1 \rightarrow 2}, P _ {\#} ^ {\theta} \mu_ {\theta} ^ {1 \rightarrow 2}\right). \tag {71}
$$
We notice that  $2W_{2}^{2}(P_{\#}^{\theta}\mu_{1},P_{\#}^{\theta}\mu_{\theta}^{1\rightarrow 2}) + 2W_{2}^{2}(P_{\#}^{\theta}\mu_{\theta}^{1\rightarrow 2},P_{\#}^{\theta}\mu_{2}) = W_{2}^{2}(P_{\#}^{\theta}\mu_{1},P_{\#}^{\theta}\mu_{2})$  (as  $P_{\#}^{\theta}\mu_{\theta}^{1\rightarrow 2}$  is the Wasserstein mean between  $P_{\#}^{\theta}\mu_{1}$  and  $P_{\#}^{\theta}\mu_{2}$ ). We also notice that  $-4W_{2}^{2}(P_{\#}^{\theta}\mu_{g,\theta}^{1\rightarrow 2},P_{\#}^{\theta}\mu_{\theta}^{1\rightarrow 2}) = 0$  (it comes from the fact that the generalized Wasserstein mean is induced by the permutations on the line), we can put all together to have:
$$
\operatorname {S W G G} _ {2} ^ {2} \left(\mu_ {1}, \mu_ {2}, \theta\right) = 2 W _ {2} ^ {2} \left(\mu_ {1}, Q _ {\#} ^ {\theta} \mu_ {1}\right) + 2 W _ {2} ^ {2} \left(\mu_ {2}, Q _ {\#} ^ {\theta} \mu_ {2}\right) - 4 W _ {2} ^ {2} \left(\mu_ {g, \theta} ^ {1 \rightarrow 2}, Q _ {\#} ^ {\theta} \mu_ {g, \theta} ^ {1 \rightarrow 2}\right) + W _ {2} ^ {2} \left(P _ {\#} ^ {\theta} \mu_ {1}, P _ {\#} ^ {\theta} \mu_ {2}\right) \tag {72}
$$
One can show that SWGG is divided into 3 Wasserstein distances between a distribution and its projections on a line and 1D Wasserstein problem. This results in a very fast computation of SWGG.
10.6 Smoothing of SWGG
In this section, we give details on the smoothing procedure of min-SWGG, an additional landscape of SWGG and its smooth counterpart SWGG and an empirical heuristic for setting hyperparameters  $s$  and  $\epsilon$ .
Smoothing Procedure. A natural surrogate would be to add an entropic regularization within the definition of  $T^{\mu_{\theta}^{1 \to 2} \to \mu_i}$ ,  $i \in \{1, 2\}$  and to solve an additional optimal transport problem. Nevertheless, it would lead to an algorithm with an  $\mathcal{O}(n^2)$  complexity. Instead, we build upon the blurred Wasserstein distance [26] between two distributions  $\nu_1$  and  $\nu_2$ :
$$
B _ {\epsilon} ^ {2} (\nu_ {1}, \nu_ {2}) \stackrel {\mathrm {d e f}} {=} W _ {2} ^ {2} (k _ {\epsilon / 4} * \nu_ {1}, k _ {\epsilon / 4} * \nu_ {2})
$$
where  $*$  denotes the smoothing (convolution) operator and  $k_{\epsilon /4}$  is the Gaussian kernel of deviation  $\sqrt{\epsilon} /2$ . In our case, it resorts in making  $s$  copies of each sorted projections  $P^{\theta}(\pmb {x}_i)$  and  $P^{\theta}(\pmb {y}_i)$  respectively, to add a Gaussian noise of deviation  $\sqrt{\epsilon} /2$  and to compute averages of sorted blurred copies  $\pmb{x}_{\sigma^s}^s$ $\pmb{y}_{\tau^s}^s$ :
$$
\left(\widetilde {\mu_ {\theta} ^ {1 \rightarrow 2}}\right) _ {i} = \frac {1}{2 s} \sum_ {k = (i - 1) s + 1} ^ {i s} \boldsymbol {x} _ {\sigma^ {s} (k)} ^ {s} + \boldsymbol {y} _ {\tau^ {s} (k)} ^ {s}. \tag {73}
$$
Further, we provide additional examples of the landscape of min-SWGG  $(\mu_1,\mu_2)$  and discuss how to choose empirically relevant  $s$  and  $\epsilon$  values.
[26] has shown that the blurred WD has the same asymptotic properties as the Sinkhorn divergence, with parameter  $\epsilon$  the strength of the blurring: it interpolates between WD (when  $\epsilon \rightarrow 0$ ) and a degenerate constant value (when  $\epsilon \rightarrow \infty$ ).
To find a minimum of Eq. (16) in the paper (i.e.  $\widetilde{\mathrm{SWGG}}_2^2 (\mu_1,\mu_2,\theta)$ ), we iterate over:
$$
\theta_ {t + 1} = \theta_ {t} + \eta \nabla_ {\theta} \widetilde {\mathbf {S W G G}} _ {2} ^ {2} (\mu_ {1}, \mu_ {2}, \theta)
$$
$$
\theta_ {t + 1} = \theta_ {t + 1} / \| \theta_ {t + 1} \| _ {2}
$$
where  $\eta \in \mathbb{R}_+$  is the learning rate. This procedure converges towards a local minima with a complexity of  $\mathcal{O}(snd + sn\log (sn))$  for each iteration. Once the optimal direction  $\theta^{\star}$  is found, the final solution resorts to be the solution provided by  $\mathrm{SWGG}_2^2 (\mu_1,\mu_2,\theta^\star)$ , where the induced optimal transport map is an unblurred matrix.
Heuristic for setting the hyperparameters of SWGG We here provide an heuristic for setting parameters  $s$  (number of copies of each points) and  $\epsilon$  (strength of the blurring). We then give an example of the behavior of SWGG w.r.t. these hyper parameters.
Let  $\mu_{1} = \frac{1}{n}\sum \delta_{\pmb{x}_{i}}$  and  $\mu_{2} = \frac{1}{n}\sum \delta_{\pmb{y}_{i}}$
-  $s \in \mathbb{N}_{+}$  represents the number of copies of each sample. We observe empirically that the quantity  $sn$  should be large to provide a smooth landscape. It means that the  $s$  values can be small when  $n$  increases, allowing to keep a competitive algorithm (as the complexity depends on  $ns$ )
-  $\epsilon \in \mathbb{R}_{+}$  represents the variance of the blurred copies of each sample. Empirically,  $\epsilon$  should depend on the variance of the distributions projected on the line. Indeed, an  $\epsilon$  very close to zero will not smooth enough the discontinuities whereas a large  $\epsilon$  will give a constant landscape.
As discussed in Section 4.3, finding an optimal  $\theta \in \mathbb{S}^{d - 1}$  is a non convex problem and provides a discontinuous loss function. We give some examples of the landscape of SWGG w.r.t. different values of the hyperparameters in Fig. 10. The landscapes were computed with a set of projections  $\theta$  regularly sampled with angles  $\in [0,2\pi ]$
We observe that the larger  $s$ , the smoother  $\widetilde{\mathrm{SWGG}}$ . Additionally, raising  $\epsilon$  tends to flatten  $\widetilde{\mathrm{SWGG}}$  w.r.t.  $\theta$  (erasing local minima). Indeed similarly to Sinkhorn, a large  $\epsilon$  blurred the transport plan and thus homogenize all the value of SWGG w.r.t.  $\theta$ .
Moreover, we empirically observe that the number of samples for  $\mu_{1}$  and  $\mu_{2}$  enforces the continuity of SWGG. We then conjecture that the discontinuities of SWGG are due to artifact of the sampling and thus the smoothing operation erases this unwanted behavior. A full investigation of this assumption is left for future work.
Figure 10: Non-convex landscapes for SWGG and SWGG with different hyper parameters.
10.7 Inconsequential of the pivot measure
Importantly, only the direction  $\theta$  is of importance for the value of SWGG. Indeed, whenever  $\nu \in \mathcal{P}_2^n (\mathbb{R}^d)$  is supported on a line of direction  $\theta$ , the position of the atoms is irrelevant for  $W_{\nu}$  and the associated transport plan whenever the atoms are distinct. Despite the fact that the pivot measure is inconsequential for the value of SWGG (at  $\theta$  fixed), we choose it to be  $\mu_{\theta}^{1\rightarrow 2}$ . This choice is supported by the fact that  $\mu_{\theta}^{1\rightarrow 2}$  can be efficiently computed (as a 1D Wasserstein mean) and that some computation can be alleviated:
$$
2 W _ {2} ^ {2} \left(Q _ {\#} ^ {\theta} \mu_ {1}, \mu_ {\theta} ^ {1 \rightarrow 2}\right) + 2 W _ {2} ^ {2} \left(\mu_ {\theta} ^ {1 \rightarrow 2}, Q _ {\#} ^ {\theta} \mu_ {2}\right) = W _ {2} ^ {2} \left(Q _ {\#} ^ {\theta} \mu_ {1}, Q _ {\#} ^ {\theta} \mu_ {2}\right) \tag {74}
$$
It is an important comment to derive the property of distance for SWGG; it also allows minimizing SWGG over  $\theta \in \mathbb{S}^{d - 1}$  without consideration for  $\nu$ , since any choice of  $\nu$  supported on the subspace generated by  $\theta$  give the same result for min-SWGG. This property of irrelevance comes from the
nature of the subspace where  $\nu$  is supported, which is uni-dimensional. More formally we give the following proposition and its associated proof.