SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $f(\pi) = \langle \mathcal{L}(D,\bar{D})\otimes \pi ,\pi \rangle$ $q(\pi) = \mathbb{I}_{\{\pi \geq 0\}}(\pi)$ $g_{1}(\pi ,\alpha) = \tau_{1}d_{\mathbf{KL}}(\pi \mathbf{1}_{m},\alpha)$ $g_{2}(\pi ,\beta) =$ $\tau_2d_{\mathbf{KL}}(\pi^T\mathbf{1}_n,\beta),h_1(\alpha) = \mathbb{I}_{\{\alpha \in \Delta^n,d_{\mathbf{KL}}(\mu ,\alpha)\leq \rho_1\}}(\alpha),$  and  $h_2(\beta) = \mathbb{I}_{\{\beta \in \Delta^m,d_{\mathbf{KL}}(\nu ,\beta)\leq \rho_2\}}(\beta).$
The following theorem states that any limit point of the sequence generated by BPALM belongs to the critical point set of problem (3).
Theorem 3.3 (Subsequence Convergence). Suppose that in Problem (1), the step size  $t_k$  in (4) satisfies  $0 < \underline{t} \leq t_k < \bar{t} \leq \sigma / L_f$  for  $k \geq 0$  where  $\underline{t}, \bar{t}$  are given constants and  $L_f$  is the gradient Lipschitz constant of  $f$ . The step size  $c_k$  in (5) and  $r_k$  in (6) satisfy  $0 < \underline{r} \leq c_k, r_k < \bar{r}$  for  $k \geq 0$  where  $\underline{r}, \bar{r}$  are given constants. Any limit point of the sequence of solutions  $\{\pi^k, \alpha^k, \beta^k\}_{k \geq 0}$  belongs to the critical point set  $\mathcal{X}$ , where  $\mathcal{X}$  is defined by
$$
\left\{ \begin{array}{c} 0 \in f (\pi) + \partial q (\pi) + \nabla_ {\pi} g _ {1} (\pi , \alpha) + \nabla_ {\pi} g _ {2} (\pi , \beta), \\ 0 \in \nabla_ {\alpha} g _ {1} (\pi , \alpha) + \partial h _ {1} (\alpha), \\ 0 \in \nabla_ {\beta} g _ {2} (\pi , \beta) + \partial h _ {2} (\beta), \\ (\pi , \alpha , \beta) \in \mathbb {R} ^ {n \times m} \times \mathbb {R} ^ {n} \times \mathbb {R} ^ {m} \end{array} \right\}.
$$
For the sake of brevity, we omit the proof. We refer the reader to Appendix C for further details.
In this section, we present comprehensive experimental results to validate the effectiveness of our proposed RGW model and BPALM algorithm in various graph learning tasks, specifically subgraph alignment and partial shape correspondence. Traditionally, balanced GW has been applied successfully in scenarios where the source and target graphs have similar sizes. However, in our approach, we treat the missing part of the target graph as outliers and leverage RGW for improved performance. All simulations are conducted in Python 3.9 on a high-performance computing server running Ubuntu 20.10, equipped with an Intel(R) Xeon(R) Silver 4214R CPU. Our code is available at https://github.com/lmkong020/outlier-robust-GW.
4.1 Partial Shape Correspondence
In this subsection, we first investigate a toy matching problem in a 2D setting to support and validate our theoretical insights and results presented in Section 2. Figure 2 (a) illustrates an example where we aim to map a two-dimensional shape without symmetries to a rotated version of the same shape while accounting for outliers in the source domain. Here, we sample 300 points from the source shape and 400 points from the target shape. Additionally, we introduce 50 outliers by randomly adding points from a discrete uniform distribution on  $[-3, -2.5] \times [0, 0.5]$  to the source domain. The distance matrices,  $D$  and  $\bar{D}$  are computed using pairwise Euclidean distances.
Figures 2 provide visualizations of the coupling matrices and objective values for all the models, highlighting the matching results. In Figure 2(c), it is evident that even a small number of outliers has a significant impact on the coupling and leads to an increased estimated GW distance. While unbalanced GW and partial GW attempt to handle outliers to some extent, they fall short in achieving accurate mappings. On the other hand, our robust GW formulation, RGW, effectively disregards outliers and achieves satisfactory performance. Additionally, the objective value of RGW closely approximates the true GW distance without outliers, as indicated by Theorem 2.3, approaching zero.
Figure 2: (a): 2D shape geometry of the source and target; (b)-(f): visualization of ground truth and the matching results of balanced GW, unbalanced GW, partial GW, and robust GW.
We evaluate the matching performance of RGW on the TOSCA dataset [6, 34] for partial shape correspondence. For the initial estimation of the transport plan in both RGW and PGW, we utilize the partial functional map method [34], employing 30 eigenfunctions. This method establishes an initial matching relationship between the source and target shapes. This method establishes an initial matching relationship between the source and target shapes by setting  $\pi_{ik}$  to 1 for matching pairs  $(i,k)$  and 0 otherwise. The resultant transport plan is scaled by  $\| \pi \| _1 = \sum_{ij}\pi_{ij}$ . UGW is not suitable for this large-scale task due to its long execution time. As shown in Figure 3, RGW outperforms PGW, and it enhances the performance of the initial point.
4.2 Subgraph Alignment
The subgraph alignment problem, which involves determining the isomorphism between a query graph and a subgraph of a larger target graph, has been extensively studied [16, 20]. While the
Figure 3: (a): 3D shape geometry of the source and target; (b)-(e): visualization of ground truth, initial point obtained from the partial functional map, and the matching results of PGW and RGW.
restricted quadratic assignment problem is commonly used for graphs of similar sizes, the GW distance provides an optimal probabilistic correspondence that preserves the isometric property. In the subgraph alignment context, the nodes in the target graph, excluding those in the source graph, can be considered outliers, making the RGW model applicable to this task. In our comparison, we evaluate RGW against various methods, including unbalanced GW, partial GW, semi-relaxed GW (srGW) [42], RGWD [25], and methods for computing balanced GW such as FW [39], BPG [46], SpecGW [15], eBPG [36], and BAPG [24].
Database Statistics We evaluate the methods on synthetic and real databases. In the synthetic database, we generate target graphs  $\mathcal{G}_t$  using Barabasi-Albert models with scales ranging from 100 to 500 nodes. The source graphs  $\mathcal{G}_s$  are obtained by sampling connected subgraphs of  $\mathcal{G}_t$  with a specified percentage of nodes. This process results in five synthetic graph pairs for each setup, totaling 200 graph pairs. The Proteins and Enzymes biological graph databases from [15] are also used, following the same subgraph generation routine. For the Proteins database, we evaluate the accuracy of matching the overlap between two subgraphs with  $90\%$  overlap and between a subgraph and the entire graph, presented in the "Proteins-2" and "Proteins-1" columns, respectively. We compute the matching accuracy by comparing the predicted correspondence set  $S_{\mathrm{pred}}$  to the ground truth correspondence set  $S_{\mathrm{gt}}$ , with accuracy calculated as  $\mathrm{Acc} = |S_{\mathrm{gt}} \cap S_{\mathrm{pred}}| / |S_{\mathrm{gt}}| \times 100\%$ . In addition, we also evaluate our methods on the Douban Online-Offline social network dataset, which consists of online and offline graphs, representing user interactions and presence at social gatherings, respectively. The online graph includes all users from the offline graph, with 1,118 users serving as ground truth alignments. Node locations are used as features in both graphs. In line with previous works [18, 37], we gauge performance using the Hit@k metric, which calculates the percentage of nodes in set  $\mathcal{V}_t$  where the ground truth alignment includes  $\mathcal{V}_s$  among the top-k candidates.
Table 1: Comparison of the average matching accuracy (%) and wall-clock time (seconds) on subgraph alignment of  $50\%$  subgraph on datasets Synthetic, Proteins and Enzymes and Hit@1 and Hit@10 of dataset Douban.
Method	Synthetic	Proteins-1	Proteins-2	Enzymes	Douban
Acc	Time	Acc	Time	Acc	Time	Acc	Time	Hit@1	Hit@10
FW	2.27	18.39	16.00	27.05	26.15	60.34	15.47	9.57	17.97	51.07
SpecGW	1.78	3.72	12.06	11.07	42.64	12.85	10.69	3.96	2.68	9.83
eBPG	3.71	85.31	19.88	1975.12	32.15	9645.05	21.58	1219.81	0.08	0.53
BPG	15.41	24.67	29.30	118.26	61.26	80.39	32.49	70.42	72.72	92.39
BAPG	48.89	27.95	30.98	122.13	66.84	16.49	35.64	16.41	72.18	92.58
srGW	1.60	152.01	21.30	63.00	12.08	172.48	24.13	19.68	4.03	11.54
RGWD	16.68	955.40	27.94	4396.41	59.69	3586.56	30.35	2629.00	4.11	16.46
UGW	89.88	176.24	25.72	4026.93	67.30	1853.82	43.73	1046.29	0.09	0.72
PGW	2.28	479.99	13.94	544.79	20.08	348.44	11.43	212.09	18.24	37.03
RGW	94.44	361.44	53.30	834.76	69.38	466.91	63.43	293.84	75.58	96.24
(b) Proteins
(c) Enzymes
Figure 4: Graph Alignment performance for selected methods in relation to varying ratios of overlapping subgraphs on Synthetic, Protein, and Enzymes databases.
Parameters Setup We use unweighted symmetric adjacency matrices  $D$  and  $\bar{D}$  as input distance matrices. SpecGW employs the heat kernel  $(\exp(-L))$  with the normalized graph Laplacian matrix  $L$ . Both  $\mu$  and  $\nu$  are set as uniform distributions. For SpecGW, BPG, eBPG, BAPG, srGW, and RGWD, we follow their respective paper setups. FW is implemented using the default PythonOT package. UGW selects the best results from regularization parameter sets  $\{0.5, 0.2, 0.1, 0.01, 0.001\}$  and marginal penalty parameter sets  $\{0.1, 0.01, 0.001\}$ . PGW reports the best results in the range of transported mass from 0.1 to 0.9. RGW sets  $\tau_{1} = \tau_{2} = 0.1$  and selects the best results from the ranges  $\{0.05, 0.1, 0.2, 0.5\}$  for marginal relaxation parameters and  $\{0.01, 0.05, 0.1, 0.5, 1\}$  for step size parameters  $(t_{k}, c_{k}, r_{k})$ . The transport plan initialization uses different approaches depending on the dataset. For the Synthetic, Proteins, and Enzymes datasets, we employ the  $1_{n\times m} / (nm)$  approach. For the Douban Online-Offline dataset, we initialize the transport plan using classical optimal transport conducted on the feature space. Additionally, for all datasets, we initialize  $\alpha$  and  $\beta$  with uniform distributions.
Result of All Methods Table 1 and Figure 4 provide a comprehensive comparison of matching accuracy and computation time across various methods on the Synthetic, Proteins, and Enzymes datasets. Notably, RGW demonstrates superior accuracy compared to other methods, while maintaining comparable computation time with UGW and PGW. On the other hand, methods for computing the balanced GW exhibit poor performance on all datasets, particularly on the extensive Synthetic graph database. This can be attributed to their struggle in fulfilling the hard marginal constraint on the source side and addressing the presence of outliers. Furthermore, the introduction of local minima in the balanced GW problem due to partial source graph structures contributes to the suboptimal results. UGW performs well with low target graph partiality but suffers significant degradation as partiality increases. PGW, aimed at mitigating outlier impact, inadvertently affects the matching between clean samples by reducing the mass transported from the source domain. In addition, RGW achieved the best results with the highest Hit@1 and Hit@10 on the Douban Online-Offline dataset, improving performance from  $4.04\%$  and  $14.9\%$  respectively for the initial point created by features.
Selection of Hyperparameters  $\rho$  and  $\tau$  The hyperparameters  $\rho_{1},\rho_{2},\tau_{1}$  and  $\tau_{2}$  in our formulation serve distinct roles. Specifically,  $\rho_{1}$  and  $\rho_{2}$  control the extent of marginal relaxation, while  $\tau_{1}$  and  $\tau_{2}$  act as the marginal penalty parameters. As indicated by Theorem 2.3, we have the flexibility to fix the value of  $\tau$  and adjust  $\rho$  to accommodate outlier effects. To determine optimal hyperparameter values, we conduct two analyses. First, we examine the effect of varying  $\rho$  while maintaining a fixed subgraph node ratio and constant  $\tau$  on the Enzymes dataset. Second, we adjust  $\tau$  while holding  $\rho$  constant and assess the impact across the Synthesis, Proteins, and Enzymes datasets in the task of matching a  $50\%$
Figure 5: Sensitivity analysis of  $\rho$  with fixed  $\tau = 0.1$  on the Enzymes database, and sensitivity analysis of  $\tau$  with fixed  $\rho = 0.2$  on the Synthetic, Proteins, and Enzymes databases.
subgraph to the entire graph. In Figure 5, the left side demonstrates that while keeping the ratio and  $\tau$  fixed but varying  $\rho$ , accuracy diminishes when  $\rho$  becomes either excessively large or too small. However, within the range of 0.05 to 1, accuracy remains resilient, highlighting the mitigating effect of marginal relaxation on outlier influence. Notably, accuracy significantly improves when  $\rho$  falls within the range of 0.05 to 1 compared to the scenario with  $\rho = 0$ , thus affirming the significance of marginal relaxation in our model. A similar trend is observed for  $\tau$  on the right side of Figure 5, where maintaining a fixed ratio and  $\rho$  while adjusting  $\tau$  results in stable accuracy within the range of 0.1 to 1 but declining beyond this range. Consequently, in RGW computations, we can initially set  $\rho$  to 0.2 and  $\tau$  to 0.1 by default and subsequently adjust  $\rho$  based on the outlier ratio: increasing it for a large ratio and decreasing it for a small ratio.
4.3 Tightness of the Bound in Theorem 2.3
Our primary focus is on scenarios where the GW distance between clean samples is nearly zero or zero due to noise, such as in partial shape correspondence and subgraph alignment tasks. In such cases, it is possible to find an isometric mapping from the query subgraph to a portion of the entire graph. By appropriately selecting the value of  $\rho$ , as discussed in Section 2.2, the upper bound in RGW becomes the GW distance between clean samples, which is zero. As RGW is always nonnegative, this upper bound is tight in this context.
To empirically validate this, we conducted experiments on the toy example in Section 4.1, and Figure 6 (a) illustrates the function values of PGW, UGW, and RGW with varying outlier ratios. The results confirm that the value of RGW can remain close to zero as the ratio of outliers increases. Additionally, Figure 6 (b) shows the function value of RGW and its upper bound as  $\rho$  varies. Both the RGW value and its upper bound decrease, converging to zero as  $\rho$  increases. This observation provides empirical support for Theorem 2.3. Regarding UGW and its upper bound with changing  $\tau$ , we observed that both the UGW value and its upper bound increase as  $\tau$  becomes larger, as shown in Figure 6 (c). Unlike RGW, UGW's  $\tau$  must strike a balance between reducing outlier impact and preserving marginal distortion in the transport plan. This demands a careful balance and caution against setting  $\tau$  excessively close to zero, which could lead to over-relaxation and potentially deteriorate the performance.
(a) Values of PGW, UGW, and RGW
Figure 6: (a) Function values of PGW, UGW, and RGW for varying  $\epsilon$ ; (b) Function value of RGW and its upper bound for different  $\rho$ ; (c) Function value of UGW and corresponding upper bound for different  $\tau$ .
(b) RGW value and upper bound
(c) UGW value and upper bound
In this paper, we introduce RGW, a robust reformulation of Gromov-Wasserstein that incorporates distributionally optimistic modeling. Our theoretical analysis demonstrates its robustness to outliers, establishing RGW as a reliable estimator. We propose a Bregman proximal alternating linearized minimization method to efficiently solve RGW. Extensive numerical experiments validate our theoretical results and demonstrate the effectiveness of the proposed algorithm. Regarding the robust estimation of Gromov-Wasserstein, a natural question is whether we can recover the transport plan from the RGW model. On the computational side, our algorithm suffers from the heavy computation cost due to the use of the unbalanced OT as our subroutine, which limits its application in large-scale real-world settings. To address this issue, a natural future direction is to develop single-loop algorithms to leverage the model benefits of robust GW for real applications.
Anthony Man-Cho So is supported in part by the Hong Kong Research Grants Council (RGC) General Research Fund (GRF) project CUHK 14203920.
[1] Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. An optimistic perspective on offline reinforcement learning. In International Conference on Machine Learning, pages 104-114. PMLR, 2020.
[2] Masoud Ahoorkhosh, Le Thi Khanh Hien, Nicolas Gillis, and Panagiotis Patrinos. Multiblock Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization. Computational Optimization and Applications, 79(3):681-715, 2021.
[3] David Alvarez-Melis and Tommi S Jaakkola. Gromov-Wasserstein alignment of word embedding spaces. arXiv preprint arXiv:1809.00013, 2018.
[4] Yogesh Balaji, Rama Chellappa, and Soheil Feizi. Robust optimal transport with applications in generative modeling and domain adaptation. Advances in Neural Information Processing Systems, 33:12934-12944, 2020.
[5] Jérôme Bolte, Shoham Sabach, and Marc Teboulle. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming, 146(1):459-494, 2014.
[6] Alexander M Bronstein, Michael M Bronstein, and Ron Kimmel. Numerical geometry of non-rigid shapes. Springer Science & Business Media, 2008.
[7] Sébastien Bubeck, Nicolo Cesa-Bianchi, et al. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning, 5(1):1-122, 2012.
[8] Charlotte Bunne, David Alvarez-Melis, Andreas Krause, and Stefanie Jegelka. Learning generative models across incomparable spaces. In International Conference on Machine Learning, pages 851-861. PMLR, 2019.
[9] Junyu Cao and Rui Gao. Contextual decision-making under parametric uncertainty and data-driven optimistic optimization. Available at Optimization Online, 2021.
[10] Laetitia Chapel, Mokhtar Z Alaya, and Gilles Gasso. Partial optimal transport with applications on positive-unlabeled learning. Advances in Neural Information Processing Systems, 33:2903-2913, 2020.
[11] Mengjie Chen, Chao Gao, and Zhao Ren. Robust covariance and scatter matrix estimation under Huber's contamination model. The Annals of Statistics, 46(5):1932-1960, 2018.
[12] Sitan Chen, Frederic Koehler, Ankur Moitra, and Morris Yau. Online and distribution-free robustness: Regression and contextual bandits with huber contamination. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pages 684-695. IEEE, 2022.
[13] Lenaic Chizat, Gabriel Peyre, Bernhard Schmitzer, and François-Xavier Vialard. Scaling algorithms for unbalanced optimal transport problems. Mathematics of Computation, 87(314):2563-2609, 2018.
[14] Samir Chowdhury and Facundo Memoli. The Gromov-Wasserstein distance between networks and stable network invariants. Information and Inference: A Journal of the IMA, 8(4):757-787, 2019.
[15] Samir Chowdhury and Tom Needham. Generalized spectral clustering via Gromov-Wasserstein learning. In International Conference on Artificial Intelligence and Statistics, pages 712-720. PMLR, 2021.
[33] Emanuele Rodola, Alex M Bronstein, Andrea Albarelli, Filippo Bergamasco, and Andrea Torsello. A game-theoretic approach to deformable shape matching. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 182-189. IEEE, 2012.
[34] Emanuele Rodolà, Luca Cosmo, Michael M Bronstein, Andrea Torsello, and Daniel Cremers. Partial functional correspondence. In Computer graphics forum, volume 36, pages 222-236. Wiley Online Library, 2017.
[35] Thibault Séjourne, François-Xavier Vialard, and Gabriel Peyré. The unbalanced Gromov Wasserstein distance: Conic formulation and relaxation. Advances in Neural Information Processing Systems, 34, 2021.
[36] Justin Solomon, Gabriel Peyre, Vladimir G Kim, and Suvrit Sra. Entropic metric alignment for correspondence problems. ACM Transactions on Graphics (ToG), 35(4):1-13, 2016.
[37] Jianheng Tang, Weiqi Zhang, Jiajin Li, Kangfei Zhao, Fugee Tsung, and Jia Li. Robust attributed graph alignment via joint structure learning and optimal transport. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 1638-1651, 2023.
[38] Jianheng Tang, Kangfei Zhao, and Jia Li. A fused Gromov-Wasserstein framework for unsupervised knowledge graph entity alignment. In Findings of the Association for Computational Linguistics: ACL 2023, pages 3320-3334.
[39] Vayer Titouan, Nicolas Courty, Romain Tavenard, and Rémi Flamary. Optimal transport for structured data with application on graphs. In International Conference on Machine Learning, pages 6275-6284. PMLR, 2019.
[40] Quang Huy Tran, Hicham Janati, Nicolas Courty, Rémi Flamary, Ievgen Redko, Pinar Demetci, and Ritambhara Singh. Unbalanced CO-Optimal transport. In Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI), 2023.
[41] Cédric Vincent-Cuaz, Titouan Vayer, Rémi Flamary, Marco Corneli, and Nicolas Courty. Online graph dictionary learning. In International Conference on Machine Learning, pages 10564-10574. PMLR, 2021.
[42] Cédric Vincent-Cuaz, Rémi Flamary, Marco Corneli, Titouan Vayer, and Nicolas Courty. Semi-relaxed Gromov Wasserstein divergence with applications on graphs. In International Conference on Learning Representations (ICLR), 2022.
[43] Hongteng Xu, Jiachang Liu, Dixin Luo, and Lawrence Carin. Representing graphs via Gromov-Wasserstein factorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[44] Hongteng Xu, Dixin Luo, and Lawrence Carin. Scalable Gromov-Wasserstein learning for graph partitioning and matching. Advances in Neural Information Processing Systems, 32, 2019.
[45] Hongteng Xu, Dixin Luo, Lawrence Carin, and Hongyuan Zha. Learning graphons via structured Gromov-Wasserstein barycenters. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10505-10513, 2021.
[46] Hongteng Xu, Dixin Luo, Hongyuan Zha, and Lawrence Carin Duke. Gromov-Wasserstein learning for graph matching and node embedding. In International Conference on Machine Learning, pages 6932-6941. PMLR, 2019.
We organize the appendix as follows:
- The proof details of Theorem 2.3 and discussion of Remark 2.4 are given in Section B.
- The proof details of the algorithm, including the properties of  $p$ , the convergence analysis of Newton's method, and Bregman proximal alternating linearized minimization method are collected in C.
Additional experiment results are summarized in Section D.
B Proof of Robust Guarantee in Section 2.2
B.1 Proof of Theorem 2.3
Proof.  $\mathrm{GW}_{\rho_1,\rho_2}^{\mathrm{rob}}(\mu ,\nu)$  is defined as
$$
\begin{array}{l} \mathbf {G W} _ {\rho_ {1}, \rho_ {2}} ^ {\mathrm {r o b}} (\mu , \nu) = \inf  _ {\alpha \in \mathcal {P} (X), \beta \in \mathcal {P} (Y)} \inf  _ {\pi \in \mathcal {M} ^ {+} (X \times Y)} \iint | d _ {X} (x, x ^ {\prime}) - d _ {Y} (y, y ^ {\prime})) | ^ {2} d \pi (x, y) d \pi (x ^ {\prime}, y ^ {\prime}) + \\ \tau_ {1} d _ {\mathbf {K L}} \left(\pi_ {1}, \alpha\right) + \tau_ {2} d _ {\mathbf {K L}} \left(\pi_ {2}, \beta\right) \\ \end{array}
$$
$$
\begin{array}{l l} \text {s . t .} & d _ {\mathbf {K L}} (\mu , \alpha) \leq \rho . \end{array}
$$
Consider  $\pi_c$ , the optimal transport plans for  $\mathrm{GW}(\mu_c, \nu_c)$ . Notably,  $\pi_c$  serves as a feasible solution for  $\mathrm{GW}_{\rho_1, \rho_2}^{\mathrm{rob}}(\mu, \nu)$ . Consequently, we can deduce that:
$$
\mathbf {G W} _ {\rho_ {1}, \rho_ {2}} ^ {\mathrm {r o b}} (\mu , \nu) \leq \inf  _ {\alpha \in \mathcal {P} (X), \beta \in \mathcal {P} (Y)} \iint | d _ {X} (x, x ^ {\prime}) - d _ {Y} (y, y ^ {\prime})) | ^ {2} d \pi_ {c} (x, y) d \pi_ {c} (x ^ {\prime}, y ^ {\prime}) +
$$