SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Beining Yang $^{1,2}$ , Kai Wang $^{3}$ , Qingyun Sun $^{1,2†}$ , Cheng Ji $^{1,2}$ , Xingcheng Fu $^{1,2}$ , Hao Tang $^{4}$ , Yang You $^{3}$ , Jianxin Li $^{1,2‡}$
$^{1}$ School of Computer Science and Engineering, Beihang University
$^{2}$ Advanced Innovation Center for Big Data and Brain Computing, Beihang University
$^{3}$ National University of Singapore  $^{4}$ Carnegie Mellon University
Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have attracted increasing concerns. Existing graph condensation methods primarily focus on optimizing the feature matrices of condensed graphs while overlooking the impact of the structure information from the original graphs. To investigate the impact of the structure information, we conduct analysis from the spectral domain and empirically identify substantial Laplacian Energy Distribution (LED) shifts in previous works. Such shifts lead to poor performance in cross-architecture generalization and specific tasks, including anomaly detection and link prediction. In this paper, we propose a novel Structure-broadcasting Graph Dataset Distillation (SGDD) scheme for broadcasting the original structure information to the generation of the synthetic one, which explicitly prevents overlooking the original structure information. Theoretically, the synthetic graphs by SGDD are expected to have smaller LED shifts than previous works, leading to superior performance in both cross-architecture settings and specific tasks. We validate the proposed SGDD across 9 datasets and achieve state-of-the-art results on all of them: for example, on the YelpChi dataset, our approach maintains  $98.6\%$  test accuracy of training on the original graph dataset with 1,000 times saving on the scale of the graph. Moreover, we empirically evaluate there exist  $17.6\% \sim 31.4\%$  reductions in LED shift crossing 9 datasets. Extensive experiments and analysis verify the effectiveness and necessity of the proposed designs. The code is available in the https://github.com/RingBDStack/SGDD.
Graphs have been applied in many research areas and achieved remarkable results, including social networks [27, 67, 73, 79], physical [16, 3, 57, 10], and chemical interactions [2, 93, 111, 80]. Graph neural networks (GNNs), a classical and wide-studied graph representation learning method [40, 84, 98, 63], is proposed to extract information via modeling the features and structures from the given graph. Nevertheless, the computational and memory costs are extremely heavy when training on a large graph [48, 89, 99]. One of the most straightforward ideas is to reduce the redundancy of the large graph. For example, graph sparsification [66, 77] and coarsening [53, 52, 46, 75] are proposed to achieve this goal by dropping redundant edges and grouping similar nodes. These methods have shown promising results in reducing the size and complexity of large graphs while preserving their essential properties.
*Equal contribution (yangbeining@buaa.edu.cn, kai.wang@comp.nus.edu.sg).
†Project lead sunqy@buaa.edu.cn
$^{\ddagger}$ Corresponding author lijx@buaa.edu.cn.
(a) Condense scheme of GCond
(b)  $\mathbf{A}'$  of GCond
(c) Cross-arch. Acc. of GCond
(d) Condense scheme of SGDD
(e)  $\mathbf{A}'$  of SGDD
(f) Cross-arch. Acc. of SGDD
Figure 1: (a) and (d) illustrate that the pipelines of SGDD and GCond [37]. One can find that SGDD broadcast A from the original graph to the generation of  $\mathbf{A}'$  while GCond synthesizes  $\mathbf{A}'$  via the operation  $f(\cdot)$  (e.g., pair-wise feature similarity [74]) on  $\mathbf{X}'$ . We also show the condensed graph and its shift coefficient (SC) of GCond and SGDD in (b) and (e), respectively. Note that we introduce  $SC$  as an approximation of the LED shift. We show the cross-architecture performance of GCond and SGDD in (c) and (f), 1, 2, 3, 4, and 5 denote APPNP [42], Cheby [13], GCN [41], SAGE [26], and SGC [92], more cross-architecture results can be found in Tab. 2.
However, graph sparsification and coarsening heavily rely on heuristics [5] (e.g., the largest principle eigenvalues [78], pairwise distances [66, 81]), which may lead to poor generalization on various architectures or tasks and sub-optimal for the downstream GNNs training [37]. Most recently, as shown in Fig. 1(a), GCond [37] follows the vision dataset distillation methods [88, 105, 24, 103] and proposes to condense the original graph by the gradient matching strategy. The feature of the synthetic graph dataset is optimized by minimizing the gradient differences between training on original and synthetic graph datasets. Then, the synthetic feature is fed into the function  $f(\mathbf{X}^{\prime})$  (e.g., pair-wise feature similarity [74]) to obtain the structure of the graph. The synthetic graph dataset (including feature and structure) is expected to preserve the task-relevant information and achieve comparable results as training on the original graph dataset at an extremely small cost.
Although the gradient matching strategy has achieved significant success in graph dataset condensation, most of these works [37, 36, 50, 109] follow the previous vision dataset distillation methods to synthesize the condensed graphs, which results in the following limitations: 1) In Fig. 1(b), we can find that the original graph structure information is not well preserved in the condensed graph. That's because they build the structure  $(\mathbf{A}^{\prime})$  upon the learned feature  $(\mathbf{X}^{\prime})$  (Fig. 1(a)), which may cause the loss of original structure information. 2) In vision dataset distillation, applying the gradient matching strategy may result in an entanglement between the synthetic dataset and the architecture that is used for condensing [106, 104, 39]. This can detrimentally impact their performance, especially when it comes to generalizing to unseen architectures [22]. This issue is compounded when dealing with graph data that comprise both features and structures. As shown in Fig. 1(c) and Tab. 2, GCond [37] explicitly displays inferior performance, demonstrating that generating structure only based on the feature degrades the generalization of the condensed graph.
To investigate the relation between the structure of the condensed graph and the performance of training on it, we follow previous works [82, 1, 13, 78] to analyze it from a spectral view. Specifically, we explore the relationship between the Laplacian Energy Distribution (LED) shift [82, 25, 12] and the generalization performance of the condensed graph. We empirically find a positive correlation between LED shift and the performance in cross-architecture settings. To address these issues, we introduce a novel Structure-broadcasting Graph Dataset Distillation (SGDD) scheme to condense the original graph dataset by broadcasting the original adjacency matrix to the generation of the synthetic one, named SGDD, which explicitly prevents overlooking the original structure information. As shown in Fig. 1(d), we propose the graphon approximation to broadcast the original structure  $\mathbf{A}$  as supervision to the generation of condensed graph structure  $\mathbf{A}'$ . Then, we optimize  $\mathbf{A}'$  by minimizing
the optimal transport distance of these two structures. For  $\mathbf{X}^{\prime}$ , we follow the [37] to synthesize the feature of the condensed graph. The  $\mathbf{A}^{\prime}$  and  $\mathbf{X}^{\prime}$  are jointly optimized in a bi-level loop. Both theoretical analysis (Sec. 3.2) and empirical studies (Fig. 1(c) and 1(f)) consistently show that our method reduces the LED shift significantly  $(0.46\rightarrow 0.18)$ .
We further evaluate the impact of LED shifts and conduct experiments in the cross-architecture setting. Comparing Fig. 1(f), 1(c), and Tab. 2, the improvement of SGDD is up to  $11.3\%$ . To evaluate the generalization of our method, we extend SGDD to node classification, anomaly detection, and link prediction tasks. SGDD achieves state-of-the-art results on these tasks in most cases. Notably, we obtain new state-of-the-art results on YelpChi and Amazon with  $9.6\%$  and  $7.7\%$  improvements, respectively. Our main contributions can be summarized as follows:
- Based on the analysis of the difference between graph and vision dataset distillation, we introduce SGDD, a novel framework for graph dataset distillation via broadcasting the original structure information to the generation of a condensed graph.
- In SGDD, the graphon approximation provides a novel scheme to broadcast the original structure information to condensed graph generation, and the optimal transport is proposed to minimize the LED shifts between original and condensed graph structures.
- SGDD effectively reduces the LED shift between the original and the condensed graphs, consistently surpassing the performance of current state-of-the-art methods across a variety of datasets.
Dataset Distillation & Dataset Condensation. Dataset distillation (DD) [88, 106, 24, 51, 110, 23, 86, 49, 87, 6, 7] aims to distill a large dataset into a smaller but informative synthetic one. The proposed method imposes constraints on the synthetic samples by minimizing the training loss difference, while ensuring that the samples remain informative. This technique is useful for applications such as continual learning [105, 106, 39, 44, 104], as well as neural architecture search [61, 62, 96]. Recently, GCond [37] is proposed to reduce a large-scale graph to a smaller one for node classification using the gradient matching scheme in DC [105]. Unlike GCond [37], who build the structure only through the learned feature, we explicitly broadcast the original graph structure to the generations to prevent the overlooking of the original graph structure information.
Graph Coarsening & Graph Sparsification. Graph Coarsening [53, 52, 14] follows the intuition that nodes in the original graph can naturally group the similar node to a super-nodes. Graph Sparsification [66, 38, 78] is aimed at reducing the edges in the original graph, as there are many redundant relations in the graphs. We can simply sum up both two methods by trying to reduce the "useless" component in the graph. Nevertheless, these methods primarily rely on unsupervised techniques such as the largest  $k$  eigenvalue [66] or the multilevel incomplete LU factorization [14], which cannot guarantee the behavior of the synthetic graph in downstream tasks. Moreover, they fail to reduce the graph size to an extremely small degree (e.g., reduce the number of nodes to  $0.1\%$  of the original). In contrast, graph condensation can aggregate information into a smaller yet informative graph in a supervised way, thereby overcoming the aforementioned limitations.
3 Preliminary and Analysis
3.1 Formulation of graph condensation
Consider a graph dataset  $\mathcal{G} = \{\mathbf{A},\mathbf{X},\mathbf{Y}\}$ , where  $\mathbf{A} \in \mathbb{R}^{N \times N}$  denotes the adjacency matrix,  $\mathbf{X} \in \mathbb{R}^{N \times d}$  is the feature, and  $\mathbf{Y} \in \mathbb{R}^{N \times 1}$  represents the labels.  $S = \{\mathbf{A}',\mathbf{X}',\mathbf{Y}'\}$  is defined as the synthetic dataset, the first dimension of  $\mathbf{A}'$ ,  $\mathbf{X}'$ , and  $\mathbf{Y}'$  are  $N' (N' \ll N)$ .
The goal of graph condensation is achieving comparable results as training on the original graph dataset  $\mathcal{G}$  via training on the synthetic one  $\mathcal{S}$ . The optimization process can be formulated as follow,
$$
\mathcal {S} ^ {*} = \arg \min  _ {\mathcal {S}} \mathrm {M} \left(\theta_ {\mathcal {S}} ^ {*}, \theta_ {\mathcal {G}} ^ {*}\right) \quad \text {s . t} \quad \theta_ {t} ^ {*} = \arg \min  _ {\theta} \operatorname {L o s s} \left(\mathrm {G N N} _ {\theta} (t)\right), \tag {1}
$$
where  $\mathrm{GNN}_{\theta}(t)$  denotes a GNN parameterized with  $\theta, \theta_{S}$  and  $\theta_{\mathcal{G}}$  are the parameters that are trained on  $S$  and  $\mathcal{G}$ , and  $t \in \{\mathcal{S}, \mathcal{G}\}$ ,  $\mathrm{M}(\cdot)$  denotes a matching function and  $\mathrm{Loss}(\cdot)$  is the loss function.
(a) LEDs in various architectures
(b) LEDs in BWGNN [82]
Figure 2: (a): Illustrations of the LEDs (use the density plotting, each peak represents the LED of a graph), SC, and Avg. (average test performances on GCN, SGC, APPNP, and SAGE) in various architectures. (b): Evaluation of these three metrics in frequency-adaptive BWGNN. We empirically find that the LED shifts (i.e., SC) are high-consistent with Avg. performances, thus SC could be a good indicator to evaluate the performance of the condensed graph.
3.2 Analysis of the structure of the condensed graph and its effects
In this section, we first introduce the definition of Laplacian Energy Distribution (LED), then analyze the LED in previous work, and finally explore its influence on generalization performance.
Laplacian Energy Distribution. Following the previous works [25, 12, 82], we introduce Laplacian energy distribution (LED) to analyze the structure of graph data. Given an adjacency matrix  $\mathbf{A} \in \mathbb{R}^{N \times N}$ , the Laplacian matrix  $\mathbf{L}$  is defined as  $\mathbf{D} - \mathbf{A}$ , where the  $\mathbf{D}$  is the degree matrix of  $\mathbf{A}$ . We utilize the normalized Laplacian matrix  $\tilde{\mathbf{L}} = \mathbf{I} - \mathbf{D}^{-1/2} \mathbf{A} \mathbf{D}^{-1/2}$ , where  $\mathbf{I}$  is an identity matrix. The eigenvalues of  $\tilde{\mathbf{L}}$  are then defined as  $\lambda_1, \ldots, \lambda_n, \ldots, \lambda_N$  by ascending order, i.e.,  $\lambda_1 \leq \dots \leq \lambda_n \leq \dots \leq \lambda_N$ , with orthonormal eigenvectors  $\mathbf{U} = (\mathrm{u}_1, \dots, \mathrm{u}_n, \dots, \mathrm{u}_N)$  correspond to these eigenvalues.
Definition 1 (Laplacian Energy Distribution [25, 12, 82]). Let  $\mathbf{X} = (x_{1},\dots x_{n},\dots x_{N})^{\top}\in \mathbb{R}^{N\times d}$  be the feature of graph, we have  $\hat{\mathbf{X}} = (\hat{x}_1\dots \hat{x}_n\dots \hat{x}_N)^\top = \pmb {U}^\top \mathbf{X}$  as the post-Graph-FourierTransform of  $\mathbf{X}$ . The formulation of LED is defined as:
$$
\eta_ {n} (\mathbf {X}, \tilde {\mathbf {L}}) = \frac {\hat {x} _ {n} ^ {2}}{\sum_ {i = 1} ^ {N} \hat {x} _ {i} ^ {2}} = \bar {x _ {n}}. \tag {2}
$$
LED analysis of previous work. First, we define  $\eta^{\mathcal{G}}$  and  $\eta^S$  as the LED of the original graph and condensed graph, respectively. Then the LED shift of the above two graphs can be formulated as:  $||\eta^{\mathcal{G}} - \eta^{\mathcal{S}}|| = ||\sum_{i=1}^{N}\eta_i(\mathbf{X},\tilde{\mathbf{L}}) - \sum_{j=1}^{N'}\eta_j(\mathbf{X}',\tilde{\mathbf{L}}')||$ . In GCond [37], the matching function M defaults as the MSE loss. Therefore, the objective of GCond can be written as  $||\theta_{\mathcal{G}} - \theta_{\mathcal{S}}|| \leq \epsilon$ , where  $\epsilon$  is a small number as expected. Incorporating such an objective in the LED shift formula, the lower bound is shown as follows.
Proposition 1. Refer to [1, 4, 82], the GNN can be recognized as the bandpass filter. Assume the frequency response area of GNN is  $(a,b)$ , where  $(a,b)$  is architecture-specific. The lower bound of GCond is shown in Eq. (3). Detailed proof can be found in Appendix B.1.
$$
\left| \left| \eta^ {\mathcal {G}} - \eta^ {\mathcal {S}} \right| \right| \geq \epsilon + \sum_ {i = 1} ^ {a} \left| \left| \bar {x} _ {i} ^ {2} - \bar {x} _ {i} ^ {' 2} \right| \right| + \sum_ {j = b} ^ {N ^ {\prime}} \left| \left| \bar {x} _ {j} ^ {2} - \bar {x} _ {j} ^ {' 2} \right| \right|. \tag {3}
$$
According to Eq. (3), we find the lower bond of LED in GCond is related to the frequency response area  $(a, b)$  (i.e., specific GNN). For example, GCN [40] or SGC [92] (utilized in GCond) is a low-pass filter [1], which emphasizes the lower Laplacian energy. As shown in Fig. 1(a), the  $\mathbf{A}'$  is built upon the learned "low-frequency" feature  $\mathbf{X}'$ , which may fail to generalize to cross-architectures (i.e., high-pass filters) and specific tasks.
Exploration of the effects of LED shift on the generalization performance. Although the LED shift phenomenon occurs in GCond, quantifying the LED shift is challenging because  $\mathcal{G}$  and  $\mathcal{S}$  have
Figure 3: The Training pipeline of the SGDD (left). We first fix  $\mathbf{A}'$  to optimize  $\mathbf{X}'$  through the gradient matching strategy and we broadcast the supervision of  $\mathbf{A}$  to the generation of the graph structure  $\mathbf{A}'$ . To mitigate the Laplacian Energy Distribution (LED) shift phenomenon, we propose the LED Matching strategy to optimize the  $\mathbf{A}'$ , which optimizes the learned structure with the optimal transport (OT) distance (right).
different numbers of nodes. To enable comparison, we follow an intuitive assumption that two nodes with similar eigenvalue distribution proportions can be aligned in comparison. Thus, we first convert LEDs of  $\mathcal{G}$  and  $S$  into probability distributions using the Kernel Density Estimation (KDE) method [60]. Then we quantify the LED shifts as the distance of the probability distributions using the Jensen-Shannon (JS) divergence [47]. We define the LED shift coefficient  $(SC)$  in Definition 2:
Definition 2 (LED shift coefficient,  $SC$ ). The LED shift coefficient between  $\mathcal{G}$  and  $S$  is:
$$
S C = \mathrm {J S} \left(\frac {1}{| V _ {\mathcal {G}} | h} \sum_ {\bar {x} _ {i} \in X _ {\mathcal {G}}} K \left(\frac {x - \bar {x} _ {i}}{h}\right) \Big | \Big | \frac {1}{| V _ {\mathcal {S}} | h} \sum_ {\bar {x} _ {j} \in X _ {\mathcal {S}}} K \left(\frac {x - \bar {x} _ {j}}{h}\right)\right), \tag {4}
$$
where the  $\mathrm{JS}(\cdot |\cdot)$  denotes the JS-divergence, the  $|V_{\mathcal{G}}|$  and the  $|V_{\mathcal{S}}|$  is the number of the nodes to corresponding graphs, the  $K(\cdot)$  represents the Gaussian kernel function with bandwidth parameter  $h$ .  $SC\in [0,1]$  reflects the divergence between  $\mathcal{G}$  and  $\mathcal{S}$  (a smaller  $SC$  indicates more similar).
In Fig. 2, we empirically study the influences of  $SC$  in two settings: various GNNs in Fig. 2(a) and fixed BWGNN [82] with adaptive bandpass in Fig. 2(b). Based on the results in Fig. 2, We have several observations: (1) The entangled learning paradigm that building structure (i.e., adjacency matrix) upon feature matrix will significantly lead to the LED shift phenomenon. (2) The positive correlation exists between the LED shift and the generalization performance of the condensed graph. (3) Preserving more information about the original graph structure may alleviate the LED shift phenomenon and improve the generalization performance of the condensed graph.
4 Structure-broadcasting Graph Dataset Distillation
In this section, we first present the pipeline and overview of SGDD in Fig. 3. Then, we introduce two modules of SGDD. Finally, we summarize the training pipeline of our SGDD.
4.1 Learning graph structure via graphon approximation
To prevent overlooking the original structure  $\mathbf{A}$  from  $\mathcal{G}$ , we broadcast  $\mathbf{A}$  as supervision for the generation of  $\mathbf{A}'$ . Considering the different shapes between  $\mathbf{A}$  and  $\mathbf{A}'$  ( $N' \ll N$ ), we introduce graphon [21, 70, 17, 33, 94] to distill the original structure information to the condensed structure  $\mathbf{A}'$ . Specifically, given random noise  $\mathcal{Z}(N') \in \mathbb{R}^{N' \times N'}$  as the input coordinates, through the generative model, we then synthesize an adjacency matrix  $\mathbf{A}'$  with  $N'$  nodes. This process can be formulated as  $\mathbf{A}' = \mathrm{GEN}(\mathcal{Z}(N'); \Phi)$ , where the  $\mathrm{GEN}(\cdot)$  is a generative model with parameter  $\Phi$ , and the optimization process is then defined as:
$$
\mathcal {L} _ {\text {s t r u c t u r e}} = \operatorname {D i s t a n c e} (\mathbf {A}, \operatorname {G E N} \left(\mathcal {Z} \left(N ^ {\prime}\right); \Phi\right)), \tag {5}
$$
where  $\mathbf{A}$  is supervision and  $\mathrm{Distance}(\cdot)$  is a metric that measures the difference of  $\mathbf{A}$  and  $\mathbf{A}'$ . The details of  $\mathrm{Distance}(\cdot)$  can be found in Sec. 4.2. To avoid overlooking of the inherent relation [32, 73] between  $\mathbf{A}'$  and the corresponding node information (i.e.,  $\mathbf{X}'$  and  $\mathbf{Y}'$ ), we jointly input  $\mathbf{X}'$  and  $\mathbf{Y}'$  as conditions to generate  $\mathbf{A}'$ . Therefore, the final version of the generative model can be written as  $\mathbf{A}' = \mathrm{GEN}(\mathcal{Z}(N') \oplus \mathbf{X}' \oplus \mathbf{Y}'; \Phi)$ , the  $\oplus$  denotes the concatenate operation.
To study the performance of SGDD in the above paradigm, we theoretically prove the upper bound of LED shift in SGDD by invoking graphon theory. The result can be presented as follows.
Proposition 2. The upper bound of the LED shift on SGDD is shown as:
$$
\left| \left| \eta^ {\mathcal {G}} - \eta^ {\mathcal {S}} \right| \right| \leq \delta_ {\square} \left(W _ {\mathbf {A}}, \mathbf {A} ^ {\prime}\right), \tag {6}
$$
where  $\delta_{\square}$  denotes the cut distance [54] and  $W_{\mathbf{A}}$  is the graphon of  $\mathbf{A}$ . See details in Appendix B.2.
Note minimizing the upper bound of Eq. (6) is equal to optimizing the  $L_{\text{structure}}$  on Eq. (5). Compared to the lower bound in (Eq. (3)), our upper bound is not related to any frequency response of specific GNN (i.e., the terms of  $\sum_{i=1}^{a} ||\bar{\boldsymbol{x}}_i^2 - \bar{\boldsymbol{x}}_i'^2|| + \sum_{i=b}^{N'} ||\bar{\boldsymbol{x}}_i^2 - \bar{\boldsymbol{x}}_i'^2||$ ). As a result, SGDD may perform better than the previous work, especially in the cross-architecture setting and specific tasks.
4.2 Optimizing the graph structure via optimal transport
To mitigate the LED shift between  $\mathbf{A}$  and  $\mathbf{A}'$ , ideally, we can directly minimize the proposed  $SC$ . However,  $SC$  requires an extremely time-consuming  $(O(N^3))$  [4, 58] eigenvalue decomposition operation. Therefore, we propose the LED Matching strategy based on the optimal transport theory to form an efficient optimizing process.
Recall in the Eq.(4) of calculating  $SC$ , we first decompose the eigenvalue, followed by aligning the node through the JS divergence, which essentially compares the distribution proportions. The key point is to decide the node mapping strategy to align such two graphs. Alternatively, assuming we know the prior distribution of the alignment of  $S$  (i.e., we know the bijection of nodes in  $S$  to the  $\mathcal{G}$ ) and denoting such alignment as  $S*$ , we can directly measure the distance between  $\mathcal{G}$  and  $S*$  by employing the 2-Wasserstein metric[15, 56].
$$
\left| \left| \eta^ {\mathcal {G}} - \eta^ {\mathcal {S} *} \right| \right| = W _ {2} ^ {2} \left(\eta^ {\mathcal {G}}, \eta^ {\mathcal {S} *}\right) \tag {7}
$$
Furthermore, following the assumption in previous work[56, 15], we have  $\eta^G \sim \mathcal{N}\left(0, L_{\mathcal{G}}^\dagger\right)$  and  $\eta^S \sim \mathcal{N}\left(0, L_{\mathcal{S}*}^\dagger\right)$ . Then, the Eq.(7) have a closed-form expression[15] as follows:
$$
\left| \left| \eta^ {\mathcal {G}} - \eta^ {\mathcal {S} *} \right| \right| = N \operatorname {t r} \left(L _ {\mathcal {G}} ^ {\dagger}\right) + N ^ {\prime} \operatorname {t r} \left(L _ {\mathcal {S} *} ^ {\dagger}\right) - 2 \operatorname {t r} \left(\sqrt {L _ {\mathcal {S} *} ^ {\ddagger} L _ {\mathcal {G}} ^ {\dagger} L _ {\mathcal {S} *} ^ {\ddagger}}\right), \tag {8}
$$
the  $L^{\dagger}$  denotes the Laplacian pseudo-inverse operation and the tr indicates the trace operation of matrix. Therefore, even though we could not know the actual mapping strategy  $S*$ , we can use the infimum of all possible strategies as a proxy solution. Formally, following the prior work [56], we employ the function  $T$  as a transport plan in the metric space  $\mathcal{X}$  to represent all feasible mapping strategies. Then, we use the  $T_{\#}\eta^{S}$  to represent the pushing forward process of transferring the distribution of  $\eta^S$  to the  $\eta^G$ . As a result, the distance can be regarded as finding the infimum.