SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xin Zheng $^{1}$ , Miao Zhang $^{2}$ , Chunyang Chen $^{1}$ , Soheila Molaei $^{3}$ , Chuan Zhou $^{4}$ , Shirui Pan $^{5*}$ $^{1}$ Monash University, Australia,  $^{2}$ Harbin Institute of Technology (Shenzhen), China  
 $^{3}$ University of Oxford, UK,  $^{4}$ Chinese Academy of Sciences, China,  $^{5}$ Griffith University, Australia  
xin.zheng@monash.edu, zhangmiao@hit.edu.cn, chunyang.chen@monash.edu  
soheila.molaei@eng.ox.ac.uk, zhouchuan@amss.ac.cn, s.pan@griffith.edu.au
Evaluating the performance of graph neural networks (GNNs) is an essential task for practical GNN model deployment and serving, as deployed GNNs face significant performance uncertainty when inferring on unseen and unlabeled test graphs, due to mismatched training-test graph distributions. In this paper, we study a new problem, GNN model evaluation, that aims to assess the performance of a specific GNN model trained on labeled and observed graphs, by precisely estimating its performance (e.g., node classification accuracy) on unseen graphs without labels. Concretely, we propose a two-stage GNN model evaluation framework, including (1) DiscGraph set construction and (2) GNNEvaluator training and inference. The DiscGraph set captures wide-range and diverse graph data distribution discrepancies through a discrepancy measurement function, which exploits the outputs of GNNs related to latent node embeddings and node class predictions. Under the effective training supervision from the DiscGraph set, GNNEvaluator learns to precisely estimate node classification accuracy of the to-be-evaluated GNN model and makes an accurate inference for evaluating GNN model performance. Extensive experiments on real-world unseen and unlabeled test graphs demonstrate the effectiveness of our proposed method for GNN model evaluation.
As prevalent graph data learning models, graph neural networks (GNNs) have attracted much attention and achieved great success for various graph structural data related applications in the real world [36, 38, 35, 41, 42, 14, 21, 46, 44]. In practical scenarios of GNN model deployment and serving [37, 40, 43], understanding and evaluating GNN models' performance is a vital step [3, 13, 22], where model designers need to determine if well-trained GNN models will perform well in practical serving, and users want to know how the in-service GNN models will perform when inferring on their own test graphs [10]. Conventionally, model evaluation utilizes well-annotated datasets for testing to calculate certain model performance metrics (e.g., accuracy and F1-score) [3, 24, 23, 20]. However, it may fail to work well in real-world GNN model deployment and serving, where the unseen test graphs are usually not annotated, making it difficult to obtain essential model performance metrics without ground-truth labels [4, 5]. As shown in Fig. 1 (a-1), taking node classification accuracy metric as an example, it is typically calculated as the percentage of correctly predicted node labels. However, when ground-truth node class labels are unavailable, we can not verify whether the predictions of GNNs are correct, and thus cannot get the overall accuracy of the model.
In light of this, a natural question, referred to GNN model evaluation problem, arises: in the absence of labels in an unseen test graph, can we estimate the performance of a well-trained GNN model?
(a-2) The proposed GNN model evaluation (w/o unseen test graph labels).
(b) An applicable case of the proposed GNNEvaluator.
Figure 1: Illustration of conventional model evaluation vs. the proposed GNN model evaluation with its applicable case. The symbol * indicates GNNs are well-trained and fixed during model evaluation.
In this work, we provide a confirmed answer together with an effective solution to this question, as shown in Fig.1 (a-2). Given a well-trained GNN model and an unseen test graph without labels, GNN model evaluation directly outputs the overall accuracy of this GNN model. This enables users to understand their GNN models at hand, benefiting many GNN deployment and serving scenarios in the real world [2, 8, 30, 25, 47, 45, 16, 15]. For example, given a GNN model in service (e.g., a GNN model trained and served with Amazon DGL and SageMaker [1]), GNN model evaluation can provide users with a confidence score based on its estimated node classification accuracy, so that users know how much faith they should place in GNN-predicted node labels on their own unlabeled test graphs. Moreover, as shown in Fig. 1 (b), given a set of available well-trained GNNs in service, GNN model evaluation can provide users with reliable guidance to make an informed choice among these deployed GNN models on their own unlabeled test graphs.
In this paper, we focus on the node classification task, with accuracy as the primary GNN model evaluation metric. Developing effective GNN model evaluation methods for this task faces threefold challenges: Challenge-1: The distribution discrepancies between various real-world unseen test graphs and the observed training graph are usually complex and diverse, incurring significant uncertainty for GNN model evaluation. Challenge-2: It is not allowed to re-train or fine-tune the practically in-service GNN model to be evaluated, and the only accessible model-related information is its outputs. Hence, how to fully exploit the limited GNN outputs and integrate various training-test graph distribution differences into discriminative discrepancy representations is critically important. Challenge-3: Given the discriminative discrepancy representations of training-test graph distributions, how to develop an accurate GNN model evaluator to estimate the node classification accuracy of an in-service GNN on the unseen test graph is the key to GNN model evaluation.
To address the above challenges, in this work, we propose a two-stage GNN model evaluation framework, including (1) DiscGraph set construction and (2) GNNEvaluator training and inference. More specifically, in the first stage, we first derive a set of meta-graphs from the observed training graph, which involves wide-range and diverse graph data distributions to simulate (ideally) any potential unseen test graphs in practice, so that the complex and diverse training-test graph data distribution discrepancies can be effectively captured and modeled (Challenge-1). Based on the meta-graph set, we build a set of discrepancy meta-graphs (DiscGraph) by jointly exploiting latent node embeddings and node class predictions of the GNN with a discrepancy measurement function, which comprehensively integrates GNN output discrepancies to expressive graph structural discrepancy representations involving discrepancy node attributes, graph structures, and accuracy labels (Challenge-2). In the second stage, we develop a GNNEvaluator composed of a typical GCN architecture and an accuracy regression layer, and we train it to precisely estimate node classification accuracy with effective supervision from the representative DiscGraph set (Challenge-3). During the inference, the trained GNNEvaluator could directly output the node classification accuracy of the in-service GNN on the unseen test graph without any node class labels.
In summary, the contributions of our work are listed as follows:
- Problem. We study a new research problem, GNN model evaluation, which opens the door for understanding and evaluating the performance of well-trained GNNs on unseen real-world graphs without labels in practical GNN model deployment and serving.
- Solution. We design an effective solution to simulate and capture the discrepancies of diverse graph data distributions, together with a GNNEvaluator to estimate node classification accuracy of the in-service GNNs, enabling accurate GNN model evaluation.
- Evaluation. We evaluate our method on real-world unseen and unlabeled test graphs, achieving a low error (e.g., as small as  $2.46\%$ ) compared with ground-truth accuracy, demonstrating the effectiveness of our method for GNN model evaluation.
Prior Works. Our research is related to existing studies on predicting model generalization error, which aims to estimate a model's performance on unlabeled data from the unknown and shifted distributions [6, 9, 34, 10, 4]. However, these researches are designed for data in Euclidean space (e.g., images) while our research is dedicatedly designed for graph structural data. Our research also significantly differs from others in unsupervised graph domain adaption [33, 39, 29], out-of-distribution (OOD) generalization [18, 48], and OOD detection [19, 26], in the sense that we aim to estimate well-trained GNN models' performance, rather than improve the generalization ability of new GNN models. Detailed related work can be found in Appendix A.
Preliminary. Consider that we have a fully-observed training graph  $S = (\mathbf{X},\mathbf{A},\mathbf{Y})$ , where  $\mathbf{X} \in \mathbb{R}^{N \times d}$  denotes  $N$  nodes with  $d$ -dimensional features,  $\mathbf{A} \in \mathbb{R}^{N \times N}$  denotes the adjacency matrix indicating the edge connections, and  $\mathbf{Y} \in \mathbb{R}^{N \times C}$  denotes the  $C$ -classes of node labels. Then, training a GNN model on  $S$  for node classification objective can be denoted as:
$$
\min  _ {\boldsymbol {\theta} _ {S}} \mathcal {L} _ {\mathrm {c l s}} (\hat {\mathbf {Y}}, \mathbf {Y}), \text {w h e r e} \mathbf {Z} _ {S}, \hat {\mathbf {Y}} = \operatorname {G N N} _ {\boldsymbol {\theta} _ {S}} (\mathbf {X}, \mathbf {A}). \tag {1}
$$
where  $\theta_{\mathcal{S}}$  denotes the parameters of GNN trained on  $\mathcal{S}$ ,  $\mathbf{Z}_{\mathcal{S}} \in \mathbb{R}^{N \times d_1}$  is the output node embedding of graph  $\mathcal{S}$  from  $\mathrm{GNN}_{\theta_{\mathcal{S}}}$ , and  $\hat{\mathbf{Y}} \in \mathbb{R}^{N \times C}$  denotes GNN predicted node labels. By optimizing the node classification loss function  $\mathcal{L}_{\mathrm{cls}}$  between GNN predictions  $\hat{\mathbf{Y}}$  and ground-truth node labels  $\mathbf{Y}$ , i.e., cross-entropy loss, we could obtain a well-trained GNN with optimal weight parameters  $\theta_{\mathcal{S}}^*$ , denoted as  $\mathrm{GNN}_{\mathcal{S}}^*$ . Then, the node classification accuracy can be calculated to reflect GNN performance as:  $\mathrm{Acc}(\mathcal{S}) = \sum_{i=1}^{N} (\hat{y}_i == y_i) / N$ , which indicates the percentage of correctly predicted node labels between the GNN predicted labels  $\hat{y}_i \in \hat{\mathbf{Y}}$  and ground truths  $y_i \in \mathbf{Y}$ . Given an unseen and unlabeled graph  $\mathcal{T} = (\mathbf{X}', \mathbf{A}')$  including  $M$  nodes with its features  $\mathbf{X}' \in \mathbb{R}^{M \times d}$  and structures  $\mathbf{A}' \in \mathbb{R}^{M \times M}$ , we assume the covariate shift between  $\mathcal{S}$  and  $\mathcal{T}$ , where the distribution shift mainly lies in node numbers, node context features, and graph structures, but the label space of  $\mathcal{T}$  keeps the same with  $\mathcal{S}$ , i.e., all nodes in  $\mathcal{T}$  are constrained in the same  $C$ -classes. Due to the absence of ground-truth node labels, we could NOT directly calculate the node classification accuracy to assess the performance of the well-trained  $\mathrm{GNN}_{\mathcal{S}}^*$  on  $\mathcal{T}$ . In light of this, we present a new research problem for GNNs as:
Definition of GNN Model Evaluation. Given the observed training graph  $S$ , its well-trained model  $\mathrm{GNN}_S^*$ , and an unlabeled unseen graph  $\mathcal{T}$  as inputs, the goal of GNN model evaluation aims to learn an accuracy estimation model  $f_{\phi}(\cdot)$  parameterized by  $\phi$  as:
$$
\operatorname {A c c} (\mathcal {T}) = f _ {\phi} \left(\mathrm {G N N} _ {\mathcal {S}} ^ {*}, \mathcal {T}\right), \tag {2}
$$
where  $f_{\phi}:(\mathrm{GNN}_{\mathcal{S}}^{*},\mathcal{T})\to a$  and  $a\in \mathbb{R}$  is a scalar denoting the overall node classification accuracy  $\mathrm{Acc}(\mathcal{T})$  for all unlabeled nodes of  $\mathcal{T}$ . When the context is clear, we will use  $f_{\phi}(\mathcal{T})$  for simplification.
3.1 Overall Framework of GNN Model Evaluation
As shown in Fig. 2, the proposed overall framework of GNN model evaluation contains two stages: (1) DiscGraph set construction; and (2) GNNEvaluator training and inference.
Stage 1. Given the observed training graph  $S$ , we first extract a seed graph from it followed by augmentations, leading to a set of meta-graphs from it for simulating any potential unseen graph distributions in practice. Then, the meta-graph set  $\mathcal{G}_{\mathrm{meta}}$  and the observed training graph  $S$  are fed into the well-trained GNN for obtaining latent node embeddings, i.e.,  $\mathbf{Z}_{\mathrm{meta}}^{(i,\ast)}$  and  $\mathbf{Z}_S^*$ , respectively. After, a
Figure 2: Overall two-stage pipeline of GNN model evaluation with (1) DiscGraph set construction and (2) GNNEvaluator training and inference.
discrepancy measurement function  $D(\cdot)$  works on these two latent node embeddings, leading to ① discrepancy node attributes  $\mathbf{X}_{\mathrm{disc}}^i$ . Meanwhile, the output node class predictions of well-trained GNN on each meta-graph are used to calculate the node classification accuracy according to its ground-truth node labels, leading to the new ② scalar accuracy labels  $y_{\mathrm{disc}}^i$ . Along with ③ graph structures  $\mathbf{A}_{\mathrm{disc}}^i$  from each meta-graph, these three important components composite the set of discrepancy meta-graphs (DiscGraph) in expressive graph structural discrepancy representations.
Stage 2. In the second stage, the representative DiscGraph set is taken as effective supervision to train a GNNEvaluator through accuracy regression. During the inference, the trained GNNEvaluator could directly output the node classification accuracy of the in-service GNN on the unseen test graph  $\mathcal{T}$  without any node class labels.
3.2 Discrepancy Meta-graph Set Construction
The critical challenge in developing GNN model evaluation methods is complex and diverse graph data distribution discrepancies between various real-world unseen test graphs and the observed training graph. As the in-service GNN in practice fits the observed training graph well, making inferences on various and diverse unseen test graphs would incur significant performance uncertainty for GNN model evaluation, especially when the node classification accuracy can not be calculated due to label absence.
In light of this, we propose to construct a set of diverse graphs to simulate wide-range discrepancies of potential unseen graph distributions. Specifically, such a graph set should have the following characteristics: (1) sufficient quantity: it should contain a relatively sufficient number of graphs with diverse node context and graph structure distributions; (2) represented discrepancy: the node attributes of each graph should indicate its distribution distance gap towards the observed training graph; (3) known accuracy: each graph should be annotated by node classification accuracy as its label.
To address the characteristic of (1) sufficient quantity, we introduce a meta-graph simulation strategy to synthesize a wide variety of meta-graphs for representing any potential unseen graph distributions that may be encountered in practice. Specifically, we extract a seed sub-graph  $S_{\mathrm{seed}}$  from the observed training graph  $S$ . The principle of seed sub-graph selection strategy is that  $S_{\mathrm{seed}}$  involves the least possible distribution shift within  $\mathbb{P}_S$  from the observed training graph, and shares the same label space with  $S$ , satisfying the assumption of covariate shift. Then,  $S_{\mathrm{seed}}$  is fed to a pool of graph augmentation operators as
$$
\mathcal {A} = \left\{\mathcal {A} _ {1} (\text {E D G E D R O P}), \mathcal {A} _ {2} (\text {S U B G R A P H}), \mathcal {A} _ {3} (\text {A T T R M A X}), \mathcal {A} _ {4} (\text {N O D E M I X}) \right\}, \tag {3}
$$
which belongs to the distribution  $\mathbb{P}_{\mathcal{A}}(p_i,|\epsilon)$  with  $\{p_i\}_{i = 1}^4\in (0,1)$  represent the augmentation ratio on  $S_{\mathrm{seed}}$  for each augmentation type, and  $\epsilon$  determines the probability of sampling a particular augmentation type. For instance,  $p_1 = 0.5$  means dropping  $50\%$  edges in  $S_{\mathrm{seed}}$ . In this way, we can obtain a set of meta-graphs  $\mathcal{G}_{\mathrm{meta}} = \left\{g_{\mathrm{meta}}^{i}\right\}_{i = 1}^{K}$  with  $K$  numbers of graphs, and we have  $\mathcal{G}_{\mathrm{meta}}\sim \mathbb{P}_{\mathcal{A}}\times \mathbb{P}_{\mathcal{S}}:\mathcal{S}_{\mathrm{seed}}\to g_{\mathrm{meta}}^{i}$ . For each meta-graph  $g_{\mathrm{meta}}^{i} = \{\mathbf{X}_{\mathrm{meta}}^{i},\mathbf{A}_{\mathrm{meta}}^{i},\mathbf{Y}_{\mathrm{meta}}^{i}\}$ , where
$\mathbf{X}_{\mathrm{meta}}^i \in \mathbb{R}^{M_i \times d}$ ,  $\mathbf{A}_{\mathrm{meta}}^i \in \mathbb{R}^{M_i \times M_i}$ , and  $\mathbf{Y}_{\mathrm{meta}}^i \in \mathbb{R}^{M_i \times C}$  have  $M_i$  number of nodes,  $d$ -dimensional features, and known node labels from  $S_{\mathrm{seed}}$  belonging to  $C$  classes.
To incorporate the characteristic of (2) represented discrepancy, we fully exploit the outputs of latent node embeddings and node class predictions from well-trained  $\mathrm{GNN}_S^*$ , and integrate various training-test graph distribution differences into discriminative discrepancy representations. Specifically, given the  $\mathrm{GNN}_S^*$  learned latent node embeddings  $\mathbf{Z}_S^* = \mathrm{GNN}_S^*(\mathbf{X}, \mathbf{A})$ , and  $\mathbf{Z}_{\mathrm{meta}}^{(i, *)}, \hat{\mathbf{Y}}_{\mathrm{meta}}^{(i, *)} = \mathrm{GNN}_S^*(\mathbf{X}_{\mathrm{meta}}^i, \mathbf{A}_{\mathrm{meta}}^i)$ , where  $\mathbf{Z}_S^* \in \mathbb{R}^{N \times d_1}$  and  $\mathbf{Z}_{\mathrm{meta}}^{(i, *)} \in \mathbb{R}^{M_i \times d_1}$ , we derive a distribution discrepancy measurement function  $D(\cdot)$  to calculate the discrepancy meta-graph node attributes as:
$$
\mathbf {X} _ {\text {d i s c}} ^ {i} = D \left(\mathbf {Z} _ {\text {m e t a}} ^ {(i, *)}, \mathbf {Z} _ {\mathcal {S}} ^ {*}\right) = \frac {\mathbf {Z} _ {\text {m e t a}} ^ {(i , *)} \mathbf {Z} _ {\mathcal {S}} ^ {* \mathrm {T}}}{\left\| \mathbf {Z} _ {\text {m e t a}} ^ {(i , *)} \right\| _ {2} \cdot \left\| \mathbf {Z} _ {\mathcal {S}} ^ {*} \right\| _ {2}}, \tag {4}
$$
where  $\mathbf{X}_{\mathrm{disc}}^i\in \mathbb{R}^{M_i\times N}$  reflects the node-level distribution discrepancy between each  $g_{\mathrm{meta}}^i$  and  $S$  with the well-trained GNN\*. Each element in  $x_{u,v}^i\in \mathbf{X}_{\mathrm{disc}}^i$  denotes the node embedding discrepancy gap between a certain node  $u$  in the  $i$ -th meta-graph and a node  $v$  in the observed training graph. Taking this representation as node attributes could effectively integrate representative node-level discrepancy produced by well-trained GNN's node embedding space.
Meanwhile, the characteristic of (3) known accuracy can be involved with the outputs of node class predictions produced by  $\mathrm{GNN}_S^*$  on meta-graph  $\hat{\mathbf{Y}}_{\mathrm{meta}}^{(i,*)} = \{\hat{y}_{\mathrm{meta}(i,*)}^j\}_{j=1}^{M_i}$ . We calculate the node classification accuracy on the meta-graph given its ground-truth node class labels  $\mathbf{Y}_{\mathrm{meta}}^i = \{y_{\mathrm{meta}(i)}^j\}_{j=1}^{M_i}$  as:
$$
y _ {\text {d i s c}} ^ {i} = \operatorname {A c c} \left(g _ {\text {m e t a}} ^ {i}\right) = \frac {\sum_ {j = 1} ^ {M _ {i}} \left(\hat {y} _ {\text {m e t a} (i , *)} ^ {j} = = y _ {\text {m e t a} (i)} ^ {j}\right)}{M _ {i}}, \tag {5}
$$
where  $y_{\mathrm{meta}(i)}^j$  and  $\hat{y}_{\mathrm{meta}(i,*)}^j$  denote the ground truth label and  $\mathrm{GNN}_S^*$  predicted label of  $j$ -th node in the  $i$ -th graph of the meta-graph set, respectively. Note that  $y_{\mathrm{disc}}^i \in \mathbb{R}$  is a continuous scalar denoting node classification accuracy under specific  $\mathrm{GNN}_S^*$  within the range of  $(0,1)$ .
In this way, by incorporating all these characteristics with the discrepancy node attributes  $(\mathbf{X}_{\mathrm{disc}}^i)$ , the scalar node classification accuracy label  $y_{\mathrm{disc}}^i$ , and the graph structure from meta-graph as  $\mathbf{A}_{\mathrm{disc}}^i = \mathbf{A}_{\mathrm{meta}}^i$  for indicating discrepancy node interconnections, we could derive the final discrepancy meta-graph set, i.e., DiscGraph set, as
$$
\mathcal {G} _ {\text {d i s c}} = \left\{g _ {\text {d i s c}} ^ {i} \right\} _ {i = 1} ^ {K}, \text {w h e r e} g _ {\text {d i s c}} ^ {i} = \left(\mathbf {X} _ {\text {d i s c}} ^ {i}, \mathbf {A} _ {\text {d i s c}} ^ {i}, y _ {\text {d i s c}} ^ {i}\right). \tag {6}
$$
The proposed DiscGraph set  $\mathcal{G}_{\mathrm{dist}}$  contains a sufficient number of graphs with diverse node context and graph structure distributions, where each discrepancy meta-graph contains the latent node embedding based discrepancy node attributes and the node class prediction based accuracy label, according to the well-trained GNN's outputs, along with diverse graph structures. All these make the proposed DiscGraph set a discriminative graph structural discrepancy representation for capturing wide-range graph data distribution discrepancies.
3.3 GNNEvaluator Training and Inference
Training. Given our constructed DiscGraph set  $\mathcal{G}_{\mathrm{disc}} = \left\{g_{\mathrm{disc}}^i\right\}_{i=1}^K$  and  $g_{\mathrm{disc}}^i = (\mathbf{X}_{\mathrm{disc}}^i, \mathbf{A}_{\mathrm{disc}}^i, y_{\mathrm{disc}}^i)$  in Eq. (6), we use it to train a GNN regressor with  $f_\phi : g_{\mathrm{disc}}^i \to a$  for evaluating well-trained GNNs, which we name as GNNEvaluator. Specifically, the proposed GNNEvaluator takes a two-layer GCN architecture as the backbone, followed by a pooling layer to average the representation of all nodes of each  $g_{\mathrm{disc}}^i$ , and then maps the averaged embedding to a scalar node classification accuracy on the whole graph. The objective function of our GNN regressor can be written as:
$$
\min  _ {\phi} \sum_ {i = 1} ^ {K} \mathcal {L} _ {\text {r e g}} \left(f _ {\phi} \left(\mathbf {X} _ {\text {d i s c}} ^ {i}, \mathbf {A} _ {\text {d i s c}} ^ {i}\right), y _ {\text {d i s c}} ^ {i}\right), \tag {7}
$$
where  $\mathcal{L}_{\mathrm{reg}}$  is the regression loss, i.e., the mean square error (MSE) loss.
Inference. During the inference in the practical GNN model evaluation, we have: (1) to-be-evaluated  $\mathrm{GNN}_S^*$ , and (2) the unseen test graph  $\mathcal{T} = (\mathbf{X}',\mathbf{A}')$  without labels. The first thing is to calculate
the discrepancy node attributes on the unseen test graph  $\mathcal{T}$  towards the observed training graph  $S$  according to Eq. (4), so that we could obtain  $\mathbf{X}_{\mathrm{disc}}^{\mathcal{T}} = D(\mathbf{Z}_{\mathcal{T}}^{*},\mathbf{Z}_{\mathcal{S}}^{*})$ , where  $\mathbf{Z}_{\mathcal{T}}^{*} = \mathrm{GNN}_{\mathcal{S}}^{*}(\mathbf{X}^{\prime},\mathbf{A}^{\prime})$ . Along with the unseen graph structure  $\mathbf{A}_{\mathrm{disc}}^{\mathcal{T}} = \mathbf{A}^{\prime}$ , the proposed GNNEvaluator could directly output the node classification accuracy of  $\mathrm{GNN}_S^*$  on  $\mathcal{T}$  as:
$$
\operatorname {A c c} (\mathcal {T}) = \hat {y} _ {\text {d i s t}} ^ {\mathcal {T}} = f _ {\phi^ {*}} \left(\mathbf {X} _ {\text {d i s c}} ^ {\mathcal {T}}, \mathbf {A} _ {\text {d i s c}} ^ {\mathcal {T}}\right), \tag {8}
$$
where  $\phi^{*}$  denotes the optimal GNNEvaluator weight parameters trained by our constructed DiscGraph set  $\mathcal{G}_{\mathrm{dist}}$
This section empirically evaluates the proposed GNNEvaluator on real-world graph datasets for node classification task. In all experiments, to-be-evaluated GNN models have been well-trained on observed training graphs and keep FIXED in GNN model evaluation. We only vary the unseen test graph datasets and evaluate various different types and architectures of well-trained GNNs. Throughout our proposed entire two-stage GNN model evaluation framework, we are unable to access the labels of unseen test graphs. We only utilize these labels for the purpose of obtaining true error estimates for experimental demonstration.