SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jun Wu<sup>1</sup>, Elizabeth Ainsworth<sup>1,2</sup>, Andrew Leakey<sup>1</sup>, Haixun Wang<sup>3</sup>, Jingrui He<sup>1</sup>
$^{2}$ USDA ARS Global Change and Photosynthesis Research Unit
{junwu3,ainswort,leakey,jingrui}@illinois.edu,{haixun}@gmail.com
Transferable graph learning involves knowledge transferability from a source graph to a relevant target graph. The major challenge of transferable graph learning is the distribution shift between source and target graphs induced by individual node attributes and complex graph structures. To solve this problem, in this paper, we propose a generic graph-structured Gaussian process framework (GraphGP) for adaptively transferring knowledge across graphs with either homophily or heterophily assumptions. Specifically, GraphGP is derived from a novel graph structure-aware neural network in the limit on the layer width. The generalization analysis of GraphGP explicitly investigates the connection between knowledge transferability and graph domain similarity. Extensive experiments on several transferable graph learning benchmarks demonstrate the efficacy of GraphGP over state-of-the-art Gaussian process baselines.
Transfer learning [40, 54] aims at studying the transfer of knowledge or information from a source domain to a relevant target domain under distribution shifts. The knowledge transferability across domains has been investigated from various aspects. For example, [3, 15, 67] proposed to empirically estimate the data-level domain discrepancy for measuring the transferability across graphs. [8, 49] adopted adaptive Gaussian processes with transfer kernels, which formulated the task relatedness as a hyper-parameter for tuning. [22, 36, 63] instead evaluated the transferability of a pre-trained source model in the target domain. However, most existing works followed the IID assumption that samples are independent and identically distributed
in each domain. This might limit their capacities in understanding the knowledge transferability across domains with non-IID data, e.g., cross-domain recommendation [55], cross-network role identification [69], etc.
In this paper, we focus on the problem of transferable graph learning over non-IID graph data. The challenge of transferable graph learning induced by the graph structure information can be explained in Figure 1. It can be seen that Figure 1(a) provides source and target samples associated with input features and output class labels (indicated by node colors). Suppose that source and target
Figure 1: Illustration of transferable graph learning. (a) Samples are independently drawn. (b) Samples are connected in each domain, and both domains follow the homophily assumption. (c) Source and target domains follow different graph structure assumptions.
distributions have been aligned (e.g., optimized by domain discrepancy minimization algorithms [15, 67]), Figure 1(a) implies that source and target domains have similar data distributions in the feature space, thereby enabling feasible knowledge transfer across domains. Compared to Figure 1(a), Figure 1(b)(c) introduce additional graph structure information. In Figure 1(b), source and target graphs follow similar structures where nodes within the same class tend to be connected (i.e., the homophily assumption [59]). In contrast, the structures of source and target graphs fundamentally differ in Figure 1(c). That is, though the target graph follows the homophily assumption, the source graph holds the heterophily assumption that connected nodes may have different classes and dissimilar node attributes [68]. Therefore, compared to standard transfer learning [31, 40] with IID data, transferable graph learning is much more challenging because knowledge transferability across graphs can be determined by both individual node features and complex graph structures.
In recent years, transferable graph learning has been studied [55, 58, 66, 69] where both source and target graphs follow the homophily assumption (Figure 1(b)). However, little effort has been devoted to investigating the knowledge transferability between homophilic and heterophilic graphs (Figure 1(c)). To bridge this gap, in this paper, we focus on studying the knowledge transferability across graphs with either the same (e.g., homophily or heterophily) or different assumptions.
We start by introducing a universal structure-aware neural network to build the input-output relationship between structure-aware input sample  $(v, G)$  and its associated output value  $y_v$  for tackling homophilic and heterophilic graphs. This model encodes both local node representation (sample-level) and global graph representation (domain-level) simultaneously. The crucial idea is to automatically select relevant neighborhoods by adding learnable weights to neighborhood aggregation. Thus, it enables knowledge transferability between homophilic and heterophilic graphs by adaptively selecting neighborhoods for each graph. The intuition is that source and target graphs might have common knowledge from the selected neighborhoods, e.g., nearby neighborhoods for the homophilic graph and high-order neighborhoods for the heterophilic graph. Then, we show the equivalence between structure-aware neural network and graph Gaussian process in the limit on the layer width. This observation sheds light on building graph-structured Gaussian processes (GraphGP) for transferable graph learning. Moreover, the generalization analysis of GraphGP shows the connection between knowledge transferability and graph domain similarity. Compared to previous works [45, 58, 69], GraphGP benefits from (i) feasible knowledge transferability between homophily and heterophily graphs, and (ii) flexible incorporation with existing graph neural networks and graph kernels. Extensive experiments on node regression tasks demonstrate the effectiveness of GraphGP over state-of-the-art Gaussian process baselines. The main contributions of this paper are summarized as follows.
- A generic graph-structured Gaussian process framework GraphGP is proposed for transferable graph learning. It is derived from a structure-aware neural network encoding local node representation (sample-level) and global graph representation (domain-level) simultaneously.
- We show that GraphGP tackles the knowledge transferability in homophily and heterophily graphs using a simple neighborhood selection strategy. In addition, we theoretically analyze the knowledge transferability across graphs from the perspective of graph domain similarity.
- Experimental results on a variety of graph learning benchmarks demonstrate the efficacy of GraphGP over state-of-the-art Gaussian process baselines.
The rest of this paper is organized as follows. Section 2 summarizes the related work, and Section 3 provides the problem definition of transferable graph learning. In Section 4, we present the graph-structured Gaussian process (GraphGP) for transferable graph learning. The experimental results are provided in Section 5. Finally, we conclude the paper in Section 6.
Transfer Learning: Transfer learning [40, 54] tackles the transfer of knowledge or information from a source domain to a relevant target domain. It is theoretically shown [3, 67] that the generalization performance of a learning algorithm can be improved by leveraging latent knowledge from the source domain. Specifically, the generalization error of a learning algorithm on the target domain can be bounded in terms of the source knowledge and the distribution discrepancy across domains. Recently, transfer learning has been explored in non-IID graph data [7, 55, 58, 64, 69]. However, most existing works focus on either investigating the transferability of a pre-trained graph neural
network [27, 45, 69] or designing graph transfer learning algorithms [58, 66] with relaxed assumptions (e.g.,  $k$ -hop ego-graphs are assumed to be independent and identically distributed [69]).
Gaussian Process: Gaussian process [9, 42] provides a principled nonparametric framework for learning stochastic functions from observations. It has been applied to transfer learning when samples in each domain are independent and identically distributed [8, 49, 56]. The crucial idea is to infer the task relatedness between source and target domains using available labeled source and target examples. More recently, Gaussian processes have also been investigated in graph learning tasks, e.g., link prediction [14, 38, 65] and node regression [6, 21, 28, 35, 37, 39]. However, little effort has been devoted to understanding transferable graph learning from Gaussian process perspectives.
3.1 Notation and Problem Definition
Suppose that a graph is represented as  $G = (V, E)$ , where  $V = \{v_1, \dots, v_{|V|}\}$  is the set of  $n$  nodes and  $E \subseteq V \times V$  is the edge set in the graph. Each node  $v \in V$  can be associated with a  $D$ -dimensional feature vector  $\mathbf{x}_v \in \mathbb{R}^D$ . In the node regression tasks, the ground-truth output of each node is denoted by  $y_v \in \mathbb{R}$ . The structure of a graph  $G$  can also be represented by an adjacency matrix  $\mathbf{A} \in \mathbb{R}^{|V| \times |V|}$ , where  $\mathbf{A}_{ij}$  is the edge weight between  $v_i$  and  $v_j$  within the graph. Following [55, 69], we introduce a generic problem setting of transferable graph learning as follows.
Definition 3.1 (Transferable Graph Learning). Given a source graph  $G_{s} = (V_{s},E_{s})$  and a target graph  $G_{t} = (V_{t},E_{t})$ , transferable graph learning aims to improve the prediction performance of a graph learning algorithm in the target graph using latent knowledge from a relevant source graph.
3.2 Message-Passing Graph Neural Networks
Weisfeiler-Lehman (WL) graph subtree kernel [46] measures the similarity of two input graphs, inspired by the Weisfeiler-Lehman test of isomorphism [53]. The crucial component of WL subtree kernel is to recursively represent the subtree structure rooted at each node. This has motivated a variety of message-passing graph neural networks (GNN), which recursively learn the node presentation of node  $v$  by aggregating the feature vectors from  $v$ 's local neighborhood [17, 18, 20, 57, 59]. Generally, a graph convolutional layer of message-passing GNNs can be summarized as follows.
$$
h _ {v} ^ {(l + 1)} = \operatorname {C O M B I N E} ^ {(l)} \left(\underbrace {h _ {v} ^ {(l)}} \quad , \operatorname {A G G R E G A T E} _ {1} ^ {(l)} \left(\left\{h _ {u} ^ {(l)} | u \in N _ {1} (v) \right\}\right), \right.
$$
Seed node representation
1-order neighborhood representation
$$
\cdot \cdot \cdot , \underbrace {\mathbf {A G G R E G A T E} _ {k} ^ {(l)}} \left(\left\{h _ {u} ^ {(l)} | u \in N _ {k} (v) \right\}\right)
$$
$k$ -order neighborhood representation
where  $\mathrm{COMBINE}^{(l)}$  function compresses the representations from the node itself and its neighbors at the  $l^{\mathrm{th}}$  graph convolutional layer into a single representation, and  $\mathrm{AGGREGATE}_j^{(l)}$  ( $j = 1, \dots, k$ ) function aims to aggregate message from  $j$ -order neighborhood  $N_j(v)$  of node  $v$ . It indicates that there are two major components for designing graph convolutional layers in GNNs: neighborhood selection and aggregation. More specifically, neighborhood aggregation allows compressing the graph structure and node attributes simultaneously, while neighborhood selection provides the flexibility of GNNs in tackling graphs with different assumptions (e.g., homophily [20] or heterophily [68]).
Homophily graphs: Homophily graph holds the assumption that nodes within the same class tend to be connected in the graph, e.g., citation networks [13], social networks [33], etc. This assumption has motivated various instantiations of message-passing graph neural networks over first-order neighborhoods, e.g., GCN [25], GraphSAGE [20], GAT [50], GIN [59].
Heterophily graphs: Heterophily graph holds that connected nodes may have different class labels and dissimilar node attributes, e.g., different classes of amino acids are more likely to connect within protein structures [30, 68]. Recently, by adaptively exploring the potential homophily in a high-order local neighborhood (e.g.,  $k > 1$ ), message-passing graph neural networks have been proposed for heterophily graphs, e.g., MixHop [1], H2GCN [68], GPR-GNN [11], HOG-GCN [52], GloGNN [29].
In this section, we propose the graph-structured Gaussian processes for transferable graph learning.
4.1 Structure-Aware Neural Network
We start by showing the graph sampling process from a probability distribution space  $\mathcal{P}$ . A graph  $G = (V, E)$  is sampled by first sampling a graph domain distribution  $\mathbb{P}_{\mathcal{G}}$  from  $\mathcal{P}$ , and then sampling a specific graph  $G$  from  $\mathbb{P}_{\mathcal{G}}$ . More specifically, one realization from  $\mathcal{P}$  is a graph domain distribution  $\mathbb{P}_{\mathcal{G}}$  (also denoted as  $\mathbb{P}_{\mathcal{G}} = (\mathbb{P}_{\mathcal{V}}, \mathbb{P}_{\mathcal{E}})$  for node and edge sampling distribution) characterizing a two-stage graph sampling as follows. A set of nodes  $V$  is sampled from the graph distribution  $\mathbb{P}_{\mathcal{G}}$  (more specifically, sampled from  $\mathbb{P}_{\mathcal{V}}$ ), and then the edge weight is sampled from  $\mathbb{P}_{\mathcal{G}}$  (more specifically, sampled from  $\mathbb{P}_{\mathcal{E}}$ ) over any pair of nodes in  $V$ . In addition to the graph structure (induced by the node dependence), the output label of a node can also be sampled accordingly in the context of node regression tasks. That is, a label-informed graph domain distribution  $\mathbb{P}_{\mathcal{G}, \mathcal{V}}$  is sampled from  $\mathcal{P}$ . Next, a graph  $G = (V, E)$  is sampled from  $\mathbb{P}_{\mathcal{G}} = (\mathbb{P}_{\mathcal{V}}, \mathbb{P}_{\mathcal{E}})$ , and then the output labels of nodes are sampled from  $\mathbb{P}_{\mathcal{V}|\mathcal{G}}$ . The goal of node regression task [55, 69] is to learn a prediction function  $f(\cdot)$  that predicts  $\hat{y}_v = f(v, G)$  for each node  $v \in V$ .
In this paper, we consider the covariate shift assumption [40] over graphs. That is, the source and target graphs share conditional distribution  $\mathbb{P}_{\mathcal{Y}|\mathcal{G}}(y|(v,G))$  but different marginal distributions  $\mathbb{P}_{\mathcal{G}}(v,G)$ . The aforementioned graph sampling process motivates us to learn the input-output relationship between an input sample  $(G,v)$  and its associated output  $y_{v}$  for transferable graph learning. To this end, for each input graph  $G$  (e.g., source  $G_{s}$  or target graph  $G_{t}$  in transferable graph learning), we define a structure-aware neural network  $f(\cdot ,\cdot)$  as follows.
$$
f _ {i} ^ {(l)} (v, G) = \frac {1}{\sqrt {M}} \sum_ {j = 1} ^ {M} \mathbf {W} _ {i j} ^ {(l)} \cdot \mu_ {j} ^ {(l)} (v | G) \cdot \nu_ {j} ^ {(l)} (G) \tag {2}
$$
$$
\mu^ {(l)} (v | G) = h _ {v} ^ {(l)} \quad \text {a n d} \quad \nu^ {(l)} (G) = \operatorname {R E A D O U T} \left(\tilde {h} _ {v} ^ {(l)} | v \in V\right)
$$
where  $M$  is the layer width $^1$ .  $\mu^{(l)}(v|G)$  denotes the node (sample-level) representation of  $v$  given the graph  $G$  at the  $l^{\text{th}}$  layer, and  $\nu^{(l)}(G)$  denotes the graph (domain-level) representation of the graph  $G$ . Here READOUT function compresses a set of node representations into a single vector representation [59, 62]. This definition indicates that the embedding representation of the input  $(v,G)$  (e.g., a pair of a node  $v$  and its associated graph  $G$ ) is given by two crucial components: an individual node embedding representation  $\mu^{(l)}(v|G)$  and a global graph embedding representation  $\nu^{(l)}(G)$ .  $h_v^{(l)}$  and  $\tilde{h}_v^{(l)}$  represent graph neural networks to learn node and graph representations separately.
The intuition behind this definition is explained below. The graph  $G$  is a realization of a graph domain distribution  $\mathbb{P}_{\mathcal{G}}$ . Then, the embedding representation of the graph  $G$  can be explained as an empirical estimate of the distribution  $\mathbb{P}_{\mathcal{G}}$ , i.e.,  $\nu(G) \approx \nu(\mathbb{P}_{\mathcal{G}})$ . Moreover, when the size of the graph  $G$  goes to infinity (i.e., associated with an infinite number of nodes),  $G$  approximates the true distribution  $\mathbb{P}_{\mathcal{G}}$  and the embedding representation of the graph  $G$  will be able to recover the true embedding of probability distribution  $\mathbb{P}_{\mathcal{G}}$  [19, 48]. Besides, the embedding representation of a node  $v$  can also be explained as an approximation of  $\mathbb{P}_{\mathcal{G}}$  at the location  $v$ , i.e.,  $\mu(v|G) \approx \mu(v|\mathbb{P}_{\mathcal{G}})$ .
Remark. The structure-aware neural network Eq. (2) is defined over message-passing graph neural networks commonly used for various single-domain graph learning tasks [18, 20, 23, 59]. Different from previous works, we focus on transferable graph learning scenarios across graphs. This motivates us to consider the task relatedness between source and target graphs, which can be explicitly measured by the domain distribution similarity [15, 56]. To this end, we design the structure-aware neural network Eq. (2) for learning domain-aware node presentation, i.e., the integration of the node (sample-level) representation and the entire graph (domain-level) representation. Specifically, we can show the connection between graph domain distribution and graph representation learning in the reproducing kernel Hilbert space for cross-graph learning tasks (see Corollary 4.4).
<sup>1</sup>Here, "layer width" indicates the number of neurons in a graph convolutional layer.
4.2 Graph-Structured Gaussian Process
Inspired by [11, 52, 68], we derive a universal message-passing graph neural network Eq. (1) for both homophily and heterophily graphs. More specifically, the message-passing graph convolutional layer of Eq. (1) for node  $v$  within an input graph  $G$  can be defined as follows.
$$
\mathbf {h} _ {v} ^ {(l)} = \sum_ {i = 0} ^ {k} \alpha_ {i} \left(\frac {1}{\sqrt {M}} \sum_ {u \in N _ {i} (v)} \mathbf {W} _ {\text {S A N N}} ^ {(l)} \mathbf {x} _ {u} ^ {(l)} + \mathbf {b} _ {\text {S A N N}} ^ {(l)}\right) \quad \text {a n d} \quad \mathbf {x} _ {u} ^ {(l)} = \phi \left(\mathbf {h} _ {u} ^ {(l - 1)}\right) \tag {3}
$$
where  $\mathbf{x}_u^{(0)} = \mathbf{x}_u\in \mathbb{R}^D$ ,  $N_{0}(v) = \{v\}$  denotes the seed node,  $\mathbf{W}_{\mathrm{SANN}}^{(l)}$ ,  $\mathbf{b}_{\mathrm{SANN}}^{(l)}$  are the weight and bias parameters, and  $\alpha_{i}$  explicitly indicates the importance of the  $i$ -order neighborhood in finding relevant neighbors around node  $v$ . Similarly, we define the domain-level graph neural layer  $\tilde{\mathbf{h}}_v^{(l)}$  parameterized by different  $\tilde{\mathbf{W}}_{\mathrm{SANN}}^{(l)}$ ,  $\tilde{\mathbf{b}}_{\mathrm{SANN}}^{(l)}$  and shared  $\alpha_{i}$ .
$$
\tilde {\mathbf {h}} _ {v} ^ {(l)} = \sum_ {i = 0} ^ {k} \alpha_ {i} \left(\frac {1}{\sqrt {M}} \sum_ {u \in N _ {i} (v)} \tilde {\mathbf {W}} _ {\text {S A N N}} ^ {(l)} \tilde {\mathbf {x}} _ {u} ^ {(l)} + \tilde {\mathbf {b}} _ {\text {S A N N}} ^ {(l)}\right) \quad \text {a n d} \quad \tilde {\mathbf {x}} _ {u} ^ {(l)} = \phi \left(\tilde {\mathbf {h}} _ {u} ^ {(l - 1)}\right) \tag {4}
$$
Generally, it is flexible to apply different model architectures to learn the node (sample-level  $\mu^{(l)}(v|G)$ ) and graph (domain-level  $\nu^{(l)}(G)$ ) representations. For simplicity, we adopt the same model architecture but different model parameters to learn node-level and graph-level representations. It is noteworthy that the neighborhood importance  $\alpha_{i}$  is shared because both node and graph representation would share the same assumption (homophily or heterophily) for a given graph. Then, the structure-aware neural network Eq. (2) can be given by
$$
f _ {i} ^ {(l)} (v, G) = \frac {1}{\sqrt {M}} \sum_ {j = 1} ^ {M} \mathbf {W} _ {i j} ^ {(l)} \cdot \mu_ {j} ^ {(l)} (v | G) \cdot \nu_ {j} ^ {(l)} (G) \tag {5}
$$
where  $\mu^{(l)}(v|G) = \mathbf{h}_v^{(l)}$  and  $\nu^{(l)}(G) = \frac{1}{|V|}\sum_{v\in V}\tilde{\mathbf{h}}_v^{(l)}$
where the READOUT function of Eq. (2) is instantiated with mean pooling [62]. Different from previous works [11, 52, 68], we would take the neighborhood importance scores  $\alpha_{i}$  as hyper-parameters for building transferable graph Gaussian process model as follows.
Theorem 4.1. Assuming that all the parameters of structure-aware graph neural network  $f(v, G)$  are independent and randomly drawn from Gaussian distributions, i.e.,  $\mathbf{W}^{(l)} \sim \mathcal{N}(\mathbf{0}, \sigma_w^2\mathbf{I})$ ,  $\mathbf{b}_{\mathrm{SANN}}^{(l)} \sim \mathcal{N}(\mathbf{0}, \varsigma_b^2\mathbf{I})$ ,  $\mathbf{W}_{\mathrm{SANN}}^{(l)} \sim \mathcal{N}(\mathbf{0}, \varsigma_w^2\mathbf{I})$ ,  $\tilde{\mathbf{b}}_{\mathrm{SANN}}^{(l)} \sim \mathcal{N}(\mathbf{0}, \tilde{\varsigma}_b^2\mathbf{I})$ ,  $\tilde{\mathbf{W}}_{\mathrm{SANN}}^{(l)} \sim \mathcal{N}(\mathbf{0}, \tilde{\varsigma}_w^2\mathbf{I})$ , when the layer width  $M$  goes to infinity, the output function  $f_i^{(l)}$  in Eq. (5) follows a Gaussian process with  $f_i^{(l)} \sim \mathcal{GP}(0, K^{(l)})$ , where the covariance function  $K^{(l)}$  is given by
$$
K ^ {(l)} \left(\left(v, G\right), \left(v ^ {\prime}, G ^ {\prime}\right)\right) = \sigma_ {w} ^ {2} \cdot K _ {\mu} ^ {(l)} \left(v, v ^ {\prime} \mid G, G ^ {\prime}\right) \cdot K _ {\nu} ^ {(l)} \left(G, G ^ {\prime}\right)
$$
$$
K _ {\mu} ^ {(l)} \left(v, v ^ {\prime} \mid G, G ^ {\prime}\right) = \sum_ {i, j = 0} ^ {k} \alpha_ {i} \alpha_ {j} ^ {\prime} \left(\varsigma_ {b} ^ {2} + \varsigma_ {w} ^ {2} \cdot \sum_ {u \in N _ {i} (v)} \sum_ {u ^ {\prime} \in N _ {j} \left(v ^ {\prime}\right)} \mathbf {C} _ {u u ^ {\prime}} ^ {(l - 1)} \left(\varsigma_ {w} ^ {2}, \varsigma_ {b} ^ {2}\right)\right)
$$
$$
K _ {\nu} ^ {(l)} \left(G, G ^ {\prime}\right) = \sum_ {i, j = 0} ^ {k} \alpha_ {i} \alpha_ {j} ^ {\prime} \left(\tilde {\varsigma} _ {b} ^ {2} + \frac {\tilde {\varsigma} _ {w} ^ {2}}{| V | \cdot | V ^ {\prime} |} \mathbf {1} ^ {T} \mathbf {P} ^ {(i)} \mathbf {C} ^ {(l - 1)} (\tilde {\varsigma} _ {w} ^ {2}, \tilde {\varsigma} _ {b} ^ {2}) \left(\mathbf {P} ^ {\prime (j)}\right) ^ {T} \mathbf {1}\right)
$$
$$
\mathbf {C} ^ {(l - 1)} (a, b) = \mathbb {E} _ {z _ {i} ^ {(l - 1)} \sim \mathcal {G P} \left(0, K _ {a b} ^ {(l - 1)}\right)} \left[ \phi (z _ {i} ^ {(l - 1)}) \phi (z _ {i} ^ {(l - 1)}) ^ {T} \right]
$$
Here  $\mathbf{P}^{(i)}(\mathbf{P}'^{(j)})$  denotes the adjacency matrix given by the  $i$ -order neighborhood from graph  $G$  ( $j$ -order neighborhood from graph  $G'$ ).  $K_{ab}^{(l-1)} = \sum_{i,j=0}^{k} \alpha_i \alpha_j'$  ( $b^2 + a^2 \mathbf{P}^{(i)} \mathbf{C}^{(l-2)}(a,b)(\mathbf{P}'^{(j)})^T$ ) and  $\mathbf{C}_{uu'}^{(0)}(a,b) = \langle \mathbf{x}_u, \mathbf{x}_{u'} \rangle$  for any  $u \in V, u' \in V'$ .
4.2.1 Implications of Theorem 4.1
In the following, we show that the sample-level covariance/kernel  $K_{\mu}^{(l)}$  of Theorem 4.1 can be explained as a message-passing operation in the kernel space. Following [4, 37], when using ReLU as activation function  $\phi(\cdot)$  (i.e.,  $\phi(x) = \max \{0, x\}$ ),  $\mathbf{C}^{(l-1)}$  is the arc-cosine kernel [12] as follows.
$$
\mathbf {C} _ {u u ^ {\prime}} ^ {(l - 1)} (\varsigma_ {w} ^ {2}, \varsigma_ {b} ^ {2}) = \frac {\kappa_ {1} (\zeta)}{2} \sqrt {K _ {\mu} ^ {(l - 1)} (u , u | G) \cdot K _ {\mu} ^ {(l - 1)} (u ^ {\prime} , u ^ {\prime} | G ^ {\prime})}
$$
where  $\zeta = \frac{K_{\mu}^{(l-1)}(u,u'|G,G')}{\sqrt{K_{\mu}^{(l-1)}(u,u|G)\cdot K_{\mu}^{(l-1)}(u',u'|G')}}$  and  $\kappa_1(\zeta) = \frac{1}{\pi}\left(\zeta \cdot (\pi - \arccos(\zeta)) + \sqrt{1 - \zeta^2}\right)$ .
Then, the following corollary derives the feature map of  $K_{\mu}^{(l)}(u,u'|G,G')$  in the kernel space.