SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To illustrate Theorem 1, we provide the specific architecture of the pre-trained GNN model  $f$ . For analytical simplicity, we initially assume that  $f$  is a single-layer GIN [Xu et al., 2019] with a linear transformation. Subsequently, we extend our conclusions to multi-layer models utilizing various transition matrices [Klicpera et al., 2019]. During the generation of graph representations, we first obtain node representations and then employ a readout function to calculate the final graph representations. Existing pre-trained GNN models commonly employ sum or mean pooling for this purpose. Previous research [Mesquita et al., 2020] suggests that complex pooling mechanisms are unnecessary, as simple readout functions can yield superior performance. It is worth mentioning that the subsequent derivations still hold when we use any other weighted aggregation readout functions, such as average pooling, min/max pooling, and hierarchical pooling. Hence, we assume that the concrete architecture of the pre-trained GNN model  $f$  can be expressed as:
$$
\mathbf {H} = \left(\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}\right) \cdot \mathbf {X} \cdot \mathbf {W} \tag {15}
$$
$$
h _ {\mathcal {G}} = \sum_ {v _ {i} \in \mathcal {V}} h _ {i} \tag {16}
$$
where  $\mathbf{W}$  is a linear projection. The parameters  $\epsilon$  and  $\mathbf{W}$  have been pre-trained in advance and remain fixed during downstream adaptation. Next, we proceed with the derivation of Theorem 1. In Theorem 1, we impose no constraints on the form of the prompting function  $\psi_t(\cdot)$ , allowing  $\hat{\mathbf{A}}$  and  $\hat{\mathbf{X}}$  to represent the adjacency matrix and feature matrix of any graph. We define the graph-level transformation  $g\colon \mathbb{G}\to \mathbb{G}$ , which satisfies  $(\hat{\mathbf{A}},\hat{\mathbf{X}}) = g(\mathbf{A},\mathbf{X})$ . Consequently, Theorem 1 is equivalent to Proposition 1.
Proposition 1. Given a pre-trained GNN model  $f$ , an input graph  $\mathcal{G} \colon (\mathbf{A},\mathbf{X})$ , for any graph-level transformation  $g \colon \mathbb{G} \to \mathbb{G}$ , there exists a GPF extra feature vector  $\hat{p}$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p}) = f (g (\mathbf {A}, \mathbf {X})) \tag {17}
$$
To further illustrate the graph-level transformation  $g(\cdot)$ , we decompose it into several specific transformations.
Proposition 2. Given an input graph  $\mathcal{G} = \{\mathbf{A},\mathbf{X}\}$ , an arbitrary graph-level transformation  $g\colon \mathbb{G}\to \mathbb{G}$  can be decoupled to a series of following transformations:
- Feature transformations. Modifying the node features and generating the new feature matrix  $\mathbf{X}' = g_{ft}(\mathbf{X})$ .
- Link transformations. Adding or removing edges and generating the new adjacency matrix  $\mathbf{A}' = g_{lt}(\mathbf{A})$ .
- Isolated component transformations. Adding or removing isolated components (sub-graphs) and generating the new adjacency matrix and feature matrix  $\mathbf{A}'$ ,  $\mathbf{X}' = g_{ict}(\mathbf{A}, \mathbf{X})$ .
Here, the word "isolated" refers to a component (sub-graph) that does not link with the rest of the graph. Proposition 2 suggests that an arbitrary graph-level transformation is a combination of the three transformations mentioned above. For instance, the deletion of a node in the initial graph can be decomposed into two steps: removing its connected edges and then removing the isolated node.
Proposition 3. Given a pre-trained GNN model  $f$ , an input graph  $\mathcal{G}$ :  $(\mathbf{A},\mathbf{X})$ , for any feature transformation  $g_{ft}$ :  $g_{ft}(\mathbf{X}) = \mathbf{X}'$ , there exists a GPF extra feature vector  $\hat{p}$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p}) = f (\mathbf {A}, \mathbf {X} ^ {\prime}) \tag {18}
$$
Proof. We set  $\Delta \mathbf{X} = \mathbf{X}' - \mathbf{X}$ . Then, we have:
$$
\begin{array}{l} \mathbf {H} ^ {\prime} = (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot \mathbf {X} ^ {\prime} \cdot \mathbf {W} (19) \\ = (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot (\mathbf {X} + \Delta \mathbf {X}) \cdot \mathbf {W} (20) \\ = (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot \mathbf {X} \cdot \mathbf {W} + (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot \Delta \mathbf {X} \cdot \mathbf {W} (21) \\ = \mathbf {H} + (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot \Delta \mathbf {X} \cdot \mathbf {W} (22) \\ \end{array}
$$
For  $\mathrm{GPF}p = [\alpha_1,\dots ,\alpha_F]\in \mathbb{R}^{1\times F}$ , we can perform a similar split:
$$
\begin{array}{l} \mathbf {H} _ {p} = (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot (\mathbf {X} + [ 1 ] ^ {N} \cdot p) \cdot \mathbf {W} (23) \\ = (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot \mathbf {X} \cdot \mathbf {W} + (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot [ 1 ] ^ {N} \cdot p \cdot \mathbf {W} (24) \\ = \mathbf {H} + (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot [ 1 ] ^ {N} \cdot p \cdot \mathbf {W} (25) \\ = \mathbf {H} + \left[ d _ {i} + 1 + \epsilon \right] ^ {N} \cdot p \cdot \mathbf {W} (26) \\ \end{array}
$$
where  $[1]^N\in \mathbb{R}^{N\times 1}$  denotes a column vector with  $N$  1s,  $[d_i + 1 + \epsilon ]^N\in \mathbb{R}^{N\times 1}$  denotes a column vector with the value of  $i$ -th row is  $d_{i} + 1 + \epsilon$  and  $d_{i}$  represents the degree number of  $v_{i}$ . To obtain the same graph representation  $h_{\mathcal{G}}$ , we have:
$$
h _ {\mathcal {G}, f t} = h _ {\mathcal {G}, p} \rightarrow \operatorname {S u m} \left(\mathbf {H} ^ {\prime}\right) = \operatorname {S u m} \left(\mathbf {H} _ {p}\right) \tag {27}
$$
where  $\mathrm{Sum}(\mathbf{M}) = \sum_{i}m_{i}$  denotes the operation that calculates the sum vector for each row in the matrix. We can further simplify the above equation as:
$$
\begin{array}{l} h _ {\mathcal {G}, f t} = h _ {\mathcal {G}, p} (28) \\ \rightarrow \operatorname {S u m} \left(\mathbf {H} ^ {\prime}\right) = \operatorname {S u m} \left(\mathbf {H} _ {p}\right) (29) \\ \rightarrow \operatorname {S u m} (\mathbf {H} + (\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}) \cdot \Delta \mathbf {X} \cdot \mathbf {W}) = \operatorname {S u m} (\mathbf {H} + [ d _ {i} + 1 + \epsilon ] ^ {N} \cdot p \cdot \mathbf {W}) (30) \\ \rightarrow \operatorname {S u m} \left(\left(\mathbf {A} + (1 + \epsilon) \cdot \mathbf {I}\right) \cdot \Delta \mathbf {X} \cdot \mathbf {W}\right) = \operatorname {S u m} \left(\left[ d _ {i} + 1 + \epsilon \right] ^ {N} \cdot p \cdot \mathbf {W}\right) (31) \\ \end{array}
$$
where the results of  $((\mathbf{A} + (1 + \epsilon) \cdot \mathbf{I}) \cdot \Delta \mathbf{X}) \in \mathbb{R}^{N \times F}$ , and the frozen linear transformation  $\mathbf{W} \in \mathbb{R}^{F \times F'}$ . We first calculate  $\Delta h_{\mathcal{G},p} = \mathrm{Sum}([d_i + 1 + \epsilon]^N \cdot p \cdot \mathbf{W}) \in \mathbb{R}^{F'}$ . We can obtain that:
$$
\begin{array}{l} \Delta h _ {\mathcal {G}, p} ^ {i} = \sum_ {j = 1} ^ {F} \sum_ {k = 1} ^ {N} \left(d _ {k} + 1 + \epsilon\right) \cdot \alpha_ {j} \cdot \mathbf {W} _ {j, i} (32) \\ = \sum_ {j = 1} ^ {F} (D + N + N \cdot \epsilon) \cdot \alpha_ {j} \cdot \mathbf {W} _ {j, i} (33) \\ \end{array}
$$
where  $h_{\mathcal{G},p}^{i}$  denotes the value of the  $i$ -th dimension in  $h_{\mathcal{G},p}$ ,  $D = \sum_{k=1}^{N} d_k$  denotes the total degree of all nodes in the whole graph,  $\alpha_j, j \in [1,F]$  denotes the  $j$ -th learnable parameter in GPF  $p$ , and  $\mathbf{W}_{j,i}, j \in [1,F], i \in [1,F']$  denotes the frozen parameter in  $\mathbf{W}$ . As for  $\Delta h_{\mathcal{G},ft} = \mathrm{Sum}((\mathbf{A} + (1 + \epsilon) \cdot \mathbf{I}) \cdot \Delta \mathbf{X} \cdot \mathbf{W})$ , we assume  $(\mathbf{A} + (1 + \epsilon) \cdot \mathbf{I}) \cdot \Delta \mathbf{X} = \mathbf{B} \in \mathbb{R}^{N \times F}$ . Then we have:
$$
\Delta h _ {\mathcal {G}, f t} ^ {i} = \sum_ {j = 1} ^ {F} \left(\sum_ {k = 1} ^ {N} \beta_ {k, j}\right) \cdot \mathbf {W} _ {j, i} \tag {34}
$$
where  $\beta_{k,j}, k \in [1,N], j \in [1,F]$  denotes the learnable parameter in  $\mathbf{B}$ . According to above analysis, to obtain a same graph representation  $h_{\mathcal{G},\hat{p}}$  with a certain  $h_{\mathcal{G},ft}$ , we have:
$$
h _ {\mathcal {G}, \hat {p}} ^ {i} = h _ {\mathcal {G}, f t} ^ {i}, \text {f o r e v e r y} i \in [ 1, F ^ {\prime} ] \tag {35}
$$
$$
\rightarrow \Delta h _ {\mathcal {G}, \hat {p}} ^ {i} = \Delta h _ {\mathcal {G}, f t} ^ {i} \tag {36}
$$
$$
\rightarrow \alpha_ {j} = \frac {\sum_ {k = 1} ^ {N} \beta_ {k , j}}{D + N + N \cdot \epsilon}, j \in [ 1, F ] \tag {37}
$$
Therefore, for an arbitrary feature transformation  $g_{ft}$ , there exists a GPF  $\hat{p}$  that satisfies the above conditions and can obtain the exact graph representation for the pre-trained GNN model  $f$ .
Proposition 3 demonstrates the comprehensive coverage of our proposed GPF for all graph-level feature transformations. GPF introduces a uniform feature modification  $p \in \mathbb{R}^{F}$  to each node in the graph. However, it can achieve an equivalent effect to adding independent feature modifications to each node individually under the pre-trained GNN model described above.
Proposition 4. Given a pre-trained GNN model  $f$ , an input graph  $\mathcal{G}$ :  $(\mathbf{A},\mathbf{X})$ , for any link transformation  $g_{lt}$ :  $g_{lt}(\mathbf{A}) = \mathbf{A}'$ , there exists a GPF extra feature vector  $\hat{p}$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p}) = f \left(\mathbf {A} ^ {\prime}, \mathbf {X}\right) \tag {38}
$$
Proof. The proof of Proposition 4 is similar to that of Proposition 3. We set  $\Delta \mathbf{A} = \mathbf{A}' - \mathbf{A}$ . It is worth mentioning that  $\mathbf{A}, \mathbf{A}' \in \{0,1\}^{N \times N}$  and  $\Delta \mathbf{A} \in \{-1,0,1\}^{N \times N}$ , which means they are of the same size, the values of  $\mathbf{A}, \mathbf{A}'$  can only be 0 or 1, and the values of  $\Delta \mathbf{A}$  can be  $-1,0$  or 1. We have:
$$
\mathbf {H} ^ {\prime} = \left(\mathbf {A} ^ {\prime} + (1 + \epsilon) \cdot \mathbf {I}\right) \cdot \mathbf {X} \cdot \mathbf {W} \tag {39}
$$
$$
= \left(\left(\mathbf {A} + \Delta \mathbf {A}\right) + (1 + \epsilon) \cdot \mathbf {I}\right) \cdot \mathbf {X} \cdot \mathbf {W} \tag {40}
$$
$$
= \mathbf {H} + \Delta \mathbf {A} \cdot \mathbf {X} \cdot \mathbf {W} \tag {41}
$$
From the proof of Proposition 3, we obtain:
$$
\mathbf {H} _ {p} = \mathbf {H} + \left[ d _ {i} + 1 + \epsilon \right] ^ {N} \cdot p \cdot \mathbf {W} \tag {42}
$$
where  $p = [\alpha_1, \dots, \alpha_F] \in \mathbb{R}^{1 \times F}$  denotes our learnable GPF,  $[d_i + 1 + \epsilon]^N \in \mathbb{R}^{N \times 1}$  denotes a column vector with the value of  $i$ -th line is  $d_i + 1 + \epsilon$  and  $d_i$  represents the degree number of  $v_i$ . With  $\Delta h_{\mathcal{G}, p} = \mathrm{Sum}([d_i + 1 + \epsilon]^N \cdot p \cdot \mathbf{W}) \in \mathbb{R}^{F'}$ , we can obtain:
$$
\Delta h _ {\mathcal {G}, p} ^ {i} = \sum_ {j = 1} ^ {F} \sum_ {k = 1} ^ {N} \left(d _ {k} + 1 + \epsilon\right) \cdot \alpha_ {j} \cdot \mathbf {W} _ {j, i} \tag {43}
$$
$$
= \sum_ {j = 1} ^ {F} (D + N + N \cdot \epsilon) \cdot \alpha_ {j} \cdot \mathbf {W} _ {j, i} \tag {44}
$$
where  $h_{\mathcal{G},p}^i$  denotes the value of the  $i$ -th dimension in  $h_{\mathcal{G},p}$ ,  $D = \sum_{k=1}^{N} d_k$  denotes the total degree of all nodes in the whole graph,  $\alpha_j, j \in [1,F]$  denotes the  $j$ -th learnable parameter in GPF  $p$ , and  $\mathbf{W}_{j,i}, j \in [1,F], i \in [1,F']$  denotes the frozen parameter in  $\mathbf{W}$ . As for  $\Delta h_{\mathcal{G},lt} = \mathrm{Sum}(\Delta \mathbf{A} \cdot \mathbf{X} \cdot \mathbf{W})$ , we have:
$$
\Delta h _ {\mathcal {G}, l t} ^ {i} = \sum_ {j = 1} ^ {F} \sum_ {(k, l) \in N \times N} \left(\Delta a _ {k, l} \cdot x _ {l, j}\right) \cdot \mathbf {W} _ {j, i} \tag {45}
$$
where  $\Delta a_{k,l}, k \in [1,N], l \in [1,N]$  denotes the element of  $\Delta \mathbf{A}$ , and  $x_{l,j}, l \in [1,N], j \in [1,F]$  denotes the element of  $\mathbf{X}$ . To obtain a same graph representation  $h_{\mathcal{G},\hat{p}}$  with a certain  $h_{\mathcal{G},lt}$ , we have:
$$
h _ {\mathcal {G}, \hat {p}} ^ {i} = h _ {\mathcal {G}, l t} ^ {i}, \text {f o r e v e r y} i \in [ 1, F ^ {\prime} ] \tag {46}
$$
$$
\rightarrow \Delta h _ {\mathcal {G}, \hat {p}} ^ {i} = \Delta h _ {\mathcal {G}, l t} ^ {i} \tag {47}
$$
$$
\rightarrow \alpha_ {j} = \frac {\sum_ {(k , l) \in N \times N} \Delta a _ {k , l} \cdot x _ {l , j}}{D + N + N \cdot \epsilon}, j \in [ 1, F ] \tag {48}
$$
Therefore, for an arbitrary link transformation  $g_{lt}$ , there exists a GPF  $\hat{p}$  that satisfies above conditions and can obtain the same graph representation for pre-trained GNN model  $f$ .
Proposition 4 demonstrates that GPF can also encompass all link transformations. Intuitively, link transformations are closely associated with changes in the adjacency matrix, which are independent of node features. However, our findings reveal that, within the architecture of most existing GNN models, modifications in the feature space and modifications in the structural space can produce equivalent effects.
Proposition 5. Given a pre-trained GNN model  $f$ , an input graph  $\mathcal{G}$ :  $(\mathbf{A},\mathbf{X})$ , for any isolated component transformation  $g_{ict} \colon g_{ict}(\mathbf{A},\mathbf{X}) = \mathbf{A}',\mathbf{X}'$ , there exists a GPF extra feature vector  $\hat{p}$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p}) = f \left(\mathbf {A} ^ {\prime}, \mathbf {X} ^ {\prime}\right) \tag {49}
$$
Proof. Unlike feature transformations and linear transformations, isolated component transformations will change the number of nodes in the graph, which means the scale of modified  $\mathbf{A}'$  and  $\mathbf{X}'$  is uncertain. We first express the isolated component transformation in more details. The adjacency matrix  $\mathbf{A}$  and feature matrix  $\mathbf{X}$  can be divided into several isolated components, which can be expressed as:
$$
\mathbf {A} = \left( \begin{array}{c c c c} \mathbf {A} _ {1} & 0 & \dots & 0 \\ 0 & \mathbf {A} _ {2} & \dots & 0 \\ \vdots & \vdots & & \vdots \\ 0 & 0 & \dots & \mathbf {A} _ {m} \end{array} \right) \quad \mathbf {X} = \left( \begin{array}{c} \mathbf {X} _ {1} \\ \mathbf {X} _ {2} \\ \vdots \\ \mathbf {X} _ {m} \end{array} \right) \tag {50}
$$
Removing an isolated component  $\mathcal{C}_k = \{\mathbf{A}_k, \mathbf{X}_k\}, k \in [1, m]$  means removing both  $\mathbf{A}_k$  in the adjacency matrix and corresponding  $\mathbf{X}_k$  in the feature matrix. Adding a new isolated component  $\mathcal{C}_{m+l} = \{\mathbf{A}_{m+l}, \mathbf{X}_{m+l}\}, l \geq 1$  means adding  $\mathbf{A}_{m+l}$  to the adjacency matrix  $\mathbf{A}$ , and adding  $\mathbf{X}_{m+l}$  to the corresponding position of  $\mathbf{X}$ . Then we have:
$$
h _ {\mathcal {G}, i s t} = \sum_ {k} \operatorname {S u m} \left(\left(\mathbf {A} _ {k} + (1 + \epsilon) \cdot \mathbf {I}\right) \cdot \mathbf {X} _ {k} \cdot \mathbf {W}\right) \tag {51}
$$
To align with the proofs of Proposition 3 and Proposition 4, we set  $\Delta h_{\mathcal{G},ist} = h_{\mathcal{G},ist} - \mathrm{Sum}((\mathbf{A} + (1 + \epsilon)\cdot \mathbf{I})\cdot \mathbf{X}\cdot \mathbf{W})$ , and it can be expressed as:
$$
\Delta h _ {\mathcal {G}, i s t} = \sum_ {k} I _ {k} \cdot \operatorname {S u m} \left(\left(\mathbf {A} _ {k} + (1 + \epsilon) \cdot \mathbf {I}\right) \cdot \mathbf {X} _ {k} \cdot \mathbf {W}\right) \tag {52}
$$
where  $I_{k}$  is an indicator that satisfies:
$$
I _ {k} = \left\{ \begin{array}{l l} 0 & \text {i f} \mathcal {C} _ {k} \text {h a s n o c h a n g e} \\ 1 & \text {i f} \mathcal {C} _ {k} \text {i s a n a d d i t i o n a l c o m p o n e n t} \\ - 1 & \text {i f} \mathcal {C} _ {k} \text {i s a r e m o v e d c o m p o n e n t} \end{array} \right. \tag {53}
$$
From the proof of Proposition 3, we have following conclusions:
$$
\mathbf {H} _ {p} = \mathbf {H} + \left[ d _ {i} + 1 + \epsilon \right] ^ {N} \cdot p \cdot \mathbf {W} \tag {54}
$$
where  $p = [\alpha_1, \dots, \alpha_F] \in \mathbb{R}^{1 \times F}$  denotes our learnable GPF,  $[d_i + 1 + \epsilon]^N \in \mathbb{R}^{N \times 1}$  denotes a column vector with the value of  $i$ -th line is  $d_i + 1 + \epsilon$  and  $d_i$  represents the degree number of  $v_i$ .
With  $\Delta h_{\mathcal{G},p} = \mathrm{Sum}([d_i + 1 + \epsilon ]^N\cdot p\cdot \mathbf{W})\in \mathbb{R}^{F'}$ , we can obtain:
$$
\begin{array}{l} \Delta h _ {\mathcal {G}, p} ^ {i} = \sum_ {j = 1} ^ {F} \sum_ {k = 1} ^ {N} \left(d _ {k} + 1 + \epsilon\right) \cdot \alpha_ {j} \cdot \mathbf {W} _ {j, i} (55) \\ = \sum_ {j = 1} ^ {F} (D + N + N \cdot \epsilon) \cdot \alpha_ {j} \cdot \mathbf {W} _ {j, i} (56) \\ \end{array}
$$
where  $h_{\mathcal{G},p}^{i}$  denotes the value of the  $i$ -th dimension in  $h_{\mathcal{G},p}$ ,  $D = \sum_{k=1}^{N} d_k$  denotes the total degree of all nodes in the whole graph,  $\alpha_j, j \in [1,F]$  denotes the  $j$ -th learnable parameter in GPF  $p$ , and  $\mathbf{W}_{j,i}, j \in [1,F], i \in [1,F']$  denotes the frozen parameter in  $\mathbf{W}$ . To obtain a same graph representation  $h_{\mathcal{G},\hat{p}}$  with a certain  $h_{\mathcal{G},ist}$ , we have:
$$
\begin{array}{l} h _ {\mathcal {G}, \hat {p}} ^ {i} = h _ {\mathcal {G}, i s t} ^ {i}, \text {f o r e v e r y} i \in [ 1, F ^ {\prime} ] (57) \\ \rightarrow \Delta h _ {\mathcal {G}, \hat {p}} ^ {i} = \Delta h _ {\mathcal {G}, i s t} ^ {i} (58) \\ \rightarrow \alpha_ {j} = \frac {\sum_ {k} I _ {k} \cdot \operatorname {S u m} \left(\left(\mathbf {A} _ {k} + (1 + \epsilon) \cdot \mathbf {I}\right) \cdot \mathbf {X} _ {k} ^ {j}\right)}{D + N + N \cdot \epsilon}, j \in [ 1, F ] (59) \\ \end{array}
$$
where  $\mathbf{X}^j$  denotes the  $j$ -th column of the matrix  $\mathbf{X}$ . Therefore, for an arbitrary isolated component transformation  $g_{ict}$ , there exists a GPF  $\hat{p}$  that satisfies above conditions and can obtain the same graph representation for pre-trained GNN model  $f$ .
The isolated component transformation possesses the capability to alter the scale of a graph, and previous studies have paid limited attention to this type of graph-level transformation.
Proposition 6. Given a pre-trained GNN model  $f$ , an input graph  $\mathcal{G}$ : (A, X), for a series of transformations  $\mathbf{g} = \{g_1, g_2, \dots, g_k\}$  composed of  $g_{ft}, g_{lt}$  and  $g_{ist}$ , there exists a GPF extra feature vector  $\hat{p}$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p}) = f (\mathbf {g} (\mathbf {A}, \mathbf {X})) \tag {60}
$$
Proof. Without loss of generality, we consider  $\mathbf{g} = \{g_1, g_2\}$  with two transformations described in Proposition 2. Now we prove there exists a GPF  $\hat{p}$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + p) = f \left(g _ {2} \left(g _ {1} (\mathbf {A}, \mathbf {X})\right)\right) \tag {61}
$$
We assume  $g_{1}(\mathbf{A},\mathbf{X}) = \mathbf{A}'$ ,  $\mathbf{X}'$ . According to Proposition 3, 4, 5, there exists a GPF  $\hat{p}_{1}$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p} _ {1}) = f \left(g _ {1} (\mathbf {A}, \mathbf {X})\right) \tag {62}
$$
and a  $\hat{p}_2$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p} _ {2}) = f \left(g _ {2} \left(\mathbf {A} ^ {\prime}, \mathbf {X} ^ {\prime}\right)\right) \tag {63}
$$
Therefore, there is a  $\hat{p} = \hat{p}_1 + \hat{p}_2$  that satisfies:
$$
f (\mathbf {A}, \mathbf {X} + \hat {p}) = f \left(g _ {2} \left(g _ {1} (\mathbf {A}, \mathbf {X})\right)\right) \tag {64}
$$
Based on the preceding analysis, we have established that the GPF can replicate the effects of any graph-level transformation on the pre-trained GNN model defined by Formula 15 and 16. Hence, we have successfully demonstrated Proposition 1 and Theorem 1 within the framework of the simple model architecture. Next, we aim to generalize our findings to more intricate scenarios.
Extension to other GNN backbones. We use GIN Xu et al. [2019] as the default backbone model in our previous derivation. When replacing GIN with another GNN model, only slight modifications are required to ensure that all propositions remain valid. As is described in Klicpera et al. [2019], various GNN models can be represented as:
$$
\mathbf {H} = \mathbf {S} \cdot \mathbf {X} \cdot \mathbf {W} \tag {65}
$$
where  $\mathbf{S} \in \mathbb{R}^{N \times N}$  denotes the diffusion matrix (e.g.,  $\mathbf{A} + (1 + \epsilon) \cdot \mathbf{I}$  is the diffusion matrix for GIN), and  $\mathbf{W}$  denotes the linear projection. In this case, we modify the Formula 26, 42 and 54 as follows:
$$
\mathbf {H} _ {p} = \mathbf {H} + \left[ d _ {i} + 1 + \epsilon \right] ^ {N} \cdot p \cdot \mathbf {W} \tag {66}
$$
$$
\rightarrow \mathbf {H} _ {p} = \mathbf {H} + \left[ \sum_ {j} s _ {i, j} \right] ^ {N} \cdot p \cdot \mathbf {W} \tag {67}
$$
where  $s_{i,j}$  is the element of the diffusion matrix  $\mathbf{S}$ . With these modifications, Propositions 3, 4, and 5 remain valid.
Extension to multi-layer models. For analytical simplification, similar to many previous works, we consider multi-layer GNN models without non-linear activation functions between layers, which can be expressed as:
$$
\mathbf {H} _ {(0)} = \mathbf {X} \tag {68}
$$
$$
\mathbf {H} _ {(k)} = \mathbf {S} _ {(k)} \cdot \mathbf {H} _ {(k - 1)} \cdot \mathbf {W} _ {(k)} \tag {69}
$$
where  $\mathbf{S}_{(k)}$  is the diffusion matrix described as Formula 65 of the  $k$ -th layer, and  $\mathbf{W}_{(k)}$  is the linear projection of the  $k$ -th layer. With such architecture, the node representations of the  $k$ -th layer  $\mathbf{H}_{(k)}$  can also be obtained as:
$$
\mathbf {H} _ {(k)} = \left(\prod_ {i = 1} ^ {k} \mathbf {S} _ {(i)}\right) \cdot \mathbf {X} \cdot \left(\prod_ {i = 1} ^ {k} \mathbf {W} _ {(i)}\right) = \mathbf {S} ^ {\prime} \cdot \mathbf {X} \cdot \mathbf {W} ^ {\prime} \tag {70}
$$
where  $\mathbf{S}' = \prod_{i=1}^{k} \mathbf{S}_{(i)}$  and  $\mathbf{W}' = \prod_{i=1}^{k} \mathbf{W}_{(i)}$ . By substituting  $\mathbf{S} = \mathbf{S}'$  and  $\mathbf{W} = \mathbf{W}'$  in Formula 65, we can find that Proposition 3, 4 and 5 still hold true.
A.3 Proof for Theorem 2
Theorem 2. (Effectiveness Guarantee of GPF) For a pre-trained GNN model  $f$ , a series of graphs  $\mathcal{D} = \{(\mathcal{G}_1: (\mathbf{A}_1, \mathbf{X}_1), y_1), \ldots, (\mathcal{G}_m: (\mathbf{A}_m, \mathbf{X}_m), y_m)\}$  under the non-degeneracy condition, and a linear projection head  $\theta$ , there exists  $\mathcal{V}' = \{y_1', \ldots, y_m'\}$  for  $y_1 = y_1', \ldots, y_m = y_m'$  that satisfies:
$$
l _ {\mathrm {G P F}} = \min  _ {p, \theta} \sum_ {i} ^ {m} (f (\mathbf {A} _ {i}, \mathbf {X} _ {i} + p) \cdot \theta - y _ {i}) ^ {2} <   l _ {\mathrm {F T}} = \min  _ {f, \theta} \sum_ {i} ^ {m} (f (\mathbf {A} _ {i}, \mathbf {X} _ {i}) \cdot \theta - y _ {i}) ^ {2}
$$
To demonstrate Theorem 2, we need to provide a more detailed description of the architecture of the GNN model  $f$ . We assume that the graph representations  $h_{\mathcal{G}_i}$  for  $\mathcal{G}_i$  are obtained through the following process:
$$
\mathbf {H} _ {i} = \mathbf {S} _ {i} \cdot \mathbf {X} _ {i} \cdot \mathbf {W} \tag {71}
$$
$$
h _ {\mathcal {G} _ {i}} = \operatorname {S u m} \left(\mathbf {H} _ {i}\right) \tag {72}
$$
where  $\mathbf{S}_i\in \mathbb{R}^{N_i\times N_i}$  denotes the diffusion matrix of  $\mathcal{G}_i$  as Formula 65,  $N_{i}$  denotes the node number of the graph  $\mathcal{G}_i$ ,  $\mathbf{W}\in \mathbb{R}^{F\times F^{\prime}}$  denotes the linear projection,  $F$  is the dimension of node features, and  $\mathrm{Sum}(\mathbf{M}) = \sum_{i}m_{i}$  denotes the operation that calculates the sum vector for each row in the matrix  $\mathbf{M}$ . When we employ our proposed GPF in the above model  $f$ , the graph representations  $h_{\mathcal{G}_i}^{\mathrm{GPF}}$  are calculated as:
$$
\mathbf {H} _ {i} ^ {\mathrm {G P F}} = \mathbf {S} _ {i} \cdot \left(\mathbf {X} _ {i} + [ 1 ] ^ {\mathrm {T}} \cdot p\right) \cdot \mathbf {W} \tag {73}
$$