SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We will once again use induction to show that such parametrizations exist.
Base case.  $L = 1$ . The properties are equivalent to the ones in the less efficient construction: property (1) is the same, and  $2^{(L - 1)} = 2^0 = L$ , so we will satisfy them with the same parameterization.
Property (1): This is satisfied in the same way as in the less efficient construction - we allocate one of the dimensions for the required count, and propagate it from the previous layer.
Property (2): Let us consider an arbitrary cut node  $u$  that has strictly less than  $2^{(L - 1)}$  cut nodes in its induced subtree  $\delta_{u}$ , and still does not satisfy property (2). We will call such nodes "improper" with respect to the current induction step, and show that there is a single layer CUTENC parameterization that makes all improper nodes satisfy these two properties, or become "proper".
First, observe that an arbitrary cut node  $u = v_{0}$  has at most one improper grandchild  $v_{1} \in \chi^{2}(u)$ , because if there were more improper grandchildren then it would have had at least  $2^{(L - 1)}$  cut nodes in it. Repeating a similar argument, we know that there is at most one improper  $v_{2} \in \chi^{2}(v_{1})$ . Continuing this until there are no improper grandchildren, we form a sequence of improper cut nodes  $u = v_{0}, v_{1}, \ldots, v_{k}$ , such that  $v_{i + 1}$  is a grandchild of  $v_{i}$  for all  $0 \leq i < k$ . The chain  $u = v_{0}, v_{1}, \ldots, v_{k}$  is precisely a chain in HLD, and we will show that a single CUTENC layer can make an "improper chain" satisfy the properties. We prove this by applying inner induction on the chain:
Induction hypothesis (for grandchildren): By the induction hypothesis, we have an established  $(L - 1)$ -layer parameterization of BASEPLANE that satisfies properties (1)-(3) for all but a single grandchild of each node in the chain  $v_{0},\ldots,v_{k}$ . This assumption is critical as it provides the ground truth that our subsequent steps will build upon.
Inner induction: Having established the hypothesis for our node's children, we initiate an inner induction, starting from the farthest improper node in the chain,  $v_{k}$ , and progressively moving towards the root of the chain,  $u = v_{0}$ . This process is akin to traversing back up the chain, ensuring each node, starting from  $v_{k}$ , satisfies property (2) as we ascend.
For each node  $v_{i}$  in our reverse traversal, we apply the special property: there exists at most one improper grandchild (potentially  $v_{i+1}$ ) within its local structure. Our CUTENC architecture then identifies this improper grandchild, which, crucially, is "proper" in its representation due to our inner induction process. This identification is facilitated by the inner MLP given in Equation (10), and property (1), as identifying is equivalent to selecting the grandchild with the largest number of cut nodes in the allocated dimension.
Having identified and isolated our improper grandchild  $v_{i+1}$  (now proper in  $\overline{h}_{\delta_{v_{i+1}}}^{(L)}$ ), CUTENO then integrates this node with the other children and grandchildren of  $v_{i}$ . It is important to note that all these other nodes have representations bijective to their respective subtree codes because of the induction hypothesis. This integration is not a mere collection; instead, the outer MLP in
Equation (10) models the "code operation" that BICODE would have done, which is possible due to the mentioned bijections to codes. This concludes the inner induction.
Our choice for  $u$  was arbitrary and the required CUTENC parameterization is independent from the exact node representation - it simply models a translation back to codes, and then injecting back to embeddings. Hence, all chains of improper nodes will satisfy property (2) after a single CUTENC layer.
Property (3): We once again follow the same approach as in the less efficient construction - we use property (2) as a bridge, as we know we have already satisfied it. Property (3) is needed to ensure that all nodes that once became "proper" will always stay so, by propagating the bijection to codes.
This concludes our construction using  $\lceil \log_2(n) \rceil + 1$  layers.
Theorem 6.1. For any planar graphs  $G_{1} = (V_{1},E_{1},\zeta_{1})$  and  $G_{2} = (V_{2},E_{2},\zeta_{2})$ , there exists a parameterization of BASEPLANE with at most  $L = \lceil \log_2(\max \{|V_1|, |V_2|\}) \rceil + 1$  layers, which computes a complete graph invariant, that is, the final graph-level embeddings satisfy  $z_{G_1}^{(L)} \neq z_{G_2}^{(L)}$  if and only if  $G_{1}$  and  $G_{2}$  are not isomorphic.
Proof. The "only if" direction is immediate because BASEPLANE is an invariant model for planar graphs. To prove the "if" direction, we do a case analysis on the root of the two Block-Cut Trees. For each case, we provide a parameterization of BASEPLANE such that  $z_{G_1}^{(L)} \neq z_{G_2}^{(L)}$  for any two non-isomorphic graphs  $G_1$  and  $G_2$ . A complete BASEPLANE model can be obtained by appropriately unifying the respective parameterizations.
Case 1.  $\mathrm{ROOT}(\delta_1)$  and  $\mathrm{ROOT}(\delta_2)$  represents two cut nodes.
Consider a parameterization of the final BASEPLANE update formula, where only cut node representation is used, and a simplified readout that only aggregates from the last layer. We can rewrite the readout for a graph in terms of the cut node representation from the last BASEPLANE layer:
$$
\boldsymbol {z} _ {G} ^ {(L)} = \operatorname {M L P} \left(\sum_ {u \in V ^ {G}} \operatorname {M L P} \left(\bar {\boldsymbol {h}} _ {\delta_ {u}} ^ {(L)}\right)\right).
$$
Let  $\mathrm{CUT}(G) = \{\{\overline{h}_{\delta_u}^{(L)}|u\in V^G\}\}$ . Intuitively,  $\mathrm{CUT}(G)$  is a multiset of cut node representations from the last BASEPLANE layer. We assume  $|V_{\delta_1}|\leq |V_{\delta_2}|$  without loss of generality. Consider the root node  $\mathrm{ROOT}(\delta_2)$  of the Block-Cut Tree  $\delta_{2}$ . By Lemma 6.3, we have  $h_{\mathrm{ROOT}(\delta_2)}$  as a complete graph invariant with  $L$  layers. Since  $\delta_{2}$  cannot appear as a subtree of  $\delta_{1}$ ,  $h_{\mathrm{ROOT}(\delta_2)}\notin \mathrm{CUT}(G_1)$ . Hence,  $\mathrm{CUT}(G_1)\neq \mathrm{CUT}(G_2)$ . Since this model can define an injective mapping on the multiset  $\mathrm{CUT}(G)$  using similar arguments as before, we get that  $z_{G_1}^{(L)}\neq z_{G_2}^{(L)}$ .
Case 2.  $\mathrm{ROOT}(\delta_1)$  and  $\mathrm{ROOT}(\delta_2)$  represents two biconnected components.
We use a similar strategy to prove Case 2. First, we consider a simplified BASEPLANE model, where the update formula considers biconnected components only and the final readout aggregates from the last BASEPLANE layer. We similarly give the final graph readout in terms of the biconnected component representation from the last BASEPLANE layer.
$$
\boldsymbol {z} _ {G} = \operatorname {M L P} \left(\sum_ {u \in V ^ {G}} \operatorname {M L P} \left(\left(\boldsymbol {h} _ {u} ^ {(L - 1)} + \sum_ {B \in \pi_ {u} ^ {G}} \widetilde {\boldsymbol {h}} _ {B} ^ {(L)}\right)\right)\right).
$$
Let  $\operatorname{Bc}(G) = \{(h_u^{(L-1)}, \{\widetilde{h}_B^{(L)} \mid B \in \pi_u^G\}) | u \in V^G\}$ . In Lemma 6.3, we prove that  $\widetilde{h}_B^{(L)}$  is also a complete invariant for the subtree rooted at  $B$  in the Block-Cut Tree. First, we show  $\operatorname{BC}(G_1) \neq \operatorname{BC}(G_2)$ . As before, we assume  $|V_{\delta_1}| \leq |V_{\delta_2}|$  without loss of generality. Consider how
the biconnected component representation  $h_{\mathrm{ROOT}(\delta_2)}$  appears in the two multisets of pairs. For  $G_2$ , there exist at least one node  $u$  and pair:
$$
\left(\boldsymbol {h} _ {u} ^ {(L - 1)}, \{\{\widetilde {\boldsymbol {h}} _ {B} ^ {(L)} \mid B \in \pi_ {u} ^ {G} \}\}\right) \in \mathrm {B c} (G _ {2}),
$$
such that  $\pmb{h}_{\mathrm{ROOT}(\delta_2)} \in \{\{\widetilde{\pmb{h}}_B^{(L)}|B \in \pi_u^G\} \}$ . However, because  $\pmb{h}_{\mathrm{ROOT}(\delta_2)}$  is a complete invariant for  $\delta_{2}$  and  $\delta_{2}$  cannot appear as a subtree in  $\delta_{1}$ , no such pair exists in  $\mathrm{BC}(G_1)$ . Given  $\mathrm{BC}(G_1) \neq \mathrm{BC}(G_2)$ , we can parameterize the MLPs to define an injective mapping to get that  $z_{G_1}^{(L)} \neq z_{G_2}^{(L)}$ .
Case 3.  $\mathrm{ROOT}(\delta_1)$  represents a cut node and  $\mathrm{ROOT}(\delta_2)$  represents a biconnected component.
We can distinguish  $G_{1}$  and  $G_{2}$  using a simple property of  $\mathrm{ROOT}(\delta_1)$ . Recall that  $\pi_u^G$  represents the set of biconnected components that contains  $u$ , and  $\chi^{\delta}(C)$  represents the C's children in the Block-Cut Tree  $\delta$ . For  $\mathrm{ROOT}(\delta_1)$ , we have  $|\pi_u^{G_1}| = |\chi^{\delta_1}(u)|$ . However, for any other cut node  $u$ , including the non-root cut node in  $\delta_1$  and all cut nodes in  $\delta_2$ , we have  $|\pi_u^G| = |\chi^\delta (u)| + 1$ , because there must be a parent node  $v$  of  $u$  such that  $v\in \pi_u^{G_1}$  but  $v\notin \chi^{\delta}(u)$ .
Therefore, we consider a parameterization of a one-layer BASEPLANE model that exploits this property. In BIENC, we have a constant vector  $[1,0]^{\top}$  for all biconnected components. In CUTENC, we learn  $[0,|\chi_u|]^{\top}$  for all cut nodes  $u$ . In the update formula, we have  $\left[|\pi_u^G | - |\chi^\delta (u)| - 1,0\right]^{\top}$ . All of the above specifications can be achieved using linear maps. For the final readout, we simply sum all node representations with no extra transformation.
Then, for  $\mathrm{ROOT}(\delta_1)$ , we have  $\pmb{h}_{\mathrm{ROOT}(\delta_u)} = [-1,0]^\top$ . For any other cut node  $u$ , we have  $\pmb{h}_u = [0,0]^\top$ . For all non-cut nodes  $u$ , we also have  $\pmb{h}_u = [0,0]^\top$  because  $|\pi_u^G| = 1$  and  $\overline{\pmb{h}}_{\delta_u} = [0,0]^\top$ . Summing all the node representations yields  $\pmb{z}_{G_1} = [-1,0]^\top$  but  $\pmb{z}_{G_2} = [0,0]^\top$ . Hence, we obtain  $\pmb{z}_{G_1}^{(L)} \neq \pmb{z}_{G_2}^{(L)}$ , as required.
B Runtime analysis of BASEPLANE
B.1 Asymptotic analysis
In this section, we study the runtime complexity of the BASEPLANE model.
Computing components of a planar graph. Given an input graph  $G = (V, E, \zeta)$ , BASEPLANE first computes the SPQR components/SPQR tree, and identifies cut nodes. For this step, we follow a simple  $O(|V|^2)$  procedure, analogously to the computation in the KHC algorithm. Note that this computation only needs to run once, as a pre-processing step. Therefore, this pre-computation does not ultimately affect runtime for model predictions.
Size and number of computed components.
1. Cut nodes: The number of cut nodes in  $G$  is at most  $|V|$ , and this corresponds to the worst-case when  $G$  is a tree.
2. Biconnected Components: The number of biconnected components  $|\pi^G|$  is at most  $|V| - 1$ , also obtained when  $G$  is a tree. This setting also yields a worst-case bound of  $2|V| - 2$  on the total number of nodes across all individual biconnected components.
3. SQPR components: As proved by Gutwenger and Mutzel [27], given a biconnected component  $B$ , the number of corresponding SPQR components, as well as their size, is bounded by the number of nodes in  $B$ . The input graph may have multiple biconnected components, whose total size is bounded by  $2|V| - 2$  as described earlier. Thus, we can apply the earlier result from Gutwenger and Mutzel [27] to obtain an analogous bound of  $2|V| - 2$  on the total number of SPQR components.
4. SPQR trees: As each SPQR component corresponds to exactly one node in a SPQR tree. The total size of all SPQR trees is upper-bounded by  $2|V| - 2$ .
Complexity of TRIENC. TRIENC computes a representation for each SPQR component edge. This number of edges, which we denote by  $e_{C}$ , is in fact linear in  $V$ : Indeed, following Euler's theorem for planar graphs  $(|E| \leq 3|V| - 6)$ , the number of edges per SPQR component is linear in its size.
Moreover, since the total number of nodes across all SPQR components is upper-bounded by  $2|V| - 2$ , the total number of edges is in  $O(V)$ . Therefore, as each edge representation can be computed in a constant time with an MLP, TRIENC computes all edge representations across all SPQR components in  $O(e_{C} \cdot d) = O(|V|d^{2})$  time, where  $d$  denotes the embedding dimension of the model.
Using the earlier edge representations, TRIENC performs an aggregation into a triconnected component representation by summing the relevant edge representations, and this is done in  $O(e_{C}d)$ . Then, the sum outputs across all SPQR components are transformed using an MLP, in  $O(|\sigma^{G}|d^{2})$ . Therefore, the overall complexity of TRIENC is  $O((e_c + |\sigma^G |)d^2) = O(|V|d^2)$ .
Complexity of BIENC. BIENC recursively encodes nodes in the SPQR tree. Each SPQR node is aggregated once by its parent and an MLP is applied to each node representation once. The total complexity of this call is therefore  $O(|\sigma^G|d^2) = O(|V|d^2)$ .
Complexity of CUTENC. A similar complexity of  $O(|V|d^2)$  applies for CUTENC, as CUTENC follows a similar computational pipeline.
Complexity of node update. As in standard MLPs, message aggregation runs in  $O(|E|d)$ , and the combine function runs in  $O(|V|d^2)$ . Global readouts can be computed in  $O(|V|d)$ . Aggregating from bi-connected components also involves at most  $O(|V|d)$  messages, as the number of messages corresponds to the total number of bi-connected component nodes, which itself does not exceed  $2|V| - 2$ . The same argument applies to messages from SPQR components: nodes within these components will message their original graph analogs, leading to the same bound. Finally, the cut node messages are linear, i.e.,  $O(|V|)$ , as each node receives exactly one message (no aggregation, no transformation). Overall, this leads to the step computation having a complexity of  $O(|V|d^2)$ .
Overall complexity. Each BASEPLANE layer runs in  $O(|V|d^2)$  time, as this is the asymptotic bound of each of its individual steps. The  $d^2$  term primarily stems from MLP computations, and is not a main hurdle to scalability, as the used embedding dimensionality in our experiments is usually small.
Parallelization. Parallelization can be used to speed up the pre-processing of large planar graphs. In particular, parallelization can reduce the runtime of KHC pre-processing, and more precisely the computation of the canonical walk in Weinberg's algorithm, which itself is the individual step with the highest complexity  $(O(|V|^2))$  in our approach. To this end, one can independently run Weinberg's algorithm across several triconnected components in parallel. Moreover, we can also parallelize the algorithm's operation within a single triconnected component: as the quadratic overhead comes from computing the lexicographically smallest TRICODE, where the computation depends on the choice of the first walk edge, we can concurrently evaluate each first edge option and subsequently aggregate over all outputs to more efficiently find the smallest code.
B.2 Empirical runtime evaluation
Experimental setup. To validate the runtime efficiency of BASEPLANE, we conduct a runtime experiment on real-world planar graphs based on the geographic faces of Alaska from the dataset TIGER, provided by the U.S. Census Bureau. The original dataset is TIGER-Alaska-93K and has 93366 nodes. We also extract the smaller datasets TIGER-Alaska-2K and TIGER-Alaska-10K, which are subsets of the original dataset with 2000 and 10000 nodes, respectively. We compare the wallclock time of BASEPLANE and PPGN which is as expressive as 2-WL.
Table 7: Runtime experiments on planar graphs TIGER-Alaska-2K, TIGER-Alaska-10K, and TIGER-Alaska-93K. We report the pre-processing and training time (per epoch) for all models.
Dataset	#Nodes	BASEPLANE	PPGN	ESAN
Pre.	Train	Pre.	Train	Pre.	Train
TIGER-Alaska-2K	2000	9.8 sec	0.1 sec	3.4 sec	5.9 sec	4.6 sec	87.73 sec
TIGER-Alaska-10K	10000	50 sec	0.33 sec	OOM	OOM	OOM	OOM
TIGER-Alaska-93K	93366	3.7 h's	2.2 sec	OOM	OOM	OOM	OOM
Results. The runtimes reported in Table 7 confirm our expectation: BASEPLANE is the only model which can scale up to all datasets. Indeed, both ESAN and PPGN fail on TIGER-Alaska-10K and TIGER-Alaska-93K. Somewhat surprisingly, ESAN training time is slower than PPGN training time
Table 8: BASEPLANE ablation study on ZINC. Each component is essential to achieve the reported performance. The use of bi- and triconnected components yields substantial improvements. Aggregating over direct neighbours in the update equation remains important.
Models	ZINC MAE
BASEPLANE (original)	0.076± 0.003
BASEPLANE (no readout)	0.079± 0.002
BASEPLANE (no CUTENC)	0.079± 0.003
BASEPLANE (no neighbours)	0.099± 0.004
BASEPLANE (only BIENC)	0.097± 0.002
BASEPLANE (only TRIENC)	0.092± 0.003
on TIGER-Alaska-2K which is likely due to the fact that the maximal degree of TIGER-Alaska-2K is 620, which the ESAN algorithm depends on. BASEPLANE is very fast for training with the main bottleneck being one-off pre-processing. This highlights that BASEPLANE is a highly efficient option for inference on large-scale graphs compared to subgraph or higher-order alternatives.
C An ablation study on ZINC
We additionally conduct extensive ablation studies on each component in our model update equation using the ZINC 12k dataset. We report the final results in Table 8. The setup and the main findings can be summarized as follows:
- BASEPLANE (no readout): We removed the global readout term from the update formula and MAE worsened by 0.003.
- BASEPLANE (no CutEnc): We removed the cut node term from the update formula and MAE worsened by 0.003.
- BASEPLANE (no neighbors): We additionally removed the neighbor aggregation from the update formula and MAE worsened by 0.023.
- BASEPLANE (only BiEnc): We only used the triconnected components for the update formula and MAE worsened by 0.021.
- BASEPLANE (only TriEnc): We only used the triconnected components for the update formula and MAE worsened by 0.016.
Hence, BasePlanE significantly benefits from each of its components: the combination of standard message passing with component decompositions is essential to obtaining our reported results.
D Further experimental details
D.1 Link to code
The code for our experiments, as well as instructions to reproduce our results and set up dependencies, can be found at this GitHub repository: https://github.com/ZZYSonny/PlanE
D.2 Computational resources
We run all experiments on 4 cores from Intel Xeon Platinum 8268 CPU @ 2.90GHz with 32GB RAM. In Table 9, we report the approximate time to train a BASEPLANE model on each dataset and with each tuned hidden dimension value.
D.3 E-BASEPLANE architecture
E-BASEPLANE builds on BASEPLANE, and additionally processes edge features within the input graph. To this end, it supersedes the original BASEPLANE update equations for SPQR components
Table 9: Approximate Training Time for BASEPLANE.
Dataset Name	Hidden Dimension	Training Time (hours)
QM9CC	32	2.5
MolHIV	64	6
QM9	128	25
ZINC(12k)	64	5
128	8
ZINC(Full)	128	45
EXP	32	0.2
and nodes with the following analogs:
$$
\widehat{\boldsymbol{h}}_{C}^{(\ell)} = \mathrm{MLP}\Big(\sum_{i = 1}^{\left|\omega \right|}\mathrm{MLP}(\boldsymbol{h}_{\omega [i]}^{(\ell -1)}\| \hat{\boldsymbol{h}}_{\omega [i],\omega [(i + 1)\% |w|]}^ {(\ell -1)}\| \boldsymbol{p}_{\kappa [i]}\| \boldsymbol{p}_{i}))\Big),\text{and}
$$
$$
\boldsymbol {h} _ {u} ^ {(\ell)} = f ^ {(\ell)} \left(g _ {1} ^ {(\ell)} \left(\boldsymbol {h} _ {u} ^ {(\ell - 1)} + \sum_ {v \in N _ {u}} g _ {5} ^ {(\ell)} \left(\boldsymbol {h} _ {v} ^ {(\ell - 1)} \| \boldsymbol {h} _ {v, u} ^ {(\ell - 1)}\right)\right) \| g _ {2} ^ {(\ell)} \left(\sum_ {v \in V} \boldsymbol {h} _ {v} ^ {(\ell - 1)}\right) \right.
$$
$$
g _ {3} ^ {(\ell)} \left(\boldsymbol {h} _ {u} ^ {(\ell - 1)} + \sum_ {B \in \pi_ {u} ^ {G}} \widetilde {\boldsymbol {h}} _ {B} ^ {(\ell)}\right) \parallel g _ {4} ^ {(\ell)} \left(\boldsymbol {h} _ {u} ^ {(\ell - 1)} + \sum_ {C \in \sigma_ {u} ^ {G}} \widehat {\boldsymbol {h}} _ {C} ^ {(\ell)}\right) \parallel \overline {{\boldsymbol {h}}} _ {\delta_ {u}} ^ {(\ell)} \Big),
$$
where  $\hat{\pmb{h}}_{i,j} = \pmb{h}_{i,j}$  if  $(i,j)\in E$  and is the ones vector  $(\mathbf{1}^d)$  otherwise
D.4 Training setups
Figure 5: Normalized clustering coefficient distribution of  $\mathrm{QM9}_{\mathrm{CC}}$
Graph classification on EXP We follow the same protocol as the original work: we use 10-fold cross validation on the dataset, train BASEPLANE on each fold for 50 epochs using the Adam [39] optimizer with a learning rate of  $10^{-3}$ , and binary cross entropy loss.
Graph classification on P3R. We follow a very similar protocol to the one in EXP: we use 10-fold cross validation, where we train BASEPLANE for 100 epochs using the Adam [39] optimizer with a learning rate of  $10^{-3}$ , and cross entropy loss.
Clustering coefficient of QM9 graphs. To train all baselines, we use the Adam optimizer with a learning rate from  $\{10^{-3}; 10^{-4}\}$ , and train all models for 100 epochs using a batch size of 256 and L2 loss. We report the overall label distribution of normalized clustering coefficients on QM9 graphs in Figure 5.
Graph classification on MolHIV. We instantiate E-BASEPLANE with an embedding dimension of 64 and a positional encoding dimensionality of 16. We further tune the number of layers within the
set  $\{2,3\}$  and use a dropout probability from the set  $\{0,0.25,0.5,0.66\}$ . Furthermore, we train our models with the Adam optimizer [39], with a constant learning rate of  $10^{-3}$ . Finally, we perform training with a batch size of 256 and train for 300 epochs.
Graph regression on QM9. As standard, we train E-BASEPLANE using mean squared error (MSE) and report mean absolute error (MAE) on the test set. For training E-BASEPLANE, we tune the learning rate from the set  $\{10^{-3}, 5 \times 10^{-4}\}$  with the Adam optimizer, and adopt a learning rate decay of 0.7 every 25 epochs. Furthermore, we use a batch size of 256, 128-dimensional node embeddings, and 32-dimensional positional encoding.
Graph regression on ZINC. In all experiments, we use a node embedding dimensionality from the set  $\{64, 128\}$ , use 3 message passing layers, and 16-dimensional positional encoding vectors. We train both BASEPLANE and E-BASEPLANE with the Adam optimizer [39] using a learning rate from the set  $\{10^{-3}, 5 \times 10^{-4}, 10^{-4}\}$ , and follow a decay strategy where the learning rate decays by a factor of 2 for every 25 epochs where validation loss does not improve. We train using a batch size of 256 in all experiments, and run training using L1 loss for 500 epochs on the ZINC subset, and for 200 epochs on the full ZINC dataset.