R EDUCING L EARNING ON C ELL C OMPLEXES TO G RAPHS
A BSTRACT
Message passing graph neural networks (GNNs) are known to have a limited expressiveness in distinguishing graphs.
A recent approach tackles this problem by transforming graphs to regular cell complexes.
This makes it possible to model higher-order structures and yields algorithms that are more powerful than the Weisfeiler Leman test (WL) or GNNs.
However, this approach cannot easily be combined with previous graph algorithms and implementations due to their fundamental differences.
We develop Cell Encoding , a novel approach of transforming regular cell complexes to graphs.
We show that cell encoding combined with
WL or a suitably expressive GNN is at least as expressive as Cellular Weisfeiler
Leman (CWL) in distinguishing cell complexes.
This means that with a simple preprocessing one can use any GNN for learning tasks on cell complexes.
Additionally, we show that this approach can make GNNs more expressive and give better results on graph classification datasets.
I NTRODUCTION
Xu et al.
and Morris et al.
(2019) showed that message passing graph neural networks (GNNs) have limited expressiveness in distinguishing graphs.
They showed that any GNN can only distinguish two graphs if the Weisfeiler Leman test (WL) can distinguish them.
This motivated the research into provably more expressive GNNs.
One approach for this is to extend the vertex features.
This can be done by adding random features (Dasoulas et al., 2020;
Abboud et al., 2021; Sato et al., 2021), subgraph counts (Bouritsas et al., 2020), or rooted subgraph homomorphism counts (Barcel ´ o et al., 2021).
Other methods change the way message passing is performed, such as higher order
GNNs (Morris et al., 2019), equivariant subgraph aggregation networks (Bevilacqua et al., 2021), structural message-passing neural networks (Vignac et al., 2020), and CW Networks (Bodnar et al., 2021a).
The latter method is especially interesting as it changes the message passing scheme by transforming a graph to a topological construct called a regular cell complex.
They define Cellular
Weisfeiler Leman (CWL) a variant of WL that works on regular cell complexes, and an equivalent of graph neural networks on regular cell complexes called CW Networks.
Depending on the transformation from graph to cell complex, these methods can have higher expressiveness and better connectivity between nodes than WL or GNNs.
Other algorithms that operate on regular cell complexes are Simplicial Networks (Bodnar et al., 2021c), Simplical Neural Networks (Ebli et al., 2020), Dist2Cycle (Keros et al., 2022), and Cell Complex Neural Networks (Hajij et al., 2020).
In this work, we present Cell Encoding an algorithm that can transform any regular cell complex to a graph.
We prove that this transformation combined with WL or a suitably expressive GNN is at least as expressive as CWL and CW Networks in distinguishing regular cell complexes.
This shows that it is possible to perform message passing on graphs instead of the corresponding cell complexes.
We also use this approach to encode structural features as additional nodes, which corresponds to lifting the graph to a cell complex and then transforming it back.
Message passing in this modified graph corresponds to message passing on the cell complex and can improve the expressiveness of GNNs 1 .
We show empirically that this approach improves the results of GNNs on graph classification tasks.
C ELLULAR W EISFEILER L EMAN
In this section, we introduce the concept of expressiveness and the necessary concepts of Bodnar et al.
(2021a)
to understand message passing on regular cell complexes.
For details we defer to to
Appendix A or Bodnar et al.
(2021a).
We say that algorithm A is at least as expressive as algorithm
B if A can distinguish every pair of graphs or cell complexes that B can distinguish.
A is more expressive than B if A is at least as expressive as B and can distinguish more pairs of graphs or cell complexes than B .
Bodnar et al.
(2021a) generalized the message passing paradigm from graphs to regular cell complexes.
Regular cell complexes generalize the simplical complexes used by Bodnar et al.
(2021b).
A regular cell complex X is a topological space consisting of subspaces { X σ } σ ∈ P X called cells together with an indexing set P X .
This indexing set encodes all topological information about X and can be used to define a boundary relation ≺ between cells.
This boundary relation can then be leveraged to define adjacencies between cells.
Cellular Weisfeiler Leman (CWL) performs message passing on cells.
In each iteration of CWL the algorithm computes a colouring for each cell depending on the colours of neighbouring cells in the previous iteration.
Similar to WL two regular cell complexes are not isomorphic if at some iteration the colour histograms of all cells are different for the two complexes.
To apply the concept of regular cell complexes to graphs, Bodnar et al.
(2021a) define the concept of a cellular lifting map , a function f that transforms a graph to a regular cell complex such that two graphs G 1 , G 2 are isomorphic if and only if f ( G 1 ) , f ( G 2 ) are isomorphic.
They prove that a class of lifting maps called skeleton preserving lifting maps together with CWL are at least as expressive as WL.
Typically, such lifting maps create cells out of vertices, together with cells that encode other structures such as induced cycles or cliques.
Figure 1 shows an example of this, the original graph (left) is turned into a cell complex (right) where the vertices are 0-dimensional cells, edges are 1-dimensional cells, and cycles are 2-dimensional cells (blue).
Bodnar et al.
(2021a) define CW Networks which combine neural networks with cellular message passing, similar to graph neural networks with message passing.
CW Networks can be made equally expressive as CWL.
Thus, by lifting graphs to cell complexes and then using a CW Network one can obtain algorithms that are strictly more expressive than WL.
C ELL E NCODING
We propose Cell Encoding , a novel algorithm that transforms a regular cell complex X to a graph G X .
A similar construction for a type of regular cell complexes called simplical complexes is already known to the topology community (Grigor et al., 2014).
We show that cell encoding combined with WL is at least as expressive as CWL in distinguishing regular cell complexes.
With this, we can perform message passing on graphs instead of cell complexes while keeping the expressiveness guarantees from CWL.
However, this approach is not limited to cell complexes obtained with a cellular lifting map.
Indeed, any cell complex can be transformed into a graph while ensuring that WL is as expressive least as CWL.
Definition 1 (Cell Encoding) .
Given a regular cell complex X with a finite indexing set P X , cell encoding transforms P X into a graph G X =
( V X , E X ) with vertex features.
Where
V X = P X , E X
= {{ τ, δ } | τ, δ ∈ P X , τ ≺ δ or δ ≺ τ } ∪ {{ τ, δ } | ∃ σ ∈ P X , τ ≺ σ, δ ≺ σ } , and the features of a vertex σ encode the dimension of cell σ .
Encoding the dimension of a cell in vertex features can be done via one-hot encoding and we use it to distinguish between cells of different dimensions.
Theorem 2.
Cell encoding together with WL is as least as expressive as CWL.
Proof Sketch.
(Full proof in Appendix B).
We prove that every pair of vertices assigned the same colour by WL imply that the underlying cells will be assigned the same colour by CWL.
We show this by induction on the iterations of CWL.
The base case directly follows from the fact that graphs obtained by applying CRE to a regular cell complex have the same number of vertices as the underlying cell complex has cells.
In the induction step, the properties of a stable WL colouring together with the vertex features encoding the dimension of cells means we can distinguish between vertices that correspond to cells of different dimensions.
This allows us to show that if cell encoding together with WL cannot distinguish a pair of regular cell complexes then neither can CWL.
While cell encoding together with WL is as expressive as CWL, this does not mean that it yields exactly the same result.
For example, when CWL passes messages via higher dimensional cells, it adds the colour of the higher dimensional cell to the message.
This is not something covered by our transformation and does not impact the expressiveness.
However, this extra information might still lead to different results.
C ELLULAR R ING E NCODING
In this section, we show that cell encoding can be used to build more expressive GNNs.
Bodnar et al.
(2021a)
present cellular lifting maps such as k -IC that when combined with CWL yield algorithms strictly more expressive than WL.
We focus on the lifting map k -IC that transforms every vertex, edge and induced cycle of length up to k into a cell.
Note that k ≥ 3 is a hyperparamter that needs to be set separately.
Combining k -IC with cell encoding gives us a transformation we call Cellular
Ring Encoding (CRE).
CRE transforms a graph into another graph with vertex features.
An example of CRE can be seen in Figure 2.
Proposition 3.
CRE together with WL is more expressive than WL.
Proof.
k -IC has been shown to be strictly more expressive than WL when combined with CWL (Bodnar et al., 2021a).
By Theorem 2 it follows that combining CRE with WL is strictly more expressive than just WL.
Furthermore, since the graph neural network GIN (Xu et al., 2019) can be made as expressive as
WL it follows that combining CRE with GIN is more expressive than WL.
E XPERIMENTS
In this section, we investigate the performance of Cellular Ring
Encoding on graph classification datasets.
The experimentsare designed to determine whether Cellular Ring Encoding is on par with the model CIN introduced in Bodnar et al.
(2021a)
and current GNNs.
Furthermore, we also investigate whether
Cellular Ring Encoding generally improves the empirical performance of graph classification methods.
Similar to CIN (Bodnar et al., 2021a), we use GIN (Xu et al., 2019) with Jumping Knowledge (Xu et al., 2018) as our model.
CIN performs message passing on cell complexes constructed with the cellular lifting map k -IC.
Analogously, we use Cellular Ring Encoding.
The size k of the largest induced cycle to lift will be tuned to the given datasets.
For details about the models we defer to Appendix C.1 and for details about hyperparameters we defer to Appendix C.4.
TUDataset.
We perform two sets of experiments on datasets from the TUDataset collection (Morris et al., 2020), the first follows Xu et al.
and the second follows Errica et al.
N/A means that the authors did not evaluate their algorithm on the given dataset.
[Table] [Table] [Table] In the first set of experiments we evaluate GIN + CRE on NCI1 , NCI109 , and PROTEINS .
These datasets consist of 1000 - 4000 graphs belonging to one of two classes.
These datasets were chosen as they are the largest commonly used molecular datasets in the collection.
We perform stratified 10-fold cross-validation and report the average and standard deviation of the epoch with the highest validation accuracy.
We use Bayesian optimization to quickly find suitable parameters within less than 20 parameter combinations.
Following Xu et al.
(2019) we report the result of the parameter configuration with the best validation accuracy.
The results can be found in Table 1.
As expected, CRE improves the accuracy of GIN.
Interestingly, GIN + CRE achieves a higher accuracy than CIN on all three datasets.
This evaluation method can overestimate the performance of models.
To get a more realistic understanding of the performance of Cellular Ring Encoding we do an ablation study by adapting an experiment setup introduced by Errica et al.
For this we perform stratified 10-fold cross validation on the NCI1 dataset and ensure that hyperparameters are only selected on the training set.
More details can be found in Appendix C.2.
We investigate three neural networks: GIN, GIN with Cellular Ring Encoding and CIN.
We also investigate the Weisfeiler-Leman Shortest Path (WL SP) and Subtree Kernels (WL ST) using an SVM as the learning algorithm, with and without CRE.
The results can be found in Table 2.
Combining GIN with with CRE improves the accuracy over
GIN.
Interestingly, CRE does not improve WL ST but improves WL SP when restricting it to a single iteration of the WL algorithm.
It is surprising that in this setting CIN performs similarly to
GIN even though we would expect it to outperform GIN and achieve similar results as GIN + CRE.
ogb-molhiv.
We evaluate GIN + CRE on the ogb-molhiv dataset (Hu et al., 2020).
Results are evaluated with the ROC-AUC score, according to Hu et al.
ogb-molhiv provides a train, validation and test split, allowing fair comparisons between different methods.
Similar to Hu et al.
we extend our setup with a virtual node (VN), that is, a node that is connected to all nodes in the graph.
We train GIN + VN + CRE to see how much we can improve upon GIN + VN.
We tune the hypereparameters via Bayesian optimization on the validation set, and train a model with the best parameters 10 times without setting a random seed.
We report the mean and standard deviation of the test ROC-AUC score in the epoch with the highest validation score in Table 3.
Cell encoding substantially improves over just GIN + VN, but does not manage to beat CIN.
C ONCLUSION
In this paper, we have shown that transforming any regular cell complex into a graph and applying
WL is at least as expressive as CWL on the cell complex.
We can adapt any GNN to operate on regular cell complexes with a single line of code.
Similarly, a simple preprocessing can make any GNN with WL expressiveness (such as GIN) at least as expressive as cellular message passing.
We have demonstrated empirically that this approach can improve the performance on graph classification datasets.
A downside of the original cellular message passing and also our cell encoding is the potentially large number of cells in a lifted graph.
The reason is that cellular lifting maps create at least a cell for every vertex and edge in the graph.
This can lead to increased runtimes on larger graphs.
For a computational analysis of computing the cell complex we refer to Bodnar et al.
(2021a).
We conclude by proposing a generalization of our ideas as future work.
The idea behind cell encoding is creating vertices corresponding to structures in graphs or cells together with a feature that gives more information about this structure.
We have already shown that this approach can be used to perform cellular message passing on graphs instead of regular cell complexes.
A similar approach could be used to encode subgraph (Bouritsas et al., 2020) or homomorphism patterns (Barcel´o et al., 2021) as additional vertices.
Finally, it is seems possible to emulate k -dimensional WL (Immerman & Lander, 1990) using 1-WL on a transformed graph.
A M ORE D ETAILS A BOUT C ELLULAR W EISFEILER L EMAN
In this section we give an explanation of cellular lifting maps and the Cellular Weisfeiler Leman algorithm.
Note that all definitions are take from the cited sources with some additional details.
We use {{·}} to denote a multiset.
Definition 4 (Hansen & Ghrist, 2019, Bodnar et al., 2021a) .
A regular cell complex is a topological space X together with an indexing set P X that defines a partition { X σ } σ ∈ P X of subspaces X σ of
X called cells , such that
For each x ∈ X , every sufficiently small neighborhood of x intersects finitely many cells.
For all σ, τ ∈ P X we have that X τ ∩ X σ ̸ = ∅ only if X τ ⊆
X σ , where X σ is the closure of a cell.
Every cell X σ is homeomorphic to R n σ for some n σ ∈ N . 4.
( Regularity )
For every σ ∈ P
X
there is a homeomorphism ϕ of a closed ball in R n σ to X σ such that the restriction of ϕ to the interior of the ball is a homeomorphism onto X σ .
For the purpose of this paper we will assume that P X is finite.
For every cell X σ we call n σ the dimension of the cell.
This definition implies that the indexing set P X has poset structure defined by τ ≤ σ iff
X τ ⊆
X σ that encodes all topological information about X .
From this Bodnar et al.
(2021a) define a boundary relation:
Definition 5 (Bodnar et al., 2021a) .
The boundary relation σ ≺ τ holds iff σ < τ
and there is no cell δ such that σ < δ < τ .
This boundary relation can then be leveraged to define adjacencies in regular cell complexes.
Note, that we simplify the definitions of Bodnar et al.
(2021a)
by applying their Theorem 7 to remove adjacencies that do not improve the expressiveness of CWL.
Definition 6 (Bodnar et al., 2021a) .
For a regular cell complex X and a cell σ ∈ P X , we define:
The boundary adjacent cells B ( σ ) = { τ | τ ≺ σ } .
These are the lower-dimensional cells on the boundary of σ .
For instance, the boundary cells of an edge are its vertices.
The co-boundary adjacent cell C ( σ ) = { τ | σ ≺ τ } .
These are the higher-dimensional cells with σ on their boundary.
For instance, the co-boundary cells of a vertex are the edges it is part of. 3.
The upper adjacent cells N ↑ ( σ ) = { τ | ∃ δ such that σ ≺ δ and τ ≺ δ } .
These are the cells of the same dimension as σ that are on the boundary of the same higher-dimensional cell as σ .
The typical graph adjacencies between vertices are the canonical example here.
Definition 7 (Bodnar et al., 2021a) .
For any cells σ, τ ∈ P X we define C ( σ, τ ) =
C ( σ ) ∩ C
( τ ) .
From this one can define how the adjacencies influence the colouring of a cell.
Definition 8 (Bodnar et al., 2021a) .
A cellular colouring is a map c that maps every cell of a regular cell complex to a colour from a fixed colour palette.
Definition 9 (Bodnar et al., 2021a) .
Let c be a celulular colouring of X with c σ denoting the colour assigned to cell σ ∈ P X .
We define the following multi-sets of colours:
The colours of the boundary cells of σ : c B ( σ ) = {{ c τ | τ ∈ B ( σ ) }} .
The upper adjacent colours of σ : c ↑ ( σ ) = {{ ( c τ , c δ ) | τ ∈ N ↑ ( σ ) and δ ∈ C ( σ, τ ) }} .
Finally, we can define CWL a colour refinement scheme for regular cell comple analogously to
Bodnar et al.
(2021a).
Given a regular cell complex X , all cells are initialised with the same colour.
Given the colour c tσ of cell σ at iteration t , we compute the colour of cell σ at the next iteration c t +1 σ by injectively
mapping the multi-sets of colours belonging to the adjacent cells of σ using a perfect HASH function: c t
[Equation]
[Equation]
The algorithm stops when a stable colouring is reached.
Two cell complexes are considered non-isomorphic if their colour histograms are different.
Otherwise, the test is inconclusive.
To apply this algorithm to graphs one needs to lift a graph to a regular cell complex.
To ensure that the expressiveness of CWL is comparable with that of WL such a lifting operation needs to respect isomorphisms.
Definition 10 (Bodnar et al., 2021a) .
A cellular lifting map is a function f : G → X from the space of graphs G to the space of regular cell complexes X with the property that two graphs G 1 , G 2 are isomorphic iff the cell complexes f ( G 1 ) , f ( G 2 ) are isomorphic.
Finally, Bodnar et al.
show that a class of cellular lifting maps called skeleton preserving lifting maps together with CWL are at least as expressive as WL.
For this one first lifts the graph to a regular cell complex with a skeleton preserving lifting map and then performs CWL on it.
Then graphs can be distinguished via their cellular colouring.
There also exist skeleton preserving lifting maps that when combined with CWL are strictly more expressive than WL.
B P ROOF OF T HEOREM
We give a proof of Theorem 2.
We use {{·}} to denote a multiset and N G ( v ) to denote the neighbors of vertex v in graph G .
Proof.
Let P X , P Y be the indexing sets of two regular cell complexes.
Let G X and H Y be the graphs obtained by applying cell encoding to P X and P Y .
We use π to denote the stable colouring obtained by WL and c t to denote the colour obtained by CWL after iteration t .
Thus π σ denotes the colours assigned to vertex σ by WL and c tσ denotes the colour assigned to cell σ by CWL.
We assume that WL with cell encoding cannot distinguish G X and H Y .
From this we show that for any iteration t ≥ 0 of CWL it holds that:
For all τ ∈ V ( G X ) , σ ∈ V ( H Y ) with π τ = π σ it holds that c tτ =
c tσ .
Note that this is equivalent to showing that if WL with cell encoding cannot distinguish G X and
H Y then CWL cannot distinguish P X and P Y .
This is because when WL with cell encoding cannot distinguish G X and H Y , then we know that for every vertex in G X there is a vertex in H Y that are assigned the same color by WL.
The statement then implies that there is a bijective mapping from cells of P X to P Y such that they are assigned the same colour by CWL which means that the histogram of colours is the same for both graphs.
We show that this statement holds by induction on the iteration t of CWL.
During the proof we will make use of the fact that π is a stable colouring.
This means that if two vertices p ∈ V ( G X ) and q ∈ V ( H Y ) are assigned the same colour π p = π q , then if WL is run for another iteration they will still have the same colour.
This implies that the multiset of colours of neighbors of p is equivalent to the multiset of colours of neighbors of q .
Thus, there exists a bijective function α : N G X ( p ) → N H Y ( q ) such that for any x ∈ N G X ( p ) it holds that π x = π α ( x ) .
Base case: We show that the statements hold for t = 0 .
CWL initializes all of its cells to the same colour.
Thus, all we need to show is that P X and P Y have the same number of cells.
The number of vertices in G X and H Y is equal to the number of cells in P X and P Y , respectively.
Since WL cannot distinguish G X and H Y we know that they must have the same number of vertices.
Induction hypothesis: We assume that the statements holds for t = n .
Induction step: We show that the statements hold for t = n + 1 .
Let τ ∈ V ( G X ) , σ ∈ V ( H Y ) be arbitrary vertices with π τ = π σ .
We need to show that c n +1 τ = c n +1 σ .
By the definition of CWL we know that c n +1 τ = HASH (cid:16) c nτ , c n B ( τ ) , c n ↑ ( τ ) (cid:17) and c n +1 σ = HASH (cid:16)
[Equation]
[Equation] [Equation]
[Equation]
We will show that the inputs into the two hash functions are equal for c n +1 τ and c n +1 σ .
First, we show that c nτ = c nσ .
This immediately follows from the assumption
π τ = π σ and the induction hypothesis.
Next, we want to show that c n B ( τ ) =
c n B ( σ ) .
The assumption π τ = π σ implies that there exists a bijective function α : N G X ( τ ) → N H Y ( σ ) such that for any x ∈ N G X ( τ ) it holds that π x = π α ( x ) .
Since the initial colours encode the dimensions of the cell, this function mus also respect the dimension of the cell meaning it only maps vertices to vertices whose cells have the same dimensions.
This implies that for every cell µ with µ ≺ τ we know that there exists a cell ν
= α ( µ ) with ν ≺ σ such that π µ = π ν .
With the induction hypothesis it follows that c nµ =
c nν .
Observe, that B ( τ ) contains only cells µ with µ ≺ τ .
Analogously, B ( σ ) contains cells ν with ν ≺ σ .
Thus c n B ( τ ) =
c n B ( σ ) .
Finally, we need to show that c n ↑ ( τ )
[Equation] [Equation]
By definition we know that
[Equation]
n
[Equation] [Equation]
We can rewrite this as c n ↑ ( τ ) =
(cid:8)(cid:8) ( c nµ , c nδ ) | τ ≺ δ and µ ≺ δ (cid:9)(cid:9) .
We will make use of the bijective function α defined in the paragraph above.
We know that for any cell δ with τ ≺ δ there exists a vertex δ adjacent to τ such that π δ = π α ( δ ) .
This implies two things: first by using the induction hypothesis we know that c nδ
= c
nα ( δ ) .
Secondly, there exist a bijective function β
δ : N G X ( δ ) → N H Y ( α ( δ )) that has the same properties as α .
That is, for any x ∈ N G X ( δ ) it holds that π x =
π β δ ( x ) .
We can now put all of this together.
The existence of α means that for any cell δ with τ ≺ δ there is a cell
α ( δ ) ∈
P Y such that c
nδ =
c
nα ( δ ) .
Next, with the existence of β δ it follows that for each cell µ with µ ≺ δ there exists a cell β δ ( µ )
∈
P Y such that c nµ =
c nβ δ ( µ ) .
With the fact that α and β δ ( µ ) are bijective, it follows that c n ↑ ( τ ) =
c n ↑ ( σ ) .
This proves the induction step and concludes the proof of
Theorem 2.
C E XPERIMENTAL D ETAILS
The neural network models are implemented in Python with PyTorch (Paszke et al., 2019) and PyTorch Geometric (Fey & Lenssen, 2019).
For kernel methods we used the GraKeL library (Siglidis et al., 2020).
The code to compute Cellular Ring Encoding is based on Bodnar et al.
(2021a)
and uses graph-tool 3 to compute induced cycles in the graphs.
We used the sweep feature from Weights & Biases 4 to perform Bayesian optimization to tune the hyperparameters in all but the ablation experiments.
All models except CIN were trained on systems with an NVIDIA GeForce RTX 3080 GPU, 64 GB of RAM and an Intel i7-10700KF CPU or an Intel i9-11900KF CPU.
CIN was trained on a system with an NVIDIA GeForce GTX TITAN X GPU, 94 GB of RAM and an Intel Xeon X5680 CPU.
C. M ODEL I MPLEMENTATIONS
Similar to Bodnar et al.
(2021a)
we implement a graph readout operation that makes better use of
CRE.
Instead of summing up the representation of each node, we keep track whether a node came from a node, an edge or a cycle in the original graph.
Then we separately sum the representations for these three types of nodes and apply a multilayer perceptron with a non linear activation function to each of the three resulting representations.
Finally, we sum up all three representations to obtain a single vector that encodes an entire graph.
For experiments on the TUDatasets we use the implementation of GIN with Jumping Knowledge from Bodnar et al.
(2021a) that is mostly equivalent to the benchmark implentation from PyTorch
Geometric.
Additionally, we add dimensional pooling to this model.
For experiments on ogb-molhiv we take the experimental setup from Hu et al.
(2020) including their implementation of GIN with Jumping Knowledge and a virtual node.
We extend this setup by adding Cellular Ring Encoding and extending the models with dimensional pooling.
C. D ETAILS ON THE A BLATION S ETUP
We perform stratified 10-fold cross validation on the NCI1 dataset to obtain tuples of training and test sets.
For the GNN methods, we split off 10% of the trainig set as a validation set, while ensuring an equal class distribution.
For each fold we tune the parameters on the training and validation set.
In total we test 20 randomly selected parameters per fold.
Then we train a model with the selected
[Table]
parameters on the training set with early stopping on the validation set and evaluate it on the test set, we do this three times for each fold to smooth out non-deterministic behaviour in the training process.
For the kernel methods we do not split off a validation set.
Instead we train on the entire training set and select the parameter configuration with the highest training accuracy, in total we try up torandom parameter configurations per fold depending on the size of the parameter grid.
We evaluate on the test set and report the average accuracy and standard deviation over all folds.
C. D ETAILS ON C ELLULAR R ING E NCODING
Cellular Ring Encoding One-Hot encodes the dimension of cells.
This means for every vertex that corresponds to a 0-dimensional cell the vector (1 , 0 , 0) will be added to the vertex features.
For 1dimensional and 2-dimensional the vectors (0 , 1 , 0) and (0 , 0 , 1) will be added to the vertex features, respectively.
We call this explicit pattern encoding .
While this is necessary for the expressiveness guarantees of Theorem 2 it can also be turned of.
Additionally, we implement the option to collect vertex features into newly created vertices by Cellular Ring Encoding.
Intuitively, if CRE adds a new vertex that corresponds to an edge { p, q } in the graph than the newly created vertex will have the average of p and q .
This feature is called aggr.
vertex features in the following section.
Finally, we developed two different ways for CRE to interact with edge features.
Edge features in vertices appends the average edge feature of a vertex to its features.
This allows GNNs that normally cannot use edge features to use them.
Aggregate edge features sets newly created edges by CRE to have the average edge features of the edge that created that edge.
For example, if { p, q } is an edge then CRE will create a vertex that corresponds to this edge.
This new vertex will be adjacent to both p and q .
Then the aggregate edge features will ensure that these newly created edges have the same features as { p, q } .
C. H YPERPARAMETER
We present all used hyperparameter configurations in Tables 4, 5 and 6.
The top part of the table contains model specific hyperparamters and the bottom part contains CRE specific hyperparameters.
More information about the CRE specific hyperparamters can be found in Appendix C.3.
As the used
TUDatasets do not provide edge features the parameters “aggr.
edge features” and “edge features in vertices” are irrelevant to those experiments.
Additionally, for GIN + CRE on TUDatasets we couple the use of dimensional pooling to the “Explicit pattern encoding” parameter meaning dimensional pooling will be used if “Explicit pattern encoding” is set to true.
For GIN + CRE on ogb-molhiv dimensional pooling is a separately tuneable parameter.
[Table] [Table]