Extended Abstract Track
On the Expressive Power of Geometric Graph Neural Networks
Editors: Sophia Sanborn, Christian Shewmake, Simone Azeglio, Arianna Di Bernardo, Nina Miolane
Abstract
We propose a geometric version of the Weisfeiler-Leman graph isomorphism test (GWL) for discriminating geometric graphs while respecting the underlying physical symmetries: permutations, rotation, reflection, and translation.
We use GWL to characterise the expressive power of Graph Neural Networks (GNNs) that are invariant or equivariant to physical symmetries in terms of the classes of geometric graphs they can distinguish.
This allows us to formalise the advantages of equivariant GNNs over invariant GNNs: equivariant layers have greater expressive power as they enable propagating geometric information beyond local neighbourhoods, while invariant layers only reason locally via scalars and cannot discriminate geometric graphs with different non-local properties.
1. Introduction
The graph isomorphism problem and the Weisfeiler-Leman (WL) (Weisfeiler and Leman, 1968) test for distinguishing non-isomorphic graphs have become a powerful tool for analysing the expressive power of Graph Neural Networks (GNNs)
(Xu et al., 2019; Morris et al., 2019).
The WL framework has been a major driver of progress for more expressive GNNs (Maron et al., 2019; Morris et al., 2020; Bodnar et al., 2021a).
However, WL does not directly apply to the increasingly relevant special case of geometric graphs – graphs embedded in
Euclidean space – which come equipped with a stronger notion of isomorphism that also takes spatial symmetries into account.
The lack of theoretical tools is becoming more apparent as geometric graphs are increasingly used to model systems in biochemistry (Jamasb et al., 2022), material science (Chanussot et al., 2021), and multiagent robotics (Li et al., 2020).
Graph Neural Networks (GNNs) with Euclidean symmetries ‘baked in’ have emerged as the architecture of choice for these domains (Geiger and Smidt, 2022).
Geometric GNNs follow the message passing paradigm (Gilmer et al., 2017) where node features are updated in a permutation equivariant manner by aggregating features from local neighbourhoods.
In addition to the permutation group, the geometric attributes of the nodes ( e.g. coordinates, velocity) transform along with Euclidean transformations of the system, i.e. they are equivariant to a Lie group such as the group of rotations ( SO ( d )) or rotations and reflections ( O ( d )).
We use G as a generic symbol for such a Lie group.
Based on this, we consider two classes of GNNs for geometric graphs: (1) G -equivariant models , where the intermediate features and propagated messages are G -equivariant geometric quantities such as vectors or tensors (Thomas et al., 2018;
Satorras et al., 2021); and (2)
G -invariant models , which only propagate G -invariant scalar features such as distances and angles (Sch¨utt et al., 2018; Gasteiger et al., 2020).
Despite promising empirical results for both classes of architectures, key theoretical questions remain unanswered: (1) How to characterise the expressive power of geometric GNNs?
And (2) what is the tradeoff between
G -equivariant and G -invariant GNNs?
Contributions.
In this work, we study the expressive power of geometric GNNs from the perspective of discriminating non-isomorphic geometric graphs.
We propose a geometric version of the Weisfeiler-Leman graph isomorphism test, termed GWL.
(Figure 1).
We use GWL to formally characterise classes of graphs that can and cannot be distinguished by G -invariant and G -equivariant GNNs.
We show how invariant models have limited expressive power as they only reason locally via scalar quantities, while equivariant models distinguish a larger class of graphs by propagating geometric vector quantities beyond local neighbourhoods.
For Background and Preliminaries , please see Appendix A.
2. The Geometric Weisfeiler-Leman Test
Assumptions.
Analogously to the WL test, the geometric and scalar features the nodes are equipped with come from countable subsets C ⊂ R d and C ′ ⊂ R , respectively.
As a
Extended Abstract Track
result, when we require functions to be injective, we require them to be injective over the countable set of G -orbits that are obtained by acting with G on the dataset.
Intuition.
For an intuition of how to generalise the WL test to geometric graphs, we note that WL uses a local, node-centric, procedure to update the colour of each node i using the colours of its the 1-hop neighbourhood N i .
In the geometric setting, N i is an attributed point cloud around the central node i .
As a result, each neighbourhood carries two types of information: (1) neighbourhood type (invariant to G ) and (2) neighbourhood geometric orientation (equivariant to G ).
From an axiomatic point of view, our generalisation of the
WL neighbourhood aggregation procedure must meet two properties:
Property 1: Orbit injectivity of colours.
If two neighbourhoods are the same up to an action of G ( e.g. rotation), then the colours of the corresponding central nodes should be the same.
Thus, the colouring must be G -orbit injective – which also makes it G -invariant – over the countable set of all orbits of neighbourhoods in our dataset.
Property 2: Preservation of local geometry.
A key property of WL is that the aggregation is injective.
A G -invariant colouring procedure that purely satisfies Propertyis not sufficient because, by definition, it loses spatial properties of each neighbourhood such as the relative pose or orientation (Hinton et al., 2011).
Thus, we must additionally update auxiliary geometric information variables in a way that is G -equivariant and injective.
Geometric Weisfeiler-Leman (GWL).
These intuitions motivate the following definition of the GWL test.
At initialisation, we assign to each node i ∈ V a scalar node colour c i ∈ C ′ and an auxiliary object g i containing the geometric information associated to it:
[Equation]
where Hash denotes an injective map over the scalar attributes s i of node i .
To define the inductive step, assume we have the colours of the nodes and the associated geometric objects at iteration t − 1.
Then, we can aggregate the geometric information around node i into a new object as follows:
[Equation]
Importantly, the group G can act on the geometric objects above inductively by acting on the geometric information inside it.
This amounts to rotating (or reflecting) the entire t -hop neighbourhood contained inside:
[Equation]
Clearly, the aggregation building g i for any time-step t is injective and G -equivariant.
Finally, we can compute the node colours at iteration t for all i ∈ V by aggregating the geometric information in the neighbourhood around node i : [Equation]
[Equation]
) by using a G -orbit injective and G -invariant function that we denote by I-Hash .
That is for any geometric objects g , g ′ ,
I-Hash
( g ) =
I-Hash ( g ′ )
if and only if there exists g ∈ G such that g = g · g ′ .
Extended Abstract Track
Overview.
With each iteration, g ( t ) i aggregates geometric information in progressively larger t -hop subgraph neighbourhoods N ( t ) i around the node i .
The node colours summarise the structure of these t -hops via the G -invariant aggregation performed by I-Hash .
The procedure terminates when the partitions of the nodes induced by the colours do not change from the previous iteration.
Finally, given two geometric graphs G and H , if there exists some iteration t for which {{ c ( t ) i
[Equation]
H ) }} , then GWL deems the two graphs as being geometrically non-isomorphic.
Otherwise, we say the test cannot distinguish the two graphs.
Invariant GWL.
Since we are interested in understanding the role of G -equivariance, we also consider a more restrictive Invariant GWL (IGWL) that only updates node colours using the G -orbit injective I-Hash function and does not propagate geometric information:
[Equation]
2.1. What Geometric Graphs can GWL and IGWL Distinguish?
In order to formalise the expressive power of GWL and IGWL, let us consider what geometric graphs can and cannot be distinguished by the tests.
As a simple first observation, we note that when all coordinates and vectors are set equal to zero GWL coincides with the standard 1-WL.
In this edge case , GWL has the same expressive power as 1-WL.
Next, let us consider consider the simplified setting of two geometric graphs G 1 =
( A 1 , S 1 , −→ V 1 , −→ X 1 ) and G
=
( A 2 , S 2 , −→ V 2 , −→ X 2 ) such that the underlying attributed graphs ( A 1 , S 1 ) and ( A 2 , S 2 ) are isomorphic.
This case frequently occurs in (bio)chemical modelling, where molecules occur in different conformations, but with the same graph topology given by the covalent bonding structure.
Recall that each iteration of GWL aggregates geometric information g ( k ) i from progressively larger neighbourhoods N ( k ) i around the node i , and distinguishes (sub-)graphs via comparing G -orbit injective colouring of g ( k ) i .
We say Gand G 2 are k -hop distinct if for all graph isomorphisms b , there is some node i ∈ V 1 , b (
i ) ∈ Vsuch that the corresponding k -hop subgraphs N ( k ) i and N ( k ) b ( i ) are distinct.
Otherwise, we say G 1 and G 2 are k -hop identical if all N ( k ) i and N ( k ) b ( i ) are identical up to group actions.
We can now formalise what geometric graphs can and cannot be distinguished by GWL.
Proposition 1 GWL can distinguish any k -hop distinct geometric graphs G 1 and G 2 where the underlying attributed graphs are isomorphic, and k iterations are sufficient.
Proposition 2 Up to k iterations, GWL cannot distinguish any k -hop identical geometric graphs G 1 and G 2 where the underlying attributed graphs are isomorphic.
Additionally, we can state the following results about the more constrained IGWL.
Proposition 3 IGWL can distinguish any 1 -hop distinct geometric graphs G 1 and G 2 where the underlying attributed graphs are isomorphic, and 1 iteration is sufficient.
Proposition 4 Any number of iterations of IGWL cannot distinguish any 1 -hop identical geometric graphs G 1 and G 2 where the underlying attributed graphs are isomorphic.
Extended Abstract Track
We can now consider the more general case where the underlying attributed graphs for G 1 =
( A 1 , S 1 , −→ V 1 , −→ X 1 ) and G
=
( A 2 , S 2 , −→ V 2 , −→ X 2 ) are non-isomorphic and constructed from point clouds using radial cutoffs, as conventional in biochemistry and material science.
Proposition
Assuming geometric graphs are constructed from point clouds using radial cutoffs, GWL can distinguish any geometric graphs G 1 and G 2 where the underlying attributed graphs are non-isomorphic.
At most k Max iterations are sufficient, where k Max is the maximum graph diameter among G 1 and G 2 .
These results enable us to compare the expressive powers of GWL and IGWL.
Theorem 6 GWL is strictly more powerful than IGWL.
This statement formalises the advantage of G -equivariant intermediate layers for graphs and geometric data, as prescribed in the Geometric Deep Learning blueprint (Bronstein et al., 2021), in addition to echoing similar intuitions in the computer vision community.
As remarked by (Hinton et al., 2011), translation invariant models do not understand the relationship between the various parts of an image (colloquially called the “Picasso problem”).
Similarly, our results explain how IGWL fails to understand how the various 1-hops of a graph are stitched together.
Finally, we identify a setting where this distinction between the two approaches disappears.
Proposition 7 IGWL has the same expressive power as GWL for fully connected geometric graphs.
2.2. Characterising the Expressive Power of Geometric GNNs
We would like to characterise the maximum expressive power of geometric GNNs based on the GWL test.
Firstly, we show that any message passing G -equivariant GNN can be at most as powerful as GWL in distinguishing non-isomorphic geometric (sub-)graphs.
Theorem 8 Any pair of geometric graphs distinguishable by a G -equivariant GNN is also distinguishable by GWL.
With sufficient iterations, the output of G -equivariant GNNs can be equivalent to GWL if certain conditions are met regarding the aggregate, update and readout functions.
Proposition 9 G -equivariant GNNs have the same expressive power as GWL if the following conditions hold: (1) The aggregation Agg is an injective, G -equivariant multiset function.
The scalar part of the update Upd s is a G -orbit injective, G -invariant multiset function.
The vector part of the update Upd v is an injective, G -equivariant multiset function.
The graph-level readout f is an injective multiset function.
Similar statements can be made for G -invariant GNNs and IGWL.
Thus, we can directly transfer our results about GWL and IGWL to the class of GNNs bounded by the respective tests.
This has several interesting practical implications, discussed in Appendix B.
3. Conclusion
This work proposes a geometric version of the Weisfeiler-Leman graph isomorphism test (GWL) for discriminating geometric graphs while respecting the underlying spatial symmetries.
We use GWL to characterise the expressive power of geometric Graph Neural
Networks and demonstrate the advantages of equivariant GNNs over invariant GNNs.
Extended Abstract Track
Acknowledgements
We would like to thank Andrew Blake, David Kovacs, Erik Thiede, Gabor Csanyi, Hannes St¨ark, Ilyes
Batatia, Iulia Duta, Mario Geiger, Petar Veliˇckovi´c, Soledad Villar, Weihua Hu and the anonymous reviewers for helpful comments and discussions.
CKJ was supported by the A*STAR Singapore
National Science Scholarship (PhD).
SVM was supported by the UKRI Centre for Doctoral Training in Application of Artificial Intelligence to the study of Environmental Risks (EP/S022961/1)
Appendix A. Background A.1. Graph Isomorphism and the Weisfeiler-Leman Test
An attributed graph G =
( A , S ) with a node set V of size n consists of an n × n adjacency matrix A and a matrix of scalar features S ∈ R n × f .
Two attributed graphs G , H are isomorphic if there exists an edge-preserving bijection b : V ( G ) → V ( H ) such that s ( G ) i =
s ( H ) b ( i ) , where the subscripts index rows and columns in the corresponding matrices.
The Weisfeiler-Leman test (WL) is an algorithm for testing whether two (attributed) graphs are isomorphic (Read and Corneil, 1977;
Weisfeiler and Leman, 1968).
At iteration zero the algorithm assigns a colour c (0) i ∈
C from a countable space of colours C to each node i .
Nodes are coloured the same if their features are the same, otherwise, they are coloured differently.
In subsequent iterations t , WL iteratively updates the node colouring by producing
a new c ( t ) i
[Equation]
Extended Abstract Track
where Hash is an injective map (i.e. a perfect hash map) that assigns a unique colour to each input and {{·}} denotes a multiset – a set that allows for repeated elements.
The test terminates when the partition of the nodes induced by the colours becomes stable.
Given two graphs G and H , if there exists some iteration t for which {{ c ( t ) i | i ∈ V ( G ) }} ̸
= {{ c ( t ) i |
i ∈ V ( H ) }} , then the graphs are not isomorphic.
Otherwise, the WL test is inconclusive, and we say it cannot distinguish the two graphs.
Xu et al.
; Morris et al.
(2019) showed that GNNs are at most as powerful as
WL at distinguishing non-isomorphic graphs.
Since then, the WL hierarchy has became a powerful tool for analysing the expressive power of GNNs and guided the search for more expressive models (Chen et al., 2019; Maron et al., 2019; Morris et al., 2020; Dwivedi et al., 2020; Bodnar et al., 2021b,a).
A.2. Group Theory
We assume basic familiarity with group theory, see Zee (2016) for an overview.
We denote the action of the group G on a space X by g · x .
If G acts on spaces X and Y , we say a function f : X → Y is G - equivariant if f ( g · x )
= g · f ( x ).
A function f : X → Y is
G - invariant if f ( g · x ) =
f
( x ).
The G -orbit of x ∈ X is O G ( x ) =
{ g · x | g ∈ G } ⊆ X .
When x and x ′ are part of the same orbit, we write x ≃ x ′ .
We say a function f : X → Y is G -orbit injective if we have f ( x 1 ) =
f
( x 2 ) if and only if x 1 ≃ x 2 for any x 1 , x 2 ∈ X .
Necessarily, such a function is G -invariant, since f ( g · x ) =
f
( x ).
We work with the permutation group over n elements S n and the Lie groups G = SO ( d ) or G = O ( d ).
Invariance to the translation group T ( d ) is conventionally handled using relative positions.
Given one of the standard groups above, for an element g we denote by
M g (or another capital letter) its standard matrix representation.
A.3. Geometric Graphs
Systems in biochemistry (Jamasb et al., 2022), material science (Chanussot et al., 2021), physical simulations (Sanchez-Gonzalez et al., 2020), and multiagent robotics (Li et al., 2020) are conventionally modelled as geometric graphs.
For example, molecules are represented as a set of nodes corresponding to atoms, which contain information about the atom type as well as its 3D spatial coordinates and other geometric quantities such as velocity or acceleration.
The geometric attributes transform along with Euclidean transformations of the system.
A geometric graph G =
( A , S , −→ V , −→ X ) with a node set V is an attributed graph that is also decorated with geometric attributes: node coordinates −→ X ∈ R n × d and (optionally) vector features 1 −→ V ∈ R n × d ( e.g. velocity, acceleration).
The geometric attributes transform as follows under the action of the relevant groups: (1) S n acts on the graph via P σ G :
= ( P σ AP ⊤ σ , P σ S , P σ −→ V , P σ −→ X ); (2) Orthogonal transformations Q g ∈ G act on −→ V , −→ X via −→ V Q g , −→ XQ g ; and (3) Translations −→ t ∈ T ( d ) act on the coordinates −→ X via −→ x i + −→ t for all nodes i .
In biochemistry and material science, the conventional procedure for constructing the geometric graph G =
( A , S , −→ V , −→ X ) is via the underlying point cloud ( S , −→ V , −→ X ) using
Extended Abstract Track
a predetermined radial cutoff r .
Thus, the adjacency matrix is defined as a ij = {+
+} if ∥−→ x i − −→ x j ∥ 2 ≤ r , or 0 otherwise, for all a ij ∈ A .
Two geometric graphs G and H are geometrically isomorphic (denoted G ≃ H ) if there exists an attributed graph isomorphism b such that the geometric attributes are equivalent, up to global group actions Q g ∈ G and −→ t ∈ T ( d ):
[Equation]
for all i ∈ V ( G ) .
Geometric graph isomorphism and distinguishing (sub-)graph geometries has important practical implications for representation learning.
For e.g. , in molecular systems, an ideal architecture should map distinct local structural environments around atoms to distinct embeddings in representation space (Bart´ok et al., 2013; Pozdnyakov et al., 2020).
A.4. Geometric Graph Neural Networks
We consider two broad classes of geometric GNN architectures.
G -equivariant GNN layers (Thomas et al., 2018; Anderson et al., 2019;
Jing et al., 2020;
Satorras et al., 2021; Brandstetter et al., 2022) update scalar and vector features from iteration t to t + 1 via learnable aggregate and update functions, Agg and Upd , respectively:
[Equation]
For e.g. , PaiNN (Sch¨utt et al., 2021) interaction layers aggregate scalar and vector features via learnt radial filters:
[Equation]
Alternatively, G -invariant layers (Sch¨utt et al., 2018; Xie and Grossman, 2018; Gasteiger et al., 2020) do not update vector features and only aggregate scalar quantities from local neighbourhoods:
[Equation]
For e.g. , SchNet (Sch¨utt et al., 2018) uses relative distances to scalarise local geometric information, while DimeNet (Gasteiger et al., 2020) uses both distances and angles, as follows:
[Equation]
Extended
Abstract
Track
For both G -invariant and G -equivariant architectures, the scalar features { s ( T ) i } at the final iteration T are mapped to graph-level features via a permutation-invariant readout f : R n × f → R f ′ .
Invariant GNNs have shown strong performance for protein design (Zhang et al., 2022;
Dauparas et al., 2022) and electrocatalysis (Gasteiger et al., 2021; Shi et al., 2022), while equivariant GNNs are being used within learnt interatomic potentials for molecular dynamics (Sch¨utt et al., 2021; Batzner et al., 2022; Batatia et al., 2022).
Appendix B. Discussion
Practical Implications.
Together, Propositions 1 and 4 highlight critical theoretical limitations of G -invariant GNNs in computing global and non-local geometric properties.
Our results suggest that G -equivariant GNNs should be preferred when working with large geometric graphs such as macromolecules with thousands of nodes, where message passing is restricted to local radial neighbourhoods around each node.
Motivated by these limitations, two straightforward approaches to improving G -invariant
GNNs may be: (1) pre-computing non-local geometric properties as input features, e.g. models such as GemNet (Gasteiger et al., 2021) and GearNet (Zhang et al., 2022) successfully use two-hop dihedral angles.
And (2) working with fully connected geometric graphs, as Proposition 7 suggests that G -equivariant
and G -invariant GNNs can be made equally powerful when performing all-to-all message passing.
This is supported by the empirical success of recent G -invariant ‘Graph Transformers’ (Joshi, 2020; Shi et al., 2022) for small molecules with tens of nodes, where working with full graphs is tractable.
Related Work.
Literature on the completeness of atom-centred interatomic potentials has focused on distinguishing 1-hop local neighbourhoods (point clouds) around atoms by building spanning sets for continuous, G -equivariant multiset functions (Shapeev, 2016;
Drautz, 2019; Dusson et al., 2019; Pozdnyakov et al., 2020).
Recent theoretical work on geometric GNNs and their universality has shown that Tensor Field Networks, GemNet and GVP (Dym and Maron, 2020;
Gasteiger et al., 2021;
Jing et al., 2020; Villar et al., 2021) can be universal approximators of continuous, G -equivariant or G -invariant multiset function over point clouds (not sparse graphs).
In contrast, the GWL framework studies the expressive power of geometric GNNs from the perspective of geometric graph isomorphism.
Overall, our work formalises what classes of geometric graphs can and cannot be distinguished by message passing G -invariant/equivariant GNNs while abstracting away implementation details.
Future Work.
GWL provides an abstraction to study the limits of geometric GNNs, but in practice it is challenging to build maximally powerful GNNs that satisfy the conditions of Proposition 9 as GWL relies on G -orbit injective colouring and G -equivariant propagation of auxiliary geometric information.
Based on the intuitions gained from GWL, future work will explore building provably powerful, practical geometric GNNs for applications in biochemistry, material science, and multiagent robotics, and better characterise the trade-offs related to practical implementation choices.
Extended Abstract Track
Appendix C. Proofs for What GWL and IGWL can Distinguish
The following results are a consequence of the construction of GWL as well as the definitions of k -hop distinct and k -hop identical geometric graphs.
Note that k -hop distinct geometric graphs are also ( k + 1)-hop distinct.
Similarly, k -hop identical geometric graphs are also ( k − 1)-hop identical, but not necessarily ( k + 1)-hop distinct.
Given two distinct neighbourhoods N 1 and N 2 , the G -orbits of the corresponding geometric multisets g 1 and g 2 are mutually exclusive, i.e. O G ( g 1 )
∩ O G ( g 2 ) ≡ ∅ .
By the properties of I-Hash this implies c 1 ̸ = c 2 .
Conversely, if N 1 and N 2 were identical up to group actions, their G -orbits would overlap, i.e. g 1 = g g 2 for some g ∈ G and
O G ( g 1 ) =
O G ( g 2 ) ⇒ c
1 = c 2 .
Proposition 10 GWL can distinguish any k -hop distinct geometric graphs G 1 and G 2 where the underlying attributed graphs are isomorphic, and k iterations are sufficient.
Proof [Proof of Proposition 1] The k -th iteration of GWL identifies the G -orbit of the k -hop subgraph N ( k ) i at each node i via the geometric multiset g ( k ) i .
G 1 and G 2 being k -hop distinct implies that there exists some bijection b and some node i ∈ V 1 , b ( i ) ∈
Vsuch that the corresponding k -hop subgraphs N ( k ) i and N ( k ) b ( i ) are distinct.
Thus, the Gorbits of the corresponding geometric multisets g ( k ) i and g ( k ) b ( i ) are mutually exclusive, i.e.
O G ( g ( k ) i ) ∩ O G ( g ( k ) b ( i ) )
≡ ∅ ⇒ c ( k ) i ̸
= c ( k ) b ( i ) .
Thus, k iterations of GWL are sufficient to distinguish G 1 and G 2 .
Proposition 11 Up to k iterations, GWL cannot distinguish any k -hop identical geometric graphs G 1 and G 2 where the underlying attributed graphs are isomorphic.
Proof [Proof of Proposition 2] The k -th iteration of GWL identifies the G -orbit of the k -hop subgraph N ( k ) i at each node i via the geometric multiset g ( k ) i .
G 1 and G 2 being k -hop identical implies that for all bijections b and all nodes i ∈ V 1 , b ( i )
∈
V 2 , the corresponding k -hop subgraphs N ( k ) i and N ( k ) b ( i ) are identical up to group actions.
Thus, the G -orbits of the corresponding geometric multisets g ( k ) i and g ( k ) b ( i ) overlap, i.e. O G ( g ( k ) i ) =
O G ( g ( k ) b ( i ) )
⇒ c
( k ) i
= c ( k ) b ( i ) .
Thus, up to k iterations of GWL cannot distinguish G 1 and G 2 .
Proposition 12 IGWL can distinguish any 1 -hop distinct geometric graphs G 1 and Gwhere the underlying attributed graphs are isomorphic, and 1 iteration is sufficient.
Proof [Proof of Proposition 3] Each iteration of IGWL identifies the G -orbit of the 1-hop local neighbourhood N ( k =1) i at each node i .
G 1 and G 2 being 1-hop distinct implies that there exists some bijection b and some node i ∈ V 1 , b ( i ) ∈
V 2 such that the corresponding 1-hop local neighbourhoods N (1) i and N (1) b ( i ) are distinct.
Thus, the G -orbits of the corresponding geometric multisets g (1) i and g (1) b ( i ) are mutually exclusive, i.e. O G ( g (1) i ) ∩ O G ( g (1) b ( i ) )
≡ ∅ ⇒ c (1) i ̸
= c (1) b ( i ) .
Thus, 1 iteration of IGWL is sufficient to distinguish G 1 and G 2 .
Proposition 13 Any number of iterations of IGWL cannot distinguish any 1 -hop identical geometric graphs G 1 and G 2 where the underlying attributed graphs are isomorphic.
Proof [Proof of Proposition 4] Each iteration of IGWL identifies the G -orbit of the 1hop local neighbourhood N ( k =1) i at each node i , but cannot identify G -orbits beyond 1hop by the construction of IGWL as no geometric information is propagated.
G 1 and Gbeing 1-hop identical implies that for all bijections b and all nodes i ∈ V 1 , b ( i )
∈ V 2 , the corresponding 1-hop local neighbourhoods N ( k ) i and N ( k ) b ( i ) are identical up to group actions.
Thus, the G -orbits of the corresponding geometric multisets g (1) i and g (1) b
( i ) overlap, i.e.
O G ( g (1) [Equation] ) = O G
[Equation]
g [Equation] ( i ) )
⇒ c
( k ) i
= c ( k ) b ( i ) .
Thus, any number of IGWL iterations cannot distinguish G 1 and G 2 .
Proposition 14 Assuming geometric graphs are constructed from point clouds using radial cutoffs, GWL can distinguish any geometric graphs G 1 and G 2 where the underlying attributed graphs are non-isomorphic.
At most k Max iterations are sufficient, where k Max is the maximum graph diameter among G 1 and G 2 .
Proof [Proof of Proposition 5]
We assume that a geometric graph G =
( A , S , −→ V , −→ X ) is constructed from a point cloud ( S , −→ V , −→ X ) using a predetermined radial cutoff r .
Thus, the adjacency matrix is defined as a ij = {+
+} if ∥−→ x i − −→ x j ∥ 2 ≤ r , or 0 otherwise, for all a ij ∈ A .
Such construction procedures are conventional for geometric graphs in biochemistry and material science.
Given geometric graphs G 1 and G 2 where the underlying attributed graphs are nonisomorphic, identify k Max the maximum of the graph diameters of G 1 and G 2 , and chose any arbitrary nodes i ∈ V 1 , j ∈ V 2 .
We can define the k Max -hop subgraphs N ( k Max ) i and N ( k Max )
j represents the maximum information contained in GWL/IGWL colours or
GNN features at iteration t by an ‘unrolling’ of the message passing procedure.
Unlike GWL, geometric orientation information cannot flow from the leaves to the root in IGWL, restricting its expressive power.
IGWL cannot distinguish Gand G 2 as all 1-hop neighbourhoods are computationally identical.
at i and j , respectively.
Thus, N ( k Max ) i
= V 1 for all i ∈ V 1 , and N ( k Max )
j = V 2 for all j ∈ V 2 .
Due to the assumed construction procedure of geometric graphs, N ( k Max ) i and N ( k Max )
j must be distinct.
Otherwise, if N ( k Max ) i and N ( k Max ) j were identical up to group actions, the sets ( S 1 , −→ V 1 , −→ X 1 ) and ( S 2 , −→ V 2 , −→ X 2 ) would have yielded isomorphic graphs.
The k Max -th iteration of GWL identifies the G -orbit of the k Max -hop subgraph N ( k Max ) i at each node i via the geometric multiset g ( k Max ) i .
As N ( k Max ) i and N ( k Max ) j are distinct for any arbitrary nodes i ∈
V 1 , j ∈ V 2 , the G -orbits of the corresponding geometric multisets g ( k Max )
i and g ( k Max )
j are mutually exclusive, i.e. O G ( g ( k Max ) i )
∩O G ( g ( k Max ) j ) ≡ ∅ ⇒ c
( k Max ) i ̸
= c ( k Max ) j .
Thus, k Max iterations of GWL are sufficient to distinguish G 1 and G 2 .
Theorem 6 GWL is strictly more powerful than IGWL.
Proof [Proof of Theorem 6]
Firstly, we can show that the GWL class contains IGWL if GWL can learn the identity when updating g i for all i ∈ V , i.e. g ( t ) i = g (
[Equation] − 1)
i = g (0) i ≡ ( s i , −→ v i ).
Thus, GWL is at least as powerful as IGWL, which does not update g i .
Secondly, to show that GWL is strictly more powerful than IGWL, it suffices to show that there exist a pair of geometric graphs that can be distinguished by GWL but not by IGWL.
We may consider any k -hop distinct geometric graphs for k > 1, where the underlying attributed graphs are isomorphic.
Proposition 1 states that GWL can distinguish any such graphs, while Proposition 4 states that IGWL cannot distinguish them.
An example is the pair of graphs in Figures 1 and 2.
Extended Abstract Track
Proposition 15 IGWL has the same expressive power as GWL for fully connected geometric graphs.
Proof [Proof of Proposition 7] We will prove by contradiction.
Assume that there exist a pair of fully connected geometric graphs G 1 and G 2 which GWL can distinguish, but IGWL cannot.
If the underlying attributed graphs of G 1 and G 2 are isomorphic, by Proposition 1 and
Proposition 4, G 1 and G 2 are 1-hop identical but k -hop distinct for some k > 1.
For all bijections b and all nodes i ∈ V 1 , b ( i )
∈ V 2 , the local neighbourhoods N (1) i and N (1) b ( i ) are identical up to group actions, and O G ( g (1) i ) =
O G ( g (1) b ( i ) )
⇒ c (1) i =
c (1) b ( i ) .
Additionally, there exists some bijection b and some nodes i ∈ V 1 , b
( i ) ∈
V
such that the k -hop subgraphs N ( k ) i and N ( k ) b ( i ) are distinct, and O G ( g ( k ) i )
∩ O G ( g ( k ) b ( i ) )
≡ ∅ ⇒ c ( k ) i ̸
= c ( k ) b ( i ) .
However, as G 1 and
G 2 are fully connected, for any k , N (1) i = N ( k )
i and N (1) b ( i ) =
N ( k ) b ( i ) are identical up to group actions.
Thus, O G ( g (1) i ) =
O G ( g ( k ) i ) =
O G ( g (1) b ( i ) )
=
O G ( g ( k ) b ( i ) )
⇒ c (1) i =
c ( k ) i
=
c ( k ) b ( i ) =
c ( k ) b ( i ) .
This is a contradiction.
If G 1 and G 2 are non-isomorphic and fully connected, for any arbitrary i ∈ V 1 , j ∈ Vand any k -hop neighbourhood, we know that N (1) i = N ( k )
i
and
N (1) j = N ( k ) j .
Thus, a single iteration of GWL and IGWL identify the same G -orbits and assign the same node colours, i.e. O G ( g (1) i ) =
O G ( g ( k ) i ) ⇒ c
(1) i =
c ( k ) i and O G ( g (1) j )
=
O G ( g ( k ) j ) ⇒ c
(1) j = c ( k ) j .
This is a contradiction.
Appendix D. Proofs for equivalence between GWL and Geometric GNNs
Our proofs adapt the techniques used in (Xu et al., 2019; Morris et al., 2019) for connecting 1-WL with GNNs.
Note that we omit including the relative position vectors −→ x ij =
−→ x i −−→ x j in GWL and geometric GNN updates for brevity, as relative positions vectors can be merged into the vector features.
Theorem 8 Any pair of geometric graphs distinguishable by a G -equivariant GNN is also distinguishable by GWL.
Proof [ Proof of Theorem 8 ]
Consider two geometric graphs G and H .
The theorem implies that if the GNN graphlevel readout outputs f ( G ) ̸ = f ( H ), then the GWL test will always determine G and H to be non-isomorphic, i.e. G ̸ = H .
We will prove by contradiction.
Suppose after T iterations, a GNN graph-level readout outputs f ( G ) ̸
= f ( H ), but the GWL test cannot decide G and H are non-isomorphic, i.e.
G and H always have the same collection of node colours for iterations 0 to T .
Thus, for iteration t and t + 1 for any t = 0 . . .
T − 1, G and H have the same collection of node colours { c ( t ) i } as well as the same collection of neighbourhood geometric multisets (cid:110)
( c ( t ) i , g ( t ) i ) , {{ ( c ( t ) j , g ( t ) j ) | j ∈ N i }} (cid:111) up to group actions.
Otherwise, the GWL test would have produced different node colours at iteration t + 1 for G and H as different geometric multisets get unique new colours.
Extended Abstract Track
We will show that on the same graph for nodes
i and k , if ( c ( t ) i , g ( t ) i ) =
( c ( t ) k , g · g ( t ) k ), we always have GNN features ( s ( t ) i , −→ v ( t ) i ) =
( s ( t ) k , Q g −→ v ( t ) k ) for any iteration t .
This holds for t = 0 because GWL and the GNN start with the same initialisation.
Suppose this holds for iteration t .
At iteration t + 1, if for any i and k , ( c ( t +1)
i , g ( t +1) i
[Equation]
= ( c
[Equation]
t +
[Equation]
k
[Equation]
g · g
[Equation]
t
[Equation]
k ), then:
[Equation]
[Equation]
By our assumption on iteration t ,
[Equation]
As the same aggregate and update operations are applied at each node within the GNN, the same inputs, i.e. neighbourhood features, are mapped to the same output.
Thus, ( s ( t +1)
i , −→ v ( t +1) i )
=
( s ( t +1) k , Q g −→ v ( t +1) k ).
By induction, if ( c ( t ) i , g ( t ) i ) =
( c ( t ) k , g · g ( t ) k ), we always have GNN node features ( s ( t ) i , −→ v ( t ) i ) =
( s ( t ) k , Q g −→ v ( t ) k ) for any iteration t .
This creates valid mappings ϕ s , ϕ v such that s ( t ) i = ϕ
s ( c ( t ) i ) and −→ v ( t ) i =
ϕ v ( c ( t ) i , g ( t ) i ) for any i ∈ V .
Thus, if G and H have the same collection of node colours and geometric multisets, then
G and H also have the same collection of GNN neighbourhood features (cid:110) ( s ( t ) i , −→ v ( t ) i ) , {{ ( s ( t ) j , −→ v ( t ) j ) | j ∈ N i }} (cid:111) =
(cid:110) ( ϕ s ( c ( t ) i ) , ϕ v ( c ( t ) i , g ( t ) i )) , {{ ( ϕ s ( c ( t ) j ) , ϕ v ( c ( t ) i , g ( t ) i )) | j ∈ N i }} (cid:111)
Thus, the GNN will output the same collection of node scalar features { s ( T ) i } for G and
H and the permutation-invariant graph-level readout will output f ( G ) =
f ( H ).
This is a contradiction.
Proposition 16 G -equivariant GNNs have the same expressive power as GWL if the following conditions hold: (1) The aggregation Agg is an injective, G -equivariant multiset function.
The scalar part of the update Upd s is a G -orbit injective, G -invariant multiset function.
The vector part of the update Upd v is an injective, G -equivariant multiset function.
The graph-level readout f is an injective multiset function.
Proof [ Proof of Theorem 9 ]
Consider a GNN where the conditions hold.
We will show that, with a sufficient number of iterations t , the output of this GNN is equivalent to GWL, i.e. s ( t ) ≡ c ( t ) .
Let G and H be any geometric graphs which the GWL test decides as non-isomorphic at iteration T .
Because the graph-level readout function is injective, i.e. it maps distinct multiset of node scalar features into unique embeddings, it suffices to show that the GNN’s neighbourhood aggregation process, with sufficient iterations, embeds G and H into different multisets of node features.
For this proof, we replace G -orbit injective functions with injective functions over the equivalence class generated by the actions of G .
Thus, all elements belonging to the same
G -orbit will first be mapped to the same representative of the equivalence class, denoted by the square brackets [ . .
. ], followed by an injective map.
The result is G -orbit injective.
Extended Abstract Track
Let us assume the GNN updates node scalar and vector features as:
[Equation]
t
[Equation] i [Equation] with the aggregation function Agg being G -equivariant and injective, the scalar update function Upd s being G -invariant and injective, and the vector update function Upd v being
G -equivariant and injective.
The GWL test updates the node colour c ( t ) i and geometric multiset g ( t ) i as:
[Equation]
where h s is a G -invariant and injective map, and h v is a G -equivariant and injective operation (e.g. in equation 2, expanding the geometric multiset by copying).
We will show by induction that at any iteration t , there always exist injective functions
φ s and φ v such that s ( t )
i =
φ s ( c ( t ) i ) and −→ v ( t ) i
=
φ v ( c ( t ) i , g ( t ) i ).
This holds for t = 0 because the initial node features are the same for GWL and GNN, c (0) i ≡ s (0) i and g (0)
i ≡ ( s (0) i , −→ v (0) i ) for all i ∈ V ( G ) , V ( H ).
Suppose this holds for iteration t .
At iteration t + 1, substituting s ( t ) i with φ s ( c ( t ) i ), and −→ v ( t ) i with φ v ( c ( t ) i , g ( t ) i ) gives us s ( t +1) i [Equation] −→ v ( t +1) i =
[Equation]
The composition of multiple injective functions is injective.
Therefore, there exist some injective functions
g s and g v such that:
[Equation]
We can then consider:
[Equation]
Then, we can denote φ s =
g s ◦ h − 1 s and φ v =
g v ◦ h − 1 v as injective functions because the composition of injective functions is injective.
Hence, for any iteration t +1, there exist injective functions φ s and φ v such that s ( t +1)
i =
φ s (cid:16) c ( t +1) i (cid:17) and −→ v ( t +1) i = φ v (cid:16) c ( t +1)
i , g ( t +1) i (cid:17) . At the T -th iteration, the GWL test decides that G and H are non-isomorphic, which means the multisets of node colours { c ( T ) i } are different for G and H .
The GNN’s node scalar features { s ( T ) i } = { φ s ( c ( T ) i ) } must also be different for G and H because of the injectivity of φ s .