Expectation Complete Graph Representations Using Graph Homomorphisms
Abstract
We propose and study a practical graph embedding that in expectation is able to distinguish all non-isomorphic graphs and can be computed in polynomial time.
The embedding is based on Lovász’ characterisation of graph isomorphism through an infinite dimensional vector of homomorphism counts.
Recent work has studied the expressiveness of graph embeddings by comparing their ability to distinguish graphs to that of the Weisfeiler-Leman hierarchy.
While previous methods have either limited expressiveness or are computationally impractical, we devise efficient sampling-based alternatives that are maximally expressive in expectation.
We empirically evaluate our proposed embeddings and show competitive results on several benchmark graph learning tasks
Introduction
We study novel efficient and expressive graph embeddings based on Lovász’ characterisation of graph isomorphism through homomorphism counts.
While most practical graph embeddings drop the property of completeness , that is, the ability to distinguish all non-isomorphic graphs, in favour of runtime, we devise efficient embeddings that retain completeness in expectation .
To achieve that, we sample pattern graphs in a particular way, simultaneously guaranteeing completeness and polynomial runtime in expectation.
We discuss related work, in particular the relationship to the k -dimensional
Weisfeiler Leman isomorphism test, and show first results on benchmark datasets.
While subgraph counts are also a reasonable choice for expectation complete graph embeddings, they have multiple drawbacks compared to homomorphism counts.
Most importantly, from a computational perspective, computing subgraph counts even for simple graphs such as trees or paths is NP-hard
[Alon et al., 1995; Marx and Pilipczuk, 2014], while we can compute homomorphism counts efficiently [Díaz et al., 2002] as long as the pattern graphs have small treewidth , a measure of ‘tree-likeness’.
In particular, all known exact algorithms for subgraph isomorphism have a runtime exponentially in the pattern size or the maximum degree of the pattern even for small treewidth — one of the main reasons why the graphlet kernel [Shervashidze et al., 2009] and similar fixed pattern based approaches [Bouritsas et al., 2022] only count subgraphs up to size around 5.
Probably most important from a conceptual perspective is the relationship of homomorphism counts to the cut distance [Borgs et al.,
Lovász, 2012].
The cut distance is a well-studied and important distance on graphs that captures global structural but also sampling-based local information.
It is well known that the distance given by (potentially approximated and sampled) homomorphism counts is close to the cut distance and hence has similar favourable properties.
The cut distance, and hence, homomorphism counts, capture the behaviour of all permutation-invariant functions on graphs.
For an ongoing discussion about the importance of the cut distance and homomorphism counts in the context of graph learning, see Dell et al.
[2018], Grohe [2020], and Hoang and Maehara [2020].
Completeness in expectation essentially implies one powerful fact that no deterministic embedding with bounded expressiveness can guarantee: repetition will make the embedding more expressive eventually.
If the graph embedding is complete in expectation it is guaranteed that sampling more patterns will eventually increase its expressiveness.
Complete Graph Embeddings
The graph isomorphism problem is a classical problem in graph theory and its computational complexity is a major open problem [Babai, 2016].
Following the classical result of Lovász [1967], two graphs are isomorphic if
and only if they have the same infinite dimensional homomorphism count vectors.
This provides a powerful graph embedding for graph classification tasks [Barceló et al., 2021; Dell et al., 2018; Hoang and Maehara, 2020].
A graph G =
( V ( G ) , E ( G )) consists of a set V ( G ) of vertices and a set E ( G ) =
{ e ⊆ V | | e | = 2 } of edges .
The size of a graph is the number of its vertices.
In the following F and G denote graphs, where F represents a pattern graph and G a graph in our training set.
A homomorphism Φ : V ( F ) → V ( G ) is a map that respects edges, i.e. { v, w } ∈ E ( F ) ⇒ { Φ( v ) , Φ( w ) } ∈ E ( G ) .
An isomorphism is a bijective homomorphism whose inverse is also a homomorphism.
We say that a distribution D over a countable domain X has full support if each x ∈ X has nonzero probability.
Let G n be the set of all finite graphs of size at most n and let hom( F, G ) denote the number of homomorphisms of F to G for arbitrarily graphs and φ n ( G )
=
hom( G n , G ) =
(hom( F, G ))
F ∈G n denote the Lovász vector of G for G n .
Lovász [1967] proved the following classical theorem.
Theorem 1 (Lovász [1967] 2 ) .
Two arbitrary graphs G, H ∈ G n are isomorphic iff φ n ( G ) =
φ n ( H ) .
We can define a simple kernel on G n with the canonical inner product using φ n .
Definition 2 (Complete Lovász kernel) .
Let k φ n ( G, H ) =
⟨ φ n ( G ) , φ n ( H ) ⟩ .
Note that k φ n is a complete graph kernel [Gärtner et al., 2003] on G n , i.e., k φ n can be used to distinguish non-isomorphic graphs of size n .
Similarly, we define complete graph embeddings.
Definition 3.
Let φ : G → V be a permutation-invariant graph embedding from a family of graphs G to a vector space V .
We call φ complete (on G ) if φ ( G ) ̸
= φ ( H ) for all non-isomorphic G, H ∈ G .
When studying graph embeddings and graph kernels we face the tradeoff between efficiency and expressiveness: complete graph representations are unlikely to be computable in polynomial time [Gärtner et al., 2003] and hence most practical graph representations drop completeness in favour of polynomial runtime.
In our work, we study random graph representations.
While dropping completeness and being efficiently computable, this allows us to keep a slightly weaker yet desirable property: completeness in expectation .
Definition 4.
A graph embedding φ X , which depends on a random variable X , is complete in expectation if the graph embedding given by the expectation, E X [ φ X ( · )] , is complete.
Similarly, we say that the corresponding kernel k X ( G, H ) =
⟨ φ X ( G ) , φ X ( H ) ⟩ is complete in expectation.
We can use Lovász’ isomorphism theorem to devise graph embeddings that are complete in expectation.
For that let e F ∈ R G n be the ‘ F th’ standard basis unit-vector of G n .
Theorem 5.
Let D be a distribution on G n with full support and G ∈ G n .
Then the graph embedding φ F ( G ) =
hom( F, G ) e F with F ∼ D and the corresponding kernel k are complete in expectation.
2.1 Expectation Complete Embeddings and Kernels on G ∞
In this section, we generalise the previous result to the set of all finite graphs G ∞ .
Theorem 1 holds for G, H ∈ G ∞ and the mapping φ ∞ that maps each G ∈ G ∞ to an infinite-dimensional vector.
The resulting vector space, however, is not a Hilbert space with the usual inner product.
To see this, consider any graph G that has at least one edge.
Then hom( P n , G ) ≥ 2 for every path P n of length n ∈ N .
Thus, the inner product ⟨ φ ∞ ( G ) , φ ∞ ( G ) ⟩ is not finite.
To define a kernel on G ∞ without fixing a maximum size of graphs, i.e., restricting to G n for some n ∈ N , we define the countable-dimensional vector φ ∞ ( G ) =
(cid:0) hom | V ( G ) | ( F, G ) (cid:1) F ∈G ∞ where hom | V ( G ) | ( F, G ) =
[Equation]
That is, φ ∞ ( G ) is the projection of φ ∞ ( G ) to the subspace that gives us the homomorphism counts for all graphs of size at most of G .
Note that this is a well-defined map of graphs to a subspace of the ℓ 2 space, i.e., sequences ( x i ) i over R with (cid:80) i
| x i | 2 < ∞ .
Hence, the kernel given by the canonical inner product k ∞ ( G, H ) = ⟨ φ ∞ ( G ) , φ ∞ ( H ) ⟩ is finite and positive semi-definite.
Note that we can rewrite k ∞ ( G, H ) =
k min ( G, H ) = ⟨ φ n ′ ( G ) , φ n ′ ( H ) ⟩ where n ′ =
min {| V ( G ) | , | V ( H ) |} .
While the first hunch might be to count patterns up to max {| V ( G ) | , | V ( H ) |} , this is not necessary to guarantee completeness.
Furthermore, the corresponding map k max is not even positive semi-definite.
Lemma 6.
k min is a complete kernel on G ∞ .
Given a sample of graphs S , we note that for n = max G ∈ S | V ( G ) | we only need to consider patterns up to size n .
As the number of graphs of a given size n are superexponential it is impractical to compute all such counts.
Hence, we propose to resort to sampling.
Theorem 7.
Let D be a distribution on G ∞ with full support and G ∈ G ∞ .
Then φ F ( G ) = hom | V ( G ) | ( F, G ) e F with F ∼ D and the corresponding kernel are complete in expectation.
2.2 Sampling multiple patterns
Sampling just one pattern F will not result in a practical graph embedding.
Thus, we propose to sample ℓ patterns F 1 , . .
F ℓ ∼ D i.i.d.
and construct the embedding φ ℓ ( G ) ∈ N ℓ 0 with ( φ ℓ ( G ))
i = hom( F i , G ) if | V ( F i ) | ≤ | V ( G ) | and 0 otherwise for all i ∈ [ ℓ ] .
For the dot product it holds that φ ℓ ( G ) T φ ℓ ( H ) = (cid:80) ℓi
=1 ⟨ φ F
i
G ) , φ F i ( H ) ⟩ as long as we do not sample patterns twice.
Computing Embeddings in Expected Polynomial Time
A graph embedding that is complete in expectation must be efficiently computable to be practical.
In this section, we describe our main result achieving polynomial runtime in expectation.
The best known algorithms [Díaz et al., 2002] to exactly compute hom( F, G ) take time
[Equation]
where tw( F ) is the treewidth of the pattern graph H .
Thus, a straightforward sampling strategy to achieve polynomial runtime in expectation is to give decreasing probability mass to patterns with higher treewidth.
Unfortunately, in the case of G ∞ , this is not possible.
Proposition 8.
There exists no distribution D with full support on G ∞ such that the expected runtime of Eq.
(1) becomes polynomial in | V ( G ) | for all G ∈ G ∞ .
To resolve this issue we have to take the size of the largest graph in our sample into account.
For a given sample S ⊆ G n of graphs, where n is the maximum number of vertices in S , we can construct simple distributions achieving polynomial time in expectation.
Theorem 9.
There exists a distribution D such that computing the expectation complete graph embedding φ X ( G ) takes polynomial time in | V ( G ) | in expectation for all G ∈ G n .
Proof.
Sketch .
We first draw a treewidth upper bound k from an appropriate distribution.
For example, a Poisson distribution with parameter λ
= O ( log n / n ) is sufficient.
We have to ensure that each possible graph with treewidth up to k gets a nonzero probability of being drawn.
For that we first draw a k -tree, a maximal graph of treewidth k , and then take a random subgraph of it.
Note that we do not require that the patterns are sampled uniformly at random.
It merely suffices that each pattern has a nonzero probability of being drawn.
To satisfy a runtime of O ( | V ( G ) | d +1 ) in expectation, for example, a Poisson distribution with λ ≤ 1+ d log n n is sufficient.
[Table]
Related Work
The k -dimensional Weisfeiler-Leman (WL) test
[Cai et al., 1992] and the Lovász vector restricted to patterns up to treewidth k are equally expressive
[Dell et al., 2018; Dvoˇrák, 2010].
We propose an efficiently computable embedding matching the expressiveness of k -WL
[Table]
and hence also MPNNs and k -GNNs [Morris et al., 2019; Xu et al., 2019], in expectation, see Appendix D.
Dell et al.
[2018] proposed a complete graph kernel based on homomorphism counts related to our k min kernel.
Instead of implicitly restricting the embedding to only a finite number of patterns, as we do, they weigh the homomorphism counts such that the inner product defined on the whole Lovász vectors converges.
However, Dell et al.
[2018] do not discuss how to compute their kernel and so, our approach can be seen as an efficient sampling-based alternative to their theoretical weighted kernel.
Using graph homomorphism counts as a feature embedding for graph learning tasks was proposed by Hoang and Maehara [2020] and Kühner [2021].
Hoang and Maehara [2020] discuss various aspects of homomorphism counts important for learning tasks, in particular, universality aspects and their power to capture certain properties of graphs, such as bipartiteness.
Instead of relying on sampling patterns, which we use to guarantee expectation in completeness, they propose to use a fixed number of small pattern graphs.
This limits the practical usage of their approach due to computational complexity reasons.
In their experiments the authors only use tree (GHC-tree(6)) and cycle patterns (GHC-cycle(8))
up to size 6 and 8, respectively, whereas we allow patterns of arbitrary size and treewidth, guaranteeing polynomial runtime in expectation.
Simiarly to Hoang and Maehara [2020], we use the computed embeddings as features for a kernel SVM (with RBF kernel) and an MLP.
Instead of embedding the whole graph into a vector of homomorphism counts, Barceló et al.
[2021] proposed to use rooted homomorphism counts as node features in conjunction with a graph neural network (GNN).
They discuss the required patterns to be as or more expressive than the k -WL test.
We achieve this in expectation when selecting an appropriate sampling distribution, see Appendix D.
Wu et al.
[2019] adapted random Fourier features [Rahimi and Recht, 2007] to graphs and proposed a sampling-based variant of the global alignment graph kernel.
Similar sampling-based ideas were discussed before for the graphlet kernel [Shervashidze et al., 2009] and frequent-subtree kernels [Welke et al., 2015].
All three papers do not discuss expressiveness aspects, however.
Experiments
We performed some preliminary experiments on some benchmark datasets.
To this end, we sample a fixed number ℓ = 30 of patterns as described in Appendix A and compute the sampled min kernel as described in Section 3.
Table 1 shows averaged accuracies of SVM and MLP classifiers trained on our feature sets.
We follow the experimental design of Hoang and Maehara [2020] and compare to their published results.
Even with as little as 30 features, the results of our approach are comparable to the competitors on real world datasets.
Furthermore, it is interesting to note that a SVM with
RBF kernel and our features performs perfectly on the PAULUS25 dataset, i.e., it is able to decide isomorphism for the strongly regular graphs in this dataset.
It also shows good performance, although with high deviation, on the CSL dataset, where only the method specifically designed for this dataset,
GHC-cycle, performs well.
We also included GNTK [Du et al., 2019], GIN [Xu et al., 2019], and
the
WL-
Conclusion
As future work we will investigate approximate counts to make our implementation more efficient [Beaujean et al., 2021].
It is unclear how this affects expressiveness, as we loose permutationinvariance.
Going beyond expressiveness results our goal is to further study graph similarities suitable for graph learning, such as the cut distance as proposed by Grohe [2020].
Finally, instead of sampling patterns from a fixed distribution a more promising variant is to adapt the sampling process in a sample-dependent manner.
One could, for example, draw new patterns until each graph in the sample has a unique embedding (up to isomorphism) or at least until we can distinguish 1-WL classes.
Alternatively, we could pre-compute frequent or interesting patterns as proposed by Schulz et al.
[2018] and use them to adapt the distribution.
Such approaches would use the power of randomisation to select an appropriate graph embedding in a data-driven manner, instead of relying on a finite set of fixed and pre-determined patterns like previous work [Barceló et al., 2021; Bouritsas et al., 2022].
A Sampling Details
Given a pattern size N ∈ N , we first draw a treewidth upper bound k < N given from some distribution.
Then we want to sample any graph with treewidth at most k with a nonzero probability.
A natural strategy is to first sample a k -tree , which is a maximal graph with treewidth k , and then take a random subgraph of it.
Uniform sampling of k -trees is described by Nie et al.
[2015] and
Caminiti et al.
Alternatively, the strategy of Yoo et al.
[2020] is also possible.
Note that we only have to guarantee that each pattern has a nonzero probability of being sampled; it does not have to be uniform.
While guaranteed uniform sampling would be preferable, we resort to a simple sampling scheme that is easy to implement.
We achieve a nonzero probability for each pattern of at most a given treewidth k by first constructing a random k -tree P through its tree decomposition, by uniformly drawing a tree T on N − k vertices and choosing a root.
We then create P as the (unique up to isomorphism)
k -tree that has T as tree decomposition.
We then randomly remove edges from that k -tree i.i.d.
with fixed probability (currently set to 0 . 1 ).
This ensures that each subgraph of P will be created with nonzero probability.
B Implementation Details and Benchmark Datasets
Our source code is available on github 6 and the datasets in the correct format can be downloaded from a google drive 7 .
We rely on the C++ code of Curticapean et al.
[2017] 8 to efficiently compute homomorphism counts.
While the code computes a tree decomposition itself we decided to simply provide it with our tree decomposition of the k -tree which we compute anyway, to make the computation more efficient.
Additionally, we use the cross-validation-based eveluation with SVM and MLP of Hoang and Maehara [2020] 9 .
MUTAG, IMDB-BIN, and IMDB-MULTI are taken from TU datasets [Morris et al., 2020],
PAULUS25 is from Hoang and Maehara [2020] 10 , and CSL is from Murphy et al.
Table 2 shows preliminary runtime results.
It reports the time taken for sampling 30 patterns and computing homomorphism counts for all patterns and all graphs in each database, i.e. the time to compute graph embeddings.
We compare to the runtime of the combined GHC-full(6)
= GHCcycle(6)
+ GHC-tree(6)
embedding computation.
GHC is much faster, as GHC-full(6) only computes counts for a total of 18 tree and cycle patterns with up to six vertices.
Our method allows for larger and more complex patterns and we pay a corresponding runtime cost.
All runtimes were obtained on an AMD Ryzen 9 3900X 12-Core Processor with 64GB of RAM running Ubuntu 20.04.
Note that our implementation is single-threaded and not yet well optimized.
C Proofs
Theorem 5.
Let D be a distribution on G n with full support and G ∈ G n .
Then the graph embedding φ F ( G ) =
hom( F, G ) e F with F ∼ D and the corresponding kernel k are complete in expectation.
Proof.
Let D and φ F with F ∼ D as stated and G ∈ G n .
Then
[Equation]
The vector g has the entries ( g ) F ′ =
Pr ( F = F ′ )
hom( F ′ , G ) .
Let G ′ be a graph that is nonisomorphic to G and let g ′ =
E F [ φ F ( G ′ )] accordingly.
By Theorem 1 we know that hom( G n , G ) ̸
= hom( G n , G ′ ) .
Thus, there is an F ′ such that hom( F ′ , G ) ̸
= hom( F ′ , G ′ ) .
By definition of D we have that Pr( F = F ′ ) > 0 and hence Pr( F = F ′ )
hom
( F ′ , G ) ̸
= Pr( F = F ′ )
hom( F ′ , G ′ ) which implies g ̸ = g ′ .
That shows that E F [ φ F ( · )] is complete and concludes the proof.
Lemma 6.
k min is a complete kernel on G ∞ .
Proof.
Let G, H ∈ G ∞ .
We have to show that
[Equation]
where G ≃ H indicates that G and H are isomorphic.
There are two cases: | V ( G ) | = | V ( H ) | : Then, by Theorem 1 we have φ N ( G ) =
φ n ( H ) iff G ≃ H for N = min {| V ( G ) | , | V ( H ) |} = | V ( G ) | = | V ( H ) | .
| V ( G ) | ̸ =
| V ( H ) | : Let w.l.o.g.
< | V ( G ) | < | V ( H ) | .
Let P be the graph on exactly one vertex.
Then hom( P, G ) < hom( P, H ) , i.e., we can distinguish graphs on different numbers of vertices using homomorphism counts.
As min {| V ( G ) | , | V ( H ) |} ≥ 1 , we have P ∈ G | V ( G ) | and hence φ | V ( G ) | ( G ) ̸
= φ | V ( G )
| ( H ) .
The other direction follows directly from the fact that homomorphism counts are invariant under isomorphism.
Theorem 7.
Let D be a distribution on G ∞ with full support and G ∈ G ∞ .
Then φ F ( G ) = hom | V ( G ) | ( F, G ) e F with F ∼ D and the corresponding kernel are complete in expectation.
Proof.
We can apply the same arguments as before from Theorem 5 to show that the expected embeddings of two graphs G, H with size n ′ =
min {| V ( G ) | , | V ( H ) |} are equal iff their Lovász vector restricted to size n ′ are equal.
By Lemma 6 we know that the latter only can happen if the two graphs are isomorphic.
Proposition 8.
There exists no distribution D with full support on G ∞ such that the expected runtime of Eq.
(1) becomes polynomial in | V ( G ) | for all G ∈ G ∞ .
Proof.
Let D be such a distribution and let D ′ be the marginal distribution on the treewidths of the graphs given by p k
=
Pr F ∼D (tw( F ) = k ) > 0 .
Let G be a given input graph in the sample with n = | V ( G ) | .
Díaz et al.
[2002] has shown that computing hom( F, G ) takes time
O (cid:0) | V ( F ) ||
V ( G ) | tw( F )+1 (cid:1) .
Assume for the purpose of contradiction that we can guarantee an expected polynomial runtime (ignoring the | V ( F ) | and constant factors for simplicity):
[Equation]
for some constants C, c ∈ N .
Then for all k ≥ c , it must hold that p k n k +1 ≤ Cn c , as all summands are positive.
However, for large enough n the left hand side is larger than the right hand side.
Contradiction.
Theorem 9.
There exists a distribution D such that computing the expectation complete graph embedding φ X ( G ) takes polynomial time in | V ( G ) | in expectation for all G ∈ G n .
Proof.
Let G ∈ G n .
Draw a treewidth upper bound k from a Poisson distribution with parameter λ to be determined later.
Select a distribution D n,k which has full support on all graphs with treewidth up to k and size up to n , for example, the one described in Appendix A. Using the algorithm of [Díaz et al., 2002] this gives, for some constant C ∈ N , an expected runtime of
E
[Equation]
We need to bound the right hand side by some polynomial Dn d for some constants D, d ∈ N .
By rearranging terms we see that
[Equation]
is sufficient.
D Matching the Expressiveness of k -WL in Expectation
We devise a graph embedding matching the expressiveness of the k -WL test in expecation.
Theorem 10.
Let D be a distribution with full support on the set of graphs with treewidth up to k .
The resulting graph embedding φ k -WL F ( · ) with F ∼ D has the same expressiveness as the k -WL test in expectation.
Furthermore, there is a specific such distribution such that we can compute φ k -WL F ( G ) in expected polynomial time O ( | V ( G ) | k +1 ) for all G ∈ G ∞ .
Proof.
Let T k be the set of graphs with treewidth up to k and D be a distribution with full support on
T k .
Then by the same arguments as before in Theorem 5, the expected embeddings of two graphs G and H are equal iff their Lovász vectors restricted to patterns in T k are equal.
By Dvoˇrák [2010] and
Dell et al.
[2018] the latter happens iff k -WL returns the same color histogram for both graphs.
This proves the first claim.
For the second claim note that the worst-case runtime for any pattern F ∈
T k is
O (cid:0) | V ( F ) ||
V ( G )
|
k +1 (cid:1) by Díaz et al.
However, the equivalence between homomorphism counts on T k and k -WL requires to inspect also patterns F of all sizes, in particular, also larger than the size n of the input graph.
To remedy this, we can draw the pattern size m = | V ( F ) | from some distribution with bounded expectation and full support on N .
For example, the geometric m ∼ Geom( p ) with any constant parameter p ∈ (0 , 1) and expectation E [ m ] = 11 − p is sufficient.
By linearity of expectation then
[Equation]
Note that for the embedding φ k -WL F ( · ) Lemma 8 does not apply.
In particular, the used distribution guaranteeing polynomial expected runtime is independent of n and can be used for all G ∞ .