Towards Bridging Generalization and Expressivity of Graph Neural Networks

## Abstract

Abstract Expressivity and generalization are two critical aspects of graph neural networks (GNNs). While significant progress has been made in studying the expressivity of GNNs, much less is known about their generalization capabilities, particularly when dealing with the inherent complexity of graph-structured data.
In this work, we address the intricate relationship between expressivity and generalization in GNNs. Theoretical studies conjecture a trade-off between the two: highly expressive models risk overfitting, while those focused on generalization may sacrifice expressivity. However, empirical evidence often contradicts this assumption, with expressive GNNs frequently demonstrating strong generalization. We explore this contradiction by introducing a novel framework that connects GNN generalization to the variance in graph structures they can capture. This leads us to propose a k 𝑘 k italic_k -variance margin-based generalization bound that characterizes the structural properties of graph embeddings in terms of their upper-bounded expressive power. Our analysis does not rely on specific GNN architectures, making it broadly applicable across GNN models. We further uncover a trade-off between intra-class concentration and inter-class separation, both of which are crucial for effective generalization. Through case studies and experiments on real-world datasets, we demonstrate that our theoretical findings align with empirical results, offering a deeper understanding of how expressivity can enhance GNN generalization.

## 1 Introduction

Graph Neural Networks (GNNs) (Scarselli et al., ) have become pivotal in modern machine learning, anchored in two main pillars: expressivity and generalization. Expressivity refers to a GNN’s capacity to distinguish between diverse graph structures, thereby determining the scope of problems it can address (Xu et al., ; Morris et al., ). Highly expressive GNNs can capture intricate dependencies, essential for tasks like molecular property prediction (Gilmer et al., ), drug discovery (Gaudelet et al., ), and protein-protein interaction prediction (Zitnik et al., ), where minor structural variations have significant implications. Generalization, on the other hand, reflects a GNN’s ability to transfer learned knowledge to unseen graphs. Given the diversity in graph structures, sizes, and complexities, GNNs that generalize well maintain consistent performance across varying datasets. Together, these properties enable GNNs to model complex graph structures while remaining effective across new, unseen data, making them invaluable for graph-based analysis.

Theoretically, a trade-off is expected between expressivity and generalization: highly expressive models can capture complex graph structures but may overfit and generalize poorly without proper regularization. Conversely, models focused on generalization often sacrifice some expressivity to perform better across diverse, unseen graph structures. Recent work indeed shows a strong correlation between a GNN’s VC dimension and its ability to distinguish non-isomorphic graphs (Morris et al., ). *A more nuanced theoretical analysis is needed*, however.
Indeed, *empirical evidence frequently contradicts the above view.*
Highly expressive models often exhibit strong generalization performance in practice (Bouritsas et al., ; Wang et al., ). In the restricted context of linear separability, margin-based bounds offer partial alignment between theory and practice (Franks et al., ), yet our broader understanding of how expressivity influences generalization remains incomplete. This raises two key questions: (i) *How does the structured nature of graphs affect GNN generalization?* (ii) *How does a GNN’s expressivity influence its ability to generalize across tasks and unseen data?* Addressing these questions is vital for advancing GNN applications in real-world scenarios.

Present work.
Building on the foundational work of Chuang et al. (), we explore how the *concentration* and *separation* of *learned features*, key factors in multiclass classification generalization, translate to graph-based models. Their bound, derived from $k$-variance (Solomon et al., ) and the expected optimal transport cost between two random subsets of the training distribution, motivates our adaptation to graph embeddings. Leveraging these insights, we extend their framework to capture the structural properties of graph embedding distributions and contribute the following:

- •
For arbitrary graph encoders, including GNNs, we show that their generalization can be bounded in terms of the generalization bound of any more expressive graph encoder. This allows capturing structural properties of graph embedding distributions with respect to their bounding encoders.
- •
Under certain margin conditions, we demonstrate that the downstream classifier generalizes well if (1) embeddings within a class are well-clustered and (2) classes are separable in the embedding space in the Wasserstein sense, extending Chuang et al. ()’s results to graphs.
- •
On the real-world PROTEINS dataset (Morris et al., ), we empirically show how a more expressive model influences generalization by measuring variance in graph embedding distributions.
- •
We apply the empirical sample-based bound of Chuang et al. () to graph classification tasks, verifying that empirical findings align with our theoretical insights, thus demonstrating the applicability of our approach to predict generalization.

Our results offer a flexible framework for analyzing generalization properties of complex graph encoders via simpler encoders, such as those based on $\mathsf{1}\text{-}\text{WL}$, its higher-order variants $k\text{-}\text{WL}$ (Cai et al., ; Grohe, ), homomorphism counts (Zhang et al., ), or $\mathcal{F}\text{-}\text{WL}$ (Barceló et al., ), provided they upper-bound the encoders under consideration. *Overall, we present a versatile tool for evaluating whether increased expressiveness improves or worsens generalization.*

## 2 Related work

Regarding generalisation of GNNs,
Scarselli et al. () utilize VC dimension to study the generalization of an older GNN architecture, distinct from modern MPNNs (Gilmer et al., ). Garg et al. () show that the Rademacher complexity of simple GNNs depends on maximum degree, layer count, and parameter norms, while Liao et al. () develop PAC-Bayesian bounds relying on node degree and spectral norms; see Karczewski et al. () for extensions. Improved bounds using the largest singular value of the diffusion matrix are proposed by Ju et al. (). Transductive PAC-Bayesian bounds for knowledge graphs are discussed by Lee et al. (). Random graph models are leveraged by Maskey et al. (), who show GNN generalization improves with larger graphs. Connections between VC-dimension and the 1-WL algorithm are made by Morris et al. (), who bound it by the number of 1-WL colors. Levie () provide bounds based on covering numbers and specialized graph metrics.

For GCNs, Verma and Zhang () derive generalization bounds using algorithmic stability, with Zhang et al. () focusing on single-layer GCNs and accelerated gradient descent. Zhou and Wang () extend this to multi-layer GCNs, showing that generalization gaps increase with depth. Similarly, Cong et al. () highlight this trend in deeper GNNs and propose detaching weight matrices to improve generalization. Further analyses of transductive Rademacher complexity using stochastic block models are offered by Oono and Suzuki (); Esser et al. (). Tang and Liu () establish bounds involving node degree, training iterations, and Lipschitz constants, while Li et al. () study topology sampling and its impact on generalization. Lastly, Franks et al. () explore margin-based bounds.

Moving to the expressivity of GNNs, MPNNs’ expressivity is bounded by $\mathsf{1}\text{-}\text{WL}$ (Xu et al., ; Morris et al., ), showing the need for more expressive methods. Many such models have been put forward. For example, the $\mathcal{F}$-MPNNs (Barceló et al., ) enhance expressivity via homomorphism counts, similar to Bouritsas et al. (). Homomorphism counts have become a popular mechanism in graph learning (Nguyen and Maehara, ; Welke et al., ; Zhang et al., ; Jin et al., ; Lanzinger and Barcelo, ), and will be central to our analysis. Additional discussion on related work can be found in Appendix A.

## 3 Preliminaries

#### Graphs and homomorphisms.

We begin by considering undirected graphs $G=(V_{G},E_{G})$, where $V_{G}$ represents the set of *vertices* and $E_{G}\subseteq V_{G}\times V_{G}$ forms the *edge* set, a symmetric relation. For any vertex $v\in V_{G}$, its set of *neighbors* is given by $N_{G}(v):=\{u\in V_{G}\mid(v,u)\in E_{G}\}$. A *homomorphism* from a graph $G$ to another graph $H$ is a mapping $h:V_{G}\to V_{H}$ such that each edge $(v,w)\in E_{G}$ is mapped to an edge $\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.0999
8pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}(h(v),h(w)
\mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.099
98pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})\in E_{H}$. An *isomorphism*, on the other hand, is a bijective function $f:V_{G}\to V_{H}$ that preserves adjacency: $(v,w)\in E_{G}$ if and only if $\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.0999
8pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}(f(v),f(w)
\mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.099
98pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})\in E_{H}$. The notation $\mathsf{Hom}(G,H)$ refers to the *number of homomorphisms* from $G$ to $H$, and the function $\mathsf{Hom}_{G}(\cdot)$ maps any graph $H$ to $\mathsf{Hom}(G,H)$. Given a sequence $\mathcal{F}=(F_{1},F_{2},\ldots)$ of graphs, we define $\mathsf{Hom}_{\mathcal{F}}(\cdot)$ as $\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.0999
8pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}(\mathsf{
Hom}_{F_{1}}(\cdot),\mathsf{Hom}_{F_{2}}(\cdot),\ldots\mathclose{{\hbox{$\left
missing
\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{
\vbox to4.04999pt{}}\right.\n@space$}}})$, a tuple of homomorphism counts.
A *graph invariant* is any function $\xi$ on graphs that is unchanged under isomorphisms, i.e., $\xi(G)=\xi(H)$ when $G$ and $H$ are isomorphic. For instance, $\mathsf{Hom}_{\mathcal{F}}(\cdot)$ serves as a graph invariant for any graph sequence $\mathcal{F}$.
Moreover, we introduce the concept of *rooted graphs*, where each graph $G^{r}$ has a distinguished root vertex $r\in V_{G}$. For two rooted graphs $G^{r}$ and $H^{s}$, a homomorphism must also map the root $r$ of $G$ to the root $s$ of $H$. The notation $\mathsf{Hom}_{F^{r}}(\cdot)$ captures the number of homomorphisms $\mathsf{Hom}(F^{r},G^{v})$ from a rooted graph $F^{r}$ to any rooted pair $(G,v)$, where $v$ is treated as the root of $G$. Similarly, $\mathsf{Hom}_{\mathcal{F}^{r}}(\cdot)$ is defined, capturing important *vertex invariants* for pairs $(G,v)$.

#### Graph neural networks and WL.

We extend these notions to *featured graphs* $G=(V_{G},E_{G},\zeta_{G})$, where each vertex is endowed with a feature vector $\zeta_{G}:V_{G}\to\mathbb{R}^{d_{0}}$ of some fixed dimension $d_{0}\in\mathbb{N}$. We focus on Message-Passing Neural Networks (MPNNs) (Gilmer et al., ), enhanced with homomorphism counts from $\mathcal{F}$ (Barceló et al., ). For a sequence of rooted graphs $\mathcal{F}=(F_{1}^{r},F_{2}^{r},\ldots)$, the initial vertex representation for a vertex $v\in V_{G}$ in an $\mathcal{F}$-MPNN is:(^1^11We ignore vertex features when considering homomorphisms.)

$$ $\phi_{\mathcal{F}}^{(0)}(G,v):=\left(\zeta_{G}(v),\mathsf{Hom}(F_{1}^{r},G^{v} ),\mathsf{Hom}(F_{2}^{r},G^{v}),\ldots\right).$ $$

At each iteration (*layer*) $0\leq\ell\leq L$, this representation is updated as follows:

$$ $\phi_{\mathcal{F}}^{(\ell+1)}(G,v):=\mathsf{upd}^{(\ell)}\Bigl{(}\phi_{ \mathcal{F}}^{(\ell)}(G,v),\mathsf{agg}^{(\ell)}\bigl{(}\{\!\!\{\phi_{\mathcal {F}}^{(\ell)}(G,u)\mid u\in N_{G}(v)\}\!\!\}\bigr{)}\Bigr{)},$ $$

where $\{\!\!\{\cdot\}\!\!\}$ denotes a multiset, and $\mathsf{upd}^{(\ell)}$ and $\mathsf{agg}^{(\ell)}$ are differentiable *update* and *aggregation* functions, respectively. After $L$ iterations, a final pooling operation produces the graph-level representation:
$\phi_{\mathcal{F}}^{L}(G):=\mathsf{readout}\bigl{(}\{\!\!\{\phi_{\mathcal{F}}^
{(L)}(G,v)\mid v\in V_{G}\}\!\!\}\bigr{)}$,
with $\mathsf{readout}$ being a differentiable function. This construction defines a graph invariant.
We also consider the $\mathcal{F}\text{-}\text{WL}$ algorithm, as introduced by Barceló et al. () as an extension of the one-dimensional Weisfeiler-Leman algorithm.
The $\mathcal{F}\text{-}\text{WL}$ algorithm iteratively updates vertex colors. Initially, each vertex is assigned a color:

$$ $\mathsf{wl}_{\mathcal{F}}^{(0)}(G,v):=\mathopen{{\hbox{$\left missing \mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{ \vbox to4.04999pt{}}\right.\n@space$}}}(\zeta_{G}(v),\mathsf{Hom}(F_{1},G^{v}) ,\mathsf{Hom}(F_{2},G^{v}),\ldots\mathclose{{\hbox{$\left missing\mathchoice{ \vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999 pt{}}\right.\n@space$}}}).$ $$

At each iteration $0\leq\ell\leq L$, new colors are assigned as follows:

$$ $\mathsf{wl}_{\mathcal{F}}^{(\ell+1)}(G,v):=\bigl{(}\mathsf{wl}_{\mathcal{F}}^{ (\ell)}(G,v),\{\!\!\{\mathsf{wl}_{\mathcal{F}}^{(\ell)}(G,u)\mid u\in N_{G}(v) \}\!\!\}\bigr{)}.$ $$

The final graph invariant is $\mathsf{wl}_{\mathcal{F}}^{(L)}(G):=\{\!\!\{\mathsf{wl}_{\mathcal{F}}^{(L)}(G,
v)\mid v\in V_{G}\}\!\!\}$. This invariant can be viewed as a *color histogram* in $\mathbb{N}^{c}$, where $c$ is the number of distinct colors, assuming a canonical ordering on colors. When the list $\mathcal{F}$ is empty we recover the $\mathsf{1}\text{-}\text{WL}$ algorithm (Weisfeiler and Leman, ).

#### Graph encoders.

*Graph encoders* are mappings $\phi$ from the set $\mathcal{G}$ of graphs to some *embedding space* $\mathcal{Z}$, typically residing in $\mathbb{R}^{k}$, for $k\in\mathbb{N}$. The space $\mathcal{Z}$ is assumed to be a *metric space* for a metric $d_{\mathcal{Z}}$. Examples of graph encoders are $\mathsf{Hom}_{\mathcal{F}}$, $\mathcal{F}$-MPNNs and $\mathcal{F}\text{-}\text{WL}$, for any sequence $\mathcal{F}$ of graphs and number $L\in\mathbb{N}$ of iterations. We will develop bounds for general graph encoders.

#### Wasserstein distance.

Let $\|\!\cdot\!\|$ denote the Euclidean norm in $\mathbb{R}^{d}$, for some $d\in\mathbb{N}$. Given two distributions $\mu$ and $\nu$ on $\mathbb{R}^{d}$, the *p-Wasserstein distance* between $\mu$ and $\nu$ is defined as:

$$ $\mathcal{W}_{p}(\mu,\nu):=\inf_{\pi\in\Pi(\mu,\nu)}\left(\mathbb{E}_{(x,y)\sim \pi}\|x-y\|^{p}\right)^{1/p},$ $$

where $\Pi(\mu,\nu)$ denotes the set of all couplings of $\mu$ and $\nu$, i.e., distributions $\pi$ on $\mathbb{R}^{d}\times\mathbb{R}^{d}$ with $\mu$ and $\nu$ as marginals. In what follows, we restrict our attention to the 1-Wasserstein distance.

## 4 Graph Encoders: Key Properties

Before presenting our generalization gap bounds, we first establish crucial properties of (classes of) graph encoders that play a significant role in our analysis. In particular, we revisit the relationship between classes of graph encoders in terms of their *distinguishing power*, i.e., their ability to map distinct graphs in $\mathcal{G}$ to distinct embeddings in their embedding spaces.

###### Definition 4.1 .

Let $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ and $\phi^{\prime}:\mathcal{G}\to\mathcal{Z}_{\phi^{\prime}}$ be two graph encoders. We say that $\phi$ *bounds $\phi^{\prime}$ in distinguishing power*, denoted by $\phi\sqsubseteq\phi^{\prime}$, if for any two graphs $G$ and $H$ in $\mathcal{G}$,

$$ $\phi^{\prime}(G)\neq\phi^{\prime}(H)\Rightarrow\phi(G)\neq\phi(H).$ $$

In other words, $\phi^{\prime}$ cannot distinguish more graphs than $\phi$.

Similarly, for classes $\Phi$ and $\Phi^{\prime}$ of graph encoders, we say that $\Phi$ *bounds $\Phi^{\prime}$ in distinguishing power*, denoted by $\Phi\sqsubseteq\Phi^{\prime}$, if no encoder in $\Phi^{\prime}$ can distinguish more graphs than any of the encoders in $\Phi$. That is, for all $\phi^{\prime}\in\Phi^{\prime}$, there exists a $\phi\in\Phi$ such that $\phi\sqsubseteq\phi^{\prime}$. If both $\Phi\sqsubseteq\Phi^{\prime}$ and $\Phi^{\prime}\sqsubseteq\Phi$ hold, then we write $\Phi\equiv\Phi^{\prime}$ and say that both classes have the same distinguishing power.

From the seminal papers by Morris et al. () and Xu et al. (), we know that
$\text{MPNN}(L)\equiv\mathsf{1}\text{-}\text{WL}(L)$, where the argument $L$ refers to the number of layers/iterations. Similarly, $\mathcal{F}$-$\text{MPNN}(L)\equiv\mathcal{F}\text{-}\text{WL}(L)$ (Barceló et al., ). It is also known that $\mathsf{Hom}_{\mathcal{T}}\sqsubseteq\text{MPNN}$ where $\mathcal{T}$ consists of all trees (Dell et al., ), and $\mathsf{Hom}_{\mathcal{T}\circ\mathcal{F}}\sqsubseteq\mathcal{F}\text{-}\text{
MPNN}(L)$ where $\mathcal{T}\circ\mathcal{F}$ consists of trees joined with copies of graphs in $\mathcal{F}$ (Barceló et al., ). Recent work by Neuen () provides valuable insights
comparing $\mathsf{Hom}_{\mathcal{F}}$ for various $\mathcal{F}$ (see also (Lanzinger and Barcelo, )).

When graph encoders are comparable in terms of distinguishing power, one can recover the least expressive encoder from the most expressive one. This is formalized in the following lemma. Proofs in this section can be found in Appendix B.

###### Lemma 4.2 .

Let $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ and $\phi^{\prime}:\mathcal{G}\to\mathcal{Z}_{\phi^{\prime}}$ be two graph encoders such that $\phi\sqsubseteq\phi^{\prime}$ holds. Then there exists a function $f:\mathcal{Z}_{\phi}\to\mathcal{Z}_{\phi^{\prime}}$ such that $\phi^{\prime}=f\circ\phi$.

As an illustration, consider an $L$-layer MPNN $\mathsf{M}$; we know that $\mathsf{1}\text{-}\text{WL}(L)\sqsubseteq\mathsf{M}$. It now suffices to define $f$ such that it maps a color histogram $\mathbf{h}$ to $\mathsf{M}(G)$, the embedding of $G$ by $\mathsf{M}$, where $G$ is a graph satisfying $\mathsf{wl}^{(L)}(G)=\mathbf{h}$. This is well-defined due to the earlier observation that $\mathsf{1}\text{-}\text{WL}(L)\sqsubseteq\mathsf{M}$.

For some classes of graph encoders, the function $f$ satisfies additional desirable properties, as we explain next.
We say that a graph encoder $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ is *$B$-bounded* if $d_{\mathcal{Z}_{\phi}}(\phi(G),\phi(H))\leq B$ for any $G,H\in\mathcal{G}$. Furthermore, a graph encoder $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ is *$S$-separating* if $d_{\mathcal{Z}_{\phi}}(\phi(G),\phi(H))\geq S$ for any $G,H\in\mathcal{G}$ such that $\phi(G)\neq\phi(H)$. We recall that a function $f$ between metric spaces $\mathcal{Z}$ and $\mathcal{Z}^{\prime}$ is *Lipschitz* with constant $\operatorname{Lip}(f)$ if for any $z_{1},z_{2}\in\mathcal{Z}$, $d_{\mathcal{Z}^{\prime}}(f(z_{1}),f(z_{2}))\leq\operatorname{Lip}(f)d_{
\mathcal{Z}}(z_{1},z_{2})$. For simplicity, we
set $\operatorname{Lip}(f)=\infty$ when $f$ is not Lipschitz for a finite constant.

###### Proposition 4.3 .

Let $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ be an $S$-separating graph encoder and $\phi^{\prime}:\mathcal{G}\to\mathcal{Z}_{\phi^{\prime}}$ be a $B$-bounded graph encoder such that $\phi\sqsubseteq\phi^{\prime}$. Then $\phi=f\circ\phi^{\prime}$ for a function $f:\mathcal{Z}_{\phi}\to\mathcal{Z}_{\phi^{\prime}}$ which is Lipschitz with constant $\operatorname{Lip}(f)=B/S$.

There are plenty of bounded graph encoders; indeed, just consider any GNN employing bounded-range activation functions such as
sigmoid, tanh, truncated ReLU (Hamilton et al., ). Other examples include normalized homomorphism count vectors or color histograms (Lovász and Szegedy, ). Similarly, any graph encoder mapping graphs into a discrete subset of $\mathbb{R}^{d}$ is $S$-separating. For example, any $\mathsf{Hom}_{\mathcal{F}}$ is $1$-separating since whenever $\mathsf{Hom}_{\mathcal{F}}(G)\neq\mathsf{Hom}_{\mathcal{F}}(H)$, there exists an $F\in\mathcal{F}$ such that $\mathsf{Hom}(F,G)\neq\mathsf{Hom}(F,H)$. Since the latter are natural numbers, and assuming a discrete metric $d$, $d\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.099
98pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}(\mathsf{
Hom}(F,G),\mathsf{Hom}(F,H)\mathclose{{\hbox{$\left missing\mathchoice{\vbox to
8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}
\right.\n@space$}}})\geq 1$. A similar argument applies to graph encoders based on $\mathsf{1}\text{-}\text{WL}$ or its higher-order variant $k\text{-}\text{WL}$.

Our generalization bounds use the 1-Wasserstein distance between distributions, as we will see shortly. Using Proposition 4.3, and in particular the Lipschitz property, we can relate the Wasserstein distance between the pushforward distributions of distributions $\mu$ and $\nu$ on $\mathcal{G}$ for the embedding spaces of the graph encoders. Formally, let $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ be a graph encoder and let $\mu$ be a distribution on $\mathcal{G}$. Then the *pushforward distribution of $\mu$ under $\phi$* is the distribution on $\mathcal{Z}_{\phi}$ given by

$$ $\phi_{\sharp}(\mu)(z):=\mu\bigl{(}\{G\in\mathcal{G}\mid\phi(G)=z\}\bigr{)},$ $$

where $z$ is an element in the embedding space $\mathcal{Z}_{\phi}$. We can now state the proposition.

###### Proposition 4.4 .

Let $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ and $\phi^{\prime}:\mathcal{G}\to\mathcal{Z}_{\phi^{\prime}}$ be two graph encoders such that $\phi^{\prime}=f\circ\phi$.
Then for any distributions $\nu$ and $\nu^{\prime}$ over $\mathcal{G}$, we have that the inequality
$\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}
{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space
$}}}(\phi_{\sharp}^{\prime}(\nu),\phi_{\sharp}^{\prime}(\nu^{\prime})
\mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.099
98pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})\leq
\operatorname{Lip}(f)\cdot\mathcal{W}_{1}\mathopen{{\hbox{$\left missing
\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{
\vbox to4.04999pt{}}\right.\n@space$}}}(\phi_{\sharp}(\nu),\phi_{\sharp}(\nu^{
\prime})\mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox
to
8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})$ holds.

We remark that the inequality above becomes vacuous when $f$ is not Lipschitz and hence $\operatorname{Lip}(f)=\infty$. As an important corollary of Propositions 4.3 and 4.4, we obtain the following.

###### Corollary 4.5 .

Let $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ be an $S$-separating graph encoder and $\phi^{\prime}:\mathcal{G}\to\mathcal{Z}_{\phi^{\prime}}$ a $B$-bounded graph encoder such that $\phi\sqsubseteq\phi^{\prime}$ holds. Then for any distributions $\nu$ and $\nu^{\prime}$ over $\mathcal{G}$, we have

$$ $\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}} {\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space $}}}(\phi_{\sharp}^{\prime}(\nu),\phi_{\sharp}^{\prime}(\nu^{\prime}) \mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.099 98pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})\leq(B/S) \cdot\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998 pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right. \n@space$}}}(\phi_{\sharp}(\nu),\phi_{\sharp}(\nu^{\prime})\mathclose{{\hbox{$ \left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.6 6995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}).$ $$

As an example, consider a $B$-bounded graph encoder $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ which is bounded in distinguishing power by the $1$-separating encoder $\mathsf{Hom}_{\mathcal{F}}$, for some sequence $\mathcal{F}$ of graphs. Then for any distributions $\nu$ and $\nu^{\prime}$ on $\mathcal{G}$, we have

$$ $\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}} {\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space $}}}(\phi_{\sharp}(\nu),\phi_{\sharp}(\nu^{\prime})\mathclose{{\hbox{$\left missing \mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{ \vbox to4.04999pt{}}\right.\n@space$}}})\leq(B/1)\cdot\mathcal{W}_{1}\mathopen {{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{ \vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}((\mathsf{Hom}_{ \mathcal{F}})_{\sharp}(\nu),(\mathsf{Hom}_{\mathcal{F}})_{\sharp}(\nu^{\prime} )\mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09 998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}).$ $$

More broadly, these results suggest that the variance of embedding distributions in $\mathcal{Z}_{\phi}$, produced by a complex graph encoder, can be effectively upper bounded by the variance of simpler, combinatorial graph invariants—such as homomorphism counts, Weisfeiler-Leman tests, and other structural descriptors, provided that the latter bound the former in terms of distinguishing power.

###### Definition 4.1 .

###### Lemma 4.2 .

###### Proposition 4.3 .

###### Proposition 4.4 .

###### Corollary 4.5 .

## 5 Generalization Analysis

We use the setup from Chuang et al. () but
translated to the graph setting.
More precisely, let $\mathcal{G}$ represent the input space of graphs, $\mathcal{Z}$ the embedding space in $\mathbb{R}^{d}$ for some $d\in\mathbb{N}$, and $\mathcal{Y}=\{1,\dots,K\}$ the output space consisting of $K$ classes. We define a set of *graph encoders* $\Phi=\{\phi:\mathcal{G}\to\mathcal{Z}\}$ and a set of *predictors* $\Psi=\{\psi=(\psi_{1},\ldots,\psi_{K}):\mathcal{Z}\to\mathbb{R}^{K}\}$. A *score-based graph classifier* $\psi\circ\phi$ simply returns $\arg\max_{y\in\mathcal{Y}}\psi_{y}(\phi(G))$ on input $G$. The graph encoders in $\Phi$ are assumed to be graph invariants, such as, e.g., $\mathcal{F}$-MPNNs, $\mathcal{F}\text{-}\text{WL}$, or $\mathsf{Hom}_{\mathcal{F}}$.

We define the *margin* of a graph classifier $\psi\circ\phi$ for a graph sample $(G,y)\in\mathcal{G}\times\mathcal{Y}$ as

$$ $\rho_{\psi}(\phi(G),y):=\psi_{y}(\phi(G))-\max_{y^{\prime}\neq y}\psi_{y^{ \prime}}(\phi(G)).$ $$

The graph classifier $\psi\circ\phi$ misclassifies $G$ if $\rho_{\psi}(\phi(G),y)<0$. Let $\mu$ be a distribution over $\mathcal{G}\times\mathcal{Y}$, and $\mathcal{S}=\{(G_{i},y_{i})\}_{i=1}^{m}$ be a set of $m$ graph samples drawn i.i.d. from $\mu$, i.e., $\mathcal{S}\sim\mu^{m}$. The *empirical distribution* $\mu_{\mathcal{S}}$ is defined as $\mu_{\mathcal{S}}:=\frac{1}{m}\sum_{i=1}^{m}\delta_{(G_{i},y_{i})}$, where $\delta_{(G_{i},y_{i})}$ denotes the Dirac delta measure centered at $(G_{i},y_{i})$.
The *expected zero-one loss* $R_{\mu}(\psi\circ\phi)$ and the *$\gamma$-margin empirical zero-one loss* $\hat{R}_{\gamma,\mathcal{S}}(\psi\circ\phi)$ are defined as

$$ $R_{\mu}(\psi\circ\phi):=\mathbb{E}_{(G,y)\sim\mu}\left[\mathbbm{1}_{\rho_{\psi }(\phi(G),y)\leq 0}\right]\text{ and }\hat{R}_{\gamma,\mathcal{S}}(\psi\circ \phi):=\mathbb{E}_{(G,y)\sim\mu_{\mathcal{S}}}\left[\mathbbm{1}_{\rho_{\psi}( \phi(G),y)\leq\gamma}\right].$ $$

We aim to bound the *generalisation gap* $R_{\mu}(\psi\circ\phi)-\hat{R}_{\gamma,\mathcal{S}}(\psi\circ\phi)$ for the graph classifier $\psi\circ\phi$.

### 5.1 Generalization bound

We are now ready to present the generalization bounds. Our results build on the margin bounds of Chuang et al. (), which are themselves based on a generalized notion of variance that involves the Wasserstein distance (Solomon et al., ). This notion more effectively captures the structural properties of the feature distribution. Crucially, we fully exploit the properties of graph encoders and, in particular, use Proposition 4.4 to derive an upper bound on the generalization gap of any graph encoder $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$
in terms of *any* graph encoder bounding $\phi$ in distinguishing power!

In order to formally state our results, some additional definitions are needed. Recall that we consider graph classifiers $(\psi_{1},\ldots,\psi_{K})\circ\phi$ where $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ is a graph encoder and the predictor $\psi=(\psi_{1},\ldots,\psi_{K})$ is such that each $\psi_{i}:\mathcal{Z}_{\phi}\to\mathbb{R}$. Recall also that the output space $\mathcal{Y}=\{1,\ldots,K\}$ and that $\mu$ is a distribution over $\mathcal{G}\times\mathcal{Y}$. We denote by $\mu_{x}$ the marginal distribution on $\mathcal{G}$, i.e., $\mu_{x}(G):=\int\mu(G,y)\text{d}y$ and by $\mu_{y}$ the marginal distribution on $\mathcal{Y}$, i.e, $\mu_{y}(c):=\int\mu(x,c)\text{d}x$. Then,
for each $c\in\mathcal{Y}$, $\mu_{c}(G)$ is the conditional distribution on $\mathcal{G}$ defined by $\mu(G,c)/\mu_{y}(c)$.

###### Theorem 5.1 .

Fix $\gamma>0$ and a graph encoder $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$. Let $\lambda:\mathcal{G}\to\mathcal{Z}_{\lambda}$ be a graph encoder that bounds $\phi$ in distinguishing power, i.e., $\lambda\sqsubseteq\phi$. Then, for every distribution $\mu$ on $\mathcal{G}\times\mathcal{Y}$, for every predictor $\psi=(\psi_{y})_{i\in\mathcal{Y}}$ and every $\delta\in(0,1)$, with probability at least $1-\delta$ over all choices of $\mathcal{S}\sim\mu^{m}$, we have that the generalization gap $R_{\mu}(\psi\circ\phi)-\hat{R}_{\gamma,\mathcal{S}}(\psi\circ\phi)$ is upper bounded by

$$ $\mathbb{E}_{c\sim\mu_{y}}\left[\frac{\operatorname{Lip}\left(\rho_{\psi}(\cdot ,c)\right)\operatorname{Lip}(f)}{\gamma}\mathbb{E}_{T,\tilde{T}\sim\mu_{c}^{m_ {c}}}\left[\mathcal{W}_{1}\bigl{(}\lambda_{\sharp}(\mu_{c,T}),\lambda_{\sharp} (\mu_{c,\tilde{T}})\bigr{)}\right]\right]+\sqrt{\frac{\log(1/\delta)}{2m}},$ ( $\dagger$ ) $$

where $\phi=f\circ\lambda$ and for each $c\in\mathcal{Y}$, $m_{c}$ denotes the number of pairs $(G,c)$ in $\mathcal{S}$. Also, recall that for $T\sim\mu_{c}^{m_{c}}$,
$\mu_{c,T}$ is the empirical distribution $\mu_{c,T}:=\sum_{G\in T}\delta_{G}$; similarly for $\mu_{c,\tilde{T}}$.

The proof is a consequence of Theorem 2 in Chuang et al. () and Proposition 4.4. As also observed by those authors, the expectation term over $T,\tilde{T}\sim\mu_{c}^{m_{c}}$ is intractable in general. To address this drawback, Chuang et al. () show how to estimate the expectation by means of *sampling*, provided that encoders are $B$-bounded. A similar approach works in our case as well. We refer to Appendix C for proofs and details.

Theorem 5.1 highlights several key factors that influence the generalization of graph classifiers: (i) the learning behavior of the predictors $\psi$, captured by $\operatorname{Lip}\left(\rho_{\psi}(\cdot,c)\right)$; (ii) the learning behavior of graph encoder $\phi$, relative to $\lambda$, described by $\operatorname{Lip}(f)$; and (iii) the variance of graph structures, in the Wasserstein distance $\mathcal{W}_{1}\bigl{(}\lambda_{\sharp}(\mu_{c,T}),\lambda_{\sharp}(\mu_{c,
\tilde{T}})\bigr{)}$ for graph samples $T,\tilde{T}\sim\mu_{c}^{m_{c}}$.

###### Theorem 5.1 .

### 5.2 Concentration and separation

In terms of concentration, since $\mathbb{E}_{T,\tilde{T}\sim\mu_{c}^{m_{c}}}[W_{1}(\lambda_{\sharp}(\mu_{c,T}),
\lambda_{\sharp}(\mu_{c,\tilde{T}}))]\leq O(m^{-1/d})$ (Chuang et al., ), a large sample size $m$ and a small dimension $d$ of the embedding space $\mathcal{Z}_{\lambda}$ lead to a smaller generalization bound. For instance, when $\mu$ is concentrated on graphs in $\mathcal{G}$ with low *color complexity* (Morris et al., )—i.e., the $\mathsf{1}\text{-}\text{WL}$ test requires only a small number of colors for the graph’s vertices—combinatorial graph encoders like $\mathsf{Hom}_{\mathcal{F}}$ and $\mathcal{F}\text{-}\text{WL}(L)$ can operate in low-dimensional spaces. This observation is consistent with earlier findings (Kiefer and McKay, ; Garg et al., ; Liao et al., ; Ju et al., ; Cong et al., ; Esser et al., ; Morris et al., ) about the effect of graph size, degree, and maximum degree on generalization performance.

Of particular interest is the case when the bounding graph classifier $\lambda$ is assumed to have a large margin. A larger margin is generally associated with better generalization (Elsayed et al., ; Chuang et al., ). If we assume the margin $\gamma$ is satisfied for $\psi\circ\lambda$, for all graph samples, and for each $c\in\mathcal{Y}$, the predictor $\psi_{c}\in\psi$ is Lipschitz, then (see Lemma 10 in Chuang et al. ()) we have

$$ $\gamma\leq\bigl{(}\max_{\begin{subarray}{c}c,c^{\prime}\in\mathcal{Y}\\ c\neq c^{\prime}\end{subarray}}W_{1}(\lambda_{\sharp}(\mu_{c}),\lambda_{\sharp }(\mu_{c^{\prime}}))\bigr{)}\bigl{(}\max_{c\in\mathcal{Y}}\mathsf{Lip}(\psi_{c })\bigr{)}.$ $$

which allows us to lower bound $1/\gamma$ in Equation $\dagger$, resulting in Proposition 5.2, hereby revealing a trade-off between concentration and separation.

###### Proposition 5.2 .

Under the same assumptions as in Theorem 5.1, but with the additional requirement that the predictors $\psi_{c}$ in $\psi$ are Lipschitz, and that the bounding graph classifier $\lambda$ has a large margin, i.e., $\rho_{\psi}(\lambda(G),y)\geq\gamma$ for all $(G,y)\sim\mu$, then for any $\delta\in(0,1)$, with probability at least $1-\delta$ over all choices $\mathcal{S}\sim\mu^{m}$, we have that the gap $R_{\mu}(\psi\circ\phi)-\hat{R}_{\gamma,\mathcal{S}}(\psi\circ\phi)$ is upper bounded by

$$ $\frac{\mathsf{Lip}(f)\cdot\mathbb{E}_{c\sim\mu_{y}}\left[\mathsf{Lip}(\rho_{ \psi}(\cdot,c))\mathbb{E}_{T,\tilde{T}\sim\mu_{c}^{m_{c}}}\left[W_{1}\left( \lambda_{\sharp}(\mu_{c,T}),\lambda_{\sharp}(\mu_{c,\tilde{T}})\right)\right] \right]}{\left(\max_{c\in\mathcal{Y}}\mathsf{Lip}(\psi_{c})\right)\left(\max_{ c,c^{\prime}\in\mathcal{Y},c\neq c^{\prime}}W_{1}\left(\lambda_{\sharp}(\mu_{c }),\lambda_{\sharp}(\mu_{c^{\prime}})\right)\right)}+\sqrt{\frac{\log(1/\delta )}{2m}}.$ $$

Furthermore, this bound is no worse than the one given in Theorem 5.1.

The above proposition highlights that, to achieve a low generalization bound, it is crucial to ensure good concentration between embeddings of the same class, i.e., $W_{1}(\lambda_{\sharp}(\mu_{c,T}),\lambda_{\sharp}(\mu_{c,\tilde{T}}))$, while maintaining a large separation between embeddings of different classes, i.e., $W_{1}(\lambda_{\sharp}(\mu_{c}),\lambda_{\sharp}(\mu_{c^{\prime}}))$, where $c\neq c^{\prime}$. This can be achieved when $\lambda$ learn embeddings in the “right” directions, where embeddings of different classes are “more separated” than those of the same class, or when the distribution $\mu$ is concentrated on graphs for which this separation happens for $\lambda$.

###### Proposition 5.2 .

#### Remarks.

In our bounds, we identify three Lipschitz constants: $\mathsf{Lip}(\rho_{\psi}(\cdot,c))$, $\mathsf{Lip}(\psi_{c})$, and $\mathsf{Lip}(f)$.
First, note that $\rho_{\psi}(\cdot,c)$ depends on $\psi_{c}$, and therefore it is Lipschitz in its first argument if $\psi_{c}$ is Lipschitz. For simplicity, we assume that the predictor $\psi=(\psi_{c})_{c\in\mathcal{Y}}$ is a softmax function with Lipschitz constant $1$.
For general $\psi_{c}$, $\mathsf{Lip}(\rho_{\psi}(\cdot,c))$ can be approximated empirically using the Jacobian, as suggested by (Chuang et al., ).

Furthermore, Corollary 4.5 states that the connecting function $f$ between the graph encoders $\lambda\sqsubseteq\phi$ is Lipschitz with constant $B/S$, provided that $\phi$ is $B$-bounded and $\lambda$ is separating. Therefore, when $\phi$ is $B$-bounded, $\mathsf{Lip}(f)$ decreases as $S$ increases. We also note that $S$ can increase with added expressivity in $\lambda$, which enhances its separation ability. In practice, both $B$ and $S$ can be computed empirically. We discuss the effect of added expressivity in $\lambda$ in more detail in the next section.

**Table 1: Two graphs with the initial vertex colors, including the input vertex features and homomorphism counts, the vertex colors after one iteration, and the dimension-wise difference and Wasserstein distance between graph embeddings for three models: (a) $\mathsf{1}\text{-}\text{WL}$, (b) $C_{4}\text{-}\text{WL}$, and (c) $K_{4}\text{-}\text{WL}$.**
| Initial Vertex Colors | After One Iteration | Graph Embeddings | Wasserstein |  |
| --- | --- | --- | --- | --- |
|  | $G$             $G^{\prime}$ | $G$             $G^{\prime}$ | (Difference) | Distance |
| (a)<br>$\mathsf{1}\text{-}\text{WL}$ |  |  |  | 4.796 |
| (b)<br>$C_{4}\text{-}\text{WL}$ |  |  |  | 4.123 |
| (c)<br>$K_{4}\text{-}\text{WL}$ |  |  |  | 5.000 |

## 6 Case Studies

In this section, we present case studies to illustrate how model expressivity influences the generalization ability of graph encoders. Recall that two key factors are considered in Theorem 5.1 and Proposition 5.2: (i) *intra-class concentration*, which reflects the variance of graph structures, measured by the Wasserstein distance $\mathcal{W}_{1}(\lambda_{\sharp}(\mu_{c,T}),\lambda_{\sharp}(\mu_{c,\tilde{T}}))$ for graph samples $T,\tilde{T}\sim\mu_{c}^{m_{c}}$, and (ii) *inter-class separation*, measured by the Wasserstein distance $\max_{c,c^{\prime}\in\mathcal{Y},c\neq c^{\prime}}\mathcal{W}_{1}(\lambda_{
\sharp}(\mu_{c}),\lambda_{\sharp}(\mu_{c^{\prime}}))$. For simplicity, we consider the following graphs $\{G,G^{\prime},H,H^{\prime}\}$ from the PROTEINS dataset (Morris et al., ), uniformly selected by the distribution $\mu$,

$$ $G=\raisebox{-0.5pt}{\leavevmode\resizebox{24.18501pt}{}{ \leavevmode\hbox to 197.09pt{\vbox to261.92pt{\pgfpicture\makeatletter\hbox{\hskip 103.92993pt \lower-157.98999pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ } \definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0} \pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ } \pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.8pt}\pgfsys@invoke{ } \par{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3 .0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 17.07166pt}{0.0pt}\pgfsys@curveto{17.07166pt}{9.42853pt}{9.42853pt}{17.07166pt }{0.0pt}{17.07166pt}\pgfsys@curveto{-9.42853pt}{17.07166pt}{-17.07166pt}{9.428 53pt}{-17.07166pt}{0.0pt}\pgfsys@curveto{-17.07166pt}{-9.42853pt}{-9.42853pt}{ -17.07166pt}{0.0pt}{-17.07166pt}\pgfsys@curveto{9.42853pt}{-17.07166pt}{17.071 66pt}{-9.42853pt}{17.07166pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{-3.22221pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{3}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{0.0pt}\pgfsys@curveto{-68.28662pt}{9.42853pt}{- 75.92975pt}{17.07166pt}{-85.35828pt}{17.07166pt}\pgfsys@curveto{-94.7868pt}{17 .07166pt}{-102.42993pt}{9.42853pt}{-102.42993pt}{0.0pt}\pgfsys@curveto{-102.42 993pt}{-9.42853pt}{-94.7868pt}{-17.07166pt}{-85.35828pt}{-17.07166pt} \pgfsys@curveto{-75.92975pt}{-17.07166pt}{-68.28662pt}{-9.42853pt}{-68.28662pt }{0.0pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{-3.22221 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{1}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{17.07166pt}{85.35828pt}\pgfsys@curveto{17.07166pt}{94.7868pt }{9.42853pt}{102.42993pt}{0.0pt}{102.42993pt}\pgfsys@curveto{-9.42853pt}{102.4 2993pt}{-17.07166pt}{94.7868pt}{-17.07166pt}{85.35828pt}\pgfsys@curveto{-17.07 166pt}{75.92975pt}{-9.42853pt}{68.28662pt}{0.0pt}{68.28662pt}\pgfsys@curveto{9 .42853pt}{68.28662pt}{17.07166pt}{75.92975pt}{17.07166pt}{85.35828pt} \pgfsys@closepath\pgfsys@moveto{0.0pt}{85.35828pt}\pgfsys@stroke\pgfsys@invoke { } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{82.13606pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{2}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{85.35828pt}\pgfsys@curveto{-68.28662pt}{94.7868 pt}{-75.92975pt}{102.42993pt}{-85.35828pt}{102.42993pt}\pgfsys@curveto{-94.786 8pt}{102.42993pt}{-102.42993pt}{94.7868pt}{-102.42993pt}{85.35828pt} \pgfsys@curveto{-102.42993pt}{75.92975pt}{-94.7868pt}{68.28662pt}{-85.35828pt} {68.28662pt}\pgfsys@curveto{-75.92975pt}{68.28662pt}{-68.28662pt}{75.92975pt}{ -68.28662pt}{85.35828pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{85.35828 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{82.13606 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{4}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -72.22612pt}{13.13216pt}\pgfsys@lineto{-13.13216pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -66.78662pt}{0.0pt}\pgfsys@lineto{-18.57166pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -85.35828pt}{18.57166pt}\pgfsys@lineto{-85.35828pt}{66.78662pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 0.0pt}{66.78662pt}\pgfsys@lineto{0.0pt}{18.57166pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -18.57166pt}{85.35828pt}\pgfsys@lineto{-66.78662pt}{85.35828pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -13.13216pt}{13.13216pt}\pgfsys@lineto{-72.22612pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{{{}}}}{{}}{{}}\hbox{ \hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt} \pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -23.36897pt}{-77.42929pt}\pgfsys@curveto{-23.36897pt}{-68.00076pt}{-31.0121pt} {-60.35764pt}{-40.44063pt}{-60.35764pt}\pgfsys@curveto{-49.86916pt}{-60.35764 pt}{-57.51228pt}{-68.00076pt}{-57.51228pt}{-77.42929pt}\pgfsys@curveto{-57.512 28pt}{-86.85782pt}{-49.86916pt}{-94.50095pt}{-40.44063pt}{-94.50095pt} \pgfsys@curveto{-31.0121pt}{-94.50095pt}{-23.36897pt}{-86.85782pt}{-23.36897pt }{-77.42929pt}\pgfsys@closepath\pgfsys@moveto{-40.44063pt}{-77.42929pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-42.94063pt}{-80.6515 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{5}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{91.6555pt}{-57.51202pt}\pgfsys@curveto{91.6555pt}{-48.0835pt }{84.01237pt}{-40.44037pt}{74.58385pt}{-40.44037pt}\pgfsys@curveto{65.15532pt} {-40.44037pt}{57.51219pt}{-48.0835pt}{57.51219pt}{-57.51202pt}\pgfsys@curveto{ 57.51219pt}{-66.94055pt}{65.15532pt}{-74.58368pt}{74.58385pt}{-74.58368pt} \pgfsys@curveto{84.01237pt}{-74.58368pt}{91.6555pt}{-66.94055pt}{91.6555pt}{-5 7.51202pt}\pgfsys@closepath\pgfsys@moveto{74.58385pt}{-57.51202pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{72.08385pt}{-60.73424 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{6}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{48.36986pt}{-139.41833pt}\pgfsys@curveto{48.36986pt}{-129.98 98pt}{40.72673pt}{-122.34668pt}{31.2982pt}{-122.34668pt}\pgfsys@curveto{21.869 67pt}{-122.34668pt}{14.22655pt}{-129.9898pt}{14.22655pt}{-139.41833pt} \pgfsys@curveto{14.22655pt}{-148.84686pt}{21.86967pt}{-156.48999pt}{31.2982pt} {-156.48999pt}\pgfsys@curveto{40.72673pt}{-156.48999pt}{48.36986pt}{-148.84686 pt}{48.36986pt}{-139.41833pt}\pgfsys@closepath\pgfsys@moveto{31.2982pt}{-139.4 1833pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{28.7982pt}{-142.64055 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{7}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -8.59778pt}{-16.4616pt}\pgfsys@lineto{-31.84285pt}{-60.9677pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 14.7069pt}{-11.34062pt}\pgfsys@lineto{59.87666pt}{-46.1714pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 4.06793pt}{-18.12079pt}\pgfsys@lineto{27.23027pt}{-121.29755pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -22.14131pt}{-74.26053pt}\pgfsys@lineto{56.28453pt}{-60.68079pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -26.38834pt}{-89.57187pt}\pgfsys@lineto{17.24591pt}{-127.27576pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 65.90645pt}{-73.93167pt}\pgfsys@lineto{39.9756pt}{-122.99869pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -76.0393pt}{-16.06429pt}\pgfsys@lineto{-49.7596pt}{-61.365pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss} \pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}} \lxSVG@closescope\endpgfpicture}}}}\hskip 28.45274ptG^{\prime}=\raisebox{-0.5 pt}{\leavevmode\resizebox{22.76228pt}{}{ \leavevmode\hbox to191.39pt{\vbox to 279.6pt{\pgfpicture\makeatletter\hbox{\hskip 103.92993pt\lower-175.66833pt \hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{ rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ } \pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt} \pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{0.8pt}\pgfsys@invoke{ } \par{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3 .0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 17.07166pt}{0.0pt}\pgfsys@curveto{17.07166pt}{9.42853pt}{9.42853pt}{17.07166pt }{0.0pt}{17.07166pt}\pgfsys@curveto{-9.42853pt}{17.07166pt}{-17.07166pt}{9.428 53pt}{-17.07166pt}{0.0pt}\pgfsys@curveto{-17.07166pt}{-9.42853pt}{-9.42853pt}{ -17.07166pt}{0.0pt}{-17.07166pt}\pgfsys@curveto{9.42853pt}{-17.07166pt}{17.071 66pt}{-9.42853pt}{17.07166pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{-3.22221pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{3}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{0.0pt}\pgfsys@curveto{-68.28662pt}{9.42853pt}{- 75.92975pt}{17.07166pt}{-85.35828pt}{17.07166pt}\pgfsys@curveto{-94.7868pt}{17 .07166pt}{-102.42993pt}{9.42853pt}{-102.42993pt}{0.0pt}\pgfsys@curveto{-102.42 993pt}{-9.42853pt}{-94.7868pt}{-17.07166pt}{-85.35828pt}{-17.07166pt} \pgfsys@curveto{-75.92975pt}{-17.07166pt}{-68.28662pt}{-9.42853pt}{-68.28662pt }{0.0pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{-3.22221 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{1}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{17.07166pt}{85.35828pt}\pgfsys@curveto{17.07166pt}{94.7868pt }{9.42853pt}{102.42993pt}{0.0pt}{102.42993pt}\pgfsys@curveto{-9.42853pt}{102.4 2993pt}{-17.07166pt}{94.7868pt}{-17.07166pt}{85.35828pt}\pgfsys@curveto{-17.07 166pt}{75.92975pt}{-9.42853pt}{68.28662pt}{0.0pt}{68.28662pt}\pgfsys@curveto{9 .42853pt}{68.28662pt}{17.07166pt}{75.92975pt}{17.07166pt}{85.35828pt} \pgfsys@closepath\pgfsys@moveto{0.0pt}{85.35828pt}\pgfsys@stroke\pgfsys@invoke { } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{82.13606pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{2}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{85.35828pt}\pgfsys@curveto{-68.28662pt}{94.7868 pt}{-75.92975pt}{102.42993pt}{-85.35828pt}{102.42993pt}\pgfsys@curveto{-94.786 8pt}{102.42993pt}{-102.42993pt}{94.7868pt}{-102.42993pt}{85.35828pt} \pgfsys@curveto{-102.42993pt}{75.92975pt}{-94.7868pt}{68.28662pt}{-85.35828pt} {68.28662pt}\pgfsys@curveto{-75.92975pt}{68.28662pt}{-68.28662pt}{75.92975pt}{ -68.28662pt}{85.35828pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{85.35828 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{82.13606 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{4}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -72.22612pt}{13.13216pt}\pgfsys@lineto{-13.13216pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -66.78662pt}{0.0pt}\pgfsys@lineto{-18.57166pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -85.35828pt}{18.57166pt}\pgfsys@lineto{-85.35828pt}{66.78662pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 0.0pt}{66.78662pt}\pgfsys@lineto{0.0pt}{18.57166pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -18.57166pt}{85.35828pt}\pgfsys@lineto{-66.78662pt}{85.35828pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -13.13216pt}{13.13216pt}\pgfsys@lineto{-72.22612pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{{{}}}}{{}}{{}}\hbox{ \hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt} \pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -23.36897pt}{-74.58385pt}\pgfsys@curveto{-23.36897pt}{-65.15532pt}{-31.0121pt} {-57.51219pt}{-40.44063pt}{-57.51219pt}\pgfsys@curveto{-49.86916pt}{-57.51219 pt}{-57.51228pt}{-65.15532pt}{-57.51228pt}{-74.58385pt}\pgfsys@curveto{-57.512 28pt}{-84.01237pt}{-49.86916pt}{-91.6555pt}{-40.44063pt}{-91.6555pt} \pgfsys@curveto{-31.0121pt}{-91.6555pt}{-23.36897pt}{-84.01237pt}{-23.36897pt} {-74.58385pt}\pgfsys@closepath\pgfsys@moveto{-40.44063pt}{-74.58385pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-42.94063pt}{-77.8060 6pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{5}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{77.42912pt}{-60.35747pt}\pgfsys@curveto{77.42912pt}{-50.9289 4pt}{69.786pt}{-43.28581pt}{60.35747pt}{-43.28581pt}\pgfsys@curveto{50.92894pt }{-43.28581pt}{43.28581pt}{-50.92894pt}{43.28581pt}{-60.35747pt} \pgfsys@curveto{43.28581pt}{-69.786pt}{50.92894pt}{-77.42912pt}{60.35747pt}{-7 7.42912pt}\pgfsys@curveto{69.786pt}{-77.42912pt}{77.42912pt}{-69.786pt}{77.429 12pt}{-60.35747pt}\pgfsys@closepath\pgfsys@moveto{60.35747pt}{-60.35747pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{57.85747pt}{-63.57968 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{6}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{85.96503pt}{-142.8703pt}\pgfsys@curveto{85.96503pt}{-133.441 77pt}{78.3219pt}{-125.79865pt}{68.89337pt}{-125.79865pt}\pgfsys@curveto{59.464 84pt}{-125.79865pt}{51.82172pt}{-133.44177pt}{51.82172pt}{-142.8703pt} \pgfsys@curveto{51.82172pt}{-152.29883pt}{59.46484pt}{-159.94196pt}{68.89337pt }{-159.94196pt}\pgfsys@curveto{78.3219pt}{-159.94196pt}{85.96503pt}{-152.29883 pt}{85.96503pt}{-142.8703pt}\pgfsys@closepath\pgfsys@moveto{68.89337pt}{-142.8 703pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{66.39337pt}{-146.0925 1pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{7}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-11.98805pt}{-157.09668pt}\pgfsys@curveto{-11.98805pt}{-147. 66815pt}{-19.63118pt}{-140.02502pt}{-29.05971pt}{-140.02502pt}\pgfsys@curveto{ -38.48824pt}{-140.02502pt}{-46.13136pt}{-147.66815pt}{-46.13136pt}{-157.09668 pt}\pgfsys@curveto{-46.13136pt}{-166.5252pt}{-38.48824pt}{-174.16833pt}{-29.05 971pt}{-174.16833pt}\pgfsys@curveto{-19.63118pt}{-174.16833pt}{-11.98805pt}{-1 66.5252pt}{-11.98805pt}{-157.09668pt}\pgfsys@closepath\pgfsys@moveto{-29.05971 pt}{-157.09668pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-31.55971pt}{-160.318 9pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{8}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -8.85225pt}{-16.32614pt}\pgfsys@lineto{-31.58838pt}{-58.2577pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 13.13216pt}{-13.13216pt}\pgfsys@lineto{47.22531pt}{-47.22531pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -22.0512pt}{-71.98837pt}\pgfsys@lineto{41.96803pt}{-62.95294pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -24.6889pt}{-84.42169pt}\pgfsys@lineto{53.14165pt}{-133.03217pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 62.26859pt}{-78.8305pt}\pgfsys@lineto{66.98225pt}{-124.39726pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -37.90324pt}{-92.98122pt}\pgfsys@lineto{-31.59738pt}{-138.69931pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 50.51442pt}{-145.53946pt}\pgfsys@lineto{-10.68076pt}{-154.42752pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -75.77689pt}{-15.90929pt}\pgfsys@lineto{-50.02202pt}{-58.67456pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss} \pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}} \lxSVG@closescope\endpgfpicture}}}}\hskip 28.45274ptH=\raisebox{-0.5pt}{ \leavevmode\resizebox{24.18501pt}{}{ \leavevmode\hbox to197.09pt{\vbox to 261.92pt{\pgfpicture\makeatletter\hbox{\hskip 103.92993pt\lower-157.98999pt \hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{ rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ } \pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt} \pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{0.8pt}\pgfsys@invoke{ } \par{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3 .0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 17.07166pt}{0.0pt}\pgfsys@curveto{17.07166pt}{9.42853pt}{9.42853pt}{17.07166pt }{0.0pt}{17.07166pt}\pgfsys@curveto{-9.42853pt}{17.07166pt}{-17.07166pt}{9.428 53pt}{-17.07166pt}{0.0pt}\pgfsys@curveto{-17.07166pt}{-9.42853pt}{-9.42853pt}{ -17.07166pt}{0.0pt}{-17.07166pt}\pgfsys@curveto{9.42853pt}{-17.07166pt}{17.071 66pt}{-9.42853pt}{17.07166pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{-3.22221pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{3}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{0.0pt}\pgfsys@curveto{-68.28662pt}{9.42853pt}{- 75.92975pt}{17.07166pt}{-85.35828pt}{17.07166pt}\pgfsys@curveto{-94.7868pt}{17 .07166pt}{-102.42993pt}{9.42853pt}{-102.42993pt}{0.0pt}\pgfsys@curveto{-102.42 993pt}{-9.42853pt}{-94.7868pt}{-17.07166pt}{-85.35828pt}{-17.07166pt} \pgfsys@curveto{-75.92975pt}{-17.07166pt}{-68.28662pt}{-9.42853pt}{-68.28662pt }{0.0pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{-3.22221 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{1}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{17.07166pt}{85.35828pt}\pgfsys@curveto{17.07166pt}{94.7868pt }{9.42853pt}{102.42993pt}{0.0pt}{102.42993pt}\pgfsys@curveto{-9.42853pt}{102.4 2993pt}{-17.07166pt}{94.7868pt}{-17.07166pt}{85.35828pt}\pgfsys@curveto{-17.07 166pt}{75.92975pt}{-9.42853pt}{68.28662pt}{0.0pt}{68.28662pt}\pgfsys@curveto{9 .42853pt}{68.28662pt}{17.07166pt}{75.92975pt}{17.07166pt}{85.35828pt} \pgfsys@closepath\pgfsys@moveto{0.0pt}{85.35828pt}\pgfsys@stroke\pgfsys@invoke { } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{82.13606pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{2}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{85.35828pt}\pgfsys@curveto{-68.28662pt}{94.7868 pt}{-75.92975pt}{102.42993pt}{-85.35828pt}{102.42993pt}\pgfsys@curveto{-94.786 8pt}{102.42993pt}{-102.42993pt}{94.7868pt}{-102.42993pt}{85.35828pt} \pgfsys@curveto{-102.42993pt}{75.92975pt}{-94.7868pt}{68.28662pt}{-85.35828pt} {68.28662pt}\pgfsys@curveto{-75.92975pt}{68.28662pt}{-68.28662pt}{75.92975pt}{ -68.28662pt}{85.35828pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{85.35828 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{82.13606 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{4}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -72.22612pt}{13.13216pt}\pgfsys@lineto{-13.13216pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -66.78662pt}{0.0pt}\pgfsys@lineto{-18.57166pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -85.35828pt}{18.57166pt}\pgfsys@lineto{-85.35828pt}{66.78662pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 0.0pt}{66.78662pt}\pgfsys@lineto{0.0pt}{18.57166pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -18.57166pt}{85.35828pt}\pgfsys@lineto{-66.78662pt}{85.35828pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -13.13216pt}{13.13216pt}\pgfsys@lineto{-72.22612pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{{{}}}}{{}}{{}}\hbox{ \hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt} \pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -23.36897pt}{-77.42929pt}\pgfsys@curveto{-23.36897pt}{-68.00076pt}{-31.0121pt} {-60.35764pt}{-40.44063pt}{-60.35764pt}\pgfsys@curveto{-49.86916pt}{-60.35764 pt}{-57.51228pt}{-68.00076pt}{-57.51228pt}{-77.42929pt}\pgfsys@curveto{-57.512 28pt}{-86.85782pt}{-49.86916pt}{-94.50095pt}{-40.44063pt}{-94.50095pt} \pgfsys@curveto{-31.0121pt}{-94.50095pt}{-23.36897pt}{-86.85782pt}{-23.36897pt }{-77.42929pt}\pgfsys@closepath\pgfsys@moveto{-40.44063pt}{-77.42929pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-42.94063pt}{-80.6515 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{5}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{91.6555pt}{-57.51202pt}\pgfsys@curveto{91.6555pt}{-48.0835pt }{84.01237pt}{-40.44037pt}{74.58385pt}{-40.44037pt}\pgfsys@curveto{65.15532pt} {-40.44037pt}{57.51219pt}{-48.0835pt}{57.51219pt}{-57.51202pt}\pgfsys@curveto{ 57.51219pt}{-66.94055pt}{65.15532pt}{-74.58368pt}{74.58385pt}{-74.58368pt} \pgfsys@curveto{84.01237pt}{-74.58368pt}{91.6555pt}{-66.94055pt}{91.6555pt}{-5 7.51202pt}\pgfsys@closepath\pgfsys@moveto{74.58385pt}{-57.51202pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{72.08385pt}{-60.73424 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{6}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{48.36986pt}{-139.41833pt}\pgfsys@curveto{48.36986pt}{-129.98 98pt}{40.72673pt}{-122.34668pt}{31.2982pt}{-122.34668pt}\pgfsys@curveto{21.869 67pt}{-122.34668pt}{14.22655pt}{-129.9898pt}{14.22655pt}{-139.41833pt} \pgfsys@curveto{14.22655pt}{-148.84686pt}{21.86967pt}{-156.48999pt}{31.2982pt} {-156.48999pt}\pgfsys@curveto{40.72673pt}{-156.48999pt}{48.36986pt}{-148.84686 pt}{48.36986pt}{-139.41833pt}\pgfsys@closepath\pgfsys@moveto{31.2982pt}{-139.4 1833pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{28.7982pt}{-142.64055 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{7}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -8.59778pt}{-16.4616pt}\pgfsys@lineto{-31.84285pt}{-60.9677pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 14.7069pt}{-11.34062pt}\pgfsys@lineto{59.87666pt}{-46.1714pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 4.06793pt}{-18.12079pt}\pgfsys@lineto{27.23027pt}{-121.29755pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -22.14131pt}{-74.26053pt}\pgfsys@lineto{56.28453pt}{-60.68079pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -26.38834pt}{-89.57187pt}\pgfsys@lineto{17.24591pt}{-127.27576pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 65.90645pt}{-73.93167pt}\pgfsys@lineto{39.9756pt}{-122.99869pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par\par \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss} \pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}} \lxSVG@closescope\endpgfpicture}}}}\hskip 28.45274ptH^{\prime}=\raisebox{-0.5 pt}{\leavevmode\resizebox{24.18501pt}{}{ \leavevmode\hbox to215.55pt{\vbox to 278.99pt{\pgfpicture\makeatletter\hbox{\hskip 155.95746pt\lower-103.92993pt \hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{ rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ } \pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt} \pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{0.8pt}\pgfsys@invoke{ } \par{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3 .0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 17.07166pt}{0.0pt}\pgfsys@curveto{17.07166pt}{9.42853pt}{9.42853pt}{17.07166pt }{0.0pt}{17.07166pt}\pgfsys@curveto{-9.42853pt}{17.07166pt}{-17.07166pt}{9.428 53pt}{-17.07166pt}{0.0pt}\pgfsys@curveto{-17.07166pt}{-9.42853pt}{-9.42853pt}{ -17.07166pt}{0.0pt}{-17.07166pt}\pgfsys@curveto{9.42853pt}{-17.07166pt}{17.071 66pt}{-9.42853pt}{17.07166pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{-3.22221pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{1}\pgfsys@stroke@opacity{1}{3}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{0.0pt}\pgfsys@curveto{-68.28662pt}{9.42853pt}{- 75.92975pt}{17.07166pt}{-85.35828pt}{17.07166pt}\pgfsys@curveto{-94.7868pt}{17 .07166pt}{-102.42993pt}{9.42853pt}{-102.42993pt}{0.0pt}\pgfsys@curveto{-102.42 993pt}{-9.42853pt}{-94.7868pt}{-17.07166pt}{-85.35828pt}{-17.07166pt} \pgfsys@curveto{-75.92975pt}{-17.07166pt}{-68.28662pt}{-9.42853pt}{-68.28662pt }{0.0pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{-3.22221 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{1}\pgfsys@stroke@opacity{1}{1}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{17.07166pt}{85.35828pt}\pgfsys@curveto{17.07166pt}{94.7868pt }{9.42853pt}{102.42993pt}{0.0pt}{102.42993pt}\pgfsys@curveto{-9.42853pt}{102.4 2993pt}{-17.07166pt}{94.7868pt}{-17.07166pt}{85.35828pt}\pgfsys@curveto{-17.07 166pt}{75.92975pt}{-9.42853pt}{68.28662pt}{0.0pt}{68.28662pt}\pgfsys@curveto{9 .42853pt}{68.28662pt}{17.07166pt}{75.92975pt}{17.07166pt}{85.35828pt} \pgfsys@closepath\pgfsys@moveto{0.0pt}{85.35828pt}\pgfsys@stroke\pgfsys@invoke { } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{82.13606pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{1}\pgfsys@stroke@opacity{1}{2}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{85.35828pt}\pgfsys@curveto{-68.28662pt}{94.7868 pt}{-75.92975pt}{102.42993pt}{-85.35828pt}{102.42993pt}\pgfsys@curveto{-94.786 8pt}{102.42993pt}{-102.42993pt}{94.7868pt}{-102.42993pt}{85.35828pt} \pgfsys@curveto{-102.42993pt}{75.92975pt}{-94.7868pt}{68.28662pt}{-85.35828pt} {68.28662pt}\pgfsys@curveto{-75.92975pt}{68.28662pt}{-68.28662pt}{75.92975pt}{ -68.28662pt}{85.35828pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{85.35828 pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{82.13606 pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{1}\pgfsys@stroke@opacity{1}{4}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -72.22612pt}{13.13216pt}\pgfsys@lineto{-13.13216pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -66.78662pt}{0.0pt}\pgfsys@lineto{-18.57166pt}{0.0pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -85.35828pt}{18.57166pt}\pgfsys@lineto{-85.35828pt}{66.78662pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 0.0pt}{66.78662pt}\pgfsys@lineto{0.0pt}{18.57166pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -18.57166pt}{85.35828pt}\pgfsys@lineto{-66.78662pt}{85.35828pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -13.13216pt}{13.13216pt}\pgfsys@lineto{-72.22612pt}{72.22612pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{{{}}}}{{}}\hbox{\hbox{ {\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{-85. 35828pt}\pgfsys@curveto{-68.28662pt}{-75.92975pt}{-75.92975pt}{-68.28662pt}{-8 5.35828pt}{-68.28662pt}\pgfsys@curveto{-94.7868pt}{-68.28662pt}{-102.42993pt}{ -75.92975pt}{-102.42993pt}{-85.35828pt}\pgfsys@curveto{-102.42993pt}{-94.7868 pt}{-94.7868pt}{-102.42993pt}{-85.35828pt}{-102.42993pt}\pgfsys@curveto{-75.92 975pt}{-102.42993pt}{-68.28662pt}{-94.7868pt}{-68.28662pt}{-85.35828pt} \pgfsys@closepath\pgfsys@moveto{-85.35828pt}{-85.35828pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{-88.5804 9pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{1}\pgfsys@stroke@opacity{1}{5}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{ \pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{17.07166pt}{-85.35828pt}\pgfsys@curveto{17.07166pt}{-75.9297 5pt}{9.42853pt}{-68.28662pt}{0.0pt}{-68.28662pt}\pgfsys@curveto{-9.42853pt}{-6 8.28662pt}{-17.07166pt}{-75.92975pt}{-17.07166pt}{-85.35828pt}\pgfsys@curveto{ -17.07166pt}{-94.7868pt}{-9.42853pt}{-102.42993pt}{0.0pt}{-102.42993pt} \pgfsys@curveto{9.42853pt}{-102.42993pt}{17.07166pt}{-94.7868pt}{17.07166pt}{- 85.35828pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{-85.35828pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{-88.58049pt} \pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{1}\pgfsys@stroke@opacity{1}{6}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -13.13216pt}{-13.13216pt}\pgfsys@lineto{-72.22612pt}{-72.22612pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ 0.0pt}{-18.57166pt}\pgfsys@lineto{0.0pt}{-66.78662pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -66.78662pt}{-85.35828pt}\pgfsys@lineto{-18.57166pt}{-85.35828pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -85.35828pt}{-18.57166pt}\pgfsys@lineto{-85.35828pt}{-66.78662pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -72.22612pt}{-13.13216pt}\pgfsys@lineto{-13.13216pt}{-72.22612pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par\par{{{{}}}}{{}}{{}} \hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt} \pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -31.29796pt}{156.49017pt}\pgfsys@curveto{-31.29796pt}{165.9187pt}{-38.94109pt} {173.56183pt}{-48.36961pt}{173.56183pt}\pgfsys@curveto{-57.79814pt}{173.56183 pt}{-65.44127pt}{165.9187pt}{-65.44127pt}{156.49017pt}\pgfsys@curveto{-65.4412 7pt}{147.06165pt}{-57.79814pt}{139.41852pt}{-48.36961pt}{139.41852pt} \pgfsys@curveto{-38.94109pt}{139.41852pt}{-31.29796pt}{147.06165pt}{-31.29796 pt}{156.49017pt}\pgfsys@closepath\pgfsys@moveto{-48.36961pt}{156.49017pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-50.86961pt}{153.2679 6pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0} \pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0} {0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{1}\pgfsys@stroke@opacity{1}{7}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \par{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -56.93791pt}{140.013pt}\pgfsys@lineto{-76.79027pt}{101.83545pt}\pgfsys@stroke \pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope \pgfsys@invoke{ }\pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{ -37.92647pt}{141.13263pt}\pgfsys@lineto{-10.44315pt}{100.71582pt} \pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par {{}}{}{{}}{{}}{{{{}}{{}}{{}}{{}}}{{{{}}{}{}{}{}{{}}}} }{{{{}}{{}}{{}}{{}}{{}}}{{{{}}{}{}{}{}{{}}}} }{{}{}}{{}} {}{}{}{{{}}{{}}{{}}} {{{}}{{}}{{}}} {}{{}}{}{{}}{}{{}}{}{}{}\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{-64.23752pt}{146. 84048pt}\pgfsys@curveto{-137.81586pt}{102.09544pt}{-154.45746pt}{-6.71533pt}{- 97.61649pt}{-71.40688pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}}{{}}{{{{}}{{}}{{}} {{}}}{{{{}}{}{}{}{}{{}}}} }{{{{}}{{}}{{}}{{}}{{}}}{{{{}}{}{}{}{}{{}}}} }{{}{}}{{}} {}{}{}{{{}}{{}}{{}}} {{{}}{{}}{{}}} {}{{}}{}{{}}{}{{}}{}{}{}\pgfsys@beginscope\pgfsys@invoke{ } \pgfsys@setlinewidth{3.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{14.22656pt}{73.42 058pt}\pgfsys@curveto{58.09277pt}{36.61186pt}{58.09277pt}{-36.61186pt}{14.2265 6pt}{-73.42058pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par\par \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss} \pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}} \lxSVG@closescope\endpgfpicture}}}}$ $$

Here, $G$ and $G^{\prime}$ belong to the class $c$, while $H$ and $H^{\prime}$ belong to the class $c^{\prime}$, where $c\neq c^{\prime}$. Assuming the margin condition is satisfied for all classes, including $c$ and $c^{\prime}$, and that embeddings of graphs within each class cluster around the embeddings of $G$, $G^{\prime}$, and $H$, $H^{\prime}$, we estimate $\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}
{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space
$}}}(\lambda_{\sharp}(\mu_{c,T}),\lambda_{\sharp}(\mu_{c,\tilde{T}})\mathclose
{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{
\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})$ using $\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}
{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space
$}}}(\lambda_{\sharp}(G),\lambda_{\sharp}(G^{\prime})\mathclose{{\hbox{$\left
missing
\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{
\vbox to4.04999pt{}}\right.\n@space$}}})$. Since there are only two classes in this dataset, we estimate $\max_{c,c^{\prime}\in\mathcal{Y},c\neq c^{\prime}}\mathcal{W}_{1}\mathopen{{
\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{
\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}(\lambda_{\sharp}(
\mu_{c}),\lambda_{\sharp}(\mu_{c^{\prime}})\mathclose{{\hbox{$\left missing
\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{
\vbox to4.04999pt{}}\right.\n@space$}}})$ by $\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}
{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space
$}}}(\lambda_{\sharp}(\mu_{c}),\lambda_{\sharp}(\mu_{c^{\prime}})\mathclose{{
\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{
\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})$ where $\mu_{c}=\{G,G^{\prime}\}$ and $\mu_{c^{\prime}}=\{H,H^{\prime}\}$.

For the bounding graph encoders $\lambda$, we use $\mathsf{1}\text{-}\text{WL}$ as the base model. Then, following the approach by Barceló et al. ()
we consider two simple rooted graphs
3124
and
3124
and append their homomorphism counts $\mathsf{Hom}(\raisebox{-0.2pt}{\leavevmode\resizebox{11.38092pt}{}{
\leavevmode\hbox to123.5pt{\vbox to123.5pt{\pgfpicture\makeatletter\hbox{
\hskip 104.42993pt\lower-19.07166pt\hbox to0.0pt{\pgfsys@beginscope
\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to
0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.8pt}
\pgfsys@invoke{ }
\par{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{4
.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
17.07166pt}{0.0pt}\pgfsys@curveto{17.07166pt}{9.42853pt}{9.42853pt}{17.07166pt
}{0.0pt}{17.07166pt}\pgfsys@curveto{-9.42853pt}{17.07166pt}{-17.07166pt}{9.428
53pt}{-17.07166pt}{0.0pt}\pgfsys@curveto{-17.07166pt}{-9.42853pt}{-9.42853pt}{
-17.07166pt}{0.0pt}{-17.07166pt}\pgfsys@curveto{9.42853pt}{-17.07166pt}{17.071
66pt}{-9.42853pt}{17.07166pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0
pt}\pgfsys@stroke\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{-3.22221pt}
\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{3}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\par{{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }
\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{
\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{
}{}\pgfsys@moveto{-68.28662pt}{0.0pt}\pgfsys@curveto{-68.28662pt}{9.42853pt}{-
75.92975pt}{17.07166pt}{-85.35828pt}{17.07166pt}\pgfsys@curveto{-94.7868pt}{17
.07166pt}{-102.42993pt}{9.42853pt}{-102.42993pt}{0.0pt}\pgfsys@curveto{-102.42
993pt}{-9.42853pt}{-94.7868pt}{-17.07166pt}{-85.35828pt}{-17.07166pt}
\pgfsys@curveto{-75.92975pt}{-17.07166pt}{-68.28662pt}{-9.42853pt}{-68.28662pt
}{0.0pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{0.0pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{-3.22221
pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{1}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
{{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }
\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{
\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{
}{}\pgfsys@moveto{17.07166pt}{85.35828pt}\pgfsys@curveto{17.07166pt}{94.7868pt
}{9.42853pt}{102.42993pt}{0.0pt}{102.42993pt}\pgfsys@curveto{-9.42853pt}{102.4
2993pt}{-17.07166pt}{94.7868pt}{-17.07166pt}{85.35828pt}\pgfsys@curveto{-17.07
166pt}{75.92975pt}{-9.42853pt}{68.28662pt}{0.0pt}{68.28662pt}\pgfsys@curveto{9
.42853pt}{68.28662pt}{17.07166pt}{75.92975pt}{17.07166pt}{85.35828pt}
\pgfsys@closepath\pgfsys@moveto{0.0pt}{85.35828pt}\pgfsys@stroke\pgfsys@invoke
{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{82.13606pt}
\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{2}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
{{{{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor[named
]{pgffillcolor}{rgb}{0,0,0}\pgfsys@color@gray@fill{0}\pgfsys@invoke{ }
\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{
\pgfsys@beginscope\pgfsys@invoke{ }\definecolor[named]{pgffillcolor}{rgb}{
0,0,0}\pgfsys@color@gray@fill{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}
\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28662pt}{85.35828pt}\pgfsys@curveto{-68.
28662pt}{94.7868pt}{-75.92975pt}{102.42993pt}{-85.35828pt}{102.42993pt}
\pgfsys@curveto{-94.7868pt}{102.42993pt}{-102.42993pt}{94.7868pt}{-102.42993pt
}{85.35828pt}\pgfsys@curveto{-102.42993pt}{75.92975pt}{-94.7868pt}{68.28662pt}
{-85.35828pt}{68.28662pt}\pgfsys@curveto{-75.92975pt}{68.28662pt}{-68.28662pt}
{75.92975pt}{-68.28662pt}{85.35828pt}\pgfsys@closepath\pgfsys@moveto{-85.35828
pt}{85.35828pt}\pgfsys@fillstroke\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{82.13606
pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{4}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\par{{}}{}{{}}
{{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-66.28662pt}{0.0pt}\pgfsys@lineto{-19.07166pt}{0.0pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}}
{{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-85.35828pt}{19.07166pt}\pgfsys@lineto{-85.35828pt}{66.28662pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}}
{{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
0.0pt}{66.28662pt}\pgfsys@lineto{0.0pt}{19.07166pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{{}}
{{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-19.07166pt}{85.35828pt}\pgfsys@lineto{-66.28662pt}{85.35828pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}
\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}
\lxSVG@closescope\endpgfpicture}}}},\cdot)$ and $\mathsf{Hom}(\raisebox{-0.2pt}{\leavevmode\resizebox{11.38092pt}{}{
\leavevmode\hbox to123.5pt{\vbox to123.5pt{\pgfpicture\makeatletter\hbox{
\hskip 104.43028pt\lower-19.07166pt\hbox to0.0pt{\pgfsys@beginscope
\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to
0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\pgfsys@setlinewidth{0.8pt}
\pgfsys@invoke{ }
\par{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{4
.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
17.07166pt}{0.0pt}\pgfsys@curveto{17.07166pt}{9.42853pt}{9.42853pt}{17.07166pt
}{0.0pt}{17.07166pt}\pgfsys@curveto{-9.42853pt}{17.07166pt}{-17.07166pt}{9.428
53pt}{-17.07166pt}{0.0pt}\pgfsys@curveto{-17.07166pt}{-9.42853pt}{-9.42853pt}{
-17.07166pt}{0.0pt}{-17.07166pt}\pgfsys@curveto{9.42853pt}{-17.07166pt}{17.071
66pt}{-9.42853pt}{17.07166pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0
pt}\pgfsys@stroke\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{-3.22221pt}
\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{3}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\par{{{{}{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }
\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{
\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{
}{}\pgfsys@moveto{-68.28662pt}{0.0pt}\pgfsys@curveto{-68.28662pt}{9.42853pt}{-
75.92975pt}{17.07166pt}{-85.35828pt}{17.07166pt}\pgfsys@curveto{-94.7868pt}{17
.07166pt}{-102.42993pt}{9.42853pt}{-102.42993pt}{0.0pt}\pgfsys@curveto{-102.42
993pt}{-9.42853pt}{-94.7868pt}{-17.07166pt}{-85.35828pt}{-17.07166pt}
\pgfsys@curveto{-75.92975pt}{-17.07166pt}{-68.28662pt}{-9.42853pt}{-68.28662pt
}{0.0pt}\pgfsys@closepath\pgfsys@moveto{-85.35828pt}{0.0pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85828pt}{-3.22221
pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{1}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
{{{{}{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }
\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{
\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{
}{}\pgfsys@moveto{17.07166pt}{85.35828pt}\pgfsys@curveto{17.07166pt}{94.7868pt
}{9.42853pt}{102.42993pt}{0.0pt}{102.42993pt}\pgfsys@curveto{-9.42853pt}{102.4
2993pt}{-17.07166pt}{94.7868pt}{-17.07166pt}{85.35828pt}\pgfsys@curveto{-17.07
166pt}{75.92975pt}{-9.42853pt}{68.28662pt}{0.0pt}{68.28662pt}\pgfsys@curveto{9
.42853pt}{68.28662pt}{17.07166pt}{75.92975pt}{17.07166pt}{85.35828pt}
\pgfsys@closepath\pgfsys@moveto{0.0pt}{85.35828pt}\pgfsys@stroke\pgfsys@invoke
{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{82.13606pt}
\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{2}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
{{{{}{}}}}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor[
named]{pgffillcolor}{rgb}{0,0,0}\pgfsys@color@gray@fill{0}\pgfsys@invoke{ }
\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{
\pgfsys@beginscope\pgfsys@invoke{ }\definecolor[named]{pgffillcolor}{rgb}{
0,0,0}\pgfsys@color@gray@fill{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}
\pgfsys@invoke{ }{}\pgfsys@moveto{-68.28697pt}{85.35828pt}\pgfsys@curveto{-68.
28697pt}{94.7868pt}{-75.9301pt}{102.42993pt}{-85.35863pt}{102.42993pt}
\pgfsys@curveto{-94.78716pt}{102.42993pt}{-102.43028pt}{94.7868pt}{-102.43028
pt}{85.35828pt}\pgfsys@curveto{-102.43028pt}{75.92975pt}{-94.78716pt}{68.28662
pt}{-85.35863pt}{68.28662pt}\pgfsys@curveto{-75.9301pt}{68.28662pt}{-68.28697
pt}{75.92975pt}{-68.28697pt}{85.35828pt}\pgfsys@closepath\pgfsys@moveto{-85.35
863pt}{85.35828pt}\pgfsys@fillstroke\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}{{{{}}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-87.85863pt}{82.13606
pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}
\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}
{0}\pgfsys@invoke{ }\hbox{\pgfsys@fill@opacity{0}\pgfsys@stroke@opacity{0}{4}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\par{{}{}}{}{{}{}}
{{{{{}}{}{}{}{}{{}}}{}}}{}{{{{{}}{}{}{}{}{{}}}{}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-71.87286pt}{13.48573pt}\pgfsys@lineto{-13.48576pt}{71.87285pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}{}}{}{{}{}}
{{{{{}}{}{}{}{}{{}}}{}}}{}{{{{{}}{}{}{}{}{{}}}{}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-66.28688pt}{0.0pt}\pgfsys@lineto{-19.07172pt}{0.0pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}{}}{}{{}{}}
{{{{{}}{}{}{}{}{{}}}{}}}{}{{{{{}}{}{}{}{}{{}}}{}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-85.35889pt}{19.07172pt}\pgfsys@lineto{-85.35896pt}{66.28688pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}{}}{}{{}{}}
{{{{{}}{}{}{}{}{{}}}{}}}{}{{{{{}}{}{}{}{}{{}}}{}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
0.0pt}{66.28688pt}\pgfsys@lineto{0.0pt}{19.07172pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}{}}{}{{}{}}
{{{{{}}{}{}{}{}{{}}}{}}}{}{{{{{}}{}{}{}{}{{}}}{}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-19.07173pt}{85.3589pt}\pgfsys@lineto{-66.28723pt}{85.35863pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}{}}{}{{}{}}
{{{{{}}{}{}{}{}{{}}}{}}}{}{{{{{}}{}{}{}{}{{}}}{}}}{{}}{}{}{}\pgfsys@beginscope
\pgfsys@invoke{ }\pgfsys@setlinewidth{4.0pt}\pgfsys@invoke{ }{}\pgfsys@moveto{
-13.48573pt}{13.48575pt}\pgfsys@lineto{-71.8732pt}{71.87286pt}\pgfsys@stroke
\pgfsys@invoke{ }
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\par
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}
\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}
\lxSVG@closescope\endpgfpicture}}}},\cdot)$ to the vertex feature of each rooted pair $(\cdot,v)$ in the graphs $G$, $G^{\prime}$, $H$ and $H^{\prime}$, respectively. This leads to slight increase in model expressivity, compared with $\mathsf{1}\text{-}\text{WL}$, allowing us to analyze how these differences impact key factors in generalization. We refer to these two graph encoders as $C_{4}\text{-}\text{WL}$, where homomorphism counts of
3124
are added, and $K_{4}\text{-}\text{WL}$, where homomorphism counts of
3124
are added, respectively.

Table 1 presents the graphs $G$ and $G^{\prime}$ along with their initial vertex colors, including input vertex features and homomorphism counts, as well as the vertex colors after one iteration. It also includes the difference between the graph embeddings for the three models.
As model expressivity increases, we observe two distinct scenarios:

- (1)
*More expressivity leads to better generalization*:
Compared to the graph embeddings from $\mathsf{1}\text{-}\text{WL}$, incorporating $C_{4}$ improves intra-class concentration. The homomorphism counts of $C_{4}$ reduce the variance between graph embeddings as shown in Table 1, resulting in a distance of 4.123, smaller than 4.796 for $\mathsf{1}\text{-}\text{WL}$.
- (2)
*More expressivity leads to worse generalization*: When $K_{4}$ is used, the graph embeddings of $G$ and $G^{\prime}$ yield a distance of 5.000, which is larger than its $\mathsf{1}\text{-}\text{WL}$ counterpart 4.796. Compared to $C_{4}\text{-}\text{WL}$, each dimension of the $K_{4}\text{-}\text{WL}$ embeddings has the same or larger magnitude, reflecting higher variance in the graph embeddings.

When measuring inter-class separation using $\mathcal{W}_{1}(\lambda_{\sharp}(\mu_{c}),\lambda_{\sharp}(\mu_{c^{\prime}}))$, the models $\mathsf{1}\text{-}\text{WL}$, $C_{4}\text{-}\text{WL}$, and $K_{4}\text{-}\text{WL}$ achieve distances of 4.582, 4.511, and 4.840, respectively. These results suggest a narrowing in the gaps of these models, compared to intra-class concentration alone. The trends in inter-class separation may change depending on the graph structure.
For instance, if graphs of class $c^{\prime}$ cluster around the embedding of $H^{\prime}$, i.e., estimating $\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}
{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space
$}}}(\lambda_{\sharp}(\mu_{c}),\lambda_{\sharp}(\mu_{c^{\prime}})\mathclose{{
\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{
\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})$ with $\mu_{c}=\{G,G^{\prime}\}$ and $\mu_{c^{\prime}}=\{H^{\prime}\}$, the reverse trend may occur, with $\mathsf{1}\text{-}\text{WL}$ achieving a distance of 4.796 and $K_{4}\text{-}\text{WL}$ achieving 4.583. This highlights the importance of inter-class separation in balancing a model’s generalization performance alongside intra-class concentration.

## 7 Experiments

Tasks and Datasets
We conduct graph classification experiments on six widely used benchmark datasets: ENZYMES, PROTEINS, and MUTAG from the TU dataset collection (Morris et al., ), as well as SIDER and BACE from the molecular dataset collection (Wu et al., ). For SIDER, which comprises 27 classification tasks, we focus specifically on the 21st task. Each dataset is randomly divided into training and test sets following a 90%/10% split.

Setup and Configuration
Each classification task is trained for 400 epochs, with five independent runs to report the mean and standard deviation of the results. Consistent with the setup in Tang and Liu (); Morris et al. (); Cong et al. (), we eliminate the use of regularization techniques such as dropout and weight decay. A batch size of 128 is utilized, with a learning rate set to $10^{-3}$, and the hidden layer dimension fixed at 64. The margin loss function is employed with a margin parameter $\gamma=1$. To compute the generalization gap, we utilize the sample-based variant of the bound as outlined in Theorem 5.1, as given in Theorem C.2 of the appendix.

For the graph encoder $\phi$, we adopt both MPNNs and $\mathcal{F}$-MPNNs, with expressivity constraints defined by $\mathsf{1}\text{-}\text{WL}$ and $\mathcal{F}\text{-}\text{WL}$, respectively, as described in Section 4. The predictor $\psi(\cdot)$ is modeled using the softmax function, which has a Lipschitz constant of 1 (Gao and Pavel, ), ensuring that $\mathsf{Lip}(\rho_{\psi}(\cdot,c))$ is also 1.
We estimate $\mathsf{Lip}(f)$ as: $\mathsf{Lip}(f)=\max_{G,H\in\mathcal{G}_{\text{train}}}\left(\frac{d_{\mathcal
{Z}_{\phi}}(\phi(G),\phi(H))}{d_{\mathcal{Z}_{\lambda}}(\lambda(G),\lambda(H))
}\right)$,
where $G$ and $H$ are sampled from the training set $\mathcal{G}_{\text{train}}$. For all experiments, we set the confidence level $\delta$ to 0.1, yielding bounds with high probability.
The experiments are conducted on a Linux-based system equipped with 96 GB of memory and a single NVIDIA RTX A6000 GPU.

**Table 2: Graph classification gaps with different numbers of MPNN layers. The MPNN embeddings are not normalized.**
|  |  | Dataset |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| # Layers |  | ENZYMES | PROTEINS | MUTAG | SIDER | BACE |
| 1 | Acc. gap | 25.41±3.82 | 2.53±1.76 | -4.84±2.52 | 4.21±0.71 | 3.63±1.63 |
| Loss gap | 0.248±0.040 | 0.029±0.015 | -0.070±0.017 | 0.037±0.003 | 0.018±0.017 |  |
| Our Bound | 7.926±1.279 | 2.193±0.702 | 1.216±0.169 | 0.511±0.286 | 1.479±0.301 |  |
| VC Bound | 586 | 929 | 51 | 960 | 621 |  |
| 2 | Acc. gap | 24.26±2.92 | 3.62±1.06 | -4.84±1.78 | 4.77±1.12 | 4.33±1.29 |
| Loss gap | 0.242±0.026 | 0.032±0.010 | -0.074±0.007 | 0.038±0.003 | 0.037±0.019 |  |
| Our Bound | 7.425±0.982 | 1.404±0.144 | 1.247±0.155 | 0.620±0.463 | 1.729±0.251 |  |
| VC Bound | 595 | 996 | 121 | 1300 | 1060 |  |
| 3 | Acc. gap | 29.89±3.01 | 2.95±1.47 | -6.60±2.57 | 4.03±0.19 | 4.99±1.22 |
| Loss gap | 0.237±0.035 | 0.025±0.009 | -0.058±0.012 | 0.038±0.002 | 0.032±0.011 |  |
| Our Bound | 6.513±0.951 | 1.421±0.220 | 1.649±0.158 | 0.409±0.253 | 1.789±0.226 |  |
| VC Bound | 595 | 996 | 135 | 1309 | 1089 |  |
| 4 | Acc. gap | 27.04±4.69 | 2.86±0.86 | -6.71±2.00 | 4.13±0.03 | 3.13±2.04 |
| Loss gap | 0.235±0.038 | 0.027±0.005 | -0.073±0.009 | 0.036±0.001 | 0.022±0.030 |  |
| Our Bound | 6.825±0.796 | 1.434±0.297 | 1.535±0.115 | 0.298±0.080 | 1.686±0.377 |  |
| VC Bound | 595 | 996 | 139 | 1309 | 1093 |  |
| 5 | Acc. gap | 29.85±4.95 | 0.79±0.78 | -4.01±1.14 | 4.09±0.06 | 3.53±1.41 |
| Loss gap | 0.256±0.037 | 0.020±0.007 | -0.071±0.021 | 0.035±0.001 | 0.020±0.020 |  |
| Our Bound | 6.384±0.813 | 1.308±0.165 | 1.773±0.194 | 0.369±0.172 | 1.662±0.120 |  |
| VC Bound | 595 | 996 | 139 | 1309 | 1093 |  |
| 6 | Acc. gap | 28.85±6.01 | 1.42±1.40 | -4.02±3.68 | 3.60±0.48 | 2.44±1.40 |
| Loss gap | 0.264±0.025 | 0.030±0.008 | -0.078±0.019 | 0.034±0.002 | 0.022±0.016 |  |
| Our Bound | 6.151±0.798 | 1.340±0.316 | 1.627±0.038 | 0.353±0.156 | 1.785±0.237 |  |
| VC Bound | 595 | 996 | 139 | 1309 | 1093 |  |

### 7.1 Results and discussion

How well can the bounds predict the generalization ability of MPNNs? To answer this, we compare the proposed bound with empirical generalization gaps, measured by loss and accuracy, while controlling MPNN expressivity by varying the number of layers.
Table 2 presents the proposed bound and the empirical generalization gaps for different numbers of MPNN layers across five datasets. For comparison, we also include the VC bound from Morris et al. (), which is based on the number of unique color histograms produced by $\mathsf{1}\text{-}\text{WL}$, indicating the number of graphs distinguishable by $\mathsf{1}\text{-}\text{WL}$.
Our results show that the proposed bounds closely mirror the empirical generalization gaps across datasets and varying layer depths, effectively predicting generalization errors. This consistency highlights the bound’s ability to reflect changes in generalization performance as model depth increases.
In contrast, the VC bound stabilizes after three iterations for most datasets, as $\mathsf{1}\text{-}\text{WL}$ can no longer distinguish more graphs. As a result, the VC bound fails to capture improved generalization performance at deeper layers and remains higher than our bound.
Our bound is less vacuous compared to the VC bound and other bounds, such as those proposed by Garg et al. (); Liao et al. (), which tend to be on the order of $10^{4}$.

To evaluate how well the proposed bound predicts the generalization gap of $\mathcal{F}$-MPNNs across different homomorphism pattern selections, we present the empirical loss gap and generalization bound for three distinct pattern sets, alongside MPNN, as shown in LABEL:fig:patterns_gap. We designate $P_{n}$, $K_{n}$, and $C_{n}$ as $n$-path, $n$-clique, and $n$-cycle graphs, respectively, and refer to the MPNN without any specific pattern as “no pattern". It can be seen that the generalization bound closely aligns with the empirical gap across different pattern choices, with some exceptions in ENZYMES. Notably, the choice of pattern influences the generalization gap in different ways. In ENZYMES, cycle patterns lead to a larger gap compared to cliques and paths. In PROTEINS, using paths or cliques increases the generalization gap, while cycles reduce it. These changes in the empirical generalization gap are largely captured by the corresponding bounds.

Why does more expressive power sometimes lead to better generalization?
In LABEL:subfig:gaps_layer, we observe two contrasting cases where increased expressivity worsens generalization (ENZYMES) and improves it (PROTEINS). To explore this further, we plot the changes of two major factors from the proposed bound in Theorem C.2: the 1-Wasserstein distance and $\mathsf{Lip}(f)$, both shown in LABEL:subfig:gaps_layer.
The 1-Wasserstein distance ($\mathcal{W}_{1}$) is computed as:
$\frac{1}{n}\sum_{j=1}^{n}\mathcal{W}_{1}\bigl{(}\lambda_{\sharp}(\mu_{c,T^{j}}
),\lambda_{\sharp}({\mu}_{c,\tilde{T}^{j}})\bigr{)}$
averaged over all graph classes. We plot these factors over four layers for both MPNN and $\{C_{3},C_{4},C_{5},C_{6}\}$-MPNN.
We observe that the inclusion of homomorphism counts worsens generalization in ENZYMES but improves it in PROTEINS. From LABEL:subfig:gaps_layer, this can be attributed to the joint influence of the Wasserstein distance and $\mathsf{Lip}(f)$. In ENZYMES, both the 1-Wasserstein distance and $\mathsf{Lip}(f)$ increase slightly when homomorphism counts are added.
While this additional expressivity leads to better separation between graphs, in ENZYMES, this increased separation hinders the ability to achieve good concentration within each graph class, ultimately worsening generalization.
In contrast, for PROTEINS, although the inclusion of homomorphism counts leads to greater graph separation, it also slightly reduces the 1-Wasserstein distance within each class, allowing for better concentration. This improved separation significantly reduces $\mathsf{Lip}(f)$, resulting in enhanced generalization.

Can generalization be improved by controlling the Lipschitz constants?
Last but not least, since $\mathsf{Lip}(f)$ plays a crucial role in the proposed bound, we aim to investigate whether controlling $\mathsf{Lip}(f)$ can serve as an effective strategy to enhance generalization. A straightforward approach to control $\mathsf{Lip}(f)$ is through normalization techniques. As demonstrated earlier, normalization effectively bounds the diameter of $\phi_{\sharp}(\mu)$, which, in turn, constrains the encoder’s boundedness and subsequently $\mathsf{Lip}(f)$.
To test this, we apply $l_{1}$-normalisation in the last layer of the MPNN. See Table 3 for results. It is evident that normalization reduces the generalization gap across all datasets. This improvement is also reflected in the computed bounds.
Interestingly, the least improvement is observed in the SIDER dataset, where $\mathsf{Lip}(f)$ is already relatively small, and the embeddings are well-concentrated even before normalization. This suggests that the impact of normalization is more pronounced when $\mathsf{Lip}(f)$ is large or when the embeddings are not already well-concentrated.

**Table 3: Graph classification gaps with different numbers of MPNN layers. The MPNN embeddings are normalized.**
|  |  | Dataset |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| # Layers |  | ENZYMES | PROTEINS | MUTAG | SIDER | BACE |
| 1 | Acc. gap | 14.67±2.85 | -1.74±0.97 | -8.82±1.65 | 1.69±1.77 | -0.34±1.26 |
| Loss gap | 0.105±0.010 | -0.018±0.009 | -0.091±0.017 | 0.013±0.013 | -0.004±0.010 |  |
| Our Bound | 0.800±0.095 | 2.203±0.134 | 1.101±0.063 | 1.137±0.552 | 1.147±0.143 |  |
| 2 | Acc. gap | 14.85±3.56 | -1.84±1.25 | -9.53±1.98 | 1.41±2.19 | 0.16±1.11 |
| Loss gap | 0.098±0.022 | -0.023±0.011 | -0.097±0.019 | 0.015±0.006 | 0.000±0.010 |  |
| Our Bound | 0.586±0.036 | 1.016±0.035 | 1.208±0.046 | 1.017±0.644 | 1.089±0.135 |  |
| 3 | Acc. gap | 19.70±3.74 | -2.72±1.27 | -7.88±0.79 | 3.71±1.50 | -0.55±1.67 |
| Loss gap | 0.118±0.023 | -0.027±0.011 | -0.083±0.006 | 0.030±0.008 | -0.006±0.015 |  |
| Our Bound | 0.572±0.024 | 0.834±0.015 | 0.993±0.039 | 1.221±0.957 | 1.167±0.610 |  |
| 4 | Acc. gap | 20.78±2.18 | -0.45±0.68 | -8.23±1.14 | 2.98±1.61 | 0.28±2.42 |
| Loss gap | 0.129±0.007 | -0.004±0.005 | -0.087±0.011 | 0.026±0.015 | 0.001±0.024 |  |
| Our Bound | 0.573±0.027 | 0.847±0.027 | 0.848±0.085 | 1.039±0.898 | 0.705±0.026 |  |
| 5 | Acc. gap | 24.70±3.08 | 0.55±3.69 | -8.47±1.26 | 1.06±4.64 | 0.07±1.85 |
| Loss gap | 0.169±0.014 | 0.003±0.035 | -0.086±0.013 | 0.006±0.038 | 0.002±0.014 |  |
| Our Bound | 0.575±0.039 | 0.713±0.246 | 0.799±0.051 | 0.923±0.438 | 0.703±0.012 |  |
| 6 | Acc. gap | 23.33±2.75 | 0.08±3.31 | -10.24±0.87 | 3.44±0.91 | -1.17±1.55 |
| Loss gap | 0.169±0.023 | -0.002±0.032 | -0.104±0.008 | 0.029±0.009 | -0.013±0.015 |  |
| Our Bound | 0.603±0.032 | 0.793±0.136 | 0.778±0.049 | 1.192±0.561 | 0.679±0.018 |  |

## 8 Conclusion and limitations

In this work, we examine the generalization of GNNs from a margin-based perspective, based on the work by Chuang et al. (). The bounds use $1$-variance and optimal transport to analyze graph embeddings. We establish a relationship between generalization and the expressive capacity of GNNs, deriving a generalization bound that demonstrates how well-clustered embeddings and separable classes lead to improved generalization. Through case studies on a real-world dataset, we empirically validate these theoretical findings. We also apply empirical sample-based bounds to graph classification tasks, confirming that our theoretical results align with empirical evidence. Our work enables analyzing the generalization of graph encoders through their bounded expressive power.

Nonetheless, our work has some limitations. While we validate the framework on real-world datasets, further large-scale studies across a wider range of datasets and applications are needed to fully establish the proposed approach’s general applicability.

### Acknowledgements

This research was supported partially by the Australian Government through the Australian Research Council’s Discovery Projects funding scheme (project DP210102273).

## Appendix A Additional related work

We provide additional references related to the expressiveness of Graph Neural Networks (GNNs). The connection with the Weisfeiler-Leman ($\mathsf{1}\text{-}\text{WL}$) test has led to the development of high-order GNNs that surpass $\mathsf{1}\text{-}\text{WL}$ and are bounded by the $k$-dimensional Weisfeiler-Leman test ($k\text{-}\text{WL}$) [Morris et al., , Maron et al., , Morris et al., , Geerts and Reutter, ]. The method by Morris et al. [, ] is strictly weaker than $k\text{-}\text{WL}$, whereas the method by Maron et al. [] can match the expressiveness of $k\text{-}\text{WL}$. However, these higher-order GNNs incur significant computational costs, rendering them impractical for large-scale datasets.

Incorporating substructure counts has been shown to be an effective strategy for enhancing GNN expressivity beyond $\mathsf{1}\text{-}\text{WL}$ [Bouritsas et al., , Barceló et al., ]. Bouritsas et al. [] integrate isomorphism counts of small subgraph patterns into the node and edge features of graphs, while Barceló et al. [] employ a similar approach using homomorphism counts. Building on this concept, Thiede et al. [] implemented convolutions on automorphism groups of subgraph patterns. Rather than directly using subgraph counts, Wijesinghe and Wang [], Wang et al. [] propose integrating local structural information into neighbor aggregation. This approach suggests that the expressivity of the model increases with the subgraph pattern size and aggregation radius.

Taking a different approach, Nguyen and Maehara [] explore the use of graph homomorphism counts directly in convolutions without message passing, demonstrating their universality in approximating invariant functions. Welke et al. [] propose combining homomorphism counts with GNN outputs in the final layer to improve expressivity. Additionally, Bevilacqua et al. [] represent graphs as collections of subgraphs derived from a predetermined policy. Zhao et al. [] and Zhang and Li [] extend this idea by representing graphs with a set of induced subgraphs. These methods are closely related to graph kernel techniques that utilize subgraph patterns [Shervashidze et al., , Horváth et al., , Costa and Grave, ].

Since the $\mathsf{WL}$-based GNN expressivity hierarchy is inherently coarse and qualitative, Zhang et al. [] propose a homomorphism-based expressivity framework, which enables direct comparisons of expressivity between common GNN models. As $\mathsf{1}\text{-}\text{WL}$ and $k\text{-}\text{WL}$ have equivalent translations in homomorphism embeddings [Dell et al., ], both MPNNs and higher-order GNNs can be expressed using homomorphism representations within this framework. Given that homomorphism embeddings are theoretically isomorphism-complete, this framework offers not only a unified but also a complete description of GNN expressivity.

## Appendix B Proofs of Section 4

See 4.2

###### Proof.

We define the function
$f:\mathcal{Z}_{\phi}\to\mathcal{Z}_{\phi^{\prime}}$, as follows. Let $z\in\mathcal{Z}_{\phi}$ and $G\in\mathcal{G}$ such that $\phi(G)=z$. Then,
define $f(z):=\phi^{\prime}(G)\in\mathcal{Z}_{\phi^{\prime}}$. Observe first that $f$ is well-defined. Indeed, if we take another $G^{\prime}\in\mathcal{G}$ such that $\phi(G^{\prime})=z$, then $\phi(G)=\phi(G^{\prime})$ and hence also $\phi^{\prime}(G)=\phi^{\prime}(G^{\prime})=f(z)$ since $\phi\sqsubseteq\phi^{\prime}$ by assumption.
Clearly, $f\circ\phi=\phi^{\prime}$, by definition
∎

See 4.3

###### Proof.

We have
$d_{\mathcal{Z}_{\phi^{\prime}}}(f(\phi(G),f(\phi(H))=0$ in case
$\phi(G)=\phi(H)$ and otherwise,
when $\phi(G)\neq\phi(H)$ also
$\phi^{\prime}(G)\neq\phi^{\prime}(H)$ and hence

$$ $d_{\mathcal{Z}_{\phi^{\prime}}}(f(\phi(G),f(\phi(H))=d_{\mathcal{Z}_{\phi^{ \prime}}}(\phi^{\prime}(G),\phi^{\prime}(H))\leq B.$ $$

Since $S\leq d_{\mathcal{Z}_{\phi}}(\phi(G),\phi(H))$ we have

$$ $d_{\mathcal{Z}_{\phi^{\prime}}}(f(\phi(G),f(\phi(H))\leq(B/S)d_{\mathcal{Z}_{ \phi}}(\phi(G),\phi(H)),$ $$

and thus

$$ $d_{\mathcal{Z}_{\phi^{\prime}}}(f(z),f(z^{\prime}))\leq(B/S)d_{\mathcal{Z}_{ \phi}}(z,z^{\prime})$ $$

since the domain of $f$ is the codomain of $\phi$. We may thus conclude that $\mathsf{Lip}(f)\leq\frac{B}{S}$, as desired.
∎

See 4.4

###### Proof.

We first show that $f\circ\phi=\phi^{\prime}$ implies the $f_{\sharp}\bigl{(}\phi_{\sharp}(\mu)\bigr{)}=\phi^{\prime}_{\sharp}(\mu)$ of the corresponding pushforward distribution of any distribution $\mu$ om $\mathcal{G}$. Indeed, this simply follows from the definitions. One the one hand, for $I\subseteq\mathcal{Z}_{\phi^{\prime}}$

$$ $\phi_{\sharp}^{\prime}(\mu)(I):=\mu\bigl{(}\{G\in\mathcal{G}\mid\phi^{\prime}( G)\in I\}\bigr{)}.$ $$

On the other hand,

$$ $\displaystyle f_{\sharp}\bigl{(}\phi_{\sharp}(\mu)\bigr{)}(I)$ $\displaystyle=\phi_{\sharp}(\mu)\bigl{(}\{z\in\mathcal{Z}_{\phi}\mid f(z)\in I \}\bigr{)}$ $\displaystyle=\mu\bigl{(}G\in\mathcal{G}\mid f(\phi(G))\in I\bigr{)}.$ $$

The equality then follows from $f\circ\phi=\phi^{\prime}$.
We assume that $f$ is Lipschitz-continuous with $\mathsf{Lip}(f)<\infty$ (otherwise the inequality is satisfied by default and there is nothing to prove).
We show that

$$ $\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}} {\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space $}}}(\phi_{\sharp}^{\prime}(\mu),\phi_{\sharp}^{\prime}(\nu)\mathclose{{\hbox{ $\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5. 66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})\leq\operatorname{Lip}(f) \cdot\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998 pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right. \n@space$}}}(\phi_{\sharp}(\mu),\phi_{\sharp}(\nu)\mathclose{{\hbox{$\left missing \mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{ \vbox to4.04999pt{}}\right.\n@space$}}}).$ $$

Let $L_{1}(\mathcal{Z}_{\phi})$ be the set of 1-Lipschitz functions on $\mathcal{Z}_{\phi}$. We use the Kantorovich-Rubinstein dual form of $\mathcal{W}_{1}$, as follows:

$$ $\displaystyle\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to 8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}} \right.\n@space$}}}(\phi_{\sharp}(\mu),\phi_{\sharp}(\nu)\mathclose{{\hbox{$ \left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.6 6995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})$ $\displaystyle=\sup_{g\in L_{1}(\mathcal{Z}_{\phi})}\mathbb{E}_{z\sim\phi_{ \sharp}(\mu)}[g(z)]-\mathbb{E}_{z\sim\phi_{\sharp}(\nu)}[g(z)]$ $\displaystyle=\sup_{g\in L_{1}(\mathcal{Z}_{\phi})}\int_{\mathcal{Z}_{\phi}}g( z)\;\mathrm{d}(\phi_{\sharp}(\mu)-\phi_{\sharp}(\nu))(z).$ $$

Note that if $g\in L_{1}(\mathcal{Z})$ then $\frac{1}{\mathsf{Lip}(f)}f\circ g\in L_{1}(\mathcal{Z})$ as well. Then, using our earlier observation about pushforward distributions,

$$ $\displaystyle\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to 8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}} \right.\n@space$}}}(\phi_{\sharp}^{\prime}(\mu),\phi_{\sharp}^{\prime}(\nu) \mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.099 98pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})$ $\displaystyle=\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to 8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}} \right.\n@space$}}}(f_{\sharp}\bigl{(}\phi_{\sharp}(\mu)\bigr{)},f_{\sharp} \bigl{(}\phi_{\sharp}(\nu)\bigr{)}\mathclose{{\hbox{$\left missing\mathchoice{ \vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999 pt{}}\right.\n@space$}}})$ $\displaystyle=\sup_{g\in L_{1}(\mathcal{Z}_{\phi^{\prime}})}\int_{\mathcal{Z}_ {\phi^{\prime}}}g(z)\>\mathrm{d}\mathopen{{\hbox{$\left missing\mathchoice{ \vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999 pt{}}\right.\n@space$}}}(f_{\sharp}\bigl{(}\lambda_{\sharp}(\mu)\bigr{)}-f_{ \sharp}\bigl{(}\lambda_{\sharp}(\nu)\bigr{)}\mathclose{{\hbox{$\left missing \mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{ \vbox to4.04999pt{}}\right.\n@space$}}})(z)$ $\displaystyle=\sup_{g\in L_{1}(\mathcal{Z}_{\phi^{\prime}})}\int_{\mathcal{Z}_ {\phi^{\prime}}}g(z)\>\mathrm{d}f_{\sharp}(\phi_{\sharp}(\mu)-\phi_{\sharp}( \nu))(z)$ $\displaystyle=\sup_{g\in L_{1}(\mathcal{Z}_{\phi^{\prime}})}\int_{\mathcal{Z}_ {\phi^{\prime}}}g\circ f(z)\>\mathrm{d}(\phi_{\sharp}(\mu)-\phi_{\sharp}(\nu)) (z)$ $\displaystyle=\mathsf{Lip}(f)\sup_{g\in L_{1}(\mathcal{Z}_{\phi^{\prime}})} \int_{\mathcal{Z}_{\phi^{\prime}}}\frac{g\circ f(z)}{\mathsf{Lip}(f)}\mathrm{d }(\mu-\nu)(z)$ $\displaystyle\leq\mathsf{Lip}(f)\sup_{h\in L_{1}(\mathcal{Z}_{\phi})}\int_{ \mathcal{Z}_{\phi}}h(x)\>\mathrm{d}(\mu-\nu)(z)$ $\displaystyle=\mathsf{Lip}(f)\cdot\mathcal{W}_{1}\mathopen{{\hbox{$\left missing \mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{ \vbox to4.04999pt{}}\right.\n@space$}}}(\phi_{\sharp}(\mu),\phi_{\sharp}(\nu) \mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.099 98pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}}),$ $$

as desired.
∎

###### Proof.

###### Proof.

###### Proof.

## Appendix C Proofs and details of Section 5

We start by restating Theorem 2 from Chuang et al. [] using encoders $\phi$ from some general set $\mathcal{X}$ to $\mathcal{Z}$.

###### Theorem C.1 (Theorem 2 in Chuang et al. [ 2021 ] ) .

Fix $\gamma>0$ and an encoder $\phi:\mathcal{X}\to\mathcal{Z}$. Then, for every distribution $\mu$ on $\mathcal{X}\times\mathcal{Y}$, for every predictor $\psi=(\psi_{y})_{i\in\mathcal{Y}}$ and every $\delta\in(0,1)$, with probability at least $1-\delta$ over all choices of $\mathcal{S}\sim\mu^{m}$, we have that the generalization gap $R_{\mu}(\psi\circ\phi)-\hat{R}_{\gamma,\mathcal{S}}(\psi\circ\phi)$ is upper bounded by

$$ $\mathbb{E}_{c\sim\mu_{y}}\left[\frac{\operatorname{Lip}\left(\rho_{\psi}(\cdot ,c)\right)}{\gamma}\mathbb{E}_{T,\tilde{T}\sim\mu_{c}^{m_{c}}}\left[\mathcal{W }_{1}\bigl{(}\phi_{\sharp}(\mu_{c,T}),\phi_{\sharp}(\mu_{c,\tilde{T}})\bigr{)} \right]\right]+\sqrt{\frac{\log(1/\delta)}{2m}},$ $$

where for each $c\in\mathcal{Y}$, $m_{c}$ denotes the number of pairs $(X,c)$ in $\mathcal{S}$. Also, recall that for $T\sim\mu_{c}^{m_{c}}$,
$\mu_{c,T}$ is the empirical distribution $\mu_{c,T}:=\sum_{X\in T}\delta_{X}$; similarly for $\mu_{c,\tilde{T}}$.

To obtain Theorem 5.1 we replace $\mathcal{X}$ by $\mathcal{G}$ and consider graph encoders $\phi:\mathcal{G}\to\mathcal{Z}_{\phi}$ and $\lambda:\mathcal{G}\to\mathcal{Z}_{\lambda}$ such that $\lambda$ upper bounds $\phi$ in expressive power. Then, Lemma 4.2 ensures the existence of $f$ such that $\phi=f\circ\lambda$ and Proposition 4.4 consequently implies
$\mathcal{W}_{1}\mathopen{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}
{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space
$}}}(\phi_{\sharp}^{\prime}(\mu_{c,T}),\phi_{\sharp}^{\prime}(\mu_{c,\tilde{T}
})\mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998pt{}}{\vbox to8.0
9998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.\n@space$}}})\leq
\operatorname{Lip}(f)\cdot\mathcal{W}_{1}\mathopen{{\hbox{$\left missing
\mathchoice{\vbox to8.09998pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{
\vbox to4.04999pt{}}\right.\n@space$}}}(\phi_{\sharp}(\mu_{c,T}),\phi_{\sharp}
(\mu_{c,\tilde{T}})\mathclose{{\hbox{$\left missing\mathchoice{\vbox to8.09998
pt{}}{\vbox to8.09998pt{}}{\vbox to5.66995pt{}}{\vbox to4.04999pt{}}\right.
\n@space$}}})$ for any $T,\tilde{T}\sum\mu_{c}^{m_{c}}$. Plugging this into the bound above results in the bound given in Theorem 5.1.

While the bound in Theorem 5.1 is theoretically useful, the expectation term over $T,\tilde{T}\sim\mu_{c}^{m_{c}}$ is intractable in general. To address this drawback, we derive another bound in Theorem C.2, which can be computed via sampling in practice and is the one used in our experiments.

###### Theorem C.2 .

Let $\{T^{j},\tilde{T}^{j}\}_{j=1}^{n}$ be $n$ pairs of graph samples where each $T^{j},\tilde{T}^{j}\sim\mu^{\lfloor m_{c}/2n\rfloor}_{c}$, $m=\sum_{c=1}^{K}\lfloor m_{c}/2n\rfloor$,
and $\Delta(\cdot)$ be the diameter of a space. For any Lipschitz continuous function $f:\mathcal{Z}_{\phi}\to\mathcal{Z}_{\lambda}$ such that $\phi=f\circ\lambda$, with probability at least $1-\delta$ for samples $\mathcal{S}\sim\mu^{m}$, we have

$$ $\displaystyle R_{\mu}(\psi\circ\phi)-\hat{R}_{\gamma,\mathcal{S}}(\psi\circ \phi)\leq\sqrt{\frac{\log(2/\delta)}{2m}}+$ $\displaystyle\mathbb{E}_{c\sim\sigma}\left[\frac{\operatorname{Lip}\left(\rho_ {\psi}(\cdot,c)\right)\operatorname{Lip}(f)}{\gamma}\left(\frac{1}{n}\sum_{j=1 }^{n}\mathcal{W}_{1}\bigl{(}\lambda_{\sharp}(\mu_{c,T^{j}}),\lambda_{\sharp}({ \mu}_{c,\tilde{T}^{j}})\bigr{)}+2\Delta(\lambda_{\sharp}(\mu_{c}))\sqrt{\frac{ \log(2K/\delta)}{n\lfloor m_{c}/2n\rfloor}}\right)\right].$ $$

The proof is again a consequence of Lemma 4.2 and Proposition 4.4, but this time relying on Corollary 6 in Chuang et al. []. We note that the diameter will be bounded when $B$-bounded graph encoders are considered.

###### Theorem C.1 (Theorem 2 in Chuang et al. [ 2021 ] ) .

###### Theorem C.2 .

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 4
questions: Is it possible to do a priori analysis on how a specific message aggregation scheme, e.g., k-WL or attention, can quantitively influence the terms in the bound?
soundness: 3
strengths: 1. The proposed framework is empirically effective to estimate the generalization of GNNs. The bound is given by the regularity and expressivity of the graph encoder itself, which are accessible measures in practice. This method has the potential to assess the generalization of real-world models.

2. The presentation of the paper is clean and easy to follow.
summary: This work present a practical framework for analyzing the generalization property of GNNs via their expressivity. The main result is an extension of previous works to graph representation learning. The theoretical results are accompanied by case studies and experimental verifications to demonstrate the tightness of the proposed bound.
weaknesses: 1. The verification and discussion are limited to MPNNs. Empirically, higher-order information and/or attention mechanism are widely adopted to enhance GNNs. While the results in this work are applicable to these models in a straightforward way, it does not provide insights how these architectures improve the B/S constants in the bound.

---

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: See the comments above.
soundness: 3
strengths: Overall the theoretical result is sound and help gain better understanding for how expressivity is related to generalization in GNNs. The paper is clearly written, with rigorous math notations and theorems. While not checking the proof, I feel the results are correct and make sense to me. The concrete example is quite interesting and makes this paper potentially useful in practice.
summary: This paper studies the generalization of graph neural networks. The paper is based on three technical tools. The first one is margin bound, which is common in deriving generalization bound. The second is the Wasserstein distance, which is implicitly linked to the Lipschitzness property of a neural network. The third one, which is the major novelty in this paper, it to bridge between a continuous GNN to a discrete graph embedding that has more separation power, such as 1-WL or homomorphism vectors. Combining the three types of techniques yields a meaningful generalization bound. The authors gave a concrete case showing that how improved expressivity benefits/harms generalization, which is quite interesting. They also conduct experiments to support the theory.
weaknesses: One major weekness of this paper is that, similar to most of the previous generalization papers, the bound given by this paper is quite sophisticated, intractable to compute, and potentially lose. It may be hard to draw the conclusion that this **upper bound** indeed reflects the real case. Another weakness is the readability. The authors did not explain the insights behind this bound clearly. I think it's not intuitive to understand why generalization error can be bounded by the discrete classifier with better separation power. What does the better separation power play a role in the bound? Also, why do the authors rely on Wasserstein distance? I should acknowledge that I am not an expert in generalization theory, but it would be nice to make the paper friendly to the expressivity community as well.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: Q1: In the experiments you only compare your bounds with VC bounds. It is encouraged to also compare with PAC-Bayesian bounds presented in (Ju et al. 2023).

Q2: Providing some examples to illustrate some definitions could help the readers better understand their meanings, e.g., the definitions of homomorphism and graph invariant in Section 3. You could add some figures to demonstrate these definitions more intuitively.
soundness: 3
strengths: The problem studied in this paper is pivotal in the theory of GNNs, i.e., revealing the connection between expressivity and generalization ability of GNNs. This paper makes a non-trivial progress in this problem, and the derived results are convinced and sound. As for the generalization bounds for GNNs in graph classification task, the derived results extend previous results (Chuang et al. 2021) to graph data, which also provides a new perspective to understand the generalization ability of GNNs. And, the experimental results show that the numerical values of the derived empirical bounds are significant sharper than that of VC bounds.
summary: The authors in this paper study the connection between expressivity and generalization ability of GNNs in the graph classification task. They first show that the generalization ability of a graph encoder can be upper bounded by that of graph more encoders with higher expressivity . After that, they establish generalization bounds for GNNs from a optimal transfer viewpoint. Concretely, the derived bound contain the Wasserstein distance of embeddings between two random subsets sampled from training nodes within the same class. In this way, the generalization ability of GNNs in the graph classification task can be depicted by the concentration of intra-class embeddings and the separation of inter-class embeddings. Finally, the authors also provide empirical bounds that adopting sampling to computed the Wasserstein distance and verify their theoretic results on real-world datasets. The experimental results generally support their derived bounds.
weaknesses: - The derived generalization bounds for GNNs contain the Lipschitz constants of graph encoders. These constants could be easily evaluated for shallow models, e.g., GCN with two or three layers, yet it may be difficulty to evaluate for more complex GNNs such as graph transformer [1,2].
- The derived bounds do not take the optimization into consideration and thus could not reflect the impact of optimization algorithms on generalization ability. It has been shown that the training trajectory could also affect the generalization ability of models [3]. Although this observation is not yet verified for GNNs, I believe that improving the generalization bounds by involving the analysis of training dynamics could be a promising direction.


[1] Chen et al., Structure-Aware Transformer for Graph Representation Learning. ICML 2022.

[2] Rampášek et al., Recipe for a General, Powerful, Scalable Graph Transformer. NeurIPS 2022.

[3] Fu et al., Learning Trajectories are Generalization Indicators. NeurIPS 2023.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: In Figure 2 (a), why not show both the loss gap and the bound for each dataset? In Figure 2 (b), why not show both $W_1$ and $Lip(f)$ for each dataset?
soundness: 2
strengths: 1. Some theoretical results seem solid and strong.
2. The problem to study is very interesting and important to the community to bridge the gap between the generalization and expressive power of GNN.
summary: This paper theoretically studies the relationship between the generalization of GNN and the expressive power of GNN in terms of the variance of graph structures. This work especially shows a trade-off between intra-class concentration and inter-class separation, which is the key insight into the generalization gap. Some experiments are conducted to support the theory.
weaknesses: 1. The theoretical proof is not complete. The proof of Corollary 4.5 and Proposition 5.2 are missing.
2. The paper is full of equations. Section 3 is not easy to follow when introducing so many terminologies with any examples. 
3. The practical insights are limited. Section 6 is quite confusing. This section shows that different graphs can lead to opposite effects on inter-class separation by the expressive power. Then, what should we do in practice based on this knowledge? Moreover, Section 6 seems irrelevant to intra-class concentration. Then, is the case study of Section 6 meaningful enough?
4. Results in Table 2 look weak. I cannot see that the proposed bound is better than VC bound in simulating the trend of accuracy or loss gap. 

Minor: 1. at the end of line 482, it shall be "Figure 2(b)" instead of "Figure 2(a)"?
2. Please resize Figure 2 to make it look nicer.

Meta-review: This paper studies expressivity and generalization of graph neural networks (GNNs). The generalization bound is characterized by the ratio between the concentration of intra-class embeddings and the separation of inter-class embeddings measured by the Wasserstein distance. The theoretical analysis is supported by some numerical experiments.   

The analysis is novel and provides a non-trivial insight. The analysis successfully characterizes the relation between expressivity and generalization with solid and convincing mathematical reasoning. This is a new perspective in the literature.    
Moreover, the empirical experiments well justifies the theoretical analysis, for example, the derived bounds are well aligned with the actual loss more than the VC bounds.    

For these reasons, this paper can be accepted by ICLR.

Decision: Accept (Poster)