SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Similarly to the classical case, Theorem 3.4 is interpreted as follows. While deterministic graph-signals may seem intricate and complex, they are actually regular, and "look like" random graph-signals that were sampled from SBMs, where the number of blocks of the SBM only depends on the desired approximation error between the SBM and the graph-signal, and not on the graph-signal itself.
Remark 3.5. The lower bound  $n \geq 2^{\lceil \frac{9c}{4\epsilon^2} \rceil}$  on the number of steps in the graphon-signal regularity lemma is essentially tight in the following sense. There is a universal constant  $C$  such that for every  $\epsilon > 0$  there exists a graphon-signal  $(W, f)$  such that no step graphon-signal  $(W', f')$  with less than  $2^{\lceil \frac{C}{\epsilon^2} \rceil}$  steps satisfies  $\delta_{\square}((W, f), (W', f')) \leq \epsilon$ . To see this, [8, Theorem 1.4, Theorem 7.1] shows that the bound in the standard weak regularity lemma (for graphs/graphons) is essentially tight in the above sense. For the graphon-signal case, we can take the graphon  $W'$  from [8, Theorem 7.1] which does not allow a regularity partition with less than  $2^{\lceil \frac{C}{\epsilon^2} \rceil}$  steps, and consider the graphon-signal  $(W', 1)$ , which then also does not allow such a regularity partition.
3.4 Compactness of the graphon-signal space and its covering number
We prove that  $\widetilde{\mathcal{WL}_r}$  is compact using Theorem 3.4, similarly to [25, Lemma 8]. Moreover, we can bound the number of balls of radius  $\epsilon$  required to cover  $\widetilde{\mathcal{WL}_r}$ .
Theorem 3.6. The metric space  $(\widetilde{\mathcal{WL}_r},\delta_\square)$  is compact. Moreover, given  $r > 0$  and  $c > 1$ , for every sufficiently small  $\epsilon >0$ , the space  $\widetilde{\mathcal{WL}_r}$  can be covered by
$$
\kappa (\epsilon) = 2 ^ {k ^ {2}} \tag {7}
$$
balls of radius  $\epsilon$ , where  $k = \lceil 2^{\frac{9c}{4\epsilon^2}}\rceil$
The Proof of Theorem 3.6 is given in Appendix C. This is a powerful result – the space of arbitrarily large graph-signals is dense in the “small” space  $\widetilde{\mathcal{WL}_r}$ . We will use this property in Section 4.3 to prove a generalization bound for MPNNs.
3.5 Graphon-signal sampling lemmas
In this section we prove that randomly sampling a graphon signal produces a graph-signal that is close in cut distance to the graphon signal. Let us first describe the sampling setting. More details on the construction are given in Appendix D.1. Let  $\Lambda = (\lambda_1, \dots, \lambda_k) \in [0, 1]^k$  be  $k$  independent uniform random samples from  $[0, 1]$ , and  $(W, f) \in \mathcal{WL}_r$ . We define the random weighted graph  $W(\Lambda)$  as the weighted graph with  $k$  nodes and edge weight  $w_{i,j} = W(\lambda_i, \lambda_j)$  between node  $i$  and node  $j$ . We similarly define the random sampled signal  $f(\Lambda)$  with value  $f_i = f(\lambda_i)$  at each node  $i$ . Note that  $W(\Lambda)$  and  $f(\Lambda)$  share the sample points  $\Lambda$ . We then define a random simple graph as follows. We treat each  $w_{i,j} = W(\lambda_i, \lambda_j)$  as the parameter of a Bernoulli variable  $e_{i,j}$ , where  $\mathbb{P}(e_{i,j} = 1) = w_{i,j}$  and  $\mathbb{P}(e_{i,j} = 0) = 1 - w_{i,j}$ . We define the random simple graph  $\mathbb{G}(W, \Lambda)$  as the simple graph with an edge between each node  $i$  and node  $j$  if and only if  $e_{i,j} = 1$ .
We note that, given a graph signal  $(G,\mathbf{f})$ , sampling a graph-signal from  $(W,f)_{(G,\mathbf{f})}$  is equivalent to subsampling the nodes of  $G$  independently and uniformly (with repetitions), and considering the resulting subgraph and subsignal. Hence, we can study the more general case of sampling a graphon-signal, where graph-signal sub-sampling is a special case. We now extend [24, Lemma 10.16], which bounds the cut distance between a graphon and its sampled graph, to the case of a sampled graphon-signal.
Theorem 3.7 (Sampling lemma for graphon-signals). Let  $r > 1$ . There exists a constant  $K_0 > 0$  that depends on  $r$ , such that for every  $k \geq K_0$ , every  $(W, f) \in \mathcal{WL}_r$ , and for  $\Lambda = (\lambda_1, \dots, \lambda_k) \in [0,1]^k$  independent uniform random samples from  $[0,1]$ , we have
$$
\mathbb {E} \left(\delta_ {\square} \left(\left(W, f\right), \left(W (\Lambda), f (\Lambda)\right)\right)\right) <   \frac {1 5}{\sqrt {\log (k)}},
$$
$$
\mathbb {E} \left(\delta_ {\square} \left(\left(W, f\right), \left(\mathbb {G} (W, \Lambda), f (\Lambda)\right)\right)\right) <   \frac {1 5}{\sqrt {\log (k)}}.
$$
The proof of Theorem 3.7 is given in Appendix D.2
4 Graphon-signal analysis of MPNNs
In this section, we propose utilizing the compactness of the graphon-signal space under cut distance, and the sampling lemma, to prove regularity results for MPNNs, uniform generalization bounds, and stability to subsampling theorems.
4.1 MPNNs on graphon signals
Next, we define MPNNs on graphon-signals, in such a way that the application of a MPNN on an induced graphon-signal is equivalent to applying the MPNN on the graph-signal and then inducing it. A similar construction was presented in [26], for average aggregation, but we use normalized sum aggregation.
At each layer, we define the message function  $\Phi(x,y)$  as a linear combination of simple tensors as follows. Let  $K \in \mathbb{N}$ . For every  $k \in [K]$ , let  $\xi_{\mathrm{r}}^{k}, \xi_{\mathrm{t}}^{k}: \mathbb{R}^{d} \to \mathbb{R}^{p}$  be Lipschitz continuous functions that we call the receiver and transmitter message functions respectively. Define the message function  $\Phi: \mathbb{R}^{2d} \to \mathbb{R}^{p}$  by
$$
\Phi (a, b) = \sum_ {k = 1} ^ {K} \xi_ {\mathrm {r}} ^ {k} (a) \xi_ {\mathrm {t}} ^ {k} (b),
$$
where the multiplication is elementwise along the feature dimension. Given a signal  $f$ , define the message kernel  $\Phi_f:[0,1]^2\to \mathbb{R}^p$  by
$$
\Phi_ {f} (x, y) = \Phi (f (x), f (y)) = \sum_ {k = 1} ^ {K} \xi_ {\mathrm {r}} ^ {k} (f (x)) \xi_ {\mathrm {t}} ^ {k} (f (y)).
$$
We see the  $x$  variable of  $\Phi_f(x,y)$  as the receiver of the message, and  $y$  as the transmitter. Define the aggregation of a message kernel  $Q:[0,1]^2\to \mathbb{R}^p$ , with respect to the graphon  $W\in \mathcal{W}_0$ , to be the signal  $\mathrm{Agg}(W,Q)\in \mathcal{L}_r^\infty [0,1]$ , defined by
$$
\operatorname {A g g} (W, Q) (x) = \int_ {0} ^ {1} W (x, y) Q (x, y) d y,
$$
for an appropriate  $r > 0$ . A message passing layer (MPL) takes the form  $f^{(t)} \mapsto \mathrm{Agg}(W, \Phi_{f^{(t)}}^{(t + 1)})$ , where  $f^{(t)}$  is the signal at layer  $t$ . Each MPL is optionally followed by an update layer, which updates the signal pointwise via  $f^{(t + 1)} = \mu^{(t + 1)}\big(f^{(t)}(x), \mathrm{Agg}(W, \Phi_{f^{(t)}}^{(t + 1)})(x)\big)$ , where  $\mu^{(t + 1)}$  is a learnable mapping called the update function. A MPNN is defined by choosing the number of layers  $T$ , and defining message and update functions  $\{\mu^t, (^t\xi_{\mathrm{r}}^k), (^t\xi_{\mathrm{t}}^k)\}_{k\in [K], t\in [T]}$ . A MPNN only modifies the signal, and keeps the graph/graphon intact. We denote by  $\Theta_t(W, f)$  the output of the MPNN applied on  $(W, f) \in \mathcal{WL}_r$  at layer  $t \in [T]$ . More details on the construction are given in Appendix E.1.
The above construction is rather general. Indeed, it is well known that many classes of functions  $F: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^C$  (e.g.,  $L^2$  functions) can be approximated by (finite) linear combinations of simple tensors  $F(a,b) \approx \sum_{k=1}^{K} \xi_1^k(a)\xi_2^k(b)$ . Hence, message passing based on general message functions  $\Phi: \mathbb{R}^{2d} \to \mathbb{R}^p$  can be approximated by our construction. Moreover, many well-known MPNNs can be written using our formulation with a small  $K$ , e.g., [29, 36] and spectral convolutional networks [9, 20, 21], if we replace the aggregation in these method with normalized sum aggregation.
In Appendix E.1 we show that for any graph-signal  $(G,\mathbf{f})$ , we have  $\Theta_t(W,f)_{(G,\mathbf{f})} = (W,f)_{\Theta_t(G,\mathbf{f})}$  where the MPNN on a graph-signal is defined with normalized sum aggregation
$$
\left(\operatorname {A g g} (G, \Phi_ {\mathbf {f}})\right) _ {i} = \frac {1}{n} \sum_ {j \in [ n ]} a _ {i, j} (\Phi_ {\mathbf {f}}) _ {i, j}.
$$
Here,  $n$  is the number of nodes, and  $\{a_{i,j}\}_{i,j\in [n]}$  is the adjacency matrix of  $G$ . Hence, we may identify graph-signals with their induced graphon-signals when analyzing MPNNs.
4.2 Lipschitz continuity of MPNNs
We now show that, under the above construction, MPNNs are Lipschitz continuous with respect to cut distance.
Theorem 4.1. Let  $\Theta$  be a MPNN with  $T$  layers. Suppose that there exist constants  $L, B > 0$  such that for every layer  $t \in [T]$ , every  $\mathrm{y} \in \{\mathrm{t}, \mathrm{r}\}$  and every  $k \in [K]$ ,
$$
\left| \mu^ {t} (0) \right|, \left| ^ {t} \xi_ {\mathrm {y}} ^ {k} (0) \right| \leq B, \quad a n d \quad L _ {\mu^ {t}}, L _ {t \xi_ {\mathrm {y}} ^ {k}} <   L,
$$
where  $L_{\mu^t}$  and  $L_{t\xi_y^k}$  are the Lipschitz constants of  $\mu^t$  and  ${}^t\xi_{\mathrm{y}}^k$ . Then, there exists a constant  $L_{\Theta}$  (that depends on  $T, K, B$  and  $L$ ) such that for every  $(W, f), (V, g) \in \mathcal{WL}_r$ ,
$$
\left\| \Theta (W, f) - \Theta (V, g) \right\| _ {\square} \leq L _ {\Theta} \Big (\left\| f - g \right\| _ {\square} + \left\| W - V \right\| _ {\square} \Big).
$$
The constant  $L_{\Theta}$  depends exponentially on  $T$ , and polynomially on  $K, B$  and  $L$ . For formulas of  $L_{\Theta}$  under different assumptions on the hypothesis class of the MPNN, we refer to Appendix F.
4.3 A generalization theorem for MPNN
In this section we prove a uniform generalization bound for MPNNs. For background on generalization analysis, we refer the reader to Appendix G.1. While uniform generalization bounds are considered a classical approach in standard neural networks, the approach is less developed in the case of MPNNs. For some works on generalization theorems of MPNNs, see [14, 23, 26, 30, 33].
When a MPNN is used for classification or regression,  $\Theta_T$  is followed by global pooling. Namely, for the output signal  $g:[0,1]\to \mathbb{R}^p$ , we return  $\int g(x)dx\in \mathbb{R}^p$ . This is then typically followed by a learnable mapping  $\mathbb{R}^p\rightarrow \mathbb{R}^C$ . In our analysis, we see this mapping as part of the loss, which can hence be learnable. The combined loss is assumed to be Lipschitz continuous<sup>2</sup>.
We model the ground truth classifier into  $C$  classes as a piecewise constant function  $\mathcal{C}:\widehat{\mathcal{WL}_r}\to$ $\{0,1\} ^C$ , where the sets of different steps in  $\widetilde{\mathcal{W}\mathcal{L}_r}$  are Borel measurable sets, correspond to different classes. We consider an arbitrary probability Borel measure  $\nu$  on  $\widetilde{\mathcal{WL}_r}$  as the data distribution. More details on the construction are given in Appendix G.2.
Let  $\mathrm{Lip}(\widetilde{\mathcal{WL}_r}, L_1)$  be the space of Lipschitz continuous mappings  $\Upsilon: \widetilde{\mathcal{WL}_r} \to \mathbb{R}^C$  with Lipschitz constant  $L_1$ . By Theorem 4.1, we may assume that our hypothesis class of MPNNs is a subset of  $\mathrm{Lip}(\widetilde{\mathcal{WL}_r}, L_1)$  for some given  $L_1$ . Let  $\mathbf{X} = (X_1, \ldots, X_N)$  be independent random samples from the data distribution  $(\widetilde{\mathcal{WL}_r}, \nu)$ . Let  $\Upsilon_{\mathbf{X}}$  be a model that may depend on the sampled data, e.g., via training. Let  $\mathcal{E}$  be a Lipschitz continuous loss function with Lipschitz constant  $L_2$ . For every function  $\Upsilon$  in the hypothesis class  $\mathrm{Lip}(\widetilde{\mathcal{WL}_r}, L_1)$  (i.e.  $\Upsilon_{\mathbf{X}}$ ), define the statistical risk
$$
\mathcal {R} (\Upsilon) = \mathbb {E} \bigl (\mathcal {E} (\Upsilon , \mathcal {C}) \bigr) = \int \mathcal {E} (\Upsilon (x), \mathcal {C} (x)) d \nu (x).
$$
We define the empirical risk  $\hat{\mathcal{R}} (\Upsilon_{\mathbf{X}},\mathbf{X}) = \frac{1}{N}\sum_{i = 1}^{N}\mathbb{E}\bigl {(}\Upsilon_{\mathbf{X}}(X_i),\mathcal{C}(X_i)\bigr).$
Theorem 4.2 (MPNN generalization theorem). Consider the above classification setting, and let  $L = L_{1}L_{2}$ . Let  $X_{1},\ldots ,X_{N}$  be independent random samples from the data distribution  $(\widetilde{\mathcal{WL}_r},\nu)$ . Then, for every  $p > 0$ , there exists an event  $\mathcal{U}^p\subset \widetilde{\mathcal{WL}_r}^N$ , with probability
$$
\nu^ {N} (\mathcal {U} ^ {p}) \geq 1 - C p - 2 \frac {C ^ {2}}{N},
$$
$$
\left| \mathcal {R} \left(\Upsilon_ {\mathbf {X}}\right) - \hat {\mathcal {R}} \left(\Upsilon_ {\mathbf {X}}, \mathbf {X}\right) \right| \leq \xi^ {- 1} (N / 2 C) \left(2 L + \frac {1}{\sqrt {2}} \left(L + \mathcal {E} (0, 0)\right) \left(1 + \sqrt {\log (2 / p)}\right)\right), \tag {8}
$$
where  $\xi (\epsilon) = \frac{\kappa(\epsilon)^2\log(\kappa(\epsilon))}{\epsilon^2}$ ,  $\kappa$  is the covering number of  $\widetilde{\mathcal{WL}_r}$  given in (7), and  $\xi^{-1}$  is the inverse function of  $\xi$ .
The theorem is proved in Appendix G.4. Note that the term  $\xi^{-1}(N / 2C)$  in (8) decreases to zero as the size of the training set  $N$  goes to infinity.
In Table 1 we compare the assumptions and dependency on the number of data points of different generalization theorems. All past works consider special assumptions. Our work provides upper bounds under no assumptions on the data distribution, and only mild assumptions on the MPNN (Lipschitz continuity of the message passing and update functions). In Table 2 in Appendix G.5 we present experiments that illustrate the generalization capabilities of MPNNs with normalized sum aggregation.
4.4 Stability of MPNNs to graph-signal subsampling
When working with very large graphs, it is often the practice to subsample the large graph, and apply a MPNN to the smaller subsampled graph [5, 7, 16]. Here, we show that such an approach is justified
2We note that loss functions like cross-entropy are not Lipschitz continuous. However, the composition of cross-entropy on softmax is Lipschitz, which is the standard way of using cross-entropy.
3The loss  $\mathcal{E}$  may have a learnable component (that depends on the dataset  $\mathbf{X}$ ), as long as the total Lipschitz bound of  $\mathcal{E}$  is  $L_{2}$ .
Table 1: Comparison of the assumptions made by different GNN generalization analysis papers.
Generalization analysis paper	Assumption on the graphs	No weight sharing	General MPL	Dependency on N
Generalization Limits of GNNs [14]	bounded degree	X	X	N-1/2
PAC-bayesian MPNN [23]	bounded degree	X	X	N-1/2
PAC-bayesian GCN [23]	bounded degree	✓	X	N-1/2
VC meets 1WL [30]	bounded color complexity	✓	X	N-1/2
Generalization Analysis of MPNNs [26]	sampled from a small set of graphons	✓	✓	N-1/2
Our graphon-signal theory	non	✓	✓	ξ-1(N)
theoretically. Namely, any (Lipschitz continuous) MPNN has approximately the same outcome on the large graph and its subsampled version.
Transferability and stability analysis [18, 22, 27, 31, 32] often studies a related setting. Namely, it is shown that a MPNN applied on a randomly sampled graph  $G$  approximates the MPNN on the graphon  $W$  from which the graph is sampled. However, previous analyses assumed that the generating graphon  $W$  has metric properties. Namely, it is assumed that there is some probability metric space  $\mathcal{M}$  which is the graphon domain, and the graphon  $W: \mathcal{M} \times \mathcal{M} \to [0,1]$  is Lipschitz continuous with respect to  $\mathcal{M}$ , where the dimension of  $\mathcal{M}$  affects the asymptotics. This is an unnatural setting, as general graphons are only assumed to be measurable, not continuous. Constraining the construction to Lipschitz continuous graphons with a uniformly bounded Lipschitz constant only accounts for a small subset of  $\mathcal{W}\mathcal{L}_r$ , and, hence, limits the analysis significantly. In comparison, our analysis applies to any graphon-signal in  $\mathcal{W}\mathcal{L}_r$ . When we only assume that the graphon is measurable,  $[0,1]$  is only treated as a standard (atomless) probability space, which is very general, and equivalent for example to  $[0,1]^d$  for any  $d \in \mathbb{N}$ , and to any Polish space. Note that graphon theory allows restricting the graphon domain to  $[0,1]$  since  $[0,1]$ , as a measure space, is very generic.
Theorem 4.3. Consider the setting of Theorem 4.2, and let  $\Theta$  be a MPNN with Lipschitz constant  $L$ . Denote
$$
\Sigma = \left(W, \Theta (W, f)\right), \quad a n d \quad \Sigma (\Lambda) = \left(\mathbb {G} (W, \Lambda), \Theta \bigl (\mathbb {G} (W, \Lambda), f (\Lambda) \bigr)\right).
$$
$$
\mathbb {E} \Big (\delta_ {\square} \big (\Sigma , \Sigma (\Lambda) \big) \Big) <   \frac {1 5}{\sqrt {\log (k)}} L.
$$
We presented an extension of graphon theory to a graphon-signal theory. Especially, we extended well-known regularity, compactness, and sampling lemmas from graphons to graphon-signals. We then showed that the normalized sum aggregation of MPNNs is in some sense compatible with the graphon-signal cut distance, which leads to the Lipschitz continuity of MPNNs with respect to cut distance. This then allowed us to derive generalization and sampling theorems for MPNNs. The strength of our analysis is in its generality and simplicity-- it is based on a natural notion of graph similarity, that allows studying the space of all graph-signals, it applies to any graph-signal data distribution, and does not impose any restriction on the number of parameters of the MPNNs, only to their regularity through the Lipschitzness of the message functions.
The main limitation of the theory is the very slow asymptotics of the generalization and subsampling theorems. This follows the fact that the upper bound on the covering number of the compact space  $\widetilde{\mathcal{WL}_r}$  grows faster than the covering number of any finite-dimensional compact space. Yet, we believe that our work can serve as a point of departure for future works, that 1) will model subspaces of  $\widetilde{\mathcal{WL}_r}$  of lower complexity, which approximate the support of the data-distribution in real-life settings of graph machine learning, and, 2) will lead to improved asymptotics. Another open problem is to find an essentially tight estimate of the covering number of  $\widetilde{\mathcal{WL}_r}$ , which may be lower than the estimate presented in this paper.
We thank Ningyuan (Teresa) Huang for providing the experiments of Table 2.
Ron Levie is partially funded by ISF (Israel Science Foundation) grant #1937/23: Analysis of graph deep learning using graphon theory.
A Basic definitions and properties of graphon-signals
In this appendix, we give basic properties of graphon-signals, cut norm, and cut distance.
A.1 Lebesgue spaces and signal spaces
For  $1 \leq p < \infty$ , the space  $\mathcal{L}^p[0,1]$  is the space of (equivalence classes up to null-set) of measurable functions  $f: [0,1] \to \mathbb{R}$ , with finite  $L_1$  norm
$$
\| f \| _ {p} = \left(\int_ {0} ^ {1} | f (x) | ^ {p} d x\right) ^ {1 / p} <   \infty .
$$
The space  $\mathcal{L}^{\infty}[0,1]$  is the space of (equivalence classes) of measurable functions with finite  $L_{\infty}$  norm
$$
\| f \| _ {\infty} = \operatorname {e s s} \sup  _ {x \in [ 0, 1 ]} | f (x) | = \inf  \left\{a \geq 0 \mid | f (x) | \leq a \text {f o r a l m o s t e v e r y} x \in [ 0, 1 ] \right\}.
$$
A.2 Properties of cut norm
Every  $f\in \mathcal{L}_r^\infty [0,1]$  can be written as  $f = f_{+} - f_{-}$ , where
$$
f _ {+} (x) = \left\{ \begin{array}{c c} f (x) & f (x) > 0 \\ 0 & f (x) \leq 0. \end{array} \right.
$$
and  $f_{-}$  is defined similarly. It is easy to see that the supremum in (3) is attained for  $S$  which is either the support of  $f_{+}$  or  $f_{-}$ , and
$$
\| f \| _ {\square} = \max  \{\| f _ {+} \| _ {1}, \| f _ {-} \| _ {1} \}.
$$
As a result, the signal cut norm is equivalent to the  $L_{1}$  norm
$$
\frac {1}{2} \| f \| _ {1} \leq \| f \| _ {\square} \leq \| f \| _ {1}. \tag {9}
$$
Moreover, for every  $r > 0$  and measurable function  $W:[0,1]^2\to [-r,r]$ ,
$$
0 \leq \| W \| _ {\square} \leq \| W \| _ {1} \leq \| W \| _ {2} \leq \| W \| _ {\infty} \leq r.
$$
The following lemma is from [24, Lemma 8.10].
Lemma A.1. For every measurable  $W:[0,1]^2\to \mathbb{R}$ , the supremum
$$
\sup  _ {S, T \subset [ 0, 1 ]} \left| \int_ {S} \int_ {T} W (x, y) d x d y \right|
$$
is attained for some  $S, T$ .
A.3 Properties of cut distance and measure preserving bijections
Recall that we denote the standard Lebesgue measure of  $[0,1]$  by  $\mu$ . Let  $S_{[0,1]}$  be the space of measurable bijections  $[0,1] \to [0,1]$  with measurable inverse, that are measure preserving, namely, for every measurable  $A \subset [0,1]$ ,  $\mu(A) = \mu(\phi(A))$ . Recall that  $S_{[0,1]}'$  is the space of measurable bijections between co-null sets of  $[0,1]$ .
For  $\phi \in S_{[0,1]}$  or  $\phi \in S_{[0,1]}'$ , we define  $W^{\phi}(x,y) \coloneqq W(\phi(x),\phi(y))$ . In case  $\phi \in S_{[0,1]}'$ ,  $W^{\phi}$  is only defined up to a null-set, and we arbitrarily set  $W$  to 0 in this null-set. This does not affect our analysis, as the cut norm is not affected by changes to the values of functions on a null sets. The cut-metric between graphons is then defined to be
$$
\begin{array}{l} \delta_ {\square} (W, W ^ {\phi}) = \inf  _ {\phi \in S _ {[ 0, 1 ]}} \| W - W ^ {\phi} \| _ {\square} \\ = \inf  _ {\phi \in S _ {[ 0, 1 ]}} \sup  _ {S, T \subseteq [ 0, 1 ]} \Big | \int_ {S \times T} \big (W (x, y) - W (\phi (x), \phi (y)) \big) d x d y \Big |. \\ \end{array}
$$
Remark A.2. Note that  $\delta_{\square}$  can be defined equivalently with respect to  $\phi \in S_{[0,1]}^{\prime}$ . Indeed, by [24, Equation (8.17) and Theorem 8.13],  $\delta_{\square}$  can be defined equivalently with respect to the measure preserving maps that are not necessarily invertible. These include the extensions of mappings from  $S_{[0,1]}^{\prime}$  by defining  $\phi(x) = 0$  for every  $x$  in the co-null set underlying  $\phi$ .
Similarly to the graphon case, the graphon-signal distance  $\delta_{\square}$  is a pseudo-metric. By introducing an equivalence relation  $(W,f)\sim (V,g)$  if  $\delta_{\square}((W,f),(V,g)) = 0$ , and the quotient space  $\widetilde{\mathcal{WL}_r} := \mathcal{WL}_r / \sim$ ,  $\widetilde{\mathcal{WL}_r}$  is a metric space with a metric  $\delta_{\square}$  defined by  $\delta_{\square}([W,f],[V,g]) = d_{\square}(W,V)$  where  $[(W,f)], [(V,g)]$ , are the equivalence classes of  $(W,f)$  and  $(V,g)$  respectively. By abuse of terminology, we call elements of  $\widetilde{\mathcal{WL}_r}$  also graphon-signals.
Remark A.3. We note that  $\widetilde{\mathcal{WL}_r} \neq \widetilde{\mathcal{W}_0} \times \widetilde{\mathcal{L}_r^\infty[0,1]}$  (for the natural definition of  $\widetilde{\mathcal{L}_r^\infty[0,1]}$ ), since in  $\widetilde{\mathcal{WL}_r}$  we require that the measure preserving bijection is shared between the graphon  $W$  and the signal  $f$ . Sharing the measure preserving bijection between  $W$  and  $f$  is an important modelling requirement, as  $\phi$  is seen as a "re-indexing" of the node set  $[0,1]$ . When re-indexing a node  $x$ , both the neighborhood  $W(x,\cdot)$  of  $x$  and the signal value  $f(x)$  at  $x$  should change together, otherwise, the graphon and the signal would fall out of alignment.
We identify graphs with their induced graphons and signal with their induced signals
B Graphon-signal regularity lemmas
In this appendix, we prove a number of versions of the graphon-signal regularity lemma, where Theorem 3.4 is one version.
B.1 Properties of partitions and step functions
Given a partition  $\mathcal{P}_k$  and  $d\in \mathbb{N}$ , the next lemma shows that there is an equiparition  $\mathcal{E}_n$  such that the space  $S_{\mathcal{E}_n}^d$  uniformly approximates the space  $S_{\mathcal{P}_k}^d$  in  $\mathcal{L}^1 [0,1]^d$  norm (see Definition 3.3).