SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Similar to how GSOs in a GNN act on graph signals, graphops act on some  $L^2$  signals (called graphop signals). The generalization is straightforward: replacing GSOs in the construction of the GNN in Eqn. (2) with graphops results in graphop convolution and replacing graph convolution with graphop convolution gives graphop neural networks.
Formally, fix a maximum order  $K \in \mathbb{N}$ . For some measure space  $(\Omega, \mathcal{B}, \mu)$ , select a graphop  $A: L^2(\Omega) \to L^2(\Omega)$  and a graphop signal  $X \in L^2(\Omega)$ . We define a graphop convolution operator as a weighted sum of at most  $K - 1$  applications of  $A$ :  $H(h, A)[X] := \sum_{k=0}^{K-1}(h_kA^k)[X]$ , where  $h \in \mathbb{R}^K$  are (learnable) filter parameters and  $A^k$  is the composition of  $k$  duplicates of  $A$ . The square bracket  $[v]_i$  indicates the  $i$ -th entry of a tuple  $v$ .
For some number of layers  $L \in \mathbb{N}$ ,  $\{n_i\}_{i \in [L]} \in \mathbb{N}$ ,  $n_0 := 1$ , define a graph hop neural network  $\Phi$  with  $L$  layers and  $n_i$  features in layer  $i$  as:
$$
\Phi (h, A, X) = X _ {L} (h, A, X), \tag {6}
$$
$$
X _ {l} (h, A, X) = \left[ \rho \left(\sum_ {g = 1} ^ {n _ {l - 1}} H \left(h _ {f, g} ^ {l}, A\right) \left[ X _ {l - 1} \right] _ {g}\right) \right] _ {f \in [ n _ {l} ]}, \quad l \in [ L ], \tag {7}
$$
$$
X _ {0} (h, A, X) = X \tag {8}
$$
with filter parameter tuple  $h = (h^{1},\dots,h^{L})$ ,  $h^l\in (\mathbb{R}^K)^{n_l\times n_{l - 1}}$  for any  $l\in [L]$ , and graphop signal tuple  $X_{l}\in (L^{2}(\Omega))^{n_{l}}$  for any  $l\in [L]\cup \{0\}$ . Eqn (7) and Eqn (2) are almost identical, with the only difference being the input/output space: graphops replacing finite graphs, and graphop signals replacing graph signals.
3.2 From graphop neural networks to finite graph neural networks
We are specifically interested in finite GNNs that are discretizations of a graphop (for instance finite grids as discretizations of infinite grids), so as to obtain a quantitative bound that depends on the resolution of discretization. To sample a GNN from a given graphop  $A: \mathcal{F} \to \mathcal{F}$ , we first sample a GSO and plug it into Eqn (2). Choose a resolution  $m \in \mathbb{N}$  and define the GSO  $A_{m}$ , for any graph signal  $X \in \mathcal{F}_{m}$  (defined in Section 2) as:
$$
A _ {m} X (v) := m \int_ {v - \frac {1}{m}} ^ {v} (A \widetilde {X}) \mathrm {d} \lambda , \quad v \in [ m ] / m, \tag {9}
$$
$$
\Phi_ {m} (h, A, X) := \Phi (h, A _ {m}, X), \tag {10}
$$
where graphop signal  $\tilde{X} \in \mathcal{F}$  is an extension of graph signal  $X \in \mathcal{F}_m$  defined as
$$
\widetilde {X} (u) := X \left(\frac {\lceil u m \rceil}{m}\right), \quad u \in [ 0, 1 ]. \tag {11}
$$
Shared structural properties of sampled graphs The discretization scheme introduced for graphop can be intuitively understood as partitioning the vertex set into finitely many sets and merging nodes and their connections in these sets, with an appropriate scaling of the edge weights. Imagine blurring an  $n \times n$  matrix into an  $n/2 \times n/2$  matrix by average-pooling over each disjoint  $2 \times 2$  square. As  $n \to \infty$  (and even for uncountable [0,1]), this makes rigorous the notion of making a high-resolution graph on a huge number of vertices more 'blurry' by merging nodes in a way that still maintains some smoothness conditions, which is reminiscent of the real-world procedures of training with low-resolution images before fine-tuning with higher-resolution ones, or sampling from low-frequency graph Fourier transform domain.
Note that if  $A$  is linear then  $A_{m}$  is necessarily linear, but our definition of graphop does not require linearity. Therefore,  $A_{m}$  is strictly more general than the matrix representation of graph shift operators. We have the following well-definedness result:
Lemma 1. If a graphop  $A: \mathcal{F} \to \mathcal{F}$  is self-adjoint, then for each resolution  $m \in \mathbb{N}$ , the discretization  $A_m: \mathcal{F}_m \to \mathcal{F}_m$  defined above is also self-adjoint.
The proof can be found in Appendix B. Compared to previous works, our discretization scheme in Eqn (9) looks slightly different. In Ruiz et al. (2023a), given a graphon  $W:[0,1]^2\to \mathbb{R}$ , the discretization at resolution  $n$  was defined by forming the matrix  $S\in \mathbb{R}^{n\times n}:S_{i,j} = W(i / n,j / n)$ . A related discretization scheme involving picking the interval endpoints at random was also used, but the resulting matrix still takes values at discrete points in  $W$ . These two sampling schemes rely crucially on their everywhere continuous assumptions for the graphon  $W$ . Indeed, but for continuity requirements, two functions that differ only at finite discrete points  $(i / n,j / n), i,j\in [n]$  are in the same  $L^2$  class of functions, but will give rise to completely different samples. Furthermore, not every  $L^2$  class of functions has a continuous representative. This means that our discretization scheme is strictly more general than that used by Ruiz et al. (2023a) even when restricted to graphons. This difference comes from the fact that we are discretizing an operator and not the graph itself. For our purpose, taking values at discrete points for some limiting object of sparse graphs will likely not work, since sparsity ensures that most discrete points are trivial.
4 Main result: Approximation and transferability
4.1 Results for  $P$ -operators
Our first set of theorems address approximation and transferability of  $P$ -operators: under certain regularity assumptions to be discussed later,  $P$ -operators are well approximated by their discretizations:
Theorem 2 (Approximation theorem). Let  $A: \mathcal{F} \to \mathcal{F}$  be a  $P$ -operator satisfying Assumption 2 with constant  $C_A$ ; Assumption 3.A or 3.B with resolutions in  $\mathcal{N}$ . Fix  $n \in \mathcal{N}$  and consider  $(k, C_v)$ -profiles. Let  $A_n: \mathcal{F}_n \to \mathcal{F}_n$  be a discretization of  $A$  as defined in Eqn (9). Then:
$$
d _ {M} (A, A _ {n}) \leq 2 \sqrt {\frac {C _ {A} C _ {v}}{n}} + \frac {C _ {v} + 1}{n}. \tag {12}
$$
Compared to theorems in Ruiz et al. (2023a), our explicit dependence on  $n$  has an extra  $n^{-1/2}$  term that stems from techniques used to bound the Lévy-Prokhorov distance between two entry distributions obtained from functions that differ by at most  $O(n^{-1})$  in  $L^2$  norm.
As an immediate corollary, invoking the triangle inequality yields a transferability bound.
Corollary 1 (Transferability). Let  $A: \mathcal{F} \to \mathcal{F}$  be a  $P$ -operator satisfying assumptions of Theorem 2 with constant  $C_A$  and resolutions  $\mathcal{N}$ . For any  $n, m \in \mathcal{N}$ , let  $A_n: \mathcal{F}_n \to \mathcal{F}_n$  and  $A_m: \mathcal{F}_m \to \mathcal{F}_m$  be discretizations as defined in Eqn (9). Then:
$$
d _ {M} \left(A _ {m}, A _ {n}\right) \leq \left(m ^ {- \frac {1}{2}} + n ^ {- \frac {1}{2}}\right) 2 \sqrt {C _ {A} C _ {v}} + \left(m ^ {- 1} + n ^ {- 1}\right) \left(C _ {v} + 1\right). \tag {13}
$$
We emphasize that these theorems work for general nonlinear  $P$ -operators and not only the linear graphops defined in (Backhausz and Szegedy, 2022).
Proof sketch The full proof of Theorem 2 is in Appendix D. To bound the distance in  $d_{M}$  between two operators, for each sample size  $k \in \mathbb{N}$ , we give a bound on the Hausdorff metric  $d_{H}$  between the two  $(k,C_v)$ -profiles. As long as the dependence on  $k$  of these bounds is polynomial, the infinite sum in the definition of  $d_{M}$  converges. We do this by picking an arbitrary distribution  $\overline{\eta}$  from  $S_{k,C_v}(A)$ , which by definition is given by a  $k$ -tuple  $F$  of functions in  $L^{\infty}_{\mathrm{reg}(C_v)}$ . Discretize each element of  $F$  and consider its entry distribution results in  $\overline{\eta}_n \in S_{k,C_v}(A_n)$ . We show that we can give an upper bound of  $d_{LP}(\overline{\eta},\overline{\eta}_n)$  that is independent of the choice of  $\overline{\eta}$  and thus same upper bound holds for  $\sup_{\eta \in S_{k,C_v}(A)}\inf_{\eta_n\in S_{k,C_v}(A_n)}d_{LP}(\eta ,\eta_n)$ . By also selecting an arbitrary element of  $S_{k,C_v}(A_n)$  and extending it to an element of  $S_{k,C_v}(A)$ , we obtain another upper bound for  $\sup_{\eta_n\in S_{k,C_v}(A_n)}\inf_{\eta \in S_{k,C_v}(A)}d_{LP}(\eta ,\eta_n)$  and thus for  $d_H$ . The different assumptions come in via different techniques used to bound  $d_{LP}$  by a high probability bound on the  $L^2$  norm of the functions in  $F$  and their discretization/extension.
4.2 Results for graphop neural networks
Not only are graphops and their discretizations close in  $d_{M}$ , but, as we show next, neural networks built from a graphop are also close to those built from graphop discretizations in  $d_{M}$ . We iterate that here we are comparing nonlinear operators (graphop neural networks) that are acting on different spaces  $(L^{2}([n] / n)$  for some finite  $n$  versus  $L^2 ([0,1])$ ).
Before stating theoretical guarantees for graphop neural networks, let us introduce some assumptions on the neural network activation function and parameters:
Assumption 1. Let the activation function  $\rho : \mathbb{R} \to \mathbb{R}$  in the definition of graphop neural networks be 1-Lipschitz and the convolution parameters  $h$  be such that  $|h| \leq 1$  element-wise.
Theorem 3 (Graphop neural network discretization). Let  $A: \mathcal{F} \to \mathcal{F}$ . Assume that  $A$  satisfies Assumption 2 with constant  $C_A$  and Assumption 3. A or 3.B with resolutions in  $\mathcal{N}$ . Fix  $n \in \mathcal{N}$  and consider  $(k, C_v)$ -profiles. Under Assumption 1, we have:
$$
d _ {M} \left(\Phi (h, A, \cdot), \Phi (h, A _ {n}, \cdot)\right) \leq P _ {1} \sqrt {\frac {\bar {C} _ {A} C _ {v}}{n}} + \frac {C _ {v} + 1}{n}, \tag {14}
$$
where  $\overline{C}_A \coloneqq (n_{\max} \sum_{i=1}^K C_A^i)^L$ ,  $n_{\max} = \max_{l \in [L]} n_l$ , and  $P_1$  is a constant depending on  $K, L$ . Furthermore, we can invoke the triangle inequality to compare outputs of graphop neural networks built from two different discretizations of  $A$ . For any  $m, n \in \mathcal{N}$ ,
$$
d _ {M} \left(\Phi \left(h, A _ {m}, \cdot\right), \Phi \left(h, A _ {n}, \cdot\right)\right) \leq P _ {1} \sqrt {\bar {C} _ {A} C _ {v}} \left(m ^ {- \frac {1}{2}} + n ^ {- \frac {1}{2}}\right) + \left(C _ {v} + 1\right) \left(n ^ {- 1} + m ^ {- 1}\right). \tag {15}
$$
Compared to the main theorems of Ruiz et al. (2023a), there are two main differences in our results. First, our rate of  $O(n^{-1/2})$  is slower than the rate of  $O(n^{-1})$  in Ruiz et al. (2023a) as a function of  $n$ . Yet, second, their bounds contain a small term that is independent of  $n$  and does not go to 0 as  $n$  goes to infinity. This small term depends on the variability of small eigenvalues in the spectral decomposition of the convolution operator associated with
Figure 1: Hausdorff metric between samples from 1-profiles of 2-hidden-layer GNN on finite polymer graphs vs on large polymer graphs (see Appendix A for polymer graphs). The GNN uses  $\mathrm{GSO}A_n^2 +A_n$  where  $A_{n}$  is the normalized adjacency matrix on  $n$  nodes and ReLU nonlinearities at each layer. Different solid lines are different random draws of functions that make up the estimated 1-profile. See Appendix A for details.
a graphon. The bound in Theorem 3, in contrast, goes to zero. We tested this bound in Figure 1 for GNNs on polymer graphs, which suggest that  $O(n^{-1})$  rate may be possible.
The proof for this theorem is in Appendix D.3 for a more general Theorem 6. Note that it does not suffice to simply use the fact that the assumptions play well with composition with Lipschitz function  $\rho$ , which would result in a bound involving  $\Phi(h, A, \cdot)$  and its discretization  $(\Phi(h, A, \cdot))_n$  as a nonlinear operator, as opposed to a bound between  $\Phi(h, A, \cdot)$  and  $\Phi(h, A_n, \cdot)$ . Our proof shares the same structure as that of Theorem 2 while making sure that the mismatch from discretizing/extending operators does not blow up with composition.
4.3 Assumptions
We discuss the main assumptions of our  $P$ -operators.
Assumption 2 (Lipschitz mapping). An operator  $A: \mathcal{F} \to \mathcal{F}$  is  $C_A$ -Lipschitz if  $\| Af - Ag\|_2 \leq C_A \| f - g\|_2$  for any  $f, g \in \mathcal{F}$ .
We have already had a finite bound on the operator norm in the definition of  $P$ -operators. For linear operators, Assumption 2 is equivalent to a bounded operator norm and is thus automatically satisfied by linear  $P$ -operators.
The next few assumptions are alternatives; only one needs be satisfied by our  $P$ -operators. Intuitively, they ensure that the images of our operator are not too discontinuous:
Assumption 3.A (Maps constant pieces to constant pieces). We say that an operator  $A: \mathcal{F} \to \mathcal{F}$  maps constant pieces to constant pieces at resolutions in  $\mathcal{N} \subset \mathbb{N}$  if for any  $n \in \mathcal{N}$ , and for any  $f \in \mathcal{F}_{[-1,1]}$  that is a.e. constant on each interval  $(u - 1/n, u]$  for  $u \in [n]/n$ ,  $Af$  is also constant on  $(u - 1/n, u]$  for each  $u$ .
Assumption 3.B (Maps Lipschitz functions to Lipschitz functions). We say that an operator  $A: \mathcal{F} \to \mathcal{F}$  maps Lipschitz functions to Lipschitz functions at resolutions in  $\mathcal{N} \subset \mathbb{N}$  if for any  $n \in \mathcal{N}$ , and for any  $f \in \mathcal{F}_{\text{reg}(C_v)}$ ,  $Af$  is  $C_v$ -Lipschitz.
This is the most restrictive assumption. However, the next lemma (proof in Appendix D) describes some dense, sparse and intermediate graphs that satisfy these assumptions.
Lemma 2 (Well-behaved operators). The following examples satisfy our assumptions:
1. Bounded-degree graphings: Let  $G$  be a graphing corresponding  $k-D$  grids or polymer graphs (not necessarily regular graphs) with fixed monomers (Appendix A). For each  $N \in \mathbb{N}$ , there exists a locally equivalent graphing  $G_N'$  such that its adjacency operator satisfies Assumption 3.A with some resolution set (see Lemma 5 and Lemma 7).
2. Lipschitz graphons: Let  $W$  be a  $C_v$ -Lipschitz graphon on  $\mathcal{F}_{reg(C_v)}$ . Then the Hilbert-Schmidt operator  $f \mapsto \int_0^1 W(\cdot, y) g(y) \, \mathrm{d}y$  satisfies Assumption 3.B with resolution in  $\mathbb{N}$ .
3. General graphs: Let  $G$  be a (potentially infinite) graph with a vertex coloring of  $N$  colors such that if two vertices have the same color, then the multiset of their neighbors' colors are the same. Then  $G$ 's adjacency operator satisfies Assumption 3.A with resolution  $N$ . An  $N$ -d hypercube (more generally, Hamming graphs) which is neither bounded-degree nor dense, satisfies the above condition with resolutions in  $\{2^n\}_{n \in [N]}$ .
All our results also hold with a less restrictive assumption that allows for a failure of Assumption 3.A and 3.B in a small set (see Assumption 4.A and 4.B in the Appendix). The most general results are proven in Appendix D and hold in even slightly more relaxed conditions which require the operators to map constant pieces to Lipschitz pieces (Assumption 5.A, 5.B in Appendix D).
4.4 Deviations and Justifications
All our theorems hold in slightly modified settings than those by Backhausz and Szegedy (2022). Namely, we allowed for nonlinear  $P$ -operators, assumed that they have finite  $\| \cdot \|_{2\to 2}$  norm, and used  $(k,L)$ -profiles where we focus on Lipschitz functions (while Backhausz and Szegedy (2022) consider all measurable functions in their profiles). Therefore, we need to ensure that our changes still give us a useful mode of convergence that generalizes dense and sparse graph convergences.
First, without the linearity assumption, the convergence proof by Backhausz and Szegedy (2022) does not hold: we do not know if all limits of nonlinear graphops are still graphops. However, our approximation results (Theorem 2) show special convergent sequences of nonlinear operators, which go beyond the settings in (Backhausz and Szegedy, 2022). Studying special nonlinear operator sequences is interesting since graphop NNs themselves are nonlinear operators. We also assert that our restriction to operators acting on  $L^2$  spaces does not affect convergence guarantees (Theorem 2.14 in (Backhausz and Szegedy, 2022)).
Next, we show that restriction to Lipschitz profiles, which is necessary for our proof technique, does not affect the original convergence either, if we allow our Lipschitz constant to grow:
Theorem 4 (Growing profiles). Let  $L: \mathbb{N} \to \mathbb{R}$  be a strictly increasing sequence such that  $L(n) \xrightarrow{n \to \infty} \infty$ . Consider a sequence of  $P$ -operators  $(A_n: \mathcal{F}_n \to \mathcal{F}_n)_{n \in \mathbb{N}}$  that is Cauchy in the sense that  $d_M'(A_n, A_m) := \sum_{k=1}^{\infty} 2^{-k} d_H(\mathcal{S}_{k,L(n)}(A_n), \mathcal{S}_{k,L(m)}(A_m)) \to 0$  as  $m, n \to \infty$ . If  $A_n \to A$  under action convergence (Backhausz and Szegedy, 2022), then  $(A_n)_{n \in \mathbb{N}}$  converges to the same limit under  $d_M'$ .
This theorem allows us to replace the  $C_v$  constant in our bound with a slowly growing function in  $n$  and get back 'action convergence' as described in Backhausz and Szegedy (2022) so that we can inherit its useful properties while still be able to draw on Lipschitz assumptions in the profiles, without any realistic slowdown in the bound.
Proof sketch For some  $k \in \mathbb{N}$ , by the completeness of the Hausdorff metric over the closed subsets of the space of probability measures supported on  $\mathbb{R}^{2k}$ , the statement is equivalent to showing  $d_{H}(\mathcal{S}_{k}(A),\mathcal{S}_{k,L(n)}(A_{n})) \to 0$  as  $n \to \infty$ . The proof uses a Lipschitz mollification argument to smooth out arbitrary measurable functions  $f_{1},\ldots ,f_{k}$  that witness a measure in the  $k$ -profile of  $A$ . By selecting a Lipschitz mollifier  $\phi$ , we ensure that convolving  $f_{j}$  with  $\phi_{\epsilon}:x \mapsto \epsilon^{-1}\phi (x\epsilon^{-1})$  results in a Lipschitz function that converges to  $f$  in  $L^2$  as  $\epsilon$  goes to 0.
5 Discussion and Future directions
In this paper, we study size transferability of finite GNNs on graphs that are discretizations of graphop, a recent notion of graph limit introduced by Backhausz and Szegedy (2022). We achieve this by viewing GNNs as operators that transform one graph signal into another. Under regularity assumptions, we proved that two GNNs, using two different-resolution GSOs discretized from the same graphop, are close in an operator metric built from weak convergence of measures.
For future direction, a principled study of spectral properties of graphops and graphop neural networks would open doors for techniques from Fourier analysis as used in (Ruiz et al., 2023a; Levie et al., 2022). This leads to distinct challenges, e.g., the spectral gap is not continuous with respect to local-global limits and thus action convergence, but many more properties of spectral measures of bounded-degree graphs are recently studied (Virag, 2018).
This work was supported by Office of Naval Research grant N00014-20-1-2023 (MURI ML-SCOPE), NSF award CCF-2112665 (TILOS AI Institute), NSF award 2134108.
K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? In Int. Conf. on Learning Representations (ICLR), 2019.
Keyulu Xu, Jingling Li, Mozhi Zhang, Simon S. Du, Ken ichi Kawarabayashi, and Stefanie Jegelka. What can neural networks reason about? In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=rJxbJeHFPS.
Keyulu Xu, Mozhi Zhang, Jingling Li, Simon Shaolei Du, Ken-Ichi Kawarabayashi, and Stefanie Jegelka. How neural networks extrapolate: From feedforward to graph neural networks. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=UH-cmocLJC.
G. Yehudai, E. Fetaya, E. Meirom, G. Chechik, and H. Maron. From local structures to size generalization in graph neural networks. In Int. Conference on Machine Learning (ICML), 2021.
A Additional details
Some highly repetitive parts in the proofs are presented informally (e.g. "use the same techniques as...") to highlight the main ideas in the NeurIPS 2023 version of the paper. For the complete formal proof, readers are advised to study a later version on arXiv.
We will use the following extra notations in the proof:
1. For some  $P$ -operator  $A \in L^2(\Omega) \to L^2(\Omega)$ ,  $k \in \mathbb{N}$  and  $\{f_1, \ldots, f_k\} =: F \subset L^2(\Omega)$ , let  $F_A$  be the ordered  $2k$ -tuple  $(f_1, \ldots, f_k, Af_1, \ldots, Af_k)$  and denote
$$
F _ {A} (x) := \left(f _ {1} (x), \dots , f _ {k} (x), A f _ {1} (x), \dots , A f _ {k} (x)\right) \in \mathbb {R} ^ {2 k}.
$$
2. For some  $P$ -operator  $A \in L^2(\Omega) \to L^2(\Omega)$ ,  $k \in \mathbb{N}$  and  $\{f_1, \ldots, f_k\} =: F \subset L^2(\Omega)$ , let  $\mathcal{D}_A(F)$  be the entry distribution  $\mathcal{D}(f_1, \ldots, f_k, Af_1, \ldots, Af_k)$ .
Cut norm and cut metric Here we define concretely the cut metric over the space of graphons. Recall that graphons are  $L^1([0,1], \mathcal{F}, \lambda)$  Lebesgue-integrable functions. The space of graphons is equipped with a norm known as the cut norm:
$$
\| W \| _ {\square} := \sup  _ {S, T \in \mathcal {F}} \left| \int_ {S \times T} W (x, y) \mathrm {d} \lambda (x) \mathrm {d} \lambda (y) \right|, \tag {16}
$$
and a metric known as the cut metric:
$$
d _ {\square} \left(W _ {1}, W _ {2}\right) := \inf  _ {\phi} \| W _ {1} - W _ {2} \circ \phi \| _ {\square}, \tag {17}
$$
where  $\phi$  is taken over all measure-preserving bijections from  $[0,1]$  to  $[0,1]$ . Intuitively, taking the inf over all measure-preserving bijections allows the cut metric to identify graphons that are just a rearrangement away from another. This generalizes symmetries in graphs, where permuting the vertices (and the corresponding edges) do not change the graph itself. Lovász (2012) shows that graphon convergence under the cut metric is well-behaved: every graphon is a limit of a convergent sequence of graphons; and every Cauchy sequences converges to a graphon. This mode of convergence is known as dense graph convergence.
Levy-Prokhorov metric The definition of Levy-Prokhorov metric on  $\mathcal{P}(\mathbb{R}^{2k})$  is:
$$
d _ {L P} \left(\eta_ {1}, \eta_ {2}\right) := \inf  \left\{\epsilon > 0: \eta_ {1} (U) \leq \eta_ {2} \left(U ^ {\epsilon}\right) + \epsilon \wedge \eta_ {2} (U) \leq \eta_ {1} \left(U ^ {\epsilon}\right) + \epsilon , \forall U \in \mathcal {B} _ {2 k} \right\},
$$
where  $\mathcal{B}_{2k}$  is the Borel  $\sigma$ -algebra generated from open subsets of  $\mathbb{R}^{2k}$  and
$$
U ^ {\epsilon} := \{y: \exists x \in \mathbb {R} ^ {2 k} \| x - y \| _ {2} <   \epsilon \}. \tag {18}
$$
Specific graphs We will also define different graphs that we mentioned in the main paper. Unless otherwise stated, we consider undirected graphs. A path on  $n$ -vertices for some  $n \geq 2$  is the graph  $G = ([n], \{(i, i + 1) : i \in [n - 1]\})$ . Its high dimensional generalization are  $k-D$  grids on  $n^k$  vertices are the graphs  $G_k = ([n]^k, \{(u, v) : \| u - v \|_1 = 1\})$ .
A Hamming graph  $H(d,q)$  with parameters  $d, q \geq 1 \in \mathbb{N}$  is the graph with vertex set  $[q]^d$  and edge set contains all pairs  $u, v$  that differs at exactly 1 coordinate (Hamming distance 1). For instance,  $H(1,q)$  is the complete graph on  $q$  vertices and  $H(2,q)$  is the hypercube on  $2^q$  vertices.