SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ningyuan (Teresa) Huang Johns Hopkins University nhuang19@jhu.edu
Ron Levie  
Technion - Israel Institute of Technology  
levieron@technion.ac.il
Soledad Villar Johns Hopkins University svillar3@jhu.edu
Graph neural networks (GNNs) are commonly described as being permutation equivariant with respect to node relabeling in the graph. This symmetry of GNNs is often compared to the translation equivariance of Euclidean convolution neural networks (CNNs). However, these two symmetries are fundamentally different: The translation equivariance of CNNs corresponds to symmetries of the fixed domain acting on the image signals (sometimes known as active symmetries), whereas in GNNs any permutation acts on both the graph signals and the graph domain (sometimes described as passive symmetries). In this work, we focus on the active symmetries of GNNs, by considering a learning setting where signals are supported on a fixed graph. In this case, the natural symmetries of GNNs are the automorphisms of the graph. Since real-world graphs tend to be asymmetric, we relax the notion of symmetries by formalizing approximate symmetries via graph coarsening. We present a bias-variance formula that quantifies the tradeoff between the loss in expressivity and the gain in the regularity of the learned estimator, depending on the chosen symmetry group. To illustrate our approach, we conduct extensive experiments on image inpainting, traffic flow prediction, and human pose estimation with different choices of symmetries. We show theoretically and empirically that the best generalization performance can be achieved by choosing a suitably larger group than the graph automorphism, but smaller than the permutation group.
Graph Neural Networks (GNNs) are popular tools to learn functions on graphs. They are commonly designed to be permutation equivariant since the node ordering can be arbitrary (in the matrix representation of a graph). Permutation equivariance serves as a strong geometric prior and allows GNNs to generalize well [1-3]. Yet in many applications, the node ordering across different graphs is matched or fixed a priori, such as a time series of social networks where the nodes identify the same users, or a set of skeleton graphs where the nodes represent the same joints. In such settings, the natural symmetries arise from graph automorphisms, which effectively only act on the graph signals; This is inherently different from the standard equivariance in GNNs that concerns all possible permutations acting on both the signals and the graph domain. A permutation of both the graph and the graph signal can be seen as a change of coordinates since it does not change the object it represents, just the way to express it. This parallels the passive symmetries in physics, where physical observables are independent of the coordinate system one uses to express them [4, 5]. In contrast, a permutation of the graph signal on a fixed graph potentially transforms the object itself, not only its representation. This parallels the active symmetries in physics, where the coordinate system (in this case, the graph or domain) is fixed but a group transformation on the signal results in a predictable
transformation of the outcome (e.g., permuting left and right joints in the human skeleton changes a left-handed person to right-handed). The active symmetries on a fixed graph are similar to the translation equivariance symmetries in Euclidean convolutional neural networks (CNNs), where the domain is a fixed-size grid and the signals are images.
In this work, we focus on active symmetries in GNNs. Specifically, we consider a fixed graph domain  $G$  with  $N$  nodes, an adjacency matrix  $A \in \mathbb{R}^{N \times N}$ , and input graph signals  $X \in \mathbb{R}^{N \times d}$ . We are interested in learning equivariant functions  $f$  that satisfy (approximate) active symmetries
$$
f (\Pi X) \approx \Pi f (X) \text {f o r p e r m u t a t i o n s} \Pi \in \mathcal {G} \subseteq \mathcal {S} _ {N}, \tag {1}
$$
where  $\mathcal{G}$  is a subgroup of the permutation group  $S_N$  that depends on  $G$ . For example,  $\mathcal{G}$  can be the graph automorphism group  $\mathcal{A}_G = \{\Pi : \Pi A = A\Pi\}$ . Each choice of  $\mathcal{G}$  induces a hypothesis class  $\mathcal{H}_{\mathcal{G}}$  for  $f$ : the smaller the group  $\mathcal{G}$ , the larger the class  $\mathcal{H}_{\mathcal{G}}$ . We aim to select  $\mathcal{G}$  so that the learned function  $f \in \mathcal{H}_{\mathcal{G}}$  generalizes well, also known as the model selection problem [6, Chp.4]. In contrast, standard graph learning methods (GNNs, spectral methods) are defined to satisfy passive symmetries, by treating  $\bar{A}$  as input and requiring  $f(\Pi A\Pi^\top, \Pi X) = \Pi f(A, X)$  for all permutations  $\Pi \in S_N$  and all  $N$ . But for the fixed graph setting, we argue that active symmetries are more relevant. Thus we use  $A$  to define the hypothesis class rather than treating it as an input. By switching from passive symmetries to active symmetries, we will show how to design GNNs for signals supported on a fixed graph with different levels of expressivity and generalization properties.
While enforcing symmetries has been shown to improve generalization when the symmetry group is known a priori [7-13], the problem of symmetry model selection is not completely solved, particularly when the data lacks exact symmetries (see for instance [14-16] and references therein). Motivated by this, we study the symmetry model selection problem for learning on a fixed graph domain. This setting is particularly interesting since (1) the graph automorphism group serves as the natural oracle symmetry; (2) real-world graphs tend to be asymmetric, but admit cluster structure or local symmetries. Therefore, we define approximate symmetries of graphs using the cut distance between graphs from graphon analysis. An approximate symmetry of a graph  $G$  is a symmetry of any other graph  $G'$  that approximates  $G$  in the cut distance. In practice, we take  $G'$  as coarse-grainings (or clusterings) of  $G$ , as these are typically guaranteed to be close in cut distance to  $G$ . We show how to induce approximate symmetries for  $G$  via the automorphisms of  $G'$ . Our main contributions include:
1. We formalize the notion of active symmetries and approximate symmetries of GNNs for signals supported on a fixed graph domain, which allows us to study the symmetry group model selection problem. (See Sections 2, 4)
2. We theoretically characterize the statistical risk depending on the hypothesis class induced from the symmetry group, and show a bias-variance tradeoff between the reduction in expressivity and the gain in regularity of the model. (See Sections 3, 4)
3. We illustrate our approach empirically for image inpainting, traffic flow prediction, and human pose estimation. (See Section 5 for an overview, and Appendix D for the details on how to implement equivariant graph networks with respect to different symmetry groups).
Graph Neural Networks and Equivariant Networks. Graph Neural Networks (GNNs) [17-19] are typically permutation-equivariant (with respect to node relabeling). These include message-passing neural networks (MPNNs) [19-21], spectral GNNs [18, 22-24], and subgraph-based GNNs [25-30]. Permutation equivariance in GNNs can extend to edge types [31] and higher-order tensors representing the graph structure [32, 33]. Equivariant networks generalize symmetries on graphs to other objects, such as sets [34], images [35], shapes [36, 37], point clouds [38-42], manifolds [1, 43, 44], and physical systems [45, 46] among many others. Notably, many equivariant machine learning problems concern the setting where the domain is fixed, e.g., learning functions on a fixed sphere [47], and thus focus on active symmetries rather than passive symmetries such as the node ordering in standard GNNs. Yet learning on a fixed graph domain arises naturally in many applications such as molecular dynamics modeling [48]. This motivates us to consider active symmetries of GNNs for learning functions on a fixed graph. Our work is closely related to Natural Graph Networks (NGNs) in [49], which use global and local graph isomorphisms to design maximally expressive GNNs for distinguishing different graphs. In contrast, we focus on generalization and thus consider symmetry model selection on a fixed graph.
Generalization of Graph Neural Networks and Equivariant Networks. Most existing works focus on the graph-level tasks, where in-distribution generalization bounds of GNNs have been derived using statistical learning-theoretic measures such as Rademacher complexity [50] and Vapnik-Chervonenkis (VC) dimension [51, 52], or uniform generalization analysis based on random graph models [53]; Out-of-distribution generalization properties have been investigated using different notions including transferability [54, 55] (also known as size generalization [56]), and extrapolation [57]. In general, imposing symmetry constraints improves the sample complexity and the generalization error [7-13]. Recently, Petrache and Trivedi [58] investigated approximation-generalization tradeoffs using approximate symmetries for general groups. Their results are based on uniform convergence generalization bounds, which measure the worst-case performance of all functions in a hypothesis class and differ from our non-uniform analysis.
Approximate Symmetries. For physical dynamical problems, Wang et al. [45] formalized approximate symmetries by relaxing exact equivariance to allow for a small equivariance error. For reinforcement learning applications, Finzi et al. [59] proposed Residual Pathway Priors to expand network layers into a sum of equivariant layers and non-equivariant layers, and thus relax strict equivariance into approximate equivariance priors. In the context of self-supervised learning, Suau et al. [60], Gupta et al. [61] proposed to learn structural latent representations that satisfy approximate equivariance. Inspired by randomized algorithms, Cotta et al. [62] formalizes probabilistic notions of invariances and universal approximation. In [63], scattering transforms (a specific realization of a CNN) are shown to be approximately invariant to small deformations in the image domain, which can be seen as a form of approximate symmetry. Scattering transforms on graphs are discussed in [64-66]. Additional discussions on approximate symmetries can be found in [67, Section 6].
2 Problem Setup: Learning Equivariant Maps on a Fixed Graph
Notations. We let  $\mathbb{R},\mathbb{R}_+$  denote the reals and the nonnegative reals,  $I$  denote the identity matrix and  $\mathbb{1}$  denote the all-ones matrix. For a matrix  $Y\in \mathbb{R}^{N\times k}$ , we write the Frobenius norm as  $\| Y\| _F$ . We denote by  $X\odot Y$  the element-wise multiplication of matrices  $X$  and  $Y$ . We write  $[N] = \{1,\dots ,N\}$  and  $\mathcal{S}_N$  as the permutation group on the set  $[N]$ . Groups are typically noted by calligraphic letters. Given  $\mathcal{H},\mathcal{K}$  groups, we denote a semidirect product by  $\mathcal{H}\rtimes \mathcal{K}$ . For a group  $\mathcal{G}$  with representation  $\phi$ , we denote by  $\chi_{\phi |_{\mathcal{G}}}$  the corresponding character (see Definition 6 in Appendix A.1). Denoting  $\mathcal{H}\leq \mathcal{G}$  means that  $\mathcal{H}$  is a subgroup of  $\mathcal{G}$ .
Equivariance. We consider a compact group  $\mathcal{G}$  with Haar measure  $\lambda$  (the unique  $\mathcal{G}$ -invariant probability measure on  $\mathcal{G}$ ). Let  $\mathcal{G}$  act on spaces  $\mathcal{X}$  and  $\mathcal{Y}$  by representations  $\phi$  and  $\psi$ , respectively. We say that a map  $f: \mathcal{X} \to \mathcal{Y}$  is  $\mathcal{G}$ -equivariant if for all  $g \in \mathcal{G}, x \in \mathcal{X}, \psi(g^{-1})f(\phi(g)x) = f(x)$ . Given any map  $f: \mathcal{X} \to \mathcal{Y}$ , a projection of  $f$  onto the space of  $\mathcal{G}$ -equivariant maps can be computed by averaging over orbits with respect to  $\lambda$
$$
\left(\mathcal {Q} _ {\mathcal {G}} f\right) (x) = \int_ {\mathcal {G}} \psi \left(g ^ {- 1}\right) f (\phi (g) x) \mathrm {d} \lambda (g). \tag {2}
$$
For  $u, v \in \mathcal{V}$ , let  $\langle u, v \rangle$  be a  $\mathcal{G}$ -invariant inner product, i.e.  $\langle \psi(g)u, \psi(g)v \rangle = \langle u, v \rangle$ , for all  $g \in \mathcal{G}$ , for all  $u, v \in \mathcal{V}$ . Given two maps  $f_1, f_2 : \mathcal{X} \to \mathcal{Y}$ , we define their inner product as  $\langle f_1, f_2 \rangle_\mu = \int_{\mathcal{X}} \langle f_1(x), f_2(x) \rangle \, \mathrm{d}\mu(x)$ , where  $\mu$  is a  $\mathcal{G}$ -invariant measure on  $\mathcal{X}$ . Let  $V$  be the space of all (measurable) map  $f : \mathcal{X} \to \mathcal{Y}$  such that  $\|f\|_\mu = \sqrt{\langle f, f \rangle_\mu} < \infty$ .
Graphs. We consider edge-node weighted graphs  $G = ([N], A, b)$ , where  $[N]$  is a finite set of nodes,  $A \in [0,1]^{N \times N}$  is the adjacency matrix describing the edge weights, and  $b = \{b_1, \ldots, b_N\} \subset \mathbb{R}$  are the node weights. An edge weighted graph is a special case of  $G$  where all node weights are 1. A simple graph is a special case of an edge weighted graph, where all edge weights are binary. Let  $\mathcal{A}_G$  be the automorphism group of a graph defined as  $\mathcal{A}_G := \{\Pi \in S_N : \Pi A \Pi^\top = A, \Pi b = b\}$ , which characterizes the symmetries of  $G$ . Hereinafter,  $\mathcal{G}$  is assumed to be a subgroup of  $S_N$ .
Graph Signals and Learning Task. We consider graph signals supported in the nodes of a fixed graph  $G$  and maps between graphs signals. Let  $\mathcal{X} = \mathbb{R}^{N\times d}$ ,  $\mathcal{Y} = \mathbb{R}^{N\times k}$  be the input and output graph signal spaces. We denote by  $f$  a map between graph signals,  $f:\mathcal{X}\to \mathcal{Y}$ . Even though the functions  $f$  depend on  $G$ , we don't explicitly write  $G$  as part of the notation of  $f$  because  $G$  is fixed. Our goal is to learn a target map between graph signals  $f^{*}:\mathcal{X}\rightarrow \mathcal{Y}$ . To this end, we assume access to a training set  $\{(X_i,Y_i)\}$  where the  $X_{i}$  are i.i.d. sampled from an  $S_N$ -invariant distribution  $\mu$  on
$\mathcal{X}$  where  $\mathcal{S}_n$  acts on  $\mathcal{X}$  by permuting the rows, and  $Y_{i} = f^{*}(X_{i}) + \xi_{i}$  for some noise  $\xi_{i}$ . A natural assumption is that  $f^{*}$  is approximately equivariant with respect to  $\mathcal{A}_G$  in some sense. Our symmetry model selection problem concerns a sequence of hypothesis class  $\{\mathcal{H}_{\mathcal{G}}\}$  indexed by  $\mathcal{G}$ , where we choose the best class  $\mathcal{H}_{\mathcal{G}}^{*}$  such that the estimator  $\hat{f}\in \mathcal{H}_{\mathcal{G}}^{*}$  gives the best generalization performance.
Equivariant Graph Networks. We propose to learn the target function  $f^{*}$  on the fixed graph using  $\mathcal{G}$ -equivariant graph networks ( $\mathcal{G}$ -Net), which are equivariant to a chosen symmetry group  $\mathcal{G}$  depending on the graph domain. Using standard techniques from representation theory, such as Schur's lemma and projections to isotopic components [68], we can parameterize  $\mathcal{G}$ -Net by interleaving  $\mathcal{G}$ -equivariant linear layers  $f_{\mathcal{G}}: \mathbb{R}^{N \times d} \to \mathbb{R}^{N \times k}$  with pointwise nonlinearity, where the weights in the linear map  $f_{\mathcal{G}}$  are constrained in patterns depending on the group structure (also known as parameter sharing or weight tying [69], see Appendix A.1 for technical details). In practice, we can make  $\mathcal{G}$ -Net more flexible by systematically breaking the symmetry, such as incorporating graph convolutions (i.e.,  $Af_{\mathcal{G}}$ ) and locality constraints (i.e.,  $A \odot f_{\mathcal{G}}$ ). Compared to standard GNNs (described in Appendix D.1),  $\mathcal{G}$ -Net uses a more expressive linear map to gather global information (i.e., weights are not shared among all nodes). Importantly,  $\mathcal{G}$ -Net yields a suite of models that allows us to flexibly choose the hypothesis class (and estimator) reflecting the active symmetries in the data, and subsumes standard GNNs that are permutation-equivariant with respect to passive symmetries (but not necessarily equivariant with respect to active symmetries).
Graphons. A graphon is a symmetric measurable function  $W:[0,1]^2\to [0,1]$ . Graphons represent (dense) graph limits where the number of nodes goes to infinity; they can also be viewed as random graph models where the value  $W(x,y)$  represents the probability of having an edge between the nodes  $x$  and  $y$ . Let  $\mathcal{W}$  denote the space of all graphons and  $\eta$  denote the Lebesgue measure. Lovász and Szegedy [70] introduced the cut norm on  $\mathcal{W}$  as
$$
\| W \| _ {\square} := \sup  _ { \begin{array}{c} S, T \subset [ 0, 1 ] \\ S, T \text {m e a s u r a b l e} \end{array} } \left| \int_ {S \times T} W (u, v) \mathrm {d} \eta (u) \mathrm {d} \eta (v) \right|. \tag {3}
$$
Based on the cut norm, Borgs et al. [71] defined the cut distance between two graphons  $W, U \in \mathcal{W}$ ,
$$
\delta_ {\square} (W, U) := \inf  _ {f \in S _ {[ 0, 1 ]}} \| W - U ^ {f} \| _ {\square}, \tag {4}
$$
where  $S_{[0,1]}$  is the set of all measure-preserving bijective measurable maps between  $[0,1]$  and itself, and  $U^{f}(x,y) = U(f(x),f(y))$ . Note that the cut distance is "permutation-invariant," where measure preserving bijections are seen as the continuous counterparts of permutations.
3 Generalization with Exact Symmetry
In this section, we assume the target map  $f^{*}$  is  $\mathcal{A}_G$ -equivariant and study the symmetry model selection problem by comparing the (statistical) risk of different models. Concretely, the risk quantifies how a given model performs on average on any potential input. The smaller the risk, the better the model performs. Thus, we use the risk gap of two functions  $f, f'$ , defined as
$$
\Delta \left(f, f ^ {\prime}\right) := \mathbb {E} \left[ \| Y - f (X) \| _ {F} ^ {2} \right] - \mathbb {E} \left[ \| Y - f ^ {\prime} (X) \| _ {F} ^ {2} \right], \tag {5}
$$
as our model selection metric. Following ideas from Elesedy and Zaidi [8], we analyze the risk gap between any function  $f \in V$  and its  $\mathcal{G}$ -equivariant version, for  $\mathcal{G} \subseteq S_N$ .
Lemma 1 (Risk Gap). Let  $\mathcal{X} = \mathbb{R}^{N\times d}$ ,  $\mathcal{Y} = \mathbb{R}^{N\times k}$  be the input and output graph signal spaces on a fixed graph  $G$ . Let  $X\sim \mu$  where  $\mu$  is a  $S_N$ -invariant distribution on  $\mathcal{X}$ . Let  $Y = f^{*}(X) + \xi$ , where  $\xi \in \mathbb{R}^{N\times k}$  is random, independent of  $X$  with zero mean and finite variance and  $f^{*}:\mathcal{X}\to \mathcal{Y}$  is  $\mathcal{A}_G$ -equivariant. Then, for any  $f\in V$  and for any compact group  $\mathcal{G}\subseteq S_N$ , we can decompose  $f$  as
$$
f = \bar {f} _ {\mathcal {G}} + f _ {\mathcal {G}} ^ {\perp},
$$
where  $\bar{f}_{\mathcal{G}} = \mathcal{Q}_{\mathcal{G}}f$ ,  $f_{\mathcal{G}}^{\perp} = f - \bar{f}_{\mathcal{G}}$ . Moreover, the risk gap satisfies
$$
\Delta (f, \bar {f} _ {\mathcal {G}}) = \mathbb {E} \left[ \| Y - f (X) \| _ {F} ^ {2} \right] - \mathbb {E} \left[ \| Y - \bar {f} _ {\mathcal {G}} (X) \| _ {F} ^ {2} \right] = \underbrace {- 2 \langle f ^ {*} , f _ {\mathcal {G}} ^ {\perp} \rangle_ {\mu}} _ {\text {m i s m a t c h}} + \underbrace {\left\| f _ {\mathcal {G}} ^ {\perp} \right\| _ {\mu} ^ {2}} _ {\text {c o n s t r a i n t}}.
$$
Lemma 1, proven in Appendix B.1, adapts ideas from [8] to equate the risk gap to the symmetry "mismatch" between the chosen group  $\mathcal{G}$  and the target group  $\mathcal{A}_G$ , plus the symmetry "constraint" captured by the norm of the anti-symmetric part of  $f$  with respect to  $\mathcal{G}$ . Our symmetry model selection problem aims to find the group  $\mathcal{G} \subseteq S_N$  such that the risk gap is maximized. Note that for  $\mathcal{G} \leq \mathcal{A}_G$ , the mismatch term vanishes since  $f^*$  is  $\mathcal{A}_G$ -equivariant, but the constraint term decreases with  $\mathcal{G}$ ; When  $\mathcal{G} = \mathcal{A}_G$ , we recover Lemma 6 in [8]. On the other hand, for  $\mathcal{G} > \mathcal{A}_G$ , the mismatch term can be positive, negative or zero (depending on  $f^*$ ) whereas the constraint term increases with  $\mathcal{G}$ .
3.1 Linear Regression
In this section, we focus on the linear regression setting and analyze the risk gap of using the equivariant estimator versus the vanilla estimator. We consider linear estimator  $\hat{\Theta}:\mathbb{R}^{N\times d}\to \mathbb{R}^{N\times k}$  that predicts  $\hat{Y} = f_{\hat{\Theta}}(X) = \hat{\Theta}^{\top}X$ . Given a compact group  $\mathcal{G}$  and any linear map  $\Theta$ , we obtain its  $\mathcal{G}$ -equivariant version via projection to the  $\mathcal{G}$ -equivariant space (also known as intertwiner average),
$$
\Psi_ {\mathcal {G}} (\Theta) = \int_ {\mathcal {G}} \phi (g) \Theta \psi (g ^ {- 1}) \mathrm {d} \lambda (g). \tag {6}
$$
We denote  $\Psi_{\mathcal{G}}^{\perp}(\Theta) = \Theta - \Psi_{\mathcal{G}}(\Theta)$  as the projection of  $\hat{\Theta}$  to the orthogonal complement of the  $\mathcal{G}$ -equivariant space. Here  $\Psi_{\mathcal{G}}$  instantiates the orbit-average operator  $\mathcal{Q}_{\mathcal{G}}$  (eqn. 2) for linear functions.
Theorem 2 (Bias-Variance-Tradeoff). Let  $\mathcal{X} = \mathbb{R}^{N\times d}$ ,  $\mathcal{Y} = \mathbb{R}^{N\times k}$  be the graph signals spaces on a fixed graph  $G$ . Let  $S_N$  act on  $\mathcal{X}$  and  $\mathcal{Y}$  by permuting the rows with representations  $\phi$  and  $\psi$ . Let  $\mathcal{G}$  be a subgroup of  $S_N$  acting with restricted representations  $\phi|_{\mathcal{G}}$  on  $\mathcal{X}$  and  $\psi|_{\mathcal{G}}$  on  $\mathcal{Y}$ . Let  $X_{[i,j]}\stackrel{i.i.d.}{\sim}\mathcal{N}(0,\sigma_X^2)$  and  $Y = f^{*}(X) + \xi$  where  $f^{*}(x) = \Theta^{\top}x$  is  $\mathcal{A}_G$ -equivariant and  $\Theta \in \mathbb{R}^{Nd\times Nk}$ . Assume  $\xi_{[i,j]}$  is random, independent of  $X$ , with mean 0 and  $\mathbb{E}\left[\xi\xi^{\top}\right] = \sigma_{\xi}^{2}I < \infty$ . Let  $\hat{\Theta}$  be the least-squares estimate of  $\Theta$  from  $n$  i.i.d. examples  $\{(X_i,Y_i):i = 1,\dots ,n\}$ ,  $\Psi_{\mathcal{G}}(\hat{\Theta})$  be its equivariant version with respect to  $\mathcal{G}$ . Let  $(\chi_{\psi|_{\mathcal{G}}}\mid \chi_{\phi|_{\mathcal{G}}}) = \int_{\mathcal{G}}\chi_{\psi|_{\mathcal{G}}}(\boldsymbol{g})\chi_{\phi|_{\mathcal{G}}}(\boldsymbol{g})\mathrm{d}\lambda(\boldsymbol{g})$  denote the inner product of the characters. If  $n > Nd + 1$  the risk gap is
$$
\mathbb {E} \left[ \Delta \left(f _ {\hat {\Theta}}, f _ {\Psi_ {\mathcal {G}} (\hat {\Theta})}\right) \right] = \underbrace {- \sigma_ {X} ^ {2} \| \Psi_ {\mathcal {G}} ^ {\perp} (\Theta) \| _ {F} ^ {2}} _ {b i a s} + \underbrace {\sigma_ {\xi} ^ {2} \frac {N ^ {2} d k - \left(\chi_ {\psi | \mathcal {G}} \mid \chi_ {\phi | \mathcal {G}}\right)}{n - N d - 1}} _ {v a r i a n c e}.
$$