SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Dian Wang $^{1}$  Xupeng Zhu $^{1}$  Jung Yeon Park $^{1}$  Mingxi Jia $^{2*}$  Guanang Su $^{3*}$  Robert Platt $^{1}$  Robin Walters $^{1}$
$^{1}$ Northeastern University  $^{2}$ Brown University  $^{3}$ University of Minnesota  $\{\text{wang.dian,zhu.xup, park.jungy,r.platt,r.walters}\} \text{@northeastern.edu}$  mingxi_jia@brown.edu su000265@umn.edu
Although equivariant machine learning has proven effective at many tasks, success depends heavily on the assumption that the ground truth function is symmetric over the entire domain matching the symmetry in an equivariant neural network. A missing piece in the equivariant learning literature is the analysis of equivariant networks when symmetry exists only partially in the domain. In this work, we present a general theory for such a situation. We propose pointwise definitions of correct, incorrect, and extrinsic equivariance, which allow us to quantify continuously the degree of each type of equivariance a function displays. We then study the impact of various degrees of incorrect or extrinsic symmetry on model error. We prove error lower bounds for invariant or equivariant networks in classification or regression settings with partially incorrect symmetry. We also analyze the potentially harmful effects of extrinsic equivariance. Experiments validate these results in three different environments.
Equivariant neural networks [9, 10] have proven to be an effective way to improve generalization and sample efficiency in many machine learning tasks. This is accomplished by encoding task-level symmetry into the structure of the network architecture so that the model does not need to explicitly learn the symmetry from the data. However, encoding a fixed type of symmetry like this can be limiting when the model symmetry does not exactly match the symmetry of the underlying function being modeled, i.e., when there is a symmetry mismatch. For example, consider the digit image classification task. Is it helpful to model this problem using
Figure 1: An example of correct, incorrect, and extrinsic equivariance. The ground truth function  $f(x)$  is shown in black and its probability density function  $p(x)$  is shown in orange. If we model  $f(x)$  using a  $G$ -invariant network where  $G$  is a reflection group that negates  $x$ , different  $f(x)$  and  $p(x)$  will lead to correct, incorrect, and extrinsic equivariance. See Section 3 for details.
a model that is invariant to 180-degree rotation of the image? For some digits, the label is invariant (e.g., 0 and 8). However, for other digits, the label changes under rotation (e.g., 6 and 9), suggesting that a rotationally symmetric model would be inappropriate here. However, recent work [58] suggests that this is not necessarily the case - symmetric models are sometimes helpful even when a symmetry mismatch exists between the problem and model. This raises the question - do the advantages obtained by using a symmetric model outweigh the errors introduced by the symmetry mismatch?
*Work done as students at Northeastern University
This paper makes four main contributions towards this problem. First, this paper extends the definitions for types of model symmetry with respect to true symmetry introduced in Wang et al. [58]. They classify models as having correct, incorrect, or extrinsic equivariance (see Figure 1), where correct means the ground truth function has the same symmetry as the equivariant model, incorrect means the ground truth function disagrees with the symmetry in the model, and extrinsic means the model symmetry transforms in-distribution data to out-of-distribution data. We generalize this system into a continuum of equivariance types to reflect the fact that a single task may have different proportions of correct, incorrect, and extrinsic symmetry across its domain. For example, in the digit classification task, 0 has correct equivariance, 6 has incorrect equivariance, and 4 has extrinsic equivariance.
Our second contribution is to introduce an analytical lower bound on model error in classification tasks resulting from incorrect model symmetry. This result can help guide model selection by quantifying error resulting from incorrect equivariance constraints. Our result generalizes that of Wang et al. [58] by removing the simplifying assumption that data density over the domain is group invariant. We prove the minimum error of an invariant classifier can be realized by assigning all data points in the same group orbit the label with the majority of the data density (Theorem 4.3).
Our third contribution is to develop new lower bounds on the  $L_{2}$  error for regression tasks in terms of the variance of the function to be modeled over the orbit of the symmetry group. Like our classification bound, this bound can assist in model selection in situations with symmetry mismatch.
Fourth, in contrast to Wang et al. [58] who show benefits of extrinsic equivariance, we theoretically demonstrate its potential harm. We perform experiments documenting the error rate across the correct-extrinsic continuum. Finally, we perform empirical studies illustrating the ideas of the paper and showing that the lower bounds obtained in our analysis appear tight in practice. This suggests our analysis can assist practitioners select symmetry groups appropriate for a given problem setting. Our code is available at https://github.com/pointW/ext理論.
Equivariant Learning. Originally used for exploiting symmetry in image domains [9, 10], equivariant learning has been very successful in various tasks including molecular dynamics [2, 4], particle physics [6], fluid dynamics [59], trajectory prediction [53], pose estimation [31, 26, 32], shape completion [7], robotics [48, 65, 21, 55, 49, 41, 23, 22, 45] and reinforcement learning [52, 54, 56, 39, 63]. However, most prior work assumes that the symmetry of the ground truth function is perfectly known and matches the model symmetry. Wang et al. [58] go further and define correct, incorrect, and extrinsic equivariance to classify the relationship between model symmetry and domain symmetry. However, they do not discuss the possible combinations of the three categories, and limit their theory to a compact group and invariant classification. Our work extends [58] and allows for a continuum of equivariance types and analyzes error bounds in a more general setup.
Symmetric Representation Learning. Various works have proposed learning symmetric representations, using transforming autoencoders [20], restricted Boltzmann machines [50], and equivariant descriptors [47]. In particular, [30] shows that convolutional neural networks implicitly learn representations that are equivariant to rotations, flips, and translations, suggesting that symmetric representations are important inductive biases. Other works have considered learning symmetry-aware features using disentanglement [43], projection mapping [25], equivariance constraints [35], separation into invariant and equivariant parts [61] or subgroups [34]. Park et al. [42] propose learning a symmetric encoder that maps to equivariant features and Dangovski et al. [13] learn features that are sensitive and insensitive to different group representations. Other works assume no prior knowledge of symmetry and learn it from data [3, 64, 14, 40]. In particular, Moskalev et al. [40] estimate the difference between the true latent symmetry and the learned symmetry. Similarly, our work considers a gap between the true symmetry and model symmetry and theoretically analyze its effects on error.
Theory of Equivariant Learning. There are several lines of work on the theory of equivariant learning. Kondor and Trivedi [27] prove that convolutions are sufficient and necessary for equivariance of scalar fields on compact groups, later generalized to the steerable case by Cohen et al. [11]. Certain equivariant networks have been proved to be universal in that such networks can approximate any  $G$ -equivariant function [36, 62]. Another line of work has considered equivariant networks in terms of generalization error. Abu-Mostafa [1] show that an invariant model has a VC dimension less than
or equal to that of a non-equivariant model. Other works studied the generalization error of invariant classifiers by decomposing the input space [51, 46]. Elesedy and Zaidi [17] quantify a generalization benefit for equivariant linear models using the notion of symmetric and anti-symmetric spaces. A PAC Bayes approach was used for generalization bounds of equivariant models [5, 33]. Our work is complimentary to these and quantifies approximation error for equivariant model classes.
Data Augmentation. Some methods use data augmentation [29, 28] to encourage the network to learn invariance with respect to transformations defined by the augmentation function [8]. Recent works have explored class-specific [19, 44] and instance-specific [38] data augmentation methods to further boost training by avoiding the potential error caused by a uniform augmentation function. Those methods can be viewed as applying data augmentation where pointwise correct or extrinsic invariance exist, while avoiding incorrect invariance.
Problem Statement. Consider a function  $f\colon X\to Y$ . Let  $p:X\to \mathbb{R}$  be the probability density function of the domain  $X$ . We assume that there is no distribution shift during testing, i.e.,  $p$  is always the underlying distribution during training and testing. The goal for a model class  $\{h:X\to Y\}$  is to fit the function  $f$  by minimizing an error function  $\mathrm{err}(h)$ . We assume the model class  $\{h\}$  is arbitrarily expressive except that it is constrained to be equivariant with respect to a group  $G$ . Let  $\mathbb{1}$  be an indicator function that equals to 1 if the condition is satisfied and 0 otherwise. In classification,  $\mathrm{err}(h)$  is the classification error rate; for regression tasks, the error function is a  $L_{2}$  norm function,
$$
\operatorname {e r r} _ {\mathrm {c l s}} (h) = \mathbb {E} _ {x \sim p} [ \mathbb {1} (f (x) \neq h (x)) ], \quad \operatorname {e r r} _ {\mathrm {r e g}} (h) = \mathbb {E} _ {x \sim p} [ | | h (x) - f (x) | | _ {2} ^ {2} ]. \tag {1}
$$
**Equivariant Function.** A function  $f: X \to Y$  is equivariant with respect to a symmetry group  $G$  if it commutes with the group transformation  $g \in G$ ,  $f(gx) = gf(x)$ , where  $g$  acts on  $x \in X$  through the representation  $\rho_X(g)$ ;  $g$  acts on  $y \in Y$  through the representation  $\rho_Y(g)$ .
3.1 Correct, Incorrect, and Extrinsic Equivariance.
Consider a model  $h$  which is equivariant with respect to a group  $G$ . Since real-world data rarely exactly conforms to model assumptions, in practice there may often be a gap between the symmetry of the model and the ground truth function. Wang et al. [58] propose a three-way classification which describes the relationship between the symmetry of  $f$  and the symmetry of  $h$ . In this system,  $h$  has correct equivariance, incorrect equivariance, or extrinsic equivariance with respect to  $f$ .
Definition 3.1 (Correct Equivariance). For all  $x \in X, g \in G$  where  $p(x) > 0$ , if  $p(gx) > 0$  and  $f(gx) = gf(x)$ ,  $h$  has correct equivariance with respect to  $f$ .
Definition 3.2 (Incorrect Equivalence). If there exist  $x \in X, g \in G$  such that  $p(x) > 0, p(gx > 0)$ , but  $f(gx) \neq gf(x)$ ,  $h$  has incorrect equivalence with respect to  $f$ .
Definition 3.3 (Extrinsic Equivalence). For all  $x \in X, g \in G$  where  $p(x) > 0$ , if  $p(gx) = 0$ ,  $h$  has extrinsic equivalence with respect to  $f$ .
Example 3.4. Consider a binary classification task where  $X = \mathbb{R}$  and  $Y = \{0,1\}$ . If the model  $h$  is invariant to a reflection group  $G$  where the group element  $g \in G$  acts on  $x \in X$  by  $gx = -x$ , Figure 1 shows examples when correct, incorrect, or extrinsic equivariance is satisfied.
3.2 Pointwise Equivariance Type.
Although Definitions 3.1- 3.3 are self-contained, they do not consider the mixture of different equivariance types in a single function. In other words, an equivariant model can have correct, incorrect, and extrinsic equivariance in different subsets of the domain. To overcome this issue, we define pointwise correct, incorrect, and extrinsic equivariance, which is a generalization of the prior work.
Definition 3.5 (Pointwise Correct Equivalence). For  $g \in G$  and  $x \in X$  where  $p(x) \neq 0$ , if  $p(gx) \neq 0$  and  $f(gx) = gf(x)$ ,  $h$  has correct equivariance with respect to  $f$  at  $x$  under transformation  $g$ .
Figure 2: Example of pointwise correct, incorrect, and extrinsic equivariance in a binary classification task.  $f(x)$  is in black and  $p(x)$  is in orange.  $G$  is a reflection group that negates  $x$ .
Definition 3.6 (Pointwise Incorrect Equivalence). For  $g \in G$  and  $x \in X$  where  $p(x) \neq 0$ , if  $p(gx) \neq 0$  and  $f(gx) \neq gf(x)$ ,  $h$  has incorrect equivalence with respect to  $f$  at  $x$  under transformation  $g$ .
Definition 3.7 (Pointwise Extrinsic Equivalence). For  $g \in G$  and  $x \in X$  where  $p(x) \neq 0$ , if  $p(gx) = 0$ ,  $h$  has extrinsic equivalence with respect to  $f$  at  $x$  under transformation  $g$ .
Notice that the definitions of pointwise correct, incorrect, and extrinsic equivariance are mutually exclusive, i.e., a pair  $(x,g)$  can only have one of the three properties. The pointwise definitions are generalizations of the global Definitions 3.1- 3.3. For example, when pointwise correct equivariance holds for all  $x\in X$  and  $g\in G$ , Definition 3.1 is satisfied.
Example 3.8 (Example of Pointwise Correct, Incorrect, and Extrinsic Equivalence). Consider the same binary classification task in Example 3.4. Figure 2 shows  $f(x)$ ,  $g(x)$ , and four subsets of  $X$  where pointwise correct, incorrect, or extrinsic holds. For  $x$  in the correct section (green),  $p(x) > 0$ ,  $p(gx) > 0$ ,  $f(x) = f(gx)$ . For  $x$  in the incorrect sections (red),  $p(x) > 0$ ,  $p(gx) > 0$ ,  $f(x) \neq f(gx)$ . For  $x$  in the extrinsic section (blue),  $p(x) > 0$ ,  $p(gx) = 0$ .
Definition 3.9 (Correct, Incorrect, and Extrinsic Sets). The Correct Set  $C \subseteq X \times G$  is a subset of  $X \times G$  where pointwise correct equivariance holds for all  $(x, g) \in C$ . Similarly, the Incorrect Set  $I$  and the Extrinsic Set  $E$  are subsets where incorrect equivariance or extrinsic equivariance holds for all elements in the subset. Denote  $U \subseteq X \times G$  as the Undefined Set where  $\forall (x, g) \in U, p(x) = 0$ . By definition we have  $X \times G = C \amalg I \amalg I \amalg E \amalg U$ , where  $\mathrm{II}$  denotes a disjoint union.
4 Approximation Error Lower Bound from Incorrect Equivariance
Studying the theoretical error lower bound of an equivariant network is essential for model selection, especially when incorrect equivariance exists. Wang et al. [58] prove an error lower bound for an incorrect equivariant network, but their setting is limited to a classification task in the global situation of Definition 3.2 with a discrete group and an invariant density function. In this section, we find the lower bound of  $\mathrm{err}(h)$  for an equivariant model  $h$  in a general setting. To calculate such a lower bound, we first define the fundamental domain  $F$  of  $X$ . Let  $d$  be the dimension of a generic orbit of  $G$  in  $X$  and  $n$  the dimension of  $X$ . Let  $\nu$  be the  $(n - d)$  dimensional Hausdorff measure in  $X$ .
Definition 4.1 (Fundamental Domain). A closed subset  $F$  of  $X$  is called a fundamental domain of  $G$  in  $X$  if  $X$  is the union of conjugates $^2$  of  $F$ , i.e.,  $X = \bigcup_{g \in G} gF$ , and the intersection of any two conjugates has 0 measure under  $\nu$ .
We assume further that the set of all  $x$  which lie in any pairwise intersection  $\bigcup_{g_1F \neq g_2F} (g_1F \cap g_2F)$  has measure 0 under  $\nu$ . Let  $Gx = \{gx : g \in G\}$  be the orbit of  $x$ , then  $X$  can be written as the union of the orbits of all points in the fundamental domain  $F$  as such  $X = \bigcup_{x \in F} Gx$ .
4.1 Lower Bound for Classification
We first show the lower bound of the error  $\mathrm{err}_{\mathrm{cls}}(h)$  (Equation 1) given the invariant constraint in  $h$ :  $h(gx) = h(x), g \in G$ . In this section, the codomain  $Y$  of  $f$  is a finite set of possible labels. Since  $h$  is  $G$ -invariant,  $h$  has the same output for all inputs in an orbit  $Gx$ . We call the label that causes the minimal error inside the orbit the majority label<sup>3</sup>, and define the error in the orbit as the total dissent.
Definition 4.2 (Total Dissent). For the orbit  $Gx$  of  $x \in X$ , the total dissent  $k(Gx)$  is the integrated probability density of the elements in the orbit  $Gx$  having a different label than the majority label
$$
k (G x) = \min  _ {y \in Y} \int_ {G x} p (z) \mathbb {1} (f (z) \neq y) d z. \tag {2}
$$
We can also lift the integral to  $G$  itself by introducing a factor  $\alpha(x, g)$  to account for the Jacobian of the action map and size of the stabilizer of  $x$ . (See Appendix A.)
$$
k (G x) = \min  _ {y \in Y} \int_ {G} p (g x) \mathbb {1} (f (g x) \neq y) \alpha (x, g) d g. \tag {3}
$$
${}^{2}$  A conjugate  ${gF}$  is defined as  ${gF} = \{ {gx} \mid  x \in  F\}$  .
3The majority label has more associated data than all other labels, but does not need to be more than  $50\%$
Theorem 4.3.  $\operatorname{err}(h)$  is lower bounded by  $\int_{F} k(Gx) dx$ .
Proof. Rewriting the error function of Equation 1, we have
$$
\operatorname {e r r} (h) = \int_ {X} p (x) \mathbb {1} (f (x) \neq h (x)) d x = \int_ {x \in F} \int_ {z \in G x} p (z) \mathbb {1} (f (z) \neq h (z)) d z d x, \tag {4}
$$
using iterated integration (Appendix B) and Definition 4.1. We assume the measure of  $F \cap gF$  is 0. Since  $h(z)$  can only have a single label in orbit  $Gx$ , we can lower bound the inside integral as
$$
\int_ {z \in G x} p (z) \mathbb {1} (f (z) \neq h (z)) d z \geq \min  _ {y \in Y} \int_ {z \in G x} p (z) \mathbb {1} (f (z) \neq y) d z = k (G x).
$$
We obtain the claim by integrating over  $F$ . Notice that this is a tight lower bound assuming universal approximation. That is, there exists  $h$  which realizes this lower bound.
We can express the total dissent in terms of the Incorrect Set  $I$  (Definition 3.9).
Proposition 4.4.  $k(Gx) = \min_{x' \in (Gx)^+} \int_G p(gx') \mathbb{1}((x', g) \in I) \alpha(x', g) dg$ , where  $(Gx)^+ = \{x_0 \in Gx | p(x_0) > 0\}$ .
Proof. Consider Equation 3, since the minimum over  $y$  is obtained for  $y = f(x')$  for some  $x' \in Gx$  such that  $p(x') > 0$  (i.e.,  $x' \in (Gx)^+$ ),
$$
k (G x) = \min  _ {x ^ {\prime} \in (G x) ^ {+}} \int_ {G} p (g x) \mathbb {1} \left(f (g x) \neq f \left(x ^ {\prime}\right)\right) \alpha (x, g) d g.
$$
Since  $x' \in Gx$ , then  $Gx' = Gx$  and we have  $k(Gx) = k(Gx')$ . Thus,
$$
\begin{array}{l} k (G x) = \min  _ {x ^ {\prime} \in (G x) ^ {+}} \int_ {G} p (g x ^ {\prime}) \mathbb {1} \left(f \left(g x ^ {\prime}\right) \neq f \left(x ^ {\prime}\right)\right) \alpha \left(x ^ {\prime}, g\right) d g \\ = \min  _ {x ^ {\prime} \in (G x) ^ {+}} \int_ {G} p \left(g x ^ {\prime}\right) \mathbb {1} \left(\left(x ^ {\prime}, g\right) \in I\right) \alpha \left(x ^ {\prime}, g\right) d g. \\ \end{array}
$$
Example 4.5 (Lower bound example for a binary classification task using Proposition 4.4).
Let  $f\colon X\to \{0,1\}$  be a binary classification function on  $X = \{x_0,x_1,x_2,x_3\}$ . Let  $G = C_{2} = \{e,r\}$  be the cyclic group of order two that permutes the elements in  $X$ . Figure 3 shows  $X$ , the label for each  $x\in X$ , and how  $e,r\in G$  acts on  $x\in X$ .  $\{x_0,x_3\}$  forms a fundamental domain  $F$ , and there are two orbits:  $Gx_{0} = \{x_{0},x_{1}\}$  and  $Gx_{2} = \{x_{2},x_{3}\}$ . Since both  $X$  and  $G$  are discrete and  $g\in G$  acts on  $X$  through permutation, The lower bound can be written as  $\mathrm{err}(h)\geq \sum_{x\in F}\min_{x'\in (Gx)^+}\sum_{g\in G}p(gx')\mathbb{1}((x',g)\in I)$ . We can then calculate  $\sum_{g\in G}p(gx')\mathbb{1}((x',g)\in I)$  for  $x^{\prime}\in X$ :  $x_0:0.4,x_1:0.3,x_2:0,x_3:0$ . Taking the min over each orbit we have  $k(Gx_0) = 0.3,k(Gx_2) = 0$ . Taking the sum over  $F = \{x_0,x_3\}$  we obtain  $\mathrm{err}(h)\geq 0.3$
Figure 3: An example binary classification task. The circles are elements of  $X$ . The arrows show how  $g \in G$  acts on  $x \in X$ . The arrow color shows whether  $(x, g) \in I$ .
Figure 4: An example multi-class classification task. Color indicates the label. The fundamental domain  $F$  is a vertical line. For a point  $x \in F$ , the orbit  $Gx$  is a horizontal line.
Example 4.6 (Lower bound example for a multi-class classification task using Proposition 4.4). Consider a multi-class classification task  $f: \mathbb{R}^2 \to Y$  with  $n = |Y|$  classes. For  $x = (u, v) \in [0,1]^2$  then  $p(u, v) = 1$  and otherwise  $p(u, v) = 0$ ; i.e., the support of  $p$  is a unit square. Let  $G$  denote the group of translations in the  $u$ -direction and  $h$  a  $G$ -invariant network. In a data distribution illustrated in Figure 4, we compute the lower bound for  $\mathrm{err}(h)$ . Consider a fundamental domain  $F$  (brown line in Figure 4). In the blue area, there is one label across the orbit (i.e., the horizontal
line), meaning  $\forall g\in G,(x',g)\in C$ , yielding Proposition 4.4 equals 0. For points in the yellow area, the majority label is yellow. This means that for  $g\in G$  such that  $gx$  is in yellow,  $(x,g)\in C$ ; for other  $g\in G,(x,g)\in I$ . Consequently, Proposition 4.4 is equivalent to the combined green and pink lengths. Taking the integral over  $F$  (Theorem 4.3), the lower bound equals the green and pink area  $(I$  in Figure 4). We define correct ratio  $(c)$  as the blue area's height and majority label ratio  $(m)$  as the yellow area's length. Adjusting  $c$  and  $m$  transitions incorrect to correct equivariance, leading to  $\mathrm{err}(h)\geq \mathrm{area}(I) = (1 - c)\times (1 - m)$ . Appendix H.2 shows an experiment where the empirical result matches our analysis.
Lower Bound When  $G$  is Finite and The Action of  $G$  is Density Preserving. In this section, we consider the lower bound in Theorem 4.3 when  $G$  is finite and the action of  $G$  is density preserving, i.e.,  $p(gx) = p(x)$ . Let  $(Gx)_y = \{z \in Gx | f(z) = y\}$  be a subset of  $Gx$  with label  $y$ . Define  $\mathcal{Q}(x) = (\max_{y \in Y} |(Gx)_y|) / |Gx|$ , which is the fraction of data in the orbit  $Gx$  that has the majority label. Denote  $Q = \{\mathcal{Q}(x) : x \in X\}$  the set of all possible values for  $\mathcal{Q}$ . Consider a partition of  $X = \coprod_{q \in Q} X_q$  where  $X_q = \{x \in X : \mathcal{Q}(x) = q\}$ . Define  $c_q = \mathbb{P}(x \in X_q) = |X_q| / |X|$ .
Proposition 4.7. The error lower bound  $\mathrm{err}(h) \geq 1 - \sum_q q c_q$  from Wang et al. [58] (Proposition 4.1) is a special case of Theorem 4.3.
Proof in Appendix C. The proposition shows Theorem 4.3 is a strict generalization of [58, Prop 4.1].
4.2 Lower Bound for Invariant Regression
In this section, we give a lower bound of the error function  $\mathrm{err}_{\mathrm{reg}}(h)$  (Equation 1) in a regression task given that  $h$  is invariant, i.e.,  $h(gx) = h(x)$  for all  $g \in G$ . Assume  $Y = \mathbb{R}^n$ . Denote by  $p(Gx) = \int_{z \in Gx} p(z) dz$  the probability of the orbit  $Gx$ . Denote by  $q(z) = \frac{p(z)}{p(Gx)}$  the normalized probability density of the orbit  $Gx$  such that  $\int_{Gx} q(z) dz = 1$ . Let  $\mathbb{E}_{Gx}[f]$  be the mean of function  $f$  on the orbit  $Gx$  defined, and let  $\mathbb{V}_{Gx}[f]$  be the variance of  $f$  on the orbit  $Gx$ ,
$$
\mathbb {E} _ {G x} [ f ] = \int_ {G x} q (z) f (z) d z = \frac {\int_ {G x} p (z) f (z) d z}{\int_ {G x} p (z) d z}, \qquad \mathbb {V} _ {G x} [ f ] = \int_ {G x} q (x) | | \mathbb {E} _ {G x} [ f ] - f (z) | | _ {2} ^ {2}.
$$
Theorem 4.8.  $\operatorname{err}(h) \geq \int_{F} p(Gx) \mathbb{V}_{Gx}[f] dx$ .
Proof. The error function (Equation 1) can be written as: