SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark 4.3 (Computational complexity). The algorithm constructed in Theorem 4.1 involves a one-time preprocessing step after which  $f(x)$  can be evaluated at any given  $x \in \Omega$  in  $O(n^2)$  time. We note that the computation at inference time matches that of (classic) Lipschitz/Hölder regression (e.g. Gottlieb et al., 2017). Furthermore, the computational complexity of the preprocessing step is similar to that in Ashlagi et al. [2021, Theorem 7] for the average Lipschitz case, where it is shown to run in time  $\widetilde{O}(n^2)$ . The complexity analysis of our prepossessing step is entirely analogous to theirs, and we forgo repeating it here.
We will now outline the proof of Theorem 4.1, which appears in Appendix B.3. The key idea is to analyze a natural fully-empirical quantity that will serve as an estimator of the true unknown average
<sup>3</sup>Namely, any ball of radius  $r > 0$  can be covered by  $2^{d}$  balls of radius  $r / 2$ .
smoothness. To that end, given a sample  $S = (X_i,Y_i)_{i=1}^n \sim \mathcal{D}^n$  and a function  $f: \Omega \to [0,1]$ , consider the following quantity which can be established directly from the data:
$$
\widehat {\Lambda} _ {f} ^ {\beta} := \frac {1}{n} \sum_ {i = 1} ^ {n} \sup _ {X _ {j} \neq X _ {i}} \frac {| f (X _ {i}) - f (X _ {j}) |}{\rho (X _ {i} , X _ {j}) ^ {\beta}}.
$$
Namely, this is the empirical average smoothness with respect to the sampled points. It would suit us well if the empirical average smoothness of a function did not greatly exceed its true average smoothness, with high probability. The fact something like this turns out to be true is somewhat surprising and may be of independent interest:
Proposition B.1. (Informal) Let  $f^{*}:\Omega \to [0,1]$ . Then with high probability  $\widehat{\Lambda}_{f^{*}}^{\beta}\lesssim \overline{\Lambda}_{f^{*}}^{\beta}$ .
The proposition above implies that restricting to the sample, and letting  $\widehat{f}(X_i) \coloneqq Y_i$  yields a function over  $\{X_1, \ldots, X_n\}$  which is empirically average-smooth over the sample (with high probability). We then turn to show that any such function can be approximately extended to the whole space, in a way that guarantees its average smoothness with respect to the underlying distribution.
Proposition B.3. (Informal) Let  $\widehat{f}:\{X_1,\ldots ,X_n\} \to [0,1]$  where  $(X_{i})_{i = 1}^{n}\sim \mu^{n}$ . Then it is possible to construct  $f:\Omega \rightarrow [0,1]$  such that with high probability  $f(X_{i})\approx \widehat{f} (X_{i})$  for all  $i\in [N]$ , and  $\overline{\Lambda}_f^\beta (\mu)\lesssim \widehat{\Lambda}_{\widehat{f}}^\beta$ .
We will now sketch the procedure described in Proposition B.3, which serves as the main challenge in proving Theorem 4.1. Roughly speaking, the algorithm sorts the sampled points with respect to their relative slope to one another. Then, it discards a fraction of the sampled points with largest relative slope, which can be thought of as "outliers". Then, the algorithm proceeds to extend the function in a smooth fashion among the remaining "well-behaved" samples. A careful probabilistic analysis shows that disregarding just the right amount of samples induces small error, while being average-smooth with high probability.
Overall this procedure yields a function  $f: \Omega \to [0,1]$  which is an approximate empirical-minimizer (since  $f(X_{i}) \approx \widehat{f}(X_{i}) = Y_{i}$ ), while guaranteed to be averagely-smooth with respect to the unknown distribution. Thus we can apply the uniform convergence of Theorem 3.4, proving Theorem 4.1.
5 Agnostic learning algorithm
Noticeably, up to this point, both the uniform convergence result we derived (Theorem 3.4) as well as the algorithmic result (Theorem 4.1) are tailored for the realizable regression setting. Inspired by a recent result of Hopkins et al. [2022] that showed a reduction from agnostic learning to realizable learning, we provide an algorithm for agnostic (i.e. noisy) regression of average-smooth functions. It is worth noting that the following algorithm does not require any prior assumption on the noise model, unlike many nonparametric regression methods, due to our distribution free analysis.
Theorem 5.1. There exists a learning algorithm  $\mathcal{A}$  such that for any metric space  $(\Omega, \rho)$ , any  $\beta \in (0,1]$ ,  $0 < \epsilon < L$ , and any distribution  $\mathcal{D}$  over  $\Omega \times [0,1]$ , given a sample  $S \sim \mathcal{D}^n$  of size  $n \geq N$  for some  $N = N(\beta, \varepsilon, \delta)$  satisfying
$$
N = \widetilde {O} \left(\frac {\mathcal {N} _ {\Omega} \left(\left(\frac {\varepsilon}{6 4 0 L \log (1 / \varepsilon)}\right) ^ {1 / \beta}\right) + \log (1 / \delta)}{\varepsilon^ {2}}\right),
$$
the algorithm constructs a hypothesis  $f = \mathcal{A}(S)$  such that  $L_{\mathcal{D}}(f) \leq \inf_{f^{*} \in \overline{\mathrm{H}\tilde{\mathrm{o}}\mathrm{l}}_{L}^{\beta}(\Omega, \mu)} L_{\mathcal{D}}(f^{*}) + \epsilon$  with probability at least  $1 - \delta$ .
Remark 5.2 (Doubling metrics). As mentioned in Corollary 3.5, in most cases of interest we have  $\mathcal{N}_{\Omega}(\varepsilon) \lesssim \left(\frac{1}{\varepsilon}\right)^d$ . In that case, the algorithm above has sample complexity
$$
N = \widetilde {O} \left(\frac {L ^ {d / \beta}}{\varepsilon^ {(d + 2 \beta) / \beta}}\right),
$$
$$
L _ {\mathcal {D}} (f) = \inf  _ {f ^ {*} \in \overline {{\mathrm {H o l}}} _ {L} ^ {\beta} (\Omega , \mu)} L _ {\mathcal {D}} (f ^ {*}) + \widetilde {O} \left(\frac {L ^ {d / (d + 2 \beta)}}{n ^ {\beta / (d + 2 \beta)}}\right),
$$
Though our agnostic algorithm is similar in spirit to that obtained by the reduction of Hopkins et al. [2022], our analysis is self-contained and crucially relies on the bracketing bound given by Theorem 3.1, as well as analyzing the empirical smoothness estimator as provided by Proposition B.1. We also note that unlike our algorithm for realizable learning, the agnostic algorithm is not computationally efficient. This seems to be inherent for such reductions, and we do not know whether this blow-up in running time can be avoided or not.
We will now describe the proof of Theorem 5.1 which appears in Appendix B.4. Given a sample  $S$  of size  $n$ , consider dividing it into two sub-samples  $S_{1} \cup S_{2} = S$  of size  $n/2$  each. We first use  $S_{1}$  in order to construct an empirical  $\epsilon$ -net  $h_{1}, \ldots, h_{N}: S_{1} \to [0,1]$ , namely a set of functions which are sufficiently empirically smooth over the sample, yet far away enough from one another when averaged over the sample. Recalling that bracketing numbers upper bound covering numbers (Eq. (1)), and since Theorem 3.1 holds true for every measure (in particular for the empirical measure), we can bound  $\log N \lesssim \mathcal{N}_{\Omega}((\epsilon/L)^{1/\beta})$ . Moreover, using Proposition B.1 we know that  $f^{*} := \arg \min_{f \in \overline{\mathrm{Hoi}}_{L}^{\beta}(\Omega, \mu)} L_{\mathcal{D}}(f)$  is likely to be  $\widetilde{O}(L)$  average-smooth over  $S_{1}$ , so there must exist some  $h_{j}$  with  $\epsilon$  excess empirical loss (since  $f^{*}$  is in the class we are  $\epsilon$ -covering). Thus running the realizable algorithm of Theorem 4.1 over all  $\{h_{1}, \ldots, h_{N}\}$ , producing  $f_{1}, \ldots, f_{N}: \Omega \to [0,1]$ , yields at least one function which has both small excess empirical error, while being smooth with respect to the underlying distribution. Finally, running ERM over  $\{f_{1}, \ldots, f_{N}\}$  with respect to the fresh sample  $S_{2}$  reveals such a good candidate function within  $\frac{\log(N) + \log(1 / \delta)}{\epsilon^{2}}$  samples by applying standard uniform convergence for finite classes (i.e. Hoeffding's inequality with the union bound).
6 Lower bound
We now turn to show that the bounds proved in Theorem 3.1, Theorem 3.4 and Theorem 4.1 are all tight up to logarithmic factors. In fact, since the bracketing entropy bound of Theorem 3.1 implies the generalization bound of Theorem 3.4 and the latter implies the sample complexity in Theorem 4.1, it is enough to show that the latter is nearly optimal.
Theorem 6.1. For any  $\beta \in (0,1]$ ,  $\varepsilon \in (0,1)$  any metric space  $(\Omega, \rho)$  and  $L \geq \frac{8}{\mathrm{diam}(\Omega)}$ , there exists a distribution  $\mathcal{D}$  over  $\Omega \times [0,1]$  which is realizable by  $\overline{\mathrm{Hol}}_L^\beta(\Omega)$  such that any learning algorithm that produces  $f = A(S)$  with  $L_{\mathcal{D}}(f) \leq \varepsilon$  with constant probability, must have sample complexity
$$
n = \Omega \left(\frac {\mathcal {N} _ {\Omega} ((\varepsilon / L) ^ {1 / \beta})}{\varepsilon}\right).
$$
Remark 6.2 (Typical case). In most cases of interest it holds that  $\mathcal{N}_{\Omega}(\varepsilon) \gtrsim \left(\frac{1}{\varepsilon}\right)^d$  for some constant  $d$ , e.g. when  $\Omega$  is a subset of non-empty interior in  $\mathbb{R}^d$  (or more generally in any  $d$ -dimensional Banach space). That being the case, Theorem 6.1 yields the simplified sample complexity lower bound of
$$
n = \Omega \left(\frac {L ^ {d / \beta}}{\varepsilon^ {(d + \beta) / \beta}}\right)
$$
Equivalently, we obtain an excess risk lower bound of
$$
L _ {\mathcal {D}} (f) = \Omega \left(\frac {L ^ {d / (d + \beta)}}{n ^ {\beta / (d + \beta)}}\right).
$$
We will now provide a proof sketch of Theorem 6.1, while the full proof appears in Appendix B.5. Suppose  $K \subset \Omega$  is a  $(\varepsilon / L)^{1/\beta}$ -net of most of  $\Omega$ , yet  $x_0 \in \Omega$  is some "isolated" point at constant
<sup>4</sup>Note that assuming a subset has nonempty interior implies that it cannot be isometrically embedded to a lower dimensional space. Hence, this  $d$  encapsulates the "true" intrinsic metric dimension.
distance away from  $K$  (we show that such  $x_0, K$  always exist). Let  $\mu$  be the measure that assigns  $1 - \varepsilon$  probability mass to  $x_0$ , while the rest of the probability mass is distributed uniformly over  $K$ . Now consider a (random) function that independently assigns either 0 or 1 to each point in  $K$  uniformly, and is constant over  $x_0$ . Since points in  $K$  are  $(\varepsilon / L)^{1/\beta}$  away from one another, the local  $\beta$ -slope at each point in  $K$  is roughly  $1 / ((\varepsilon / L)^{1/\beta})^\beta = L / \varepsilon$ , while the slope at  $x_0$  is small since it is far enough from other points. Averaging over the space with respect to  $\mu$ , we see that the function is  $\mu(K) \cdot L / \varepsilon = L$  average-Hölder. Now, we imitate the standard lower bound proof for VC classes over  $K$ : Since any point in  $K$  is sampled with probability  $\varepsilon / |K|$ , any learning algorithm with much fewer than  $|K| / \varepsilon \approx \mathcal{N}_{\Omega}((\varepsilon / L)^{1/\beta}) / \varepsilon$  examples will guess wrong a large portion of  $K$ , suffering  $L_1$ -loss of at least order of  $\mu(K) = \varepsilon$ .
7 Illustrative examples
Having established the control that average-Hölder smoothness has on generalization, we illustrate the vast possible gap between the average smoothness and it's "worst-case" classic counterpart. Indeed, in the examples we provide, the gap is infinite. Moreover, we also show that classes of average-Hölder smoothness are significantly richer than the previously studied average-Lipschitz, motivating the more general Hölder framework considered in this work. Finally, it is illuminating to notice that both claims to follow actually consist of the same simple function  $f(x) = \mathbf{1}[x > \frac{1}{2}]$  though with respect to different distributions, emphasizing the crucial role of the underlying distribution in terms of establishing the function classes.
Claim 7.1. For any  $L > 0, \beta \in (0,1)$ , there exist  $f: \Omega \to [0,1]$  and a probability measure  $\mu$  such that
-  $f$  is average-Hölder:  $f \in \overline{\mathrm{H\ddot{o}l}}_L^\beta(\Omega, \mu)$ .
-  $f$  is not Hölder with any finite Hölder constant: For all  $M > 0$ :  $f \notin \mathrm{Hil}_M^\beta(\Omega)$ .
-  $f$  is not (even) weakly-average-Lipschitz with any finite modulus: For all  $M > 0 : f \notin \widetilde{\mathrm{Lip}}_M(\Omega, \mu)$ .
Thus,  $\overline{\mathrm{H}\ddot{\mathrm{o}}\mathrm{l}}_L^\beta (\Omega ,\mu)\not\subset\bigcup_{M = 0}^\infty \left(\mathrm{H}\ddot{\mathrm{o}}\mathrm{l}_M^\beta (\Omega)\cup \widetilde{\mathrm{Lip}}_M(\Omega ,\mu)\right).$
Claim 7.2. For any  $L > 0, \beta \in (0,1)$ , there exist  $f: \Omega \to [0,1]$  and a probability measure  $\mu$  such that
-  $f$  is weakly-average-Hölder:  $f \in \widetilde{\mathrm{H\ddot{o}l}}_L^\beta(\Omega, \mu)$ .
-  $f$  is not strongly-average-Hölder with any finite modulus: For all  $M > 0$ :  $f \notin \overline{\mathrm{H\ddot{o}l}}_M^\beta(\Omega)$ .
-  $f$  is not (even) weakly-average-Lipschitz with any finite modulus: For all  $M > 0 : f \notin \widetilde{\mathrm{Lip}}_M(\Omega, \mu)$ .
Thus,  $\widetilde{\mathrm{H}\bar{\mathrm{o}}\mathrm{l}}_L^\beta (\Omega ,\mu)\not\subset\bigcup_{M = 0}^{\infty}\left(\overline{\mathrm{H}\bar{\mathrm{o}}\mathrm{l}}_M^\beta (\Omega ,\mu)\cup \widetilde{\mathrm{Lip}}_M(\Omega ,\mu)\right).$
We prove both of the claims above in Appendix B.6.
In this work, we have defined a notion of an average-Hölder smoothness, extending the average-Lipschitz one introduced by Ashlagi et al. [2021]. Using proof techniques based on bracketing numbers, we have established the minimax rate for average-smoothness classes in the realizable setting with respect to the  $L_{1}$  risk up to logarithmic factors, and have provided a nontrivial learning algorithm that attains this nearly-optimal learning rate. Moreover, we have also provided yet another learning algorithm for the agnostic setting. All of these results improve upon previously known rates even in the special case of average-Lipschitz classes.
A few notes are in order. First, the choice of focusing on  $L_{1}$  risk as opposed to general  $L_{p}$  losses is merely a matter of conciseness, as to avoid introducing additional parameters. Indeed, the only place throughout the proofs which we use the  $L_{1}$  loss is in the proof of Proposition 3.2, where we show that the loss-class  $\mathcal{L}_{\mathcal{F}} \coloneqq \{x \mapsto |f(x) - f^{*}(x)| : f \in \mathcal{F}\}$  satisfies
$$
\mathcal {N} _ {[ ]} (\mathcal {L} _ {\mathcal {F}}, L _ {1} (\mu), \alpha) \leq \mathcal {N} _ {[ ]} (\mathcal {F}, L _ {1} (\mu), \alpha).
$$
It is easy to show via essentially the same proof that for any  $p \in [1, \infty)$ , the  $L_{p}$ -composed loss-class satisfies  $\mathcal{N}_{[]}(\mathcal{L}_{\mathcal{F}}, L_1(\mu), \alpha) \leq \mathcal{N}_{[]}(\mathcal{F}, L_1(\mu), \alpha^{1/p})$ , and the remaining proofs can be invoked verbatim. This yields a realizable sample complexity (in the typical,  $d$ -dimensional case) of order  $N = \widetilde{O}\left(\frac{L^{d/p\beta}}{\varepsilon^{(d+p\beta)/p\beta}}\right)$ , or equivalently  $L_{p}$ -risk decay rate of  $L_{\mathcal{D}}(f) = \widetilde{O}\left(\frac{L^{d/(d+p\beta)}}{n^{p\beta/(d+p\beta)}}\right)$  which are also easily translatable to their corresponding agnostic rates.
Focusing again on  $L_{1}$  minimax rates of average-Hölder classes, it is interesting to compare them to the minimax rates of "classic" (i.e., worst-case) Hölder classes. Schreuder [2020] has shown the minimax risk to be of order  $n^{-\beta /d}$ , whereas we showed the average-smooth case has the slightly worse rate of  $n^{-\beta /(d + \beta)}$  (which cannot be improved, due to our matching lower bound). However, comparing the rates alone is rather misleading, since both risks are multiplied by a factor depending on their corresponding Hölder constant, which can be considerably smaller in the average-case result. Still, it is interesting to note that in the asymptotic regime there is a marginal advantage in case the learned function is worst-case Hölder, as opposed to Hölder on average.
Our work leaves open several questions. A relatively straightforward one is to compute the minimax rates and construct an optimal algorithm for the classification setting, which is not addressed by this paper. Moreover, there is a slight mismatch between our established upper and lower bounds in the agnostic setting, ranging between  $\widetilde{O}(n^{-\beta/(d+2\beta)})$  and  $\Omega(n^{-\beta/(d+\beta)})$ . Closing this gap is an interesting problem which we leave for future work.
Mikhail S. Agranovich. Sobolev spaces, their generalizations and elliptic problems in smooth and Lipschitz domains. Springer Monographs in Mathematics. Springer, Cham, 2015. ISBN 978-3-319-14647-8; 978-3-319-14648-5. doi: 10.1007/978-3-319-14648-5. URL https://doi.org/10.1007/978-3-319-14648-5. Revised translation of the 2013 Russian original.
Martin Anthony and Peter L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge, 1999. ISBN 0-521-57353-X. doi: 10.1017/CBO9780511624216. URL http://dx.doi.org/10.1017/CBO9780511624216.
Jürgen Appell, Jozef Banaš, and Nelson Merentes. Bounded variation and around, volume 17 of De Gruyter Series in Nonlinear Analysis and Applications. De Gruyter, Berlin, 2014. ISBN 978-3-11-026507-1; 978-3-11-026511-8.
Yair Ashlagi, Lee-Ad Gottlieb, and Aryeh Kontorovich. Functions with average smoothness: structure, algorithms, and learning. In Conference on Learning Theory, pages 186-236. PMLR, 2021.
Peter L. Bartlett, Sanjeev R. Kulkarni, and S. E. Posner. Covering numbers for real-valued function classes. IEEE Trans. Information Theory, 43(5):1721-1724, 1997. doi: 10.1109/18.623181. URL https://doi.org/10.1109/18.623181.
Kamalika Chaudhuri and Sanjoy Dasgupta. Rates of convergence for nearest neighbor classification. In NIPS, 2014.
David L. Donoho and Iain M. Johnstone. Minimax estimation via wavelet shrinkage. Ann. Statist., 26 (3):879-921, 1998. ISSN 0090-5364. doi: 10.1214/aos/1024691081. URL https://doi.org/10.1214/aos/1024691081.
Evarist Gine and Richard Nickl. Mathematical foundations of infinite-dimensional statistical models. Cambridge university press, 2021.
Lee-Ad Gottlieb, Aryeh Kontorovich, and Robert Krauthgamer. Adaptive metric dimensionality reduction. Theoretical Computer Science, 620:105-118, 2016.
A Minimal  $\beta$ -slope Hölder extension
In this section we describe a procedure that extends Hölder functions in an optimally smoothest manner at every point, as it will serve as a crucial ingredient in our proofs. That is, given a subset of a metric space  $A \subset \Omega$  and a function  $f: \Omega \to [0,1]$ , it produces  $F_A: \Omega \to [0,1]$  such that
1. It extends  $f|_{A}: F_{A}|_{A} = f|_{A}$ .
2. For any  $\widetilde{F}:\Omega \to [0,1]$  that extends  $f|_{A}$ , it holds that  $\Lambda_{F_A}^\beta (x)\leq \Lambda_{\widetilde{F}}^\beta (x)$  for all  $x\in \Omega$
Such a procedure was described for Lipschitz extensions (namely when  $\beta = 1$ ) in Ashlagi et al. [2021]. The purpose of this section is to generalize this procedure to any Hölder exponent.
Throughout this section we fix  $\beta \in (0,1]$ ,  $\emptyset \neq A \subset \Omega$  and  $f: \Omega \to [0,1]$ , and will always assume the following.
Assumption A.1.  $\| f|_{A}\|_{\mathrm{H}\ddot{\circ}\mathrm{l}^{\beta}} < \infty$  and  $\mathrm{diam}(A) < \infty$ .
Keeping in mind that the case we are really interested in is when  $A$  is finite (i.e. a sample), the conditions above are trivially satisfied. Nonetheless, everything we will present continues to hold in this more general setting. For  $u, v \in A$  we introduce the following notation:
$$
\begin{array}{l} R _ {x} (u, v) := \frac {f (v) - f (u)}{\rho (x , v) ^ {\beta} + \rho (x , u) ^ {\beta}}, \\ F _ {x} (u, v) := f (u) + R _ {x} (u, v) \rho (x, u) ^ {\beta}, \\ R_{x}^{*}:= \sup_{u,v\in A}R_{x}(u,v)  , \\ W _ {x} (\varepsilon) := \left\{\left(u, v\right) \in A \times A: R _ {x} (u, v) > R _ {x} ^ {*} - \varepsilon \right\}, \quad 0 <   \varepsilon <   R _ {x} ^ {*} \\ \Phi_ {x} (\varepsilon) := \left\{F _ {x} (u, v): (u, v) \in W _ {x} (\varepsilon) \right\}. \\ \end{array}
$$
Definition A.2. We define the  $\beta$ -pointwise minimal slope extension ( $\beta$ -PMSE) to be the function  $F_A: \Omega \to \mathbb{R}$  satisfying
$$
F _ {A} (x) := \lim  _ {\varepsilon \rightarrow 0 ^ {+}} \Phi_ {x} (\varepsilon).
$$
In the degenerate case in which  $f(u) = f(v)$  for all  $u, v \in A$ , define  $F_A(x) \coloneqq f(u)$  for some (and hence any)  $u \in A$ .
Theorem A.3. Let  $\emptyset \neq A\subset \Omega$ ,  $f:\Omega \to [0,1]$ , such that Assumption A.1 holds. Then  $F_{A}:\Omega \to [0,1]$  is well defined, and satisfies for any  $x\in \Omega : \Lambda_{F_A}^\beta (x)\leq \Lambda_f^\beta (x)$ . Furthermore, if  $A$  is finite, then  $F_{A}(x)$  can be computed for any  $x\in \Omega$  within  $O(|A|^2)$  arithmetic operations.
Remark A.4. When  $R_{x}(\cdot, \cdot)$  has a unique maximizer  $(u_{x}^{*}, v_{x}^{*}) \in A \times A$ , the definition of  $F_{A}$  simplifies to
$$
F _ {A} (x) = f \left(u _ {x} ^ {*}\right) + \frac {\rho \left(x , u _ {x} ^ {*}\right) ^ {\beta}}{\rho \left(x , u _ {x} ^ {*}\right) ^ {\beta} + \rho \left(x , v _ {x} ^ {*}\right) ^ {\beta}} \left(f \left(v _ {x} ^ {*}\right) - f \left(u _ {x} ^ {*}\right)\right). \tag {3}
$$
We conclude that under Assumption A.1, we can assume without loss of generality that for each  $x \in \Omega$  there is such a unique maximizer (since the function is well defined, thus does not depend on the choice of the maximizer). Furthermore, this readily shows that when  $A$  is finite, we can compute  $F_{A}(x)$  for any  $x \in \Omega$  within  $O(|A|^{2})$  arithmetic operations — simply by finding this maximizer.
Proof. (of Theorem A.3)
We will assume that there exist  $u, v \in A$  such that  $f(u) \neq f(v)$ , since the degenerate (constant extension) case is trivial to verify. This assumption implies that  $R_{x}^{*} > 0$ . It is also easy to verify that  $\sup_{x \in \Omega} R_{x}^{*} < \infty \iff \|f\|_{\mathrm{H}\circ \mathrm{l}^{\beta}} < \infty$ .
Lemma A.5.  $F_{A}$  is well defined. Namely, under Assumption A.1 the limit  $\lim_{\varepsilon \to 0^{+}}\Phi_x(\varepsilon)\in [0,1]$  exists.
Proof. Fix  $x \in \Omega$  (we will omit the  $x$  subscripts from now on). Let  $\varepsilon < R^{*} / 2$ ,  $(u, v), (u', v') \in W(\varepsilon)$ . Note that  $R(u, v) > 0$  and that  $F(u, v) = f(v) - R(u, v)\rho(x, v)^{\beta}$ . Hence
$$
f (u) \leq F (u, v) \leq f (v), \tag {4}
$$
and the same clearly holds if we replace  $(u,v)$  by  $(u^{\prime},v^{\prime})$ . Assume without loss of generality that  $F(u,v)\leq F(u^{\prime},v^{\prime})$ , hence  $f(u)\leq F(u,v)\leq F(u^{\prime},v^{\prime})\leq f(v^{\prime})$ . We get
$$
\begin{array}{l} R (u ^ {\prime}, v ^ {\prime}) + \varepsilon > R ^ {*} \\ \geq \frac {f (v ^ {\prime}) - f (u)}{\rho (x , v ^ {\prime}) ^ {\beta} + \rho (x , u) ^ {\beta}} \\ = \frac {f \left(v ^ {\prime}\right) - F \left(u ^ {\prime} , v ^ {\prime}\right) + F (u , v) - f (u)}{\rho \left(x , v ^ {\prime}\right) ^ {\beta} + \rho (x , u) ^ {\beta}} + \frac {F \left(u ^ {\prime} , v ^ {\prime}\right) - F (u , v)}{\rho \left(x , v ^ {\prime}\right) ^ {\beta} + \rho (x , u) ^ {\beta}} \\ = \frac {R \left(u ^ {\prime} , v ^ {\prime}\right) \rho \left(x , v ^ {\prime}\right) ^ {\beta} + R (u , v) \rho \left(x , u\right) ^ {\beta}}{\rho \left(x , v ^ {\prime}\right) ^ {\beta} + \rho (x , u) ^ {\beta}} + \frac {F \left(u ^ {\prime} , v ^ {\prime}\right) - F (u , v)}{\rho \left(x , v ^ {\prime}\right) ^ {\beta} + \rho (x , u) ^ {\beta}} \\ \geq \frac {R \left(u ^ {\prime} , v ^ {\prime}\right) \rho \left(x , v ^ {\prime}\right) ^ {\beta} + \left(R \left(u ^ {\prime} , v ^ {\prime}\right) - \varepsilon\right) \rho \left(x , u\right) ^ {\beta}}{\rho \left(x , v ^ {\prime}\right) ^ {\beta} + \rho \left(x , u\right) ^ {\beta}} + \frac {F \left(u ^ {\prime} , v ^ {\prime}\right) - F (u , v)}{2 \operatorname {d i a m} (A) ^ {\beta}} \\ \geq R (u ^ {\prime}, v ^ {\prime}) - \varepsilon + \frac {F (u ^ {\prime} , v ^ {\prime}) - F (u , v)}{2 \operatorname {d i a m} (A) ^ {\beta}} \\ \Longrightarrow | F _ {x} (u, v) - F _ {x} \left(u ^ {\prime}, v ^ {\prime}\right) | \leq 4 \varepsilon \operatorname {d i a m} (A) ^ {\beta}. \\ \end{array}
$$