SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To compute the exact formula, we need to obtain  $S_{\pi_n}$  and  $S_{\pi_n}^{-1}$ . Since the loss function is  $\mathcal{L}(\hat{y}, y) = (\hat{y} - y)^2$ , we have
$$
S _ {\pi_ {n}} (f) = 2 \lambda_ {0} f + \mathbb {E} _ {\pi_ {n}} [ \mathcal {L} ^ {\prime \prime} (Y, g _ {\pi_ {n}, \lambda_ {0}} (X)) f (X) \Phi (X) ] = 2 \lambda_ {0} f + \frac {2}{n} \sum_ {j = 1} ^ {n} f (x _ {j}) \Phi (x _ {j}).
$$
Suppose  $S_{\pi_n}^{-1}(2\lambda_0g_{\pi_n,\lambda_0}) = \tilde{g}_1$ . Then at  $x_0$ , we have
$$
2 \lambda_ {0} g _ {\pi_ {n}, \lambda_ {0}} (x _ {0}) = S _ {\pi_ {n}} (\tilde {g} _ {1} (x _ {0})) = 2 \lambda_ {0} \tilde {g} _ {1} (x _ {0}) + \frac {2}{n} \sum_ {j = 1} ^ {n} \tilde {g} _ {1} (x _ {j}) k (x _ {0}, x _ {j}).
$$
$$
\tilde {g} _ {1} (x _ {0}) = g _ {\pi_ {n}, \lambda_ {0}} (x _ {0}) - \frac {1}{\lambda_ {0} n} \sum_ {j = 1} ^ {n} \tilde {g} _ {1} (x _ {j}) k (x _ {0}, x _ {j}).
$$
This implies that we need to evaluate  $\tilde{g}_1(x_j)$  on training data first, which is straightforward by letting  $x_0 = x_1,\ldots ,x_n$ :
$$
\tilde {g} _ {1} (\boldsymbol {x}) = g _ {\pi_ {n}, \lambda_ {0}} (\boldsymbol {x}) - \frac {1}{\lambda_ {0} n} \boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) \tilde {g} _ {1} (\boldsymbol {x})
$$
$$
\tilde {g} _ {1} (\boldsymbol {x}) = \left(\boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) + \lambda_ {0} n \boldsymbol {I}\right) ^ {- 1} \left(\lambda_ {0} n \boldsymbol {I}\right) g _ {\pi_ {n}, \lambda_ {0}} (\boldsymbol {x})
$$
$$
\begin{array}{l} \tilde {g} _ {1} (x _ {0}) = g _ {\pi_ {n}, \lambda_ {0}} (x _ {0}) - \frac {1}{\lambda_ {0} n} \boldsymbol {k} (x _ {0}, \boldsymbol {x}) ^ {T} \tilde {g} _ {1} (\boldsymbol {x}) \\ = g _ {\pi_ {n}, \lambda_ {0}} (x _ {0}) - \boldsymbol {k} (x _ {0}, \boldsymbol {x}) ^ {T} (\boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) + \lambda_ {0} n \boldsymbol {I}) ^ {- 1} g _ {\pi_ {n}, \lambda_ {0}} (\boldsymbol {x}) \\ \end{array}
$$
Next we compute  $S_{\pi_n}^{-1}\Phi (z_x) = \tilde{g}_2$  . At  $x_0$  ,we have
$$
k \left(z _ {x}, x _ {0}\right) = \Phi \left(z _ {x}\right) \left(x _ {0}\right) = S _ {\pi_ {n}} \left(\tilde {g} _ {2} \left(x _ {0}\right)\right) = 2 \lambda_ {0} \tilde {g} _ {2} \left(x _ {0}\right) + \frac {2}{n} \sum_ {j = 1} ^ {n} \tilde {g} _ {2} \left(x _ {j}\right) k \left(x _ {0}, x _ {j}\right)
$$
$$
\tilde {g} _ {2} (x _ {0}) = \frac {1}{2 \lambda_ {0}} k (z _ {x}, x _ {0}) - \frac {1}{\lambda_ {0} n} \sum_ {j = 1} ^ {n} \tilde {g} _ {2} (x _ {j}) k (x _ {0}, x _ {j})
$$
This implies that we need to evaluate  $\tilde{g}_2(x_j)$  on training data first, which is straightforward by letting  $x_0 = x_1,\ldots ,x_n$ :
$$
\tilde {g} _ {2} (\boldsymbol {x}) = \frac {1}{2 \lambda_ {0}} \boldsymbol {k} \left(z _ {x}, \boldsymbol {x}\right) - \frac {1}{\lambda_ {0} n} \boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) \tilde {g} _ {2} (\boldsymbol {x})
$$
$$
\tilde {g} _ {2} (\boldsymbol {x}) = \left(\boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) + \lambda_ {0} n \boldsymbol {I}\right) ^ {- 1} \left(\frac {n}{2} \boldsymbol {I}\right) \boldsymbol {k} \left(z _ {x}, \boldsymbol {x}\right)
$$
$$
\begin{array}{l} \tilde {g} _ {2} (x _ {0}) = \frac {1}{2 \lambda_ {0}} k (z _ {x}, x _ {0}) - \frac {1}{\lambda_ {0} n} \boldsymbol {k} (x _ {0}, \boldsymbol {x}) ^ {T} \tilde {g} _ {2} (\boldsymbol {x}) \\ = \frac {1}{2 \lambda_ {0}} k (z _ {x}, x _ {0}) - \frac {1}{2 \lambda_ {0}} \pmb {k} (x _ {0}, \pmb {x}) ^ {T} (\pmb {k} (\pmb {x}, \pmb {x}) + \lambda_ {0} n \pmb {I}) ^ {- 1} \pmb {k} (z _ {x}, \pmb {x}) \\ \end{array}
$$
Combing previous results, we obtain
$$
\begin{array}{l} I F (z; T, \pi_ {n}) (x _ {0}) \\ = - g _ {\pi_ {n}, \lambda_ {0}} (x _ {0}) + \boldsymbol {k} (x _ {0}, \boldsymbol {x}) ^ {T} (\boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) + \lambda_ {0} n \boldsymbol {I}) ^ {- 1} g _ {\pi_ {n}, \lambda_ {0}} (\boldsymbol {x}) \\ + 2 \left(z _ {y} - g _ {\pi_ {n}, \lambda_ {0}} \left(z _ {x}\right)\right) \left(\frac {1}{2 \lambda_ {0}} k \left(z _ {x}, x _ {0}\right) - \frac {1}{2 \lambda_ {0}} \boldsymbol {k} \left(x _ {0}, \boldsymbol {x}\right) ^ {T} \left(\boldsymbol {k} \left(\boldsymbol {x}, \boldsymbol {x}\right) + \lambda_ {0} n \boldsymbol {I}\right) ^ {- 1} \boldsymbol {k} \left(z _ {x}, \boldsymbol {x}\right)\right) \\ = \boldsymbol {k} \left(x _ {0}, \boldsymbol {x}\right) ^ {T} \left(\boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) + \lambda_ {0} n \boldsymbol {I}\right) ^ {- 1} \left(g _ {\pi_ {n}, \lambda_ {0}} (\boldsymbol {x}) - \frac {1}{\lambda_ {0}} \left(z _ {y} - g _ {\pi_ {n}, \lambda_ {0}} \left(z _ {x}\right)\right) \boldsymbol {k} \left(z _ {x}, \boldsymbol {x}\right)\right) \\ - g _ {\pi_ {n}, \lambda_ {0}} (x _ {0}) + \frac {1}{\lambda_ {0}} (z _ {y} - g _ {\pi_ {n}, \lambda_ {0}} (z _ {x})) k (z _ {x}, x _ {0}) \\ = \boldsymbol {k} (x _ {0}, \boldsymbol {x}) ^ {T} (\boldsymbol {k} (\boldsymbol {x}, \boldsymbol {x}) + \lambda_ {0} n \boldsymbol {I}) ^ {- 1} M _ {z} (\boldsymbol {x}) - M _ {z} (x _ {0}) \\ \end{array}
$$
Proof of Theorem B.7. Recall from Section B.2, we use the equivalence notations:
$$
I F (z; T, P) = T _ {P} ^ {\prime} (\delta_ {z} - P).
$$
First, we prove the following Claim:
$$
\sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \| T _ {\pi} ^ {\prime} \left(\delta_ {z} - \pi\right) - T _ {\pi_ {n}} ^ {\prime} \left(\delta_ {z} - \pi_ {n}\right) \| _ {\mathcal {H}} \rightarrow 0, \quad a. s. \tag {23}
$$
From Lemma A.6. and Lemma A.7. in [50], we have
$$
T _ {P} ^ {\prime} (Q) = - S _ {P} ^ {- 1} (W _ {P} (Q)).
$$
In this equation,
$$
W _ {P}: \operatorname {l i n} \left(B _ {S}\right)\rightarrow \mathcal {H}, \quad Q \mapsto \mathbb {E} _ {Q} \left[ \mathcal {L} ^ {\prime} \left(Y, g _ {P, \lambda_ {0}} (X)\right) \Phi (X) \right]
$$
is a continuous linear operator on  $\mathrm{lin}(B_S)$  where the space  $\mathrm{lin}(B_S)$  is defined in [50]. Moreover, the operator norm of  $W_{P}$  satisfies  $\| W_P\| \leq 1$ . In addition,
$$
S _ {P}: \mathcal {H} \rightarrow \mathcal {H}, \quad f \mapsto 2 \lambda_ {0} f + \mathbb {E} _ {P} \left[ \mathcal {L} ^ {\prime \prime} (Y, g _ {P, \lambda_ {0}} (X)) f (X) \Phi (X) \right]
$$
is an invertible continuous linear operator on  $\mathcal{H}$  where  $\Phi$  is the feature map of  $\mathcal{H}$ . This implies that  $S_P^{-1}:\mathcal{H}\to \mathcal{H}$  is also a continuous linear operator on  $\mathcal{H}$ .
Next, we apply Lemma A.7 in [50] by considering the following two sequences: The first sequence is  $\delta_z - \pi_n \in \mathrm{lin}(B_S)$  which obviously satisfies
$$
\left\|\left(\delta_ {z} - \pi_ {n}\right) - \left(\delta_ {z} - \pi\right)\right\| _ {\infty} = \| \pi - \pi_ {n} \| _ {\infty} \rightarrow 0, \quad a. s.
$$
The second sequence is  $\pi_n\in B_S$  which obviously satisfies
$$
\| \pi - \pi_ {n} \| _ {\infty} \to 0, \quad a. s.
$$
Following the proof of Lemma A.7 in [50], we have that
$$
\begin{array}{l} \left\| T _ {\pi} ^ {\prime} \left(\delta_ {z} - \pi\right) - T _ {\pi_ {n}} ^ {\prime} \left(\delta_ {z} - \pi_ {n}\right) \right\| _ {\mathcal {H}} \\ = \| S _ {\pi} ^ {- 1} \left(W _ {\pi} \left(\delta_ {z} - \pi\right)\right) - S _ {\pi_ {n}} ^ {- 1} \left(W _ {\pi_ {n}} \left(\delta_ {z} - \pi_ {n}\right)\right) \| _ {\mathcal {H}} \\ \leq \left\| S _ {\pi} ^ {- 1} \left(W _ {\pi} \left(\delta_ {z} - \pi\right)\right) - S _ {\pi} ^ {- 1} \left(W _ {\pi_ {n}} \left(\delta_ {z} - \pi\right)\right) \right\| _ {\mathcal {H}} + \left\| S _ {\pi} ^ {- 1} \left(W _ {\pi_ {n}} \left(\delta_ {z} - \pi\right)\right) - S _ {\pi} ^ {- 1} \left(W _ {\pi_ {n}} \left(\delta_ {z} - \pi_ {n}\right)\right) \right\| _ {\mathcal {H}} \\ + \| S _ {\pi} ^ {- 1} \left(W _ {\pi_ {n}} \left(\delta_ {z} - \pi_ {n}\right)\right) - S _ {\pi_ {n}} ^ {- 1} \left(W _ {\pi_ {n}} \left(\delta_ {z} - \pi_ {n}\right)\right) \| _ {\mathcal {H}} \\ \leq \| S _ {\pi} ^ {- 1} \| \| W _ {\pi} (\delta_ {z} - \pi) - W _ {\pi_ {n}} (\delta_ {z} - \pi) \| _ {\mathcal {H}} + \| S _ {\pi} ^ {- 1} \| \| W _ {\pi_ {n}} \| \| (\delta_ {z} - \pi) - (\delta_ {z} - \pi_ {n}) \| _ {\infty} \\ + \| S _ {\pi} ^ {- 1} - S _ {\pi_ {n}} ^ {- 1} \| \| W _ {\pi_ {n}} \| \| \delta_ {z} - \pi_ {n} \| _ {\infty} \\ \leq \| S _ {\pi} ^ {- 1} \| \| W _ {\pi} (\pi) - W _ {\pi_ {n}} (\pi) \| _ {\mathcal {H}} + \| S _ {\pi} ^ {- 1} \| \| W _ {\pi_ {n}} \| \| \pi - \pi_ {n} \| _ {\infty} \quad (s i n c e W _ {\pi_ {n}} i s a l i n e a r o p e r a t o r) \\ + \left\| S _ {\pi} ^ {- 1} - S _ {\pi_ {n}} ^ {- 1} \right\| \left\| W _ {\pi_ {n}} \right\| \left(\left\| \delta_ {z} - \pi \right\| _ {\infty} + \left\| \pi - \pi_ {n} \right\| _ {\infty}\right) \\ \end{array}
$$
Therefore, we only need to study the three terms in the last equation. Note that the first two terms are independent of  $z$ , and thus it follows from Step 3 and Step 4 in the proof of Lemma A.7 in [50] that
$$
\begin{array}{l} \sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \| S _ {\pi} ^ {- 1} \| \| W _ {\pi} (\pi) - W _ {\pi_ {n}} (\pi) \| _ {\mathcal {H}} + \| S _ {\pi} ^ {- 1} \| \| W _ {\pi_ {n}} \| \| \pi - \pi_ {n} \| _ {\infty} \\ = \| S _ {\pi} ^ {- 1} \| W _ {\pi} (\pi) - W _ {\pi_ {n}} (\pi) \| _ {\mathcal {H}} + \| S _ {\pi} ^ {- 1} \| \| W _ {\pi_ {n}} \| \| \pi - \pi_ {n} \| _ {\infty} \rightarrow 0, \quad a. s. \\ \end{array}
$$
To show that the third term also satisfies that
$$
\sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \| S _ {\pi} ^ {- 1} - S _ {\pi_ {n}} ^ {- 1} \| \| W _ {\pi_ {n}} \| \left(\| \delta_ {z} - \pi \| _ {\infty} + \| \pi - \pi_ {n} \| _ {\infty}\right)\rightarrow 0, \quad a. s.,
$$
we only need to note the following fact:
1)  $\| S_{\pi}^{-1} - S_{\pi_n}^{-1} \| \to 0$ , a.s., by Step 2 in the proof of Lemma A.7 in [50]. Moreover, this equation is independent of  $z$ .
2)  $\| W_{\pi_n} \| \leq 1$  by Step 1 in the proof of Lemma A.7 in [50]. Moreover, this equation is independent of  $z$ .
3)  $\| \pi - \pi_n \|_{\infty} < +\infty$ , a.s., since  $\| \pi - \pi_n \|_{\infty} \to 0$ , a.s.. Moreover, this equation is independent of  $z$ .
4)  $\sup_{z\in \mathcal{X}\times \mathcal{Y}}\| \delta_z - \pi \|_\infty < + \infty$  . To see this, we note that by definition
$$
\sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \| \delta_ {z} - \pi \| _ {\infty} = \sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \sup  _ {g \in \mathcal {G}} \left| \int g d (\delta_ {z} - \pi) \right| = \sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \sup  _ {g \in \mathcal {G}} \left| g (z) - \int g d \pi \right| \leq 2 \sup  _ {g \in \mathcal {G}} \| g \| _ {\infty}
$$
where the last term is independent of  $z$  and the space  $\mathcal{G} = \mathcal{G}_1 \cup \mathcal{G}_2 \cup \{b\}$  is defined in [50].  $\sup_{g \in \mathcal{G}_1} \| g \|_{\infty} \leq 1$  by the definition of  $\mathcal{G}_1$ .  $\sup_{g \in \{b\}} \| g \|_{\infty} < +\infty$  by our additional assumption on  $b$ . Since both  $\mathcal{X}$  and  $\mathcal{Y}$  are bounded and closed,  $\sup_{x \in \mathcal{X}} \sqrt{k(x, x)}$  is bounded above by, say  $\kappa < \infty$ . Thus, for every  $h \in \mathcal{H}$  with  $\| h \|_{\mathcal{H}} \leq C_1$ , we have  $\| h \|_{\infty} \leq C_1 \kappa$ . By definition of  $\mathcal{G}_2$ , for any  $g \in \mathcal{G}_2$ , we can write  $g(x, y) = \mathcal{L}'(y, f_0(x)) f_1(x)$  with  $\| f_0 \|_{\mathcal{H}} \leq c_0$  and  $\| f_1 \|_{\mathcal{H}} \leq 1$ . Note that  $\| f_1 \|_{\mathcal{H}} \leq 1$  implies that  $\| f_1 \|_{\infty} \leq \kappa$  and  $\| f_0 \|_{\mathcal{H}} \leq c_0$  implies that  $\| f_0 \|_{\infty} \leq c_0 \kappa$ . Thus Assumption B.4 shows that  $\| \mathcal{L}'(y, f_0(x)) \|_{\infty} \leq b_{c_0 \kappa}'(y)$  which is bounded on  $\mathcal{Y}$  by our additional assumption (uniformly for  $f_0$ ). Hence we conclude that  $\sup_{g \in \mathcal{G}_2} \| g \|_{\infty} \leq \kappa \sup_{y \in \mathcal{Y}} b_{c_0 \kappa}'(y) < +\infty$ . Summarizing the above discussion, we obtain that
$$
\sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \| \delta_ {z} - \pi \| _ {\infty} \leq 2 \sup  _ {g \in \mathcal {G}} \| g \| _ {\infty} <   + \infty
$$
Combining the above points 1)-4), we conclude that
$$
\sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \| S _ {\pi} ^ {- 1} - S _ {\pi_ {n}} ^ {- 1} \| \| W _ {\pi_ {n}} \| \left(\| \delta_ {z} - \pi \| _ {\infty} + \| \pi - \pi_ {n} \| _ {\infty}\right) \to 0, \quad a. s.
$$
Hence, we obtain our Claim (23).
Applying  $k_{x_0}$  to (23), the reproducing property of  $k$  implies that
$$
\sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \left| T _ {\pi} ^ {\prime} (\delta_ {z} - \pi) (x _ {0}) - T _ {\pi_ {n}} ^ {\prime} (\delta_ {z} - \pi_ {n}) (x _ {0}) \right| \to 0, \quad a. s.
$$
Note that since  $\mathcal{X}$  and  $\mathcal{Y}$  are bounded and closed by our assumptions, we have that
$$
\begin{array}{l} \left| \int_ {z \in \mathcal {X} \times \mathcal {Y}} \left(T _ {\pi} ^ {\prime} \left(\delta_ {z} - \pi\right) \left(x _ {0}\right)\right) ^ {2} d \pi (z) - \int_ {z \in \mathcal {X} \times \mathcal {Y}} \left(T _ {\pi_ {n}} ^ {\prime} \left(\delta_ {z} - \pi_ {n}\right) \left(x _ {0}\right)\right) ^ {2} d \pi (z) ] \right| \\ \leq \left| \pi (\mathcal {X} \times \mathcal {Y}) \right| \times \sup  _ {z \in \mathcal {X} \times \mathcal {Y}} \left| T _ {\pi} ^ {\prime} \left(\delta_ {z} - \pi\right) \left(x _ {0}\right) - T _ {\pi_ {n}} ^ {\prime} \left(\delta_ {z} - \pi_ {n}\right) \left(x _ {0}\right) \right| \\ \times \left| \int_ {z \in \mathcal {X} \times \mathcal {Y}} \left(T _ {\pi} ^ {\prime} \left(\delta_ {z} - \pi\right) \left(x _ {0}\right) + T _ {\pi_ {n}} ^ {\prime} \left(\delta_ {z} - \pi_ {n}\right) \left(x _ {0}\right)\right) d \pi (z) ] \right| \\ \rightarrow 0, \quad a. s. \\ \end{array}
$$
where we use the fact that  $\int_{z\in \mathcal{X}\times \mathcal{Y}}(T_{\pi}'(\delta_z - \pi)(x_0))^2 d\pi (z) = \xi^2 (x_0) < + \infty$  . On the other hand, it follows from the strong law of large numbers that
$$
\frac {1}{n} \sum_ {z _ {i} \in \mathcal {D}} \left(T _ {\pi_ {n}} ^ {\prime} (\delta_ {z _ {i}} - \pi_ {n}) (x _ {0})\right) ^ {2} - \int_ {z \in \mathcal {X} \times \mathcal {Y}} \left(T _ {\pi_ {n}} ^ {\prime} (\delta_ {z} - \pi_ {n}) (x _ {0})\right) ^ {2} d \pi (z) \to 0, a. s.
$$
Hence, we conclude that
$$
\frac {1}{n} \sum_ {z _ {i} \in \mathcal {D}} \left(T _ {\pi_ {n}} ^ {\prime} (\delta_ {z _ {i}} - \pi_ {n}) (x _ {0})\right) ^ {2} - \int_ {z \in \mathcal {X} \times \mathcal {Y}} \left(T _ {\pi} ^ {\prime} (\delta_ {z} - \pi) (x _ {0})\right) ^ {2} d \pi (z) \to 0, a. s.
$$
For confidence intervals, the proof is straightforward. Theorem B.5 shows that as  $n\to \infty$  we have
$$
\frac {g _ {\pi_ {n} , \lambda_ {n}} (x _ {0}) - g _ {\pi , \lambda_ {0}} (x _ {0})}{\xi (x _ {0}) / \sqrt {n}} \Rightarrow \mathcal {N} (0, 1)
$$
By Slutsky's theorem and  $\frac{\xi(x_0)}{\hat{\xi}(x_0)} \to 1$ , a.s., we have
$$
\frac {g _ {\pi_ {n} , \lambda_ {n}} (x _ {0}) - g _ {\pi , \lambda_ {0}} (x _ {0})}{\hat {\xi} (x _ {0}) / \sqrt {n}} \Rightarrow \mathcal {N} (0, 1)
$$
This implies that asymptotically as  $n\to \infty$  , we have
$$
\mathbb {P} \left(- q _ {1 - \frac {\alpha}{2}} \leq \frac {g _ {\pi_ {n} , \lambda_ {n}} (x _ {0}) - g _ {\pi , \lambda_ {0}} (x _ {0})}{\hat {\xi} (x _ {0}) / \sqrt {n}} \leq q _ {1 - \frac {\alpha}{2}}\right)\rightarrow 1 - \alpha
$$
$$
\left[ g _ {\pi_ {n}, \lambda_ {n}} (x _ {0}) - \frac {\hat {\xi} (x _ {0})}{\sqrt {n}} q _ {1 - \frac {\alpha}{2}}, g _ {\pi_ {n}, \lambda_ {n}} (x _ {0}) + \frac {\hat {\xi} (x _ {0})}{\sqrt {n}} q _ {1 - \frac {\alpha}{2}} \right]
$$
is an asymptotically exact  $(1 - \alpha)$ -level confidence interval of  $g_{\pi, \lambda_0}(x)$ .
Appendix F Experiments: Details and More Results
F.1 Experimental Details
We provide more details about our experimental implementation in Section 4.
Throughout our experiments, we use a two-layer fully-connected neural network as the base predictor based on the NTK specifications in Section C (Proposition C.3). However, we need to resolve the conflicts between the theoretical assumptions therein (e.g., continuous-time gradient flow) and practical implementation (e.g., discrete-time gradient descent), and at the same time, guarantee that the training procedure indeed operates in the NTK regime so that the first-order Taylor expansion (linearized neural network assumption) works well [64, 34, 80, 118]. Therefore, we will use the following specifications in all experiments.
1. The network adopts the NTK parametrization, and its parameters are randomly initialized using He initialization. The ReLU activation function is used in the network.
2. The network has  $32 \times n$  hidden neurons in its hidden layer where  $n$  is the size of the entire training data. The network should be sufficiently wide so that the NTK theory holds.
3. The network is trained using the regularized square loss (1) with regularization hyperparameter  $\lambda_{n} \equiv 0.1^{10}$ .
4. The network is trained using the (full) batch gradient descent (by feeding the whole dataset).
5. The learning rate and training epochs are properly tuned based on the specific dataset. The epochs should not be too small since the training needs to converge to a good solution, but the learning rate should also not be too large because we need to stipulate that the training procedure indeed operates in the NTK regime (an area around the initialization). Note that in practice, we cannot use the continuous-time gradient flow, and the network can never be infinitely wide. Therefore, with the fixed learning rate in gradient descent, we do not greatly increase the number of epochs so that the training procedure will likely find a solution that is not too far from the initialization.
6. We set  $m' = 4$  in the PNC-enhanced batching approach and  $R = 4$  in the PNC-enhanced cheap bootstrap approach.
7. In DropoutUQ, the dropout rate is a crucial hyper-parameter. We find that the dropout rate has a significant impact on the interval width of DropoutUQ: A large dropout rate produces a wide interval since the dropout rate is linked to the variance of the prior Gaussian distribution [43]. Therefore, to make fair comparisons between different approaches, we adjust the dropout rate in DropoutUQ so that they have a similar or larger interval width as PNC-enhanced batching or PNC-enhanced cheap bootstrap.
8. All experiments are conducted on a single GeForce RTX 2080 Ti GPU.
F.2 Additional Experiments
In this section, we present additional experimental results on more datasets. These experimental results further support our observations and claims in Section 4, demonstrating the robustness and effectiveness of our proposals.
First, we consider additional synthetic datasets.
Synthetic Datasets #2:  $X \sim \mathrm{Unif}([0, 0.2]^d)$  and  $Y \sim \sum_{i=1}^{d} X^{(i)} \sin(X^{(i)}) + \mathcal{N}(0, 0.001^2)$ . The training set  $\mathcal{D} = \{(x_i, y_i) : i = 1, \dots, n\}$  is formed by i.i.d. samples of  $(X, Y)$  with sample size  $n$  from the above data generating process. We use  $x_0 = (0.1, 0.1, \dots, 0.1)$  and  $y_0 = \sum_{i=1}^{d} 0.1 \sin(0.1)$  as the fixed test point in the confidence interval task. The rest of the setup is the same as Section 4. The implementation specifications are the same as in Section F.1. The results for constructing confidence intervals are displayed in Table 3, and the results for reducing procedural variability are displayed in Table 4.
Next, we consider real-world datasets.
It is worth mentioning the main reason for adopting synthetic datasets in our experiments. That is, precise evaluation of a confidence interval requires the following two critical components, which nevertheless are not available in real-world data: 1) One needs to know the ground-truth regression function to check the coverage of confidence intervals, while the label in real-world data typically contains some aleatoric noise. 2) To estimate the coverage rate of the confidence interval, we need to repeat the experiments multiple times by regenerating new independent datasets from the same data distribution. In practice, we cannot regenerate new real-world datasets.
However, to conduct simulative experiments on realistic real-world data, we provide a possible way to mimic an evaluation of the confidence interval on real-world datasets as follows.
Step 1. For a given real-world dataset, select a fixed test data point  $(x_0,y_0)$  and a subset of the real-world data  $(x_{i},y_{i})$ $(i\in \mathcal{I})$  that does not include  $(x_0,y_0)$ .
Step 2. For each experimental repetition  $j \in [J]$ , add certain artificial independent noise on the label to obtain a "simulated" real-world dataset  $(x_i, y_i + \epsilon_{i,j})$  ( $i \in \mathcal{I}$ ) where  $\epsilon_{i,j}$  are all independent
Table 3: Confidence interval construction on synthetic datasets #2 with different data sizes  $n = 128,256,512,1024$  and different data dimensions  $d = 2,4,8,16$ . The CR results that attain the desired confidence level  $95\% / 90\%$  are in bold.
PNC-enhanced batching	PNC-enhanced cheap bootstrap	DropoutUQ
95%CI(CR/IW)	90%CI(CR/IW)	MP	95%CI(CR/IW)	90%CI(CR/IW)	MP	95%CI(CR/IW)	90%CI(CR/IW)	MP
(d=2)
n=128	0.98/0.0075	0.91/0.0056	0.0201	1.00/0.0343	0.99/0.0264	0.0184	0.99/0.0392	0.97/0.0330	0.0216
n=256	0.97/0.0058	0.93/0.0043	0.0197	0.97/0.0215	0.93/0.0165	0.0185	0.96/0.0213	0.94/0.0169	0.0214
n=512	0.96/0.0050	0.95/0.0037	0.0199	0.94/0.0148	0.90/0.0114	0.0186	0.96/0.0164	0.90/0.0135	0.0216
n=1024	0.96/0.0041	0.94/0.0030	0.0199	0.95/0.0115	0.93/0.0088	0.0203	0.91/0.0124	0.86/0.0102	0.0210
(d=4)
n=128	0.98/0.0111	0.97/0.0082	0.0395	1.00/0.0443	0.98/0.0340	0.0370	0.96/0.0462	0.93/0.0375	0.0441
n=256	0.98/0.0088	0.92/0.0065	0.0401	0.98/0.0346	0.94/0.0266	0.0374	0.98/0.0344	0.94/0.0282	0.0426
n=512	0.94/0.0073	0.91/0.0054	0.0403	0.99/0.0236	0.94/0.0181	0.0373	0.94/0.0310	0.88/0.0260	0.0427
n=1024	1.00/0.0055	0.91/0.0040	0.0399	0.94/0.0167	0.87/0.0128	0.0373	0.95/0.0241	0.90/0.0203	0.0417
(d=8)
n=128	0.97/0.0145	0.93/0.0107	0.0798	1.00/0.0683	1.00/0.0524	0.0849	0.94/0.0661	0.86/0.0548	0.0876
n=256	0.96/0.0110	0.94/0.0082	0.0801	0.99/0.0540	0.97/0.0414	0.0846	0.92/0.0538	0.91/0.0446	0.0853
n=512	0.97/0.0092	0.92/0.0068	0.0793	0.99/0.0393	0.96/0.0302	0.0846	0.91/0.0427	0.85/0.0359	0.0862
n=1024	0.95/0.0084	0.90/0.0062	0.0792	0.97/0.0284	0.95/0.0218	0.0820	0.85/0.0295	0.79/0.0248	0.0809
(d=16)
n=128	0.95/0.0214	0.94/0.0158	0.1608	1.00/0.1004	0.99/0.0771	0.1568	0.92/0.1124	0.87/0.0958	0.1814
n=256	0.97/0.0174	0.91/0.0129	0.1595	1.00/0.0725	1.00/0.0556	0.1578	0.95/0.0800	0.88/0.0677	0.1720
n=512	0.96/0.0140	0.92/0.0104	0.1604	0.99/0.0516	0.98/0.0396	0.1569	0.88/0.0565	0.80/0.0475	0.1689
n=1024	0.95/0.0114	0.88/0.0084	0.1599	0.98/0.0350	0.95/0.0269	0.1568	0.86/0.0584	0.83/0.0498	0.1685