SlowGuess/NeurIPS_2023_OCR huggingface dataset:
During the execution of Algorithm 1, two types of updates are made. The first type of update is done within the inner loop and corresponds to beginning from a predictor  $p_{\mathrm{old}}$  and acquiring  $p_{\mathrm{new}}$  which is the function  $\mathbf{x} \mapsto (p_{\mathrm{old}}(\mathbf{x}) + \sigma (\mathbf{w}\cdot \mathbf{x}))_{[0,1]}$ , where  $\mathbf{w}$  is the output of the weak learner of our Proposition A.1, run with  $\varepsilon \gets \varepsilon_3$  ( $\varepsilon_3$  will be specified later). To lower bound the decrease in the potential function during this type of update, we use a version of Lemma 7.6 in Gopalan et al. [2023]. In this case, one needs to pick a step size  $\sigma$  that is polynomially smaller than the guarantee that the
weak learner provides. In particular, if the weak learner of our Proposition A.1 is run with  $\varepsilon \gets \varepsilon_3$ , then one acquires (following the proof of Lemma 7.6 in Gopalan et al. [2023])
$$
\mathbf {E} [ (p ^ {*} (\mathbf {x}) - p _ {\mathrm {o l d}} (\mathbf {x})) ^ {2} ] - \mathbf {E} [ (p ^ {*} (\mathbf {x}) - p _ {\mathrm {n e w}} (\mathbf {x})) ^ {2} ] \geq \frac {\sigma \cdot \varepsilon_ {3}}{2} - B ^ {2} \lambda \sigma^ {2}.
$$
If  $\sigma$  is picked to be  $\sigma = \frac{\varepsilon_3}{4B^2\lambda}$ , then the quantity of interest  $\mathbf{E}[(p^{*}(\mathbf{x}) - p_{\mathrm{old}}(\mathbf{x}))^{2}] - \mathbf{E}[(p^{*}(\mathbf{x}) - p_{\mathrm{new}}(\mathbf{x}))^{2}]$  is lower bounded by  $\frac{\varepsilon_3^2}{16B^2\lambda}$ . Hence the number of iterations of the inner loop is upper bounded by  $O\left(\frac{B^2\lambda}{\varepsilon_3^2}\right)$ . Note that in their original algorithm,  $\sigma$  was picked equal to  $\varepsilon_3$  and this is why  $\varepsilon_3$  (or another corresponding parameter) does not appear in their proofs. The updated choice of  $\sigma$  generates a polynomial overhead in time and sample complexity. We pick  $\varepsilon_3 = \frac{1}{2} (\varepsilon_1 - B\sqrt{\lambda\delta})$  so that each time we exit the inner loop, we have that, with high probability, the current value of  $p$  corresponds to an  $(\varepsilon_1 - B\sqrt{\lambda\delta})$ -multiaccurate predictor.
The second type of update (the outer loop update) is a calibration step where  $p_{\mathrm{new}}$  is set to be the following function with the notation of Algorithm 1
$$
\mathbf {x} \mapsto \sum_ {j = 0} ^ {1 / \delta} \bar {y} _ {j} \mathbb {1} \{p _ {\mathrm {o l d}} (\mathbf {x}) \in I _ {j} \}
$$
In this case, Corollary 7.5 of Gopalan et al. [2023] can be used as is to acquire that  $\mathbf{E}[(p^{*}(\mathbf{x}) - p_{\mathrm{old}}(\mathbf{x}))^{2}] - \mathbf{E}[(p^{*}(\mathbf{x}) - p_{\mathrm{new}}(\mathbf{x}))^{2}] \geq \varepsilon_{1}^{2}/8$  if we set  $\delta \leq \varepsilon_{1}^{2}/C$  for some large enough universal constant  $C > 0$ . Hence, the number of iterations of the outer loop is upper bounded by  $O(1/\varepsilon_{1}^{2})$ .
It remains to show that once the algorithm terminates, the output is approximately calibrated and multiaccurate. Regarding multiaccuracy, whenever we exit the inner loop, the current value of  $p$  corresponds (with high probability) to an  $(\varepsilon_1 - B\sqrt{\lambda\delta})$ -multiaccurate predictor. The predictor  $p^\delta$  is close to  $p$  in absolute distance and therefore can be shown to be approximately multiaccurate using some version of Lemma 7.2 in Gopalan et al. [2023]. In particular, following the proof of Lemma 7.2, the multiaccuracy guarantee for  $p^\delta$  changes to  $(\mathcal{C}, \alpha + B\sqrt{\lambda\delta})$ -multiaccuracy, where  $\alpha$  is the multiaccuracy guarantee for  $p$ , by using a Cauchy-Schwarz inequality and bounding  $\mathbb{E}[(\mathbf{w} \cdot \mathbf{x})^2]$  by  $B^2 \cdot \lambda$ . Since  $\alpha = \varepsilon_1 - B\sqrt{\lambda\delta}$ , the multiaccuracy guarantee is  $\varepsilon_1$ . The calibration guarantee follows by the fact that the termination criterion corresponds to empirically checking whether  $p^\delta$  is calibrated and by using Lemma 7.4 in Gopalan et al. [2023], the empirical estimate should be close to the true calibration error. Overall, once we terminate, the output is  $\varepsilon_1$  calibrated and multiaccurate.
Since the time and sample complexity of each of the calls of the weak learner depends on  $\varepsilon_3$ , we pick  $\delta \gets \frac{\varepsilon_1^2}{C(1 + B^2\lambda)}$  so that  $\varepsilon_3 = \Theta (\varepsilon_1)$  and  $\delta \leq \frac{\varepsilon_1^2}{C}$  as required.
Sample Complexity. For each of the inner loop iterations, we require a fresh sample of size  $O\left(\frac{d^2B^2\lambda}{\varepsilon_1^2}\right)$ , as specified by Proposition A.1 (by setting the probability of success to a sufficiently small constant). For the outer loop, we require  $O\left(\frac{1 + B^8\lambda^4}{\varepsilon_1^8}\log \left(\frac{1 + B\lambda}{\varepsilon_1}\right)\right)$  samples per iteration, so that the calibration error estimate is accurate enough, according to Lemma 7.4 in Gopalan et al. [2023].
Time Complexity. Each of the inner loop iterations requires  $O\left(\frac{d^3B^2\lambda}{\varepsilon_1^2}\right)$  time and each of the outer loop iterations requires an additional  $O\left(d\frac{1 + B^2\lambda}{\varepsilon_1^2} + \frac{1 + B^{10}\lambda^5}{\varepsilon_1^{10}}\log \left(\frac{1 + B\lambda}{\varepsilon_1}\right)\right)$  time.
The final technical complication we need to address is that their algorithm works only given binary labels. We can, however, form binary labels as follows. Let  $(\mathbf{x},y)$  be drawn from  $D$ . We have that  $y \in [0,1]$ . Given  $y$ , we draw an independent Bernoulli random variable  $y'$  with probability of success  $y$ , forming the distribution  $D'$  over  $\mathbb{R}^d \times \{0,1\}$ . We run the algorithm of Gopalan et al. [2023] on  $D'$  and receive a predictor  $p$  such that
$$
\mathcal {L} _ {g} \left(f ^ {\prime} \circ p; D ^ {\prime}\right) \leq \min  _ {\mathbf {w} \in \mathcal {W}} \mathcal {L} _ {g} \left(\mathbf {w}; D ^ {\prime}\right) + \varepsilon , \text {f o r a n y} (f, g) \in \mathcal {F}
$$
We have that
$$
\begin{array}{l} \mathcal {L} _ {g} (c; D ^ {\prime}) = \underset {\mathbf {x}, y ^ {\prime}} {\mathbb {E}} [ g (c (\mathbf {x})) - y ^ {\prime} c (\mathbf {x}) ] \\ = \underset {\mathbf {x}} {\mathbb {E}} \left[ g (c (\mathbf {x})) - \underset {y} {\mathbb {E}} \left[ \underset {y ^ {\prime}} {\mathbb {E}} [ y ^ {\prime} | y ] \right| \mathbf {x} \right] c (\mathbf {x}) \Bigg ] \\ = \underset {\mathbf {x}} {\mathbb {E}} \left[ g (c (\mathbf {x})) - \underset {y} {\mathbb {E}} [ y \mid \mathbf {x} ] c (\mathbf {x}) \right] \\ = \underset {\mathbf {x}, y} {\mathbb {E}} \left[ g (c (\mathbf {x})) - y c (\mathbf {x}) \right] = \mathcal {L} _ {g} (c; D) \\ \end{array}
$$
This concludes the proof of Theorem 3.2.
C.2 Proof of Lemma 3.3
We first apply Theorem 2.1 with  $g^{\prime}\gets \phi^{\prime}$  to get that for  $\phi_{\mathbf{w}}^{\prime}(\mathbf{x}) = \phi^{\prime}(\mathbf{w}\cdot \mathbf{x})$  , we have
$$
\operatorname {e r r} _ {2} (p) \leq \frac {\beta}{\alpha} \operatorname {e r r} _ {2} \left(\phi_ {\mathbf {w} ^ {*}} ^ {\prime}\right) + 2 \beta \varepsilon
$$
(since inequality holds for  $\mathbf{w}\in \mathcal{W}$ )
Moreover, we have
$$
\begin{array}{l} \operatorname {e r r} _ {2} \left(\phi_ {\mathbf {w} ^ {*}} ^ {\prime}\right) = \mathbb {E} \left[ \left(y - \phi^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right)\right) ^ {2} \right] \\ = \mathbb {E} \left[ \left(y - g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right) + g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right) - \phi^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right)\right) ^ {2} \right] \\ \leq 2 \mathbb {E} \left[ \left(y - g ^ {\prime} (\mathbf {w} ^ {*} \cdot \mathbf {x})\right) ^ {2} \right] + 2 \mathbb {E} \left[ \left(g ^ {\prime} (\mathbf {w} ^ {*} \cdot \mathbf {x}) - \phi^ {\prime} (\mathbf {w} ^ {*} \cdot \mathbf {x})\right) ^ {2} \right] \\ = 2 \operatorname {o p t} _ {g} + 2 \mathbb {E} \left[ \left(g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right) - \phi^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right)\right) ^ {2} \right] \\ \end{array}
$$
This concludes the proof of lemma 3.3.
D Proofs from Section 4
D.1 Proof of Theorem 4.1
In the case we consider here,  $g'$  is the sigmoid activation, i.e.,  $g'(t) = (1 + e^{-t})^{-1}$  for any  $t \in \mathbb{R}$ . In particular, the pointwise surrogate loss we consider satisfies
$$
\ell_ {g} (y, f ^ {\prime} (p)) = y \ln \frac {1}{p} + (1 - y) \ln \frac {1}{1 - p} - \ln 2,
$$
for any  $y \in [0,1]$  and  $p \in (0,1)$ . We may extend Lemma 4.2 to also capture  $y \in \{0,1\}$ , by defining  $\ell_g(0,f'(0)) = \ell_g(1,f'(1)) = -\ln 2$  (the inequality would hold under this definition). Hence, following a similar procedure as the one used for proving Theorem B.1, we obtain the following by applying Lemma 4.2
$$
\operatorname {e r r} _ {2} (p) \leq 2 \left(\mathcal {L} _ {g} \left(f ^ {\prime} \circ p\right) - \mathbb {E} \left[ \ell_ {g} \left(y, f ^ {\prime} (y)\right) \right]\right) \tag {D.1}
$$
$$
\mathcal {L} _ {g} \left(\mathbf {w} ^ {*}\right) - \mathbb {E} \left[ \ell_ {g} (y, f ^ {\prime} (y)) \right] \leq \mathbb {E} \left[ \frac {2}{g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right) \vee \left(1 - g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right)\right)} \cdot \left(y - g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right)\right) ^ {2} \right] \tag {D.2}
$$
$$
\mathcal {L} _ {g} \left(f ^ {\prime} \circ p\right) \leq \mathcal {L} _ {g} \left(\mathbf {w} ^ {*}\right) + \varepsilon \tag {D.3}
$$
Therefore, in order to prove Theorem 4.1, it is sufficient to provide a strong enough upper bound for the quantity of the right hand side of Equation (D.2) in terms of  $\mathrm{opt}_g$ . We observe that
$$
\frac {2}{g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right) \vee \left(1 - g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right)\right)} \leq 4 \exp \left(\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right|\right), \text {f o r a n y} \mathbf {x} \in \mathbb {R} ^ {d}
$$
It remains to bound the quantity  $\mathbb{E}[e^{|\mathbf{w}^* \cdot \mathbf{x}|}(y - g_{\mathbf{w}}'(\mathbf{x}))^2]$ . Set  $Q = e^{|\mathbf{w}^* \cdot \mathbf{x}|}(y - g_{\mathbf{w}}'(\mathbf{x}))^2$  ( $Q$  is a random variable). Then for any  $r \geq 0$  we have
$$
\begin{array}{l} \mathbb {E} [ Q ] = \mathbb {E} [ Q \cdot \mathbb {1} \{| \mathbf {w} ^ {*} \cdot \mathbf {x} | \leq r \} ] + \mathbb {E} [ Q \cdot \mathbb {1} \{| \mathbf {w} ^ {*} \cdot \mathbf {x} | > r \} ] \\ \leq e ^ {r} \mathbb {E} [ (y - g _ {\mathbf {w} ^ {*}} ^ {\prime} (\mathbf {x})) ^ {2} \mathbb {1} \left\{\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right| \leq r \right\} ] + \mathbb {E} \left[ e ^ {\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right|} \mathbb {1} \left\{\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right| > r \right\} \right] \\ \leq e ^ {r} \cdot \operatorname {o p t} + \mathbb {E} \left[ e ^ {\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right|} \mathbb {1} \left\{\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right| > r \right\} \right] \\ \end{array}
$$
To bound the quantity  $\mathbb{E}[e^{|\mathbf{w}^* \cdot \mathbf{x}|} \mathbb{1}\{| \mathbf{w}^* \cdot \mathbf{x} | > r\}]$ , consider  $F(s) \coloneqq \operatorname*{Pr}[e^{|\mathbf{w}^* \cdot \mathbf{x}|} \mathbb{1}\{| \mathbf{w}^* \cdot \mathbf{x} | > r\} \geq s]$ . We have that
$$
F (s) = \mathbb {1} \{s = 0 \} \Pr [ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \leq r ] + \Pr [ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \geq \max  \{\ln s, r \} ].
$$
Since  $\mathbb{E}[e^{|\mathbf{w}^* \cdot \mathbf{x}|} \mathbb{1}\{|\mathbf{w}^* \cdot \mathbf{x}| > r\}] = \int_{s=0}^{\infty} F(s) ds = \int_{s=0}^{\infty} \Pr[|\mathbf{w}^* \cdot \mathbf{x}| \geq \max \{\ln s, r\}] ds$ , we obtain
$$
\begin{array}{l} \mathbb {E} [ Q ] = \int_ {s = 0} ^ {\infty} \Pr \left[ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \geq \max  \{\ln s, r \} \right] d s \\ \leq \int_ {s = 0} ^ {e ^ {r}} \Pr \left[ | \widehat {\mathbf {w}} ^ {*} \cdot \mathbf {x} | \geq \frac {r}{B} \right] d s + \int_ {s = e ^ {r}} ^ {\infty} \Pr \left[ | \widehat {\mathbf {w}} ^ {*} \cdot \mathbf {x} | \geq \frac {\ln s}{B} \right] d s \quad (\text {s i n c e} \| \mathbf {w} ^ {*} \| _ {2} \leq B) \\ \leq e ^ {r} \cdot e ^ {- (r / B) ^ {2}} + \int_ {s = e ^ {r}} ^ {\infty} e ^ {- (\ln s / B) ^ {2}} d s \tag {seeDef.1.6} \\ = e ^ {r} \cdot e ^ {- (r / B) ^ {2}} + e ^ {r} \cdot \int_ {u = 0} ^ {\infty} e ^ {u - (\frac {u + r}{B}) ^ {2}} d u \quad \text {(d e f i n e} u = \ln s - r) \\ \leq e ^ {r} \cdot e ^ {- (r / B) ^ {2}} + e ^ {r} \cdot e ^ {- (r / B) ^ {2}} \cdot B \cdot e ^ {\frac {B ^ {2}}{2}} \cdot \int_ {u = 0} ^ {\infty} e ^ {- (u - \frac {B}{2}) ^ {2}} d u \\ \leq C e ^ {B ^ {2}} e ^ {r} e ^ {- (r / B) ^ {2}} \\ \end{array}
$$
Therefore, in total, we have that  $\mathrm{err}_2(p) \leq 8e^r\mathrm{opt}_g + 8Ce^{B^2}e^r e^{-(r / B)^2} + 2\varepsilon$  and we may obtain Theorem 4.1 by picking  $r = B(\ln \frac{1}{\mathrm{opt}})^{1 / 2}$ .
D.2 Proof of Theorem 4.3
We first prove Lemma 4.4. We have that for any  $y \in \{0,1\}$  and  $p \in (0,1)$
$$
\ell_ {g} (y, f ^ {\prime} (p)) = y \ln \frac {1}{p} + (1 - y) \ln \frac {1}{1 - p} - \ln 2 = \mathrm {C E} (y, p) - \ln 2,
$$
where  $\mathrm{CE}(y,p)$  is the cross entropy function. It is sufficient to show that for  $y\in \{0,1\}$  and  $p\in [0,1]$ ,
$$
| y - p | \leq \operatorname {C E} (y, p) \leq 2 | y - p | \log \left(\frac {1}{p (1 - p)}\right) \tag {D.4}
$$
$$
\operatorname {C E} (0, p) = \log \left(\frac {1}{1 - p}\right) = \sum_ {i = 1} ^ {\infty} \frac {p ^ {i}}{i} \tag {D.5}
$$
where the series on the right converges for all  $p < 1$ . We can also write
$$
\operatorname {C E} (1, p) = \log \left(\frac {1}{p}\right) = \sum_ {i = 1} ^ {\infty} \frac {(1 - p) ^ {i}}{i} \tag {D.6}
$$
with the series converging for  $p > 0$ .
For the lower bound, we observe the following inequalities hold for all  $p \in [0,1]$
$$
\operatorname {C E} (0, p) \geq p = | 0 - p |
$$
$$
\operatorname {C E} (1, p) \geq 1 - p = | 1 - p |.
$$
For the upper bound, we first prove the claim for  $y = 0$ , where it states that
$$
\operatorname {C E} (0, p) = \log \left(\frac {1}{1 - p}\right) \leq 2 p \log \left(\frac {1}{p (1 - p)}\right). \tag {D.7}
$$
When  $p \leq 1/2$  we can use Eq. (D.5) to bound
$$
\operatorname {C E} (0, p) \leq \sum_ {i = 1} ^ {\infty} \frac {p ^ {i}}{i} \leq \sum_ {i = 1} ^ {\infty} p ^ {i} = \frac {p}{1 - p} \leq 2 p. \tag {D.8}
$$
The bounds hold by observing that since  $p(1 - p) \geq 1 / 4$ ,
$$
\log \left(\frac {1}{p (1 - p)}\right) \geq \log (4) \geq 1
$$
When  $p \geq 1/2$ , we note that
$$
\log \left(\frac {1}{p (1 - p)}\right) \geq \log \left(\frac {1}{1 - p}\right) = \operatorname {C E} (0, p)
$$
and  $2p \geq 1$ . Hence the bound holds in this case too.
In the case where  $y = 1$ , the bound states that
$$
\operatorname {C E} (1, p) = \log \left(\frac {1}{p}\right) \leq 2 (1 - p) \log \left(\frac {1}{p (1 - p)}\right).
$$
This is implied by our bound for  $y = 0$  by taking  $q = 1 - p$ . This concludes the proof of Lemma 4.4 and we are ready to prove Theorem 4.3. The following are true
$$
\operatorname {e r r} _ {1} (p) \leq \mathcal {L} _ {g} \left(f ^ {\prime} \circ p\right) - \ln 2 \tag {D.9}
$$
$$
\mathcal {L} _ {g} \left(\mathbf {w} ^ {*}\right) - \ln 2 \leq 2 \cdot \mathbb {E} \left[ \ln \left(\frac {1}{g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right) \cdot \left(1 - g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right)\right)}\right) \cdot | y - g ^ {\prime} \left(\mathbf {w} ^ {*} \cdot \mathbf {x}\right) | \right] \tag {D.10}
$$
$$
\mathcal {L} _ {g} \left(f ^ {\prime} \circ p\right) \leq \mathcal {L} _ {g} \left(\mathbf {w} ^ {*}\right) + \varepsilon \tag {D.11}
$$
Similarly to the proof of Theorem 4.1, we observe that
$$
\ln \left(\frac {1}{g ^ {\prime} (\mathbf {w} ^ {*} \cdot \mathbf {x}) \cdot (1 - g ^ {\prime} (\mathbf {w} ^ {*} \cdot \mathbf {x}))}\right) \leq \ln 4 + | \mathbf {w} ^ {*} \cdot \mathbf {x} |
$$
It remains to bound the quantity  $\mathbb{E}[|\mathbf{w}^* \cdot \mathbf{x}| \cdot |y - g_{\mathbf{w}}'(x)|]$ . Set  $Q = |\mathbf{w}^* \cdot \mathbf{x}| \cdot |y - g_{\mathbf{w}}'(x)|$  ( $Q$  is a random variable). Then for any  $r \geq 0$  we have
$$
\begin{array}{l} \mathbb {E} [ Q ] = \mathbb {E} [ Q \cdot \mathbb {1} \left\{\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right| \leq r \right\} ] + \mathbb {E} [ Q \cdot \mathbb {1} \left\{\left| \mathbf {w} ^ {*} \cdot \mathbf {x} \right| > r \right\} ] \\ \leq r \mathbb {E} [ | y - g _ {\mathbf {w} ^ {*}} ^ {\prime} (\mathbf {x}) | \cdot \mathbb {1} \{| \mathbf {w} ^ {*} \cdot \mathbf {x} | \leq r \} ] + \mathbb {E} [ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \cdot \mathbb {1} \{| \mathbf {w} ^ {*} \cdot \mathbf {x} | > r \} ] \\ \leq r \cdot \operatorname {o p t} + \mathbb {E} \left[ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \cdot \mathbb {1} \left\{| \mathbf {w} ^ {*} \cdot \mathbf {x} | > r \right\} \right] \\ \end{array}
$$
To bound the quantity  $\mathbb{E}[|\mathbf{w}^* \cdot \mathbf{x}| \cdot \mathbb{1}\{| \mathbf{w}^* \cdot \mathbf{x} | > r\}]$ , consider  $F(s) \coloneqq \operatorname*{Pr}[|\mathbf{w}^* \cdot \mathbf{x}| \cdot \mathbb{1}\{| \mathbf{w}^* \cdot \mathbf{x} | > r\} \geq s]$ . We have that
$$
F (s) = \mathbb {1} \{s = 0 \} \Pr [ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \leq r ] + \Pr [ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \geq \max  \{s, r \} ].
$$
Since  $\mathbb{E}[|\mathbf{w}^* \cdot \mathbf{x}| \cdot \mathbb{1}\{| \mathbf{w}^* \cdot \mathbf{x} | > r\}] = \int_{s=0}^{\infty} F(s) ds = \int_{s=0}^{\infty} \Pr[|\mathbf{w}^* \cdot \mathbf{x}| \geq \max\{s, r\}] ds$ , we obtain
$$
\begin{array}{l} \mathbb {E} [ Q ] = \int_ {s = 0} ^ {\infty} \Pr \left[ | \mathbf {w} ^ {*} \cdot \mathbf {x} | \geq \max  \{s, r \} \right] d s \\ \leq \int_ {s = 0} ^ {r} \Pr \left[ | \widehat {\mathbf {w}} ^ {*} \cdot \mathbf {x} | \geq \frac {r}{B} \right] d s + \int_ {s = r} ^ {\infty} \Pr \left[ | \widehat {\mathbf {w}} ^ {*} \cdot \mathbf {x} | \geq \frac {s}{B} \right] d s \quad (\text {s i n c e} \| \mathbf {w} ^ {*} \| _ {2} \leq B) \\ \leq r \cdot e ^ {- r / B} + \int_ {s = r} ^ {\infty} e ^ {- s / B} d s \tag {seeDef.1.6} \\ = r \cdot e ^ {- r / B} + B \cdot e ^ {- r / B} \\ \end{array}
$$
Therefore, in total, we have that  $\mathrm{err}_1(p) \leq 2\ln 4 \cdot \mathrm{opt}_g + 2(r + B)e^{-r / B} + 2r \cdot \mathrm{opt}_g + \varepsilon$  and we may obtain Theorem 4.3 by picking  $r = B \cdot \ln \frac{1}{\mathrm{opt}}$ .