SlowGuess/NeurIPS_2023_OCR huggingface dataset:
with  $\mathfrak{T}^{\mathrm{comp}}$  an  $\mathcal{H}$ -estimation error transformation for comp-sum losses defined for all  $t\in [0,1]$  by
$$
\mathfrak {T} ^ {\mathrm {c o m p}} (t) =
$$
$$
\left\{ \begin{array}{l l} \inf  _ {\tau \in [ 0, \frac {1}{2} ]   \mu \in [ - \tau , 1 - \tau ]} \sup  _ {\substack {\mu \in [ - \tau , 1 - \tau ]}} \Bigl \{\frac {1 + t}{2} \bigl [ \Phi (\tau) - \Phi (1 - \tau - \mu) \bigr ] + \frac {1 - t}{2} \bigl [ \Phi (1 - \tau) - \Phi (\tau + \mu) \bigr ] \Bigr \} & n = 2 \\ \inf  _ {P \in [ \frac {1}{n - 1} \vee t, 1 ]} \inf  _ {\substack {\tau_ {1} \geq \max  (\tau_ {2}, 1 / n) \\ \tau_ {1} + \tau_ {2} \leq 1, \tau_ {2} \geq 0}} \sup  _ {\mu \in [ - \tau_ {2}, \tau_ {1} ]} \Bigl \{\frac {P + t}{2} \bigl [ \Phi (\tau_ {2}) - \Phi (\tau_ {1} - \mu) \bigr ] + \frac {P - t}{2} \bigl [ \Phi (\tau_ {1}) - \Phi (\tau_ {2} + \mu) \bigr ] \Bigr \} & n > 2. \end{array} \right.
$$
Proof. For the comp-sum loss  $\ell^{\mathrm{comp}}$ , the conditional  $\ell^{\mathrm{comp}}$ -risk can be expressed as follows:
$$
\begin{array}{l} \mathcal {C} _ {\ell^ {\operatorname {c o m p}}} (h, x) = \sum_ {y \in \mathcal {Y}} p (x, y) \ell^ {\operatorname {c o m p}} (h, x, y) \\ = \sum_ {y \in \mathcal {Y}} p (x, y) \Phi \left(\frac {e ^ {h (x , y)}}{\sum_ {y ^ {\prime} \in \mathcal {Y}} e ^ {h (x , y ^ {\prime})}}\right) \\ = \sum_ {y \in \mathcal {Y}} p (x, y) \Phi \left(S _ {h} (x, y)\right) \\ = p (x, y _ {\max }) \Phi \left(S _ {h} (x, y _ {\max })\right) + p (x, \mathsf {h} (x)) \Phi \left(S _ {h} (x, \mathsf {h} (x))\right) \\ + \sum_ {y \notin \{y _ {\max }, h (x) \}} p (x, y) \Phi \left(S _ {h} (x, y)\right). \\ \end{array}
$$
where we let  $S_{h}(x,y) = \frac{e^{h(x,y)}}{\sum_{y^{\prime}\in\mathcal{Y}}e^{h(x,y^{\prime})}}$  for any  $y\in \mathcal{Y}$  with the constraint that  $\sum_{y\in \mathcal{Y}}S_h(x,y) = 1$ . For any  $h\in \mathcal{H}$  such that  $\mathsf{h}(x)\neq y_{\max}$  and  $x\in \mathcal{X}$ , we can always find a family of hypotheses  $\{h_\mu \} \subset \mathcal{H}$  such that  $S_{h,\mu}(x,\cdot) = \frac{e^{h_{\mu}(x,\cdot)}}{\sum_{y^{\prime}\in\mathcal{Y}}e^{h_{\mu}(x,y^{\prime})}}$  take the following values:
$$
S _ {h, \mu} (x, y) = \left\{ \begin{array}{l l} S _ {h} (x, y) & \text {i f} y \notin \{y _ {\max }, h (x) \} \\ S _ {h} (x, y _ {\max }) + \mu & \text {i f} y = h (x) \\ S _ {h} (x, h (x)) - \mu & \text {i f} y = y _ {\max }. \end{array} \right.
$$
Note that  $S_{h,\mu}$  satisfies the constraint:
$$
\sum_ {y \in \mathcal {Y}} S _ {h, \mu} (x, y) = \sum_ {y \in \mathcal {Y}} S _ {h} (x, y) = 1.
$$
Let  $p_1 = p(x, y_{\max})$ ,  $p_2 = p(x, \mathsf{h}(x))$ ,  $\tau_1 = S_h(x, \mathsf{h}(x))$  and  $\tau_2 = S_h(x, y_{\max})$  to simplify the notation. Then, by the definition of  $S_{h,\mu}$ , we have for any  $h \in \mathcal{H}$  and  $x \in \mathcal{X}$ ,
$$
\begin{array}{l} \mathcal {C} _ {\ell^ {\operatorname {c o m p}}} (h, x) - \inf  _ {\mu \in \left[ - \tau_ {2}, \tau_ {1} \right]} \mathcal {C} _ {\ell^ {\operatorname {c o m p}}} (h _ {\mu}, x) \\ = \sup  _ {\mu \in [ - \tau_ {2}, \tau_ {1} ]} \left\{p _ {1} \left[ \Phi (\tau_ {2}) - \Phi (\tau_ {1} - \mu) \right] + p _ {2} \left[ \Phi (\tau_ {1}) - \Phi (\tau_ {2} + \mu) \right] \right\} \\ = \sup  _ {\mu \in [ - \tau_ {2}, \tau_ {1} ]} \left\{\frac {P + p _ {1} - p _ {2}}{2} \left[ \Phi \left(\tau_ {2}\right) - \Phi \left(\tau_ {1} - \mu\right) \right] + \frac {P - p _ {1} + p _ {2}}{2} \left[ \Phi \left(\tau_ {1}\right) - \Phi \left(\tau_ {2} + \mu\right) \right] \right\} \\ (P = p _ {1} + p _ {2} \in \left[ \frac {1}{n - 1} \vee p _ {1} - p _ {2}, 1 \right]) \\ \leq \inf  _ {P \in \left[ \frac {1}{n - 1} \vee p _ {1} - p _ {2}, 1 \right]} \inf  _ {\substack {\tau_ {1} \geq \max  (\tau_ {2}, 1 / n) \\ \tau_ {1} + \tau_ {2} \leq 1, \tau_ {2} \geq 0}} \sup  _ {\mu \in \left[ - \tau_ {2}, \tau_ {1} \right]} \left\{\frac {P + p _ {1} - p _ {2}}{2} \left[ \Phi \left(\tau_ {2}\right) - \Phi \left(\tau_ {1} - \mu\right) \right] \right. \\ \left. + \frac {P - p _ {1} + p _ {2}}{2} \left[ \Phi \left(\tau_ {1}\right) - \Phi \left(\tau_ {2} + \mu\right) \right] \right\} \quad (\tau_ {1} \geq \max  \left(\tau_ {2}, 1 / n\right), \tau_ {1} + \tau_ {2} \leq 1, \tau_ {2} \geq 0) \\ = \mathfrak {T} ^ {\text {c o m p}} \left(p _ {1} - p _ {2}\right) \\ = \mathfrak {T} ^ {\operatorname {c o m p}} \left(\Delta \mathcal {C} _ {\ell_ {0 - 1}, \mathcal {H}} (h, x)\right), \quad (\text {b y}) \\ \end{array}
$$
where for  $n = 2$ , an additional constraint  $\tau_{1} + \tau_{2} = 1$  is imposed and the expression of  $\mathcal{T}^{\mathrm{comp}}$  is simplified. Since  $\mathcal{T}^{\mathrm{comp}}$  is convex, by Jensen's inequality, we obtain for any hypothesis  $h \in \mathcal{H}$  and any distribution,
$$
\begin{array}{l} \mathfrak {T} ^ {\text {c o m p}} \left(\mathcal {R} _ {\ell_ {0 - 1}} (h) - \mathcal {R} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell_ {0 - 1}} (\mathcal {H})\right) \\ = \mathfrak {T} ^ {\operatorname {c o m p}} \left(\mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\ell_ {0 - 1}, \mathcal {H}} (h, x) \right]\right) \\ \leq \mathbb {E} _ {X} \left[ \mathcal {T} ^ {\operatorname {c o m p}} \left(\Delta \mathcal {C} _ {\ell_ {0 - 1}, \mathcal {H}} (h, x)\right) \right] \\ \leq \mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\ell \text {c o m p}, \mathcal {H}} (h, x) \right] \\ = \mathcal {R} _ {\ell^ {\operatorname {c o m p}}} (h) - \mathcal {R} _ {\ell^ {\operatorname {c o m p}}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell^ {\operatorname {c o m p}}} (\mathcal {H}). \\ \end{array}
$$
For the second part, we first consider  $n = 2$ . For any  $t \in [0,1]$ , we consider the distribution that concentrates on a singleton  $\{x\}$  and satisfies  $p(x,1) = \frac{1 + t}{2}$ ,  $p(x,2) = \frac{1 - t}{2}$ . For any  $\epsilon > 0$ , by the definition of infimum, we can take  $h \in \mathcal{H}$  such that  $S_{h}(x,1) = \tau_{\epsilon} \in \left[0,\frac{1}{2}\right]$  and satisfies
$$
\sup  _ {\mu \in [ - \tau_ {\epsilon}, 1 - \tau_ {\epsilon} ]} \left\{\frac {1 + t}{2} \bigl [ \Phi (\tau_ {\epsilon}) - \Phi (1 - \tau_ {\epsilon} - \mu) \bigr ] + \frac {1 - t}{2} \bigl [ \Phi (1 - \tau_ {\epsilon}) - \Phi (\tau_ {\epsilon} + \mu) \bigr ] \right\} <   \mathfrak {T} ^ {\operatorname {c o m p}} (t) + \epsilon .
$$
$$
\begin{array}{l} \mathcal {R} _ {\ell_ {0 - 1}} (h) - \mathcal {R} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell_ {0 - 1}} (\mathcal {H}) = \mathcal {R} _ {\ell_ {0 - 1}} (h) - \mathbb {E} _ {X} \left[ \mathcal {C} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}, x) \right] \\ = \mathcal {C} _ {\ell_ {0 - 1}} (h, x) - \mathcal {C} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}, x) \\ = t \\ \end{array}
$$
$$
\begin{array}{l} \mathfrak {T} ^ {\operatorname {c o m p}} (t) \leq \mathcal {R} _ {\ell^ {\operatorname {c o m p}}} (h) - \mathcal {R} _ {\ell^ {\operatorname {c o m p}}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell^ {\operatorname {c o m p}}} (\mathcal {H}) \\ = \mathcal {R} _ {\ell^ {\operatorname {c o m p}}} (h) - \mathbb {E} _ {X} \left[ \mathcal {C} _ {\ell^ {\operatorname {c o m p}}} ^ {*} (\mathcal {H}, x) \right] \\ = \mathcal {C} _ {\ell \operatorname {c o m p}} (h, x) - \mathcal {C} _ {\ell \operatorname {c o m p}} ^ {*} (\mathcal {H}, x) \\ = \sup  _ {\mu \epsilon [ - \tau_ {\epsilon}, 1 - \tau_ {\epsilon} ]} \left\{\frac {1 + t}{2} \left[ \Phi \left(\tau_ {\epsilon}\right) - \Phi (1 - \tau_ {\epsilon} - \mu) \right] + \frac {1 - t}{2} \left[ \Phi (1 - \tau_ {\epsilon}) - \Phi (\tau_ {\epsilon} + \mu) \right] \right\} \\ <   \mathfrak {T} ^ {\operatorname {c o m p}} (t) + \epsilon . \\ \end{array}
$$
By letting  $\epsilon \to 0$ , we prove the tightness for  $n = 2$ . The proof for  $n > 2$  directly extends from the case when  $n = 2$ . Indeed, for any  $t \in [0,1]$ , we consider the distribution that concentrates on a singleton  $\{x\}$  and satisfies  $p(x,1) = \frac{1 + t}{2}$ ,  $p(x,2) = \frac{1 - t}{2}$ ,  $p(x,y) = 0$ ,  $3 \leq y \leq n$ . For any  $\epsilon > 0$ , by the definition
of infimum, we can take  $h \in \mathcal{H}$  such that  $S_{h}(x,1) = \tau_{1,\epsilon}, S_{h}(x,2) = \tau_{2,\epsilon}, S_{h}(x,y) = 0, 3 \leq y \leq n$  and satisfies  $\tau_{1,\epsilon} + \tau_{2,\epsilon} = 1$ , and
$$
\begin{array}{l} \inf  _ {P \in \left[ \frac {1}{n - 1} \vee t, 1 \right]} \sup  _ {\mu \in \left[ - \tau_ {2, \epsilon}, \tau_ {1, \epsilon} \right]} \left\{\frac {P + t}{2} \left[ \Phi \left(\tau_ {2, \epsilon}\right) - \Phi \left(\tau_ {1, \epsilon} - \mu\right) \right] + \frac {P - t}{2} \left[ \Phi \left(\tau_ {1, \epsilon}\right) - \Phi \left(\tau_ {2, \epsilon} + \mu\right) \right] \right\} \\ = \sup  _ {\mu \in [ - \tau_ {2, \epsilon}, \tau_ {1, \epsilon} ]} \left\{\frac {1 + t}{2} \left[ \Phi (\tau_ {2, \epsilon}) - \Phi (\tau_ {1, \epsilon} - \mu) \right] + \frac {1 - t}{2} \left[ \Phi (\tau_ {1, \epsilon}) - \Phi (\tau_ {2, \epsilon} + \mu) \right] \right\} \\ <   \mathfrak {T} ^ {\operatorname {c o m p}} (t) + \epsilon . \\ \end{array}
$$
$$
\mathcal {R} _ {\ell_ {0 - 1}} (h) - \mathcal {R} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell_ {0 - 1}} (\mathcal {H}) = t
$$
$$
\mathfrak {T} ^ {\operatorname {c o m p}} (t) <   \mathfrak {T} ^ {\operatorname {c o m p}} (t) + \epsilon .
$$
By letting  $\epsilon \rightarrow 0$ , we prove the tightness for  $n > 2$ .
C.2 Characterization of  $\mathfrak{T}^{\mathrm{comp}}$  (Theorem 3)
$$
\mathfrak{T}^{\mathrm{comp}}(t) = \left\{ \begin{array}{ll}\Phi \Big(\frac{1}{2}\Big) - \inf_{\mu \in \bigl [-\frac{1}{2},\frac{1}{2}\bigr ]}\Big\{\frac{1 - t}{2}\Phi \Big(\frac{1}{2} +\mu \Big) + \frac{1 + t}{2}\Phi \Big(\frac{1}{2} -\mu \Big)\Big\} & n = 2\\ \inf_{\tau \in \bigl [ \frac{1}{n},\frac{1}{2}\bigr ]}\Big\{\Phi (\tau) - \inf_{\mu \in \bigl [-\tau ,\tau \bigr ]}\Big\{\frac{1 + t}{2}\Phi (\tau -\mu) + \frac{1 - t}{2}\Phi (\tau +\mu)\Big\} \Big\} & n > 2. \end{array} \right.
$$
Proof. For  $n = 2$ , we have
(  $\Phi$  is convex)
$$
\begin{array}{l} \mathfrak{T}^{\mathrm{comp}}(t) = \inf_{\tau \in \left[0,\frac{1}{2}\right]}\sup_{\mu \in [-\tau ,1 - \tau ]}\left\{\frac{1 + t}{2}\bigl[\Phi (\tau) - \Phi (1 - \tau -\mu)\bigr ] + \frac{1 - t}{2}\bigl[\Phi (1 - \tau) - \Phi (\tau +\mu)\bigr ]\right\} \\ = \inf  _ {\tau \in [ 0, \frac {1}{2} ]} \left(\frac {1 + t}{2} \Phi (\tau) + \frac {1 - t}{2} [ \Phi (1 - \tau) ] - \inf  _ {\mu \in [ - \tau , 1 - \tau ]} \left\{\frac {1 + t}{2} \Phi (1 - \tau - \mu) + \frac {1 - t}{2} \Phi (\tau + \mu) \right\}\right) \\ = \inf  _ {\tau \in [ 0, \frac {1}{2} ]} \left(\frac {1 + t}{2} \Phi (\tau) + \frac {1 - t}{2} [ \Phi (1 - \tau) ]\right) - \inf  _ {\mu \in [ - \frac {1}{2}, \frac {1}{2} ]} \left\{\frac {1 - t}{2} \Phi \left(\frac {1}{2} + \mu\right) + \frac {1 + t}{2} \Phi \left(\frac {1}{2} - \mu\right) \right\} \\ \geq \inf  _ {\tau \in [ 0, \frac {1}{2} ]} \left(\Phi \left(\frac {1}{2}\right) + \Phi^ {\prime} \left(\frac {1}{2}\right) t \left(\tau - \frac {1}{2}\right)\right) - \inf  _ {\mu \in \left[ - \frac {1}{2}, \frac {1}{2} \right]} \left\{\frac {1 - t}{2} \Phi \left(\frac {1}{2} + \mu\right) + \frac {1 + t}{2} \Phi \left(\frac {1}{2} - \mu\right) \right\} \\ = \Phi \left(\frac {1}{2}\right) - \inf  _ {\mu \in \left[ - \frac {1}{2}, \frac {1}{2} \right]} \left\{\frac {1 - t}{2} \Phi \left(\frac {1}{2} + \mu\right) + \frac {1 + t}{2} \Phi \left(\frac {1}{2} - \mu\right) \right\} \quad (\Phi^ {\prime} \left(\frac {1}{2}\right) <   0, t \left(\tau - \frac {1}{2}\right) \leq 0) \\ \end{array}
$$
where the equality can be achieved by  $\tau = \frac{1}{2}$ .
For  $n > 2$ , we have
$$
\mathfrak{T}^{\mathrm{comp}}(t) = \inf_{P\in \left[\frac{1}{n - 1},1\right]}\inf_{\substack{\tau_{1}\geq \max (\tau_{2},1 / n)\\ \tau_{1} + \tau_{2}\leq 1,\tau_{2}\geq 0}}\sup_{\mu \in \left[-\tau_{2},\tau_{1}\right]}F(P,\tau_{1},\tau_{2},\mu)
$$
where we let  $F(P, \tau_1, \tau_2, \mu) = \frac{P + t}{2} \left[ \Phi(\tau_2) - \Phi(\tau_1 - \mu) \right] + \frac{P - t}{2} \left[ \Phi(\tau_1) - \Phi(\tau_2 + \mu) \right]$ . For simplicity, we assume that  $\Phi$  is differentiable. For general convex  $\Phi$ , we can proceed by using left and right derivatives, which are non-decreasing. Differentiate  $F$  with respect to  $\mu$ , we have
$$
\frac {\partial F}{\partial \mu} = \frac {P + t}{2} \Phi^ {\prime} (\tau_ {1} - \mu) + \frac {t - P}{2} \Phi^ {\prime} (\tau_ {2} + \mu).
$$
Using the fact that  $P \in \left[\frac{1}{n - 1} \vee t, 1\right], t \in [0,1]$  and  $\Phi'$  is non-decreasing, we obtain that  $\frac{\partial F}{\partial \mu}$  is non-increasing. Furthermore,  $\Phi'$  is non-decreasing and non-positive,  $\Phi$  is non-negative, we obtain that
$\Phi^{\prime}(+\infty) = 0$ . This implies that  $\frac{\partial F}{\partial\mu} (+\infty)\leq 0$  and  $\frac{\partial F}{\partial\mu} (-\infty)\geq 0$ . Therefore, there exists  $\mu_0\in \mathbb{R}$  such that
$$
\frac {\partial F}{\partial \mu} (\mu_ {0}) = \frac {P + t}{2} \Phi^ {\prime} (\tau_ {1} - \mu_ {0}) + \frac {t - P}{2} \Phi^ {\prime} (\tau_ {2} + \mu_ {0}) = 0
$$
By taking  $\mu = \tau_{1} - \tau_{2}$  and using the fact that  $\tau_{2} \leq \frac{1}{2}$ ,  $\Phi' \left(\frac{1}{2}\right) < 0$ , we have
$$
\frac {\partial F}{\partial \mu} \left(\tau_ {1} - \tau_ {2}\right) = \frac {P + t}{2} \Phi^ {\prime} \left(\tau_ {2}\right) + \frac {t - P}{2} \Phi^ {\prime} \left(\tau_ {1}\right) <   0.
$$
Thus, since  $\frac{\partial F}{\partial \mu}$  is non-increasing, we obtain  $\mu_0 < \tau_1 - \tau_2$ . Differentiate  $F$  with respect to  $\tau_2$  at  $\mu_0$ , we have
$$
\frac {\partial F}{\partial \tau_ {2}} = \frac {P + t}{2} \Phi^ {\prime} (\tau_ {2}) + \frac {t - P}{2} \Phi^ {\prime} (\tau_ {2} + \mu_ {0}).
$$
Since  $\Phi^{\prime}$  is non-decreasing, we obtain
$$
\frac {\partial F}{\partial \tau_ {2}} \leq \frac {P + t}{2} \Phi^ {\prime} (\tau_ {2}) + \frac {t - P}{2} \Phi^ {\prime} (\tau_ {2} + \tau_ {1} - \tau_ {2}) = \frac {\partial F}{\partial \mu} (\tau_ {1} - \tau_ {2}) <   0,
$$
which implies that the infimum  $\inf_{\tau_1 \geq \max \{\tau_2, \frac{1}{n}\}}$  is achieved when  $\tau_2 = \tau_1$ . Differentiate  $F$  with respect to  $P$  at  $\mu_0$  and  $\tau_1 = \tau_2$ , by the convexity of  $\Phi$ , we obtain
$$
\frac {\partial F}{\partial P} = \Phi (\tau_ {1}) - \Phi (\tau_ {1} - \mu_ {0}) + \Phi (\tau_ {1}) - \Phi (\tau_ {1} + \mu_ {0}) \leq 0,
$$
which implies that the infimum  $\inf_{P\in \left[\frac{1}{n - 1},1\right]}$  is achieved when  $P = 1$ . Above all, we obtain
$$
\begin{array}{l} \mathfrak{T}^{\mathrm{comp}}(t) = \inf_{\tau \in \left[\frac{1}{n},\frac{1}{2}\right]}\sup_{\mu \in [-\tau ,\tau ]}F(1,\tau ,\tau ,\mu) \\ = \inf  _ {\tau \in \left[ \frac {1}{n}, \frac {1}{2} \right]} \left\{\Phi (\tau) - \inf  _ {\mu \in [ - \tau , \tau ]} \left\{\frac {1 + t}{2} \Phi (\tau - \mu) + \frac {1 - t}{2} \Phi (\tau + \mu) \right\} \right\}. \\ \end{array}
$$
C.3 Computation of examples
Example:  $\Phi(t) = -\log(t)$ . For  $n = 2$ , plugging in  $\Phi(t) = -\log(t)$  in Theorem 3, gives
$$
\begin{array}{l} \mathfrak{T}^{\mathrm{comp}} = \log 2 - \inf_{\mu \in \left[ - \frac{1}{2},\frac{1}{2}\right]}\left\{-\frac{1 - t}{2}\log \left(\frac{1}{2} +\mu\right) - \frac{1 + t}{2}\log \left(\frac{1}{2} -\mu\right)\right\} \\ = \frac {1 + t}{2} \log (1 + t) + \frac {1 - t}{2} \log (1 - t). \quad (\text {m i n i m u m a c h i e v e d a t} \mu = - \frac {t}{2}) \\ \end{array}
$$
Similarly, for  $n > 2$ , plugging in  $\Phi(t) = -\log(t)$  in Theorem 3 yields
$$
\begin{array}{l} \mathfrak{T}^{\mathrm{comp}} = \inf_{\tau \in \left[ \frac{1}{n},\frac{1}{2}\right]}\left\{-\log \tau -\inf_{\mu \in \left[ -\tau ,\tau \right]} \left\{-\frac{1 - t}{2}\log (\tau +\mu) - \frac{1 + t}{2}\log (\tau -\mu)\right\} \right\} \\ = \frac {1 + t}{2} \log (1 + t) + \frac {1 - t}{2} \log (1 - t). \quad (\text {m i n i m u m a c h i e v e d a t} \mu = - \tau t) \\ \end{array}
$$
Example:  $\Phi(t) = \frac{1}{t} - 1$ . For  $n = 2$ , plugging in  $\Phi(t) = \frac{1}{t} - 1$  in Theorem 3, gives
$$
\begin{array}{l} \mathcal {T} ^ {\mathrm {c o m p}} = 2 - \inf  _ {\mu \in \left[ - \frac {1}{2}, \frac {1}{2} \right]} \left\{\frac {1 - t}{2} \frac {1}{\frac {1}{2} + \mu} + \frac {1 + t}{2} \frac {1}{\frac {1}{2} - \mu} \right\} \\ = 1 - \sqrt {1 - t ^ {2}}. \quad (\text {m i n i m u m a c h i e v e d a t} \mu = \frac {(1 - t) ^ {\frac {1}{2}} - (1 + t) ^ {\frac {1}{2}}}{2 \left((1 + t) ^ {\frac {1}{2}} + (1 - t) ^ {\frac {1}{2}}\right)} \\ \end{array}
$$
Similarly, for  $n > 2$ , plugging in  $\Phi(t) = \frac{1}{t} - 1$  in Theorem 3 yields
$$
\begin{array}{l} \mathfrak{T}^{\text{comp}} = \inf_{\tau \in \left[\frac{1}{n},\frac{1}{2}\right]}\left\{\frac{1}{\tau} -\inf_{\mu \in [-\tau ,\tau ]}\left\{\frac{1 + t}{2}\frac{1}{\tau - \mu} +\frac{1 + t}{2}\frac{1}{\tau + \mu}\right\} \right\} \\ = \inf  _ {\tau \in \left[ \frac {1}{n}, \frac {1}{2} \right]} \frac {1}{2 \tau} \left(1 - \sqrt {1 - t ^ {2}}\right) \quad (\text {m i n i m u m a c h i e v e d a t} \mu = \frac {(1 - t) ^ {\frac {1}{2}} - (1 + t) ^ {\frac {1}{2}}}{(1 + t) ^ {\frac {1}{2}} + (1 - t) ^ {\frac {1}{2}}} \tau) \\ = 1 - \sqrt {1 - t ^ {2}}. \quad (\text {m i n i m u m a c h i e v e d a t} \tau = \frac {1}{2}) \\ \end{array}
$$
Example:  $\Phi(t) = \frac{1}{q}(1 - t^q), q \in (0,1)$ . For  $n = 2$ , plugging in  $\Phi(t) = \frac{1}{q}(1 - t^q)$  in Theorem 3, gives
(minimum achieved at  $\mu = \frac{(1 - t)^{\frac{1}{1 - q}} - (1 + t)^{\frac{1}{1 - q}}}{2\left((1 + t)^{\frac{1}{1 - q}} + (1 - t)^{\frac{1}{1 - q}}\right)}$
$$
\begin{array}{l} \mathfrak{T}^{\mathrm{comp}} = -\frac{1}{q2^{q}} -\inf_{\mu \in \left[-\frac{1}{2},\frac{1}{2}\right]}\left\{-\frac{1 - t}{2q}\bigg(\frac{1}{2} +\mu \bigg)^{q} - \frac{1 + t}{2q}\bigg(\frac{1}{2} -\mu \bigg)^{q}\right\} \\ = \frac {1}{q 2 ^ {q}} \left(\frac {(1 + t) ^ {\frac {1}{1 - q}} + (1 - t) ^ {\frac {1}{1 - q}}}{2}\right) ^ {1 - q} - \frac {1}{q 2 ^ {q}}. \\ \end{array}
$$
Similarly, for  $n > 2$ , plugging in  $\Phi(t) = \frac{1}{q} (1 - t^q)$  in Theorem 3 yields
$$
\begin{array}{l} \mathfrak{T}^{\text{comp}} = \inf_{\tau \in \left[\frac{1}{n},\frac{1}{2}\right]}\left\{-\frac{\tau^{q}}{q} -\inf_{\mu \in \left[ - \tau ,\tau \right]} \left\{-\frac{1 + t}{2q} (\tau -\mu)^{q} - \frac{1 - t}{2q} (\tau +\mu)^{q}\right\} \right\} \\ = \inf  _ {\tau \in \left[ \frac {1}{n}, \frac {1}{2} \right]} \frac {\tau^ {q}}{q} \left(\frac {(1 + t) ^ {\frac {1}{1 - q}} + (1 - t) ^ {\frac {1}{1 - q}}}{2}\right) ^ {1 - q} - \frac {\tau^ {q}}{q} \\ \end{array}
$$
(minimum achieved at  $\mu = \frac{(1 - t)^{\frac{1}{1 - q}} - (1 + t)^{\frac{1}{1 - q}}}{(1 + t)^{\frac{1}{1 - q}} + (1 - t)^{\frac{1}{1 - q}}} \tau$ )
$$
= \frac {1}{q n ^ {q}} \left(\frac {(1 + t) ^ {\frac {1}{1 - q}} + (1 - t) ^ {\frac {1}{1 - q}}}{2}\right) ^ {1 - q} - \frac {1}{q n ^ {q}}.
$$
(minimum achieved at  $\tau = \frac{1}{n}$ )
Example:  $\Phi(t) = 1 - t$ . For  $n = 2$ , plugging in  $\Phi(t) = 1 - t$  in Theorem 3, gives
$$
\mathcal {T} ^ {\mathrm {c o m p}} = \frac {1}{2} - \inf _ {\mu \in [ - \frac {1}{2}, \frac {1}{2} ]} \left\{\frac {1 - t}{2} \bigg (\frac {1}{2} - \mu \bigg) + \frac {1 + t}{2} \bigg (\frac {1}{2} + \mu \bigg) \right\} = \frac {1}{2} - \frac {1 - t}{2} = \frac {t}{2}.
$$
Similarly, for  $n > 2$ , plugging in  $\Phi(t) = 1 - t$  in Theorem 3 yields
$$
\begin{array}{l} \mathcal {T} ^ {\text {c o m p}} = \inf  _ {\tau \in \left[ \frac {1}{n}, \frac {1}{2} \right]} \left\{(1 - \tau) - \inf  _ {\mu \in [ - \tau , \tau ]} \left\{\frac {1 + t}{2} (1 - \tau + \mu) + \frac {1 - t}{2} (1 - \tau - \mu) \right\} \right\} \\ = \inf  _ {\tau \in \left[ \frac {1}{n}, \frac {1}{2} \right]} \tau t \quad (\text {m i n i m u m a c h i e v e d a t} \mu = - \tau) \\ = \frac {t}{n}. \quad (\text {m i n i m u m a c h i e v e d a t} \tau = \frac {1}{n}) \\ \end{array}
$$
Example:  $\Phi(t) = (1 - t)^2$ . For  $n = 2$ , plugging in  $\Phi(t) = (1 - t)^2$  in Theorem 3, gives
$$
\mathfrak {T} ^ {\mathrm {c o m p}} = \frac {1}{4} - \inf _ {\mu \in [ - \frac {1}{2}, \frac {1}{2} ]} \left\{\frac {1 - t}{2} \left(\frac {1}{2} - \mu\right) ^ {2} + \frac {1 + t}{2} \left(\frac {1}{2} + \mu\right) ^ {2} \right\} = \frac {1}{4} - \frac {1 - t ^ {2}}{4} = \frac {t ^ {2}}{4}.
$$
$$
\begin{array}{l} \mathfrak{T}^{\text{comp}} = \inf_{\tau \in \left[ \frac{1}{n},\frac{1}{2}\right]}\left\{(1 - \tau)^{2} - \inf_{\mu \in [-\tau ,\tau ]}\left\{\frac{1 + t}{2} (1 - \tau +\mu)^{2} + \frac{1 - t}{2} (1 - \tau -\mu)^{2}\right\} \right\} \\ = \inf  _ {\tau \in \left[ \frac {1}{n}, \frac {1}{2} \right]} \left\{\left(1 - \tau\right) ^ {2} t ^ {2} \right\} \quad (\text {m i n i m u m a c h i e v e d a t} \mu = t (\tau - 1)) \\ = \frac {t ^ {2}}{4}. \quad \left(\text {m i n i m u m a c h i e v e d a t} \tau = \frac {1}{2}\right) \\ \end{array}
$$
D Proofs for constrained losses
D.1 Proof of  $\mathcal{H}$ -consistency bounds with  $\mathcal{T}^{\mathrm{cstnd}}$  (Theorem 10)
with  $\mathcal{H}$ -estimation error transformation for constrained losses defined on  $t \in [0,1]$  by  $\mathcal{T}^{\mathrm{cstnd}}(t) =$
$$
\left\{ \begin{array}{l l} \inf  _ {\tau \geq 0} \sup  _ {\mu \in \mathbb {R}} \big \{\frac {1 - t}{2} \big [ \Phi (\tau) - \Phi (- \tau + \mu) \big ] + \frac {1 + t}{2} \big [ \Phi (- \tau) - \Phi (\tau - \mu) \big ] \big \} & n = 2 \\ \inf  _ {P \in \big [ \frac {1}{n - 1}, 1 \big ]} \inf  _ {\tau_ {1} \geq \max  \{\tau_ {2}, 0 \}} \sup  _ {\mu \in \mathbb {R}} \big \{\frac {2 - P - t}{2} \big [ \Phi (- \tau_ {2}) - \Phi (- \tau_ {1} + \mu) \big ] + \frac {2 - P + t}{2} \big [ \Phi (- \tau_ {1}) - \Phi (- \tau_ {2} - \mu) \big ] \big \} & n > 2. \end{array} \right.
$$
Proof. For the constrained loss  $\ell^{\mathrm{cstnd}}$ , the conditional  $\ell^{\mathrm{cstnd}}$ -risk can be expressed as follows:
$$
\begin{array}{l} \mathcal {C} _ {\ell^ {\mathrm {c s t n d}}} (h, x) = \sum_ {y \in \mathcal {Y}} p (x, y) \ell^ {\mathrm {c s t n d}} (h, x, y) \\ = \sum_ {y \in \mathcal {Y}} p (x, y) \sum_ {y ^ {\prime} \neq y} \Phi (- h (x, y ^ {\prime})) \\ = \sum_ {y \in \mathcal {Y}} \Phi (- h (x, y)) \sum_ {y ^ {\prime} \neq y} p (x, y ^ {\prime}) \\ = \sum_ {y \in \mathcal {Y}} \Phi (- h (x, y)) (1 - p (x, y)) \\ = \Phi (- h (x, y _ {\max }) (1 - p (x, y _ {\max })) + \Phi (- h (x, h (x))) (1 - p (x, h (x))) \\ + \sum_ {y \notin \{y _ {\max }, h (x) \}} \Phi (- h (x, y)) (1 - p (x, y)). \\ \end{array}
$$
For any  $h \in \mathcal{H}$  and  $x \in \mathcal{X}$ , by the symmetry and completeness of  $\mathcal{H}$ , we can always find a family of hypotheses  $\{h_{\mu} : \mu \in \mathbb{R}\} \subset \mathcal{H}$  such that  $h_{\mu}(x, \cdot)$  take the following values:
$$
h _ {\mu} (x, y) = \left\{ \begin{array}{l l} h (x, y) & \text {i f} y \notin \{y _ {\max }, \mathsf {h} (x) \} \\ h (x, y _ {\max }) + \mu & \text {i f} y = \mathsf {h} (x) \\ h (x, \mathsf {h} (x)) - \mu & \text {i f} y = y _ {\max }. \end{array} \right.
$$
Note that the hypotheses  $h_\mu$  satisfies the constraint:
$$
\sum_ {y \in \mathcal {Y}} h _ {\mu} (x, y) = \sum_ {y \in \mathcal {Y}} h (x, y) = 0, \forall \mu \in \mathbb {R}.
$$
Let  $p_1 = p(x, y_{\max})$ ,  $p_2 = p(x, \mathsf{h}(x))$ ,  $\tau_1 = h(x, \mathsf{h}(x))$  and  $\tau_2 = h(x, y_{\max})$  to simplify the notation. Then, by the definition of  $h_\mu$ , we have for any  $h \in \mathcal{H}$  and  $x \in \mathcal{X}$ ,