SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where we denote by  $\overline{q}(x, y') = \sum_{y \in \mathcal{Y}} p(x, y) \overline{\ell}(y', y) \in [0,1]$  and  $\mathsf{S}(x, y) = \frac{e^{h(x, y)}}{\sum_{y'' \in \mathcal{Y}} e^{h(x, y'')} \in [0,1]}$  with the constraint that  $\sum_{y \in \mathcal{Y}} \mathsf{S}(x, y) = 1$ . Let  $y_{\min} = \operatorname*{argmin}_{y' \in \mathcal{Y}} \sum_{y \in \mathcal{Y}} p(x, y) \ell(y', y)$ , where we choose the label with the highest index under the natural ordering of labels as the tie-breaking strategy. For any  $h \in \mathcal{H}$  such that  $\mathsf{h}(x) \neq y_{\min}$  and  $x \in \mathcal{X}$ , by the symmetry and completeness of  $\mathcal{H}$ , we can always find a family of hypotheses  $\left\{\overline{h}_{\mu} : \mu \in [-\mathsf{S}(x, y_{\min}), \mathsf{S}(x, \mathsf{h}(x))]\right\} \subset \mathcal{H}$  such that  $\overline{\mathsf{S}}_{\mu}(x, \cdot) = \frac{e^{h_{\mu}(x, \cdot)}}{\sum_{y' \in \mathcal{Y}} e^{h_{\mu}(x, y')} \in [0,1]}$  take the following values:
$$
\bar {\mathcal {S}} _ {\mu} (x, y) = \left\{ \begin{array}{l l} \mathcal {S} (x, y) & \text {i f} y \notin \left\{y _ {\min }, \mathrm {h} (x) \right\} \\ \mathcal {S} (x, y _ {\min }) + \mu & \text {i f} y = \mathrm {h} (x) \\ \mathcal {S} (x, \mathrm {h} (x)) - \mu & \text {i f} y = y _ {\min } \end{array} \right. \tag {20}
$$
Note that  $\overline{\mathcal{S}}_{\mu}$  satisfies the constraint:
$$
\sum_ {y \in \mathcal {Y}} \overline {{\mathcal {S}}} _ {\mu} (x, y) = \sum_ {y \in \mathcal {Y}} \mathcal {S} (x, y) = 1, \forall \mu \in [ - \mathcal {S} (x, y _ {\min}), \mathcal {S} (x, \mathsf {h} (x)) ].
$$
By (20) and using the fact that  $\mathsf{H}(x) = \mathcal{Y}$  when  $\mathcal{H}$  is symmetric, we obtain
$$
\begin{array}{l} \Delta \mathcal {C} _ {\mathsf {L} ^ {\mathrm {c o m p}}, \mathcal {H}} (h, x) = \mathcal {C} _ {\mathsf {L} ^ {\mathrm {c o m p}}} (h, x) - \mathcal {C} _ {\mathsf {L} ^ {\mathrm {c o m p}}} ^ {*} (\mathcal {H}, x) \\ \geq \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (h, x) - \inf  _ {\mu \in \mathbb {R}} \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (\overline {{h}} _ {\mu}, x) \\ = \sup  _ {\mu \in [ - \mathcal {S} (x, y _ {\min }), \mathcal {S} (x, \mathrm {h} (x)) ]} \left\{\overline {{q}} (x, y _ {\min }) \left[ - \log \left(\mathcal {S} (x, y _ {\min })\right) + \log (\mathcal {S} (x, \mathrm {h} (x)) - \mu) \right] \right. \\ \left. \right.\left. + \bar {q} (x, h (x)) \left[ - \log \left(S (x, h (x))\right) + \log \left(S (x, y _ {\min }) + \mu\right)\right]\right\}. \\ \end{array}
$$
Differentiating with respect to  $\mu$  yields the optimal value  $\mu^{*} = \frac{\overline{q}(x,\mathsf{h}(x))\mathcal{S}(x,\mathsf{h}(x)) - \overline{q}(x,y_{\min})\mathcal{S}(x,y_{\min})}{\overline{q}(x,y_{\min}) + \overline{q}(x,\mathsf{h}(x))}$ . Plugging in that value gives:
$$
\begin{array}{l} \Delta \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}} (h, x) \geq \overline {{q}} (x, y _ {\min }) \log \frac {(\mathcal {S} (x , \mathsf {h} (x)) + \mathcal {S} (x , y _ {\min }) \big) \overline {{q}} (x , y _ {\min })}{\mathcal {S} (x , y _ {\min }) (\overline {{q}} (x , y _ {\min }) + \overline {{q}} (x , \mathsf {h} (x)))} \\ + \bar {q} (x, \mathsf {h} (x)) \log \frac {(\mathsf {S} (x , \mathsf {h} (x)) + \mathsf {S} (x , y _ {\min })) \bar {q} (x , \mathsf {h} (x))}{\mathsf {S} (x , \mathsf {h} (x)) (\bar {q} (x , y _ {\min }) + \bar {q} (x , \mathsf {h} (x)))}. \\ \end{array}
$$
Differentiating with respect to  $\mathcal{S}$  shows that the minimum is attained for  $\mathcal{S}(x, \mathsf{h}(x)) = \mathcal{S}(x, y_{\min})$  which gives:
$$
\begin{array}{l} \Delta \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}} (h, x) \geq \bar {q} (x, y _ {\min }) \log \frac {2 \bar {q} (x , y _ {\min })}{\bar {q} (x , y _ {\min }) + \bar {q} (x , \mathsf {h} (x))} + \bar {q} (x, \mathsf {h} (x)) \log \frac {2 \bar {q} (x , \mathsf {h} (x))}{\bar {q} (x , y _ {\min }) + \bar {q} (x , \mathsf {h} (x))} \\ \geq \frac {\left(\bar {q} (x , \mathsf {h} (x)) - \bar {q} (x , y _ {\min })\right) ^ {2}}{2 \left(\bar {q} (x , \mathsf {h} (x)) + \bar {q} (x , y _ {\min })\right)} \\ (a \log \frac {2 a}{a + b} + b \log \frac {2 b}{a + b} \geq \frac {(a - b) ^ {2}}{2 (a + b)}, \forall a, b \in [ 0, 1 ] [ \text {M o h r i e t a l . , 2 0 1 8 , P r o p o s i t i o n E . 7} ]) \\ \geq \frac {\left(\bar {q} (x , \mathsf {h} (x)) - \bar {q} (x , y _ {\min })\right) ^ {2}}{4} \quad (0 \leq \bar {q} (x, \mathsf {h} (x)) + \bar {q} (x, y _ {\min }) \leq 2) \\ = \frac {\left(\sum_ {y \in \mathcal {Y}} p (x , y) \ell (\mathsf {h} (x) , y) - \sum_ {y \in \mathcal {Y}} p (x , y) \ell \left(y _ {\min } , y\right)\right) ^ {2}}{4} \\ = \frac {1}{4} \Delta \mathcal {C} _ {\mathrm {L}, \mathcal {H}} (h, x) ^ {2}. \quad (\text {b y}) \\ \end{array}
$$
Since the function  $t \mapsto \frac{t^2}{4}$  is convex, by Jensen's inequality, we obtain for any hypothesis  $h \in \mathcal{H}$  and any distribution,
$$
\begin{array}{l} \frac {\left(\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L} , \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} , \mathcal {H}}\right) ^ {2}}{4} = \frac {\left(\mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\mathrm {L} , \mathcal {H}} (h , x) \right]\right) ^ {2}}{4} \\ \leq \mathbb {E} _ {X} \left[ \frac {\Delta \mathcal {C} _ {\mathrm {L} , \mathcal {H}} (h , x) ^ {2}}{4} \right] \\ \leq \mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\mathrm {L} ^ {\text {c o m p}}, \mathcal {H}} (h, x) \right] \\ = \mathcal {R} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}}, \\ \end{array}
$$
$$
\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L}, \mathcal {H}} ^ {*} \leq 2 \Big (\mathcal {R} _ {\mathrm {L} _ {\log} ^ {\mathrm {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} _ {\log} ^ {\mathrm {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} _ {\log} ^ {\mathrm {c o m p}}, \mathcal {H}} \Big) ^ {\frac {1}{2}} - \mathcal {M} _ {\mathrm {L}, \mathcal {H}}.
$$
E.2 Structured sum-exponential loss
Theorem 12 ( $\mathcal{H}$ -consistency bound of  $\mathsf{L}_{\mathrm{exp}}^{\mathrm{comp}}$ ). Assume that  $\mathcal{H}$  is symmetric and complete. Then, for any target loss  $\ell$ , hypothesis  $h \in \mathcal{H}$  and any distribution,
$$
\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L}, \mathcal {H}} ^ {*} \leq 2 \left(\mathcal {R} _ {\mathrm {L} _ {\exp} ^ {\text {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} _ {\exp} ^ {\text {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} _ {\exp} ^ {\text {c o m p}}, \mathcal {H}}\right) ^ {\frac {1}{2}} - \mathcal {M} _ {\mathrm {L}, \mathcal {H}}. \tag {21}
$$
Proof. For the comp-sum structured loss  $\mathsf{L}_{\mathrm{exp}}^{\mathrm{comp}}$ , the conditional  $\mathsf{L}_{\mathrm{exp}}^{\mathrm{comp}}$ -risk can be expressed as follows:
$$
\mathcal {C} _ {\mathrm {L} _ {\exp} ^ {\mathrm {c o m p}}} (h, x) = \sum_ {y \in \mathcal {Y}} p (x, y) \sum_ {y ^ {\prime} \in \mathcal {Y}} \overline {{\ell}} (y ^ {\prime}, y) \sum_ {y ^ {\prime \prime} \neq y ^ {\prime}} e ^ {h (x, y ^ {\prime \prime}) - h (x, y ^ {\prime})} = \sum_ {y ^ {\prime} \in \mathcal {Y}} \left(\frac {1}{\mathcal {S} (x , y ^ {\prime})} - 1\right) \overline {{q}} (x, y ^ {\prime}),
$$
where we denote by  $\overline{q}(x, y') = \sum_{y \in \mathcal{Y}} p(x, y) \overline{\ell}(y', y) \in [0,1]$  and  $\mathcal{S}(x, y) = \frac{e^{h(x, y)}}{\sum_{y'' \in \mathcal{Y}} e^{h(x, y'')} \in [0,1]}$  with the constraint that  $\sum_{y \in \mathcal{Y}} \mathcal{S}(x, y) = 1$ . Let  $y_{\min} = \operatorname*{argmin}_{y' \in \mathcal{Y}} \sum_{y \in \mathcal{Y}} p(x, y) \ell(y', y)$ , where we choose the label with the highest index under the natural ordering of labels as the tie-breaking strategy. For any  $h \in \mathcal{H}$  such that  $\mathsf{h}(x) \neq y_{\min}$  and  $x \in \mathcal{X}$ , by the symmetry and completeness of  $\mathcal{H}$ , we can always find a family of hypotheses  $\left\{\overline{h}_{\mu} : \mu \in [-\mathcal{S}(x, y_{\min}), \mathcal{S}(x, h(x))]\right\} \subset \mathcal{H}$  such that  $\overline{\mathcal{S}}_{\mu}(x, \cdot) = \frac{e^{h_{\mu}(x, \cdot)}}{\sum_{y' \in \mathcal{Y}} e^{h_{\mu}(x, y')} \in [0,1]}$  take the following values:
$$
\bar {\mathcal {S}} _ {\mu} (x, y) = \left\{ \begin{array}{l l} \mathbb {S} (x, y) & \text {i f} y \notin \left\{y _ {\min }, h (x) \right\} \\ \mathbb {S} (x, y _ {\min }) + \mu & \text {i f} y = h (x) \\ \mathbb {S} (x, h (x)) - \mu & \text {i f} y = y _ {\min } \end{array} \right. \tag {22}
$$
$$
\sum_ {y \in \mathcal {Y}} \overline {{\mathcal {S}}} _ {\mu} (x, y) = \sum_ {y \in \mathcal {Y}} \mathcal {S} (x, y) = 1, \forall \mu \in [ - \mathcal {S} (x, y _ {\mathrm {m i n}}), \mathcal {S} (x, \mathsf {h} (x)) ].
$$
By (22) and using the fact that  $\mathsf{H}(x) = \mathcal{Y}$  when  $\mathcal{H}$  is symmetric, we obtain
$$
\begin{array}{l} \Delta \mathcal {C} _ {\mathrm {L} ^ {\mathrm {c o m p}}, \mathcal {H}} (h, x) \\ = \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (h, x) - \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} ^ {*} (\mathcal {H}, x) \\ \geq \mathcal{C}_{\mathsf{L}^{\mathrm{comp}}}(h,x) - \inf_{\mu \in \mathbb{R}}\mathcal{C}_{\mathsf{L}^{\mathrm{comp}}}(\overline{h}_{\mu},x) \\ = \sup  _ {\mu \in [ - \mathcal {S} (x, y _ {\min }), \mathcal {S} (x, \mathrm {h} (x)) ]} \left\{\bar {q} (x, y _ {\min }) \left[ \frac {1}{\mathcal {S} (x , y _ {\min })} - \frac {1}{\mathcal {S} (x , \mathrm {h} (x)) - \mu} \right] \right. \\ \left. + \bar {q} (x, h (x)) \left[ \frac {1}{S (x , h (x))} - \frac {1}{S (x , y _ {\min }) + \mu} \right] \right\} \\ = \frac {\bar {q} (x , y _ {\operatorname* {m i n}})}{\mathcal {S} (x , y _ {\operatorname* {m i n}})} + \frac {\bar {q} (x , \mathsf {h} (x))}{\mathcal {S} (x , \mathsf {h} (x))} - \frac {\left(\sqrt {\bar {q} (x , y _ {\operatorname* {m i n}})} + \sqrt {\bar {q} (x , \mathsf {h} (x))}\right) ^ {2}}{\mathcal {S} (x , y _ {\operatorname* {m i n}}) + \mathcal {S} (x , \mathsf {h} (x))} \\ \end{array}
$$
(differentiating with respect to  $\mu$  to optimize, optimal  $\mu^{*} = \frac{\sqrt{q}(x,\mathsf{h}(x))\mathcal{S}(x,\mathsf{h}(x)) - \sqrt{q}(x,y_{\min})\mathcal{S}(x,y_{\min})}{\sqrt{q}(x,y_{\min}) + \sqrt{q}(x,\mathsf{h}(x))}$
$$
\geq \left(\sqrt {\bar {q} (x , y _ {\operatorname* {m i n}})} - \sqrt {\bar {q} (x , \mathsf {h} (x))}\right) ^ {2}
$$
(differentiating with respect to  $\mathcal{S}$  to minimize, minimum is attained when  $\mathcal{S}(x,\mathsf{h}(x)) = \mathcal{S}(x,y_{\min}) = \frac{1}{2})$
$$
\begin{array}{l} \geq \frac {\left(\bar {q} (x , h (x)) - \bar {q} (x , y _ {\min })\right) ^ {2}}{\left(\sqrt {\bar {q} (x , h (x))} + \sqrt {\bar {q} (x , y _ {\min })}\right) ^ {2}} \\ \geq \frac {\left(\bar {q} (x , h (x)) - \bar {q} (x , y _ {\min })\right) ^ {2}}{4} \quad (\sqrt {a} + \sqrt {b} \leq 2, \forall a, b \in [ 0, 1 ], a + b \leq 2) \\ = \frac {\left(\sum_ {y \in \mathcal {Y}} p (x , y) \ell (\mathsf {h} (x) , y) - \sum_ {y \in \mathcal {Y}} p (x , y) \ell (y _ {\min} , y)\right) ^ {2}}{4} \\ = \frac {1}{4} \Delta \mathcal {C} _ {\mathrm {L}, \mathcal {H}} (h, x) ^ {2}. \quad (\text {b y}) \\ \end{array}
$$
$$
\begin{array}{l} \frac {\left(\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L} , \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} , \mathcal {H}}\right) ^ {2}}{4} = \frac {\left(\mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\mathrm {L} , \mathcal {H}} (h , x) \right]\right) ^ {2}}{4} \\ \leq \mathbb {E} _ {X} \left[ \frac {\Delta \mathcal {C} _ {\mathrm {L} , \mathscr {H}} (h , x) ^ {2}}{4} \right] \\ \leq \mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\mathrm {L} ^ {\text {c o m p}}, \mathcal {H}} (h, x) \right] \\ = \mathcal {R} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}}, \\ \end{array}
$$
$$
\mathcal {R} _ {\mathsf {L}} (h) - \mathcal {R} _ {\mathsf {L}, \mathcal {H}} ^ {*} \leq 2 \Big (\mathcal {R} _ {\mathsf {L} _ {\exp} ^ {\mathrm {c o m p}}} (h) - \mathcal {R} _ {\mathsf {L} _ {\exp} ^ {\mathrm {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathsf {L} _ {\exp} ^ {\mathrm {c o m p}}, \mathcal {H}} \Big) ^ {\frac {1}{2}} - \mathcal {M} _ {\mathsf {L}, \mathcal {H}}.
$$
E.3 Structured generalized cross-entropy loss
Theorem 13 ( $\mathcal{H}$ -consistency bound of  $\mathsf{L}_{\mathrm{gce}}^{\mathrm{comp}}$ ). Assume that  $\mathcal{H}$  is symmetric and complete. Then, for any target loss  $\ell$ , hypothesis  $h \in \mathcal{H}$  and any distribution,
$$
\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L}, \mathcal {H}} ^ {*} \leq 2 n ^ {\frac {\alpha}{2}} \left(\mathcal {R} _ {\mathrm {L} _ {\text {g c e}} ^ {\text {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} _ {\text {g c e}} ^ {\text {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} _ {\text {g c e}} ^ {\text {c o m p}}, \mathcal {H}}\right) ^ {\frac {1}{2}} - \mathcal {M} _ {\mathrm {L}, \mathcal {H}}. \tag {23}
$$
Proof. For the comp-sum structured loss  $\mathsf{L}_{\mathrm{gce}}^{\mathrm{comp}}$ , the conditional  $\mathsf{L}_{\mathrm{gce}}^{\mathrm{comp}}$ -risk can be expressed as follows:
$$
\begin{array}{l} \mathcal {C} _ {\mathrm {L} _ {\mathrm {g c e}} ^ {\mathrm {c o m p}}} (h, x) = \sum_ {y \in \mathcal {Y}} p (x, y) \sum_ {y ^ {\prime} \in \mathcal {Y}} \bar {\ell} \left(y ^ {\prime}, y\right) \frac {1}{\alpha} \left(1 - \left(\frac {e ^ {h \left(x , y ^ {\prime}\right)}}{\sum_ {y ^ {\prime \prime} \in \mathcal {Y}} e ^ {h \left(x , y ^ {\prime \prime}\right)}}\right) ^ {\alpha}\right) \\ = \frac {1}{\alpha} \sum_ {y ^ {\prime} \in \mathcal {Y}} \left(1 - \left(\frac {e ^ {h (x , y ^ {\prime})}}{\sum_ {y ^ {\prime \prime} \in \mathcal {Y}} e ^ {h (x , y ^ {\prime \prime})}}\right) ^ {\alpha}\right) \sum_ {y \in \mathcal {Y}} p (x, y) \bar {\ell} (y ^ {\prime}, y) \\ = \frac {1}{\alpha} \sum_ {y ^ {\prime} \in \mathcal {Y}} (1 - \mathsf {S} (x, y ^ {\prime}) ^ {\alpha}) \bar {q} (x, y ^ {\prime}), \\ \end{array}
$$
where we denote by  $\overline{q}(x, y') = \sum_{y \in \mathcal{Y}} p(x, y) \overline{\ell}(y', y) \in [0,1]$  and  $\mathcal{S}(x, y) = \frac{e^{h(x, y)}}{\sum_{y'' \in \mathcal{Y}} e^{h(x, y'')} \in [0,1]}$  with the constraint that  $\sum_{y \in \mathcal{Y}} \mathcal{S}(x, y) = 1$ . Let  $y_{\min} = \operatorname*{argmin}_{y' \in \mathcal{Y}} \sum_{y \in \mathcal{Y}} p(x, y) \ell(y', y)$ , where we choose the label with the highest index under the natural ordering of labels as the tie-breaking strategy. For any  $h \in \mathcal{H}$  such that  $\mathsf{h}(x) \neq y_{\min}$  and  $x \in \mathcal{X}$ , by the symmetry and completeness of  $\mathcal{H}$ , we can always find a family of hypotheses  $\left\{\overline{h}_\mu : \mu \in [-\mathcal{S}(x, y_{\min}), \mathcal{S}(x, \mathsf{h}(x))]\right\} \subset \mathcal{H}$  such that  $\overline{\mathcal{S}}_\mu(x, \cdot) = \frac{e^{h_\mu(x, \cdot)}}{\sum_{y' \in \mathcal{Y}} e^{h_\mu(x, y')} }$  take the following values:
$$
\bar {\mathcal {S}} _ {\mu} (x, y) = \left\{ \begin{array}{l l} \mathbb {S} (x, y) & \text {i f} y \notin \left\{y _ {\min }, h (x) \right\} \\ \mathbb {S} (x, y _ {\min }) + \mu & \text {i f} y = h (x) \\ \mathbb {S} (x, h (x)) - \mu & \text {i f} y = y _ {\min }. \end{array} \right. \tag {24}
$$
By (24) and using the fact that  $\mathsf{H}(x) = \mathcal{Y}$  when  $\mathcal{H}$  is symmetric, we obtain
$$
\begin{array}{l} \Delta \mathcal {C} _ {\mathrm {L} ^ {\text {c o m p}}, \mathcal {H}} (h, x) \\ = \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (h, x) - \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} ^ {*} (\mathcal {H}, x) \\ \geq \mathcal{C}_{\mathsf{L}^{\mathrm{comp}}}(h,x) - \inf_{\mu \in \mathbb{R}}\mathcal{C}_{\mathsf{L}^{\mathrm{comp}}}\left(\overline{h}_{\mu},x\right) \\ = \frac {1}{\alpha} \sup  _ {\mu \in [ - \mathcal {S} (x, y _ {\min }), \mathcal {S} (x, \mathrm {h} (x)) ]} \left\{\bar {q} (x, y _ {\min }) \left[ - \mathcal {S} (x, y _ {\min }) ^ {\alpha} + (\mathcal {S} (x, \mathrm {h} (x)) - \mu) ^ {\alpha} \right] \right. \\ \left. + \bar {q} (x, \mathsf {h} (x)) \left[ - \mathcal {S} (x, \mathsf {h} (x)) ^ {\alpha} + (\mathcal {S} (x, y _ {\min }) + \mu) ^ {\alpha} \right] \right\} \\ = \frac {1}{\alpha} \left(\mathcal {S} (x, h (x)) + \mathcal {S} (x, y _ {\min })\right) ^ {\alpha} \left(\bar {q} (x, y _ {\min }) ^ {\frac {1}{1 - \alpha}} + \bar {q} (x, h (x)) ^ {\frac {1}{1 - \alpha}}\right) ^ {1 - \alpha} \\ - \frac {1}{\alpha} \bar {q} (x, y _ {\min }) \mathcal {S} (x, y _ {\min }) ^ {\alpha} - \frac {1}{\alpha} \bar {q} (x, \mathsf {h} (x)) \mathcal {S} (x, \mathsf {h} (x)) ^ {\alpha} \\ \end{array}
$$
(differentiating with respect to  $\mu$  to optimize, optimum  $\mu^{*} = \frac{\overline{q}(x,h(x))^{\frac{1}{1 - \alpha}}\mathcal{S}(x,h(x)) - \overline{q}(x,y_{\min})^{\frac{1}{1 - \alpha}}\mathcal{S}(x,y_{\min})}{\overline{q}(x,y_{\min})^{\frac{1}{1 - \alpha}} + \overline{q}(x,h(x))^{\frac{1}{1 - \alpha}}}$
$$
\geq \frac {1}{\alpha n ^ {\alpha}} \left[ 2 ^ {\alpha} \left(\bar {q} (x, y _ {\min }) ^ {\frac {1}{1 - \alpha}} + \bar {q} (x, h (x)) ^ {\frac {1}{1 - \alpha}}\right) ^ {1 - \alpha} - \bar {q} (x, y _ {\min }) - \bar {q} (x, h (x)) \right]
$$
(differentiating with respect to  $\mathcal{S}$  to minimize, minimum is attained when  $\mathcal{S}(x,\mathsf{h}(x)) = \mathcal{S}(x,y_{\min}) = \frac{1}{n})$
$$
\begin{array}{l} \geq \frac {\left(\overline {{q}} (x , \mathsf {h} (x)) - \overline {{q}} (x , y _ {\min })\right) ^ {2}}{4 n ^ {\alpha}} \quad \left(\frac {a ^ {\frac {1}{1 - \alpha}} + b ^ {\frac {1}{1 - \alpha}}}{2}\right) ^ {1 - \alpha} - \frac {a + b}{2} \geq \frac {\alpha}{4} (a - b) ^ {2}, \forall a, b \in [ 0, 1 ], 0 \leq a + b \leq 1) \\ = \frac {\left(\sum_ {y \in \mathcal {Y}} p (x , y) \ell (\mathsf {h} (x) , y) - \sum_ {y \in \mathcal {Y}} p (x , y) \ell \left(y _ {\min } , y\right)\right) ^ {2}}{4 n ^ {\alpha}} \\ = \frac {\Delta \mathcal {C} _ {\mathrm {L} , \mathcal {H}} (h , x) ^ {2}}{4 n ^ {\alpha}}. \quad (\text {b y}) \\ \end{array}
$$
Since the function  $t \mapsto \frac{t^2}{4n^\alpha}$  is convex, by Jensen's inequality, we obtain for any hypothesis  $h \in \mathcal{H}$  and any distribution,
$$
\begin{array}{l} \frac {\left(\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L} , \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} , \mathcal {H}}\right) ^ {2}}{4 n ^ {\alpha}} = \frac {\left(\mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\mathrm {L} , \mathcal {H}} (h , x) \right]\right) ^ {2}}{4 n ^ {\alpha}} \\ \leq \mathbb {E} _ {X} \left[ \frac {\Delta \mathcal {C} _ {\mathsf {L} , \mathcal {H}} (h , x) ^ {2}}{4 n ^ {\alpha}} \right] \\ \leq \mathbb {E} _ {X} \left[ \Delta \mathcal {C} _ {\mathrm {L} ^ {\text {c o m p}}, \mathcal {H}} (h, x) \right] \\ = \mathcal {R} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} ^ {\operatorname {c o m p}}, \mathcal {H}}, \\ \end{array}
$$
$$
\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L}, \mathcal {H}} ^ {*} \leq 2 n ^ {\frac {\alpha}{2}} \left(\mathcal {R} _ {\mathrm {L} _ {\mathrm {g c e}} ^ {\mathrm {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} _ {\mathrm {g c e}} ^ {\mathrm {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} _ {\mathrm {g c e}} ^ {\mathrm {c o m p}}, \mathcal {H}}\right) ^ {\frac {1}{2}} - \mathcal {M} _ {\mathrm {L}, \mathcal {H}}.
$$
E.4 Structured mean absolute error loss
Theorem 14 (H-consistency bound of  $\mathsf{L}_{\mathrm{mae}}^{\mathrm{comp}}$ ). Assume that  $\mathcal{H}$  is symmetric and complete. Then, for any target loss  $\ell$ , hypothesis  $h \in \mathcal{H}$  and any distribution,
$$
\mathcal {R} _ {\mathrm {L}} (h) - \mathcal {R} _ {\mathrm {L}, \mathcal {H}} ^ {*} \leq n \left(\mathcal {R} _ {\mathrm {L} _ {\text {m a e}} ^ {\text {c o m p}}} (h) - \mathcal {R} _ {\mathrm {L} _ {\text {m a e}} ^ {\text {c o m p}}, \mathcal {H}} ^ {*} + \mathcal {M} _ {\mathrm {L} _ {\text {m a e}} ^ {\text {c o m p}}, \mathcal {H}}\right) - \mathcal {M} _ {\mathrm {L}, \mathcal {H}}. \tag {25}
$$
Proof. For the comp-sum structured loss  $\mathsf{L}_{\mathrm{mae}}^{\mathrm{comp}}$ , the conditional  $\mathsf{L}_{\mathrm{mae}}^{\mathrm{comp}}$ -risk can be expressed as follows:
$$
\begin{array}{l} \mathcal {C} _ {\mathrm {L} _ {\text {m a e}} ^ {\text {c o m p}}} (h, x) = \sum_ {y \in \mathcal {Y}} p (x, y) \sum_ {y ^ {\prime} \in \mathcal {Y}} \bar {\ell} \left(y ^ {\prime}, y\right) \left(1 - \frac {e ^ {h \left(x , y ^ {\prime}\right)}}{\sum_ {y ^ {\prime \prime} \in \mathcal {Y}} e ^ {h \left(x , y ^ {\prime \prime}\right)}}\right) \\ = \sum_ {y ^ {\prime} \in \mathcal {Y}} \left(1 - \frac {e ^ {h (x , y ^ {\prime})}}{\sum_ {y ^ {\prime \prime} \in \mathcal {Y}} e ^ {h (x , y ^ {\prime \prime})}}\right) \sum_ {y \in \mathcal {Y}} p (x, y) \bar {\ell} \left(y ^ {\prime}, y\right) \\ = \sum_ {y ^ {\prime} \in \mathcal {Y}} (1 - \mathcal {S} (x, y ^ {\prime})) \overline {{q}} (x, y ^ {\prime}), \\ \end{array}
$$
where we denote by  $\overline{q}(x,y') = \sum_{y\in \mathcal{Y}}p(x,y)\overline{\ell}(y',y) \in [0,1]$  and  $\mathcal{S}(x,y) = \frac{e^{h(x,y)}}{\sum_{y''\in\mathcal{Y}}e^{h(x,y'')} \in [0,1]}$  with the constraint that  $\sum_{y\in \mathcal{Y}}\mathcal{S}(x,y) = 1$ . Let  $y_{\min} = \operatorname*{argmin}_{y'\in \mathcal{Y}}\sum_{y\in \mathcal{Y}}p(x,y)\ell(y',y)$ , where we choose the label with the highest index under the natural ordering of labels as the tie-breaking strategy. For any  $h \in \mathcal{H}$  such that  $\mathsf{h}(x) \neq y_{\min}$  and  $x \in \mathcal{X}$ , by the symmetry and completeness of  $\mathcal{H}$ , we can always find a family of hypotheses  $\left\{\overline{h}_{\mu} : \mu \in [-\mathcal{S}(x,y_{\min}),\mathcal{S}(x,\mathsf{h}(x))]\right\} \subset \mathcal{H}$  such that  $\overline{\mathcal{S}}_{\mu}(x,\cdot) = \frac{e^{h_{\mu}(x,\cdot)}}{\sum_{y'\in\mathcal{Y}}e^{h_{\mu}(x,y')}}$  take the following values:
$$
\bar {S} _ {\mu} (x, y) = \left\{ \begin{array}{l l} S (x, y) & \text {i f} y \notin \left\{y _ {\min }, h (x) \right\} \\ S (x, y _ {\min }) + \mu & \text {i f} y = h (x) \\ S (x, h (x)) - \mu & \text {i f} y = y _ {\min } \end{array} \right. \tag {26}
$$
By (26) and using the fact that  $\mathsf{H}(x) = \mathcal{Y}$  when  $\mathcal{H}$  is symmetric, we obtain
$$
\Delta \mathcal {C} _ {\mathrm {L} ^ {\text {c o m p}}, \mathcal {H}} (h, x)
$$
$$
= \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} (h, x) - \mathcal {C} _ {\mathrm {L} ^ {\operatorname {c o m p}}} ^ {*} (\mathcal {H}, x)
$$
$$
\geq \mathcal{C}_{\mathsf{L}^{\mathrm{comp}}}(h,x) - \inf_{\mu \in \mathbb{R}}\mathcal{C}_{\mathsf{L}^{\mathrm{comp}}}(\overline{h}_{\mu},x)
$$
$$
= \sup  _ {\mu \in [ - \mathcal {S} (x, y _ {\min }), \mathcal {S} (x, \mathrm {h} (x)) ]} \left\{\bar {q} (x, y _ {\min }) [ - \mathcal {S} (x, y _ {\min }) + \mathcal {S} (x, \mathrm {h} (x)) - \mu ] \right.
$$
$$
\left. + \bar {q} (x, \mathsf {h} (x)) [ - \mathcal {S} (x, \mathsf {h} (x)) + \mathcal {S} (x, y _ {\min }) + \mu ] \right\}
$$
$$
= \bar {q} (x, y _ {\min }) \mathcal {S} (x, h (x)) - \bar {q} (x, h (x)) \mathcal {S} (x, h (x))
$$
(differentiating with respect to  $\mu$  to optimize, optimum  $\mu^{*} = -\mathcal{S}(x,y_{\mathrm{min}})$ )
$$
\geq \frac {1}{n} (\bar {q} (x, y _ {\min }) - \bar {q} (x, h (x)))
$$
(differentiating with respect to  $S$  to minimize, minimum is attained when  $S(x, h(x)) = \frac{1}{n}$ )
$$
\begin{array}{l} = \frac {\sum_ {y \in \mathcal {Y}} p (x , y) \ell (\mathrm {h} (x) , y) - \sum_ {y \in \mathcal {Y}} p (x , y) \ell (y _ {\min } , y)}{n} \\ = \frac {\Delta \mathcal {C} _ {\mathrm {L} , \mathcal {H}} (h , x)}{n} \quad (\text {b y}) \\ \end{array}
$$
Therefore, we obtain for any hypothesis  $h\in \mathcal{H}$  and any distribution,