SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$\mathcal{H}$ -Consistency Bounds: Characterization and Extensions
Anqi Mao  
Courant Institute  
New York, NY 10012  
aqmao@cims.nyu.edu
Mehryar Mohri  
Google Research & CIMS  
New York, NY 10011  
mohri@google.com
Yutao Zhong  
Courant Institute  
New York, NY 10012  
yutao@cims.nyu.edu
A series of recent publications by Awasthi, Mao, Mohri, and Zhong [2022b] have introduced the key notion of  $\mathcal{H}$ -consistency bounds for surrogate loss functions. These are upper bounds on the zero-one estimation error of any predictor in a hypothesis set, expressed in terms of its surrogate loss estimation error. They are both non-asymptotic and hypothesis set-specific and thus stronger and more informative than Bayes-consistency. However, determining if they hold and deriving these bounds have required a specific proof and analysis for each surrogate loss. Can we derive more general tools and characterizations? This paper provides both a general characterization and an extension of  $\mathcal{H}$ -consistency bounds for multi-class classification. We present new and tight  $\mathcal{H}$ -consistency bounds for both the family of constrained losses and that of comp-sum losses, which covers the familiar cross-entropy, or logistic loss applied to the outputs of a neural network. We further extend our analysis beyond the completeness assumptions adopted in previous studies and cover more realistic bounded hypothesis sets. Our characterizations are based on error transformations, which are explicitly defined for each formulation. We illustrate the application of our general results through several special examples. A by-product of our analysis is the observation that a recently derived multi-class  $\mathcal{H}$ -consistency bound for cross-entropy reduces to an excess bound and is not significant. Instead, we prove a much stronger and more significant guarantee.
Bayes-consistency is an important property of surrogate loss functions. It requires that minimizing the surrogate excess error over the family of all measurable functions leads to the minimization of the target error loss in the limit [Steinwart, 2007]. This property applies to a broad family of convex margin-based losses in binary classification [Zhang, 2004a, Bartlett et al., 2006], as well as some extensions in multi-class classification [Tewari and Bartlett, 2007]. However, Bayes-consistency does not apply to the hypothesis sets commonly used for learning, such as the family of linear models or that of neural networks, which of course do not include all measurable functions. Furthermore, it is also only an asymptotic property and does not supply any convergence guarantee.
To address these limitations, a series of recent publications by Awasthi, Mao, Mohri, and Zhong [2022b] introduced the key notion of  $\mathcal{H}$ -consistency bounds for surrogate loss functions. These are upper bounds on the zero-one estimation error of any predictor in a hypothesis set, expressed in terms of its surrogate loss estimation error. They are both non-asymptotic and hypothesis set-specific and thus stronger and more informative than Bayes-consistency. However, determining the validity of these bounds and deriving them have required a specific proof and analysis for each surrogate loss. Can we derive more general tools and characterizations for  $\mathcal{H}$ -consistency bounds?
This paper provides both a general characterization and an extension of  $\mathcal{H}$ -consistency bounds for multi-class classification. Previous approaches to deriving these bounds required the development of new proofs for each specific case. In contrast, we introduce the general concept of an error transformation function that serves as a very general tool for deriving such guarantees with tightness guarantees. We show that deriving an  $\mathcal{H}$ -consistency bound for comp-sum losses and constrained losses for both complete and bounded hypothesis sets can be reduced to the calculation of their corresponding error transformation function. Our general tools and tight bounds show several remarkable advantages: first, they improve existing bounds for complete hypothesis sets previously proven in [Awasthi et al., 2022b]; second, they encompass all previously comp-sum and constrained losses studied thus far as well as many new ones [Awasthi et al., 2022a, Mao et al., 2023h]; third, they extend beyond the completeness assumption adopted in previous work; fourth, they provide novel guarantees for bounded hypothesis sets; and, finally, they help prove a much stronger and more significant guarantee for logistic loss with linear hypothesis set than [Zheng et al., 2023].
Previous work. Here, we briefly discuss recent studies of  $\mathcal{H}$ -consistency bounds by Awasthi et al. [2022a,b], Mao et al. [2023h] and Zheng et al. [2023]. Awasthi et al. [2022a] introduced and studied  $\mathcal{H}$ -consistency bounds in binary classification. They provided a series of tight  $\mathcal{H}$ -consistency bounds for bounded hypothesis set of linear models and one-hidden-layer neural networks. The subsequent study [Awasthi et al., 2022b] further generalized the framework to multi-class classification and presented an extensive study of  $\mathcal{H}$ -consistency bounds for diverse multi-class surrogate losses, including negative results for max losses [Crammer and Singer, 2001] and positive results for sum losses [Weston and Watkins, 1998], and constrained losses [Lee et al., 2004]. However, the hypothesis sets examined in their analysis were assumed to be complete, which rules out the bounded hypothesis sets typically used in practice. Moreover, the final bounds derived from [Awasthi et al., 2022b] are based on ad hoc methods and may not be tight. [Mao et al., 2023h] complemented this previous work by studying a wide family of comp-sum losses in the multi-class classification, which generalizes the sum-losses and includes as special cases the logistic loss [Verhulst, 1838, 1845, Berkson, 1944, 1951], the generalized cross-entropy loss [Zhang and Sabuncu, 2018], and the mean absolute error loss [Ghosh et al., 2017]. Here too, the completeness assumption on the hypothesis sets was adopted and their  $\mathcal{H}$ -consistency bounds do not apply to common bounded hypothesis sets in practice. Recently, Zheng et al. [2023] proved  $\mathcal{H}$ -consistency bounds for multi-class logistic loss with bounded linear hypothesis sets. However, their bounds require a crucial distributional assumption, under which the minimizability gaps coincide with the approximation errors. Thus, their bounds can be recovered as excess error bounds, which are less significant.
Other related work on  $\mathcal{H}$ -consistency bounds includes  $\mathcal{H}$ -consistency bounds for pairwise ranking [Mao, Mohri, and Zhong, 2023d,e]; theoretically grounded surrogate losses and algorithms for learning with abstention supported by  $\mathcal{H}$ -consistency bounds, including the study of score-based abstention [Mao, Mohri, and Zhong, 2023c] and learning to abstain with a fixed predictor with application in decontextualization [Mohri, Andor, Choi, Collins, Mao, and Zhong, 2023]; principled approaches for learning to defer with multiple experts that benefit from strong  $\mathcal{H}$ -consistency bounds, including the single-stage scenario [Mao, Mohri, and Zhong, 2023b] and a two-stage scenario [Mao, Mohri, Mohri, and Zhong, 2023a];  $\mathcal{H}$ -consistency theory and algorithms for adversarial robustness [Awasthi et al., 2021a,b, 2023a, Mao et al., 2023h, Awasthi et al., 2023b]; and efficient algorithms and loss functions for structured prediction with stronger  $\mathcal{H}$ -consistency guarantees [Mao et al., 2023g].
Structure of this paper. We present new and tight  $\mathcal{H}$ -consistency bounds for both the family of comp-sum losses (Section 4.1) and that of constrained losses (Section 5.1), which cover the familiar cross-entropy, or logistic loss applied to the outputs of a neural network. We further extend our analysis beyond the completeness assumptions adopted in previous studies and cover more realistic bounded hypothesis sets (Section 4.2 and 5.2). Our characterizations are based on error transformations, which are explicitly defined for each formulation. We illustrate the application of our general results through several special examples. A by-product of our analysis is the observation that a recently derived multi-class  $\mathcal{H}$ -consistency bound for cross-entropy reduces to an excess bound independent of the hypothesis set. Instead, we prove a much stronger and more significant guarantee (Section 4.2).
We give a comprehensive discussion of related work in Appendix A. We start with some basic definitions and notation in Section 2.
We denote by  $\mathcal{X}$  the input space, by  $\mathcal{Y}$  the output space, and by  $\mathcal{D}$  a distribution over  $\mathcal{X} \times \mathcal{Y}$ . We consider the standard scenario of multi-class classification, where  $\mathcal{Y} = \{1, \ldots, n\}$ . Given a hypothesis set  $\mathcal{H}$  of functions mapping  $\mathcal{X} \times \mathcal{Y}$  to  $\mathbb{R}$ , the multi-class classification problem consists of finding a hypothesis  $h \in \mathcal{H}$  with small generalization error  $\mathcal{R}_{\ell_{0-1}}(h)$ , defined by  $\mathcal{R}_{\ell_{0-1}}(h) = \mathbb{E}_{(x,y) \sim \mathcal{D}}[\ell_{0-1}(h,x,y)]$ , where  $\ell_{0-1}(h,x,y) = \mathbb{1}_{\mathfrak{h}(x) \neq y}$  is the multi-class zero-one loss with  $\mathfrak{h}(x) = \operatorname{argmax}_{y \in \mathcal{Y}} h(x,y)$  the prediction of  $h$  for the input point  $x$ . We also denote by  $\mathsf{H}(x)$  the set of all predictions associated to input  $x$  generated by functions in  $\mathcal{H}$ , that is,  $\mathsf{H}(x) = \{\mathfrak{h}(x) : h \in \mathcal{H}\}$ .
We will analyze the guarantees of surrogate multi-class losses in terms of the zero-one loss. We denote by  $\ell$  a surrogate loss and by  $\mathcal{R}_{\ell}(h)$  its generalization error,  $\mathcal{R}_{\ell}(h) = \mathbb{E}_{(x,y)\sim \mathcal{D}}[\ell (h,x,y)]$ . For a loss function  $\ell$ , we define the best-in-class generalization error within a hypothesis set  $\mathcal{H}$  as  $\mathcal{R}_{\ell}^{*}(\mathcal{H}) = \inf_{h\in \mathcal{H}}\mathcal{R}_{\ell}(h)$ , and refer to  $\mathcal{R}_{\ell}(h) - \mathcal{R}_{\ell}^{*}(\mathcal{H})$  as the estimation error. We will study the key notion of  $\mathcal{H}$ -consistency bounds [Awasthi et al., 2022a,b], which are upper bounds on the zero-one estimation error of any predictor in a hypothesis set, expressed in terms of its surrogate loss estimation error, for some real-valued function  $f$  that is non-decreasing:
$$
\forall h \in \mathcal {H}, \mathcal {R} _ {\ell_ {0 - 1}} (h) - \mathcal {R} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}) \leq f \big (\mathcal {R} _ {\ell} (h) - \mathcal {R} _ {\ell} ^ {*} (\mathcal {H}) \big).
$$
These bounds imply that the zero-one estimation error is at most  $f(\epsilon)$  whenever the surrogate loss estimation error is bounded by  $\epsilon$ . Thus, the learning guarantees provided by  $\mathcal{H}$ -consistency bounds are both non-asymptotic and hypothesis set-specific. The function  $f$  appearing in these bounds is expressed in terms of a minimizability gap, which is a quantity measuring the difference of best-in-class error  $\mathcal{R}_{\ell}^{*}(\mathcal{H})$  and the expected best-in-class conditional error  $\mathbb{E}_x[\mathcal{C}_{\ell}^*(\mathcal{H}, x)]$ :  $\mathcal{M}_{\ell}(\mathcal{H}) = \mathcal{R}_{\ell}^{*}(\mathcal{H}) - \mathbb{E}_X[\mathcal{C}_{\ell}^*(\mathcal{H}, x)]$ , where  $\mathcal{C}_{\ell}(h, x) = \mathbb{E}_{y|x}[\ell(h, x, y)]$  and  $\mathcal{C}_{\ell}^{*}(\mathcal{H}, x) = \inf_{h \in \mathcal{H}} \mathcal{C}_{\ell}(h, x)$  are the conditional error and best-in-class conditional error respectively. We further write  $\Delta \mathcal{C}_{\ell, \mathcal{H}} = \mathcal{C}_{\ell}(h, x) - \mathcal{C}_{\ell}^{*}(\mathcal{H}, x)$  to denote the conditional regret. Note that the minimizability gap is an inherent quantity depending on a hypothesis set  $\mathcal{H}$  and the loss function  $\ell$ .
By Lemma 1, the minimizability gap for the zero-one loss,  $\mathcal{M}_{\ell_{0 - 1}}(\mathcal{H})$ , coincides with its approximation error  $\mathcal{A}_{\ell_{0 - 1}}(\mathcal{H}) = \mathcal{R}_{\ell_{0 - 1}}^{*}(\mathcal{H}) - \mathcal{R}_{\ell_{0 - 1}}^{*}(\mathcal{H}_{\mathrm{all}})$  when the set of all possible predictions generated by  $\mathcal{H}$  covers the label space  $\mathcal{Y}$ . This holds for typical hypothesis sets used in practice. However, for a surrogate loss  $\ell$ , the minimizability gap  $\mathcal{M}_{\ell}(\mathcal{H})$  is always upper bounded by and in general finer than its approximation error  $\mathcal{A}_{\ell}(\mathcal{H}) = \mathcal{R}_{\ell}^{*}(\mathcal{H}) - \mathcal{R}_{\ell}^{*}(\mathcal{H}_{\mathrm{all}})$  since  $\mathcal{M}_{\ell}(\mathcal{H}) = \mathcal{A}_{\ell}(\mathcal{H}) - I_{\ell}(\mathcal{H})$  where  $\mathcal{H}_{\mathrm{all}}$  is the family of all measurable functions and  $I_{\ell}(\mathcal{H}) = \mathbb{E}_x[\mathcal{C}_{\ell}^{*}(\mathcal{H},x) - \mathcal{C}_{\ell}^{*}(\mathcal{H}_{\mathrm{all}},x)]$  (see Appendix B for a more detailed discussion). Thus, an  $\mathcal{H}$ -consistency bound, expressed as follows for some increasing function  $\Gamma$ :
$$
\left. \mathcal {R} _ {\ell_ {0 - 1}} (h) - \mathcal {R} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell_ {0 - 1}} (\mathcal {H}) \leq \Gamma \left(\mathcal {R} _ {\ell} (h) - \mathcal {R} _ {\ell} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell} (\mathcal {H})\right), \right. \tag {1}
$$
is more favorable than an excess error bound expressed in terms of approximation errors  $\mathcal{R}_{\ell_{0-1}}(h) - \mathcal{R}_{\ell_{0-1}}^*(\mathcal{H}) + \mathcal{A}_{\ell_{0-1}}(\mathcal{H}) \leq \Gamma(\mathcal{R}_\ell(h) - \mathcal{R}_\ell^*(\mathcal{H}) + \mathcal{A}_\ell(\mathcal{H}))$ . Here,  $\Gamma$  is typically linear or the square-root function modulo constants. When  $\mathcal{H} = \mathcal{H}_{\mathrm{all}}$ , the family of all measurable functions, an  $\mathcal{H}$ -consistency bound coincides with the excess error bound and implies Bayes-consistency by taking the limit. It is therefore a stronger guarantee than an excess error bound and Bayes-consistency.
The minimizability gap is always non-negative, since the infimum of the expectation is greater than or equal to the expectation of infimum. Furthermore, as shown in Appendix B, when  $\mathcal{H}$  is the family of all measurable functions or when the Bayes-error coincides with the best-in-class error, that is,  $\mathcal{R}_{\ell}^{*}(\mathcal{H}) = \mathcal{R}_{\ell}^{*}(\mathcal{H}_{\mathrm{all}})$ , the minimizability gap vanishes. In such cases, (1) implies the  $\mathcal{H}$ -consistency of a surrogate loss  $\ell$  with respect to the zero-one loss  $\ell_{0-1}$ :
$$
\mathcal {R} _ {\ell} \left(h _ {n}\right) - \mathcal {R} _ {\ell} ^ {*} (\mathcal {H}) \xrightarrow {n \to + \infty} 0 \Longrightarrow \mathcal {R} _ {\ell_ {0 - 1}} \left(h _ {n}\right) - \mathcal {R} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}) \xrightarrow {n \to + \infty} 0.
$$
In the next sections, we will provide both a general characterization and an extension of  $\mathcal{H}$ -consistency bounds for multi-class classification. Before proceeding, we first introduce a useful lemma from [Awasthi et al., 2022b] which characterizes the conditional regret of zero-one loss explicitly. We denote by  $p(x) = (p(x,1),\ldots ,p(x,n))$  as the conditional distribution of  $y$  given  $x$ .
Lemma 1. For zero-one loss  $\ell_{0-1}$ , the best-in-class conditional error and the conditional regret for  $\ell_{0-1}$  can be expressed as follows: for any  $x \in \mathcal{X}$ , we have
$$
\mathcal {C} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}, x) = 1 - \max  _ {y \in \mathsf {H} (x)} p (x, y) \quad a n d \quad \Delta \mathcal {C} _ {\ell_ {0 - 1}, \mathcal {H}} (h, x) = \max  _ {y \in \mathsf {H} (x)} p (x, y) - p (x, \mathsf {h} (x)).
$$
3 Comparison with previous work
Here, we briefly discuss previous studies of  $\mathcal{H}$ -consistency bounds [Awasthi et al., 2022a,b, Zheng et al., 2023, Mao et al., 2023h] in standard binary or multi-class classification and compare their results with those we present.
Awasthi et al. [2022a] studied  $\mathcal{H}$ -consistency bounds in binary classification. They provided a series of tight  $\mathcal{H}$ -consistency bounds for the bounded hypothesis set of linear models  $\mathcal{H}_{\mathrm{lin}}^{\mathrm{bi}}$  and one-hidden-layer neural networks  $\mathcal{H}_{\mathrm{NN}}^{\mathrm{bi}}$ , defined as follows:
$$
\mathcal {H} _ {\mathrm {l i n}} ^ {\mathrm {b i}} = \left\{x \mapsto w \cdot x + b | \| w \| \leq W, | b | \leq B \right\}
$$
$$
\mathcal {H} _ {\mathrm {N N}} ^ {\mathrm {b i}} = \left\{x \mapsto \sum_ {j = 1} ^ {n} u _ {j} \left(w _ {j} \cdot x + b\right) _ {+} \mid \| u \| _ {1} \leq \Lambda , \| w _ {j} \| \leq W, | b | \leq B \right\},
$$
where  $B$ ,  $W$ , and  $\Lambda$  are positive constants and where  $(\cdot)_{+} = \max (\cdot ,0)$ . We will show that our bounds recover these binary classification  $\mathcal{H}$ -consistency bounds.
The scenario of multi-class classification is more challenging and more crucial in applications. Recent work by Awasthi et al. [2022b] showed that max losses [Crammer and Singer, 2001], defined as  $\ell^{\max}(h,x,y) = \max_{y' \neq y} \Phi(h(x,y) - h(x,y'))$  for some convex and non-increasing function  $\Phi$ , cannot admit meaningful  $\mathcal{H}$ -consistency bounds, unless the distribution is deterministic. They also presented a series of  $\mathcal{H}$ -consistency bounds for sum losses [Weston and Watkins, 1998] and constrained losses [Lee et al., 2004] for symmetric and complete hypothesis sets, that is such that:
$$
\mathcal {H} = \{h: \mathcal {X} \times \mathcal {Y} \rightarrow \mathbb {R}: h (\cdot , y) \in \mathcal {F}, \forall y \in \mathcal {Y} \} \quad \text {(s y m m e t r y)}
$$
$$
\forall x \in \mathcal {X}, \{f (x): f \in \mathcal {F} \} = \mathbb {R}, \tag {completeness}
$$
for some family  $\mathcal{F}$  of functions mapping from  $\mathcal{X}$  to  $\mathbb{R}$ . The completeness assumption rules out the bounded hypothesis sets typically used in practice such as  $\mathcal{H}_{\mathrm{lin}}$ . Moreover, the final bounds derived from [Awasthi et al., 2022b] are based on ad hoc proofs and may not be tight. In contrast, we will study both the complete and bounded hypothesis sets, and provide a very general tool to derive  $\mathcal{H}$ -consistency bounds. Our bounds are tighter than those of Awasthi et al. [2022b] given for complete hypothesis sets and extend beyond the completeness assumption.
[Mao et al., 2023h] complemented the work of [Awasthi et al., 2022b] by studying a wide family of comp-sum losses in multi-class classification, which generalized the sum-losses and included as special cases the logistic loss [Verhulst, 1838, 1845, Berkson, 1944, 1951], the generalized cross-entropy loss [Zhang and Sabuncu, 2018], and the mean absolute error loss [Ghosh et al., 2017]. Here too, the completeness assumption was adopted, thus their  $\mathcal{H}$ -consistency bounds do not apply to common bounded hypothesis sets used in practice. We illustrate the application of our general results through a broader set of surrogate losses than [Mao et al., 2023h] and significantly generalize the bounds of [Mao et al., 2023h] to bounded hypothesis sets.
Recently, Zheng et al. [2023] proved  $\mathcal{H}$ -consistency bounds for logistic loss with linear hypothesis sets in the multi-class classification:  $\mathcal{H}_{\mathrm{lin}} = \{x \mapsto w_y \cdot x + b_y \mid \|w_y\| \leq W, |b_y| \leq B, y \in \mathcal{Y}\}$ . However, their bounds require a crucial distributional assumption under which, the minimizability gaps  $\mathcal{M}_{\ell_{0-1}}(\mathcal{H}_{\mathrm{lin}})$  and  $\mathcal{M}_{\ell_{\log}}(\mathcal{H}_{\mathrm{lin}})$  coincide with the approximation errors  $\mathcal{R}_{\ell_{0-1}}(\mathcal{H}_{\mathrm{lin}}) - \mathcal{R}_{\ell_{0-1}}^*(\mathcal{H}_{\mathrm{all}})$  and  $\mathcal{R}_{\ell_{\log}}(\mathcal{H}_{\mathrm{lin}}) - \mathcal{R}_{\ell_{\log}}^*(\mathcal{H}_{\mathrm{all}})$  respectively (see the note before [Zheng et al., 2023, Appendix F]). Thus, their bounds can be recovered as excess error bounds  $\mathcal{R}_{\ell_{0-1}}(h) - \mathcal{R}_{\ell_{0-1}}^*(\mathcal{H}_{\mathrm{all}}) \leq \sqrt{2} \left( \mathcal{R}_{\ell_{\log}}(h) - \mathcal{R}_{\ell_{\log}}^*(\mathcal{H}_{\mathrm{all}}) \right)^{\frac{1}{2}}$  which are less significant. In contrast, our  $\mathcal{H}_{\mathrm{lin}}$ -consistency bound is much finer and takes into account the role of the parameter  $B$  and that of the number of labels  $n$ . Thus, we provide stronger and more significant guarantees for logistic loss with linear hypothesis set than [Zheng et al., 2023].
In summary, our general tools offer the remarkable advantages of deriving tight bounds, which improve upon the existing bounds of Awasthi et al. [2022b] given for complete hypothesis sets, cover the comp-sum and constrained losses considered in [Awasthi et al., 2022a, Mao et al., 2023h] as well as new ones, extend beyond the completeness assumption with novel guarantees valid for bounded hypothesis sets, and are much stronger and more significant guarantees for logistic loss with linear hypothesis sets than those of Zheng et al. [2023].
4 Comp-sum losses
In this section, we present a general characterization of  $\mathcal{H}$ -consistency bounds for comp-sum losses, a family of loss functions including the logistic loss [Verhulst, 1838, 1845, Berkson, 1944, 1951], the sum exponential loss [Weston and Watkins, 1998, Awasthi et al., 2022b], the generalized cross entropy loss [Zhang and Sabuncu, 2018], the mean absolute error loss [Ghosh et al., 2017], and many other loss functions used in applications.
This is a family of loss functions defined via the composition of a non-negative and non-decreasing function  $\Psi$  with the sum exponential losses (see [Mao et al., 2023h]):
$$
\forall h \in \mathcal {H}, \forall (x, y) \times \mathcal {X} \times \mathcal {Y}, \quad \ell^ {\operatorname {c o m p}} (h, x, y) = \Psi \left(\sum_ {y ^ {\prime} \neq \mathcal {Y}} e ^ {h (x, y ^ {\prime}) - h (x, y)}\right). \tag {2}
$$
This expression can be equivalently written as  $\ell^{\mathrm{comp}}(h,x,y) = \Phi \left(\frac{e^{h(x,y)}}{\sum_{y'\in\mathcal{Y}}e^{h(x,y')}}\right)$ , where  $\Phi: u \mapsto \Psi\left(\frac{1-u}{u}\right)$  is a non-increasing auxiliary function from  $[0,1]$  to  $\mathbb{R}_+ \cup \{+\infty\}$ . As an example, the logistic loss corresponds to the choice  $\Phi: u \mapsto -\log(u)$  and the sum exponential loss to  $\Phi: u \mapsto \frac{1-u}{u}$ .
4.1  $\mathcal{H}$ -consistency bounds
In previous work, deriving  $\mathcal{H}$ -consistency bounds has required giving new proofs for each instance. The following result provides a very general tool for deriving such bounds with tightness guarantees. We introduce an error transformation function and show that deriving an  $\mathcal{H}$ -consistency bound for comp-sum losses can be reduced to the calculation of this function.
Theorem 2 (H-consistency bound for comp-sum losses). Assume that  $\mathcal{H}$  is symmetric and complete and that  $\mathfrak{T}^{\mathrm{comp}}$  is convex. Then, the following inequality holds for any hypothesis  $h\in \mathcal{H}$  and any distribution
$$
\mathfrak {T} ^ {\operatorname {c o m p}} \left(\mathcal {R} _ {\ell_ {0 - 1}} (h) - \mathcal {R} _ {\ell_ {0 - 1}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell_ {0 - 1}} (\mathcal {H})\right) \leq \mathcal {R} _ {\ell^ {\operatorname {c o m p}}} (h) - \mathcal {R} _ {\ell^ {\operatorname {c o m p}}} ^ {*} (\mathcal {H}) + \mathcal {M} _ {\ell^ {\operatorname {c o m p}}} (\mathcal {H}), \tag {3}
$$
with  $\mathfrak{T}^{\mathrm{comp}}$  an  $\mathcal{H}$ -estimation error transformation for comp-sum losses defined for all  $t\in [0,1]$  by  $\mathfrak{T}^{\mathrm{comp}}(t) =$