Title: A Unifying Post-Processing Framework for Multi-Objective Learn-to-Defer Problems
Venue: NeurIPS 2024
Accepted: yes
Keywords: Learn-to-Defer

Human-AI Teaming

Neyman-Pearson Lemma

Hypothesis Testing

Bayes Optimal

PAC Generalization
Abstract: Learn-to-Defer is a paradigm that enables learning algorithms to work not in isolation but as a team with human experts. In this paradigm, we permit the system to defer a subset of its tasks to the expert. Although there are currently systems that follow this paradigm and are designed to optimize the accuracy of the final human-AI team, the general methodology for developing such systems under a set of constraints (e.g., algorithmic fairness, expert intervention budget, defer of anomaly, etc.) remains largely unexplored. In this paper, using a d-dimensional generalization to the fundamental lemma of Neyman and Pearson (d-GNP), we obtain the Bayes optimal solution for learn-to-defer systems under various constraints. Furthermore, we design a generalizable algorithm to estimate that solution and apply this algorithm to the COMPAS, Hatespeech, and ACSIncome datasets. Our algorithm shows improvements in terms of constraint violation over a set of learn-to-defer baselines and can control multiple constraint violations at once. The use of d-GNP is beyond learn-to-defer applications and can potentially obtain a solution to decision-making problems with a set of controlled expected performance measures.

A Unifying Post-Processing Framework for Multi-Objective Learn-to-Defer Problems

Abstract

Learn-to-Defer is a paradigm that enables learning algorithms to work not in isolation but as a team with human experts. In this paradigm, we permit the system to defer a subset of its tasks to the expert. Although there are currently systems that follow this paradigm and are designed to optimize the accuracy of the final human-AI team, the general methodology for developing such systems under a set of constraints (e.g., algorithmic fairness, expert intervention budget, defer of anomaly, etc.) remains largely unexplored. In this paper, using a d -dimensional generalization to the fundamental lemma of Neyman and Pearson ( d -GNP), we obtain the Bayes optimal solution for learn-to-defer systems under various constraints. Furthermore, we design a generalizable algorithm to estimate that solution and apply this algorithm to the COMPAS, Hatespeech, and ACSIncome datasets. Our algorithm shows improvements in terms of constraint violation over a set of learn-to-defer baselines and can control multiple constraint violations at once. The use of d -GNP is beyond learn-to-defer applications and can potentially obtain a solution to decision-making problems with a set of controlled expected performance measures.

1 Introduction

Machine learning algorithms are increasingly used in diverse fields, including critical applications, such as medical diagnostics [72] and predicting optimal prognostics [63]. To address the sensitivity of such tasks, existing approaches suggest keeping the human expert in the loop and using the machine learning prediction as advice [35], or playing a supportive role by taking over the tasks on which machine learning is uncertain [39, 60, 4]. The abstention of the classifier in making decisions, and letting the human expert do so, is where the paradigm of learn-to-defer (L2D) started to exist.

The development of L2D algorithms has mainly revolved around optimizing the accuracy of the final system under such paradigm [60, 50]. Although they achieve better accuracy than either the machine learning algorithm or the human expert in isolation, these works provide inherently single-objective solutions to the L2D problem. In the critical tasks that are mentioned earlier, more often than not, we face a challenging multi-objective problem of ensuring the safety, algorithmic fairness, and practicality of the final solution. In such settings, we seek to limit the cost of incorrect decisions [46], algorithmic biases [13], or human expert intervention [57], while optimizing the accuracy of the system. Although the seminal paper that introduced the first L2D algorithm targeted an instance of such multi-objective problem [44], a general solution to such class of problems, besides specific examples [26, 57, 51, 52], has remained unknown to date. Multi-objective machine learning extends beyond the realm of L2D problems. A prime example that is extensively studied in various settings is ensuring algorithmic fairness [18] while optimizing accuracy. Recent advances in the algorithmic fairness literature have suggested the superiority of post-processing methodology for tackling this

Train

Validation

suph,p E[Ldef(h(x), r(x)]

[&train| = Ntrain

constraints

Xval

Xtrain

Wo(x)

Wo(x)

Wm(x)

Yes

truelabelsand

yo(x)

Embedding

Wm(x)

Linear Combination

Fine-tune

using

+

Predict

g

No

g=L

expert decisions

Function

Argmax

{(deer,horse),(frog,frog),(dog,bird),..(deer, deer)}

multi-objective problem [73, 14, 20, 76]. Post-processing algorithms operate in two steps: first, they find a calibrated estimation of a set of probability scores for each input via learning algorithms, and then they obtain the optimal predictor as a function of these scores. Similarly, in a recent set of works, optimal algorithms to reject the decision-making under a variety of secondary objectives are determined via post-processing algorithms [51, 52], which is in line with classical results such as Chow's rule [16] that is the simplest form of a post-processing method, thresholding the likelihood.

Inspired by the above works, in this paper, we fully characterize the solution to multi-objective L2D problems using a post-processing framework. In particular, we consider a deferral system together with a set of conditional performance measures { Ψ 0 , . . . , Ψ m } that are functions of the system outcome ˆ Y , the target label Y , and the input X . The goal is to optimize the average value of Ψ 0 over data distribution while keeping the average value of the rest of performance measures Ψ 1 , . . . , Ψ m for all inputs under control. As an example, in binary classification, Ψ 0 can be the 0 -1 deferral loss function, while Ψ 1 can be the difference between positive prediction rates of ˆ Y for all instances of X that belong to demographic group A = 0 or A = 1 . The solution for which we aim optimizes the accuracy while assuring that the demographic parity measure between the two groups is bounded by a tolerance value δ 1 ∈ [0 , 1] .

To provide the optimal solution, we move beyond staged learning [12] methodology, in which the classifier h ( x ) is trained in the absence of human decision-makers, and then the optimal rejection function r ( x ) is obtained for that classifier to decide when the human expert should intervene ( r ( x ) = 1 ). Instead, we jointly obtain the classifier and rejection function. The reason that we avoid this methodology is that firstly, objectives such as algorithmic fairness are not compositional, i.e., even if the classifier and the human are fair, due to the emergence of Yule's effect [62] the obtained deferral system is not necessarily fair (see Appendix A), and in fact abstention systems can deter the algorithmic biases [36]. Secondly, the feasibility of constraints is not guaranteed under staged learning methodology [74], e.g., there can be cases in which achieving a purely fair solution is impossible, while this occurs neither in vanilla classification [20] nor in our solution.

This paper shows that the joint learning of classifier and rejection function for finding the optimal multi-objective L2D solution boils down to a generalization of the fundamental Neyman-Pearson lemma [55]. This lemma is initially introduced in studying hypothesis testing problems and characterizes the most powerful test (i.e., the test with the highest true positive rate) while keeping the significance level (true negative rate) under control. As a natural extension to this paradigm, we consider a multi-hypothesis setting where for each true positive prediction and false negative prediction, we receive a reward and loss, respectively. Then, we show that the extension of Neyman-Pearson lemma to this setting provides us with a solution for our multi-objective L2D problem.

In summary, the contribution of this paper is as below:

In Section 3, we show that obtaining the optimal deterministic classifier and rejection function under a constraint is, in general, an NP-Hard problem, then

by introducing randomness, we rephrase the multi-objective L2D problem into a functional linear programming.

In Section 4, we show that such linear programming problem is an instance of d -dimensional generalized Neyman-Pearson ( d -GNP) problem, then

we characterize the solution to d -GNP problem, and we particularly derive the corresponding parameters of the solution when the optimization is restricted by a single constraint.

In Section 5, we show that a post-processing algorithm that is based on d -GNP solution generalizes in constraints and objective with the rate O ( √ log n/n, √ log(1 /ϵ ) /n, ϵ ′ ) and O ((log n/n ) 1 / 2 γ , (log(1 /ϵ ) /n ) 1 / 2 γ , ϵ ′ ) , respectively, with probability at least 1 -ϵ where n is the size of the set using which we fine-tune the algorithm, ϵ ′ measures the accuracy of learned post-processing scores, and γ is a parameter that measures the sensitivity of the constraint to the change of the predictor. Then,

we show that the use of in-processing methods in L2D problem does not necessarily generalize to the unobserved data, and finally

we experiment our post-processing algorithm on two tabular datasets and a text dataset, and observe its performance compared to the baselines for ensuring demographic parity and equality of opportunity on final predictions.

Lastly, the d -GNP theorem has potential use cases beyond the L2D problem, particularly in vanilla classification problems under constraints. However, such applications are beyond the scope of this paper, and except for a brief explanation of the use of d -GNP in algorithmic fairness for multiclass classification, we leave them to future works.

2 Related Works

Human and ML's collaboration in decision-making has been demonstrated to enhance the accuracy of final decisions compared to predictions that are made solely by humans or ML [37, 68]. This overperformance is due to the ability to estimate the accuracy and confidence of each agent on different regions of data and subsequently allocate instances between human and ML to optimize the overall accuracy [2]. Since the introduction of the L2D problem, the implementation of its optimal rule has been the focus of interest in this field [8, 50, 12, 51, 9, 43, 48, 45]. The multiobjective classification with abstention problems is studied for specific objectives in [44, 57, 48] via in-processing methods. The application of Neyman-Pearson lemma for learning problems with fairness criteria is recently introduced in [75].

We refer the reader to Appendix B for further discussion on related works.

3 Problem Setting

Assume that we are given input features x i ∈ X , corresponding labels y i ∈ Y = { 1 , . . . , L } , and the human expert decision m i for such input, and assume that these are i.i.d. realizations of random variables X,Y,M ∼ µ = µ XYM . Since there exists randomness in the human decision-making process, for the sake of generality, we treat M as a random variable similar to Y and do not assume that m i = m ( x i ) for some function m . Further, assume that for the true label y and a certain feature vector x , the cost of incorrect predictions is measured by a loss function ℓ AI ( y, h ( x )) for the classifier prediction h ( x ) , and a loss function ℓ H ( y, m ) for human's prediction m . The question that we tackle in this paper is the following: What is an optimal classifier and otherwise an optimal way of deferring the decision to the human when there are constraints that limit the decision-making? The constraints above can be algorithmic fairness constraints (e.g., demographic parity, equality of opportunity, equalized odds), expert intervention constraints (e.g., when the human expert can classify up to b proportion of the data), or spatial constraints to enforce deferral on certain inputs, or any combination thereof.

Let us put the above question in a formal optimization form. To that end, let r ( x ) ∈ { 0 , 1 } be the rejection function 1 , i.e., when r ( x ) = 0 the classifier makes the decision for input x and otherwise x is deferred to the expert. We obtain the deferral loss on x and given a label y and the expert decision m as

I

Y

A

=1

=

y,A

I

Y

A

=0

=

y,A

̸

Therefore, we can find the average deferral loss on distribution µ as

We aim to find a randomized algorithm A that defines a probability distribution µ A on H×R that solves the optimization problem

where Ψ i is a performance measure that induces the desired constraint in our optimization problem. We assume that Ψ i , similar to ℓ def , is an outcome-dependent function, i.e., if the deferral occurs, the outcome of the classifier does not change Ψ i , and otherwise, if deferral does not occur, the human decision does not change Ψ i . In other words, the value of the constraints can only be a function of input feature x and of the deferral system prediction ˆ Y = r ( x ) M + ( 1 -r ( x ) ) h ( x ) . Here, ˆ Y is the expert decision when deferral occurs, and is the classifier decision otherwise.

Types of constraints. Before we discuss our methodology to solve (2), it is beneficial to review the types of constraints with which we are concerned: (1) expert intervention budget that can be written in form of Pr ( r ( X ) = 1 ) ≤ δ , limits the rejection function to defer up to δ proportion of the instance,

(2) demographic parity that is formulated as ∣ ∣ P ( ˆ Y = 1 | A = 0) -P ( ˆ Y = 1 | A = 0) ∣ ∣ ≤ δ , ensures that the proportion of positive predictions for the first demographic group ( A = 0 ) is comparable to that for the second demographic group ( A = 1 ). (3) equality of opportunity that is defined as | Pr ( ˆ Y = 1 | A = 1 , Y = 1) -Pr ( ˆ Y | A = 0 , Y = 1) | ≤ δ limits the differences between correct positive predictions among two demographic groups, (4) equalized odds that is similar to equality of opportunity but targets the differences of correct positive and negative predictions among two groups, i.e., max y =0 , 1 ∣ ∣ Pr ( ˆ Y = 1 | A = 1 , Y = y ) -Pr ( ˆ Y = 1 | A = 0 , Y = y ) ∣ ∣ ≤ δ , (5) out-ofdistribution (OOD) detection that is written as Pr out ( r ( X ) = 0) ≤ δ limits the prediction of the classifier on points that are outside its training distribution and incentivizes deferral in such cases, (6) long-tail classification deals with high class imbalances. This method aims to minimize a balanced error of classifier prediction on instances where deferral does not occur. Achieving this objective as

-

.

Pr

(

=1)

Pr

(

=0)

̸

̸

̸

mentioned in [53] is equivalent to minimizing ∑ K i =1 1 α i Pr( Y = h ( X ) , r ( X ) = 0 | Y ∈ G i ) when the feasible set is Pr( r ( X ) = 0 , Y ∈ G i ) = α i K , and where { G i } K i =1 is a partition of classes, and finally (7) typek error bounds that is a generalization of Type-I and Type-II errors, limits errors of a specific class k using Pr( ˆ Y = k | Y = k ) ≤ δ .

All above constraints are expected values of outcome-dependent functions (see Appendix D for proof). To put it informally, if we change the classifier outcome after the rejection, such constraints do not vary.

Linear Programming Equivalent to (2) . The outcome-dependence property helps us to show that (see Appendix C) obtaining the optimal classifier and rejection function is equivalent to obtaining the solution of

where ∆ d is a simplex of d dimensions, d = L +1 , and ψ i : X → R d is defined as

̸

that we name the embedding function 2 corresponding to the performance measure Ψ i for i ∈ [0 : m ] , where for simplifying the notation we define Ψ 0 ≡ -ℓ def . Furthermore, the optimal algorithm is obtained by predicting h ( x ) = i with normalized probability of f ∗ i ( x ) / ∑ d -1 j =1 f ∗ j ( x ) , where ∑ d -1 j =1 f ∗ j ( x ) = 0 , and rejecting r ( x ) = 1 with probability f ∗ d ( x ) . In case of ∑ d -1 j =1 f ∗ j ( x ) = 0 the classifier is defined arbitrarily. A list of embedding functions for the mentioned constraints and objectives is provided in Table 1 (See Appendix D for derivations).

Hardness. We first derive the following negative result for the optimal deterministic predictor in (3). Weuse the similarity between (3) and 0 -1 Knapsack problem (see [58, pp. 374]) to show that there are cases in which solving the former is equivalent to solving an NP-Hard problem. More particularly, if we assume that the distribution of X contains finite atoms x 1 , . . . , x n , each of which have probability of Pr( X = x i ) = p i , and if we set ψ 1 ( x i ) = [0 , w i p i ] and ψ 0 ( x i ) = [0 , v i p i ] for v i , w i ∈ R + , then (3) reduces in argmax ∑ i f 1 ( x i ) v i subjected to f 1 : X → { 0 , 1 } and ∑ i f 1 ( x i ) w i ≤ δ 1 , which is the main form of the Knapsack problem. In the following theorem, we show that a similar result can be obtained if we choose ψ 0 and ψ 1 to be embedding functions corresponding to accuracy and expert intervention budget. All proofs of theorems can be found in the appendix.

Theorem 3.1 (NP-Hardness of (2)) . Let the human expert and the classifier induce 0 -1 losses and assume X to be finite. Finding an optimal deterministic classifier and rejection function for a bounded expert intervention budget is an NP-Hard problem.

Note that the above finding is different from the complexity results for deferral problems in [49, Theorem 1] and [23, Theorem 1]. NP-hardness results in these settings are consequences of restricting the search to a specific space of models, i.e., the intersection of half-spaces and linear models on a subset of the data. However, in our theorem, the hardness arises due to a possibly complex data distribution and not because of the complex model space.

The above hardness theorem for deterministic predictors justifies our choice of using randomized algorithms to solve multi-objective L2D. In the next section, by finding a closed-form solution for the randomized algorithm, we show that such relaxation indeed simplifies the problem.

4 d -dimensional Generalization of Neyman-Pearson Lemma

The idea behind minimizing an expected error while keeping another expected error bounded is naturally related to the problem that is designed by Neyman and Pearson [55]. They consider two hypotheses H 0 , H 1 as two distributions with density functions g 0 ( x ) and g 1 ( x ) for which a given point x can be drawn. Then, they maximize the probability of correctly rejecting H 0 , while bounding the probability of incorrectly rejecting H 0 , i.e., for a test T ( x ) ∈ [0 , 1] that rejects the null hypothesis when T ( x ) = 1 , they solved the problem

̸

They concluded that thresholding the likelihood ratio is a solution to the above problem. Formally, they show that all optimal hypothesis tests take the value T ( x ) = 1 when g 1 ( x ) /g 0 ( x ) > k and take the value T ( x ) = 0 when g 1 ( x ) /g 0 ( x ) < k , where k is a scalar and dependent on α .

Multi-hypothesis testing with rewards. In this section, we aim to solve (3) as a generalization of Neyman-Pearson lemma for binary testing to the case of multi-hypothesis testing, in which correctly and incorrectly rejecting each hypothesis has a certain reward and loss. To clarify how the extension of this setting and the problem (3) are equivalent, assume the general case of d hypotheses H 0 , . . . , H d -1 , each of which corresponding to X being drawn from the density function g i ( x ) for i ∈ { 0 , . . . , d -1 } . Further, assume that for each hypothesis H i , in case of true positive, we receive the reward r i ( x ) , and in case of false negative, we receive the loss ℓ i ( x ) . Assume that we aim to find a test f : X → ∆ d that for each input x ∈ X rejects d -1 hypotheses, each hypothesis H i with probability 1 -f i ( x ) and maximizes a sum of true positive rewards, and that keeps the sum of false negative losses under control. Then, this is equivalent to argmax f ∈ ∆ X d ∑ d -1 i =0 E X ∼ g i [ f i ( x ) r i ( x ) ]

subjected to ∑ d -1 i =0 E X ∼ g i [ (1 -f i ( x )) ℓ i ( x ) ] ≤ δ 1 which in turns is equivalent to

̸

g ( x )

This problem can be seen as instance of (3), when we set ψ 0 ( x ) = [ r 0 ( x ) , . . . , r d -1 ( x ) d -1 g 0 ( x ) ] and ψ 1 ( x ) = [ ∑ j =0 ℓ j ( x ) g j ( x ) g 0 ( x ) , . . . , ∑ j = d -1 ℓ j ( x ) g j ( x ) g 0 ( x ) ] . Similarly, we can show that for all ψ 0 ( x ) , ψ 1 ( x ) in (3) there exists a set of densities g 1 ( x ) , . . . , g d -1 ( x ) and rewards and losses such that (6) and (3) are equivalent. This can be done by setting g i ≡ g 0 and noting that the mapping from ℓ i s and r i s into ψ 0 and ψ 2 is invertible.

̸

̸

The formulation of (3) can be seen as an extension of the setting in [69] when we move beyond typek error bounds to a general set of constraints. That work achieves the optimal test by applying strong duality on the Lagrangian form of the constrained optimization problem. However, we avoided using this approach in proving our solution, since finding f ∗ , and not the optimal objective, is possible via strong duality only when we know apriori that the Lagrangian has a single saddle point (for more details and fallacy of such approach, see Section E). As another improvement to the duality method, we not only find a solution to (3), but also show that there is no other solution that works as well as ours.

Before we express our solution in the following theorem, we define an import notation as an extension of the argmax function that helps us articulate the optimal predictor. In fact, we define

that is a set of functions that result in one-hot encoded argmax when there is a clear maximum, and otherwise, based on its second argument, results in a probability distribution on all components that achieved the maximum value.

Theorem 4.1 ( d -GNP) . For a set of functions ψ i where i ∈ [0 , m ] , assume that ( δ 1 , . . . , δ m ) is an interior point 3 of the set F = { ( E [ ⟨ r ( x ) , ψ 1 ( x ) ⟩ ] , . . . , E [ ⟨ r ( x ) , ψ m ( x ) ⟩ ] ) : f ∈ ∆ X d } . Then, there is a set of fixed values k 1 , . . . , k m and τ ∈ T d such that the predictor

obtains the optimal solution of sup f ∈ ∆ X d E [ ⟨ f ( x ) , ψ 0 ( x ) ⟩ ] , subjected to the constraints being achieved tightly, i.e., when for i ∈ [1 : m ] we have E [ ⟨ f ( x ) , ψ i ( x ) ⟩ ] = δ i . If k 1 , . . . , k m are further non-negative, then f ∗ ( x ) is the optimal solution to (3) . Moreover, all optimal solutions of (3) that tightly achieve the constraints are in form of (8) almost everywhere on X .

Example 1 (L2D with Demographic Parity) . In the setting that we have a deferral system and we aim for controlling demographic disparity under the tolerance δ , we can set ψ 0 ( x ) = [ Pr( Y =

0 | x ) , Pr( Y = 1 | x ) , Pr( Y = M | x ) ] and ψ 1 ( x ) = s ( A ) [ 0 , 1 , Pr( M = 1 | x ) ] , using Table 1, where s ( A ) := ( I A =1 Pr ( A =1) -I A =0 Pr ( A =0) ) . Therefore, d -GNP, together with the discussion after (4) shows that the optimal classifier and rejection function are obtained as

and

for a fixed value k ∈ R , and where λ ( A,x ) := max { Pr( Y = 0 | x ) , Pr( Y = 1 | x ) -ks ( A ) } . The above identities imply that the optimal fair classifier for the deferral system thresholds the scores for different demographic groups using two thresholds ks (0) and ks (1) . This is similar in form to the optimal fair classifier in vanilla classification problem [14, 20]. However, the rejection function does not merely threshold the scores for different groups, but adds an input-dependent threshold ks ( A ) Pr( M = 1 | x ) to the unconstrained deferral system scores.

It is important to note that although we have a thresholding rule for the classifier, the thresholds are not necessarily the same as of isolated classifier under fairness criteria. Furthermore, the deferral rule is dependent on the thresholds that we use for the classifier. Therefore, we cannot train the classifier for a certain demographic parity and a rejection function in two independent stages. This further affirms the lack of compositionality of algorithmic fairness that we discussed earlier in the introduction of this paper.

Example 2 (L2D with Equality of Opportunity) . Here, similar to the previous example, we can obtain the embedding function for accuracy and equality of opportunity constraint as ψ 0 ( x ) = [ p 0 x , p 1 x , p x M ] and ψ 1 ( x ) = t ( A, 1) [ 0 , p 1 x , Pr( M = 1 , Y = 1 | x ) ] , respectively, where p i x := Pr( Y = i | x ) for i ∈ { 1 , 2 } and similarly p x M = Pr( Y = M | x ) . Therefore, the characterization of optimal classifier and rejection function using d -GNP results in

and

for k ∈ R and where ν ( A,x ) := max { p 0 x , ( 1 -kt ( A, 1) ) p 1 x } . Assuming 2 -kt ( A, 1) takes positive values for all choices of A , we conclude that the optimal classifier is to threshold positive scores differently for different demographic groups. However, the optimal deferral is a function of probability of positive prediction by human expert.

Example 3 (Algorithmic Fairness for Multiclass Classification) . In addition to addressing the L2D problem, the formulation of d -GNP in Theorem 4.1 allows for finding the optimal solution in vanilla classification. In fact, for an L -class classifier, if we aim to set constraints on demographic parity ∣ ∣ Pr( ˆ Y = 0 | A = 0) -Pr( ˆ Y = 0 | A = 1) ∣ ∣ ≤ δ or equality of opportunity ∣ ∣ Pr( ˆ Y = 0 | Y = 0 , A = 0) -Pr( ˆ Y = 0 | Y = 0 , A = 1) ∣ ∣ ≤ δ on Class 0 , then we can follow similar steps as in Appendix D to find the embedding functions as ψ DP = s ( A ) [ 1 , 0 , . . . , 0 ] and ψ EO = t ( A, 0) [ p 0 x , 0 , . . . , 0 ] , where p i x := Pr( Y = i | x ) for i ∈ [ L ] .

As a result, since the accuracy embedding function is ψ 0 ( x ) = [ p 0 x , . . . , p L x ] , then, by neglecting the effect of randomness, the optimal classifier under such constraints are as

Equivalently, for demographic parity, the optimal classifier includes a shift on the score of Class 0 as a function of demographic group, and for equality of opportunity, the optimal classifier includes a multiplication of the score of Class 0 with a value that is a function of demographic group. It is easy to show that under condition of positivity of the multiplied value, these classifiers both reduce to thresholding rules in binary setting.

and

Note that although Theorem 4.1 characterizes the optimal solution of (3), it leaves us uninformed regarding parameters k 1 , . . . , k m , and further does not give us the form of the optimal solution when ψ 0 ( x ) -∑ m i =1 k i ψ i ( x ) has more than one maximizer. In the following theorem, we address these issues for the case that we have a single constraint.

Theorem 4.2 ( d -GNP with a single constraint) . The optimal solution (8) of the optimization problem (3) with one constraint is equal to f ∗ k,p ( x ) = τ ( ψ 0 ( x ) -kψ 1 ( x ) , x ) where τ is a member of T d such that if there is a non-singleton set I of maximizers of a vector y ∈ R d , then we have ( τ ( y , x ) ) ( i ) = p and ( τ ( y , x ) ) ( j ) = 1 -p , where i and j are the first indices in I that minimizes ψ 1 ( x ) , and maximizes ψ 0 ( x ) , respectively. In this case, k is a member of the set K = { t : δ ∈ [ lim τ ↑ t C ( τ ) , C ( t ) ] } where C ( t ) = E [ ⟨ f ∗ t, 0 ( x ) , ψ 0 ( x ) ⟩ ] is the expected constraint of the predictor f ∗ t, 0 . Moreover, p = C ( k ) -c C ( k ) -lim τ ↑ t -C ( τ ) , if C ( · ) is lower-discontinuous at k , and otherwise p = 0 .

This theorem reduces the complexity of finding k i s from the complexity of an exhaustive search to the complexity of finding the root of the monotone function C ( t ) -δ (see Lemma J.2 for the proof of monotonicity), and further finds the randomized response for the cases that Theorem 4.1 leaves undetermined.

Before we proceed to the designed algorithm based on d -GNP, we should address two issues. Firstly, during the course of optimization, it can occur that the solution of Theorem 4.1 does not compute non-negative values k i for an i ∈ [1 : m ] . This means that the constraints are not achieved tightly in the final solution of (3). Therefore, we are able to achieve the optimal solution with the constraint δ ′ i < δ i . Now, if we can assure that the constraint tuples are still inner points of F when we substitute δ i by δ ′ i , then Theorem 4.1 shows that (8) is still an optimal solution to (3).

Secondly, for tackling various objectives that are defined in Section 3, we usually need to upper- and lower-bound a performance measure by δ and -δ . However, since both bounds cannot hold tightly and simultaneously unless the tolerance is δ = 0 , then we can use only one of the constraints in turn and apply the result of Theorem 4.2 and check whether the constraint is active in the final solution. In the next section, we design an algorithm based on these results and show its generalization to the unseen data.

5 Empirical d -GNP and its Statistical Generalization

In previous sections, we obtained the optimal solution to the constrained optimization problem (3) using d -GNP. Based on this optimal solution, we can design a plug-in method (see Algorithm 1 in Appendix F) to solve the constrained learning problem using empirical data. This algorithm varies from many Lagrangian-based algorithms for solving constrained learning problem (e.g., Primal-Dual method [10]) in which the optimal predictor parameter and constraint penalties are dependent to each other, and therefore we should learn them iterativaly. However, as we saw in Theorem 5.1 (respectively in Algorithm 1), the solution of d -GNP is a mere thresholding on the corresponding embedding functions, where the threshold is obtained in a post-hoc manner and from validation dataset. Therefore, although Lagrangian-based algorithms can lead to oscillations or converge with a large computational cost, the d -GNP can potentially reduce such complexity costs and improve convergence conditions. To show such convergence, we bound the generalization error of the objective and constraints based on this solution. These results are extensions of the generalization results for Neyman-Pearson [1, 71] and further hold when multiple constraints should be controlled at once. The first result is the following theorem that shows if the solution to our plug-in method meets constraints of the optimization problem on training data, this generalizes to the unseen data.

Theorem 5.1 (Generalization of the Constraints) . For the approximation of the Neyman-Pearson solution ˆ f ˆ k, ˆ p ( x ) in Algorithm 1 such that E S n [ ⟨ ˆ f ˆ k, ˆ p ( x ) , ˆ ψ i ( x ) ⟩ ] ≤ δ i for i ∈ [1 : m ] , if we assume that embedding functions are bounded, then for d n ( ϵ ) ≃ O ( √ log n + √ log 1 /ϵ √ n ) and S n ∼ µ we have E µ [ ⟨ ˆ f ˆ k, ˆ p ( x ) , ψ i ( x ) ⟩ ] ≤ δ i + d n ( ϵ m ) for all i ∈ [1 : m ] and with probability at least 1 -ϵ .

In the above theorem, we show that the optimal empirical solution for the constraint, probably and approximately satisfies the constraint on true distribution. Therefore, if we assume that we have an approximation ˆ ψ i ( x ) in hand where ∥ ˆ ψ i ( x ) -ψ i ( x ) ∥ ∞ ≤ ϵ ′ with high probability, this theorem together with Hölder's inequality shows that we need to assure E S n [ ⟨ ˆ f ˆ k, ˆ p ( x ) , ˆ ψ i ( x ) ⟩ ] ≤ δ -d n ( ϵ m ) -ϵ ′ to achieve the corresponding generalization with high probability.

0

.

60

0

.

62

0

.

64

0

.

66

0

.

68

0

.

70

0

.

72

0

.

74

Test Accuracy

0

.

05

0

.

10

0

.

15

0

.

20

0

.

25

0

.

30

0

.

35

Demographic Parity

d-GNP

0

.

025

0

.

050

0

.

075

0

.

100

0

.

125

0

.

150

0

.

175

0

.

200

Constraint Tolerance

0

.

866

0

.

868

0

.

870

0

.

872

Test Accuracy

0

.

025

0

.

050

0

.

075

0

.

100

0

.

125

0

.

150

0

.

175

0

.

200

Constraint Tolerance

0

.

000

0

.

025

0

.

050

0

.

075

0

.

100

0

.

125

0

.

150

0

.

175

0

.

200

Constraint Violation

TPR

TNR

Next, we ask whether the objectives of the empirical optimal solution and the true optimal solution are close. We answer to this question positively in the following theorem. First, however, let us define the notions of ( γ , ∆ )-sensitivity condition as the following. This is an extension to detection condition in [71] and assumes that changing the parameter in predictor leads to a detectable change in constraints.

Definition 5.2. For an embedding function ψ 1 , and a distribution µ X on X , we refer to a function r k ( x ) as a prediction with ( γ , ∆ )-sensitivity around k , if there exists C ∈ R + such that for all δ ∈ (0 , ∆] we have

Now, we express the following generalization theorem for predictors that address the above conditions:

Theorem 5.3 (Generalization of Objective) . Assume that ( δ -ϵ l , δ + ϵ u ) is a subset of of all achievable constraints E [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] , and that ∥ ψ i ( x ) ∥ ∞ ≤ 1 for i = 1 , 2 . Further, let the size n of validation data be large enough such that d n ( δ/ 3) ≤ ϵ l 2 . Now, if the optimal predictor f ∗ k, 0 ( x ) is ( γ , ∆ )-sensitive around optimal k ∗ for ∆ ≃ Ω ( d 1 /γ n ( δ/ 3) , δ 1 / 2 γ 0 , δ 1 / 2 γ 1 ) and γ ≤ 1 , then for n ≥ 16 ϵ 2 l log 3 δ , and with probability at least 1 -δ , the optimal empirical classifier, as of Algorithm 1 has an objective that is at most O ( d 1 /γ n ( δ/ 3) , δ 1 /γ 0 , δ 1 / 2 0 , δ 1 / 2 1 , C -1 /γ , C -1 / 2 ) -far from the true optimal objective.

Now that we have proven generalization of our post-processing method, we should briefly compare this to other possible algorithms to learn an approximation of the optimal classifier and rejection function pair. A possible method is to find the appropriate 'defer' or 'no defer' value for each instance in the training dataset, and for a given set of constraints. Although these types of in-processing algorithms can perform computationally efficient (e.g., O ( n log n ) complexity for 1 n -suboptimal solution for human intervention budget as shown in Theorem G.1), they do not necessarily generalize to unseen data. In particular, we can show that for all algorithms that estimate deferral labels from empirical data, there exist two underlying distributions on the data on which the algorithm results in similar deferral labels, while the optimal rejection functions for these two distributions are not interchangeable. This argument is further formalized in the following proposition:

̸

̸

Proposition 5.4 (Impossibility of generalization of deferral labels) . For every deterministic deferral rule ˆ r for empirical distributions and based on the two losses 1 m = y and 1 h ( x ) = y , there exist two probability measures µ 1 and µ 2 on X × Y × M such that the corresponding (ˆ r, X ) for both measures is distributed equally. However, the optimal deferral r ∗ µ 1 and r ∗ µ 2 for these measures are not interchangeable, that is L µ i def ( h, r ∗ µ i ) ≤ 1 3 while L µ i def ( h, r ∗ µ j ) = 2 3 for i = 1 , 2 and j = i .

̸

In a nutshell, this proposition implies that, every algorithm that reduces the two-bit data of human accuracy and AI accuracy for an input into a single-bit data of 'defer' or 'no defer' looses the information that is important for obtaining the optimal rejection function that generalizes to the unseen data. This is a drawback of in-processing algorithms that are used in multi-objective L2D problems. We refer the reader to Appendix M for more details and proof of aforementioned proposition.

6 Experiments

COMPAS dataset. We implemented 4 Algorithm 1, first for COMPAS dataset [27] in which the recidivism rate of 7214 criminal defendants is predicted. The human assessment is done in this

dataset on 1000 cases by giving humans a description of the case and asking them whether the defendant would recidivate within two years of their most recent crime. 5 The demographic parity is assessed for two racial groups of white and non-white defendants. Figure 2 shows the average performance of d -GNP over 10 random seeds compared to two baselines: (1) Madras et al. [44] in which a demographic parity regularizer is added to the surrogate loss, and over a variation of 100 regularizer coefficient, and (2) Mozannar et al. [50] in which after training the classifier and rejector pair, we shift the corresponding scores to find a new thresholding rule. All scores, classifiers, and rejection functions are trained on a 1 -layer feed-forward neural network. The figure shows that achieving better fairness criteria is possible using d -GNP, while this might not lead to better accuracy when the constraint violation is not of interest.

Hatespeech dataset. The next experiment is on flagging offensive tweets in Hatespeech dataset [22]. This dataset contains 24,802 tweets that are labeled by at least three crowd workers as hate speech, offensive but not hate speech, or neither hate speech nor offensive. We used a pre-trained model [5] to detect whether the tweet contains an African-American dialect. Next, we used d -GNP method to control the demographic disparity of predicting a tweet hate speech or offensive bounded by δ HS = 0 . 1 and δ O = 0 . 01 . In the result of this experiment that is displayed in Figure 3 we can observe the following points: (i) in test-time the resulting demographic disparity for both classes are bounded as expected, (ii) the accuracy of d -GNP method is bounded by the vanilla deferral method, while stricter constraint control (in here offensive prediction parity) keeps the accuracy lower, and (iii) interestingly, the performance of d -GNP for controlled offensive prediction parity copies that of human. Therefore, a good strategy for obtaining such constrained learn-to-defer system seems to be to defer the offensive tweet prediction to human, when the tweet contains African-American dialect, and otherwise either bias the classifier scores or use a mixture of human and classifier involvement to achieve the final controlled disparity.

ACS dataset. We further tested our method on folktables dataset [25] that contains an income prediction task based on 1.6M rows of American Community Survey data. Since we had no access to human expert data, we simulated a human expert that has different accuracy on two racial groups of white and non-white individuals (85% and 60%, respectively). We considered the L2D problem with bounded equalized odds violation. Figure 2 shows our method's accuracy and constraint violation, coupled with a confidence bound that is obtained using ten iterations of bootstrapping. This figure shows that violation bounds are accurately met for the test data, and the performance increases when these bounds are loosened.

7 Conclusion

The d -GNP is a general framework that obtains the optimal solution to various constrained learning problems, including but not limited to multi-objective L2D problems. Using this post-processing framework, we can first estimate the scores related to our problem and then find a linear rule of these scores by fine-tuning for specific violation tolerances. This method reduces the computational complexity of in-processing methods while guaranteeing achieving a near-optimal solution in a large data regime.

8 Acknowledgment

M.A. Charusaie thanks the International Max Planck Research School for Intelligent Systems (IMPRSIS) and Tübingen AI Center for the support and funding of this project. He is further grateful to Matthäus Kleindessner for his significant intellectual contributions to the first draft of this paper. The idea of obtaining an extension to the Neyman-Pearson lemma emerged from discussions with André Cruz and Florian Dorner. The very initial draft of this paper was written during an hours-long train delay in Germany, and thus, M.A. Charusaie is thankful to Deutsche Bahn in that regard.

References

Jean-Yves Audibert and Alexandre B Tsybakov. Fast learning rates for plug-in classifiers. 2007.

Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, and Daniel S Weld. Is the most accurate ai the best teammate? optimizing ai for teamwork. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 11405-11414, 2021.

Peter L Bartlett and Marten H Wegkamp. Classification with a reject option using a hinge loss. Journal of Machine Learning Research , 9(8), 2008.

Emma Beede, Elizabeth Baylor, Fred Hersch, Anna Iurchenko, Lauren Wilcox, Paisan Ruamviboonsuk, and Laura M Vardoulakis. A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. In Proceedings of the 2020 CHI conference on human factors in computing systems , pages 1-12, 2020.

Su Lin Blodgett, Lisa Green, and Brendan O'Connor. Demographic dialectal variation in social media: A case study of african-american english. arXiv preprint arXiv:1608.08868 , 2016.

Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, and Manfred K Warmuth. Learnability and the vapnik-chervonenkis dimension. Journal of the ACM (JACM) , 36(4):929-965, 1989.

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning , 3(1):1-122, 2011.

Yuzhou Cao, Tianchi Cai, Lei Feng, Lihong Gu, GU Jinjie, Bo An, Gang Niu, and Masashi Sugiyama. Generalizing consistent multi-class classification with rejection to be compatible with arbitrary losses. In Advances in Neural Information Processing Systems .

Yuzhou Cao, Hussein Mozannar, Lei Feng, Hongxin Wei, and Bo An. In defense of softmax parametrization for calibrated and consistent learning to defer. Advances in Neural Information Processing Systems , 36, 2024.

Luiz FO Chamon, Santiago Paternain, Miguel Calvo-Fullana, and Alejandro Ribeiro. Constrained learning with non-convex losses. IEEE Transactions on Information Theory , 69(3):17391760, 2022.

Nontawat Charoenphakdee, Zhenghang Cui, Yivan Zhang, and Masashi Sugiyama. Classification with rejection based on cost-sensitive classification. In International Conference on Machine Learning , pages 1507-1517. PMLR, 2021.

Mohammad-Amin Charusaie, Hussein Mozannar, David Sontag, and Samira Samadi. Sample efficient learning of predictors that complement humans. In Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 2972-3005. PMLR, 2022.

Richard J Chen, Judy J Wang, Drew FK Williamson, Tiffany Y Chen, Jana Lipkova, Ming Y Lu, Sharifa Sahai, and Faisal Mahmood. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nature biomedical engineering , 7(6):719-742, 2023.

Wenlong Chen, Yegor Klochkov, and Yang Liu. Post-hoc bias scoring is optimal for fair classification. arXiv preprint arXiv:2310.05725 , 2023.

Xin Cheng, Yuzhou Cao, Haobo Wang, Hongxin Wei, Bo An, and Lei Feng. Regression with cost-based rejection. Advances in Neural Information Processing Systems , 36, 2024.

C Chow. On optimum recognition error and reject tradeoff. IEEE Transactions on information theory , 16(1):41-46, 1970.

Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil. Leveraging labeled and unlabeled data for consistent fair binary classification. Advances in Neural Information Processing Systems , 32, 2019.

Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining , pages 797-806, 2017.

Corinna Cortes, Giulia DeSalvo, and Mehryar Mohri. Learning with rejection. In Algorithmic Learning Theory: 27th International Conference , pages 67-82. Springer, 2016.

André F Cruz and Moritz Hardt. Unprocessing seven years of algorithmic fairness. arXiv preprint arXiv:2306.07261 , 2023.

George B Dantzig and Abraham Wald. On the fundamental lemma of neyman and pearson. The Annals of Mathematical Statistics , 22(1):87-93, 1951.

Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media , volume 11, pages 512-515, 2017.

Abir De, Paramita Koley, Niloy Ganguly, and Manuel Gomez-Rodriguez. Regression under human assistance. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 2611-2620, 2020.

Abir De, Nastaran Okati, Ali Zarezade, and Manuel Gomez Rodriguez. Classification under human assistance. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 5905-5913, 2021.

Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances in neural information processing systems , 34:6478-6490, 2021.

Kate Donahue, Alexandra Chouldechova, and Krishnaram Kenthapadi. Human-algorithm collaboration: Achieving complementarity and avoiding unfairness. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , pages 1639-1656, 2022.

Julia Dressel and Hany Farid. The accuracy, fairness, and limits of predicting recidivism. Science advances , 4(1):eaao5580, 2018.

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference , pages 214-226, 2012.

Cynthia Dwork and Christina Ilvento. Fairness under composition. arXiv preprint arXiv:1806.06122 , 2018.

Ran El-Yaniv et al. On the foundations of noise-free selective classification. Journal of Machine Learning Research , 11(5), 2010.

David Heaver Fremlin. Measure theory , volume 4. Torres Fremlin, 2000.

Aditya Gangrade, Anil Kag, and Venkatesh Saligrama. Selective classification via one-sided prediction. In International Conference on Artificial Intelligence and Statistics , pages 21792187. PMLR, 2021.

Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. Advances in neural information processing systems , 30, 2017.

Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems , 29, 2016.

Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Ohno-Machado. Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Association , 19(2):263-274, 2012.

Erik Jones, Shiori Sagawa, Pang Wei Koh, Ananya Kumar, and Percy Liang. Selective classification can magnify disparities across groups. arXiv preprint arXiv:2010.14134 , 2020.

Ece Kamar, Severin Hacker, and Eric Horvitz. Combining human and machine intelligence in large-scale crowdsourcing. In AAMAS , volume 12, pages 467-474, 2012.

Achim Klenke. Probability theory: a comprehensive course . Springer Science & Business Media, 2013.

Benjamin Kompa, Jasper Snoek, and Andrew L Beam. Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digital Medicine , 4(1):1-6, 2021.

Thomas Landgrebe and R Duin. On neyman-pearson optimisation for multiclass classifiers. In Proceedings 16th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA , pages 165-170, 2005.

Joshua K Lee, Yuheng Bu, Deepta Rajan, Prasanna Sattigeri, Rameswar Panda, Subhro Das, and Gregory W Wornell. Fair selective classification via sufficiency. In International conference on machine learning , pages 6076-6086. PMLR, 2021.

Erich Leo Lehmann, Joseph P Romano, and George Casella. Testing statistical hypotheses , volume 3. Springer, 1986.

Shuqi Liu, Yuzhou Cao, Qiaozhen Zhang, Lei Feng, and Bo An. Mitigating underfitting in learning to defer with consistent losses. In International Conference on Artificial Intelligence and Statistics , pages 4816-4824. PMLR, 2024.

David Madras, Toni Pitassi, and Richard Zemel. Predict responsibly: improving fairness and accuracy by learning to defer. Advances in Neural Information Processing Systems , 31, 2018.

Anqi Mao, Mehryar Mohri, and Yutao Zhong. Predictor-rejector multi-class abstention: Theoretical analysis and algorithms. In International Conference on Algorithmic Learning Theory , pages 822-867. PMLR, 2024.

Charles E Metz. Basic principles of roc analysis. In Seminars in nuclear medicine , volume 8, pages 283-298. Elsevier, 1978.

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning . MIT press, 2018.

Hussein Mozannar, Hunter Lang, Dennis Wei, Prasanna Sattigeri, Subhro Das, and David Sontag. Who should predict? exact algorithms for learning to defer to humans. In International conference on artificial intelligence and statistics , pages 10520-10545. PMLR, 2023.

Hussein Mozannar, Hunter Lang, Dennis Wei, Prasanna Sattigeri, Subhro Das, and David Sontag. Who should predict? exact algorithms for learning to defer to humans. arXiv preprint arXiv:2301.06197 , 2023.

Hussein Mozannar and David Sontag. Consistent estimators for learning to defer to an expert. In International Conference on Machine Learning , pages 7076-7087. PMLR, 2020.

Harikrishna Narasimhan, Wittawat Jitkrittum, Aditya K Menon, Ankit Rawat, and Sanjiv Kumar. Post-hoc estimators for learning to defer to an expert. Advances in Neural Information Processing Systems , 35:29292-29304, 2022.

Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum, Neha Gupta, and Sanjiv Kumar. Learning to reject meets long-tail learning. In The Twelfth International Conference on Learning Representations .

Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum, and Sanjiv Kumar. Plugin estimators for selective classification with out-of-distribution detection. arXiv preprint arXiv:2301.12386 , 2023.

Harikrishna Narasimhan, Harish G Ramaswamy, Shiv Kumar Tavker, Drona Khurana, Praneeth Netrapalli, and Shivani Agarwal. Consistent multiclass algorithms for complex metrics and constraints. arXiv preprint arXiv:2210.09695 , 2022.

Jerzy Neyman and Egon Sharpe Pearson. Ix. on the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character , 231(694-706):289-337, 1933.

Jerzy Neyman and Egon Sharpe Pearson. Contributions to the theory of testing statistical hypotheses. Statistical research memoirs , 1936.

Nastaran Okati, Abir De, and Manuel Rodriguez. Differentiable learning under triage. Advances in Neural Information Processing Systems , 34:9140-9151, 2021.

Christos H Papadimitriou and Kenneth Steiglitz. Combinatorial optimization: algorithms and complexity . Courier Corporation, 1998.

Charles Chapman Pugh and CC Pugh. Real mathematical analysis , volume 2011. Springer, 2002.

Maithra Raghu, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer, and Sendhil Mullainathan. The algorithmic automation problem: Prediction, triage, and human effort. arXiv preprint arXiv:1903.12220 , 2019.

Philippe Rigollet and Xin Tong. Neyman-pearson classification, convexity and stochastic constraints. Journal of machine learning research , 2011.

Salvatore Ruggieri, Jose M Alvarez, Andrea Pugnana, Franco Turini, et al. Can we trust fair-ai? In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 15421-15430, 2023.

Stephen-John Sammut, Mireia Crispin-Ortuzar, Suet-Feung Chin, Elena Provenzano, Helen A Bardwell, Wenxin Ma, Wei Cope, Ali Dariush, Sarah-Jane Dawson, Jean E Abraham, et al. Multi-omic machine learning predictor of breast cancer therapy response. Nature , 601(7894):623-629, 2022.

Clayton Scott. Performance measures for neyman-pearson classification. IEEE Transactions on Information Theory , 53(8):2852-2863, 2007.

Clayton Scott and Robert Nowak. A neyman-pearson approach to statistical learning. IEEE Transactions on Information Theory , 51(11):3806-3819, 2005.

Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms . Cambridge University Press, 2014.

Dharmesh Tailor, Aditya Patra, Rajeev Verma, Putra Manggala, and Eric Nalisnick. Learning to defer to a population: A meta-learning approach. In International Conference on Artificial Intelligence and Statistics , pages 3475-3483. PMLR, 2024.

Sarah Tan, Julius Adebayo, Kori Inkpen, and Ece Kamar. Investigating human+ machine complementarity for recidivism predictions. arXiv preprint arXiv:1808.09123 , 2018.

Ye Tian and Yang Feng. Neyman-pearson multi-class classification via cost-sensitive learning. arXiv preprint arXiv:2111.04597 , 2021.

Alexandru Tifrea, Preethi Lahoti, Ben Packer, Yoni Halpern, Ahmad Beirami, and Flavien Prost. Frappé: A group fairness framework for post-processing everything. In Forty-first International Conference on Machine Learning .

Xin Tong. A plug-in approach to neyman-pearson classification. The Journal of Machine Learning Research , 14(1):3011-3040, 2013.

C Vermeulen, M Pagès-Gallego, L Kester, MEG Kranendonk, P Wesseling, N Verburg, P de Witt Hamer, EJ Kooi, L Dankmeijer, J van der Lugt, et al. Ultra-fast deep-learned cns tumour classification during surgery. Nature , 622(7984):842-849, 2023.

Ruicheng Xian, Lang Yin, and Han Zhao. Fair and optimal classification via post-processing. In International Conference on Machine Learning , pages 37977-38012. PMLR, 2023.

Tongxin Yin, Jean-François Ton, Ruocheng Guo, Yuanshun Yao, Mingyan Liu, and Yang Liu. Fair classifiers that abstain without harm. arXiv preprint arXiv:2310.06205 , 2023.

Xianli Zeng, Guang Cheng, and Edgar Dobriban. Bayes-optimal fair classification with linear disparity constraints via pre-, in-, and post-processing. arXiv preprint arXiv:2402.02817 , 2024.

Xianli Zeng, Edgar Dobriban, and Guang Cheng. Bayes-optimal classifiers under group fairness. arXiv preprint arXiv:2202.09724 , 2022.

Content of Appendices

A Lack of Compositionality of Fairness Criteria

Here, we show an example of lack of compositionality of fairness criteria for learn-to-defer problems. This falls in line with [29], where the authors studied the effect of the operators such as 'OR' or 'AND'. Here, we show that a similar non-compositionality holds for the operator 'DEFER'. The following example is found based on the insight that a fair predictor is fair over all the space X , and if it could take a decision over only a subset of X it will not necessarily be a fair predictor. This can be seen as a particular application of Yule's effect [62] which explains that vanishing correlation in a mixture of distributions does not necessarily concludes vanishing correlation on each of such distributions.

Let us assume that the space X contains only four points x 1 , x 2 , x 3 , and x 4 , and that the input takes these values with probability Pr( X = x 1 ) = Pr( X = x 2 ) = Pr( X = x 3 ) = Pr( X = x 4 ) = 1 4 . The first two points x 1 , x 2 are corresponded to the demographic group A = 0 and the last two points are corresponded to the demographic group A = 1 . Further, assume that the conditional target probability is Pr( Y = 1 | x 1 ) = Pr( Y = 1 | x 2 ) = Pr( Y = 1 | x 3 ) = Pr( Y = 1 | x 4 ) = 1 . Moreover, we consider the equality of opportunity as the measure of fairness. Now, assume that the classifier h ( · ) : X → { 0 , 1 } is taking values h ( x 1 ) = 1 , h ( x 2 ) = 0 , h ( x 3 ) = 1 , and h ( x 4 ) = 0 and the human decision maker predicts M = 0 conditioned on x 1 , M = 1 conditioned on x 2 , and M = 1 conditioned on x 3 , and M = 0 conditioned on x 4 . Therefore, both classifier and human expert have accuracy of 1 2 on the data.

Following the above assumptions, we can find the fairness measure for classifier as

which means that the classifier is fully fair. We can derive a similar result for the human expert, i.e.,

Now that we established a fair classifier and a fair expert, we take the step to find an optimal deferral solution, i.e., a deferral system that minimizes the overall loss. We can observe that for x 1 the classifier is accurate, while for x 2 the human expert is accurate. Furthermore, for x 3 and x 4 they both are equally inaccurate. Therefore, an optimal solution is not to defer for x 1 , and defer for x 2 , and take an arbitrary decision for x 3 and x 4 . Now, if we find the fairness measure of the resulting deferral predictor, we have

or equivalently the resulting predictor is unfair for the demographic group A = 1 . This means the 'DEFER' composition of the predictors does not preserve fairness. One can further easily show that no deferral system from the above classifier and human expert that has the accuracy better than 1 2 is fair.

B Extended Related Works

The deferral problem has been studied under a variety of conditions. Rejection learning [19, 3, 11, 15] or selective classification [30, 33, 32], assumes that a fixed cost is incurred to the overall loss, when ML decides not to make a prediction on an input. The first Bayes optimal rule for rejection learning was derived in [16]. Assuming that the accuracy of human, and consequently the cost of deferring to

the human, can vary for different inputs, [50] obtained the Bayes optimal deferral rule. The deferral problem is further studied assuming that the number of available instances for deferral are bounded and a near-optimal classifier and deferral rule is required as a solution of empirical risk minimization [23, 24]. Most recently, the implementation of deferral rules using neural networks and surrogate losses is studied for binary and multi-class classification [8, 50, 12, 51, 9, 43, 48, 45]. A possible shift in human expert for L2D methods recently studied in [67]. The problem multi-objective L2D and rejection learning is mainly studied in an in-processing approach. A few instances of tackling such problems can be found in [57, 52, 53] and [74, 41] for L2D and rejection learning, respectively. Neyman-Pearson's fundamental lemma is introduced in [55] originally for binary hypothesis testing and later was generalized in [56] to give a close-form formulation for a variety of binary constrained optimization problems. Later, [21] found conditions for which Neyman and Pearson solution exists and is unique. The generalization error of the empirical solution to Neyman-Pearson problem is studied in two lines of works: (i) the generalization of direct (in-processing) solutions to the optimization problem [65, 64, 61], and (ii) the generalization of plug-in methods [71] that first approximate the score functions and then use Neyman-Pearson lemma to approximate the predictor. The generalization of Neyman-Pearson lemma to multiclass setting is first empirically studied in [40] and under strong duality assumption is proved in [69]. Our lemma d -GNP extends these works in order to (i) be able to control a general set of constraints instead of TypeK errors, and (ii) be valid in absence of strong duality assumption. Further, the idea of using Neyman-Pearson lemma for controlling fairness criteria originally dates back to [76] (later as [75]). More recently, a similar post-processing method is introduced in [14] using cost-sensitive learning and strong duality technique. Although these works cover binary classification problem, in this paper we focus on solving multi-class classification problem, and particularly in a deferral system.

Moreover, his work differs from multi-class classification with complex performance metrics [54] in the sense that they consider constraints that are non-linear functions of confusion matrix, while ignoring the dependence on input x . In our setting, the constraints are linear in terms of confusion matrix when conditioned on the input, but the linear coefficients vary with the input.

Finally, the work [70] has recently studied an extension of post-processing method to other constrained learning problems. The difference of that work with our method is threefold: (i) while we prove that the optimal post-processing method is a linear combination of scores, they have no such claim, (ii) we have no assumption on the format of the loss function, while they assume a particular set of strictly convex loss functions, (iii) we have no bound on our hypothesis class while they assume the representation of the predictor with a multidimensional vector and a fixed dimension.

C Rephrasing (2) into Linear Functional Programming

Here, we first characterize functions that are outcome-dependent. To that end, we define ı ( x ) as

This function can retrieve the value of r ( x ) and can retrieve the value of h ( x ) only if r ( x ) = 0 . In fact, we can obtain r ( x ) = ( ı ( x ) ) ( L +1) and h ( x ) = i if r ( x ) = 0 and ( ı ( x ) ) ( i ) = 1 . Therefore, for a function Ψ( x, h ( x ) , r ( x )) = E Y,M | X = x [ Ψ( x, Y, M, h ( x ) , r ( x )) ] and ℓ def ( x, h ( x ) , r ( x )) = E Y,M | X = x [ ℓ def ( x, Y, M, h ( x ) , r ( x )) ] to be outcome dependent, it must only be a function of x and ı ( x ) . In fact, we must have

for a choice of Ψ ′ and ℓ ′ def , where ℓ def ( x, h ( x ) , r ( x )) = E Y,M | X = x [ ℓ def ( x, Y, M, h ( x ) , r ( x )) ] . Now, we can check that ı ( x ) can take L +1 different values, in each of which one of its components takes the value 1 and others take the value 0 . Therefore, by conditioning on each of these L + 1 values we have

and

where ψ i ( x ) is defined as

Similarly, we can show that where ℓ ⃗ def ( x ) is defined as

Next, we know that due to the randomization of A , the vector ı ( x ) can take various values on each instance x . This, however, is not the case for ψ i ( x ) and ℓ ⃗ def ( x ) , since they are defined independent of r ( x ) and h ( x ) . Therefore, the average of constraints and loss can be rewritten as

and

where f ( x ) is defined as

Therefore, the optimization problem in (2) is effectively reduced to the linear programming problem in (3). Moreover, if f ∗ ( x ) is the solution to that linear program, then the corresponding r ( x ) should be distributed as Pr( r ( x ) = 1) = ( f ∗ ( x ) ) ( L + 1) , where h ( x ) should be distributed

independence of h ( x ) and r ( x ) comes with no loss of generality, since the value of h ( x ) does not variate the loss or constraints in the system when we have r ( x ) = 1 .

D Derivation of Embedding Functions

In this appendix we derive the embedding functions in Table 1 that are corresponded to the constraints of choice, as named in Section 3. The trick that we use for all these constraints is that we first rewrite the constraint in terms of the expected value of a function over the randomness of the algorithm A and the input variable X , and then we use (17) to transform that function into the embedding function.

Overall Loss : To find the embedding function that is corresponded to the overall loss of the system, we should first note that by loss we mean the probability of incorrectness of ˆ Y . Therefore, the corresponding ℓ def ( x, h ( x ) , r ( x )) in this case, as defined in (1) is obtained as

̸

̸

Therefore, using (19) we find ℓ ⃗ def as

̸

̸

̸

Expert intervention budget : In this case, similar to the case before, we first derive Ψ( x, h ( x ) , r ( x )) . To that end, we first note that the expert intervention constraint in Section 3 is equivalent with

which in turn suggests that

Next, we find ψ ( x ) using (17), as

̸

OODDetection : To obtain the corresponding embedding function to the OOD detection constraint in Section 3, we can rewrite Pr out ( r ( X ) = 1 ) as

where the last equation holds when X and X out are absolutely continuous distributions, and therefore have probability density functions. A similar assumption is made by [53]. This results in Ψ( x, h ( x ) , r ( x )) being obtained as

Therefore, we conclude that the embedding function can be calculated using (17) as

In the simple case that f out X ( x ) = f in X ( x ) I f in X ( x ) ≤ ϵ ∫ f in X ( x ) I f in X ( x ) ≤ ϵ d x , the embedding function is equal to

Long-Tail Classification : This methodology aims to minimize the balanced loss

̸

However, as mentioned in [52], this optimization problem can be rewritten as

̸

Therefore, the objective can rewritten as

̸

which together with (17) shows that

̸

̸

The reason that we use negative sign is because in the definition of (3) we aim to maximize the objective.

Similarly, we can rewrite the objectives as

Therefore, using (17) we can obtain ψ i ( x ) as

Typek Error Bound : We first rewrite Typek constraint in 3 as

̸

̸

̸

̸

where ( a ) is followed by chain rule.

̸

Next, we condition Pr( ˆ Y = k | Y = k, X = x ) on r ( X ) being 1 and 0 , which concludes that

̸

Therefore, using (24) we conclude that

̸

̸

which together with (17) shows that the embedding function is obtained as

̸

Note that here we used the assumption that ( Y, M ) and A are independent for each choice of X , i.e., the value noise that is introduced in A for each X = x is generated independent of the value of Y and M , which is the true assumption, since the algorithm only has access to X and not true label or the human label.

Demographic Parity : We know that the demographic parity constraint in Section 3 can be written as

Here, we find the corresponding embedding function ψ ( x ) for the upper-bound in the above inequality. For the lower-bound, we can use -ψ ( x ) and follow the steps that are proposed in the main text of the manuscript.

To find the embedding function that corresponds to the upper-bound of (25), we first rewrite Pr( ˆ Y = 1 | A = 0) -Pr( ˆ Y = 1 | A = 1) as

Now, similar to what we did in previous section, we condition Pr( ˆ Y = 1 , A = a ) for a ∈ { 0 , 1 } on the value of h ( x ) and r ( x ) , and we conclude

Here, we used the assumption of independence of X and ( M,Y ) given a choice of X .

As a result of (26), (27), and (17) we can find the embedding function as

̸

̸

̸

̸

̸

̸

̸

̸

̸

(In-)Equality of Opportunity : Similar to the previous items, we rewrite equality of opportunity constraint in Section 3 as

Again, we only consider the upper-bound and rewrite Pr( ˆ Y = 1 | Y = 1 , A = 1) -Pr( ˆ Y = 1 | Y = 1 , A = 0) as

Next, by conditioning on r ( X ) = 1 and r ( X ) = 0 , we rewrite Pr( ˆ Y = 1 , Y = 1 , A = a ) for a ∈ { 0 , 1 } as

where the last identity is followed by the assumption of independence of A and ( Y, M, A ) given an instance X = x .

As a result of (28), (29), and (17) we can obtain the embedding function as

(In-)Equality of Odds : This induces the same constraint as that of equality of opportunity, and further induces an extra constraint that is in nature similar to equality of opportunity with the difference that it uses Y = 0 instead of Y = 1 . Therefore, we have two embedding functions, one is similar to that of equality of opportunity as

and another similar to that with changing Y = 1 into Y = 0 , and therefore as

E Limitations of Cost-Sentitive Methods

A variety of works have tackled constrained classification problems using cost-sensitive modeling [42, 17, 57]. In other words, they use the expected loss that is penalized with the constraints and solve that for certain coefficients for those constraints (a.k.a., they form Lagrangian from that problem). In the next step, they optimize the coefficients and obtain the optimal predictor. The issue that we discuss further in the following we concern is that during this process, the optimal predictor is achieved only

when the corresponding cost-sensitive Lagrangian has a single saddle point in terms of coefficients and predictors. Such assumption, unless by analyzing the Lagrangian closely, is hard to be validated. However, our results in this paper have no such assumption, and instead use statistical hypothesis testing methods to show their optimality.

To further clarify the issue with such methodology, we bring an example of L2D problem when human intervention budget is controlled. Suppose that the features in X are distributed with an atomless probability measure µ X (e.g., normal or uniform distribution). 6 Further, assume that the human has perfect information of the label, i.e. Y = M , while the input features have no information of the label, i.e., Pr( Y = 1 | X = x ) = 1 / 2 for all x ∈ X . Moreover, let the classifier and the human induce the 0 -1 loss function. In this case, we can see that the optimal classifier is the maximizer of the scores (see the early discussion of Section H), which in this case, since there is no clear maximizer, without loss of generality can be set to h ( x ) ≡ 1 .

For such assumptions, if we write the Lagrangian in form of

then strong duality shows that

or to put it informally, the objective is invariant under the interchange of minimum and maximum over Lagrange multipliers and the variable of interest. However, this does not prove the interchangeability of the saddle points in these settings, i.e., we cannot guarantee argmin r ∈ [0 , 1] L ( λ ∗ , r ) = f ∗ , where λ ∗ ∈ argmax λ min r ∈ [0 , 1] L ( λ, r ) , and f ∗ ∈ argmin r ∈ [0 , 1] max λ L ( λ, r ) . In fact, this guarantee holds only in particular examples, e.g., when L ( λ ∗ r , r ) is strictly convex [7, page 8].

In fact, if we optimize r for all λ as in RHS of (30), we can show that r λ ( x ) = { 1 λ < 1 2 0 otherwise

.

Therefore, λ ∗ can be obtained as λ ∗ = argmax λ ≥ 0 ( λ -1 / 2) --λb where ( x ) -:= min { x, 0 } . This can be rewritten as

Hence, the condition λ < 1 / 2 is never satisfied and we have r λ ∗ ( x ) = 0 , i.e., we should never defer. For the deferral rule r λ ∗ , the deferral loss (1) is

To show that r λ ∗ is not optimal, we provide random and deterministic deferral rules f ∗ and r ∗∗ that satisfy the constraint in (2), while having a smaller deferral loss:

⋄ Let f ∗ ( x ) = b , that is a random deferral rule that defers with probability b everywhere on X . Therefore, on average b proportion of inquiries are deferred and thus it satisfies the constraint in (2). The deferral loss for f ∗ ( x ) is equal to

⋄ The second example is a deterministic deferral rule. Since the probability measure on X is atomless, for all b ∈ [0 , 1] there exists a set A such that Pr( X ∈ A ) = b [31, Proposition 215D]. Hence, defining r ∗∗ ( x ) = 1 x ∈A the constraint in (2) is met. Similar to the last example L µ def ( h, r ∗∗ ) = 1 -b 2 < 1 2 .

The above two examples show that the deferral rule r λ ∗ is sub-optimal. The reason is that, for optimality of r λ ∗ we should make sure that L ( λ ∗ r , r ) has a single minimizer of r . However, in our example we had L ( 1 2 , r ) = -λb has infinite number of minimizers in terms of r ( x ) . Therefore, the equality of the solutions to minimax problem and maximin problem is not guaranteed.

F d -GNP Learning Algorithm

G On Failure of In-Processing Methods

One might think that the need of using post-processing methods does not necessarily appear in some examples of multi-objective L2D problem. As an instance, for the expert intervention budget we can rank samples based on the difference between machine and human loss and defer the top b -proportion of samples for which the machine loss is higher than the human one. This method is illustrated in Algorithm 2. Indeed, in the following we show that the sub-optimality of such deterministic deferral rule, compared to the optimal deferral rule diminishes as the size of training set increases.

̸

Theorem G.1 (Optimal Deferral for Empirical Distribution) . For a classifier h ( x ) and dataset D = { ( x i , y i , m i ) } n i =1 , where we assume x i = x j , i = j , the deterministic deferral rule as in Algorithm 2 is (i) the optimal deterministic deferral rule for the empirical distribution on D and bounded expert intervention budget, and (ii) at most 1 n -suboptimal (in terms of deferral loss) compared to the optimal random deferral rule for the empirical distribution on D .

̸

Next in the following, we show that such policy does not provide sufficient information to determine the optimal deferral rule for the true distribution. To that end, we first recall that in classification tasks, the optimal classifier typically thresholds an estimation of conditional probability of the label Y given X that is obtained using the available dataset. As a result, if we observe enough pairs of ( x i , y i ) , then

Algorithm 2 Deterministic Algorithm for Deferring Tasks to Human or AI for the Empirical Distribution and Expert Intervention Budget

Input : The dataset D , he human and classifier loss ℓ H and ℓ AI and available proportion b of instances to defer

Output : Labels of 'defer" or 'no defer" for each instance in D

1: procedure DEFERTASKS( D , ℓ , ℓ , b )

H AI 2: Make the set A = { ( x, y, m ) ∈ D : ℓ H ( y, m ) -ℓ AI ( y, h ( x )) ≤ 0 } 3: if | A | ≥ b |D| then 4: Defer all tasks in A to human 5: else 6: Defer the b |D| tasks with the lowest ℓ H ( x, y, m ) -ℓ AI ( x, y ) to human 7: end if

8: end procedure

we improve upon such estimation of conditional probability and increase the accuracy of the obtained classifier. However, we argue that this paradigm is inapplicable in the case of deferral rule as follows. Although the output ˆ r of Algorithm 2 for each feature x is a deterministic 0 or 1 label, it varies with the choice of the dataset D . Hence, if we draw datasets from a distribution µ , the outcome of ˆ r becomes probabilistic. In the following, we introduce two probability distributions µ 1 and µ 2 over ( X,Y,M ) such that for random draws of the dataset from µ i , the conditional probability of such optimal deferral label ˆ r given X is equal, yet the optimal deferral rule for the true distribution is different.

Although the following discussion bears some resemblance with the No-Free-Lunch theorem [e.g. 66], there exists the following difference between the two. The No-Free-Lunch theorem states that for each learning algorithm, there exists a data distribution on which the algorithm does not generalize well. However, in the following discussion, we assume that we can observe infinite number of datasets and indeed, we can find the underlying probability of the deferral labels. In fact, we show that the limiting factor for finding the optimal deferral for the true distribution is that we only use deferral labels and if we use values of both losses, we can accordingly find the optimal deferral rule as suggested in Theorem 4.1.

Assume that we have a dataset D = { ( x i , y i , m i ) } n i =1 that contains i.i.d. samples from the distribution µ XYM . Further, assume that we have no budget constraint, that is b = 1 in Algorithm 2. Therefore, the optimal randomized deferral rule over the empirical distribution is the solution of the problem

̸

̸

It is easy to see that the solution to this problem is given by ˆ r i = 0 if 1 h ( x i ) = y i < 1 m i = y i and ˆ r i = 1 if 1 h ( x i ) = y i > 1 m i = y i . As a result, the optimal deferral is obtained as

̸

̸

̸

Among all the possible policies, we can choose

̸

̸

.

Next, we assume that we have a classifier h and two probability distributions µ 1 and µ 2 over ( X,Y,M ) . For both distributions X is uniformly distributed over [0 , 1] , and we have µ 1 ( Y = M,h ( X ) = Y ) = 2 3 , µ 1 ( Y = M,h ( X ) = Y )= 1 3 and µ 2 ( Y = M,h ( X ) = Y ) = 2 3 , µ 2 ( Y = M,h ( X ) = Y )= 1 3 . We can see that although the observed ˆ r s are fixed for a given choice of D , since D is randomly drawn, ˆ r values are randomly distributed. Furthermore, the distribution of Pr(ˆ r | X ) is according to Bern ( 1 3 ) , since in both cases we have µ i ( Y = M,h ( X ) = Y ) = 1 3 . However, the optimal deferral rule for the first distribution is r ∗ 1 ( x ) = 1 for all x ∈ X , since we have L µ 1 def ( h, r ∗ 1 ) = 0 , while for the second case the optimal deferral rule is r ∗ 2 ( x ) = 0 for all x ∈ X because we have L µ 2 def ( h, r ∗ 2 ) = 1 3 . Furthermore, such deferral

̸

̸

̸

̸

̸

̸

and

̸

̸

rules are not interchangeable, because we have L µ 1 def ( h, r ∗ 2 ) = L µ 2 def ( h, r ∗ 1 ) = 2 3 . As a result, Pr(ˆ r | X does not provide sufficient information for obtaining optimal deferral rule on true distribution. For an arbitrary choice of deterministic deferral rule for empirical distribution, we state the following proposition as a proof of insufficiency of deferral labels for obtaining optimal deferral rule over the true distribution.

̸

̸

)

Proposition G.2 (Impossibility of generalization of deferral labels) . For every deterministic deferral rule ˆ r for empirical distributions and based on the two losses 1 m = y and 1 h ( x ) = y , there exist two probability measures µ 1 and µ 2 on X × Y × M such that the corresponding (ˆ r, X ) for both measures is distributed equally. However, the optimal deferral r ∗ µ 1 and r ∗ µ 2 for these measures are not interchangeable, that is L µ i def ( h, r ∗ µ i ) ≤ 1 3 while L µ i def ( h, r ∗ µ j ) = 2 3 for i = 1 , 2 and j = i .

̸

̸

Proof. As mentioned in (31), there are four possibilities of a deterministic deferral rule for empirical distribution based on the events h ( X ) = Y and M = Y . The reason is that

̸

̸

̸

the parameters a and b can take binary values. One of the cases in which a = b = 0 is analyzed previously in this section. We study the other cases as follows:

1. a = 1 , b = 0 : In this case we have

̸

.

If we define a measure µ 1 such that

̸

and a measure µ 2 such that

̸

then on one hand one can see that ˆ r is according to Bern ( 1 3 ) in both cases. On the other hand, because the probability of classifier accuracy is larger than human accuracy in µ 1 and is smaller than human accuracy in µ 2 , we have r ∗ µ 1 ( x ) = 0 while r ∗ µ 2 ( x ) = 1 . Therefore, we conclude that

while the losses with interchanging deferral policies are equal to

2. a = 0 , b = 1 : In this case, the deferral rule is as

̸

.

Next, if we set two probability measures µ 1 and µ 2 such that

̸

̸

and

̸

̸

then ˆ r is according to Bern ( 2 3 ) in both cases. However, r ∗ µ 1 = 1 while r ∗ µ 2 = 0 . Furthermore, the expected deferral losses are equal to

while after interchanging the deferral policies we have

3. a = 1 , b = 1 : In this case, the deferral rule is as

̸

.

Next, if we set two probability measures µ 1 and µ 2 such that

̸

̸

and

̸

̸

̸

then we can see that ˆ r has the distribution Bern ( 2 3 ) . However, one can find the optimal deferral policies for the true distributions are r ∗ µ 1 = 1 and r ∗ µ 2 = 0 . Furthermore, we have

and while

H Proof of Theorem 3.1

Let X = { x 1 , . . . , x n } and Y = { 1 , . . . , n } . We first show that obtaining the optimal classifier is of O ( n ) complexity, since in this case is equivalent to obtaining the Bayes optimal classifier in isolation. The reason is that, the unconstrained Bayes optimal classifier is a deterministic classifier that minimizes

for all x ∈ X . This is regardless of whether the deferral occurs or not. Therefore, this solution is further the solution to

for every rejection function r , including the optimal rejection function of the constrained optimization problem. In the particular case of expert intervention budget, the constraint is further independent of h and is only a function of r . Therefore, the unconstrained Bayes classifier is an optimal classifier for the constrained L2D problem with human intervention budget.

Next, we consider a specific case in which E µ Y | X [ ℓ AI ( Y, 1 , X ) | X = x ] > E µ Y | X [ ℓ AI ( Y, 0 , X ) | X = x ] for all x ∈ X , and therefore h ( x ) = 1 over all input space.

Further, we assume the data distribution has the property µ XYM = µ XY δ ( M = Y ) , i.e. M = Y on all the data. In this case, we know that

̸

and we define

̸

Now, if we set Pr( X = x i ) = p i , and r ( x i ) = r i , then the optimization problem

is equivalent to

that is equivalent to

Next, we show that the above problem spans all instances of the 0 -1 knapsack problem, which is known to be NP-hard (Theorem 15.8 of [58]). Let

be an instance of the 0 -1 knapsack problem 7 with w i , c i > 0 , i ∈ [ n ] , and K > 0 . With ℓ i = c i /w i ∑ n i =1 c i /w i , p i = w i ∑ n i =1 w i and b = K ∑ n i =1 w i , problem (33) can be written in the form of (32). Because of ∑ n i =1 l i = ∑ n i =1 p i = 1 this yields indeed a valid problem.

I Proof of Theorem 4.1

We start this proof by introducing a few useful lemmas:

Lemma I.1. The set F = ∆ X n of all functions that map X to an n -dimensional probability is weakly compact, i.e., for each sequence { f n } ∞ n =1 , there is a sub-sequence { f n i } and a function f ∗ ∈ F such that for all measurable embedding functions ψ , we have

Proof. We know that all components of each element of the function sequence is bounded by 1 . We define { f i m } ∞ m =1 as the sequence of the i th component of the function sequence. Therefore, using [42, Theorem A.5.1] we know that there is a sub-sequence { f 1 m 1 k } ∞ k =1 and a non-negative 1 -bounded function f ∗ 1 , such that for each µ -integrable function ψ 1 ( x ) we have

Next, we can repeat the same process for { f i m 1 k } ∞ k =1 where i ∈ [2 : n ] , and we can find a sub-sequence m i +1 k of m i k and a non-negative 1 -bounded function f ∗ i +1 for which

Now, since all sub-sequences of a converging sequence converges to the same limit, we can use m n k that is the intersection of all sequences and show that

for all i ∈ [1 : n ] and integrable functions ψ i . As a result, due to interchangeability of limit and summation, when the sum is over a finite set of elements, it is easy to show that

Next, we need to show that f ∗ ∈ F . We already know that all components of f ∗ is 1 -bounded and non-negative. Therefore, we only need to prove that all elements of f ∗ sum up to 1 almost everywhere. If not, then assume that there is a non-zero set A where µ ( A ) > 0 and there exists l > 0 such that | ∑ i f ∗ i -1 | ≥ l for all x ∈ A . We know that there is either a subset B ⊆ A with µ ( B ) > 0 such that for all x ∈ B we have ∑ i f ∗ i ( x ) ≥ 1 + l , or similarly a subset for which ∑ i f ∗ i ( x ) ≤ 1 -l . The reason is that otherwise a non-zero measure set A is a union of two zero-measure set, which is a contradiction. Without loss of generality we assume the first, which means ∑ i f ∗ i ( x ) ≥ 1 + l for x ∈ B . Now, if we define ˆ ψ ( x ) = [1 , . . . , 1] for x ∈ B and otherwise ˆ ψ ( x ) = [0 , . . . , 0] , then we have

for all k ∈ N . This is a contradction, because the limit of a constant sequence is not different from that constant value. Hence, f ∗ sums up to 1 almost everywhere, and that completes the proof. Proof of Theorem 4.1 : We prove the theorem using the following steps: (i) for the class C of prediction functions for which E [ ⟨ f ( x ) , ψ i ( x ) ⟩ ] = δ i for i ∈ [1 : m ] , we show that the supremum of the objective function E [ ⟨ f ( x ) , ψ 0 ( x ) ⟩ ] is a maximum, (ii) we show that it is sufficient for a predictor f ∈ C to be in form of (8) to achieve the maximum objective E [ ⟨ f ( x ) , ψ 0 ( x ) ⟩ ] in C and also for all predictors with E [ ⟨ f ( x ) , ψ i ( x ) ⟩ ] ≤ δ i , (iii) we show that the space of all possible constraints for any prediction function in ∆ X d is convex and compact, and (iv) we show that if the tuple of constraints is an interior point of all possible tuples of constraints, then a point in C achieves its maximum if and only if it follows the thresholding rule (8) almost everywhere.

Step (i) : Due to the definition of supremum, we know that for each ϵ > 0 , there exists a function f ϵ in C such that E [ ⟨ f ϵ , ψ 0 ( x ) ⟩ ] ≥ sup f ∈C E [ ⟨ f, ψ 0 ( x ) ⟩ ] -ϵ . Equivalently, there is a sequence of functions f n for which lim n →∞ E [ ⟨ f n , ψ 0 ( x ) ⟩ ] = sup f ∈C E [ ⟨ f, ψ 0 ( x ) ⟩ ] . Using weakly-compactness of the function class ∆ X n +1 as in Lemma I.1, we know that for the sequence f n , there exists a subsequence f n k and a function f ∗ ∈ ∆ X n +1 such that

Furthermore, we know that each subsequence a n k of a converging sequence a n has the same limit as the limit of the mother sequence a n [59, Chapter 2, Theorem 1]. Therefore, we have

which means that the supremum of the objective is achievable by f ∗ . Moreover, for ψ i ( x ) where i ∈ [1 : m ] , we have E [ ⟨ f n , ψ i ( x ) ⟩ ] = δ i for all n , which concludes

This means that the equality constraints holds for f ∗ , i.e., f ∗ ∈ C , if it holds for each predictor f n .

Step (ii) : Assume that there is a predictor ˆ f such that E [ ⟨ ˆ f, ψ i ⟩ ] ≤ δ i . In this step, we show that if exists a predictor f in form of (8) and in C , then ˆ f always has smaller objective than ˆ f . To that end, consider the following expression:

while

Now, we know that

where ( a ) holds because f ∈ C . As a result, if A ≥ 0 , then we could show that

and complete the proof.

To that end, first note that both f and ˆ f are in ∆ X d , and therefore

As a result, for any fixed scalar c , we have

Next, we fix c to be the maximum component of the vector ψ 0 ( x ) -∑ m i =1 k i ψ i ( x ) , i.e.,

Now, we rewrite A using (35) as

Now, we consider two cases for which E i 1 ( x ) : f i ( x ) > ˆ f i ( x ) , and E i 2 ( x ) : f i ( x ) ≤ ˆ f i ( x ) . If f i ( x ) > ˆ f i ( x ) , then we have f i ( x ) > 0 , because 1 ≥ ˆ f i ( x ) ≥ 0 for all i ∈ [1 : d ] . Therefore, using the definition of S d and because f ∈ S d we have

Consequently, we have

where ( a ) holds due to (36) and ( b ) holds because f i ( x ) ≤ ˆ f i ( x ) and ψ i m +1 ( x ) -∑ m j =1 k j ψ i j ( x ) ≤ c = max i ∈ [1: n +1] { ψ i m +1 ( x ) -∑ m j =1 k j ψ i j ( x ) } . As a result, we have A ≥ 0 that concludes (34) and completes the proof of this step.

v

M

v

∗∗

v

∗

v

′

v

∗∗

v

∗

v

Step (iii) : In this step, we show that the space of joint set of expected inner-products

is compact under Euclidean metric, and convex.

To show the compactness of that space, assume that there is a sequence { g n } ∞ n =1 such that lim n →∞ g n = g , or accordingly there is a sequence of f n ∈ ∆ X d for which lim n →∞ ( E [ ⟨ f n ( x ) , ψ 1 ( x ) ⟩ ] , . . . , E [ f n ( x ) , ψ m ( x )] ) = ( g 1 , . . . , g m ) . Since the metric is Euclidean, this is equivalent to lim n →∞ E [ ⟨ f n ( x ) , ψ i ( x ) ⟩ ] = g i for all i ∈ [1 : m ] . The weak compactness of ∆ X d , as proved in Lemma I.1, shows that there exists f ∗ and a subsequence f n k such that lim k →∞ E [ ⟨ f n k ( x ) , ψ i ( x ) ⟩ ] = E [ ⟨ f ∗ , ψ i ( x ) ⟩ ] for all i ∈ [1 : d ] . Therefore, because of the choice of Euclidean metric, we have

which is equivalent to compactness of G .

To show the convexity of G , it is enough to prove the convexity of ∆ X d . The reason is that g ( f ) = ( E [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] , . . . , E [ f ( x ) , ψ m ( x )] ) is a linear functional of f , and a linear functional images a convex set to another convex set.

To prove the convexity of ∆ X d , let f, f ′ ∈ ∆ X d . This means that f i ( x ) , f ′ i ( x ) ∈ [0 , 1] for all i ∈ [1 : d ] and ∑ d i =1 f i ( x ) = ∑ d i =1 f ′ i ( x ) = 1 . Consequently, af i ( x ) + (1 -a ) f ′ i ( x ) ≥ 0 , since a, 1 -a ≥ 0 . Moreover, ∑ d i =1 af i ( x ) + (1 -a ) f ′ i ( x ) = a ∑ d i =1 f i ( x ) + (1 -a ) ∑ d i =1 f ′ i ( x ) = a +1 -a = 1 . As a result of these two facts, af +(1 -a ) f ′ ∈ ∆ X d , and the proof of this step is completed.

Step (iv) : In this step we show that if the tuple of constraints is an interior points of all possible achievable tuples of constraints using different prediction functions, then a point in C achieves its supremum in terms of objective E [ ⟨ f ( x ) , ψ 0 ( x ) ] if and only if it is in form of (8) almost everywhere. This is an extension to [21, Theorem 3.1] and its proof resembles to the proof that is provided there. The sufficiency is already shown in Step (ii). Therefore, we only need to show that if a prediction function in C maximizes the objective, then it is in form of (8).

Firstly, using Step (iii), we know that the space N of all points ( E [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] , . . . , E [ f ( x ) , ψ m ( x )] ) and the space M of all points ( E [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] , . . . , E [ f ( x ) , ψ 0 ( x )] ) are compact and convex. Now, assume that v = ( δ 1 , . . . , δ m ) is an interior point of N . Then, the corresponding set in M , i.e., B v = { ( δ 0 , . . . , δ m ) ∈ M : δ 0 ∈ R } has a supremum and an infimum of the first component that we name δ ∗∗ and δ ∗ . Now, since M is compact, then v ∗∗ = ( δ ∗∗ , δ 1 , . . . , δ m ) and v ∗ = ( δ ∗ , δ 1 , . . . , δ m ) are contained in M . Next, assume the following two cases:

δ ∗∗ = δ ∗ : In this case for all other points v = ( δ 1 , . . . , δ m ) of N , the corresponding set B v ′ in M is a single point. The reason is that, if it is not so, then we have two points v ∗∗ = ( δ ∗∗ , δ 1 , . . . , δ m ) and v ∗ = ( δ ∗ , δ 1 , . . . , δ m ) where δ ∗∗ > δ ∗ . Now, since v is an interior point of N , then on any direction in a small neighborhood around v there exists a point v ′ within N . Let that direction be opposite the connecting line of v and v i.e., let v be on a connecting line of v ′ and v ∗ . Now, make a convex hull using the three points v ′ , v ∗∗ , and v ∗ , which are all in M . Because of the convexity of M , the convex hull is also a subset of M . Since v is an interior point of the convex hull, this means that a neighborhood of v along any direction is inside M . Now, if we set ( m +1) th axis to be that direction, we contradict with the fact that δ ∗ = δ ∗∗ . (See Figure 4)

Now, we know that in such case all points within N have one corresponding point in M . Because of the convexity of M this is equivalent to M being a subset of a hyperplane with the generating formula x 0 = ∑ m i =1 k i x i + k 0 . Therefore, we have E [ ⟨ f, ψ 0 ⟩ ] = E [ ⟨ f, ∑ m i =1 k i ψ i ⟩ ] + k 0 for all f ∈ ∆ X d . Therefore, for d ≥ 2 if we set f 1 = ( p ( x ) d -2 , . . . , p ( x ) d -2 , 1 -p ( x ) ︸ ︷︷ ︸ , p ( x ) d -2 , . . . , 0 ︸︷︷︸ j , p ( x ) d -2 , . . . , p ( x ) d -2 ) and f 2 =

, , i

( p ( x ) d -2 , . . . , p ( x ) d -2 , 0 ︸︷︷︸ i , p ( x ) d -2 , . . . , 1 -p ( x ) ︸ ︷︷ ︸ j , p ( x ) d -2 , . . . , p ( x ) d -2 ) for p ( x ) ∈ [0 , 1] X , then we have

or equivalently

for all function p ( x ) ∈ ∆ X d . A similar result can be achieved for d = 2 and by setting f 1 = ( p ( x ) , 1 -p ( x )) and f 2 = (1 -p ( x ) , p ( x )) . As a result, we have

̸

for all i = j ∈ [1 : d ] , and consequently

for all i ∈ [1 : n + 1] . As a result, there is a set of k 1 , . . . , k m such that ψ 0 ( x ) -∑ m i =1 k i ψ i ( x ) has equal components almost everywhere. As a result, due to the freedom of choice for τ ( ψ 0 ( x ) -∑ m i =1 k i ψ i ( x ) , x ) where τ ∈ S d and when we have more than one maximizer component, then, without loss of generality we can assume that every prediction function f almost everywhere is in form of τ ( ψ m +1 ( x ) -∑ m i =1 k i ψ i ( x ) , x ) .

̸

δ ∗∗ > δ ∗ : In such case, for all δ 0 ∈ [ δ ∗ , δ ∗∗ ] , we can show that v = ( δ 0 , . . . , δ m ) is an interior point of M . To show that, we first find m points v ′ 1 , . . . , v ′ m ∈ N that are linearly independent and such that their convex hull include ( δ 1 , . . . , δ m ) . Using the definition of M , for each of these points v ′ i = ( δ ′ 1 i , . . . , δ ′ m i ) , there exists h ′ i ∈ R such that v ′′ i = ( h ′ i , δ ′ 1 i , . . . , δ ′ m i ) is within M . Now, we add the two points v ∗∗ and v ∗ to these sets of points. It is easy to see that v ′′ i s are linearly independent. Furthermore, we know that ( δ 1 , . . . , δ m ) is a convex combination of v ′ i s, i.e., ∑ i a i v ′ i = ( δ 1 , . . . , δ m ) . As a result, if ∑ i b i v ′′ i -v ∗∗ = (0 , . . . , 0) , then we have b i = a i for i ∈ [1 : m ] . Furthermore, we have ∑ a i h ′ i = ∑ b i h ′ i = δ ∗∗ . Similarly, if ∑ i c i v ′′ i -v ∗ = (0 , . . . , 0) we have c i = a i and ∑ a i h ′ i = ∑ c i h ′ i = δ ∗ . As a result, since δ ∗ = δ ∗∗ at least one of these cases would not occur, or equivalently, the dimension of the convex hull of v ′′ 1 , . . . v ′′ m , v ∗∗ , v ∗ is of dimension m +1 . As a result, v is an interior point of

this convex hull, and because the convex hull is ( m +1) -dimensional, it is an interior point of M .

̸

Now, since v ∗∗ is a border point in M and due to the convexity of M there is a hyperplane P such that it passes v ∗∗ and it lays above all points of M . Since v is an interior point of M , a neighborhood of v is laid under the hyperplane P , hence v cannot be laid on the hyperplane. Therefore, if the generating formula of such hyperplane is ∑ m i =0 k i x i = ∑ m i =1 k i δ i + k 0 δ ∗∗ , since v is not laid on the hyperplane we assure that ∑ m i =1 k i δ i + k 0 δ 0 = ∑ m i =1 k i δ i + k 0 δ ∗∗ , or equivalently k 0 = 0 . Hence, without loss of generality assume that k 0 = -1 . This shows that for all points in ( u 0 , . . . , u m ) ∈ M we have

or equivalently, by the definition of M , for all prediction function f , we have

Assuming that ˆ f ∈ C maximizes the objective, we have

̸

This shows that almost everywhere when there is a unique maximizing component j in ψ 0 ( x ) -∑ m i =1 k i ψ i ( x ) , then ˆ f j ( x ) = 1 . The reason is that otherwise and if there is a set A such that µ ( A ) > 0 and for x ∈ A and a choice of l ∈ [0 , 1) , ϵ ∈ R , and all t = j we have ψ j m +1 ( x ) -∑ m i =1 k i ψ j i ( x ) ≥ ϵ + ψ t m +1 ( x ) -∑ m i =1 k i ψ t i ( x ) while f j ≤ 1 -l , then we can make a function f ( x ) that is f ( x ) = ˆ f ( x ) for x ∈ X \ A and f ( x ) = [0 , . . . , 1 ︸︷︷︸ j , . . . , 0] for x ∈ A . Such function leads to

that is in contradiction with (37). This completes the proof of this step.

J Proof of Theorem 4.2

In the following, we introduce a few lemmas that are useful in our proofs.

Lemma J.1. For every random variable X on R we have

Proof. For each increasing sequence { τ i } ∞ i =1 we show that the first limit is zero, which proves the claim that the function of τ has a zero limit. We define

̸

and notice that

Further, we note that

As a result

and

Next, because probability measure is σ -additive, we conclude its lower-semicontinuity [38, Theorem 13.6], and therefore we have

which proves X ∈ S

We could take similar steps to show that since ⋂ ( t, τ ′ ) = ∅ for decreasing τ ′ we have lim i →∞ Pr( i ) = 0 . ∞ i =1 i i

Lemma J.2. Let ψ 1 : X → R d be a bounded function. Further, we define two functions C ( k ) = E [ ⟨ f ∗ k, 0 ( x ) , ψ 1 ( x ) ⟩ ] , D ( k ) = E [ ⟨ f ∗ k, 1 ( x ) , ψ 1 ( x ) ⟩ ] , and F ( k ) = E [ ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ ] , where f ∗ k,p is defined in Theorem 4.2. Then,

C ( k ) is monotonically non-increasing,

C ( k ) is upper semi-continuous,

F ( k ) is monotonically non-decreasing,

D ( k ) is lower semi-continuous, and we have

lim k ′ ↑ k C ( k ) = lim k ′ ↑ k D ( k )

Proof. 1. Firstly, let us define ℓ k ( x ) = ψ 0 ( x ) -kψ 1 ( x ) . For the setting where p = 0 , the prediction function f ∗ k,p ( x ) is defined as

Further, for k 1 , k 2 such that k 1 ≤ k 2 , let us define j 1 and j 2 as the only non-zero index of f ∗ k 1 , 0 ( x, p ) and f ∗ k 2 , 0 ( x, p ) , respectively. To show that C ( k ) is monotonically non-increasing we only need to show that ( ψ 1 ( x ) ) ( j 1 ) = ⟨ f ∗ k 1 , 0 ( x ) , ψ 1 ( x ) ⟩ ≥ ⟨ f ∗ k 2 , 0 ( x ) , ψ 1 ( x ) ⟩ = ( ψ 1 ( x ) ) ( j 2 ) . Assume that this does not occur, or equivalently ( ψ 1 ( x ) ) ( j 1 ) < ( ψ 1 ( x ) ) ( j 2 ) . In such case we have

where ( a ) and ( b ) holds due to the definition of j 1 and j 2 , and ( c ) holds due to the assumption ( ψ 1 ( x ) ) ( j 1 ) < ( ψ 1 ( x ) ) ( j 2 ) . The last inequality is clearly a contradiction, and shows that ⟨ f ∗ k 1 , 0 ( x ) , ψ 1 ( x ) ⟩ ≥ ⟨ f ∗ k 2 , 0 ( x ) , ψ 1 ( x ) ⟩ , and therefore C ( k 1 ) ≥ C ( k 2 ) .

Let us divide the space X into two subsets

For each x ∈ A k we know

Using previous part, we know that by increasing k we have no increase in ⟨ f ∗ k, 0 ( x ) , ψ 1 ( x ) ⟩ , and in this case since ⟨ f ∗ k, 0 ( x ) , ψ 1 ( x ) ⟩ = min j ( ψ 1 ( x ) ) ( j ) , then this value cannot reduce with the change of k . Therefore, ⟨ f ∗ k, 0 ( x ) , ψ 1 ( x ) ⟩ is a constant function for all k ′ ≥ k , and consequently E [ ⟨ f ∗ k ′ , 0 ( x ) , ψ 1 ( x ) ⟩| x ∈ A k ] Pr( x ∈ A k ) is a constant function for k ′ ≥ k . If x ∈ B k , then for j / ∈ argmax i ( ℓ k ( x ) ) ( i ) and l ∈ argmax i ( ℓ k ( x ) ) ( i ) , we have

(

ℓ

k

(

x

)

)

(

j

)

<

(

ℓ

k

(

x

)

)

(

l

)

. Define the set

C

δ

for

δ

0

as

Using Lemma J.1 we know that

This means that

This, together with the previous part in which we showed ⟨ f ∗ k, 0 ( x ) , ψ 0 ( x ) ⟩ ≥ ⟨ f ∗ k ′ , 0 ( x ) , ψ 0 ( x ) ⟩ , concludes that ⟨ f ∗ k, 0 ( x ) , ψ 0 ( x ) ⟩ = ⟨ f ∗ k ′ , 0 ( x ) , ψ 0 ( x ) ⟩ . This means that E [ ⟨ f ∗ k ′ , 0 ( x ) , ψ 0 ( x ) ⟩| x ∈ C δ ] Pr( x ∈ C δ ) is a constant function for all k ′ ≥ k . Finally, since we have

and because the first term and the third term in RHS are constant in terms of k ′ and for k ′ ≥ k , and the second term is diminishing, then we have

which is equivalent to say that lim k ′ ↑ k C ( k ′ ) = C ( k ) .

For p = 1 , we know that the prediction function f ∗ k,p ( x ) is obtained as

or equivalently for all ϵ ≥ 0 , there exists δ such that

Therefore, if without loss of generality, we assume that ψ 1 ( x ) is bounded by 1 , then there exists δ ≥ 0 such that we have

where ( a ) holds due to Hölder's inequality. If x ∈ C δ , and because we assumed ∥ ψ 1 ( x ) ∥ ∞ ≤ 1 , then we know that by increasing k to k ′ ∈ [ k -δ/ 2 , k + δ/ 2) , we have

.

If we define ψ ′ 1 ( x ) := -ψ 0 ( x ) , then we have

.

Since the above is equal to (38), then using the first part of this lemma, we know that E [ ⟨ f ∗ k, 1 ( x ) , ψ ′ 1 ( x ) ⟩ ] = -E [ ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ ] is monotonically non-increasing, which is equivalent to F ( k ) = E [ ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ ] being monotonically non-decreasing.

This part is similar to the second part of the proof. In fact, if x ∈ A k , then we have

For k ′ ≤ k and because of the third part of this lemma, we know that ⟨ f ∗ k ′ , 1 ( x ) , ψ 0 ( x ) ⟩ ≥ ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ . Furthermore, because of (41) we know that ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ = max ψ 0 ( x ) , and therefore by reducing k ′ , the prediction function f ∗ k ′ , 1 ( x ) stays constant. As a result, E [ ⟨ f ∗ k ′ , 1 ( x ) , ψ 1 ( x ) ⟩| x ∈ A k ] Pr( x ∈ A k ) is a constant function for k ′ ≤ k .

Furthermore, similar to the second part of this lemma, we can show that for each ϵ > 0 , there exists δ ′ ≥ 0 such that for all 0 ≤ δ ≤ δ ′ we have

Moreover, for the case of x ∈ C δ , since in this case J ⊆ I , then we know that

Next, using the third part of this lemma, we know that for k ′ ≤ k we have ⟨ f ∗ k ′ , 1 ( x ) , ψ 0 ( x ) ⟩ ≤ ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ , which together with (43) concludes that ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ = ⟨ f ∗ k ′ , 1 ( x ) , ψ 0 ( x ) ⟩ . Next, because ( ψ 0 ( x ) -kψ 1 ( x ) ) ( i ) = ( ψ 0 ( x ) -kψ 1 ( x ) ) ( j ) for i, j ∈ J , then we know that ∣ ∣ ∣ ( ℓ k ′ ( x ) ) ( i ) -( ℓ k ′ ( x ) ) ( j ) ∣ ∣ ∣ = ∣ ∣ ( k -k ′ ) ( ( ψ 1 ( x ))( i ) -( ψ 1 ( x ))( j ) ) ≤ 2 | k -k ′ | . Therefore, if for i, j ∈ J we know that ( ψ 0 ( x ) ) ( i ) = ( ψ 0 ( x ) ) ( j ) , then the difference between ψ 1 for those indices is bounded as

Now, we know that because x ∈ C δ , then ⟨ f ∗ k, 1 ( x ) , ψ 1 ( x ) ⟩ = ( ψ 1 ( x ) ) ( i ) for i ∈ argmax j ∈J ( ψ 0 ( x ) ) ( j ) , and ⟨ f ∗ k ′ , 1 ( x ) , ψ 1 ( x ) ⟩ = ( ψ 1 ( x ) ) ( j ) for j ∈ argmax k ∈I ( ψ 0 ( x ) ) ( j ) .

Hence, we can see that i ∈ J ⊆ I and j ∈ I , and because ( ψ 0 ( x ) ) ( i ) = ⟨ f ∗ k, 1 ( x ) , ψ 0 ( x ) ⟩ = ⟨ f ∗ k ′ , 1 ( x ) , ψ 0 ( x ) ⟩ = ( ψ 0 ( x ) ) ( j ) , and due to (44) we have

as long as k ′ ∈ [ k -δ/ 2 , k ) . Therefore, if we set δ = max { δ ′ , ϵ/ 2 } we have

and therefore

Finally, we can rewrite D ( k ′ ) as

and because of (42) and (45), and since the first term is a constant function in terms of k ′ and for all k ′ ∈ [ k -δ/ 2 , k ] , then we have

This shows that D ( k ′ ) is lower semi-continuous around k ′ = k .

To prove this part, we first divide X into two subsets

and H k ′ = X \ G k ′ . We know that for x ∈ G k ′ we have

This concludes that

Moreover, let us define the set Ψ k ′ 1 = { x ∈ X : ∃ c ∈ R , ∀ j ∈ argmax ℓ k ′ ( x ) , ( ψ 1 ( x ) ) ( j ) = c } . We show that sum of the probabilities of H k ′ \ Ψ k ′ 1 is always bounded by 2 d for a set of choices for k ′ , or equivalently

for all finite or countably infinite choice of K ⊆ R + . In fact, we know that for each instance x , argmax j ∈ [1: d ] ( ℓ k ( x ) ) ( j ) can take up to 2 d cases of all subsets of { 1 , . . . , d } . Therefore, we need to show that there cannot exist two values of k, k ′ such that for x ∈ ( H k \ Ψ k 1 ) ∩ ( H k ′ \ Ψ k ′ 1 ) we have

If we prove such identity, then due to pigeonhole principle, we have

which by integration over all values of x concludes in (50). We prove this claim by contradiction. If we assume k, k ′ ∈ K such that for x ∈ ( H k \ Ψ k 1 ) ∩ ( H k ′ \ Ψ k ′ 1 ) the identity (51) holds, then because x ∈ H k ∩ H k ′ , then the size of argmax ( ℓ k ( x ) ) ( j ) and

j

argmax j ( ℓ k ′ ( x ) ) ( j ) is at least 2 . This concludes that

as well as

for all choices of i, j ∈ argmax ℓ k ( x ) . As a result, we have

̸

and because k ′ = k , we have

for all i, j ∈ argmax ℓ k ( x ) . Therefore, x ∈ Ψ k ′ 1 and that is a contradiction. Now that we know that the sum of the probabilities of Pr( x ∈ H k ′ \ Ψ k ′ 1 ) is bounded, we can renormalize that and make a probability measure as

Due to Lemma J.1, for all ϵ ≥ 0 we can find a small enough δ ≥ 0 such that g ([ k -δ, k )) ≤ ϵ/ 2 d +1 , and therefore for all k ′ ∈ [ k -δ, k ) we have

where the last inequality holds because of (50). Now, using this and due to (49), and by defining g i ( x ) = ⟨ f ∗ k,i ( x ) , ψ 0 ( x ) ⟩ for i = 1 , 2 , we can bound the difference of D ( k ) and C ( k ) as

where ( a ) holds because ∥ f ∗ k, 0 -f ∗ k, 1 ∥ 1 ≤ ∥ f ∗ k, 0 ∥ 1 + ∥ f ∗ k, 1 ∥ 1 = 2 and because of Hölder inequality we have ∣ ∣ ⟨ f ∗ k, 0 ( x ) -f ∗ k, 1 , ψ 1 ( x ) ⟩ ∣ ∣ ≤ ∥ f ∗ k, 0 -f ∗ k, 1 ∥ 1 ∥ ψ 1 ( x ) ∥ ∞ ≤ 2 . Moreover, to show that ( b ) holds we know that for x ∈ Ψ k ′ 1 we have ( ψ 1 ( x ) ) ( i ) = ( ψ 1 ( x ) ) ( j ) for all i, j ∈ argmax ℓ k ′ ( x ) . Therefore, because we know g 0 ( x ) = ( ψ 1 ( x ) ) ( i ) for i ∈ argmin j ∈ argmax l ( ℓ k ′ ( x ) ) ( l ) ( ψ 1 ( x ) ) ( j ) ⊆ argmax l ( ℓ k ′ ( x ) ) ( l ) and g 1 ( x ) = ( ψ 1 ( x ) ) ( j ) for j ∈

argmax j ∈ argmax l ( ℓ k ′ ( x ) ) ( l ) ( ψ 0 ( x ) ) ( j ) ⊆ argmax l ( ℓ k ′ ( x ) ) ( l ) , we have g 0 ( x ) = g 1 ( x ) . The above inequality proves that the limit of C ( k ′ ) and D ( k ′ ) for k ′ ↑ k are equal and that completes the proof.

To prove this theorem, we take the following steps: (i) We show that the set K has a non-negative member, (ii) we show that the prediction function f ∗ k,p ( x ) achieves the inequality constraint tightly, and by Theorem 4.1 we can conclude that f ∗ k,p ( x ) is the optimal solution.

step (i) : It is easy to see that the Bayes optimal solution of the prediction function in (3) without any constraint is

̸

where p i ( x ) ∈ ∆ d is an arbitrary vector. We can see that by setting

then the two prediction functions f ∗ ( x ) and f ∗ 0 , 0 ( x ) are equal (See statement of Theorem 4.2).

Now, in the first and second part of Lemma J.2 we have shown that E [ ⟨ f ∗ k, 0 ( x ) , ψ 1 ( x ) ⟩ ] is upper semi-continuous and monotonically non-increasing. Therefore, for all k ∈ R + we have

Similarly, we can show that for k →∞ , the solution is equivalent to the Bayes minimizer of

Therefore, since δ is an interior point of all possible values, it lays on the interval ( E [ ⟨ f ∗∗ ( x ) , ψ 1 ( x ) ⟩ ] , E [ ⟨ f ∗ ( x ) , ψ 1 ( x ) ⟩ ]) , due to the montonicity and upper semicontinuity of E [ ⟨ f ∗ k, 0 , ψ 1 ( x ) ⟩ ] , we can find t such that

Moreover, this t should be a positive scalar, since otherwise we have

which is a contradiction to (54).

step (ii) : In this step, we consider the following two cases:

-C ( t ) is continuous at t : In this case, (54) is equivalent to δ = C ( t ) = E [ ⟨ f ∗ t, 0 ( x ) , ψ 0 ( x ) ⟩ ] , which means that the prediction function f ∗ k, 0 ( x ) achieves the constraint tightly, and therefore using Theorem 4.1 f ∗ k, 0 ( x ) is the optimal solution.

-C ( t ) is discontinuous at t : To show that we can achieve the highest constraint in this case, we first condition the constraint into two events x ∈ G k and x ∈ X \ G k , where G k is defined in (47). We know that in the latter case x ∈ X \ G k , the prediction function f ∗ k,p can be decomposed into two components

while for x ∈ G k the prediction function f ∗ k,p ( x ) = f ∗ k, 0 ( x ) = f ∗ k, 1 ( x ) for all p ∈ [0 , 1] . Therefore, in both cases (55) holds, and we have

where C ( · ) and D ( · ) are defined in Lemma J.2. Using this lemma, we know that D ( · ) is lower semi-continuous, and lim k ′ ↑ k C ( k ) = lim k ′ ↑ k D ( k ) . Therefore, together with (56) and the definition of p in the statement of theorem, we have

Equivalently, the prediction function achieves the constraint inequality tightly, and therefore by Theorem 4.1 this is sufficient to be the optimal solution to the constrained optimization problem.

K Proof of Theorem 5.1

Through the proof of this theorem, we use [6, Lemma 3.2.3] that implies that the class of multiplications of k binary functions f i ( x ) for i ∈ [1 : k ] within a hypothesis class with VC dimension V C ( f i ) = d itself has a VC dimension that is bounded as

In fact, we use a simple extension to this lemma for which the VC dimension of the functions is not d itself but is bounded above by d . In such case we claim that (58) still holds. The starting point for the proof to this lemma is bounding the size of the restriction Π H ( S ) = |{ h ∩ S : h ∈ H}| for the hypothesis class H by

where V C ( H ) = d and m = | S | . However, this inequality holds for the hypothesis classes that have VC dimensions that are bounded by d . The reason is increasingly monotonicity of RHS of (59). In fact, by obtaining the gradient of ( em d ) d in terms of d we have

which is nonnegative as long as m ≥ d . If we particularly set m ∗ = 2 dk log 3 k , then m ∗ ≥ d and therefore (59) holds. Next, similar to the proof of [6, Lemma 3.2.3], we can show that for the set S with size m ∗ we have

which means that S cannot be shattered by H ′ , and therefore the VC dimension of this hypothesis class must be bounded by m ∗ .

We further use the following lemma:

Lemma K.1. For arbitrary sets of functions { ϕ i 1 ( x ) } n i =1 and { ϕ i 2 ( x ) } n i =1 on R and for a given d ∈ R the hypothesis class

has the VC dimension of at most 4 .

Proof. To prove this lemma, we show that the form of the product in the definition of H reduces to the form of an interval on R , which is known to have VC dimension of 2 . In fact, each term sgn( ϕ i 1 ( x ) -kϕ i 2 ( x ) -d ) can be rewritten as

As a result, by multiplying all terms we have

where A x , B x , and C x are defined as A x = { i ∈ [1 : n ] : ϕ i 2 ( x ) > 0 } , B x = { i ∈ [1 : n ] : ϕ i 2 ( x ) < 0 } , and C x = { i ∈ [1 : n ] : ϕ i 2 ( x ) = 0 } . Now, we see that the first two terms define an interval for k ∈ ( f 1 ( x ) , f 2 ( x ) ) where f 1 ( x ) = max i ∈B x ϕ i 1 ( x ) -d ϕ i 2 ( x ) and f 2 ( x ) = min i ∈A x ϕ i 1 ( x ) -d ϕ i 2 ( x ) . Next,

̸

we prove that the VC dimension of the hypothesis class of all such functions is less than the VC dimension of G = { f : R × R →{ 0 , 1 } : f ( x, y ) = sgn( x -k 1 )sgn( k 2 -y ) , k 1 , k 2 ∈ R } . The reason is that if the aforementioned interval can shatter a set S , then we can find the corresponding values of f 1 ( x ) and f 2 ( x ) for each x ∈ S , and then form the pair ( x i , y i ) where x i = f 1 ( x ) and y i = f 2 ( x ) , and by setting k 1 = k 2 = k , we can shatter the set { ( x i , y i ) } |S| i =1 with G . Note that here all pairs are identical. The reason is that if not, i.e., if f 1 ( x ) = f 1 ( x ′ ) and f 2 ( x ) = f 2 ( x ′ ) for x, x ′ ∈ S and x = x ′ , then, for all possible k , we have sgn( k -f 1 ( x ))sgn( f 2 ( x ) -k ) = sgn( k -f 1 ( x ′ ))sgn( f 2 ( x ′ ) -k ) , and therefore we cannot shatter S by sgn( k -f 1 ( x ))sgn( f 2 ( x ) -k ) . Therefore, the set { ( x i , y i ) } |S| i =1 has the same cardinality of S , which in consequence proves that the VC dimension of all sgn( k -f 1 ( x ))sgn( f 2 ( x ) -k ) is bounded by V C ( G ) . Moreover, V C ( G ) ≤ 4 , since for each 5 points in two-dimensional space, one is in the convex hull of the others, and in case that all others are labeled as 1 , the one in the convex hull also must be labeled as 1 . As a result, G cannot shatter 5 points, and therefore V C ( G ) ≤ 4 .

̸

Up to now, we have shown that the class of functions equal to the first two terms of (60) has a VC dimension that is bounded by 4 . Next, we show that multiplying a hypothesis class H with a binary function ϕ ( x ) does not increase the VC dimension of that class. More formally, if we define

then V C ( H ) ≤ V C ( H ′ ) . The reason is that if we can shatter a set S using H , then for each member x ∈ S there exists two members f 1 , f 2 of H ′ such that f 1 ( x ) = 1 and f 2 ( x ) = 0 . This means that ϕ ( x ) = 0 , because otherwise f 1 ( x ) = 1 would not be achievable. Therefore, ϕ ( x ) = 1 for all x ∈ S , and as a result similarly H ′ can shatter S , which proves that V C ( H ) ≤ V C ( H ′ ) . Finally, since we know that the class of all functions in H is in form of sgn( k -f 1 ( x ))sgn( f 2 ( x ) -k ) multiplied with a binary function, then we conclude that V C ( H ) ≤ 4 .

To prove the rest of the theorem, we need to show that for all choices of ˆ k and ˆ p the difference of the empirical and the true loss is bounded. In fact, we should find a bound in form of

Here, we divide the class X into two subsets G k and H k = X \ G k , where G k is defined in (47). Now, using the definition of f ∗ k,p ( x ) , we know that within G k , the inner-product ⟨ f ∗ k,p ( x ) , ψ 1 ( x ) ⟩ can be rewritten as

̸

̸

Now, we can condition x on being a member of G k , and therefore the maximum difference between the two empirical and true expectation is as

Now, we bound the inner term of (61) in a high probability setting. To that end, we use Rademacher's inequality in [66, Theorem 26.5], which shows that maximum difference between the expected value of a function h ∈ H over empirical distribution and the true distribution is 2 R ( H ) +4 c √ ln 4 /ϵ n where R ( H ) is the Rademacher's complexity of the class of function H and c is maximum value that h can take. By defining

we have c = ∥ ( ψ 1 ( x ) ) ( j ) ∥ ∞ ≤ 1 . Therefore, we have for all h ,

with probability at least 1 -ϵ d . Now, we can use contraction Lemma [66, Lemma 26.9] to show that since ∥ ( ψ 1 ( x ) ) ( j ) ∥ ∞ ≤ 1 , then R ( H ) ≤ R ( F ) , where F = { Φ k j ( x ) , k ∈ R } . Moreover, F contains functions that are all multiplication of d -1 binary functions all in form of

Lemma K.1 shows that the hypothesis class that contains products of all such function has a VCdimension that is bounded by 4 . As a result, the Rademacher's complexity of F is bounded using [47, Corollary 3.8, Corollary3.18] as

and therefore together with (62) for all h ∈ H we have

with probability at least 1 -ϵ d . Hence, using (61) we have

with probability at least 1 -ϵ . In the last inequality, we used Bonferroni's inequality on ϵ/d bad events that each summand of (61) is not within the concentration bound.

Next, we consider the region H k in which there are at least two maximizer components of ℓ k ( x ) . In this case, by definition of ˆ f k,p ( x ) , among these maximizers, we choose the first maximizer of ψ 0 ( x ) with probability p and the first minimizer of ψ 1 ( x ) with probability 1 -p . Therefore, by condition on these cases, and if we define

then we have

Now, to bound E ( k, 1) , we first rewrite the closed-form solution of ˆ f k, 1 ( x ) as

where l ij ( x ) and u ij ( x ) are defined as

and

respectively. Note that the only difference between the definition of u ij ( x ) and l ij ( x ) is that u ij ( x ) permits the equality of ( ψ 0 ( x ) ) ( i ) with other components, while that is not the case for l ij ( x ) . This difference lets us find the first component with the largest value of ψ 0 ( x ) .

Now, we can rewrite sgn ( ( ℓ k ( x ) ) ( j ) ≥ max t ( ℓ k ( x ) ) ( t ) ) as the product

As shown in Lemma K.1, the class of such function has VC dimension of at most 4 . Furthermore, multiplying a hypothesis class with a function such as sgn ( ( ψ 0 ( x ) ) ( i ) ≥ ( ψ 0 ( x ) ) ( j ) ) and sgn ( ( ψ 0 ( x ) ) ( i ) > ( ψ 0 ( x ) ) ( j ) ) does not increase the VC dimension (See proof of Lemma K.1, and neither does negation. Therefore, in RHS of (66) we can count d number of functions, each with a hypothesis class with the VC dimension of at most 4 , and therefore using the early discussions in this proof (58), ( ˆ f k, 1 ( x ) ) ( i ) is within a function class with the VC dimension of at most 8 d log(3 d ) . Therefore, similar to (63) in previous part, we can bound sup k,p E ( k, 1) as

for l ≥ 8 d log(3 d ) with probability at least 1 -ϵ .

We can similarly, show that sup k,p E ( k, 0) is bounded as

Therefore, using (63), (64), (65), (67), (68), and the application Bonferonni's inequality we have

with probability at least 1 -ϵ . Therefore, by assuming E S n [ ⟨ f ∗ k,p ( x ) , ψ 1 ( x ) ⟩ ] ≤ α -d n ( ϵ ) , we assure that E µ [ ⟨ f ∗ k,p ( x ) , ψ 1 ( x ) ⟩ ] ≤ α , with probability at least 1 -ϵ , and this completes the proof.

L Proof of Theorem 5.3

We first introduce three lemmas that are useful in proving this theorem.

Lemma L.1. If δ is an ϵ -interior point of the set C = { E µ [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] : f ∈ ∆ X d } , then δ is ( ϵ/ 2) -interior point of D = { E S n [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] : f ∈ ∆ X d } with probability 1 -2 e -lϵ 2 4 .

Proof. The proof of this lemma is a direct application of Hoeffding's inequality. In fact, for ∥ ψ 1 ∥ ∞ ≤ C that inequality together with Hölder's inequality imply that

Therefore, if there exists f 1 such that E µ [ ⟨ f 1 ( x ) , ψ 1 ( x ) ⟩ ] = ϵ , then with probability at least 1 -e -nϵ 2 4 C 2 we have E S n [ ⟨ f 1 ( x ) , ψ 1 ( x ) ⟩ ] ∈ [ ϵ/ 2 , 3 ϵ/ 2] . Similarly, if f 2 exists such that E µ [ ⟨ f 1 ( x ) , ψ 1 ( x ) ⟩ ] = -ϵ , then with probability 1 -e -nϵ 2 4 C 2 we have E S n [ ⟨ f 2 ( x ) , ψ 1 ( x ) ⟩ ] ∈ [ -3 ϵ/ 2 , -ϵ/ 2] . As a result of Bonferroni's inequality, with probability at least 1 -2 e -nϵ 2 4 C 2 both these events happen, and because of the convexity of the set D we can say that with such probability all values between a 0 ∈ [ -3 ϵ/ 2 , -ϵ/ 2] and a 1 ∈ [ ϵ/ 2 , 3 ϵ/ 2] are in D too. This, of course at least contains the interval [ -ϵ/ 2 , ϵ/ 2] .

Lemma L.2. Assume that we have an approximation ˆ ψ 1 ( x ) of ψ 1 ( x ) with the error bounded as ∥ ˆ ψ 1 ( x ) -ψ 1 ( x ) ∥ ∞ ≤ ϵ . Further let ϵ ′ ∈ R + such that ϵ ′ ≥ ϵ . Now, if for σ ∈ {-ϵ ′ , ϵ ′ } there exists a rule f ∈ ∆ X d such that E µ [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] = δ + σ , then there exists k ∈ R as well as p ∈ [0 , 1] such that E µ [ ⟨ ˆ f k,p ( x ) , ˆ ψ 1 ( x ) ⟩ ] = δ + ϵ ′ -ϵ 2 .

Proof. Firstly, because of Hölder's inequality we know that

for all f ∈ ∆ X d . Therefore, by setting σ = ϵ ′ and σ = -ϵ ′ , we can show that for f 1 ∈ ∆ X d such that

Now, because of step (iii) of the proof of Theorem 4.1, we know that the set of constraints for all rules within ∆ X d is convex. Therefore, since we can achieve two points f 1 , f 2 such that the constraint E µ [ ⟨ f i ( x ) , ˆ ψ 1 ( x ) ⟩ ] can achieve two points above δ + ϵ ′ -ϵ and below δ -ϵ ′ + ϵ , then for each c ∈ [ δ -ϵ ′ + ϵ, δ + ϵ ′ -ϵ ] there exists f ∈ ∆ X d such that E µ [ ⟨ f ( x ) , ˆ ψ 1 ( x ) ⟩ ] = c . Now, let c = δ + ϵ ′ -ϵ 2 . In the following, we show that there exists k ∈ R and p ∈ [0 , 1] such that further E µ [ ⟨ ˆ f k,p ( x ) , ˆ ψ 1 ( x ) ⟩ ] = c .

To that end, we first remind that Lemma J.2 shows that E µ [ ⟨ ˆ f k, 0 ( x ) , ˆ ψ 1 ( x ) ⟩ ] is monotonically nonincreasing in terms of k . We show that for k ∈ R -we have max ˆ ψ 1 ( x ) -⟨ ˆ f k, 0 ( x ) , ˆ ψ 1 ( x ) ⟩ ≤ -2 k . The reason is that if j ∈ argmax l ( ˆ ψ 0 ( x ) -k ˆ ψ 1 ( x ) ) ( l ) and j ′ ∈ argmax l ( ˆ ψ 1 ( x ) ) ( l ) , then we have

then and where for f 2 ∈ ∆ X d

then which concludes that

Therefore, since

where the last inequality holds due to the existence of f 1 , then for k ≤ -8 / ( ϵ ′ -ϵ ) we have

Similarly, if we let k ≥ 8 / ( ϵ ′ -ϵ ) we can prove that

As a result, the set C = { k : E µ [ ⟨ ˆ f k, 0 ( x ) , ˆ ψ 1 ( x ) ⟩ ] ≥ c } is non-empty and bounded below by -8 ϵ ′ -ϵ . Therefore, its infimum exists and is also bounded below by -8 ϵ ′ -ϵ . Let us name that infimum ˆ k . Now, if E µ [ ⟨ ˆ f k, 0 , 0 ( x ) , ˆ ψ 1 ( x ) ⟩ ] is continuous at k = ˆ k , then we can show that E µ [ ⟨ ˆ f ˆ k, 0 ( x ) , ˆ ψ 1 ( x ) ⟩ ] = c . If not, then as shown in step (ii) of the proof of Theorem 4.1, and in particular in (57), there exists p such that E µ [ ⟨ ˆ f ˆ k,p ( x ) , ˆ ψ 1 ( x ) ⟩ ] = c . This completes the proof.

Lemma L.3. If ∥ ˆ ψ 0 -ψ 0 ∥ ∞ ≤ δ 0 and ∥ ˆ ψ 1 -ψ 1 ∥ ∞ ≤ δ 1 , and for k ∈ [ -K,K ] , and k ′ ≤ k -2( δ 0 + Kδ 1 ) T for T ∈ R + , then we have

Proof. The proof of this lemma bears similarity to that of Lemma J.2. Here too, we define ˆ ℓ k ( x ) = ˆ ψ 0 ( x ) -k ˆ ψ 1 ( x ) . Next, we have

Next, we need to show that ( ψ 1 ( x ) ) ( j 1 ) = ⟨ r k ′ , 0 ( x ) , ψ 1 ( x ) ⟩ ≥ ⟨ ˆ f k, 0 , 0 ( x ) , ψ 0 ( x ) ⟩ -T = ( ψ 0 ( x ) ) ( j 2 ) -T . Assume otherwise, meaning that ( ψ 1 ( x ) ) ( j 1 ) < ( ψ 1 ( x ) ) ( j 2 ) -T . In this case, we have

which is a contradiction. Note that ( a ) holds because of definition of j 2 and (71), ( b ) holds due to approximation assumptions ∥ ˆ ψ 0 -ψ 0 ∥ ∞ ≤ δ 0 and ∥ ˆ ψ 1 -ψ 1 ∥ ∞ ≤ δ 1 , ( c ) holds because of the assumption ( ψ 1 ( x ) ) ( j 1 ) < ( ψ 1 ( x ) ) ( j 2 ) -T , ( d ) is followed by the definition of j 1 on maximizing ℓ k ′ ( x ) , and ( e ) holds because k ≥ k ′ + 2( δ 0 + Kδ 1 ) T , and ( f ) is followed by approximation assumptions.

We first formally express Theorem 5.3 as following:

Theorem L.4. Assume that ( δ -ϵ l , δ + ϵ u ) is a subset of of all achievable constraints E [ ⟨ f ( x ) , ψ 1 ( x ) ⟩ ] , and that ∥ ψ i ( x ) ∥ ∞ ≤ 1 for i = 1 , 2 . Further, let the size n of validation data be large enough such that d n ( δ/ 3) ≤ ϵ l 2 . Now, if the optimal predictor f ∗ k, 0 ( x ) is ( γ , ∆ )- sensitive around optimal k ∗ for ∆ ≥ ( 2 max { d n ( δ/ 3) ,δ 1 } + √ 2 γC ( δ 0 + Kδ 1 ) ) C 1 /γ and γ ≤ 1 , then for n ≥ 16 ϵ 2 l log 3 δ , and with probability at least 1 -δ , the optimal empirical classifier, as of Algorithm 1 has an objective that is at most D 0 -far from the true optimal objective where D 0 is defined as

where K is an upper-bound to the absolute value of k ∗ .

In order to prove this theorem, we first define a measure of distance between two rules f 1 , f 2 ∈ ∆ R d as

Using this measure of distance, the difference of objectives between two rules f 1 and f 2 can be written as

Therefore, if two rules achieve similar constraints, and if D k ( f 1 , f 2 ) is small enough, we can prove that the two rules achieve similar objectives too, since k is bounded above by K .

In fact, if we let f 1 ( x ) = f ∗ k,p ( x ) and f 2 ( x ) := ˆ f ˆ k, ˆ p , where k and p are optimal solutions as in Theorem 4.2, then due to this optimality, and because E [ ⟨ ˆ f ˆ k, ˆ p , ψ 1 ( x ) ⟩ ] ≤ δ with probability at least 1 -ϵ as shown in Theorem 5.1, then LHS of (74) is positive with at least the same probability. In this proof, we show that how large is that term, and therefore, we show that how sub-optimal is ˆ f ˆ k, ˆ p in terms of the objective.

To that end, we first bound the difference between constraints. This bound can be achieved similar to the proof of Theorem 5.1. In fact, there we showed that if the empirical constraint E S n [ ⟨ ˆ f ˆ k, ˆ p , ψ 1 ( x ) ⟩ ] ≤ δ -d n ( π ) , then using (69) the true expectation is bounded as E µ [ ⟨ ˆ f ˆ k, ˆ p , ψ 1 ( x ) ⟩ ] ≤ δ with probability at least 1 -π . However, (69) is symmetric in empirical and true constraint, i.e., if we show that E S n [ ⟨ ˆ f ˆ k, ˆ p , ψ 1 ( x ) ⟩ ] ≥ δ -d n ( π ) , then we have E µ [ ⟨ ˆ f ˆ k, ˆ p , ψ 1 ( x ) ⟩ ] ≥ δ -2 d n ( π ) with probability at least 1 -π .

To show E S n [ ⟨ ˆ f ˆ k, ˆ p , ψ 1 ( x ) ⟩ ] ≥ δ -d n ( π ) , we follow three steps, (i) because δ is ( ϵ l , ϵ u ) -interior point of the set of constraints, i.e., ( δ -ϵ l , δ + ϵ u ) is a subset of all plausible constraints, then δ -d n ( π ) is ( ϵ l -d n ( π ) , ϵ u + d n ( π )) -interior point. Now, using Lemma L.1 and by setting ϵ ′ = min { ϵ l -d n ( π ) , ϵ u + d n ( π ) } we can show that δ -d n ( π ) is ϵ ′ / 2 -interior point of the empirical constraints with probability at least 1 -2 e -nϵ ′ 2 4 , (ii) using the first step and assuming d n ( π ) ≤ ϵ l / 2 we conclude that δ -d n ( π ) is d n ( π ) / 2 -interior point of the empirical constraints with the aforementioned probability, (iii) because of Lemma L.2, we conclude that for ϵ = d n ( π ) / 2 , and with probability at least 1 -2 e -nϵ ′ 2 4 there exists k ∈ R and p ∈ [0 , 1] such that E S n [ ⟨ ˆ f k,p ( x ) , ˆ ψ 1 ( x ) ⟩ ] = δ -d n ( π ) + d n ( π ) / 2 -ϵ 2 = δ -d n ( π ) . As a result of the above discussion we conclude that with probability at least 1 -π -2 e -nϵ ′ 2 4 there exists k and p such that δ ≥ E [ ⟨ ˆ f k,p ( x ) , ψ 1 ( x ) ⟩ ] ≥ δ -2 d n ( π ) . Now, since we know that E [ ⟨ f 1 ( x ) , ψ 1 ( x ) ⟩ ] = E [ ⟨ f ∗ k,p ( x ) , ψ 1 ( x ) ⟩ ] = δ , then we have

with probability at least 1 -π -2 e -nϵ ′ 2 4 .

The above discussion together with (74) and the assumption of boundedness of k shows that the difference of objectives is bounded with a high probability, if we bound D k ( f 1 , f 2 ) . However, before we proceed with bounding that term, we should derive a relationship between ˆ k and k ∗ for the reasons that we see in proving boundedness of D k ( f 1 , f 2 ) .

We have already shown that there exists ˆ p ∈ [0 , 1] such that δ ≥ E [ ⟨ ˆ f ˆ k, ˆ p , ψ 1 ( x ) ⟩ ] ≥ δ -2 d l ( π ) . Here, Lemma L.3 shows that for k ′ = k -2( δ 0 + Kδ 1 ) T we have E [ ⟨ f ∗ k ′ , 0 , ψ 1 ( x ) ⟩ ] ≥ δ -2 d l ( π ) -T with probability at least 1 -π -2 e -nϵ ′ 2 4 . Moreover, using symmetry in Lemma L.3 and for k ′′ = k + 2( δ 0 + Kδ 1 ) T we have E [ ⟨ f ∗ k ′′ , 0 ( x ) -ˆ f k ( x ) , ˆ ψ 1 ( x ) ⟩ ] ≤ T . Now, since ∥ ψ 1 ( x ) -ˆ ψ 1 ( x ) ∥ ∞ ≤ δ 0 , using Hölder's inequality we conclude that E [ ⟨ f ∗ k ′′ , 0 ( x ) -ˆ f k ( x ) , ˆ ψ 1 ( x ) ⟩ ] ≤ T + 2 δ 1 , and consequently E [ ⟨ f ∗ k ′′ , 0 ( x ) , ˆ ψ 1 ( x ) ⟩ ] ≤ δ + T +2 δ 0

Now that we have found a lower-bound on constraint of the rule f ∗ k -q ( x ) for q = 2( δ 0 + Kδ 1 ) T , then if we find an upper bound on the constraint of the rule f ∗ k ∗ + e ( x ) for an e ∈ R + , then we can use monotonicity of the constraint of f ∗ k in terms of k and prove a relationship between k and k ∗ . To that end, we use detection assumption with which we can show that

where we assume that d n ( π ) ≤ ( C ∆) γ -T 2 . Now, using previous discussions conclude that

with probability at least 1 -π -2 e -nϵ ′ 2 4 . This together with the first part of Lemma J.2 shows that k ′ ≤ k ∗ + 1 C (2 d n ( π ) + T ) 1 /γ , or equivalently k ≤ k ∗ + 2( δ 0 + Kδ 1 ) T + 1 C (2 d n ( π ) + T ) 1 /γ with probability at least 1 -π -2 e -nϵ ′ 2 4 . Since we know that γ is clamped above by 1 , and using the inequality (1 + x ) a ≤ 1 + ax for a ≥ 1 we can substitute the above inequality with k ≤ k ∗ + 2( δ 0 + Kδ 1 ) T + ( 2 d n ( π ) ) 1 /γ C ( 1 + T γ (2 d n ( π )) 1 /γ ) . Now optimizing over T leads in T = √ 2 γC ( δ 0 + Kδ 1 ) , which concludes that k ≤ k ∗ +∆ u k with the aforementioned probability, where 1 /γ √

Similarly, using sensitivity assumption, we have

where (2 δ 1 + T ) 1 /γ C ≤ ∆ . Next, using previous discussions conclude that

with the aforementioned probability. This, again, together with the first part of Lemma J.2 shows that k ′′ ≥ k ∗ -1 C (2 δ 1 + T ) 1 /γ , or equivalently k ≥ k ∗ -1 C (2 δ 1 + T ) 1 /γ -2( δ 0 + Kδ 1 ) T . Therefore, by setting T = √ 2 γC ( δ 0 + Kδ 1 ) we conclude that k ≥ k ∗ -∆ l k where ∆ l k = (2 δ 1 ) 1 /γ C +2 √ 2( δ 0 + Kδ 1 ) γC , and assuming ( 2 δ 1 + √ 2 γC ( δ 0 + Kδ 1 ) ) 1 /γ C ≤ ∆ .

Next, we turn into bounding D k ∗ ( f 1 , f 2 ) . To that end, we first note that

for all p ∈ [0 , 1] . This is followed by the definition of f ∗ k ∗ ,p ( · ) . Similarly, we can show that

for all

where ( a ) and ( b ) hold due to Hölder's inequality.

Next, we show Lipschitzness of t ( k ) using its structure. In fact, due to its definition, t ( k ) is the maximum of a set of lines with { t i = ( ψ 0 ( x ) ) ( i ) -k ( ψ 1 ( x ) ) ( i ) } n +1 i =1 in terms of k with slope m i = -( ψ 1 ( x ) ) ( i ) and y -intercept of b i = ( ψ 0 ( x ) ) ( i ) . Therefore, such piecewise-linear function has a Lipschitz factor equal to the maximum slope of the lines, which in here is equal to max i m i = max i ∣ ∣ ∣ ( ψ 1 ( x ) ) ( i ) ∣ ∣ ∣ ≤ 1 . Therefore, t ( k ) is a 1 -Lipschitz function. Therefore, using (77) we can bound D k ∗ ( f 1 , f 2 ) as

where ( a ) holds because each component of ( ψ 0 ( x ) -ˆ kψ 1 ( x ) ) and ( ˆ ψ 0 ( x ) -ˆ k ˆ ψ 1 ( x ) ) is bounded by δ 0 + Kδ 1 , and because the maximum operator is a norm, and therefore satisfies sub-additivity. Finally, since we have bounded ∆ ≤ k ∗ -ˆ k ≤ ∆ l with probability at least 1 -π -2 e -nϵ ′ 2 / 4 , then we have

with such probability. This, together with (74) and (75) shows that

which completes the proof.

M Proof of Theorem G.1

To prove this theorem, we first prove the following auxiliary lemma

Lemma M.1. For α, ϵ ≥ 0 , the following holds

Proof of lemma. We know that for every positive vector r with ∑ n i =1 r i ≤ α + ϵ , we could rewrite that as a sum of two vectors r = r ′ + r ′′ for which

As a result, we can rewrite min r i ≥ 0 , ∑ n i =1 r i ≤ α + ϵ ∑ n i =1 r i d i as

Hence, we have that

where ( a ) holds using Hölder's inequality.

Next, we know that the optimal deterministic deferral policy should satisfy

and

where ( a ) holds because r ( x i ) ∈ { 0 , 1 } and therefore ∑ r ( x i ) ≤ bn if and only if ∑ r ( x i ) ≤ ⌊ bn ⌋ . Now, we turn to examining B . To that end, we first consider the following optimization problem:

For a minimizer r ∗ of the above problem, we could form ˆ r as

One can see that ˆ r ( x ) is also a minimizer of the above problem. Hence, without loss of generality, we assume that there is an optimal deferral policy that has only non-zero value when ( x, y, m ) ∈ A = { ( x, y, m ) ∈ D : } . Furthermore, we know that since ˆ r ( x i ) ≤ 1 , then ∑ i ˆ r ( x i ) ≤ min {⌊ nb ⌋ , | A |} . We argue that this inequality does not hold in a strict form, i.e., we have ∑ i ˆ r ( x i ) = min {⌊ nb ⌋ , | A |} . The reason is that otherwise one can find r ′ ( x ) ∈ [0 , 1] X such that ∑ ( x i ,y i ,m i ) ∈ A ˆ f ( x i ) + r ′ ( x i ) = min { nb, | A |} and because of negativity of ℓ H ( x i , y i , m i ) -ℓ AI ( x i , y i ) , we can strictly reduce the objective function that is a contradiction.

Next, we order ℓ H ( x i , y i , m i ) -ℓ AI ( x i , y i ) increasingly and we name them d j . In fact, we define k j such that d j = ℓ H ( x k j , y k j , m k j ) -ℓ AI ( x k j , y k j ) and that d 1 ≤ d 2 . . . ≤ d | A | ≤ 0 . For the sake of simplicity, we further define r j := r ( x k j ) . As a result, the optimization problem in (78) can be rewritten as

Here, we show that the optimizer of the above problem is r i = 1 i ≤ min {⌊ nb ⌋ , | A |} . To show that, we consider r ′ i ∈ [0 , 1] such that ∑ n i =1 r ′ i = min {⌊ nb ⌋ , | A |} . Then, we have

Now, since we know that ∑ i 1 i ≤ min {⌊ nb ⌋ , | A |} = ∑ i r ′ i , we can define a parameter Q as

Next, by defining p i := 1 i ≤ min {⌊ nb ⌋ , | A |} -r ′ i Q for i s in which 1 i ≤ min {⌊ nb ⌋ , | A |} -r ′ i > 0 and q i := r ′ i 1 i ≤ min {⌊ nb ⌋ , | A |} Q for i s in which 1 i ≤ min {⌊ nb ⌋ , | A |} -r ′ i < 0 and 0 otherwise, we conclude that { p i } i and { q i } i are probability mass functions. Hence, using (79) and (80), we have

The above identity contains the difference of two expected value over random variables that one is always smaller than the other. As a result, we show that

which completes the proof.

Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: Yes

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: Yes

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: Yes

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: Yes

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: Yes

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: Yes

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: Yes

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: Yes

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: NA

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: NA

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: NA

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: Yes

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: Yes

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: NA

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: NA

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWhile there has been extensive research on L2D, general methods for designing such systems under various constraints (e.g., algorithmic fairness, expert intervention budget, defer of anomaly, etc.) remain largely unexplored. This paper utilizes $d$-dimensional generalization to the fundamental lemma of Neyman and Pearson ($d$-GNP) to obtain Bayes optimal solutions for L2D under various constraint conditions and designs a generic algorithm to estimate these solutions.\n\nSTRENGTHS:\n1. This paper proposes a novel general framework for addressing L2D under various constraint conditions.\n2. Different constraint conditions can be simultaneously addressed.\n3. The proposed method is a post-processing approach that can be easily extended to existing models.\n\nWEAKNESSES:\n1. The general framework provided by the author can address various constraint problems, but it does not seem to be reflected in the experimental section (especially in cases involving multiple constraints), only considering simple single-constraint situations and basic comparisons.\n\nExpert Intervention Budget: \n\nLiu S, Cao Y, Zhang Q, et al. Mitigating Underfitting in Learning to Defer with Consistent Losses. ICML, 2024.\n\nNarasimhan H, Jitkrittum W, Menon A K, et al. Post-hoc estimators for learning to defer to an exper. NeurIPS, 2022.\n\nLong-Tail Classification:\n\nNarasimhan H, Menon A K, Jitkrittum W, et al. Learning to Reject Meets Long-tail Learning. ICLR, 2024.\n\nQUESTIONS:\n1. Post-Processing Framework has many benefits, but it also faces challenges in confidence calibration, which is particularly evident in deep models. Inevitable overfitting makes the model's output probabilities unreliable, indirectly leading to biases in approaching the Bayes optimal solution through post-processing methods. Does the method proposed in this paper include mechanisms to mitigate these issues?\n2. There are many studies similar to L2D, such as the dynamic classifier selection literature (DCS). What is the main difference between L2D and DCS?\n\nCruz R M O, Sabourin R, Cavalcanti G D C. Dynamic classifier selection: Recent advances and perspectives[J]. Information Fusion, 2018, 41: 195-216.\n\nLIMITATIONS:\nN/A.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies multi-objective learn-to-defer problems, where the objectives include minimizing deferral loss and satisfying several constraints. It demonstrates that these problems are generally NP-hard and can be reduced to functional linear programming. Additionally, it shows that the problem can be further reduced to a d-dimensional generalized Neyman-Pearson problem and characterizes its solution when there is only one constraint. The paper also presents a unifying post-processing algorithm with generalization bounds and provides numerical validations.\n\nSTRENGTHS:\nThe paper offers several theoretical insights into multi-objective learn-to-defer problems, enhancing the understanding of these problems. It also introduces a unifying post-processing algorithm with provable performance guarantees.\n\nWEAKNESSES:\n1.Inconsistent Notations and Typos: The paper is difficult to follow due to inconsistent notations and potential typos. For instance, $m$ can mean the expert decision (line 111) or the number of constraints (Equation 3). $r$ can mean the rejection function (line 110) or a distribution vector (Theorem 4.1).  In line 7 of Algorithm 1, should it be $\\hat{C}(k)$ instead of $\\hat{C}(t)$? And line 208 only defines  $\\psi_{i}$ for $i=1,...,m+1$, then in line 209, what is $\\psi_{0}$?\n\n2.Limitations on Multiple Constraints: The paper acknowledges that the provided analyses for the algorithm (and possibly the algorithm itself) only apply to single constraint settings. More discussion on the challenges of extending these analyses (and potentially the algorithm) to multiple constraints settings would be helpful.\n\nQUESTIONS:\n1.Why in Theorem 4.2, $f^*_{k,p}(x)=\\tau(\\psi_1(x)-k\\psi_0(x))$, while in Algorithm 1, its estimation version is $\\hat{f}_{k,p}(x)=\\tau(\\hat{\\psi}_0(x)-k\\hat{\\psi}_1(x))$ (where the positions are swapped)?\n\n\n2.Can Algorithm 1 handle multiple constraints settings? Figure 1 depicts a diagram of multiple constraints settings, but it seems Algorithm 1 is designed for single constraint settings.\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a unifying post-processing framework for multi-objective learn-to-defer problems, allowing the system to defer tasks to an expert under specified constraints. By generalizing the Neyman-Pearson lemma, the paper derives the Bayes optimal solution for this framework and develops an algorithm to estimate it. The proposed algorithm is evaluated using the COMPAS and ACSIncome datasets.\n\nSTRENGTHS:\n1. The paper introduces a general post-processing framework.\n2. The paper is well-written and has excellent illustrations, for example, Figure 1.\n3. The framework accounts for multiple objectives in L2D.\n4. The method can potentially be extended beyond the L2D setting to apply to other constrained objectives.\n\nWEAKNESSES:\n1. The proposed algorithm requires the estimation of scores and $\\hat \\psi$ (Lines 4 and 5 of Algorithm 1). However, accurately estimating these values can be potentially challenging.\n2. The experiments could potentially be expanded further.\n\nQUESTIONS:\n1. The multi-class Neyman-Pearson (NP) paradigm has been studied in [66]. Could the authors summarize the key differences between the framework proposed in this paper and the prior work?\n\n2. What is the motivation for imposing specific constraints in learning to defer? Could this compromise the accuracy of the process?\n\n3. How does the proposed algorithm empirically compare with the baselines on the ACSIncome datasets? Do you think the experiments could be expanded further?\n\n4. It seems that the method could potentially be applicable to other constrained objectives beyond the L2D setting. Could the authors further elaborate on this?\n\n5. Could the authors further comment on the challenges of extending the analysis and algorithm to multiple constraint cases?\n\n6. How are the tolerances chosen during the experiments? Can the assumptions in Theorem 5.3 be verified in practice?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to provide a provably consistent unified post-processing framework of learning to defer with constraints. The problem of constrained L2D is first reduced to a linear programming problem. Then the linear programming problem is further tackled with a generalized version of Neymar-Pearson lemma, which lead to an efficient solver of this problem. Non-asymptotic analysis is conducted to further provide guarantee for the empirical version of the algorithm. Experimental results validate the efficiency of the proposed solver.\n\nSTRENGTHS:\n1. A unified post-processing framework for constrained L2D is proposed in this paper, which allows training a randomized classifier-rejector tuple with small number of validation data combined with a trained model’s confidence output.\n\n2. The hardness of directly solving the original problems and in-processing methods are thoroughly analyzed in this paper, which provides enough rationale for the proposed method.\n\n3. The introduction of the generalized Neyman-Pearson lemma is novel to the field of L2D, which can provide insights for the future works of this field.\n\nWEAKNESSES:\n1. The related works are moved to the appendices, which can be confusing to readers that are new to this field. I suggest the authors use a more compact type setting in the contribution part of Section 1 and reduce/integrate some of the contributions to make more spaces. The paragraph ‘Type of Constraints’ can also be reduced given the Table 1.\n\n2. The notations in this paper need further proofread, e.g., the realization and random variables are used improperly in line 141, line 188; the quantity $a^{i}$ in the algorithm 1 seems to be unused in this algorithm.\n\nQUESTIONS:\n1. It is mentioned that solving (2) is NP-hard. While the proof is quite clear, I wonder if such hardness is important. In my opinion, a common practice that can avoid directly solving this problem is using a surrogate loss instead of 0-1 loss and integrating the expert coverage constraint into the loss like that in the Selective-net. Can you further make some discussions on this point?\n\n2. If we have rather accurate class probability estimates and expert accuracy, why not use it as the solver of this problem?\n\nLIMITATIONS:\nPlease see the weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-28"}