SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. To prove the above claim, we construct a monotone confidence function  $f_{H}$ , perfectly calibrated confidence function  $f_{B}$  and distribution  $P^{\mathcal{M}}$  for which any monotone AI-assisted decision policy  $\pi \in \Pi(H, B)$  achieves strictly lower utility than a carefully constructed non-monotone AI-assisted decision policy  $\tilde{\pi} \in \Pi(H, B)$ .
We will present the proof in three parts. First, we will introduce the main building block and idea behind the proof by a small construction of  $f_{H}, f_{B}$  and  $P^{\mathcal{M}}$  with  $|\mathcal{H}| = |\mathcal{B}| = 3$ , where  $\mathcal{B} \subseteq [0,1]$  denotes the (discrete) output space of the classifier's confidence function. We then construct examples of  $f_{H}, f_{B}$  and  $P^{\mathcal{M}}$  for arbitrary  $|\mathcal{H}| = k$  and  $|\mathcal{B}| = m$  with  $m, k \in \mathbb{N}$ ,  $m > k \geq 2$ . Lastly, we construct examples where  $\mathcal{B}$  is non-discrete and  $|\mathcal{H}| = k$  with  $k > 2$ .
Main building block and small example.
We start by presenting the main idea of the proof using an example with a small set of confidence values  $\mathcal{H}$  and  $\mathcal{B}$ . Let the values of the decision maker's confidence  $H$  be in  $\mathcal{H} = \{h_1, h_2, h_3\}$  and the values of the classifier's confidence  $B$  be in  $\mathcal{B} = \{b_1, b_2, b_3\}$ , with order  $h_i < (h_i + 1)$  and  $b_i < (b_i + 1)$  respectively.
Our main building block, consists of two distributions  $P^{-}, P^{+} \in \mathcal{P}(\{0,1\})$  with  $P^{-}(Y = 1) < c$  and  $P^{+}(Y = 1) > c$ , where  $c$  depends on utility  $u$  as described by Eq. 8 in Lemma 2. We use these distributions for our constructions of  $f_{H}, f_{B}$  and  $P^{\mathcal{M}}$ , so that for some realizations of  $H, B$  distribution  $P(Y = 1 \mid H, B)$  is either  $P^{-}$  or  $P^{+}$ . Using Lemma 2 and from Eq. 10, we have that:
(I) For any  $h_i, b_i$  such that  $P(Y \mid H = h_i, B = b_i) = P^-$ , it holds that
$$
\mathbb {E} [ u (1, Y) \mid H = h _ {i}, B = b _ {i} ] <   \mathbb {E} [ u (0, Y) \mid H = h _ {i}, B = b _ {i} ].
$$
Hence, decreasing  $P_{\pi}(T = 1 \mid H, B)$  increases  $\mathbb{E}[u(T, Y) \mid H = h_i, B = b_i]$ .
(II) For any  $h_i, b_i$  such that  $P(Y \mid H = h_i, B = b_i) = P^+$ , it holds that
$$
\mathbb {E} [ u (1, Y) \mid H = h _ {i}, B = b _ {i} ] > \mathbb {E} [ u (0, Y) \mid H = h _ {i}, B = b _ {i} ].
$$
Hence, increasing  $P_{\pi}(T = 1 \mid H, B)$  increases  $\mathbb{E}[u(T, Y) \mid H = h_i, B = b_i]$ .
Intuitively, suppose we now have that, for confidence values  $h_2, b_2, Y \sim P^+$  and, for confidence values  $h_3, b_2, Y \sim P^-$ , i.e.,  $P(Y \mid H = h_2, B = b_2) = P^+$  and  $P(Y \mid H = h_3, B = b_2) = P^-$ . Then, any non-monotone AI-assisted decision policy  $\tilde{\pi}$  with  $P_{\tilde{\pi}}(T = 1 \mid H = h_2, B = b_2) > P_{\tilde{\pi}}(T = 1 \mid H = h_3, B = b_2)$  will have higher expected utility than any monotone AI-assisted decision policy given confidence values  $h_2, b_2$  and  $h_3, b_2$ . Finally, under an appropriate choice of distribution  $P(H, B)$ , such non-monotone AI-assisted decision policies  $\tilde{\pi}$  will offer higher overall utility in expectation.
We formalize this intuition with the following lemma:
Lemma 3. Let  $\mathcal{M}$  be any AI-assisted decision making process satisfying Eqs. 2 and 3, with utility function  $u(T,Y)$  satisfying Eq. 1. If  $f_{H}, f_{B}$  and  $P^{\mathcal{M}}$  are such that there exists confidence values  $b \in \mathcal{B}, h_i, h_j \in \mathcal{H}$ , with  $h_i < h_j$ , which satisfy
$$
P (H = h _ {i}, B = b) > 0, \quad P (H = h _ {j}, B = b) > 0,
$$
$$
P (Y \mid H = h _ {i}, B = b) = P ^ {+} \quad a n d \quad P (Y \mid H = h _ {j}, B = b) = P ^ {-}, \tag {11}
$$
for some distributions  $P^{-}, P^{+}$  with  $P^{-}(Y = 1) < c$  and  $P^{+}(Y = 1) > c$ , where
$$
c = \frac {u (0 , 0) - u (1 , 0)}{u (1 , 1) - u (1 , 0) + u (0 , 0) - u (0 , 1)}. \tag {12}
$$
Then, for any monotone AI-assisted decision policy  $\pi \in \Pi (H,B)$ , there exists an AI-assisted decision policy  $\tilde{\pi}\in \Pi (H,B)$  which is not monotone and achieves a strictly greater utility than  $\pi$ , i.e.,  $\mathbb{E}_{\pi}[u(T,Y)] < \mathbb{E}_{\tilde{\pi}}[u(T,Y)]$ .
Proof. Let  $\pi$  be a monotone AI-assisted decision policy, then it must hold that  $P_{\pi}(T = 1 \mid H = h_i, B = b) \leq P_{\pi}(T = 1 \mid H = h_j, B = b)$  (see Eq. 4). Let  $\tilde{\pi}$  be an identical AI-assisted decision
policy to  $\pi$  up to the decision for confidence values  $h_i, b$  and  $h_j, b$ . We distinguish between three cases.
— Case 1:  $P_{\pi}(T = 1 \mid H = h_i, B = b) < P_{\pi}(T = 1 \mid H = h_j, B = b)$ .
Let the probability of  $T = 1$  under  $\tilde{\pi}$  for confidence values  $h_i, b$  and  $h_j, b$  be switched compared to  $\pi$ , i.e.,
$$
P _ {\pi} (T = 1 \mid H = h _ {i}, B = b) = P _ {\pi} (T = 1 \mid H = h _ {j}, B = b),
$$
$$
P _ {\bar {\pi}} (T = 1 \mid H = h _ {j}, B = b) = P _ {\pi} (T = 1 \mid H = h _ {i}, B = b).
$$
Then,  $\tilde{\pi}$  is not monotone, as Eq. 4 is not satisfied, and it holds that
$$
P _ {\pi} (T = 1 \mid H = h _ {i}, B = b) > P _ {\pi} (T = 1 \mid H = h _ {i}, B = b),
$$
$$
P _ {\bar {\pi}} (T = 1 \mid H = h _ {j}, B = b) <   P _ {\pi} (T = 1 \mid H = h _ {j}, B = b).
$$
As we decreased  $P(T = 1 \mid H = h_j, B = b)$  and increased  $P(T = 1 \mid H = h_i, B = b)$ , by properties (I) and (II), it must hold that the expected utility of  $\tilde{\pi}$  given confidence values  $h_i, b$  and  $h_j, b$  is higher than the one of  $\pi$ , i.e.,
$$
\mathbb {E} _ {\hat {\pi}} [ u (T, Y) \mid H = h _ {i}, B = b ] > \mathbb {E} _ {\pi} [ u (T, Y) \mid H = h _ {i}, B = b ] \quad \text {a n d} \tag {13}
$$
$$
\mathbb {E} _ {\pi} [ u (T, Y) \mid H = h _ {j}, B = b ] > \mathbb {E} _ {\pi} [ u (T, Y) \mid H = h _ {j}, B = b ]. \tag {14}
$$
Case 2:  $0 < P_{\pi}(T = 1 \mid H = h_i, B = b) = P_{\pi}(T = 1 \mid H = h_j, B = b) \leq 1.$
Let the probability of  $T = 1$  under  $\tilde{\pi}$  for confidence values  $h_j, b$  be strictly lower compared to  $\pi$  and be the same as  $\pi$  for  $h_i, b$ . Then,  $\tilde{\pi}$  is not monotone, since by case assumption
$$
P _ {\tilde {\pi}} (T = 1 \mid H = h _ {i}, B = b) = P _ {\pi} (T = 1 \mid H = h _ {j}, B = b) > P _ {\tilde {\pi}} (T = 1 \mid H = h _ {j}, B = b)
$$
and the inequality in Eq. 14 holds by property (I).
Case 3:  $P_{\pi}(T = 1 \mid H = h_i, B = b) = P_{\pi}(T = 1 \mid H = h_j, B = b) = 0$ .
Let the probability of  $T = 1$  under  $\tilde{\pi}$  for confidence values  $h_i, b$  be strictly higher compared to  $\pi$  and be the same as  $\pi$  for  $h_j, b$ . Then,  $\tilde{\pi}$  is not monotone, since by case assumption
$$
P _ {\bar {\pi}} (T = 1 \mid H = h _ {j}, B = b) = P _ {\pi} (T = 1 \mid H = h _ {i}, B = b) <   P _ {\bar {\pi}} (T = 1 \mid H = h _ {i}, B = b)
$$
and the inequality in Eq. 13 holds by property (II).
As in all three cases at least one of the strict inequalities in Eqs. 13 or 14 holds and  $\tilde{\pi}$  is equivalent to  $\pi$  (i.e., it has the same expected conditional utility) given any other pair of confidence values  $h' \in \mathcal{H}, b' \in \mathcal{B}$ , we have that
$$
\mathbb {E} _ {\bar {\pi}} [ u (T, Y) ] = \mathbb {E} [ \mathbb {E} _ {\bar {\pi}} [ u (T, Y) ] | H, B ] > \mathbb {E} [ \mathbb {E} _ {\pi} [ u (T, Y) | H, B ] = \mathbb {E} _ {\pi} [ u (T, Y) ].
$$
Before proceeding further, we would like to note that we may also state Lemma 3 using  $h \in \mathcal{H}$ ,  $b_{i}, b_{j} \in \mathcal{B}$ , with  $b_{i} < b_{j}$ , the proof would follow analogously.
Now, we construct an AI-decision making process  $\mathcal{M}$ , with  $\mathcal{H} = \{h_1, h_2, h_3\}$  and  $\mathcal{B} = \{b_1, b_2, b_3\}$ , such that the decision maker's confidence  $f_H$  is monotone, the classifier's confidence  $f_B$  is perfectly calibrated, and the conditions of Lemma 3 are satisfied. First, let  $f_H, f_B$  and  $P^{\mathcal{M}}$  be such that
$$
P (f _ {B} (Z) = b _ {j}) = \left\{ \begin{array}{l l} 3 / 6 & \text {i f} j = 1 \\ 2 / 6 & \text {i f} j = 2 \\ 1 / 6 & \text {i f} j = 3 \\ 0 & \text {o t h e r w i s e} \end{array} \right. \quad \text {a n d}
$$
$$
P (H = h _ {i} \mid B = b _ {j}) := P _ {X, V} (H = h _ {i} \mid f _ {B} (Z) = b _ {j}) = \left\{ \begin{array}{l l} \frac {1}{4 - j} & \text {i f i \geq j} \\ 0 & \text {o t h e r w i s e .} \end{array} \right.
$$
Then, it readily follows that  $P(H = h_i, B = b_j) = 1/6$  for  $i \geq j$  and  $P(H = h_i, B = b_j) = 0$  otherwise. Moreover, for each pair of confidence values  $(h_i, b_j)$  with positive probability  $P(H = h_i, B = b_j)$ , we set
$$
P (Y = 1 \mid H = h _ {i}, B = b _ {j}) = \left\{ \begin{array}{l l} P ^ {+} & \text {i f} i = j = 2 \text {o r} (i = 3 \text {a n d} j \in \{1, 3 \}) \\ P ^ {-} & \text {i f} (j = 2 \text {a n d} i = 3) \text {o r} (j = 1 \text {a n d} i \in \{1, 2 \}), \end{array} \right.
$$
Figure 4: Nonzero values of  $P(Y = 1|H = h_i, B = b_j)$  and  $P(H = h_i, B = b_j)$  for every  $h_i \in \mathcal{H}$  and  $b_j \in \mathcal{B}$  used in the first (left) and second (right) part of the proof of Theorem 3. In each cell  $(h_i, b_j)$  in both panels,  $P^+$  or  $P^-$  is the value of  $P(Y = 1|H = h_i, B = b_j)$  and lighter color means lower value of  $P(H = h_i, B = b_j)$ , where white means  $P(Y = 1|h = h_i, B = b_j) = 0$  and  $P(H, B) = 0$ . In both panels, the assignment of values is very stylized to facilitate the proof—the classifier's confidence function  $f_B$  partitions the feature space in a way such that a rational decision maker is unable to take decisions that maximize utility for almost all confidence values. However, less stylized examples also satisfy the conditions of Lemma 3. For example, as long as there is one triplet of confidence values  $b_2, h_2, h_3$  (or  $h_3, b_1, b_2$  in the left example) for which a rational decision maker is unable to take decisions that maximize utility, Lemma 3 can be applied.
as shown in Figure 4 (left). Then, it readily follows that  $f_{H}$  is monotone with respect to the probability that  $Y = 1$ , i.e.,  $P(Y = 1 \mid H = h_{i}) \leq P(Y = 1 \mid H = h_{i + 1}))$ , and we have that the classifier's confidence values
$$
\begin{array}{l} b _ {j} := \sum_ {i: i \geq j} P (H = h _ {i} \mid B = b _ {j}) \cdot P (Y = 1 \mid H = h _ {i}, B = b _ {j}) \\ = \left\{ \begin{array}{l l} 2 / 3 \cdot P ^ {-} + 1 / 3 \cdot P ^ {+} & \text {i f} j = 1 \\ 1 / 2 \cdot P ^ {-} + 1 / 2 \cdot P ^ {+} & \text {i f} j = 2 \\ P ^ {+} & \text {i f} j = 3 \\ 0 & \text {o t h e r w i s e} \end{array} \right. \\ \end{array}
$$
are perfectly calibrated and satisfy that  $b_{j} < b_{j + 1}$ .
Finally, using Lemma 3 with  $b = b_{2}$ ,  $h_{i} = h_{2}$ ,  $h_{j} = h_{3}$ , we have that any monotone AI-assisted decision policy is suboptimal for any  $\mathcal{M}$  with  $f_{H}$ ,  $f_{B}$  and  $P^{\mathcal{M}}$  as defined above.
Construction with arbitrary  $|\mathcal{H}| = k$  and  $|\mathcal{B}| = m, m > k \geq 2$ .
In this second part of the proof, we construct an AI-assisted decision making processes  $\mathcal{M}$ , with  $|\mathcal{H}| = k$  and  $|\mathcal{B}| = m$  such that  $m > k \geq 2$ , such that the decision maker's confidence  $f_{H}$  is monotone, the classifier's confidence  $f_{B}$  is perfectly calibrated and the conditions of Lemma 3 are satisfied.
First, let the space of confidence values be  $\mathcal{H} = \{h_i\}_{i\in [k]}$  and  $\mathcal{B} = \{b_j\}_{j\in [m]}$ , with order  $h_i < h_{i+1}$  and  $b_i < b_{i+1}$ , respectively, and  $f_H, f_B$  and  $P^{\mathcal{M}}$  be such that  $P(f_B(Z) = b_j) = 1/m$  and
$$
P \left(H = h _ {i} \mid B = b _ {j}\right) := P _ {X, V} \left(H = h _ {i} \mid f _ {B} (Z) = b _ {j}\right) = \left\{ \begin{array}{l l} \frac {m - j + 1}{m} & \text {i f} j = i \\ \frac {m - j + 1}{m} & \text {i f} i = 1, j > k \\ \frac {j - 1}{m} & \text {i f} j = i + 1, j \leq k \\ \frac {j - 1}{m} & \text {i f} i = k, j > k \\ 0 & \text {o t h e r w i s e .} \end{array} \right. \tag {15}
$$
Moreover, for each pair of confidence values  $(h_i, b_j)$  with positive probability  $P(H = h_i, B = b_j)$ , we set
$$
P (Y = 1 \mid H = h _ {i}, B = b _ {j}) = \left\{ \begin{array}{l l} P ^ {-} & \text {i f} j = i \\ P ^ {-} & \text {i f} i = 1, j > k \\ P ^ {+} & \text {i f} j = i + 1, j \leq k \\ P ^ {+} & \text {i f} i = k, j > k, \end{array} \right. \tag {16}
$$
as shown in Figure 4 (right). Further, we set the classifier's confidence values  $b_{j}$  to
$$
b _ {j} := \frac {m - j + 1}{m} \cdot P ^ {-} + \frac {j - 1}{m} \cdot P ^ {+}.
$$
Then, it holds that  $b_{j} < b_{j + 1}$  and  $f_{B}$  is perfectly calibrated as
$$
P (Y = 1 \mid B = b _ {j}) = \left\{ \begin{array}{l l} P (H = h _ {j} \mid B = b _ {j}) \cdot P ^ {-} + P (H = h _ {j - 1} \mid B = b _ {j}) \cdot P ^ {+} & \text {i f} j \leq k \\ P (H = h _ {1} \mid B = b _ {j}) \cdot P ^ {-} + P (H = h _ {k} \mid B = b _ {j}) \cdot P ^ {+} & \text {i f} j > k \end{array} \right.
$$
and thus, using the definitions of  $P(H \mid B)$  and  $P(Y \mid H, B)$ , we have that  $P(Y \mid B = b_j) = b_j$ .
To show that  $f_{H}$  is monotone with respect to the probability that  $Y = 1$ , first note that  $P(H = h_i, B = b_i)$  decreases as  $i$  increases and  $P(H = h_i, B = b_{i+1})$  increases as  $i$  increases. Moreover, further note that  $P(Y = 1 \mid H = h_i, B = b_i) = P^{-} < P(Y = 1 \mid H = h_i, B = b_{i+1}) = P^{+}$ . Hence, for any  $i \in \{2, \ldots, k-1\}$ , it readily follows that
$$
\begin{array}{l} P (Y = 1 \mid H = h _ {i}) = P ^ {+} \cdot P (B = b _ {i + 1} | H = h _ {i}) + P ^ {-} \cdot P (B = b _ {i} | H = h _ {i}) \\ \leq P (Y = 1 \mid H = h _ {i + 1}), \\ \end{array}
$$
and, for  $i = 1$ , it is evident that  $P(Y = 1 \mid H = h_1) < P(Y = 1 \mid H = h_2)$ .
Finally, using Lemma 3 with any choice of confidence values  $b = b_{j}$ ,  $h_{i} = h_{j-1}$  and  $h_{j} = h_{j}$  with  $j \in \{2, \ldots, k\}$ , we have that any monotone AI-assisted decision policy  $\pi$  is suboptimal for any  $\mathcal{M}$  with  $|\mathcal{H}| = k$  and  $|\mathcal{B}| = m$ ,  $m > k \geq 2$ , and  $f_{H}, f_{B}$  and  $P^{\mathcal{M}}$  as defined above. Here, note that, as we do not fix the exact distributions  $P^{-}$  and  $P^{+}$ , the above Lemma applies to infinitely many AI-assisted decision making processes  $\mathcal{M}$ .
Construction with  $\mathcal{B} \subseteq [0,1]$  and  $|\mathcal{H}| = k$ .
In this last part of the proof, we construct an AI-assisted decision making process  $\mathcal{M}$ , with  $|\mathcal{H}| = k \geq 2$  and  $\mathcal{B} \subseteq [0,1]$ , such that the decision maker's confidence function  $f_{H}$  is monotone, the classifier's confidence function  $f_{B}$  is perfectly calibrated and the conditions of Lemma 3 are satisfied.
First, let the space of confidence values be  $\mathcal{H} = \{h_i\}_{i\in [k]}$ , with order  $h_i < h_{i + 1}$ , the feature space<sup>10</sup>  $\mathcal{X} = [0,1]$ , and  $f^{-}, f^{+}$  be two strictly monotone increasing functions with
$$
f ^ {-}: [ 0, 1 ] \rightarrow [ 0, c) \quad \text {a n d} \quad f ^ {+}: [ 0, 1 ] \rightarrow (c, 1 ], \tag {17}
$$
$$
c = \frac {u (0 , 0) - u (1 , 0)}{u (1 , 1) - u (1 , 0) + u (0 , 0) - u (0 , 1)}. \tag {18}
$$
Further, let  $Q_{k + 1} = \{q_0,q_1,\ldots q_k,q_{k + 1}\}$  be a set of quantiles such that  $P(X\leq q_j) = j / (k + 1)$  for all  $j\in \{0,1,\dots ,k + 1\}$  and thus, we have that, for all  $j\in [k + 1]$ ,
$$
\text {f o r} \quad I _ {j} := \left(q _ {j - 1}, q _ {j} \right], \quad \text {i t h o l d s t h a t} \quad P (X \in I _ {j}) = \frac {1}{k + 1}.
$$
Now, let  $f_{H}$  and  $P^{\mathcal{M}}$  be such that
$$
P _ {V} \left(H = h _ {i} \mid X, X \in I _ {j}\right) = \left\{ \begin{array}{l l} 1 / 2 & \text {i f} i \in \{j - 1, j \} \\ 1 & \text {i f} i = j = 1 \text {o r} (i = k \text {a n d} j = k + 1) \\ 0 & \text {o t h e r w i s e ,} \end{array} \right. \tag {19}
$$
For a more general feature space  $\mathcal{X}$ , we can use a mapping  $\phi$  of  $\mathcal{X}$  to  $[0,1]$ . The proof works analogously by substituting  $X$  with  $\phi(X)$ .
Figure 5: Nonzero values of  $P(Y = 1|X, H = h_i, X \in I_j)$  for every  $h_i \in \mathcal{H}$ , with  $|\mathcal{H}| = 3$ , and  $I_j = (q_{j-1}, q_j]$ , with  $q_j \in Q_4$  used in the last part of the proof of Theorem 3. Lighter color means lower value of  $f^-$  or  $f^+$ .
$$
P (Y = 1 \mid X, H = h _ {i}, X \in I _ {j}) = \left\{ \begin{array}{l l} f ^ {-} (X) & \text {i f} j = i \text {o r} (i = j = 1) \\ f ^ {+} (X) & \text {i f} j = i + 1 \text {o r} (i = k \text {a n d} j = k + 1), \end{array} \right. \tag {20}
$$
as shown in Figure 5. Next, we define
$$
f _ {B} (Z) = f _ {B} (X) := P (Y = 1 \mid X) = \left\{ \begin{array}{l l} f ^ {-} (X) & \text {i f X \in I _ {1}} \\ f ^ {+} (X) & \text {i f X \in I _ {k + 1}} \\ (f ^ {-} (X) + f ^ {+} (X)) / 2 & \text {o t h e r w i s e}, \end{array} \right.
$$
which, by construction, is perfectly calibrated.
To show that the decision maker's confidence function  $f_{H}$  is monotone with respect to the probability that  $Y = 1$ , we first note that, using Eq. 19, we have that
$$
P \left(X \in I _ {j} \mid H = h _ {i}\right) = \left\{ \begin{array}{l l} 1 / 2 & \text {i f} 1 <   i <   k \text {a n d} j \in \{i, i + 1 \} \text {a n d} \\ 1 & \text {i f} i = j = 1 \\ 1 & \text {i f} i = k \text {a n d} j = k + 1 \\ 0 & \text {o t h e r w i s e .} \end{array} \right. \tag {21}
$$
Hence, using Eq. 21 and the law of total probability, for any  $i \in \{2, \dots, k - 2\}$ , we have that
$$
\begin{array}{l} P (Y = 1 \mid H = h _ {i}) = \frac {1}{2} \left[ P (Y = 1 \mid H = h _ {i}, X \in I _ {i}) + P (Y = 1 \mid H = h _ {i}, X \in I _ {i + 1}) \right] \\ \leq \frac {1}{2} [ f ^ {-} (q _ {i}) + f ^ {+} (q _ {i + 1}) ] \\ = \frac {1}{2} \left[ f ^ {-} (\inf  I _ {i + 1}) + f ^ {+} (\inf  I _ {i + 2}) \right] \\ \leq \frac {1}{2} [ P (Y = 1 \mid H = h _ {i + 1}, X \in I _ {i + 1}) + P (Y = 1 \mid H = h _ {i + 1}, X \in I _ {i + 2}) ] \\ = P (Y = 1 \mid H = h _ {i + 1}), \\ \end{array}
$$
where the inequalities follow from the fact that  $f^{-}$  and  $f^{+}$  are strictly monotone increasing. Corner cases for  $i = 1$  and  $i = k - 1$  can be shown analogously by further using that  $f^{-}(X) < c < f^{+}(X)$  for all  $X$ .
Finally, using Lemma 3 with any choice of confidence values  $h_i = h_{j-1}$ ,  $h_j = h_j$ ,  $j \in \{2, \dots, k-1\}$  and  $b = f_B(X)$  with  $X \in I_j$ , we have that any monotone AI-assisted decision policy  $\pi$  is suboptimal for any  $\mathcal{M}$  with  $|\mathcal{B}| \subseteq [0,1]$  and  $|\mathcal{H}| = k$ ,  $k \geq 2$  and  $f_H, f_B$  and  $P^{\mathcal{M}}$  as defined above.
A.3 Proof of Theorem 5
We prove the statement by contraposition. Let  $\mathcal{M}$  be an AI-assisted decision making process satisfying Eqs. 2 and 3, with a utility function  $u(T,Y)$  satisfying Eq. 1 and let  $\mathcal{M}$  be such that  $f_{B}$  satisfies  $\alpha$ -alignment with respect to  $f_{H}$  and  $f_{B}$  has output space  $\mathcal{B} \subseteq [0,1]$ . Assume there exists no (near-)optimal monotone AI-assisted decision policy for utility  $u$ . Thus, there must exist an optimal AI-assisted decision policy  $\pi \in \Pi(H,B)$  which is not monotone and has strictly greater expected utility than any monotone policy. However, we show that we can modify  $\pi$  to a monotone AI-assisted decision policy  $\hat{\pi} \in \Pi(H,B)$  with near-optimal expected utility.
As  $\pi$  is not monotone, there must exist confidence values  $h_1, h_2 \in \mathcal{H}$ ,  $h_1 \leq h_2$ , and  $b_1, b_2 \in \mathcal{B}$ ,  $b_1 \leq b_2$ , such that
$$
\pi \left(h _ {1}, b _ {1}, w\right) > \pi \left(h _ {2}, b _ {2}, w\right) \quad \text {f o r s o m e} \quad w \in \mathcal {W}, \tag {22}
$$
where  $\mathcal{W}$  denotes the space of noise values. In what follows, let  $\tilde{\mathcal{W}}_{h_1,b_1}^{(\pi ,h_2,b_2)}\subseteq \mathcal{W}$  denote the set containing any such  $w$  and let  $\tilde{\mathcal{W}}^{(\pi ,h_2,b_2)} = \bigcup_{h,b\in \mathcal{H}\times \mathcal{B}}\tilde{\mathcal{W}}_{h,b}^{(\pi ,h_2,b_2)}$ .
For any confidence value  $h', b' \in \mathcal{H} \times [0,1]$ , we modify policy  $\pi$  to a policy  $\hat{\pi}$  as follows. Let  $\{\tilde{S}_h\}_{h \in \mathcal{H}}$  denote the sets satisfying the  $\alpha$ -alignment condition for  $f_B$  with respect to  $f_H$  and, given confidence  $h'$ , let  $\hat{b}_{h'}$  denote the smallest confidence value of  $f_B$ , such that there exist  $h \leq h'$  with  $P(Y = 1 \mid B = \hat{b}_{h'}, Z \in \tilde{S}_h) \geq c$ , i.e.,
$$
\hat {b} _ {h ^ {\prime}} := \min  \{b \in \mathcal {B} \mid P (Y = 1 \mid B = b, Z \in \tilde {S} _ {h}) \geq c \text {f o r} h \leq h ^ {\prime} \}. \tag {23}
$$
Now, we define a new AI-assisted policy  $\hat{\pi}$  from  $\pi$  as follows,
$$
\hat {\pi} \left(h ^ {\prime}, b ^ {\prime}, w\right) := \left\{ \begin{array}{l l} 1 & \text {i f} b ^ {\prime} \geq \hat {b} _ {h} \text {a n d} w \in \bigcup_ {h \leq h ^ {\prime}, b \in \left[ \hat {b} _ {h ^ {\prime}}, b ^ {\prime} \right]} \tilde {\mathcal {W}} ^ {(\pi , h, b)} \\ 0 & \text {i f} b ^ {\prime} <   \hat {b} _ {h} \text {a n d} w \in \bigcup_ {h \geq h ^ {\prime}, b \in \left[ b ^ {\prime}, \hat {b} _ {h ^ {\prime}}\right)} \tilde {\mathcal {W}} ^ {(\pi , h, b)} \\ \pi \left(h ^ {\prime}, b ^ {\prime}, w\right) & \text {o t h e r w i s e .} \end{array} \right. \tag {24}
$$
Next, we show that  $\hat{\pi}$  is monotone and  $\mathbb{E}_{\hat{\pi}}[u(T,Y)]\geq \mathbb{E}_{\pi}[u(T,Y)] + \alpha \cdot a$  for some constant  $a$ .
Proof  $\hat{\pi}$  is a monotone assisted policy.
To prove that  $\hat{\pi} \in \Pi(H, B)$  is a monotone AI-assisted decision policy, we show that, for all  $h', h'' \in \mathcal{H}, b', b'' \in \mathcal{B}$ , with  $h' \leq h'', b' \leq b''$ , it holds that  $\tilde{\mathcal{W}}_{h', b'}^{(\hat{\pi}, h'', b'')} = \emptyset$ . We distinguish between three cases.
Case 1:  $b' \geq \hat{b}_{h'}$  and  $b'' \geq \hat{b}_{h''}$ .
Since  $h' \leq h''$ ,  $b' \leq b''$  and, by definition,  $\hat{b}_{h''} \leq \hat{b}_{h'}$  since  $h' \leq h''$ , we have that
$$
\bigcup_ {h \leq h ^ {\prime}, b \in \left[ \hat {b} _ {h ^ {\prime}}, b ^ {\prime} \right]} \tilde {\mathcal {W}} ^ {(\pi , h, b)} \subseteq \bigcup_ {h \leq h ^ {\prime \prime}, b \in \left[ \hat {b} _ {h ^ {\prime \prime}}, b ^ {\prime \prime} \right]} \tilde {\mathcal {W}} ^ {(\pi , h, b)}.
$$
Hence, we can conclude that
$$
\hat {\pi} \left(h ^ {\prime}, b ^ {\prime}, w\right) \leq 1 = \hat {\pi} \left(h ^ {\prime \prime}, b ^ {\prime \prime}, w\right) \text {f o r a l l} w \in \bigcup_ {h \leq h ^ {\prime \prime}, b \in \left[ \hat {b} _ {h ^ {\prime \prime}}, b ^ {\prime \prime} \right]} \tilde {\mathcal {W}} ^ {(\pi , h, b)}. \tag {25}
$$
Further, for any other  $w \in \mathcal{W} - \bigcup_{h \leq h'', b \in [\hat{b}_{h''}, b'']} \tilde{\mathcal{W}}^{(\pi, h, b)} \subseteq \mathcal{W} - \tilde{\mathcal{W}}_{h', b'}^{(\pi, h'', b'')}$ , we have that  $\hat{\pi}(h', b', w) = \pi(h', b', w)$  and  $\hat{\pi}(h'', b'', w) = \pi(h'', b'', w)$  and, by definition of  $\tilde{\mathcal{W}}_{h', b'}^{(\pi, h'', b'')}$ , it follows that
$$
\hat {\pi} \left(h ^ {\prime}, b ^ {\prime}, w\right) \leq \hat {\pi} \left(h ^ {\prime \prime}, b ^ {\prime \prime}, w\right) \text {f o r a l l} w \in \mathcal {W} - \bigcup_ {h \leq h ^ {\prime \prime}, b \in \left[ \hat {b} _ {h ^ {\prime \prime}}, b ^ {\prime \prime} \right]} \tilde {\mathcal {W}} ^ {(\pi , h, b)}. \tag {26}
$$
From Eqs. 25 and 26, it follows that  $\tilde{\mathcal{W}}_{h',b'}^{(\hat{\pi},h'',b'')} = \emptyset$ .
Case 2:  $b' < \hat{b}_{h'}$  and  $b'' \geq \hat{b}_{h''}$ .
By definition of  $\hat{\pi}$ , we have that
$$
\hat {\pi} \left(h ^ {\prime}, b ^ {\prime}, w\right) \leq 1 = \hat {\pi} \left(h ^ {\prime \prime}, b ^ {\prime \prime}, w\right) \text {f o r a l l} w \in \bigcup_ {h \leq h ^ {\prime \prime}, b \in \left[ \hat {b} _ {h ^ {\prime \prime}}, b ^ {\prime \prime} \right]} \tilde {\mathcal {W}} ^ {(\pi , h, b)} \tag {27}
$$