SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where the first inequality is from the definition of  $\mathfrak{D}_q(\hat{\pmb{p}}_{sa}^i,\pmb {b}_{sa},\alpha)$ , the second equality is due to the selection of  $j$ , and the last inequality is due to the selection of  $\alpha$ . Hence  $\alpha^{\star}\in [0,\bar{\alpha} ]$  by  $f(\alpha)$  is concave w.r.t.  $\alpha$  (otherwise  $\exists \alpha^{\star} > \bar{\alpha}$  such that  $f(\alpha^{\star}) > f(0)\geq f(\bar{\alpha})$ , where nonconcavity of  $f$  follows), and (8) is true. One can further compute the subdifferential of  $f(\alpha)$  by Danskin's theorem (Bertsekas, 1999). Typically,
$$
- \gamma + \frac {1}{N} \sum_ {i = 1} ^ {N} \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i, \star} \in \partial f (\alpha),
$$
where  $\pmb{p}_{sa}^{i,\star}$  is any minimizer of the inner minimization problem corresponding with given  $\alpha \geq 0$ .
Proof of Theorem 4.4. For simplicity of notations, throughout this proof we use
$$
\delta \triangleq \frac {A \epsilon_ {2}}{2} \left(\max  _ {a \in \mathcal {A}} \max  \{\boldsymbol {b} _ {s a} \} + \bar {\gamma}\right) \quad \text {a n d} \quad f (\gamma) \triangleq \sum_ {a \in \mathcal {A}} \mathfrak {P} \left(\left\{\hat {\boldsymbol {p}} _ {s a} ^ {i} \right\} _ {i = 1} ^ {N}; \boldsymbol {b} _ {s a}, \gamma\right), \quad \forall \gamma \in [ \underline {{\gamma}}, \bar {\gamma} ].
$$
By (6), we can get that  $\mathfrak{P}\left(\{\hat{p}_{sa}^i\}_{i=1}^N; \boldsymbol{b}_{sa}, \gamma\right)$  and  $f(\gamma)$  are non-increasing in  $[\underline{\gamma}, \bar{\gamma}]$ . For each given  $\gamma \in [\underline{\gamma}, \bar{\gamma}]$  and  $a \in \mathcal{A}$ , we denote  $\hat{\mathfrak{P}}\left(\{\hat{p}_{sa}^i\}_{i=1}^N; \boldsymbol{b}_{sa}, \gamma\right)$  the corresponding value calculated by the Algorithm 1. Furthermore, we call
$$
\hat {f} (\gamma) \triangleq \sum_ {a \in \mathcal {A}} \hat {\mathfrak {P}} \left(\left\{\hat {\boldsymbol {p}} _ {s a} ^ {i} \right\} _ {i = 1} ^ {N}; \boldsymbol {b} _ {s a}, \gamma\right), \quad \forall \gamma \in [ \underline {{\gamma}}, \bar {\gamma} ].
$$
We can show that
$$
| f (\gamma) - \hat {f} (\gamma) | \leq \delta , \quad \forall \gamma \in [ \underline {{\gamma}}, \bar {\gamma} ]. \tag {14}
$$
We fix  $\gamma \in [\underline{\gamma},\bar{\gamma} ]$ , and consider (8). Algorithm 1 provides the optimal solution for (8) with tolerance  $\epsilon_2 / 2$ , so we have  $|\alpha_{sa,\gamma}^{\star} - \hat{\alpha}_{sa,\gamma}|\leq \epsilon_2 / 2$ , where  $\alpha_{sa,\gamma}^{\star}$  is the true optimal solution of (8) and  $\hat{\alpha}_{sa,\gamma}$  is the solution computed in the inner bisection in Algorithm 1. Thus we get
$$
\begin{array}{l} \left| \mathfrak {P} \left(\{\hat {\boldsymbol {p}} _ {s a} ^ {i} \} _ {i = 1} ^ {N}; \boldsymbol {b} _ {s a}, \gamma\right) - \hat {\mathfrak {P}} \left(\{\hat {\boldsymbol {p}} _ {s a} ^ {i} \} _ {i = 1} ^ {N}; \boldsymbol {b} _ {s a}, \gamma\right) \right| \\ = - \alpha_ {s a, \gamma} ^ {\star} \gamma + \frac {1}{N} \sum_ {i = 1} ^ {N} \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \alpha_ {s a, \gamma} ^ {\star}\right) - \left(- \hat {\alpha} _ {s a, \gamma} \gamma + \frac {1}{N} \sum_ {i = 1} ^ {N} \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \hat {\alpha} _ {s a, \gamma}\right)\right) \\ \leq \left| \alpha_ {s a, \gamma} ^ {\star} - \hat {\alpha} _ {s a, \gamma} \right| \gamma + \frac {1}{N} \sum_ {i = 1} ^ {N} \left| \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \alpha_ {s a, \gamma} ^ {\star}\right) - \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \hat {\alpha} _ {s a, \gamma}\right) \right| \\ \leq \frac {\epsilon_ {2}}{2} \left(\bar {\gamma} + \max  _ {a \in \mathcal {A}} \max  \{\boldsymbol {b} _ {s a} \}\right), \\ \end{array}
$$
where the last step is due to that
$$
\begin{array}{l} \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \alpha_ {s a, \gamma} ^ {\star}\right) - \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \hat {\alpha} _ {s a, \gamma}\right) \\ = \min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {q} ^ {q} + \alpha_ {s a, \gamma} ^ {\star} \cdot \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i} - \left(\min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {q} ^ {q} + \hat {\alpha} _ {s a, \gamma} \cdot \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i}\right) \\ \leq \left(\alpha_ {s a, \gamma} ^ {\star} - \hat {\alpha} _ {s a, \gamma}\right) \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i, \hat {\alpha}} \\ \leq \frac {\epsilon_ {2}}{2} \max  _ {a \in \mathcal {A}} \max  \left\{\boldsymbol {b} _ {s a} \right\}, \\ \end{array}
$$
$$
\begin{array}{l} \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \hat {\alpha} _ {s a, \gamma}\right) - \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \alpha_ {s a, \gamma} ^ {\star}\right) \\ = \min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {q} ^ {q} + \hat {\alpha} _ {s a, \gamma} \cdot \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i} - \left(\min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {q} ^ {q} + \alpha^ {\star} \cdot \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i}\right) \\ \leq \left(\hat {\alpha} _ {s a, \gamma} - \alpha_ {s a, \gamma} ^ {\star}\right) \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i, \alpha^ {*}} \\ \leq \frac {\epsilon_ {2}}{2} \max  _ {a \in \mathcal {A}} \max  \left\{\boldsymbol {b} _ {s a} \right\}, \\ \end{array}
$$
here  $p_{sa}^{i,\hat{\alpha}}$  and  $p_{sa}^{i,\alpha^{\star}}$  are the optimal solutions to (9) for  $\alpha = \hat{\alpha}_{sa,\gamma}$  and  $\alpha = \alpha_{sa,\gamma}^{\star}$  respectively.
Hence, (14) is the direct consequence of above estimation, together with the definitions of  $f, \hat{f}$  and  $\delta$ . And we have  $\gamma^{\star} = \inf \{\gamma \in [\underline{\gamma},\bar{\gamma} ]:f(\gamma)\leq \theta^{q}\}$ . We further define  $\hat{\gamma}\triangleq \inf \{\gamma \in [\underline{\gamma},\bar{\gamma} ]:\hat{f} (\gamma)\leq \theta^{q}\}$ . The outer bisection of Algorithm 1 implies that  $|\gamma^{\prime} - \hat{\gamma} |\leq \frac{\epsilon_1}{2}$ , so to prove the claimed result in the theorem, it suffices to show that
$$
\left| \gamma^ {\star} - \hat {\gamma} \right| \leq \frac {2 \delta \left(\max  _ {a \in \mathcal {A}} \max  \left\{\boldsymbol {b} _ {s a} \right\} - \underline {{\gamma}}\right)}{\theta^ {q}}. \tag {15}
$$
Notice that  $\hat{\mathfrak{P}}\left(\{\hat{\pmb{p}}_{sa}^i\}_{i=1}^N; \pmb{b}_{sa}, \gamma\right) \leq \mathfrak{P}\left(\{\hat{\pmb{p}}_{sa}^i\}_{i=1}^N; \pmb{b}_{sa}, \gamma\right), \forall \gamma \in [\underline{\gamma}, \bar{\gamma}]$  by definitions, we get  $\hat{f}(\gamma) \leq f(\gamma)$ , so  $\{\gamma \in [\underline{\gamma}, \bar{\gamma}]: f(\gamma) \leq \theta^q\} \subseteq \{\gamma \in [\underline{\gamma}, \bar{\gamma}]: \hat{f}(\gamma) \leq \theta^q\}$ , hence we get  $\hat{\gamma} \leq \gamma^\star$ .
We assume that  $\gamma^{\star} - \underline{\gamma}\geq \frac{2\delta\left(\max_{a\in\mathcal{A}}\max\{\pmb{b}_{sa}\} - \underline{\gamma}\right)}{\theta^{q}}$  otherwise (15) is trivially satisfied. To achieve (15), we claim the following statement:
$$
f \left(\gamma^ {\star} - \frac {2 \delta \left(\max  _ {a \in \mathcal {A}} \max  \left\{\boldsymbol {b} _ {s a} \right\} - \underline {{\gamma}}\right)}{\theta^ {q}}\right) > \theta^ {q} + \delta . \tag {16}
$$
If the statement is true, we have  $\forall \gamma \in [\underline{\gamma},\gamma^{\star} - (\max_{a\in \mathcal{A}}\max \{\pmb {b}_{sa}\} -\underline{\gamma})(2\delta /\theta^{q})]$
$$
\hat {f} (\gamma) \geq f (\gamma) - \delta \geq f \left(\gamma^ {\star} - \frac {2 \delta \left(\max  _ {a \in \mathcal {A}} \max  \left\{\boldsymbol {b} _ {s a} \right\} - \underline {{\gamma}}\right)}{\theta^ {q}}\right) - \delta > \theta^ {q},
$$
where the first inequality is from (14) and  $\hat{f} (\gamma)\leq f(\gamma)$ , the second inequality is due to that  $f(\gamma)$  is non-increasing, and the third inequality is from the above statement.
So  $\{\gamma \in [\underline{\gamma},\bar{\gamma} ]:\hat{f} (\gamma)\leq \theta^q\} \subseteq (\gamma^\star -\left(\max_{a\in A}\max \{\pmb {b}_{sa}\} -\underline{\gamma}\right)(2\delta /\theta^q),\bar{\gamma} ],$  thus
$$
\gamma^ {\star} - \frac {2 \delta (\operatorname* {m a x} _ {a \in \mathcal {A}} \operatorname* {m a x} \{\boldsymbol {b} _ {s a} \} - \underline {{\gamma}})}{\theta^ {q}} \leq \hat {\gamma} \leq \gamma^ {\star},
$$
which implies the desired (15).
For the proof of the statement (16). We argue that
$$
\forall \gamma \in [ \gamma^ {\star} - \frac {2 \delta (\max  _ {a \in \mathcal {A}} \max  \left\{\boldsymbol {b} _ {s a} \right\} - \underline {{\gamma}})}{\theta^ {q}}, \gamma^ {\star}): \sum_ {a \in \mathcal {A}} \alpha_ {s a, \gamma} ^ {\star} \geq \frac {\theta^ {q}}{(\max  _ {a \in \mathcal {A}} \max  \left\{\boldsymbol {b} _ {s a} \right\} - \underline {{\gamma}})}, \tag {17}
$$
by contradiction. Assume there is some  $\gamma'' \in [\gamma^{\star} - \frac{2\delta(\max_{a \in \mathcal{A}} \max\{\pmb{b}_{sa}\} - \underline{\gamma})}{\theta^{q}}, \gamma^{\star})$  with that  $\sum_{a \in \mathcal{A}} \alpha_{sa, \gamma''}^{\star} < \frac{\theta^{q}}{(\max_{a \in \mathcal{A}} \max\{\pmb{b}_{sa}\} - \underline{\gamma})}$ . Clearly  $f(\gamma'') > \theta^{q}$  since  $\gamma'' < \gamma^{\star}$ . Then
$$
\begin{array}{l} \theta^ {q} > \sum_ {a \in \mathcal {A}} \alpha_ {s a, \gamma^ {\prime \prime}} ^ {\star} \left(\max _ {a \in \mathcal {A}} \max \{\pmb {b} _ {s a} \} - \underline {{\gamma}}\right) \\ \geq \sum_ {a \in \mathcal {A}} \alpha_ {s a, \gamma^ {\prime \prime}} ^ {\star} \left(\frac {1}{N} \sum_ {i = 1} ^ {N} \left(\boldsymbol {b} _ {s a} ^ {\top} \hat {\boldsymbol {p}} _ {s a} ^ {i} - \gamma\right)\right) \\ = \sum_ {a \in \mathcal {A}} \frac {1}{N} \sum_ {i = 1} ^ {N} \alpha_ {s a, \gamma^ {\prime \prime}} ^ {\star} \left(\boldsymbol {b} _ {s a} ^ {\top} \hat {\boldsymbol {p}} _ {s a} ^ {i} - \gamma\right) \\ \geq \sum_ {a \in \mathcal {A}} \frac {1}{N} \sum_ {i = 1} ^ {N} \min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {\mathcal {S}}} \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {q} ^ {q} + \alpha_ {s a, \gamma^ {\prime \prime}} ^ {\star} \cdot \left(\boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i} - \gamma\right) \\ = \sum_ {a \in \mathcal {A}} - \alpha_ {s a, \gamma^ {\prime \prime}} ^ {\star} \gamma^ {\prime \prime} + \frac {1}{N} \sum_ {i = 1} ^ {N} \mathfrak {D} _ {q} (\hat {\pmb {p}} _ {s a} ^ {i}, \pmb {b} _ {s a}, \alpha_ {s a, \gamma^ {\prime \prime}} ^ {\star}) \\ = \sum_ {a \in \mathcal {A}} \Pr \left(\left\{\hat {\boldsymbol {p}} _ {s a} ^ {i} \right\} _ {i = 1} ^ {N}; \boldsymbol {b} _ {s a}, \gamma^ {\prime \prime}\right) \\ = f \left(\gamma^ {\prime \prime}\right), \\ \end{array}
$$
which is contradicted with  $f(\gamma^{\prime \prime}) > \theta^{q}$ . This contradiction implies that (17) is true.
For simplicity of notations, we call  $\gamma^{\prime \prime \prime} = \gamma^{\star} - \frac{2\delta\left(\max_{a\in\mathcal{A}}\max\{\pmb{b}_{sa}\} - \underline{\gamma}\right)}{\theta^{q}}$ , and we fix any  $\gamma^{\ell}\in (\gamma^{\prime \prime \prime},\gamma^{\star})$  then
$$
\begin{array}{l} \theta^ {q} - f \left(\gamma^ {\star} - \frac {2 \delta \left(\max _ {a \in \mathcal {A}} \max \{\boldsymbol {b} _ {s a} \} - \underline {{\gamma}}\right)}{\theta^ {q}}\right) \\ <   f (\gamma^ {\ell}) - f \left(\gamma^ {\prime \prime \prime}\right) \\ = \sum_ {a \in \mathcal {A}} \left(\mathfrak {P} \left(\{\hat {\boldsymbol {p}} _ {s a} ^ {i} \} _ {i = 1} ^ {N}; \boldsymbol {b} _ {s a}, \gamma^ {\ell}\right) - \mathfrak {P} \left(\{\hat {\boldsymbol {p}} _ {s a} ^ {i} \} _ {i = 1} ^ {N}; \boldsymbol {b} _ {s a}, \gamma^ {\prime \prime \prime}\right)\right) \\ \leq \sum_ {a \in \mathcal {A}} - \alpha_ {s a, \gamma^ {\ell}} ^ {\star} \gamma^ {\ell} + \frac {1}{N} \sum_ {i = 1} ^ {N} \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \alpha_ {s a, \gamma^ {\ell}} ^ {\star}\right) - \left(- \alpha_ {s a, \gamma^ {\ell}} ^ {\star} \gamma^ {\prime \prime \prime} + \frac {1}{N} \sum_ {i = 1} ^ {N} \mathfrak {D} _ {q} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \alpha_ {s a, \gamma^ {\ell}} ^ {\star}\right)\right) \\ = \sum_ {a \in \mathcal {A}} \alpha_ {s a, \gamma^ {\ell}} ^ {\star} \left(\gamma^ {\prime \prime \prime} - \gamma^ {\ell}\right) \\ \leq (\gamma^ {\prime \prime \prime} - \gamma^ {\ell}) \frac {\theta^ {q}}{\left(\max _ {a \in \mathcal {A}} \max \{\pmb {b} _ {s a} \} - \underline {{\gamma}}\right)}, \\ \end{array}
$$
where the first step is due to  $\gamma^{\ell} < \gamma^{\star}$  and definition of  $\gamma^{\prime \prime \prime}$ , the second step is from the definition of function  $f$ , the third step is from the definition of  $\mathfrak{P}(\{\hat{\pmb{p}}_{sa}^{i}\}_{i = 1}^{N};\pmb{b}_{sa},\gamma)$  and  $\alpha_{sa,\gamma}^{\star}$ , and the last step is due to (17) and  $\gamma^{\ell}\in (\gamma^{\prime \prime \prime},\gamma^{\star})$ . Notice that above inequality is true for all  $\gamma^{\ell}\in (\gamma^{\prime \prime \prime},\gamma^{\star})$ , we could let  $\gamma^{\ell}\rightarrow \gamma^{\star}$ , which leads to
$$
\theta^ {q} - f \left(\gamma^ {\star} - \frac {2 \delta \left(\max _ {a \in \mathcal {A}} \max \{\pmb {b} _ {s a} \} - \underline {{\gamma}}\right)}{\theta^ {q}}\right) \leq - 2 \delta <   - \delta .
$$
Then (16) is the direct consequence of above inequality. This finishes the proof of statement, hence finishes the proof of the theorem.
Proof of Theorem 4.5. The Algorithm 1 is the direct consequence of the procedures in the content, except computing the slope, which has been explained at the end of the proof of Proposition 4.3.
For the time complexity, we can see that the bisection method on  $\gamma$  and  $\alpha$  uses complexity  $\mathcal{O}(\log \epsilon_1^{-1}\log \epsilon_2^{-1})$ . For the subproblem (9), which costs time complexity  $h_q(S)$ , we need to solve
Algorithm 2: Fast algorithm to solve (9) with  $q = 1$
Input: Sorted  $b_{sa}$  with  $b_{san_1} \geq b_{san_2} \geq \dots \geq b_{san_S}$ . Initialization:  $r \leftarrow b_{sa}^{\top} \hat{p}_{sa}^{i}$  and  $p_{sa}^{i} \leftarrow \hat{p}_{sa}^{i}$ . for  $k = 1, \ldots, S - 1$  do if  $2\hat{p}_{san_k}^{i} + \alpha (b_{san_S} - b_{san_k}) \hat{p}_{san_k}^{i} \geq 0$  then break else  $r+ = 2\hat{p}_{san_k}^{i} + \alpha (b_{san_S} - b_{san_k}) \hat{p}_{sas'}^{i}$ .  $p_{san_S}^{i} + = p_{san_k}^{i}$  and  $p_{san_k}^{i} = 0$ . end
Result: Optimal objective value  $r$  and optimal solution  $\pmb{p}_{sa}^{i}$  of (9) with  $q = 1$ .
it  $NA$  times. Besides, computing the upper bound claimed in Proposition 4.2 requires finding  $\min \{\pmb{b}_{sa}\}$  for each  $a\in \mathcal{A}$ , which is in time complexity  $\mathcal{O}(AS)$ . So we get the time complexity of Algorithm 1 is  $\mathcal{O}\left(h_q(S)NA\log \epsilon_1^{-1}\log \epsilon_2^{-1} + AS\right)$ .
Proof of Proposition 4.6. Plug  $\mathbb{B}_s^\infty$  in (3) into (4), we get the Bellman update is given by
$$
\begin{array}{l} \frac {1}{N} \max  _ {\boldsymbol {\pi} _ {s} \in \Delta_ {A}} \left\{\min  \sum_ {a \in \mathcal {A}} \sum_ {i = 1} ^ {N} \pi_ {s a} \boldsymbol {p} _ {s a} ^ {i} ^ {\top} (\boldsymbol {r} _ {s a} + \lambda \boldsymbol {v}): \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {\infty} \leq \theta ,   \boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S},   \forall i \in [ N ],   \forall a \in \mathcal {A} \right\} \\ = \frac {1}{N} \max  _ {\boldsymbol {\pi} _ {s} \in \Delta_ {A}} \sum_ {a \in \mathcal {A}} \pi_ {s a} \sum_ {i = 1} ^ {N} \min  \left\{\boldsymbol {p} _ {s a} ^ {i} ^ {\top} (\boldsymbol {r} _ {s a} + \lambda \boldsymbol {v}): \| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \| _ {\infty} \leq \theta , \boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S} \right\} \\ = \frac {1}{N} \max  _ {a \in \mathcal {A}} \sum_ {i = 1} ^ {N} \min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \left\{\boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i}: \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {\infty} \leq \theta \right\}. \\ \end{array}
$$
The first equality is due to that the decision variables  $\pmb{p}_{sa}^{i}$ ,  $\forall i \in [N]$  and  $\forall a \in \mathcal{A}$ , are independent from each other. Then we can divide the original problem into  $NA$  subproblems. The second equality is from the fact that the objective function is affine w.r.t.  $\pi_{sa}$ , and the maximum is simply the greatest coefficient of  $\pi_{sa}$ .
A.2 Proofs of results in Section 5
A.2.1 Proof of results in Section 5.1
Proof of Proposition 5.1. We introduce the variable  $v \geq \| \pmb{p}_{sa}^{i} - \hat{\pmb{p}}_{sa}^{i} \|_{1}$ , so
$$
\begin{array}{l} \mathfrak {D} _ {1} \left(\hat {\boldsymbol {p}} _ {s a} ^ {i}, \boldsymbol {b} _ {s a}, \alpha\right) = \min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \left\| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \right\| _ {1} + \alpha \cdot \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i} \\ = \min  _ {v \geq 0} \min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \left\{v + \alpha \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i}: \| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \| _ {1} \leq v \right\} \tag {18} \\ = \min  _ {v \geq 0} \left\{\bar {v} + \alpha \min  _ {\boldsymbol {p} _ {s a} ^ {i} \in \Delta_ {S}} \boldsymbol {b} _ {s a} ^ {\top} \boldsymbol {p} _ {s a} ^ {i}: \| \boldsymbol {p} _ {s a} ^ {i} - \hat {\boldsymbol {p}} _ {s a} ^ {i} \| _ {1} \leq v \right\}. \\ \end{array}
$$
Proof of Theorem 5.2. We denote the objective function in (11) as
$$
F(v) = v + \alpha \left\{\min_{\boldsymbol{p}_{sa}^{i}\in \Delta_{S}}\boldsymbol{b}_{sa}^{\top}\boldsymbol{p}_{sa}^{i}:\| \boldsymbol{p}_{sa}^{i} - \hat{\boldsymbol{p}}_{sa}^{i}\|_{1}\leq v\right\} .
$$
W.L.O.G., we assume that  $b_{sa1} > b_{sa2} > \dots > b_{saS}$ , and  $\hat{p}_{sa}^i > 0$ . We claim that
$$
F (v) = \left\{ \begin{array}{l l} & v + \alpha \left(- \frac {v \left(b _ {s a 1} - b _ {s a S}\right)}{2} + \boldsymbol {b} _ {s a} ^ {\top} \hat {\boldsymbol {p}} _ {s a} ^ {i}\right) \quad \text {i f} v \in \left[ 0, 2 \hat {p} _ {s a 1} ^ {i}\right), \\ & v + \alpha \sum_ {k = K + 1} ^ {S} r _ {k} b _ {s a k} \\ & v + \alpha b _ {s a S} \end{array} \right. \quad \text {i f} v \in \left[ 2 \sum_ {k = 1} ^ {K} \hat {p} _ {s a k} ^ {i}, 2 \sum_ {k = 1} ^ {K + 1} \hat {p} _ {s a k} ^ {i}\right), \text {f o r s o m e} K \in [ S - 2 ],
$$
$$
r _ {k} = \left\{ \begin{array}{l l} & \hat {p} _ {s a (K + 1)} ^ {i} - \left(v - 2 \sum_ {k = 1} ^ {K} \hat {p} _ {s a k} ^ {i}\right) \Bigg / 2 \quad \text {i f} k = K + 1, \\ & \hat {p} _ {s a k} ^ {i} \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \text {i f} K + 2 \leq k \leq S - 1, \\ & \hat {p} _ {s a S} ^ {i} + \frac {v}{2} \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \text {i f} k = S. \end{array} \right.
$$
To prove the first case of the claim, it suffices to show that the optimal solution for the minimization problem in  $F(v)$  is given by  $\pmb{p}^{\star}$ , whose components are  $p_1^\star = \hat{p}_{sa1}^i -\frac{v}{2}, p_k^\star = \hat{p}_{sak}^i$ ,  $\forall 2\leq k\leq S - 1$  and  $p_S^\star = \hat{p}_{saS}^i +\frac{v}{2}$ . It can be easily verified that  $\pmb{p}^{\star}$  defined in this way satisfies the constraints of the minimization problem in  $F(v)$ .
To see the optimality, we consider any optimal solution  $\bar{\pmb{p}}$ . We first notice that  $\bar{p}_1 > 0$  and  $\bar{p}_S < 1$ . Actually, Let
$$
\mathcal {N} = \{k \in [ S ]: \bar {p} _ {k} <   \hat {p} _ {s a k} ^ {i} \},
$$
$$
\mathcal {P} = \{k \in [ S ]: \bar {p} _ {k} > \hat {p} _ {s a k} ^ {i} \},
$$
$$
\mathcal {E} = \{k \in [ S ]: \bar {p} _ {k} = \hat {p} _ {s a k} ^ {i} \}.
$$
Then by  $\boldsymbol{e}^{\top}\bar{\boldsymbol{p}} = \boldsymbol{e}^{\top}\hat{\boldsymbol{p}}_{sa}^{i} = 1$  and  $\| \bar{\boldsymbol{p}} -\hat{\boldsymbol{p}}_{sa}^{i}\|_{1}\leq v$ , we get
$$
\sum_ {k \in \mathcal {N}} \hat {p} _ {s a k} ^ {i} - \bar {p} _ {k} = \sum_ {k \in \mathcal {P}} \bar {p} _ {k} - \hat {p} _ {s a k} ^ {i}
$$
$$
\sum_ {k \in \mathcal {N}} \left(\hat {p} _ {s a k} ^ {i} - \bar {p} _ {k}\right) + \sum_ {k \in \mathcal {P}} \left(\bar {p} _ {k} - \hat {p} _ {s a k} ^ {i}\right) \leq v
$$
Hence  $\sum_{k\in \mathcal{N}}\hat{p}_{sak}^i -\bar{p}_k = \sum_{k\in \mathcal{P}}\bar{p}_k - \hat{p}_{sak}^i\leq \frac{v}{2} <  \hat{p}_{sa1}^i$  , so we get  $\bar{p}_1 > 0$  and  $\bar{p}_S <   1$
Next we show that  $\bar{p}_k = \hat{p}_{sak}^i$ ,  $\forall 2 \leq k \leq S - 1$ . Otherwise we have some  $2 \leq \hat{k} \leq S - 1$  such that  $|\bar{p}_{\hat{k}} - \hat{p}_{sa\hat{k}}^i| > 0$ . If  $\bar{p}_{\hat{k}} > \hat{p}_{sa\hat{k}}^i$ , we define  $\tilde{\boldsymbol{p}}$  with that  $\tilde{p}_{\hat{k}} = \bar{p}_{\hat{k}} - \varepsilon$  and  $\tilde{p}_S = \bar{p}_S + \varepsilon$ , here  $0 < \varepsilon < \min\{\frac{|\bar{p}_k - \hat{p}_{sa\hat{k}}^i|}{2}, \frac{1 - \bar{p}_S}{2}\}$ , while keeping the other components of  $\tilde{\boldsymbol{p}}$  same as  $\bar{p}$ . We can see that  $\tilde{\boldsymbol{p}}$  achieves smaller objective value than  $\bar{p}$  does due to  $b_{sa\hat{k}} > b_{saS}$ , which contradicts the optimality of  $\bar{p}$ . If  $\bar{p}_{\hat{k}} < \hat{p}_{sa\hat{k}}^i$ , then we define  $\tilde{\boldsymbol{p}}$  with that  $\tilde{p}_{\hat{k}} = \bar{p}_{\hat{k}} + \varepsilon$  and  $\tilde{p}_1 = \bar{p}_1 - \varepsilon$ , here  $0 < \varepsilon < \min\{\frac{|\bar{p}_k - \hat{p}_{sa\hat{k}}^i|}{2}, \frac{\bar{p}_1}{2}\}$ , while keeping the other components of  $\tilde{\boldsymbol{p}}$  same as  $\bar{p}$ . Similarly,  $\tilde{\boldsymbol{p}}$  achieves smaller objective value and this implies the contradiction.
Finally, we verify the rest two components. We introduce the variables  $d_{1} = \bar{p}_{1} - \hat{p}_{sa1}^{i}$  and  $d_{S} = \bar{p}_{S} - \hat{p}_{saS}^{i}$ , then we get the equivalent reformulation of the inner minimization in  $F(v)$ :
$$
\min  _ {d _ {1}, d _ {S}} \quad b _ {s a 1} d _ {1} + b _ {s a S} d _ {S}
$$
$$
\begin{array}{l} \text {s . t .} \quad d _ {1} + d _ {S} = 0, | d _ {1} | + | d _ {S} | \leq v. \end{array}
$$
The optimal  $d_{1}$  and  $d_{S}$  are given by  $-v / 2$  and  $v / 2$  respectively. Hence we proved  $p^{\star}$  is indeed an optimal solution.
To prove the second case of the claim, the optimal solution for the minimization problem in  $F(v)$  is given by  $p^{\star}$ , whose components are  $p_k^\star = 0$ ,  $\forall k\in [K]$ , and  $p_k^\star = r_k$ ,  $\forall K + 1\leq k\leq S$ . The decision variables  $p^{\star}$  defined in this way satisfies the constraints of the minimization problem in  $F(v)$ .
To see the optimality, we consider any optimal solution  $\bar{p}$ . We first notice that  $\bar{p}_{K + 1} > 0$  and  $\bar{p}_S < 1$ .
To argue this by contradiction, we suppose the contrary  $\bar{p}_{K + 1} = 0$ . Define the notations  $\mathcal{N}, \mathcal{P}$  and  $\mathcal{E}$  same as before. Then there exists  $\hat{k} \leq K$  with  $\bar{p}_{\hat{k}} > 0$ , otherwise we assume that  $\bar{p}_k = 0$ ,  $\forall k \in [K]$ , which implies the contradiction as follows.