SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \beta_ {\phi , p} ^ {+} (\alpha) - \beta_ {\phi , \hat {p}} ^ {+} (\alpha) \\ = P (Y \geq k) - P (Y \geq \hat {k}) + \frac {P (Y = k) [ P (X <   k) - \alpha ]}{P (X = k)} - \frac {P (Y = \hat {k}) [ P (\hat {X} <   \hat {k}) - \alpha ]}{P (\hat {X} = \hat {k})} \tag {55} \\ = P (Y > k) - P (Y > \hat {k}) + \frac {P (Y = k) [ P (X \leq k) - \alpha ]}{P (X = k)} - \frac {P (Y = \hat {k}) [ P (\hat {X} \leq \hat {k}) - \alpha ]}{P (\hat {X} = \hat {k})}. \\ \end{array}
$$
Obviously,  $P(Y \geq k) - P(Y \geq \hat{k}) \leq 0$  and  $P(Y > k) - P(Y > \hat{k}) \leq 0$  for  $k \geq \hat{k}$ . Observing that  $\beta_{\phi,p}^{+}(\alpha) - \beta_{\phi,\hat{p}}^{+}(\alpha)$  is a linear function of  $\alpha \in [\max \{P(X < k), P(\hat{X} < \hat{k})\}, \min \{P(X \leq k), P(\hat{X} \leq \hat{k})\}]$  given  $Y, X, \hat{X}, k$  and  $\hat{k}$ , we consider the following four possible cases:
1)  $P(X < k) \leq P(\hat{X} < \hat{k})$  and  $\alpha = P(\hat{X} < \hat{k})$ : In this case,  $\frac{P(Y=k)[P(X<k)-\alpha]}{P(X=k)} = \frac{P(Y=k)[P(X<k)-P(\hat{X}<\hat{k})]}{P(X=k)} \leq 0$ . As a result,  $\beta_{\phi,p}^{+}(\alpha) - \beta_{\phi,\hat{p}}^{+}(\alpha) \leq 0$ .
2)  $P(X < k) > P(\hat{X} < \hat{k})$  and  $\alpha = P(X < k)$ : In this case,
$$
\begin{array}{l} \beta_ {\phi , p} ^ {+} (\alpha) - \beta_ {\phi , \hat {p}} ^ {+} (\alpha) \\ = P (Y \geq k) - P (Y \geq \hat {k}) + \frac {P (Y = k) [ P (X <   k) - \alpha ]}{P (X = k)} - \frac {P (Y = \hat {k}) [ P (\hat {X} <   \hat {k}) - \alpha ]}{P (\hat {X} = \hat {k})} \tag {56} \\ = P (Y \geq k) - P (Y \geq \hat {k}) - \frac {P (Y = \hat {k}) [ P (\hat {X} <   \hat {k}) - P (X <   k) ]}{P (\hat {X} = \hat {k})}. \\ \end{array}
$$
When  $k = \hat{k}$ , since  $p > \hat{p}$ , we have  $P(\hat{X} < \hat{k}) - P(X < \hat{k}) > 0$ , which violates the condition that  $P(X < k) > P(\hat{X} < \hat{k})$ .
When  $k > \hat{k}$ , we have  $P(Y \geq k) - P(Y \geq \hat{k}) \leq -P(Y = \hat{k})$ . Therefore,
$$
\begin{array}{l} \beta_ {\phi , p} ^ {+} (\alpha) - \beta_ {\phi , \hat {p}} ^ {+} (\alpha) \leq - P (Y = \hat {k}) - \frac {P (Y = \hat {k}) [ P (\hat {X} <   \hat {k}) - P (X <   k) ]}{P (\hat {X} = \hat {k})} \\ = - \frac {P (Y = \hat {k}) [ P (\hat {X} \leq \hat {k}) - P (X <   k) ]}{P (\hat {X} = \hat {k})} \tag {57} \\ \leq 0. \\ \end{array}
$$
3)  $P(X \leq k) \leq P(\hat{X} \leq \hat{k})$  and  $\alpha = P(X \leq k)$ : In this case,
$$
\begin{array}{l} \frac {P (Y = k) [ P (X \leq k) - \alpha ]}{P (X = k)} - \frac {P (Y = \hat {k}) [ P (\hat {X} \leq \hat {k}) - \alpha ]}{P (\hat {X} = \hat {k})} = \tag {58} \\ - \frac {P (Y = \hat {k}) [ P (\hat {X} \leq \hat {k}) - P (X \leq k) ]}{P (\hat {X} = \hat {k})} \leq 0 \\ \end{array}
$$
As a result,  $\beta_{\phi ,p}^{+}(\alpha) - \beta_{\phi ,\hat{p}}^{+}(\alpha)\leq P(Y > k) - P(Y > \hat{k})\leq 0.$
4)  $P(X \leq k) > P(\hat{X} \leq \hat{k})$  and  $\alpha = P(\hat{X} \leq \hat{k})$ : In this case, when  $k = \hat{k}$ ,  $P(X \leq \hat{k}) - P(\hat{X} \leq \hat{k}) > 0$ , which violates the condition that  $P(X \leq k) > P(\hat{X} \leq \hat{k})$ .
When  $k > \hat{k}$
$$
\begin{array}{l} \beta_ {\phi , p} ^ {+} (\alpha) - \beta_ {\phi , \hat {p}} ^ {+} (\alpha) \\ = P (Y \geq k) - P (Y \geq \hat {k}) + \frac {P (Y = k) [ P (X <   k) - P (\hat {X} \leq \hat {k}) ]}{P (X = k)} \\ - \frac {P (Y = \hat {k}) [ P (\hat {X} <   \hat {k}) - P (\hat {X} \leq \hat {k}) ]}{P (\hat {X} = \hat {k})} \tag {59} \\ = P (Y \geq k) - P (Y > \hat {k}) + \frac {P (Y = k) [ P (X <   k) - P (\hat {X} \leq \hat {k}) ]}{P (X = k)}. \\ \end{array}
$$
Since  $k > \hat{k}$ ,  $P(Y \geq k) - P(Y > \hat{k}) \leq 0$ . In addition,  $P(X < k) - P(\hat{X} \leq \hat{k}) \leq 0$  since  $\alpha \in [\max\{P(X < k), P(\hat{X} < \hat{k})\}, P(\hat{X} \leq \hat{k})]$ . As a result,  $\beta_{\phi, p}^{+}(\alpha) - \beta_{\phi, \hat{p}}^{+}(\alpha) \leq P(Y > k) - P(Y > \hat{k}) \leq 0$ .
Now that  $\beta_{\phi, p}^{+}(\alpha) - \beta_{\phi, \hat{p}}^{+}(\alpha)$  is a linear function of  $\alpha \in [\max \{P(X < k), P(\hat{X} < \hat{k})\}, \min \{P(X \leq k), P(\hat{X} \leq \hat{k})\}]$ , which is non-positive in the extreme points (i.e., the boundaries), we can conclude that  $\beta_{\phi, p}^{+}(\alpha) - \beta_{\phi, \hat{p}}^{+}(\alpha) \leq 0$  for any  $\alpha \in [\max \{P(X < k), P(\hat{X} < \hat{k})\}, \min \{P(X \leq k), P(\hat{X} \leq \hat{k})\}]$ . Therefore, the infimum of  $\beta_{\phi}^{+}(\alpha)$  is attained when  $p = p_{\text{max}}$ .
Case 2:  $q > p$
In this case, we can find that  $\frac{P(Y=k)}{P(X=k)}$  is an increasing function of  $k$ . As a result, according to Lemma 2, we have
$$
\beta_ {\phi} ^ {-} (\alpha) = P (Y \leq k) + \frac {P (Y = k) P (X > k)}{P (X = k)} - \frac {P (Y = k)}{P (X = k)} \alpha . \tag {60}
$$
Similarly, it can be shown that the infimum is attained when  $q = p_{max}$  and  $p = p_{min}$ .
$$
T (P, Q) (\alpha) = \min  \left\{\beta_ {\phi , \inf } ^ {+} (\alpha), \beta_ {\phi , \inf } ^ {-} (\alpha) \right\} \tag {61}
$$
$$
f ^ {\text {t e r n a r y}} (\alpha) = \left\{ \begin{array}{c} 1 - \frac {A + c}{A - c} \alpha , \quad \text {f o r} \alpha \in [ 0, \frac {A - c}{2 B} ], \\ 1 - \frac {c}{B} - \alpha , \text {f o r} \alpha \in [ \frac {A - c}{2 B}, 1 - \frac {A + c}{2 B} ], \\ \frac {A - c}{A + c} - \frac {A - c}{A + c} \alpha , \text {f o r} \alpha \in [ 1 - \frac {A + c}{2 B}, 1 ]. \end{array} \right. \tag {62}
$$
We provide the  $f$ -DP analysis for a generic ternary stochastic compressor defined as follows.
Definition 8 (Generic Ternary Stochastic Compressor). For any given  $x \in [-c, c]$ , the generic compressor ternary outputs ternary $(x, p_1, p_0, p_{-1})$ , which is given by
$$
\operatorname {t e r n a r y} (x, p _ {1}, p _ {0}, p _ {- 1}) = \left\{ \begin{array}{c c} 1, & \text {w i t h p r o b a b i l i t y} p _ {1} (x), \\ 0, & \text {w i t h p r o b a b i l i t y} p _ {0}, \\ - 1, & \text {w i t h p r o b a b i l i t y} p _ {- 1} (x), \end{array} \right. \tag {63}
$$
where  $p_0$  is the design parameter that controls the level of sparsity and  $p_1(x), p_{-1}(x) \in [p_{min}, p_{max}]$ . It can be readily verified that  $p_1 = \frac{A + x}{2B}$ ,  $p_0 = 1 - \frac{A}{B}$ ,  $p_{-1} = \frac{A - x}{2B}$  (and therefore  $p_{min} = \frac{A - c}{2B}$  and  $p_{max} = \frac{A + c}{2B}$ ) for the ternary stochastic compressor in Definition 6.
In the following, we show the  $f$ -DP of the generic ternary stochastic compressor, and the corresponding  $f$ -DP guarantee for the compressor in Definition 6 can be obtained with  $p_{\min} = \frac{A - c}{2B}$ ,  $p_{\max} = \frac{A + c}{2B}$ , and  $p_0 = 1 - \frac{A}{B}$ .
Lemma 5. Suppose that  $p_0$  is independent of  $x$ ,  $p_{\text{max}} + p_{\text{min}} = 1 - p_0$ , and  $p_1(x) > p_1(y), \forall x > y$ . The ternary compressor is  $f^{\text{ternary}}(\alpha)$ -differentially private with
$$
f ^ {\text {t e r n a r y}} (\alpha) = \left\{ \begin{array}{c} 1 - \frac {p _ {\text {m a x}}}{p _ {\text {m i n}}} \alpha , \quad \text {f o r} \alpha \in [ 0, p _ {\text {m i n}} ], \\ p _ {0} + 2 p _ {\text {m i n}} - \alpha , \text {f o r} \alpha \in [ p _ {\text {m i n}}, 1 - p _ {\text {m a x}} ], \\ \frac {p _ {\text {m i n}}}{p _ {\text {m a x}}} - \frac {p _ {\text {m i n}}}{p _ {\text {m a x}}} \alpha , \quad \text {f o r} \alpha \in [ 1 - p _ {\text {m a x}}, 1 ], \end{array} \right. \tag {64}
$$
Proof. Similar to the binomial mechanism, the output space of the ternary mechanism remains the same for different inputs. Let  $Y = \text{ternary}(x_i', p_1, p_0, p_{-1})$  and  $X = \text{ternary}(x_i, p_1, p_0, p_{-1})$ , we have
$$
\frac {P (Y = - 1)}{P (X = - 1)} = \frac {p _ {- 1} (x _ {i} ^ {\prime})}{p _ {- 1} (x _ {i})},
$$
$$
\frac {P (Y = 0)}{P (X = 0)} = 1, \tag {65}
$$
$$
\frac {P (Y = 1)}{P (X = 1)} = \frac {p _ {1} \left(x _ {i} ^ {\prime}\right)}{p _ {1} \left(x _ {i}\right)}.
$$
When  $x_{i} > x_{i}^{\prime}$ , it can be observed that  $\frac{P(Y=k)}{P(X=k)}$  is a decreasing function of  $k$ . According to Lemma 2, we have
$$
\beta_ {\phi} ^ {+} (\alpha) = \left\{ \begin{array}{c c} 1 - \frac {p _ {- 1} \left(x _ {i} ^ {\prime}\right)}{p _ {- 1} \left(x _ {i}\right)} \alpha , & \text {f o r} \alpha \in [ 0, p _ {- 1} \left(x _ {i}\right) ], \\ p _ {0} + p _ {1} \left(x _ {i} ^ {\prime}\right) + p _ {- 1} \left(x _ {i}\right) - \alpha , & \text {f o r} \alpha \in [ p _ {- 1} \left(x _ {i}\right), 1 - p _ {1} \left(x _ {i}\right) ], \\ \frac {p _ {1} \left(x _ {i} ^ {\prime}\right)}{p _ {1} \left(x _ {i}\right)} - \frac {p _ {1} \left(x _ {i} ^ {\prime}\right)}{p _ {1} \left(x _ {i}\right)} \alpha , & \text {f o r} \alpha \in [ 1 - p _ {1} \left(x _ {i}\right), 1 ]. \end{array} \right. \tag {66}
$$
When  $x_{i} < x_{i}^{\prime}$ , it can be observed that  $\frac{P(Y = k)}{P(X = k)}$  is an increasing function of  $k$ . According to Lemma 2, we have
$$
\beta_ {\phi} ^ {-} (\alpha) = \left\{ \begin{array}{c c} 1 - \frac {p _ {1} \left(x _ {i} ^ {\prime}\right)}{p _ {1} \left(x _ {i}\right)} \alpha , & \text {f o r} \alpha \in [ 0, p _ {1} \left(x _ {i}\right) ], \\ p _ {0} + p _ {- 1} \left(x _ {i} ^ {\prime}\right) + p _ {1} \left(x _ {i}\right) - \alpha , & \text {f o r} \alpha \in \left[ p _ {1} \left(x _ {i}\right), 1 - p _ {- 1} \left(x _ {i}\right) \right], \\ \frac {p _ {- 1} \left(x _ {i} ^ {\prime}\right)}{p _ {- 1} \left(x _ {i}\right)} - \frac {p _ {- 1} \left(x _ {i} ^ {\prime}\right)}{p _ {- 1} \left(x _ {i}\right)} \alpha , & \text {f o r} \alpha \in [ 1 - p _ {- 1} \left(x _ {i}\right), 1 ]. \end{array} \right. \tag {67}
$$
The infimum of  $\beta_{\phi}^{+}(\alpha)$  is attained when  $p_{-1}(x_i') = p_{max}$  and  $p_{-1}(x_i) = p_{min}$ , while the infimum of  $\beta_{\phi}^{-}(\alpha)$  is attained when  $p_1(x_i') = p_{max}$  and  $p_1(x_i) = p_{min}$ . As a result, we have
$$
f ^ {\text {t e r n a r y}} (\alpha) = \left\{ \begin{array}{c} 1 - \frac {p _ {\text {m a x}}}{p _ {\text {m i n}}} \alpha , \quad \text {f o r} \alpha \in [ 0, p _ {\text {m i n}} ], \\ p _ {0} + 2 p _ {\text {m i n}} - \alpha , \quad \text {f o r} \alpha \in [ p _ {\text {m i n}}, 1 - p _ {\text {m a x}} ], \\ \frac {p _ {\text {m i n}}}{p _ {\text {m a x}}} - \frac {p _ {\text {m i n}}}{p _ {\text {m a x}}} \alpha , \quad \text {f o r} \alpha \in [ 1 - p _ {\text {m a x}}, 1 ], \end{array} \right. \tag {68}
$$
B.4 Proof of Theorem 4
Theorem 4. Given a vector  $x_{i} = [x_{i,1}, x_{i,2}, \dots, x_{i,d}]$  with  $|x_{i,j}| \leq c, \forall j$ . Applying the ternary compressor to the  $j$ -th coordinate of  $x_{i}$  independently yields  $\mu$ -GDP with  $\mu = -2\Phi^{-1}\left(\frac{1}{1 + (\frac{A + c}{A - c})^d}\right)$ .
Before proving Theorem 4, we first introduce the following lemma.
Lemma 6. [42, 43] Any  $(\epsilon, 0)$ -DP algorithm is also  $\mu$ -GDP for  $\mu = -2\Phi^{-1}\left(\frac{1}{1 + e^{\epsilon}}\right)$ , in which  $\Phi(\cdot)$  is the cumulative density function of normal distribution.
Proof. According to Theorem 3, in the scalar case, the ternary stochastic compressor is  $f^{\text{ternary}}(\alpha)$ -differentially private with
$$
f ^ {\text {t e r n a r y}} (\alpha) = \left\{ \begin{array}{c} 1 - \frac {A + c}{A - c} \alpha , \quad \text {f o r} \alpha \in [ 0, \frac {A - c}{2 B} ], \\ 1 - \frac {c}{B} - \alpha , \text {f o r} \alpha \in [ \frac {A - c}{2 B}, 1 - \frac {A + c}{2 B} ], \\ \frac {A - c}{A + c} - \frac {A - c}{A + c} \alpha , \quad \text {f o r} \alpha \in [ 1 - \frac {A + c}{2 B}, 1 ]. \end{array} \right. \tag {69}
$$
It can be easily verified that  $f^{ternary}(\alpha) \geq \max \{0, 1 - \left(\frac{A + c}{A - c}\right)\alpha, \left(\frac{A - c}{A + c}\right)(1 - \alpha)\}$ . Invoking Lemma 1 suggests that it is  $(\log \left(\frac{A + c}{A - c}\right), 0)$ -DP. Extending it to the  $d$ -dimensional case yields  $(d\log \left(\frac{A + c}{A - c}\right)^M, 0)$ -DP. As a result, according to Lemma 6, it is  $-2\Phi^{-1}\left(\frac{1}{1 + \left(\frac{A + c}{A - c}\right)^d}\right)$ -GDP.
B.5 Proof of Theorem 5
$$
G _ {\mu} (\alpha + \gamma) - \gamma \leq f ^ {t e r n a r y} (\alpha) \leq G _ {\mu} (\alpha - \gamma) + \gamma , \tag {70}
$$
$$
\mu = \frac {2 \sqrt {d c}}{\sqrt {A B - c ^ {2}}}, \gamma = \frac {0 . 5 6 \left[ \frac {A - c}{2 B} \left| 1 + \frac {c}{B} \right| ^ {3} + \frac {A + c}{2 B} \left| 1 - \frac {c}{B} \right| ^ {3} + \left(1 - \frac {A}{B}\right) \left| \frac {c}{B} \right| ^ {3} \right]}{\left(\frac {A}{B} - \frac {c ^ {2}}{B ^ {2}}\right) ^ {3 / 2} d ^ {1 / 2}}. \tag {71}
$$
Before proving Theorem 5, we first define the following functions as in [15],
$$
\operatorname {k l} (f) = - \int_ {0} ^ {1} \log | f ^ {\prime} (x) | d x, \tag {72}
$$
$$
\kappa_ {2} (f) = \int_ {0} ^ {1} \log^ {2} | f ^ {\prime} (x) | d x, \tag {73}
$$
$$
\kappa_ {3} (f) = \int_ {0} ^ {1} | \log | f ^ {\prime} (x) | | ^ {3} d x, \tag {74}
$$
$$
\bar {\kappa} _ {3} (f) = \int_ {0} ^ {1} | \log | f ^ {\prime} (x) | + \mathrm {k l} (f) | ^ {3} d x. \tag {75}
$$
The central limit theorem for  $f$ -DP is formally introduced as follows.
Lemma 7 ([15]). Let  $f_1, \ldots, f_n$  be symmetric trade-off functions such that  $\kappa_3(f_i) < \infty$  for all  $1 \leq i \leq d$ . Denote
$$
\mu = \frac {2 \| k l \| _ {1}}{\sqrt {\left| | \kappa_ {2} \| _ {1} - \| k l \| _ {2} ^ {2} \right.}}, a n d \gamma = \frac {0 . 5 6 \| \bar {\kappa} _ {3} \| _ {1}}{(\left| | \kappa_ {2} \| _ {1} - \| k l \| _ {2} ^ {2}\right) ^ {3 / 2}},
$$
and assume  $\gamma < \frac{1}{2}$ . Then, for all  $\alpha \in [\gamma, 1 - \gamma]$ , we have
$$
G _ {\mu} (\alpha + \gamma) - \gamma \leq f _ {1} \otimes f _ {2} \otimes \dots \otimes f _ {d} (\alpha) \leq G _ {\mu} (\alpha - \gamma) + \gamma . \tag {76}
$$
Given Lemma 7, we are ready to prove Theorem 5.
Proof. Given  $f_{i}(\alpha)$  in (62), we have
$$
\begin{array}{l} \operatorname {k l} (f) = - \left[ \frac {A - c}{2 B} \log \left(\frac {A + c}{A - c}\right) + \frac {A + c}{2 B} \log \left(\frac {A - c}{A + c}\right) \right] \\ = \left[ \frac {A + c}{2 B} - \frac {A - c}{2 B} \right] \log \left(\frac {A + c}{A - c}\right) \tag {77} \\ = \frac {c}{B} \log \left(\frac {A + c}{A - c}\right), \\ \end{array}
$$
$$
\begin{array}{l} \kappa_ {2} (f) = \left[ \frac {A - c}{2 B} \log^ {2} \left(\frac {A + c}{A - c}\right) + \frac {A + c}{2 B} \log^ {2} \left(\frac {A - c}{A + c}\right) \right] \tag {78} \\ = \frac {A}{B} \log^ {2} \left(\frac {A + c}{A - c}\right), \\ \end{array}
$$
$$
\begin{array}{l} \kappa_ {3} (f) = \left[ \frac {A - c}{2 B} \left| \log \left(\frac {A + c}{A - c}\right) \right| ^ {3} + \frac {A + c}{2 B} \left| \log \left(\frac {A - c}{A + c}\right) \right| ^ {3} \right] \tag {79} \\ = \frac {A}{B} \left| \log \left(\frac {A + c}{A - c}\right) \right| ^ {3}, \\ \end{array}
$$
$$
\bar {\kappa} _ {3} (f) = \left[ \frac {A - c}{2 B} \left| 1 + \frac {c}{B} \right| ^ {3} + \frac {A + c}{2 B} \left| 1 - \frac {c}{B} \right| ^ {3} + \left(1 - \frac {A}{B}\right) \left| \frac {c}{B} \right| ^ {3} \right] \left| \log \left(\frac {A + c}{A - c}\right) \right| ^ {3}. \tag {80}
$$
The corresponding  $\mu$  and  $\gamma$  are given as follows
$$
\mu = \frac {2 d \frac {c}{B}}{\sqrt {\frac {A}{B} d - \frac {c ^ {2}}{B ^ {2}} d}} = \frac {2 \sqrt {d c}}{\sqrt {A B - c ^ {2}}}, \tag {81}
$$
$$
\gamma = \frac {0 . 5 6 \left[ \frac {A - c}{2 B} \left| 1 + \frac {c}{B} \right| ^ {3} + \frac {A + c}{2 B} \left| 1 - \frac {c}{B} \right| ^ {3} + \left(1 - \frac {A}{B}\right) \left| \frac {c}{B} \right| ^ {3} \right]}{\left(\frac {A}{B} - \frac {c ^ {2}}{B ^ {2}}\right) ^ {3 / 2} d ^ {1 / 2}}, \tag {82}
$$
C  $f$ -DP of the Poisson Binomial Mechanism
The Poisson binomial mechanism [9] is presented in Algorithm 3. In the following, we show the
Algorithm 3 Poisson Binomial Mechanism
Input:  $p_i \in [p_{min}, p_{max}]$ ,  $\forall i \in \mathcal{N}$
Privatization:  $Z_{pb} \triangleq PB(p_1, p_2, \dots, p_N) = \sum_{i \in \mathcal{N}} \text{Binom}(M, p_i)$ .
$f$ -DP guarantee of the Poisson binomial mechanism with  $M = 1$ . The extension to the proof for  $M > 1$  is straightforward by following a similar technique.
Theorem 6. The Poisson binomial mechanism with  $M = 1$  in Algorithm 3 is  $f^{pb}(\alpha)$ -differentially private with
$$
f ^ {p b} (\alpha) = \min  \left\{\max  \left\{0, 1 - \frac {1 - p _ {\text {m i n}}}{1 - p _ {\text {m a x}}} \alpha , \frac {p _ {\text {m i n}}}{p _ {\text {m a x}}} (1 - \alpha) \right\}, \right. \tag {83}
$$
$$
\left. \max  \left\{0, 1 - \frac {p _ {m a x}}{p _ {m i n}} \alpha , \frac {1 - p _ {m a x}}{1 - p _ {m i n}} (1 - \alpha) \right\} \right\}.
$$
Proof. For Poisson Binomial, let
$$
X = P B \left(p _ {1}, p _ {2}, \dots , p _ {i - 1}, p _ {i}, p _ {i + 1}, \dots , p _ {N}\right),
$$
$$
Y = P B \left(p _ {1}, p _ {2}, \dots , p _ {i - 1}, p _ {i} ^ {\prime}, p _ {i + 1}, \dots , p _ {N}\right), \tag {84}
$$
$$
Z = P B \left(p _ {1}, p _ {2}, \dots , p _ {i - 1}, p _ {i + 1}, \dots , p _ {N}\right),
$$
in which  $PB$  stands for Poisson Binomial. In this case,
$$
\frac {P (Y = k + 1)}{P (X = k + 1)} = \frac {P (Z = k + 1) \left(1 - p _ {i} ^ {\prime}\right) + P (Z = k) p _ {i} ^ {\prime}}{P (Z = k + 1) \left(1 - p _ {i}\right) + P (Z = k) p _ {i}}. \tag {85}
$$
$$
P (Y = k + 1) P (X = k) - P (Y = k) P (X = k + 1) \tag {86}
$$
$$
= \left[ P (Z = k + 1) P (Z = k - 1) - \left(P (Z = k)\right) ^ {2} \right] \left(p _ {i} - p _ {i} ^ {\prime}\right).
$$
Since  $P(Z = k + 1)P(Z = k - 1) - (P(Z = k))^2 < 0$  for Poisson Binomial distribution, we have
$$
P (Y = k + 1) P (X = k) - P (Y = k) P (X = k + 1) \left\{ \begin{array}{l l} > 0, & \text {i f} p _ {i} <   p _ {i} ^ {\prime}, \\ <   0, & \text {i f} p _ {i} > p _ {i} ^ {\prime}. \end{array} \right. \tag {87}
$$
That being said,  $\frac{P(Y=k)}{P(X=k)}$  is an increasing function of  $k$  if  $p_i < p_i'$  and a decreasing function of  $k$  if  $p_i > p_i'$ . Following the same analysis as that in the proof of Theorem 2, for  $p_i > p_i'$ , we have
$$
\begin{array}{l} \beta_ {\phi} ^ {+} (\alpha) = 1 - \left[ P (Y <   k) + \gamma P (Y = k) \right] \\ = P (Y \geq k) - P (Y = k) \frac {\alpha - P (X <   k)}{P (X = k)} \tag {88} \\ = P (Y \geq k) + \frac {P (Y = k) P (X <   k)}{P (X = k)} - \frac {P (Y = k)}{P (X = k)} \alpha , \\ \end{array}
$$
for  $\alpha \in [P(X < k), P(X \leq k)]$  and  $k \in \{0, 1, 2, \dots, N\}$ .
In the following, we show that the infimum of  $\beta_{\phi}^{+}(\alpha)$  is attained when  $p_i = p_{max}$  and  $p_i' = p_{min}$ .
Case 1:  $k = 0$ . In this case,
$$
P (Y \geq 0) = 1,
$$
$$
\begin{array}{l} P (Y = 0) = P (Z = 0) \left(1 - p _ {i} ^ {\prime}\right), \\ P (X = 0) = 0 \end{array} \tag {89}
$$
$$
P (X <   0) = 0,
$$
$$
P (X = 0) = P (Z = 0) \left(1 - p _ {i}\right).
$$
Plugging (89) into (88) yields
$$
\beta_ {\phi} ^ {+} (\alpha) = 1 - \frac {1 - p _ {i} ^ {\prime}}{1 - p _ {i}} \alpha . \tag {90}
$$
It is obvious that the infimum is attained when  $p_i = p_{max}$  and  $p_i' = p_{min}$ .
Case 2:  $k > 0$ . In this case,
$$
P (Y \geq k) = P (Z \geq k) + P (Z = k - 1) p _ {i} ^ {\prime},
$$
$$
\begin{array}{l} P (Y = k) = P (Z = k) \left(1 - p _ {i} ^ {\prime}\right) + P (Z = k - 1) p _ {i} ^ {\prime}, \\ P (X = k) = P (Z = k) = P (Z = k - 1). \end{array} \tag {91}
$$
$$
P (X <   k) = P (Z <   k) - P (Z = k - 1) p _ {i},
$$
$$
P (X = k) = P (Z = k) \left(1 - p _ {i}\right) + P (Z = k - 1) p _ {i}.
$$
Plugging (91) into (88) yields
$$
\beta_ {\phi} ^ {+} (\alpha) = p (Z > k) + P (Z = k) p _ {i} ^ {\prime} + \left[ P (X \leq k) - \alpha \right] \frac {\left[ P (Z = k) - \left[ P (Z = k) - P (Z = k - 1) p _ {i} ^ {\prime} \right] \right]}{P (X = k)}. \tag {92}
$$
The  $p_i^\prime$  related term is given by
$$
\left[ \frac {P (X = k) P (Z = k)}{P (X = k)} - \frac {[ P (Z = k) - P (Z = k - 1) ] [ P (X \leq k) - \alpha ]}{P (X = k)} \right] p _ {i} ^ {\prime}. \tag {93}
$$
Observing that (93) is a linear function of  $\alpha$ , we only need to examine  $\alpha \in \{P(X < \bar{k}), P(X \leq k)\}$ . More specifically, when  $\alpha = P(X \leq k)$ , it is reduced to  $P(Z = k)p_i'$ ; when  $\alpha = P(X < k)$ , it is reduced to  $P(Z = k - 1)p_i'$ . In both cases, the infimum is attained when  $p_i' = p_{min}$ .
Given that  $p_i' = p_{min}$ , the same technique as in the proof of Theorem 2 can be applied to show that the infimum is attained when  $p = p_{max}$ .
Since  $\frac{P(Y=k)}{P(X=k)}$  is a decreasing function of  $k$  when  $p_i > p_i'$ , we have
$$
\frac {p _ {\text {m i n}}}{p _ {\text {m a x}}} \leq \frac {P (Y = k)}{P (X = k)} \leq \frac {1 - p _ {\text {m i n}}}{1 - p _ {\text {m a x}}}. \tag {94}
$$
Given that  $\beta_{\phi}^{+}(\alpha)$  is a decreasing function of  $\alpha$  with  $\beta_{\phi}^{+}(0) = 1$  and  $\beta_{\phi}^{+}(1) = 0$ , we can readily conclude that  $\beta_{\phi}^{+}(\alpha) \geq \max \{0, 1 - \frac{1 - p_{min}}{1 - p_{max}}\alpha\}$  and  $\beta_{\phi}^{+}(\alpha) \geq \frac{p_{min}}{p_{max}}(1 - \alpha)$ . That being said,  $\beta_{\phi}^{+}(\alpha) \geq \max \{0, 1 - \frac{1 - p_{min}}{1 - p_{max}}\alpha, \frac{p_{min}}{p_{max}}(1 - \alpha)\}$ .
Similarly, for  $p_i < p_i'$ , we have
$$
\begin{array}{l} \beta_ {\phi} ^ {-} (\alpha) = 1 - \left[ P (Y > k) + \gamma P (Y = k) \right] \\ = P (Y \leq k) - P (Y = k) \frac {\alpha - P (X > k)}{P (X = k)} \tag {95} \\ = P (Y \leq k) + \frac {P (Y = k) P (X > k)}{P (X = k)} - \frac {P (Y = k)}{P (X = k)} \alpha \\ \end{array}
$$
for  $\alpha \in [P(X > k), P(X \geq k)]$  and  $k \in \{0, 1, 2, \dots, N\}$ . The infimum is attained when  $p_i = p_{min}$ ,  $p_i' = p_{max}$ .
Since  $\frac{P(Y=k)}{P(X=k)}$  is an increasing function of  $k$  when  $p_i < p_i'$ , we have
$$
\frac {1 - p _ {\text {m a x}}}{1 - p _ {\text {m i n}}} \leq \frac {P (Y = k)}{P (X = k)} \leq \frac {p _ {\text {m a x}}}{p _ {\text {m i n}}}. \tag {96}
$$
Given that  $\beta_{\phi}^{-}(\alpha)$  is an increasing function of  $\alpha$  with  $\beta_{\phi}^{-}(0) = 1$  and  $\beta_{\phi}^{-}(1) = 0$ , we can easily conclude that  $\beta_{\phi}^{-}(\alpha) \geq \max \{0, 1 - \frac{p_{max}}{p_{min}}\alpha\}$  and  $\beta_{\phi}^{-}(\alpha) \geq \frac{1 - p_{max}}{1 - p_{min}}(1 - \alpha)$ . That being said,  $\beta_{\phi}^{-}(\alpha) \geq \max \{0, 1 - \frac{p_{max}}{p_{min}}\alpha, \frac{1 - p_{max}}{1 - p_{min}}(1 - \alpha)\}$ .