SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 3. For an input instance  $(n_p, n_r, \ell_p, \ell_r, \mathbf{S})$ , where each entry of  $\mathbf{S}$  is i.i.d. sampled from  $\{v_1, \ldots, v_k\} (0 < v_1 < \dots < v_k)$  uniformly, assume (i)  $k \leq \frac{1}{c} \cdot n_p$ , (ii)  $\ell_r \geq c \cdot \ln(n_r)$ , (iii)  $2 \cdot n_p \cdot \ell_p \leq n_r \cdot \ell_r$  and (iv)  $Q \cdot (n_r - 1) \geq \ell_p$ . Let  $f$  be a strictly concave perturbation function and  $f'(0) < \frac{v_i}{v_{i-1}} f'(1), \forall i \in \{2, \ldots, k\}$ . With probability  $1 - e^{-\Omega(c)}$ ,  $PM$  with  $f(x)$  as the perturbation function (weakly) dominates PLRA in quality and all randomness metrics. Formally,
Proof of Theorem 3: To prove this statement, first consider the following 2 algorithms.
Algorithm 4: Greedy
For each paper  $p$ , let the reviewers sorted by decreasing similarity with  $p$  be  $\{r_1, r_2, \dots, r_{n_r}\}$ . Greedily assign  $p$  to the first few reviewers as follows: assign  $p$  to  $r_1, r_2, \dots, r_{\lfloor \ell_p / Q \rfloor}$  with probability  $Q$ , and assign  $p$  to  $r_{\lfloor \ell_p / Q \rfloor + 1}$  with probability  $\ell_p - Q \lfloor \ell_p / Q \rfloor$ .
Algorithm 5: Balanced Greedy
For each paper  $p$ , suppose  $\forall i \in \{1, 2, \dots, k\}$  the set of reviewers with similarity  $v_i$  is  $R_{p,i}$ . Initialize remaining paper requirement  $\ell_{\text{remain}} = \ell_p$ . For  $i$  from  $k$  to 1:
- If  $\ell_{\mathrm{remain}} \geq Q \cdot |R_{p,i}|$ , assign  $p$  to each reviewer in  $R_{p,i}$  with probability  $Q$ .
- Otherwise uniformly assign  $p$  to each reviewer in  $R_{p,i}$  with probability  $\ell_{\text{remain}} / |R_{p,i}|$ .
- Update  $\ell_{\mathrm{remain}}$  to the current remaining paper requirement max  $(0,\ell_{\mathrm{remain}} - Q|R_{p,i}|)$ .
Note that Balanced Greedy is almost the same algorithm as Greedy, except that Balanced Greedy groups reviewers with the same similarity  $v_{i}$  with  $p$  together as  $R_{p,i}$  and always assigns the same probability to them, while Greedy treats each reviewer individually.
These two Greedies do not always produce feasible assignments because they both consider each paper  $p$  individually and do not take the constraint that each reviewer is assigned at most  $\ell_r$  papers into account. Nevertheless, we will show that with high probability, they are feasible, and if this is the case, we can relate them to PM & PLRA and prove the Theorem 3.
At a high level, we will prove the Theorem 3 with the following steps.
(1) Greedy and Balanced Greedy are feasible with probability  $1 - e^{-\Omega (c)}$ .
(2) When Greedy and Balanced Greedy are feasible:
Greedy produces a possible solution from PLRA.
- PM does exactly the same with Balanced Greedy.
(3) Balanced Greedy dominates Greedy with probability  $1 - e^{-\Omega (c)}$
We will formalize and prove the above 3 steps below.
Claim B.3.1. Greedy and Balanced Greedy are feasible with probability  $1 - e^{-\Omega(c)}$ .
Proof of Claim B.3.1: We use Bernstein's inequality: Suppose  $x_{1}, x_{2}, \ldots, x_{n}$  are i.i.d. from a distribution with mean  $\mu$ , bounded support  $[a, b]$ , with variance  $\sigma^{2}$ . Then
$$
\Pr \left[ | \hat {\mu} - \mu | \geq t \right] \leq 2 \exp \left(- \frac {n \cdot t ^ {2}}{2 \left(\sigma^ {2} + (b - a) t\right)}\right) \text {w h e r e} \hat {\mu} = \frac {1}{n} \sum_ {i = 1} ^ {n} x _ {i}.
$$
In Greedy and Balanced Greedy, the output  $\mathbf{x}$  is a  $n_p\times n_r$  matrix of random variables, where variables from different rows (papers) are independent as different papers are considered individually. These algorithms are feasible if and only if for each reviewer  $r$ ,  $\sum_{p}x_{p,r}\leq \ell_r$ . Note that as the similarities are i.i.d. random, the reviewers are symmetric. Using Union Bound across reviewers,
$$
\Pr [ \mathrm {A L G} \text {i s i n f e a s i b l e} ] \leq n _ {r} \cdot \Pr \left[ \sum_ {p} x _ {p, r _ {1}} > \ell_ {r} \right], \forall \mathrm {A L G} \in \{\text {G r e e d y , B a l a n c e d G r e e d y} \}.
$$
Moreover, the papers are also symmetric. So  $\sum_{p}x_{p,r_1}$  is then a sum of  $n_p$  i.i.d. random variables. Let their distribution be  $D$ . We will bound the probability using Bernstein's inequality. To do this, consider the properties of the distribution  $D$ . For both Greedy and Balanced Greedy,
- The expectation  $\mu$  of  $D$  should be  $\ell_p / n_r$ .
- Also,  $D$  is supported on  $[0, Q] \subseteq [0, 1]$ .
- Therefore,  $\sigma^2 = \mathbf{E}_{x\sim D}[x^2 ] - \mu^2\leq \mathbf{E}_{x\sim D}[x^2 ]\leq \mathbf{E}_{x\sim D}[x] = \ell_p / n_r$
Also, by assumption (iii),  $2 \cdot n_{p} \cdot \ell_{p} \leq n_{r} \cdot \ell_{r}$ , we then know  $\ell_r / n_p \geq 2 \cdot \ell_p / n_r$ . So that
$$
\mathbf {P r} \left[ \sum_ {p} x _ {p, r _ {1}} > \ell_ {r} \right] = \mathbf {P r} \left[ | \hat {\mu} - \mu | > \frac {\ell_ {r}}{n _ {p}} - \mu \right] \leq \mathbf {P r} \left[ | \hat {\mu} - \mu | \geq \frac {\ell_ {r}}{2 n _ {p}} \right].
$$
Letting  $t = \frac{\ell_r}{2n_p}$ , we will use Bernstein's inequality on this formula. According to Bernstein's inequality,
$$
\mathbf {P r} \left[ \sum_ {p} x _ {p, r _ {1}} > \ell_ {r} \right] \leq 2 \exp \left(- \frac {n _ {p} \cdot (\ell_ {r} / 2 n _ {p}) ^ {2}}{2 (\ell_ {p} / n _ {r} + (1 - 0) (\ell_ {r} / 2 n _ {p}))}\right) \leq 2 \exp \left(- \frac {1}{8} \ell_ {r}\right).
$$
Then, by assumption (ii),  $\ell_r\geq c\cdot \ln (n_r)$  , and we will see  $\forall \mathrm{ALG}\in \{\mathrm{Greedy},\mathrm{Balanced~Greedy}\} ,$
$$
\Pr [ \mathrm {A L G} \text {i s i n f e a s i b l e} ] \leq 2 n _ {r} \cdot e ^ {- \frac {1}{8} \ell_ {r}} = 2 e ^ {\left(1 - \frac {1}{8} c\right) \ln \left(n _ {r}\right)} = e ^ {- \Omega (c)}.
$$
This concludes the proof of Claim B.3.1
Let  $\mathbf{x}^{(b)} = \text{Balanced Greedy}(Q), \mathbf{x}^{(g)} = \text{Greedy}(Q)$ . For paper  $p$ , both Balanced Greedy and Greedy maximize the total Quality of  $p$ , so  $\text{Quality}(\mathbf{x}^{(b)}) = \text{Quality}(\mathbf{x}^{(g)})$ . Moreover, this property of the Greedies also gives us the following Claim B.3.2.
Claim B.3.2. For any feasible assignment  $\mathbf{x}$ ,  $\operatorname{Quality}(\mathbf{x}^{(b)}) \geq \operatorname{Quality}(\mathbf{x})$ . The equality holds if and only if for each paper  $p$ ,  $\mathbf{x}$  maximizes the total Quality of  $p$ .
And as PLRA is maximizing Quality, we further get the following Corollary B.3.1.
Corollary B.3.1. If  $\mathbf{x}^{(b)}$  and  $\mathbf{x}^{(g)}$  are feasible, then  $\mathbf{x}^{(b)},\mathbf{x}^{(g)}\in \mathrm{PLRA}(Q)$  and  $\forall \mathbf{x}\in \mathrm{PLRA}(Q),\mathbf{x}$  maximizes the total Quality of  $p$ .
Claim B.3.3. For each paper  $p$ ,  $\mathbf{x}^{(b)}$  uniquely maximizes the total PQuality  $f$  of  $p$ .
Proof of Claim B.3.3: For each paper  $p$ , recall that  $\forall i \in \{1, 2, \dots, k\}$ , the set of reviewers with similarity  $v_i$  to  $p$  is  $R_{p,i}$ . By the execution of Balanced Greedy,  $\mathbf{x}^{(b)}$  assigns  $p$  to every reviewer in the  $i$ -th set,  $R_{p,i}$ , with the same probability. Let this probability be  $A_{p,i}$ . Then, there exists a  $t \in \{1, 2, \dots, k\}$ , such that  $A_{p,t+1} = \dots = A_{p,k} = Q$ ,  $A_{p,t} < Q$  and  $A_{p,t-1} = \dots = A_{p,1} = 0$ .
Suppose that for an assignment  $\mathbf{x}$ , there are both a reviewer  $r_{\mathrm{high}} \in R_{p,i}$  such that  $x_{p,r_{\mathrm{high}}} < Q$ , and another reviewer  $r_{\mathrm{low}} \in R_{p,j} (j < i)$  such that  $x_{p,r_{\mathrm{low}}} > 0$ . Let  $\delta = \min \{Q - x_{p,r_{\mathrm{high}}}, x_{p,r_{\mathrm{low}}}\}$ . Consider adjusting  $\mathbf{x}$  by decreasing  $x_{p,r_{\mathrm{low}}}$  by  $\delta$  and increasing  $x_{p,r_{\mathrm{high}}}$  by  $\delta$ . The total PQuality  $f$  of
$p$  will first decrease by at most  $\delta f'(0)v_j$  and then increase by at least  $\delta f'(1)v_i$ . The net increase in the total PQuality  $f$  of  $p$  will be
$$
\geq \delta \left(f ^ {\prime} (1) v _ {i} - f ^ {\prime} (0) v _ {j}\right) \geq \delta \left(f ^ {\prime} (1) v _ {i} - f ^ {\prime} (0) v _ {i - 1}\right) > 0.
$$
This shows that the adjustment increases the total PQuality  $f$  of  $p$ , so  $\mathbf{x}$  does not maximize it. Therefore, for an assignment  $\mathbf{x}$  to maximize the total PQuality  $f$  of  $p$ ,  $\mathbf{x}$  must assign  $p$  to every reviewer in  $R_{p,t+1}, \ldots, R_{p,k}$  with probability  $Q$ , and assign  $p$  to every reviewer in  $R_{p,t-1}, \ldots, R_{p,1}$  with probability 0. It remains to consider the assignment to group  $R_{p,t}$ .
Write the total PQuality  $f$  of  $p$  in  $\mathbf{x}$  as
$$
\sum_ {i = t + 1} ^ {k} | R _ {p, i} | \cdot f (1) v _ {i} + \sum_ {r \in R _ {p, t}} f (x _ {p, r}) v _ {t}
$$
For fixed  $p$ , the first summation is constant. To maximize the second summation, according to Jensen's inequality,  $x_{p,r}, \forall r \in R_{p,t}$  must be the same, which is exactly  $\mathbf{x}^{(b)}$ .
This concludes the proof of Claim B.3.3.
Claim B.3.3 gives us the following Corollary B.3.2.
Corollary B.3.2. If  $\mathbf{x}^{(b)}$  is feasible,  $\mathrm{PM}(Q,f) = \{\mathbf{x}^{(b)}\}$ .
Now we are ready to proceed to prove Theorem 3. We first prove Theorem 3 (a).
For Quality, Claim B.3.2 implies that  $\forall \mathbf{x} \in \mathrm{PLRA}(Q)$ ,  $\mathrm{Quality}(\mathbf{x}^{(b)}) \geq \mathrm{Quality}(\mathbf{x})$ .
For the randomness metrics, consider each paper  $p$  and again let the set of reviewers with similarity  $v_{i}$  to  $p$  be  $R_{p,i}$ . Like in the proof of Claim B.3.3, by the execution of Balanced Greedy,  $\mathbf{x}^{(b)}$  assigns  $p$  to every reviewer in  $R_{p,i}$  with the same probability  $A_{p,i}$ , and there exists a  $t \in \{1,2,\dots,k\}$ , such that  $A_{p,t+1} = \dots = A_{p,k} = Q$ ,  $A_{p,t} < Q$  and  $A_{p,t-1} = \dots = A_{p,1} = 0$ .
By Corollary B.3.1,  $\forall \mathbf{x} \in \mathrm{PLRA}(Q)$ ,  $\mathbf{x}$  must assign  $p$  to all reviewers in  $R_{p,t+1}, \ldots, R_{p,k}$  with probability  $Q$  and assign  $p$  to all reviewers in  $R_{p,1}, \ldots, R_{p,t-1}$  with probability 0. Among all such assignments, with an argument using Jensen's inequality or simple categorical discussion, we will see that  $\mathbf{x} = \mathbf{x}^{(b)}$  maximizes  $\sum_{r} x_{p,r} \ln(1 / x_{p,r})$ ,  $\sum_{r} \mathbb{I}[x_{p,r} > 0]$  and minimizes  $\max_{r} \{x_{p,r}\}$ ,  $\sum_{r} x_{p,r}^2$ .
This shows that Theorem 3 (a) holds.
To show Theorem 3 (b), we will first prove the following Claim B.3.4.
Claim B.3.4. With probability  $1 - e^{-\Omega (c)}$
$\operatorname{Support}(\mathbf{x}^{(b)}) > \operatorname{Support}(\mathbf{x}^{(g)}), \operatorname{L2Norm}(\mathbf{x}^{(b)}) < \operatorname{L2Norm}(\mathbf{x}^{(g)}), \operatorname{Entropy}(\mathbf{x}^{(b)}) > \operatorname{Entropy}(\mathbf{x}^{(g)}).$
Proof of Claim B.3.4: Recall in Greedy, for each paper  $p$ , the sorted reviewer list by decreasing similarity with  $p$  is  $\{r_1, r_2, \ldots, r_{n_r}\}$ . By assumption (iv),  $Q \cdot (n_r - 1) \geq \ell_p$ , so  $\lceil \ell_p / Q \rceil + 1 \leq n_r$ .
Suppose for some paper  $p$  and  $i \in \{1, \dots, k\}$ ,  $S_{p,r_{\lceil \ell_p / Q \rceil}} = S_{p,r_{\lceil \ell_p / Q \rceil + 1}} = v_i$ . Then, as by the execution of Greedy,  $x_{p,r_{\lceil \ell_p / Q \rceil}}^{(g)} > 0$  and  $x_{p,r_{\lceil \ell_p / Q \rceil + 1}}^{(g)} = 0$ . Let  $\{r_{\mathrm{left}}, \dots, r_{\mathrm{right}}\}$  be the set of reviewers with similarity  $v_i$  to  $p$ , where left  $\leq \lceil \ell_p / Q \rceil < \lceil \ell_p / Q \rceil + 1 \leq \mathrm{right}$ . Then
$$
\mathbf {x} _ {p, r _ {j}} ^ {(b)} = \left\{ \begin{array}{c l} Q & (j \leq \text {l e f t} - 1) \\ \frac {\ell_ {p} - Q (\text {l e f t} - 1)}{\text {r i g h t} - \text {l e f t} + 1} & (\text {l e f t} \leq j \leq \text {r i g h t}) \\ 0 & (j \geq \text {r i g h t} + 1) \end{array} \right.,
$$
$$
\mathbf {x} _ {p, r _ {j}} ^ {(g)} = \left\{ \begin{array}{c l} Q & (j \leq \lceil \ell_ {p} / Q \rceil - 1) \\ \ell_ {p} - Q (\lceil \ell_ {p} / Q \rceil - 1) & (j = \lceil \ell_ {p} / Q \rceil) \\ 0 & (j \geq \lceil \ell_ {p} / Q \rceil + 1) \end{array} \right..
$$
Note that  $\mathbf{x}_{p,r_j}^{(b)} = \mathbf{x}_{p,r_j}^{(g)}$  for any  $j\leq$  left-1 and any  $j\geq$  right  $+1$  . For left  $\leq j\leq$  right,  $\mathbf{x}_{p,r_j}^{(b)}$  are all equal, but as  $x_{p,r\lceil \ell_p / Q\rceil +1}^{(g)} = 0$ $\mathbf{x}_{p,r_j}^{(g)}$  are not all equal. So Support  $(\mathbf{x}^{(b)}) > \mathrm{Support}(\mathbf{x}^{(g)})$  , and by Jensen's inequality, L2Norm  $(\mathbf{x}^{(b)}) <   \mathrm{L2Norm}(\mathbf{x}^{(g)})$  , Entropy  $(\mathbf{x}^{(b)}) > \mathrm{Entropy}(\mathbf{x}^{(g)})$
Therefore, it remains to show that with probability  $1 - e^{-\Omega (c)}$ , for some paper  $p$ ,  $S_{p,r_{[\ell_p / Q]}} = S_{p,r_{[\ell_p / Q]} + 1}$ . For a fixed  $p$ , denote the event that  $S_{p,r_{[\ell_p / Q]}} = S_{p,r_{[\ell_p / Q] + 1}}$  as  $E_{p}$ . Recall that entries in  $\mathbf{S}$  are i.i.d. and uniformly chosen from  $\{v_{1},\ldots ,v_{k}\}$ . Consider fixing  $S_{p,1},\ldots ,S_{p,n_r - 1}$ , and let the  $[\ell_p / Q]$ -th largest number in this set be  $\alpha$ . Then  $\operatorname*{Pr}[S_{p,n_r} = \alpha ]\geq \frac{1}{k}$ . As  $S_{p,n_r} = \alpha$  implies  $E_{p}$ ,  $\operatorname*{Pr}[E_p = 1]\geq \frac{1}{k}$ . Therefore, according to assumption (i),  $k\leq \frac{1}{c}\cdot n_p$ , we know that
$$
\operatorname * {P r} [ \exists p, E _ {p} = 1 ] \geq 1 - \left(1 - \frac {1}{k}\right) ^ {n _ {p}} \geq 1 - \left(1 - \frac {1}{k}\right) ^ {c k} \geq 1 - e ^ {- c}.
$$
This concludes the proof of Claim B.3.4.
Theorem 3 (b) is implied by Claim B.3.1, Corollary B.3.1, Corollary B.3.2 and Claim B.3.4.
C Discussions
C.1 Directly Optimizing Specific Randomness Metrics with PM
While PM provides one way to introduce randomness into the paper assignment, one natural alternative approach is to simply maximize one specific randomness metric, subject to a constraint on the minimum solution quality. In this section, we show that a slight modification to PM is general enough to capture such approaches.
Specifically, suppose we want to maximize a concave randomness metric  $\mathrm{RM}:[0,1]^{n_p\times n_r}\to \mathbb{R}$  over the set of feasible assignments  $S_{\mathrm{feasible}} = \{\mathbf{x}\in [0,1]^{n_p\times n_r}|\sum_rx_{p,r} = \ell_p,\forall p$  and  $\sum_{p}x_{p,r}\leq \ell_r,\forall r\}$  subject to a minimum requirement of the solution quality.  $\mathrm{Quality}(\mathbf{x})\geq \tau \cdot \mathrm{Quality}_{\mathrm{OPT}}$  where  $\mathrm{Quality}_{\mathrm{OPT}} = \max_{\mathbf{x}\in S_{\mathrm{feasible}}}\mathrm{Quality}(\mathbf{x})$  and  $\tau \in [0,1]$ . The problem can be formulated as
Maximize  $\operatorname {RM}(\mathbf{x})$
Subject to Quality  $(\mathbf{x})\geq \tau \cdot \mathrm{Quality}_{\mathrm{OPT}}$  (5)
$$
\mathbf {x} \in S _ {\mathrm {f e a s i b l e}}.
$$
We then consider a slight generalization of PM that allows different perturbation functions for each reviewer-paper pair. That is, we change the definition of  $\mathrm{PQuality}_f$  (the objective of PM) in Definition 4.2 to
$$
\mathrm {P Q a l i t y} _ {f} (\mathbf {x}) = \sum_ {p} \sum_ {r} S _ {p, r} \cdot f _ {p, r} (x _ {p, r}).
$$
Then we have the following result:
Theorem 4. If  $f_{p,r}(x)$  is (i)  $x + \frac{\lambda}{S_{p,r}}\mathbb{I}[x > 0]$ , (ii)  $x - \frac{\lambda}{S_{p,r}}x\ln(x)$ , or (iii)  $x - \frac{\lambda}{S_{p,r}}x^2$ , then PM achieves the optimal trade-offs between Quality(x) and (i) Support(x), (ii) Entropy(x), or (iii) L2Norm²(x) respectively with different values of  $\lambda \in [0, +\infty)$ .
In fact, the proof of Theorem 4 also shows that PM-Q can be viewed as a algorithm that achieves the optimal trade-off between Quality(x) and another randomness metric  $\left( \sum_{p,r} S_{p,r} x_{p,r}^2 \right)$ , a similarity-weighted version of squared L2 norm of x. Next, we will proceed to prove Theorem 4.
Proof of Theorem 4: When  $\tau \in [0,1]$ , the maximization problem (5) always has at least one feasible solution. Let the optimal value of (5) be  $\mathrm{RM}^* (\tau)$ .  $\mathrm{RM}^* (\tau)$  describes the optimal trade-off between Quality(x) and  $\mathrm{RM}(\mathbf{x})$ . We can show the following property of  $\mathrm{RM}^* (\tau)$ .
Lemma C.1.  $\mathrm{RM}^* (\tau)$  is a concave and non-increasing function of  $\tau \in [0,1]$ .
Proof of Lemma C.1: Let  $\mathbf{x}_1$  be the optimal solution of (5) when  $\tau = \tau_1$  and  $\mathbf{x}_2$  be the optimal solution of (5) when  $\tau = \tau_2$  where  $0 \leq \tau_1 < \tau_2 \leq 1$ . Then, we have  $\mathrm{Quality}(\mathbf{x}_2) \geq \tau_2 \cdot \mathrm{Quality}_{\mathrm{OPT}} > \tau_1 \cdot \mathrm{Quality}_{\mathrm{OPT}}$  and  $\mathbf{x}_2$  is a feasible solution of (5) when  $\tau = \tau_1$ . Therefore,  $\mathrm{RM}(\mathbf{x}_1) \geq \mathrm{RM}(\mathbf{x}_2)$ , i.e.,  $\mathrm{RM}^*(\tau_1) \geq \mathrm{RM}^*(\tau_2)$ . This shows that  $\mathrm{RM}^*(\tau)$  is a non-increasing function of  $\tau$ .
Moreover, let  $\mathbf{x}_m = \frac{1}{2} (\mathbf{x}_1 + \mathbf{x}_2)$ . Then,  $\mathrm{Quality}(\mathbf{x}_m) \geq \frac{1}{2} (\tau_1 + \tau_2) \cdot \mathrm{Quality}_{\mathrm{OPT}}$  and  $\mathbf{x}_m \in S_{\mathrm{feasible}}$ . Thus  $\mathbf{x}_m$  is a feasible solution of (5) when  $\tau = \frac{1}{2} (\tau_1 + \tau_2)$ . Since RM is a concave function,
$\mathrm{RM}^{*}(\frac{1}{2} (\tau_{1} + \tau_{2}))\geq \mathrm{RM}(\mathbf{x}_{m})\geq \frac{1}{2} (\mathrm{RM}(\mathbf{x}_{1}) + \mathrm{RM}(\mathbf{x}_{2})) = \frac{1}{2} (\mathrm{RM}^{*}(\tau_{1}) + \mathrm{RM}^{*}(\tau_{2}))$  . This shows that  $\mathrm{RM}^{\ast}(\tau)$  is a concave function of  $\tau$
Now, for  $\lambda \geq 0$ , consider another optimization program as follows:
$$
\begin{array}{l l} \text {M a x i m i z e} & \operatorname {Q u a l i t y} (\mathbf {x}) + \lambda \cdot \operatorname {R M} (\mathbf {x}) \\ \text {S u b j e c t t o} & \mathbf {x} \in S _ {\text {f e a s i b l e}}. \end{array} \tag {6}
$$
Let the solution of (6) be  $x_{\lambda}^{*}$ . Lemma C.1 gives us the following corollary.
Corollary C.1.1.
$$
\left\{\left(\tau , \operatorname {R M} ^ {*} (\tau)\right) \mid \tau \in (0, 1 ] \right\} = \left\{\left(\text {Q u a l i t y} \left(\mathbf {x} _ {\lambda} ^ {*}\right) / \text {Q u a l i t y} _ {\text {O P T}}, \operatorname {R M} \left(\mathbf {x} _ {\lambda} ^ {*}\right)\right) \mid \lambda \in [ 0, + \infty) \right\}.
$$
Plugging in different randomness metrics  $\mathrm{RM}(\mathbf{x})$  concludes the proof of Theorem 4.
C.2 Incorporating Various Constraints in PM
In some of the currently-deployed conference review systems, there are various additional constraints on the assignment of papers to reviewers. We will discuss in this section how to incorporate some of these constraints in PM. Specifically, we consider the following constraints from [20]:
(1) Seniority: Each paper is assigned to  $\geq 1$  senior reviewer.
(2) Geographic diversity: No 2 reviewers assigned to the same paper belong to the same region.
Seniority. Let the set of senior reviewers be  $S_{\mathrm{senior}}$ . We can incorporate the seniority constraint in PM by modifying PM to the following optimization program.
$$
\begin{array}{l l} \text {M a x i m i z e} & \text {P Q u a l i t y} _ {f} (\mathbf {x}) \\ \text {S u b j e c t t o} & \sum_ {r} x _ {p, r} = \ell_ {p} \quad \forall p \in \mathcal {P}, \\ & \sum_ {r \in S _ {\text {s e n i o r}}} x _ {p, r} \geq 1 \quad \forall p \in \mathcal {P}, \\ & \sum_ {p} x _ {p, r} \leq \ell_ {r} \quad \forall r \in \mathcal {R}, \\ & 0 \leq x _ {p, r} \leq Q \quad \forall p \in \mathcal {P}, r \in \mathcal {R}. \end{array}
$$
With a slightly different sampling algorithm, we can show that the obtained randomized assignment can be realized by a distribution of deterministic assignments that satisfies the seniority constraint [22].
Geographic diversity. Let the set of regions be  $S_{\mathrm{region}}$ . We can incorporate the geographic diversity constraint in PM by modifying PM to the following optimization program.
$$
\begin{array}{l l} \text {M a x i m i z e} & \text {P Q u a l i t y} _ {f} (\mathbf {x}) \\ \text {S u b j e c t t o} & \sum_ {r} x _ {p, r} = \ell_ {p} \quad \forall p \in \mathcal {P}, \\ & \sum_ {r \text {b e l o n g s t o} g} x _ {p, r} \leq 1 \quad \forall p \in \mathcal {P}, g \in S _ {\text {r e g i o n}}, \\ & \sum_ {p} x _ {p, r} \leq \ell_ {r} \quad \forall r \in \mathcal {R}, \\ & 0 \leq x _ {p, r} \leq Q \quad \forall p \in \mathcal {P}, r \in \mathcal {R}. \end{array}
$$
The obtained assignment can also be realized by a distribution of deterministic assignments that satisfies the geographic diversity constraint with a slightly different sampling algorithm [22].
Apart from the constraints mentioned above, there are also some other common constraints that cannot be easily incorporated in PM [20]. We leave incorporating them for future work.