Fair Sortition Made Transparent
Authors: Bailey Flanigan
Gregory Kehne
Ariel D. Procaccia
Venue: NeurIPS 2021 Poster (2021)
Keywords: Fairness
Transparency
Discrepancy
Fair Division
Sortition

Abstract: Sortition is an age-old democratic paradigm, widely manifested today through the random selection of citizens' assemblies. Recently-deployed algorithms select assemblies \textit{maximally fairly}, meaning that subject to demographic quotas, they give all potential participants as equal a chance as possible of being chosen.  While these fairness gains can bolster the legitimacy of citizens' assemblies and facilitate their uptake, existing algorithms remain limited by their lack of transparency. To overcome this hurdle, in this work we focus on panel selection by uniform lottery, which is easy to realize in an observable way. By this approach, the final assembly is selected by uniformly sampling some pre-selected set of $m$ possible assemblies.
We provide theoretical guarantees on the fairness attainable via this type of uniform lottery, as compared to the existing maximally fair but opaque algorithms, for two different fairness objectives. We complement these results with experiments on real-world instances that demonstrate the viability of the uniform lottery approach as a method of selecting assemblies both fairly and transparently.

Fair Sortition Made Transparent
Bailey Flanigan Department of Computer Science Carnegie Mellon University bfl bflaniga@andres. cmu edu
Gregory Kehne
School Engineering and Applied Sciences Harvard University gkehne@g. htw.gg..aaaar..ddeoo
Ariel D. Procaccia
School of Engineering and Applied Sciences
Harvard University
arielpro@seas harvard. edu
Abstract
Sortition is an age-old democratic paradigm, widely manifested today through the random selection of citizens' assemblies. Recently-deployer algorithms select as- semblies maximally fairly, meaning that subject to demographic quotas, they give all potential participants as equal chance as possible of being chosen. While these fairness gains can bolster the legitimacy of citizens' assemblies and facili- tate their uptake, existing algorithms remain limited by their lack of transparency. To overcome this hurdle, in this work we focus panel selection by uniform lottery, which is easy to realize an observable way. By this approach, the final assembly is selected by uniformly sampling some pre-selected set of v possible assemblies. We provide theoretical guarantees on the fairness attainable via this type of uniform lottery, as compared to the existing maximally fair but opaque al- gorithms, for two different fairness objectives. We complement these results with experiments on weal-world instances that demonstrate the viability of the uniform lottery approach as method selecting assemblies both fairly and transparently.
Introduction
In citizens' assembly, panel of randomly chosen citizens is convened deliberate and ulti- mately make recommendations on a policy issue. The defining aspect of citizens' assemblies is the randomness of the process, sortition, by which participants are chosen. In practice, the sorti- tion process works as follows: first, volunteers are solicited via thousands of letters or phone calls, which target individuals chosen uniformly at random. Those who respond affirmatively form the pool of volunteers, from which a final panel will be chosen. Finally, selection algorithm is used randomly select some pre-speeiiee specified number of pool members for the panel. To ensure adequate representation of demographic groups, the chosen panel is often constrained to satisfy some upper and lower quotas on feature categories such as age, gender, and ethnicity. We call quota-satisfying panel of size k feasible panel. As this process illustrates, citizens' assemblies offer way to in- volve the public in informed decision-making This potential for civic participation has recently spurred global resurgence in the popularity of citizens assemblies; they have been commissioned by governments and led to policy changes at the national level [19, 23, 12].
Prompted by the growing impact of citizens' assemblies, there has been recent flurry of computer scientific research on sortition, and in particular, on the fairness of the procedure by which partici- pants are chosen [2. 13, 12]. The most practicable result to date is a family of selection algorithms proposed by Flanigan et al. [12], which are distinguished from their predecessors by their use of randomness toward the goal of fairnes:: while previously-use used algorithms selected pool members in
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

a random but ad-hoc fashion, these new algorithms are maximally fair, ensuring that pool members have as equal probability as possible of being chosen for the panel, subject to the quotas. To en- compass the many interpretations of "as equal as possible," these algorithms permit the optimization of any fairness objective with certain convexity properties. There is now a publicly available imple- mentation of the techniques of Flanigan et al. [12], called Panelot, which optimizes the egalitarian notion that no pool member has too little selection probability via the Leximin objective from fair division [21, 14]. This algorithm has already been deployed by several groups of panel organizers, and has been used to select dozens of panels worldwide.
Fairness gains in the panel selection process can lend legitimacy to citizens' assemblies and poten- tially increase their adoption, but only insofar as the public trusts that these gains are truly realized. Currently, the potential for public trust in the panel selection process is limited by multiple factors. First, the latest panel selection algorithms select the final panel via When panels are selected in this manner, observers cannot even verify that any given pool mem- ber has any chance of being chosen for the panel. A second and more fundamental hurdle that randomness and probability, which are central to the sortition process, have been shown in many contexts to be difficult for people to understand and reason about [24, 20, 28]. Aiming to address these shortcomings, we propose and pursue the following notion of transparency in panel selection:
computation.
Transparency: Observers should be able to, without reasoning in-depth about probability. (1) understand the probabilities with which each individual will be chosen for the panel in theory, and (2) verify that individuals are actually selected with these probabilities in practice.
In this paper, we aim to achieve transparency and fairness simultaneouslyy this means advancing the defined goal of transparency, while preserving the fairness gains obtained by maximally fair selection algorithms. Although this task is reminiscent of existing Al research on trade-offs between fairness or transparency with other desirable objectives [4, 11, 3, 27], to our knowledge, this is the first investigation of the trade-off between fairness and transparency.
Setting aside for a moment the goal of fairness, we consider a method of random decision that is already common in the public sphere: the uniform lottery. To satisfy quotas, uniform lottery for sortition must randomize not over individuals, but over entire feasible panels. In fact, this ap- proach has been suggested by practitioners, and was even used in 2020 to select citizens' assembly in Michigan. The following example, which closely mirrors that real-world pilot,' 2 illustrates that panel selection via uniform lottery is naturally consistent with the transparency notion we pursue.
Suppose we construct 1000 feasible panels from a pool (possibly with duplicates), numbered 000- 999, and publish an (anonymized) list of which pool members are on each panel. We then inform spectators that we will choose each panel with equal probability. This satisfies criterion (1): specta- tors can easily understand that all panels will be chosen with the same probability of 1/1000, and can easily determine each individual's selection probability by counting the number of panels containing the individual. To satisfy criterion (2), we enact the lottery by drawing each of the three digits of the final panel number individually from lottery machines. Lottery spectators can confirm that each ball is drawn with equal probability; this provides confirmation that panels are indeed being chosen with uniform probabilities, thus confirming the enactment of the proposed individual selection prob- abilities. In addition to its conventionality lottery balls invites an exciting spectacle, which can promote engagement with citizens' assemblies.
source of randomness, decision-making via drawing
This simple method neatly satisfies our transparency criteria, but it has one obvious downside: uniform lottery over an arbitrary set of feasible panels does not guarantee any measure of equal probabilities to individuals. In fact, it is not even clear that the fairest possible uniform lottery over in panels, where m is a number conducive to selection by physical lottery (e.g. m = =1000). would not be significantly less fair than maximally fair algorithms, which sample the fairest possible unconstrained distribution over panels. For example, if m is too small, there may be no uniform lottery which gives all individuals non-zero selection probability, even if each individual appears
'Qutaa can preclude giving individuals exactly equal probabilities: if the panel must be 1/2 men, 1/2 women but the pool is split 3/4 men, 1/4 women, then some women must be chosen more often than some men. *Of By For's pilot of live panel selection via lottery can be viewed at https: //vwiv/vv...///////e///e com/ 4583048888 t=17m59s from 17:59 to 21:23. For more detailed description, see Figure 3 and surrounding text in [12].

---

on some feasible panel (and so can attain a non-zero selection probability under an unconstrained distribution).
Fortunately, empirical evidence suggests that there is hope: in the 2020 pilot mentioned above, a uniform lottery over m =100 panels was found that nearly matched the fairness of the maximally fair distribution generated by Panelot. Motivated by this anecdotal evidence, we aim to understand whether such fair uniform lottery is guaranteed to exist in general, and if it does, how find it. We summarize this goal in the following research questions:
Does there exist a uniform lottery over m panels that nearly preserves the fairness of the maximally fair unconstrained distribution over panels? And. Algorithmically, how do we compute such uniform lottery?
Results and Contributions. After describing the model in Section 2, in Section 3 we prove that is possible to round an (essentially) arbitrary distribution over panels to a uniform lottery while preserving all individuals' selection probabilities up to only small bounded deviation. These results use tools from discrepancy theory and randomized rounding. Intuitively, this bounded change in selection probabilities implies bounded losses in fairness; we formalize this intuition in Section 4, showing that there exists in general a uniform lottery that is nearly maximally fair, with respect to multiple choices of fairness objective. Although we would ideally like to give such bounds for the Leximin fairness objective, due to its use practice, we cannot succinctly represent bounds for this objective because it is not scalar valued. We therefore give bounds for Maximin, closely related egalitarian objective which only considers the minimum selection probability given to any pool member [7]. We discuss in Section 4 why bounds on loss in Maximin faimess are, in the most meaningful sense, also bounds on loss in Leximin fairness. We additionally give upper bounds on the loss in Nash Welfare [21], similarly well-established fairnsss objective that has also been implemented in panel selection tools [18].
Finally, in Section 5, we consider the algorithmic question in practice: given a maximally fair dis- tribution over panels, can we actually find nearly maximally fair uniform lotteries that match our theoretical guarantees? To answer this question. we implement two standard rounding algorithms, along with near-optimal (but more computationally intensive) integer programming methods, for finding uniform lotteries. We then evaluate the performance of these algorithms in 11 real-world panel selection instances. We find that in all instances, we can compute uniform lotteries that nearly exactly preserve not only fairness with respect to both objectives, but entire sets of Leximin-optimal marginals, meaning that from the perspective of individuals, there is essentially no difference be- tween using a uniform lottery versus the optimal unconstrained distribution sampled by the latest algorithms. We discuss these results, their implications, and how they can be deployed directly into the existing panel selection pipeline in Section 6.
2 Model
Panel Selection Problem. First, we formally define the task of panel selection for citizens' semblies. Let N = [nx] be the pool of volunteers for the panel-individuals from the population who have indicated their willingness denote a fixed set of features of interest. Each feature ft N + Ω4 maps each pool member to their value of that feature, where Qt is the set of ft's possible values. For example, for feature f. "gender", we might have Ωr ("male", "female", "non-binary"), We define individual i's feature vector F(i) = (f)( E II. 22 to be the vector encoding their values for all features in F.
participate in response to an invitation. Let F {fehe
As is done in practice and in previous research [13, 12], we impose that the chosen panel P must be a subset of the pool of size k. and must be representative of the broader population with respect the features in F. This representativeness is imposed via quotas: for each feature I and corresponding value V E L. may have lower and upper quotas f,,0 and uf.u- These quotas require that the panel contain between f,, and u individuals i such that f(() = U.
In terms of these parameters, we define an instance of the panel selection problem as: given (N, k. F,,,u)aa pool, panel size, set of features, and sets of lower and upper quotas-randomly select feasible panel, where a feasible panel is any set of individuals P from the collection K::
k: (P= (2) 1, S 1(i E P f(i) = w}1 S uf, for all f.u].
3

---

Maximally Fair Selection Algorithms. A selection algorithm is procedure that solves instances of the panel selection problem. selection algorithm's level of fairness on a given instance is determined by its panel distribution p. the (possibly implicit) distribution over IC from which it draws the final panel. Because we care about fairness to individual pool members, we evaluate the fairness of p in terms of the fairness of selection probabilities, or marginals, that p implies for all pool members." 3 We denote the vector of marginals implied by p as n, and we will sometimes specity panel distribution p, π to explicitly denote this pair. We say that is realizable if it is implied by some distribution over the feasible panels K.
Maximally fair selection algorithms are those which solve the panel selection problem by sampling specifically chosen p. one which implies marginals т that allocate probability as fairly as possible across pool members. The fairess of p, π is measured by a fairess objective F, which maps an allocation-in this case, of selection probability to pool members- to a real number measuring the allocation's fairness. Fixing an instance, a fairness objective F, and a panel distribution p. we express the fairness of p as F()). Existing maximally fair selection algorithms can maximize a wide range of fairness objectives, including those considered in this paper.
Leximin, Maximin, and Nash Welfare. Of the three fairness objectives we consider in this paper. Maximnn and Nash Welfare (NW) have succinct formulae. For p; ग they are defined follows. where Ty is the marginal of individual i:
1/n
Maximin(p) == min Ti. EEN
(p :-
II
T
Intuitively, NW maximizes the geometric mean, prioritizing the marginal of each individual in proportion to प... Maximin maximizes the marginal probability of the individual least likely to be selected. Finally, Leximin is refinement of Maximin, and is defined by the following algorithm: first, optimize Maximin; then, fixing the minimum marginal as a lower bound on any marginal, maximize the second-lowest marginal: and so on.
Our task: quantize a maximally fair panel distribution with minimal fairness loss. We define a 1 1/m-quantized panel distribution as a distribution over all feasible panels K. in which all prob- abilities are integer multiples of 1/m. We use 5 to denote a panel distribution with this property. Formally, while an (unconstrained) panel distribution p lies in D == {p E R! |lp|| = 1}. a 1/m-quantized panel distribution in P lies in D == {ø E (Z+/m)lKi ||pll1 = 1}. Note that a //m-quantized distribution p immediately translates to a physical uniform lottery of over m panels (with duplicates): if ğ assigns probability f/m to panel P. then the corresponding physical uniform lottery would contain f duplicates of P. Thus, if we can compute 1 1/m-quantized panel distribu- tion p with fairness F()), then we have designed a physical uniform lottery over m panels with that same level of fairness.
Our goal follows directly from this observation: we want to show that given an instance and desired lottery size m, we can compute 1/ //mm-quanized distribution ğ that is nearly as fair, with respect to a fairness notion F, as the maximally fair panel distribution in this instance p* E arg max F(p). We
We define the fairness loss in this quantization process to be the difference F(p") F(p")-F(p) are aided in this task by the existence of practical algorithms for computing p" Flanigan et al. [12], which allows us to use p" as an input to the quantization procedure we hope to design. For intu- ition, we illustrate this quantization task in Figure 1, where x², म are the marginals implied by p", P", respectively. Since the fairness of p" P are computed in terms of л², म, is intuitive that quan- tization process that results in small marginal discrepancy, defined as the maximum change in any marginal ||T - ooooo. should also have small fairness loss. This idea motivates the upcoming section, in which we give quantization procedures with provably bounded marginal discrepancy, forming the foundation for our later bounds on fairness loss.
'A panel distribution p implies unique vector of marginals и as follows: fixing p. x, pool member i's marginal selection probability n, is equal to the probability of drawing a panel from p containing that pool member. For a more detailed introduction to the connection between panel distributions and marginals, we refer readers to Flanigan al. [12].

---

input p
output
quantize
99999 all pool members
1/m
all pool members
all Jeasible panels (K)
all Geasible panels (KC
Figure The quantization task takes as input maximally fair panel distribution p" (implying marginals n and outputs 1/m-quantized panel distribution (implying marginals π).
3 Theoretical Bounds Marginal Discrepancy
Here we prove that for a fixed panel distribution p, π, there exists a uniform lottery p, म such that || - - oo bounded. Preliminarily, we note that intuitive that bounds this discrepancy should approach as m becomes large with respect to n and k. To see why, begin by fixing some distribution p. over panels: as m becomes large, we approach the scenario in which uniform lottery can assign panels arbitrary probabilities, providing increasingly close approximations to p. Since the marginals n; are continuous with respect p. as p p we have that म. + π. for all
While this argument demonstrates convergence, it provides neither efficient algorithms nor tight bounds the rate of convergence. In this section, our task is therefore bound the rate of this convergence as a function of m and the other parameters of the instance. All omitted proofs of results from this section are included in Appendix B.
3.1 Worst-Case Upper Bounds
Our first set of upper bounds result from rounding STANDARD LP, the LP that most directly arises from our problem. This LP is defined in terms of panel distribution p, and M, an [K] matrix describing which individuals are on which feasible panels: M, = if E and Mr = otherwise.
STANDARD LP
Mp
(3.1)
||p||
(3.2)
p20 0.
Here, (3.1) specifies n total constraints. Our goal is to round p to a uniform lottery over m panels (so the entries p are multiples of 1/m) such that (3.2) is maintained exactly, and constraint in (3.1) is relaxed by too much, i.e., M| Mp = || = remains small.
Randomized rounding is natural first approach. Any randomized rounding scheme satisfying neg- ative association (which includes several that respect (3.2)) yields the following bound:
Theorem 3.1. For any realizable n,, we may efficiently randomly generate such that its marginals म satisfy
(
[V log
- न 11000 = -1x00000
Fortunately, there is potential for improvement: randomized rounding does not make full use of the fact that M is k-column sparse, due to each panel in X. containing exactly & individuals. We use this sparsity get stronger bound when n k2, which is practically significant parameter regime. The proof applies dependent rounding algorithm based on theorem of Beck and Fiala [1], to which a modification ensures the exact satisfaction of constraint (3.2).
Theorem 3.2. For any realizable n, we may efficiently construct such that its marginals ß satisfy
| - # < k/m.
This bound already meaningful in practice, where h & m insured by the fact that m is pre- chosen along with k prior to panel selection. Note also that k is typically the order of 100
5

---

(Table 1), whereas uniform lottery can in practice be easily made orders of magnitude larger, as each additional factor of 10 in the size of the uniform lottery requires drawing only one more ball (and there is no fairness cost to drawing larger lottery, since increasing m allows for uniform lotteries which better approximate the unconstrained optimal distribution).
3.2 Beyond-Worst-Can Upper Bounds
As we will demonstrate in Section 3.3, we cannot hope for a better worst-case upper bound than poly(k)/m. We thus shift our consideration to instances which are "simple" in their feature struc- ture. having small number of features (Theorem B.7), a limited number unique feature vectors in the pool (Theorem 3.3), multiple individuals that share each feature vector present (Theo- rem B.8). The beyond-worst-cee dominate worst-case bounds in Theorem 3.1 and Theorem 3.2, respectively. Moreover, Theo- rem 3.3 dominates all other upper bounds in 10 of the 11 practical instances studied in Section 5.
bounds given by Theorem 3.3 and Theorem B.8 asymptotically
We note that while our worst-case upper bounds implied the near pear-preservation of any realizable set of marginals т, some of our syystd-worst-case results apply to only realizable т which are anonymous, meaning that are equal for all with equal feature vectors. We contend that any reasonable set of marginals should have this property," and furthermore that the "anonymization" of any realizable is also realizable (Claim B.6); hence this restriction is insignificant. Our beyond- worst-case bounds also differ from our worst-case bounds in that they depart from the paradigm of rounding p. instead randomizing over panels that may fall outside the support of p.
The main beyond-worste worst-case bound we give, stated below, is parameterized by |C], where the set of unique feature vectors that appear in the pool. All omitted proofs and other beyond worst-case results are stated and proven in Appendix B.
Theorem 3.3. If क is anonymous and realizable, then we may efficiently construct p such that its marginals गे satisfy
|| - 1 0 / Io [c]
m
[C] is at most n,, this bound dominates Theorem 3.1. In 10 of the 11 real-world instances we study, [C] is also smaller than k22 (Appendix A), in which case this bound also dominates Theorem 3.2.
At high level, our beyond-worst--ed instead using the structure of the sortition instance abstract the problem into one about "types." " For this bound we then solve an LP in terms of "types," round that LP, and then reconstruct rounded panel distribution D, from the "type" solution. In particular, the types of individuals are the feature vectors which appear in the pool, and types of panels are the multisets of k feature vectors that satisfy the instance quotas. Fixing an instance, we project some p into type space by viewing it as a distribution p over types of panels R, inducing marginals for each type individuals E C.
upper bounds are obtained not by directly rounding p. but
To begin, we define the TYPE LP, which is analogous Eq. (3.1). We let Q be the type analog of M, so that entry Qej is the number of individuals i with F(1) = c contained in panels of type j E R.5 Then,
TYPE LP
QP = |p =
(3.3)
(3.4)
p 2 0.
We round p in this tP a panel type distribution P while preserving (3.4). All that remains, then, is to construct some p, न such that is consistent with P and || म ol is small. This p is in general supported by panels outside of supp(p), unlike the p obtained by Theorem 3.1. It is the anonymity of π which allows us to construct these new panels and prove that they are feasible for the instance.
"The class of all anonymous marginals я includes the maximizers and second, this condition is satisfied by all existing selection algorithms used in practice, to our knowledge. CCompleting the analogy, C, R, Q, Q.P.P. P. the "type" versions of N. KC, M, p, p, from the original LP.
all reasonable fairness objectives,

---

3.3 Lower Bounds
This method of using bounded discrepancy to derive nearly cairness-optimn uniform lotteries has its limits, since there are even sparse M and fractional for which no integer ž yields nearby Ma. In the worst case, we establish lower bounds by modifying those of Beck and Fiala [25]: Theorem 3.4. There exist p, n for which for all uniform lotteries p, म.
k m
min || - 52 DED
Our k-dependent upper and lower bounds are separated by a factor of Vk, matching the current upper and lower bounds of the Beck-Fiala conjecture as applied to linear discrepancy (also known as the lattice approximation problem [26]). The respective gaps are incomparable, however, since for given [0, 1]", the former problem aims to minimize | E)|[ao over E E {0, 1]", while we aim to do the same over a subset of the ø E 2" for which Σ, 23 = EE By (see Lemma B.4).
Theoretical Bounds on Fairness Loss
Since the fairness of distribution is determined by its marginals æ, it is intuitive that if uniform lotteries incur only small marginal discrepancy (per Section 3), then they should also incur only small faimess losses. This should hold for any fairness notion that is sufficiently "smooth" (i.e., doesn't 't change too quickly with changing marginals) in the vicinity of p, n.
Although our bounds from Section 3 apply to any reasonable initial distribution we are particu- larly concerned with bounding fairness loss from maximally fair initial distributions y". Here, specifically consider such p" that are optimal with respect to Maximin and NW. We note that, since there exist anonymous p" x² that maximize these objectives, we can apply any upper bound from Section 3 to upper bound || - Flooo. We defer omitted proofs to Appendix C.
4.1 Maximin
Since Leximin is the fairness objective optimized by the maximally fair algorithm used in practice, would be most natural to start with a p" that Leximin-optimal and bound fairness loss with respect to this objective. However, the fact that Leximin fairness cannot be represented by a single scalar value prevents us from formulating such an approximation guarantee. Instead, we first pursue bounds the closely-related objective, Maximin. We argue that in the most meaningful sense, worst-case Maximin guarantee is a Leximin guarantee: such a bound would show limited loss the minimum marginal, and it is Leximin's 's lexicographically first priority to maximize the minimum marginal.
First, we show there exists some p. π that gives bounded Maximin loss from p", n,, the Maximin- optimal unconstrained distribution. This bound follows from Theorems 3.3 and B.8, using the simple observation that Mmin = min, n, denotes the smallest number of individuals which share any feature vector E
decrease the lowest marginal given by p" by no more than || #||xo- Here
Corollary 4.1. By Theorem 3.3 and B.8, for Maximin-optima p', there exists uniform lottery that satisfies
Maximin(p") - Maximin(p) 1 .o ((min
0 min {
{Vic) V log C),
k + 11)
m
Theorem 3.4 demonstrates that we cannot get an upper bound on Maxmin loss stronger than O(Vk/m) using uniform bound on changes in all w,. However, since Maximin is concerned only with the smallest n. it seems plausible that better upper bounds Maximin loss could sult from rounding т while tightly controlling only losses in the smallest n, 's while giving freer reign to larger marginals. We show that this is not the case by further modifying the instances from Theorem 3.4 to obtain the following lower bound on the Maximin loss:
Theorem 4.1. There exists a Maximin-optimal p" such that, for all uniform lotteries p.
Maximin(p") - Maximin(p)) =
7

---

4.2 Nash Welfare
As NW has also garnered interest by practitioners and is applicable in practice [18], we upper-bound the NW faireess loss. Unlike Maximin loss, an upper bound on NW loss does not immediately follow from one on || - * lls because decreases in smaller marginals have larger negative impact on the NW. As result, the upper bound on NW resulting from Section 3 is slightly weaker than that on Maximin:
Theorem 4.2. For NW -W-optimal p", there exists a uniform lottery p that satisfies
NW("") - NW()) = m
k 10iin
-0 -
VICI log CJ
11)
Periin
We give an overview of the proof of Theorem 4.2. To begin, fix a NWW w-ttpiiiizing panel distribution p", n² Before applying our upper bounds on marginal discrepancy from Section 3, must contend with the fact that this bounded loss is suffered by already-tiny marginals, the NW may decrease substantially or even go O. Thus, first prove Lemmas 4.1 and 4.2, which together imply that no marginal in π² is smaller than 1/n.
Lemma 4.1. For NW- -optimal p" over support of panels supp(p"), there exists a constant À E R such that, for all P E supplp"), Eiep X.
Lemma 4.2. For NW "Woptiiaal p², x², we have that Tद 2 1/n for all E.
Lemma 4.1 follows from the fact that the partial derivative of NW with respect to the probability assigns a given panel must be the same as that with respect any other panel at p" (otherwise, mass in the distribution could be shifted to increase the NW). Lemma 4.2 then follows by the additional observation that E Ep
EEEP 1/mā] Y..
Finally Lemma 4.3 follows from the fact that Lemma 4.2 limits the potential multiplicative, and therefore additive, impact on the NW of decreasing any marginal by || R Alooo:
Lemma 4.3. For opttimlll p", n², there exists a uniform lottery p, that satisfies NW (p")
(p) < k || - +| |
As the NW-optilal marginals x* are anonymous, we can apply the upper bounds given by Theo- rem 3.3 and Theorem B.8 to show the existence of a p. ग satisfying the claim of the theorem.
5 Practical Algorithms for Computing Fair Uniform Lotteries
Algorithms. First, we implement versions of two existing rounding algorithms, which are implicit in our worst-case upper bounds. The first is Pipage rounding [16], or PIPAGE, a randomized rounding scheme satisfying negative association [10]. The second is BECK-FIALA, the dependent rounding scheme used in the proof of Theorem 3.2. To benchmark these algorithms against the highest level of fairness they could possibly achieve, we use integer programming (IP) to compute the fairest possible uniform lotteries over supp(p")) the panels over which p² randomizes." We define IP- MAXIMIN and IP-NW to find uniform lotteries over supp(p") maximizing Maximin and NW, respectively. We remark that the performance of these IPs is still subject to our theoretical upper and lower bounds. We provide implementation details in Appendix D.I.
One question is whether we should prefer the IPs or the rounding algorithms for real-world ap- plications. Although IP-MAXIMIN appears to find good solutions at practicable speeds, IP-NW converges to optimality prohibitively slowly in some instances (see Appendix D.2 for runtimes). At the same time, we find that our simpler rounding algorithms give near-optimal uniform lotter- ies with respect to both fairness objectives. Also in favor simpler rounding algorithms, many randomized rounding procedures (including Pipage rounding) have the advantage that they exactly
WWe do not implement the algorithm implicit in Theorem 3.3 because our results already present sufficient alternatives for finding excellent uniform lotteries in practice.
"Note that these lotteries are not necessarily universally optimal, as they can randomize over only supp(p"), ): conceivably, one could find fairer uniform lottery by also randomizing over panels not in supp(p"), ). However, PIPAGE and BECK- -FALA also restricted in this way. and thus must be weakly dominated by the IP.
8

---

0.40
IP-Maximin
-1. -1.6/m
m 1000
Pipage
-6 6/m
0.35
Beck-Fiala
-28.4mm
Theoretical Bound
Maximin objective value
0.30
0.25
..0/m
-0.0/m
0.00mm
+8.0/m
0.20
0.15
-0.9 9/
-1.1/m
-2.9/m
0.10
-24.2/m
-3.1/m
.4.1/m
-1.1/m
-16.5/m
-3.9/m
-1.5/m
-3.5/m
66.5/m
-2.1/m
-6.1/m
.25.6/m
-3.4/
-1.4/m
-1.4/m
-3.5/m
-055/m
-4.5m
-1.4/m
-4.4/m
0.05
99.mmm
45.1/m
0.00
sf(a)
sf(b)
sf(c)
sf(d)
sf(e)
cca
hd
mass
nexus
obf
nndm
Figure 2: 1000. Shaded regions extend from Maximin(p"), the faimess of the optimal uncon- strained distribution, down to the minimum fairness implied by the tightest theoretical upper bound in that instance (in all instances but "obf" Theorem 3.3 is tightest). Each algorithm or bound's 's loss relative to Maximin(p") is written above in the corresponding color. We show a representative run of PIPAGE, randomized algorithm.
preserve marginals over the combined steps of randomly rounding to uniform lottery and then randomly sampling it it-a guarantee that much more challenging to achieve with IPs.
Uniform lotteries nearly exactly preserve Maximin, Nash Welfare fairness. We first measure the fairness of uniform lotteries produced by these algorithms in 1l real-world panel selection instances from different organizations worldwide (instance details in Appendix A). In all experiments, we generate lottery size m 1000. This is fairly small; it requires drawing only 3 balls from lottery machines, and in one instance we have that m < We nevertheless see excellent performance of all algorithms, and note that this performance will only improve with larger m.
Figure shows the Maximin fairness of the uniform lottery computed by PIPAGE, BECK-FIALA, and IP-MAXIMIN for each instance. For intuition, recall that the level of Maximin fairness given by any lottery is exactly the minimum marginal assigned to any individual by that lottery. The upper edges of the gray boxes Fig. 2 correspond strained distribution p". These experiments reveal that the cost of transparency to Maximiin-fairne practically non-existent: across instances, the quantized distributions computed by IP-MAXIMIN decrease the minimum marginal by at most 2.1 2. amounting to a loss of no more than 0.0021 in the minimum marginal probability in any instance. Visually, we can see that this loss negligible relative to the original magnitude of even the smallest marginals given by p². Surprisingly, though PIPAGE and BECK-FIALA do not aim optimize any fairness objective, they achieve only slightly larger losses in Maximin fairness, with PIPAGE outperforming BECC-FIALLA Finally, the heights of the gray boxes indicate that our theoretical bounds are often meaningful in practice, giving lower bounds on Maximin fairness well above zero in nine out of cleven instances. We note these bounds only tighten with larger m. We present similarly encouraging results on NW loss in Appendix D.3.
the optimal fairness attained by an uncon-
Uniform lotteries nearly preserve all Leximin marginals. We still remain one step away from practice: our examination of Maximin does not address whether uniform lotteries can attain the finer-tuned faimess properties of the Leximin-opúmal distributions currently used in practice. For- tunately, our results from Section imply the existence of a quantized ğ that closely approximates all marginals given by the Leximin-optimal distribution p", x² We evaluate the extent to which PIPAGE and BECK-FIALA preserve these marginals in Fig. 3. They are benchmarked against new IP, IP-MARGINAL.S. which computes the uniform lottery over supp(p")) minimizing ||T"" ß loo-

---

1.00
marginal probability
0.75
P-Maaginals
Pipage
0.50
Beck-Fiala
13/m
Lel
0.25
0.00
agents sorted by marginal given by p
Figure 3: Instance = sf(a), 1000. Line plot shows the Leximin-optimal marginals T" (implied by panel distribution p²), along with marginals given by all algorithms sorted according to T². Note that each z coordinate then corresponds to an individual. The zoomed box shows the magnitude of marginal discrepancy around x². The surrounding shaded region shows the tightest theoretical bound on the marginal discrepancy, in this case from Theorem 3.3, around the optimal marginals. We show a representative run PIPAGE, a randomized algorithm.
Figure demonstrates that in the instance "sf(a)", all algorithms produce marginals that deviate neg- ligibly from those given by n² Analogous results on remaining instances appear in Appendix D.4 and show similar results. As was the case for Maximin, we see that our theoretical bounds are meaningful, but that we can consistently outperform them in real-world instances.
Discussion
Our aim to show that uniform lotteries can preserve fairness, and our results ultimately suggest this, along with something stronger: that practical instances, uniform lotteries can reliably almost exactly replicate the entire set of marginals given by the optimal unconstrained panel distribution. Our rounding algorithms can thus be plugged directly into the existing panel selection pipeline with essentially impact individuals' selection probabilities, thus enabling translation of the output of Panclot (and other maximally fair algorithms)) to a nearly maximally fair and transparent panel selection procedure. We note that methods not just compatible with ball ball-drawing lotteries, but any form of uniform physical randomness (e.g. dice, wheel-spinning, etc.).
Although we achieve our stated notion of transparency, a limitation of this notion is that it focuses on the final stage of the panel selection process. A more holistic notion transparency might require that onlookers can verify that the panel is not being intentionally stacked with certain individuals. This work does not fully enable such verification: although onlookers can now observe individuals' marginals, they still cannot verify that these marginals are actually maximally fair without verifying the underlying optimization algorithms. In particular, in the common case where quotas require even maximally fair panel distributions to select certain individuals with probability near one, onlookers cannot distinguish those from unfair distributions engineered such that one or more pool members chosen with probability near
In research on economies, fair division, and other areas of AI, randomness is often proposed tool to make real-world systems fairer [17, 6, 15]. Nonetheless, in practice, these systems (with few exceptions, such as school choice [22]) remain stubbornly deterministic. Among the hurdles to bringing the theoretical benefits of randomness into practice is that allocation mechanisms fare best when they can be readily understood, and that randomness can perceived as undesirable suspect. Sortition is rather unique paradigm the heart of this tension: it relies centrally therefore a
randomness, while in the public sphere attaining increasing political influence. uniquely high-impact domain in which study how to combine the benefits of randomness, such as fairness, with transparency. We hope that this work and its potential for impact will inspire the investigation of fairness-trant transparery tradeoffs in other AI applications.
10

---

Acknowledgements. We would foremost like to thank Paul Gölz for helpful technical conversa- tions and insights on the practical motivations for this research. We also thank Anupam Gupta for helpful technical conversations. Finally, several organizations for supplying real-world citizens' assembly data, including the Sortition Foundation, the Center for Climate Assemblies, Healthy Democracy, MASS LBP, Nexus Institute, Of by For, and New Democracy.
Funding and Competing Interests. This work was partially supported by National Science Foun- dation grants CCF-2007088 IIS-2024287 and CCF-17335566 and by Office of Naval Research grant Bailey Flanigan is supported by the National Science Foundation Graduate Re- search Fellowship and the Fannic and John Hertz Foundation. None of the authors have competing interests.
References
[1] József Beck and Tibor Fiala. "Integer-making theorems. Discrete Applied Mathematics, 3(1):
1-8, 1981.
[2] Gerdus Benadè, Paul Gölz, and Ariel D. Procaccia. No stratification without representation. In Proceedings of the 2019 ACM Conference on Economics and Computation (EC), pages 281-314, 2019.
[3] Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50: 3-44, 2018
[4] Dimitris Bertsimas, Vivek F. Farias, and Nikolaos Trichakis. On the salciencc-fairmess trade- off. Management Science, 58(12):2234-222 2250, 2012.
[5] Petter Brändén and Johan Jonasson. Negative dependence in sampling. Scandinavian Journal of Statistics, 3944)8830-838 2012.
[6] Eric Budish, Yeon-Koo Che. Fuhito Kojima, and Paul Milgrom. Designing random allocation mechanisms: theory and applications. American Economic Review, 103(2):585-623, 2013.
[7] Yann Chevaleyre, Ulle Endriss, Jérôme Lang, and Nicolas Maudet. A short introduction to computational social choice. In Proceedings of the 33rd Conference on Current Trends in Theory and Practice of Computer Science (SOFSEM), pages 51-69, 2007.
[8] Benjamin Doerr. Roundings respecting hard constraints. Theory of Computing Systems, 40(4): 483, 2007.
[9] Devdatt Dubhashi and Desh Ranjan, Balls and bins: study negative dependence. BRICS Report Series, 3(25):30, 1996.
[10] Devdatt Dubhashi, Johan Jonasson, and Desh Ranjan. Positive influence and negative depen- dence. Combinatorics, Probability & Computing, 16(1):29, 2007.
[11] Benjamin Fish, Jeremy Kun, and Ádám D Lelkes. A confidence-basee approach for balancing fairness and accuracy. In Proceedings of the 2016 SIAM International Conference Data Mining (ICDM), pages 144-152, 2016.
[12] B. Flanigan, P. Gölz, A. Gupta. B. Hennig, and D. Procaccia. Fair algorithms for selecting citizens' assemblies. Nature, 596:548-5535 552, 2021.
[13] Bailey Flanigan, Paul Gölz, Anupam Gupta, and Ariel D Procaccia. Neutralizing self-selection bias in sampling for sortition. In Proceedings of the 34th Annual Conference on Neural Infor- mation Processing Systems (NewrIPS), pages 6528-6539, 2020.
[14] Bailey Flanigan, Paul Gölz, Anupam Gupta, Ariel D. Procaccia, and Gili Rusak. Panelot, 2020.
http://www. http///wwwwpello.oorooo
[15] Rupert Freeman, Nisarg Shah, and Rohit Vaish. Best of both worlds: ex-ante and ex-post fairness in resource allocation. In Proceedings of the 21 ACM Conference on Economics and Computation (EC), pages 21-22, 2020.

---

[16] Rajiv Gandhi, Samir Khuller, Srinivasan Parthasarathy, and Aravind Srinivasan. Dependent rounding and its applications approximation algorithms. Journal of the ACM (JACM), 53 (3):324-360) 360, 2006.
[17] Geoffrey Grimmett. Stochastic apportionment. The American Mathematical Monthly, 111(4): 299-307, 2004.
[18] Brett Hennig and Paul Gölz. Sortition foundation stratification application, 2020. https: wtt.tt/ooooooaaaao/ee/eetteeeeet
// //github.ccm
[19] The Irish Citizens' Assembly Project, 2019, http:::/ww..iiiiieeeeeee. http://www.iiiieeasemmllllbl.b.a wetplly.ee//ooo ie/work//
[20] Sam J. Maglio and Evan Polman. Revising probability estimates: means increasing impact. Journal of Personality and Social Psychology, 111(2):141, 2016.
Why increasing likelihood
[21] Hervé Moulin. Fair division and collective welfare. MIT press, 2003.
[22] New York missions, 2021.
City
Department
of Education.
Random selection in ad-
https://wwes https://ww..schols.. schools www.c..oo/eeeeeo /orec/o////rrrlmen// enroll-grade beerrade----aeee grwggradee/ee hande//hw-sae sttoonstuuuttsggeeee wtttooofeee-ttttt-o-o-----------
vandom-numbers admissions.
[23] OECD. Innovative Citizen Participation and New Democratic Institutions. OECD Publishing, 2020.
[24] Valerie Reyna and Charles J Brainerd. judgments of risk and probability. Learning and Individual Differences, 18(1):89-107, 2008
Numeracy, ratio bias, and denominator neglect in
[25] Joel Spencer. Six standard deviations suffice. Transactions of the American Mathematical Society, 289(2):679-706, 1985.
[26] Aravind Srinivasan. Improving the discrepancy bound for sparse matrices: Better approxima- tions for sparse lattice approximation problems. In Proceedings of the 8th Annual ACM-SIAM Symposium on Discrete Algorithmss (SODA), pages 692-701, 1997.
[27] Andrew Trask, Emma Bluemke, Ben Garfinkel, Claudia Ghezzou Cuervas-Mons, and Allan Dafoe. Beyond privacy trade-off's with structured transparency. arXiv 2012.08347, 2020.
[28] Sean Jeremy Westwood, Solomon Messing, and Yphtach Lelkes. Projecting confidence: How the probabilistic horse race confuses and demobilizes the public. The Journal of Politics, 82 (4):1530-1544 2020.
12


Reviews:
[{"review_id": "nIDQ1R_dll", "reviewer": "Reviewer_tP62", "summary": "Additional to the maximally-fair consideration, this paper concerns the transparency in assembly selection problem. Specifically, the paper introduces the notion of transparency in panel selection as people can easy understand the probabilities with which each individual will be chosen for the panel and verify that individuals are actually selected with these probabilities. To achieve this, the paper studies the $m$-uniform lottery where the selection probability of each feasible panel must be multiples of $1/m$.\nThe paper shows that there exists a uniform lottery over $m$ panels that can nearly preserve the fairness of the maximally-fair unconstrained distribution over panels.  \n\nFurthermore, the paper uses the fairness loss and marginal discrepancy to quantify the closeness of such lottery and the lottery obtained in an unconstrained (no transparency requirement) setting. The paper characterizes several upper bounds for these measures and further strengthen these bounds in more structured settings. \n\nLastly, the paper conducts experiments on real-world panel selection instances to demonstrate the viability of the uniform lottery approach as a method of selecting assemblies both fairly and transparently.\n\n", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality, quality, clarity, and significance\n\nI should first mention that I feel I'm not a suitable reviewer for this paper, as the main topics are outside my areas of expertise – I have indicated this in my low confidence for my score. Although I cannot judge its novelty adequately, the problem studied in this paper is well motivated and exhibits solid applications in real-world. The theoretically results around the problem setting appear correct and the experiments seem to be comprehensive. \nIn terms of the presentation, the paper is overall well written. I would like to appreciate author's efforts in introduction section to make the motivation of the problem well defined. \n\nQuestions and comments: \n\n1. are there any connections between $|\\mathcal{K}|$ and $m$? I'm a bit confusing here, as in Line 90, you mentioned \"uniform lottery over $m$ panels\", and $\\mathcal{K}$ is the set of all feasible panels by definition in Line 129. \n\n2. I found it a bit difficult to parse the definition of Maximin, it might simply because that I'm not in this area. Seems that Maximin can not bound the difference of marginal of individuals, which is usually the focus of individual fairness. Could author explain this a little bit? \n\n3. The fairness objectives considered in the paper are more from the point of view of individuals. I'm wondering whether the current results still hold if you replace the objectives to a group fairness measure? \n\n4. Does $k\\le m$ hold for sure? If $k>m$, Theorem 3.2 seems to be meaningless?  \n\n\nOthers:\n\nLine 248: \"be most natural start with\" -> starting\n\nLine 281, you probably want to unify Nash-Welfare optimal and NW-optimal. \n\nLine 291, it should be \"Claim B.6\"?\n\n==== Post Rebuttal =====\n\nThank the authors for the clear and detailed feedback. Taking everything into consideration, I'd like to keep my rating as-is.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "20"}, "extra_text": {}}, {"review_id": "IDWNCHanQG", "reviewer": "Reviewer_oxDH", "summary": "The paper discusses transparency in the context of sortition - selecting citizens to participate in a “Citizens’ Assembly” for the purpose of making policy recommendations. They seek to design algorithms which are approximately fair (with respect to three metrics - Maximin, Leximin, and NW) while being more interpretable/verifiable than disclosing selection probabilities. Their proposed solution is to design a uniform lottery, which selects a panel uniformly at random from a precomputed set of panels. They bound the fairness loss and deviation in marginals due to restricting to uniform lotteries, discuss LP rounding algorithms to find such lotteries, and run experiments on 11 real-world instances comparing the algorithms.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written, clearly explaining the problem and techniques used. The theoretical results are basically applications of known randomized rounding algorithms over a natural linear program for this problem. The empirical results seem to indicate that imposing their definition of transparency is not harmful in terms of performance, and their methods seem implementable in practice. I liked the premise of the paper to make randomization more \"transparent\" by preselecting a set of $m$ panels, each of which could be a feasible panel, and then providing a uniform lottery on these to make the process look transparent. The main drawback to the paper is that the theoretical contribution is small—the LP rounding techniques used are not quite original. \n\n- I wonder though, if the set of m panels that are computed could be modified in a way to actually bias any uniformly selected panel. What's the guarantee that the $m$ panels selected are all possible panels? In fact, moving the upper and lower bounds slightly could have a significant impact on the kind of panels obtained by uniformly sampling the feasible set. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Societal impact and limitations are adequately addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1 (by me), 3 (by my student)"}, "extra_text": {"ethical_concerns": "Privacy is a potential concern, as was alluded to briefly in line 68. Anonymization of pool members is mentioned (presumably for the sake of privacy), but there is no discussion of the extent of recoverability of personal information."}}, {"review_id": "65aNbVaehd", "reviewer": "Reviewer_5XCt", "summary": "This paper develops and characterizes methods for selecting a citizens' panel that is representative (satisfying upper and lower quotas on demographics' representation), fair (each eligible citizen has as equal a chance as possible of being selected), and transparent (where the final panel is chosen uniformly at random from a set of candidate panels in a public lottery). ", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While I am not familiar with the sortition literature and did not check the mathematical details of this paper, I found it interesting and reasonably easy to follow. I'll let other reviewers to determine whether the problem studied (in particular, the transparency requirement) is important. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This paper is grounded in a read-world problem and is obviously concerned with the values of the people involved in and affected by sortition. The limitations of the proposed method (notably, that outside observers cannot distinguish between an individual's high selection probability being necessary for representation and being a way to rig the panel to include them) are clearly explained.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "-uBjqKZI9uC", "reviewer": "Reviewer_jMTv", "summary": "This paper gives a method for selecting panels via sortition that offers transparency that can be theoretically guaranteed and empirically audited. Their method is based on uniform lotteries and aims to achieve Maximin and Nash Welfare notions of fairness.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While previous works present the merits of sortition algorithms and give methods to better sample for sortition based committee selection, this work furthers these goals by analyzing uniform lottery with fairness constraints. \n\nThe worst-case marginal discrepancy bounds seem reasonable and it is great to see a tighter upper bound for n >> k^2 case since that is likely applicable in most applications. \n\nI am not sure I fully understand the argument of why a worst-case Maximin guarantee is a Leximin guarantee. It seems like uniform lottery does not necessarily preclude Leximin solutions. \n\nIt is nice that the authors consider both Maximin and Nash Welfare when bounding the fairness difference between the p* and \\bar{p}. What explains the extra factor of k between the Maximin and Nash upper bounds?\n\nConsidering that sortition is increasingly used in the real world, I believe this paper makes an important contribution in analyzing fairness costs of a transparent sortition process: uniform lottery. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper briefly acknowledges that the notion of transparency suggested in the paper looks at only the final stage of the panel selection process. I agree that thinking about transparency throughout the selection process is important from recruitment to final selection. \n\nAnother major consideration is that quotas may not be legally compatible in certain domains. Transparency may lead to legal challenges that violate equal protection laws (particularly in the United States). What does this mean for comparing against quota systems of sortition-based selection or stratified sampling? \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Thanks for the strong submission; the reviewers unanimously enjoyed it. 
Decision: Accept (Poster)