Is Sortition Both Representative and Fair?
Abstract
Sortition is a form of democracy built on random selection of representatives.
Two of the key arguments in favor of sortition are that it provides representation (a random panel reflects the composition of the population) and fairness (everyone has a chance to participate).
Uniformly random selection is perfectly fair, but is it representative?
Towards answering this question, we introduce the notion of a representation metric on the space of individuals, and assume that the cost of an individual for a panel is determined by the q -th closest representative; the representation of a (random) panel is measured by the ratio between the (expected) sum of costs of the optimal panel for the individuals and that of the given panel.
For k/ 2 < q ≤ k − Ω( k ) , where k is the panel size, we show that uniform random selection is indeed representative by establishing a constant lower bound on this ratio.
By contrast, for q ≤ k/ 2 , no random selection algorithm that is almost fair can give such a guarantee.
We therefore consider relaxed fairness guarantees and develop a new random selection algorithm that sheds light on the tradeoff between representation and fairness.
Introduction
Most people think of democracy as synonymous with elections.
But that has not always been the case: from the inception of democracy in ancient Athens until the American and French revolutions, democracy had typically been associated with random selection of representatives [1], a paradigm known as sortition .
These days, sortition is mainly practiced in the form of citizens’ assemblies — randomly selected groups of people who deliberate on central questions, with the goal of generating recommendations and informing policy.
The impact and prevalence of citizens’ assemblies around the world have motivated computational work on how to fairly and transparently select assembly members [2– 4].
But there are signs that sortition is becoming even more widely accepted, including its recent institutionalization in Belgium, where permanent sortition-based bodies are now working alongside the parliaments of the German-speaking region and the Brussels region.
In light of this progress, it may only be a matter of time until one of the many blueprints for sortition-based democracy [5] is implemented at the level of an entire country.
The excitement about sortition is driven by several appealing qualities, which are seen as providing solutions to some of the problems plaguing electoral democracy.
We briefly present two of them in the context of uniform selection , which selects a uniformly random panel and is considered to be the ideal sortition method [6]
• Descriptive representation : A panel selected uniformly at random is likely to reflect the composition of the population from which it was drawn.
Representation lends legitimacy to the process [7, 8], as individuals are able to identify some panelists who are similar to themselves.•
Fairness : Under uniform selection, each individual has an equal chance to participate.
Political theorists have argued that this quality realizes philosophical ideals like equality of opportunity and allocative justice [10].
By any reasonable measure of the fairness of selection probabilities — e.g., the minimum selection probability of any individual [2] — uniform selection achieves perfect fairness, as selection probabilities are equalized.
We ask: Is uniform selection also representative in a rigorous sense?
If we had an analogous measure of representation, we would be able to explore whether this is the case.
But quantifying representation poses a conceptual challenge.
Our approach.
We address this challenge by assuming that there exists a representation metric on individuals, which measures to what degree one individual represents another (smaller distance means better representation).
Readers familiar with the algorithmic fairness literature will no doubt make the connection to the similarity metric of Dwork et al.
[11], which has been criticized on the grounds that it is difficult to explicitly construct [12]; a major obstacle is that the question of whether certain features should be used to determine similarity is domain-specific and tied to legal interpretation.
By contrast, the prospect of constructing a representation metric is perhaps more tractable for practitioners. This is because the organizers of a citizens assembly (say) can choose it to be a function of a common set of features which are already routinely used by practitioners for the purpose of guaranteeing the representation of selection rules.
Moreover, some of our main results, which pertain to uniform selection, are fully independent of the metric — for these results it suffices to suppose that such a metric exists .
Note, however, that a distance metric (on individuals) does not directly define a measure of the degree to which an individual is represented by a panel.
Following very recent work by Caragiannis et al.
[13], we assume that the cost of a panel for an individual is determined by the q -th closest member of the panel, and our results are parameterized by q .
We can now define representation by taking a page from the literature on distortion in social choice [14].
Specifically, for a given selection algorithm, we measure its representation via the ratio between the social cost (sum of costs) of the optimal panel and that of the panel chosen by the algorithm, in the worst case over underlying representation metrics .
Our results.
Returning to the question of whether uniform selection is also representative, and, more generally, the eponymous question of whether sortition is both representative and fair, our answer is that “it depends” — on the value of q .
When k/ 2 < q ≤ k − Ω( k ) , where k is the size of the panel, we show that uniform selection (which is perfectly fair) achieves constant representation.
Qualitatively, we view this as providing positive answers to our questions in the regime of q > k/ 2 .
Note that this regime has a natural interpretation: each individual wants a majority of the panel to be representative of themselves.
This is especially justifiable when the panel makes decisions or recommendations through voting, which is often the case in citizens’ assemblies.
By contrast, for the regime of q ≤ k/ 2 , we prove that any selection algorithm that chooses each individual with probability somewhat higher than q/n , where n is the number of individuals, must in the worst case have representation of precisely 0.
This result clearly applies to uniform selection, where the minimum selection probability is k/n , and it motivates us to consider weaker fairness guarantees.
We design an algorithm, R ANDOM R EPLACE , which selects each individual with probability at least q/n and has a nontrivial representation guarantee of 1 / ( q + 1) for any value of q .
Finally, we evaluate the average-case representation of uniform selection and R ANDOM R EPLACE on inferred metrics derived from two demographic datasets.
For q > k/ 2 , our experiments show that uniform selection achieves representation consistently greater than 55%.
Even for q ≤ k/ 2 , in contrast to our worst-case result, uniform selection achieves good average-case representation.
These results suggest that, in practice, uniform selection may be a good choice for all q -cost distance functions.
While R ANDOM R EPLACE outperforms uniform selection across the board in terms of representation (at the expense of fairness), its advantage is small for values of q around k/ 2 and close to k , which means it does not offer a good representation-fairness tradeoff in those regimes.
Lastly, for both algorithms, we observe a spike in representation at q = k/ 2 , which demonstrates that the chasm between provable guarantees for q ≤ k/ 2 versus q > k/ 2 is not merely a theoretical curiosity.
Related work.
The design of practical, fair and transparent algorithms for selecting citizens’ assemblies was explored in several previous papers [2–4] — two of which appeared in previous
NeurIPS conferences.
Assemblies are required to be representative of the population with respect to features like gender, age, ethnicity, education and geography.
This is generally done by setting quotas on individual features; for example, a panel of 100 people might be required to include at least 48 men, at least 48 women, and at least 2 people who identify as non-binary.
The challenge is that an assembly is selected from a pool of volunteers, and this pool is typically unrepresentative of the population due to self-selection bias.
Uniformly random selection from the pool, therefore, would likely itself result in an unrepresentative panel.
Instead, the primary selection algorithm advocated by Flanigan et al.
[2] computes a distribution over quota-compliant panels that (roughly speaking) maximizes the minimum selection probability of any volunteer, thereby maximizing fairness subject to these hard demographic constraints.
By contrast, like Benadè et al.
[15] and the political theory literature, we take a longer-term view: We are interested in random selection directly from the population, which is a hallmark of some plans for sortition-based democracy [5].
In addition, we take a fundamentally different, and arguably more nuanced, view of representation.
Indeed, our framework can theoretically accommodate considerations of intersectionality (to what degree is a rural, college-educated man represented by a rural, college-educated woman?) and could also capture a more holistic analysis of the composition of the panel (to what degree is a rural, college-educated man represented by a panel that includes 10 rural, college-educated men and 90 urban women with no college education?).
Lang and Skowron [16], Celis et al.
[17], and Do et al.
[18] also study the problem of creating committees or assemblies in which the features of the participants satisfy some desired quotas, but in the absence of an underlying representation metric and any fairness constraints.
Our approach to evaluating representation through a metric is rooted in spatial theories of voting from political theory [19, 20].
The idea of measuring how poorly a panel represents an individual by the distance of the q -th closest panel member to the individual was introduced by Caragiannis et al.
[13] in the context of committee elections.
Finally, aiming to minimize the total misrepresentation across all people and comparing that to the optimal panel makes our representation measure the inverse of distortion in voting theory [21, 22].
In voting, it is assumed that we only have partial access to the metric in the form of voters’ ranked preferences over the candidates, which are induced by distance comparisons.
In contrast, our results for uniform selection use no knowledge of the underlying metric, while our other algorithms assume complete access.
When selecting a single candidate as the winner, it is known that the best distortion achievable by deterministic selection is 3 (which maps to 1 / 3 representation in our formulation) [23].
Our setting is closer to committee selection, where a committee of k candidates is selected.
Here, Caragiannis et al.
[13] show that when each voter measures her distance to the q -th closest committee member, there is a trichotomy: the best possible distortion is infinite when q ≤ k/ 3 , linear in the number of voters when q ∈ ( k/ 3 , k/ 2] , and 3 for deterministic selection when q > k/ 2 .
Sortition may be viewed as a special case of randomized committee elections, in which the set of candidates is the same as the set of voters.
Hence, all the positive results from Caragiannis et al.
[13] carry over in the absence of any fairness constraints.
However, our results show that when (perfect) fairness in selection probabilities is sought in conjunction with representation, the distortion becomes infinite (zero representation) for all q ≤ k/ 2 but constant distortion can still be achieved for q > k/ 2 .
The idea of the set of voters acting as the set of candidates was explored by Cheng et al.
However, they model infinitely many voters using a continuous distribution over the metric space.
Preliminaries
For all t ∈ N , define [ t ] = { 1 , . .
. , t } .
Let N =
[ n ] be the set that indexes the underlying population.
A panel P is a subset of the population.
Let S k ( N ) denote the set of all subsets of N of size k .
(We omit N when it is clear form the context.)
The population lies in a metric space with distance d , which we view as the representation metric discussed earlier.
For each i, j ∈ N , d ( i, j ) denotes the distance between i and j ; d is a metric if the following properties are satisfied: (a) d ( i, j ) ≥ 0 , and d ( i, j )[-= 
-]
= if and only if i = j , (b) d ( i, j ) =
d ( j, i ) , and (c) for each i, j, ℓ ∈ N , d ( i, ℓ ) + d ( ℓ, j ) ≥ d ( i, j ) .
The last property is known as the triangle inequality.
An instance of our problem is given by the underlying population along with distances as defined by d ; hereinafter, we simply denote such an instance by d .
Given a panel P of size k and a positive integer q ∈ [ k ] , the q -cost of individual i for P , denoted by c q ( i, P ; d ) , is equal to the distance of i from her q -th closest representative in P .
Note that for q = 1 , we have c 1 ( i, P ; d ) =
min j ∈ P d ( i, j ) and for q =
k we get c k
( i, P ; d ) = max j ∈ P d ( i, j ) .
Let top q
( i, P ; d ) be the set of q closest members of P to i (with ties broken arbitrarily).
The q -social cost of panel P is given by SC q ( P ; d )
= (cid:80) i ∈ N c q
( i, P ; d ) , i.e., the sum of the q -costs over all individuals.
(Observe that if q = 1 then SC q ( P ; d ) is the standard k -medians clustering objective evaluated for centers P .)
We omit d from the notation when it is clear from the context.
In this setting, a selection algorithm
A k,q parameterized by k and q takes as input the metric d and outputs a distribution over all panels of size k .
We are especially interested in the uniform selection algorithm, denoted by U k , that always outputs a uniform distribution over S k , regardless of q .
In other words, it does not take into account the underlying metric space or q , but instead outputs a committee of size k chosen uniformly at random.
We now formally introduce the fairness and representation of a selection algorithm. Fairness: One appealing property of uniform selection is that each individual is selected to be part of the panel with probability exactly equal to k/n , i.e., Pr[ i ∈
U k ]
= kn .
In particular, all individuals have an equal chance of being chosen.
We call this property perfect fairness ; in general an algorithm A k,q provides perfect fairness when for each instance d , it ensures that min i ∈ N Pr[ i ∈ A k,q ] = k/n .
When perfect fairness is too restrictive, we relax this constraint by allowing individuals to be selected with probability less than k/n .
In this case, the fairness of an algorithm A k,q is the worst-case ratio of the minimum probability of an individual to be selected by the algorithm and the ideal selection probability of k/n .
Formally,
[Equation]
Representation: The other key property we would like to measure is representation.
To do so, we consider a panel to be a good representative of the whole population when its q -social cost is not much larger than the best possible.
In other words, a selection algorithm
A k,q that outputs a distribution over the different committees of size k provides good representation when the expected q -social cost of the panel is similarly small.
More formally, we define the representation of a selection algorithm
A k,q as the worst-case ratio of the minimum possible social q -cost of any panel and the expected q -social cost of the panel chosen by A k,q over all possible instances; i.e., [Equation]
Representation with Perfect Fairness for q > k/
We begin by considering the case that q > k/ 2 .
We show that in this case uniform selection is asymptotically optimal with respect to representation among all selection algorithms that are perfectly fair.
Moreover, the representation of uniform selection is constant for any q = c · k for 1 / 2 < c < 1 .
Theorem 1.
For q > k/ 2 , uniform selection satisfies repr q ( U k ) ≥ 12 · k − q +1 k .
A crucial property that we exploit in this section is that the q -costs of the individuals satisfy the triangle inequality when q > k/ 2 .
This observation was first made by Caragiannis et al.
[13]; we present the lemma below for completeness.
Lemma 1.
For q > k/ 2 , individuals i, j ∈ N , and a panel P ,
c q ( i, P ; d ) + c q ( j, P ; d ) ≥ d ( i, j ) .
Proof.
Let T i =
top q ( i, P ) and T j = top q
( i, P ) be the q closest neighbors of i and j , respectively, in the panel P .
As | T i | =
| T j | > k/ 2 , there exists an individual k ∈ T i ∩ T j .
Therefore, d ( i, j ) ≤ d ( i, k ) + d ( k, j ) ≤ c q ( i, P ) + c q ( j, P ) .
We use this observation to lower bound the social cost of the optimal committee.
Lemma 2.
For q > k/ 2 , the q -social cost of the optimal panel P ∗ is at least
[Equation]
Proof.
By applying Lemma 1 for all pairs of individuals ( i, j ) , we get
[Equation]
The q -cost of each person appears exactly 2( n − 1) times on the left hand side.
Thus,
[Equation]
We are now ready to prove the theorem.
Proof of Theorem 1.
For any committee P of size k ,
[Equation]
where in the first inequality the minimum is upper bounded by the average.
Therefore, the expected social cost of uniform selection is at most
[Equation]
By Lemma 2 and the upper bound shown above [Equation] we have repr
[Equation]
U k
[Equation]
≥ [Equation] In the proof above, the only property of uniform selection we use is that the marginal probabilities are equal to k/n .
Hence, this lower bound also holds for any perfectly fair selection algorithm.
We next establish an upper bound on the representation of any perfectly fair selection algorithm.
It shows that the lower bound of Theorem 1 is tight up to a factor of 4.
Theorem 2.
For any q > k/ 2 , every selection algorithm
A k,q with fairness ( A k,q ) = 1 satisfies {+repr q ( A k,q ) ≤ 2 · k − q +
 k+}
+1 .
Proof. First, note that if q = k +12 , the statement trivially holds, since repr q ( A k,q ) ≤ {+2 · k − q +1 k +
 =+}
which is true for any algorithm.
Thus, we assume q > k +12 .
Consider an instance with n = k +individuals where k − q + 1 individuals are located at 0 and q individuals are at 1 , denoted by Nand N 1 , respectively.
Any committee of size k leaves one person out of the committee, and for each individual i ∈ N , this happens with probability of
[Equation]
Individuals in N 0 will always have a q -cost of 1 , because k + 1 − q <
k +12 <
q individuals are located there.
Therefore, E P ∼A k,q [ (cid:80) i ∈ N 0 c q ( i, P )] =
| N 0 | .
For individuals in N 1 , their q -cost is 1 if and only if less than q individuals are selected from N 1 , i.e., the single person left out of the committee is located at 1 .
This event happens with probability
[Equation]
where the first equality comes from the fact that the events are disjoint (which holds because any committee leaves out exactly one individual).
Therefore, E P ∼A k,q [ (cid:80) i ∈ N 1 c
[Equation]
N
[Equation]
| · | N 1 | k +1 , and the expected social cost of any perfectly fair algorithm is
[Equation]
The optimal committee would leave out a person from N 0 and achieve a social cost of | N 0 | .
Therefore, the representation of any algorithm with perfect fairness is at most [Equation]  Representation with Relaxed Fairness for q ≤ k/ In stark contrast to the case of q > k/ 2 , uniform selection and, more generally, any perfectly fair selection algorithm, cannot obtain bounded representation when q ≤ k/ 2 .
In fact, the following theorem shows that selection algorithms with fairness strictly more than q +( k mod q ) k
(which itself is upper bounded by (2 q − 1) /k ) suffer from unbounded representation.
The proof is in Appendix A.
Theorem 3.
For q ≤ k/ 2 , ϵ > 0 , and any selection algorithm
A k,q with fairness ( A k,q ) ≥ q +( k mod q ) k
+ ϵ , repr q ( A k,q ) is 0.
Although bounded representation is not feasible with fairness slightly larger than qk , we design a selection algorithm that, given an α -representative panel, can achieve representation of at least αq +1 with fairness of qk .
In particular, starting from the optimal panel ( α = 1 ), it achieves 1 q + representation.
Although finding the optimal panel given the metric space is an NP-hard problem for most metric spaces, computing a constant-factor approximation (i.e., constant α ) is feasible in polynomial time [26], as we explain in Section 5.
Our Algorithm, R ANDOM R EPLACE q , is given in Algorithm 1.
It starts from a panel P , randomly selects a panel S of q individuals, and replaces individuals in P with individuals in S as follows.
First, individuals in S ∩ P remain in the final panel.
Then, for i ∈ S \ P , swap i with one of its q -closest neighbors in the optimal panel top q ( i, P ) that has not been replaced by the algorithm yet.
The next theorem establishes the fairness and representation guarantees of R ANDOM R EPLACE .
Theorem 4.
For any q ∈ [ k ] and any panel P with repr q ( P ) =
α , we have that repr q ( R ANDOM R EPLACE q ( P )) ≥ αq +1 and fairness ( R ANDOM R EPLACE q ) ≥ qk .
Proof.
Let S ⊆
N be a set of size q chosen uniformly at random.
We denote with P S the panel that is returned from the algorithm.
First, we show that Line 4 of the algorithm is valid.
The algorithm reaches this line when it considers i ∈ S \ P S , meaning that i is not included in the panel P S and therefore is not included in P .
Hence, top q ( i, P ) \ S cannot be empty since | top q ( i, P ) | = q , | S | =
q and there exists i in S but not in top q ( i, P ) .
We see that every individual in S is included in P S as in Line 5 the algorithms ensures that each such individual is included and is never excluded afterwards.
Hence, as each individual is chosen in S with probability at least q/n , we can see that fairness ( R ANDOM R EPLACE q ) ≥ q/k .
Now, we prove that for any S ∈ S q and any individual i ′ ∈ N ,
[Equation]
The second inequality holds as the maximum is at most the sum.
Therefore, we focus on the first inequality.
If c q ( i ′ , P S ) ≤ c q ( i ′ , P ) , then it trivially holds.
Otherwise, c q ( i ′ , P S ) >
c q ( i ′ , P ) .
In this case, we can show that there exists some i ∈ S \ P such that d ( i ′ , i ) ≥ c q ( i ′ , P S ) and j i ∈ top q ( i ′ , P ) .
First, note if for each i ∈ S \ P ∗ , j i does not belong in top q ( i ′ , P ) , then it is not possible that c q ( i ′ , P S ) >
c q ( i ′ , P ) , since top q ( i ′ , P ) ⊆
P S .
Next, suppose for contradiction that for every i that was included in P S when j i ∈ top q ( i ′ , P ) was excluded from it, it holds that d ( i ′ , i ) <
c q ( i ′ , P ) .
Then, in P S there are | top q ( i ′ , P ) \∪ i ∈ S \ P ( { j i }∩ top q ( i ′ , P ))
| individuals that have distance at most c q ( i ′ , P ) <
c q ( i ′ , P S ) from i ′ and |{ i ∈ S \ P : j i ∈ top q ( i ′ , P ) }| individuals that have distance less than c q ( i ′ , P S ) from i ′ .
Note that
[Equation] and hence we get that there are at least | top q ( i ′ , P ) | =
q individuals in P S with distance strictly less than c q ( i ′ , P S ) from i ′ , which is a contradiction.
From the above observation , we have that
[Equation]
where the penultimate inequality holds because j i ∈ top q ( i ′ , P ) and j i ∈ top q ( i, P ) from the definition of j i .
This proves (1).
Summing (1) over all i ′ ∈ N , we have SC q ( P S ) ≤ SC q ( P ) + n · (cid:80) i ∈
S c q ( i, P ) .
Taking the expectation of this equation with respect to S , and using the fact that Pr[ i ∈ S ] = q/n , we have E [Equation] [Equation]
· (cid:88)
[Equation] , that is, 1 q +1 · E [SC q ( P S )] ≤ SC q ( P ) .
Combined with the assumption that P is α -representative, i.e., α · SC q ( P ) ≤ min P ′ ∈
S k ( N ) SC q ( P ′ ) , we get repr q ( R ANDOM R EPLACE q
( P )) ≤ αk +1 .
The next theorem shows that when q = Ω( k ) , R ANDOM R EPLACE q attains an asymptotically optimal representation-fairness tradeoff.
Theorem 5.
For any q ≤ k/ 2 such that k mod q = 0 , every selection algorithm
A k,q with fairness ( A k,q ) ≥ q/k satisfies repr q ( A k,q ) ≤ k/q 2 . Proof.
Let m = k/q . Consider an instance with n > 2 k individuals on the real line, where one individual i 0 is located at 0 , ⌈ n/m ⌉ − 1 people are at 1 , and at least ⌊ n/m ⌋ people are located at each position j ∈ { 2 , . . , m } .
This way, there are at least n/m − 1 = ( n/k ) · q − 1 ≥ 2 q − 1 ≥ q individuals located at each j ∈ [ m ] . Any optimal panel would include q individuals from each position j ∈ [ m ] , which results in a q -cost of 1 for i 0 and a q -cost of 0 for the rest. Hence, SC q ( P ∗ ) = 1 .
However, any selection algorithm with fairness of at least q/k selects i 0 with probability of at least q/n .
When i 0 is selected, there must exist a group j ∈ [ m ] from which the algorithm selects at most q − 1 people, incurring a q -cost of 1 for at least n/m people (person i 0 and at least n/m − 1 people at position j ).
Hence,
[Equation]
which completes the proof.
From the above theorem, it follows that Algorithm 1 achieves the highest possible fairness of q/k subject to positive representation.
Experiments
Next, to complement the worst-case analysis of the selection algorithms we have considered so far, we conduct an empirical comparison between the average-case q -social cost and representation ( repr q ) of the different selection methods.
Datasets.
Data on the metric-structure preferences of groups in their full richness are difficult to come by, but it is reasonable to expect that the extent to which individuals feel well-represented by one another is at least partly a function of their relative characteristics along some observable features.
We therefore begin with two datasets that express the characteristics of populations along a range of features, and randomly construct synthetic metric preferences from these feature signatures.
Adult dataset.
Our first source of demographic data is the UCI Adult dataset, which was derived from the 1994 Current Population Survey of the US Census Bureau, and is made available by the UCI Machine Learning Repository under a CC BY 4.0 license [27].
It contains a range of demographic variables principally related to employment.
Our experiments do not require Adult to be representative of any actual population, nor should this an assumption be made lightly [28].
For
Adult we choose the features workclass , education , marital status and sex .
Adult contains n = 30162 individuals with values for each feature, who may be viewed as a distribution over the 721 unique feature vectors which they collectively hold.
ESS dataset.
Our second source of demographic data is the European Social Survey ( ESS ) [29], which is made available by the Norwegian Centre for Research Data under a CC BY 4.0 license.
We use the ESS Round 9 (2018) data, which consists of 46,276 people in 27 countries, and contains ∼features regarding socioeconomic demographic, political beliefs, geographical region, house-hold composition, personal values, media use and trust, etc.
Most of the features are country-specific, which leaves roughly 250 features available per country, while each country has between 781 and 2745 entries (with a mean of 1713).
Each entry is assigned an analysis weight which is aimed to correct the differential selection probabilities.
In contrast to our experiments with Adult , we use all of the available features available in ESS .
We report the experiment results based on the data of the
United Kingdom (2204 entries).
Similar results were obtained using the data from other countries.
Metric construction.
We choose some set of features along which to evaluate the individuals in the population.
These are the features which will inform our metric.
For each feature s , F s is the set of possible values that this feature can take.
Each individual i is then represented as a vector of feature values, where f si is the value that i has for feature s .
For categorical features (e.g., marital status, sex), we define d ( i, j ; s )
=
− 1 { f si = f sj } .
For range (e.g., income) features we define d ( i, j ; s ) =
| f si − f sj | / (max i ′ ,j ′ | f si ′ − f sj ′ | ) , where the normalization makes d ( i, j ; s ) ∈ [0 , 1] .
Then, for each feature we sample a weight w t ∼ U [0 , 1] uniformly at random from the interval [0 , 1] .
Finally the distance between individuals i and j and our metric d is defined to be d ( i, j ) :
= (cid:80) s w s · d ( i, j ; s ) .
For the empirical evaluation of our selection algorithms on these randomly generated metrics, we suppose that this is in fact the distribution for a population large enough that there are at least k individuals with any given feature vector.
As these metrics represent populations, the q -social costs in Figures 1 and 2 are normalized by population size.
Figures 1 to 3 depict data averaged overrandom metrics constructed in this manner, and the error bars show the standard error of the mean.
20 k 0.
S C _ q q=1 q=2 q= q=10 q= ( [Figure] ELECTION   20 k 0.4 0.
S C _ q q=1 q=2 q= q=10 q=
[Figure]
10 q 0.
S C _ q opt opt
[Figure]
apx
[Figure]
Proxy for the optimum.
In evaluating U NIFORM S ELECTION and R ANDOM R EPLACE , we use a proxy for the optimal q -social cost, since n and k are too large to support finding SC q ( P ∗ , d ) exactly.
This selection algorithm O PT P ROXY (indicated as ‘opt_apx’ in legends) is an implementation of the fault-tolerant metric k -medians algorithm of Kumar et al.
[26], which guarantees a constant-factor approximation to SC q ( P ∗ , d ) .
This algorithm uses a constant-factor metric k -medians algorithm as a primitive; we implement the local search algorithm of Arya et al.
[30] with single swaps.
More specifically, O PT P ROXY works as follows.
It finds the (approximately) optimal
⌊ k/q ⌋ -median solution and picks q individuals from each of these ⌊ k/q ⌋ positions, and then selects the remaining k mod q people uniformly at random.
To evaluate the fidelity of O PT P ROXY we compare it with SC q ( P ∗ , d ) for 100 metrics d constructed by drawing 30 randomly chosen feature vectors from the supports of Adult instances described above.
For panels of size k = 10 and all values of q we find that O PT P ROXY recovers SC q ( P ∗ , d ) exactly (Figure 2).
The fact that the q -social cost attained by O PT P ROXY for fixed q and q ∈ ( k/ 2 , k ] appears constant, on the other hand, is not universally true of SC q ( P ∗ , d ) .
These plateaus are due to the way O PT P ROXY selects panels by returning q copies of the optimal 1 -median location.
Since P ∗ q need not be of this form, it is interesting that it exhibits this same step-like behavior in Figure 2.
Experimental results.
In Figure 1a, we see U NIFORM S ELECTION behaving as expected.
For a fixed k , the q -social cost of a uniformly random panel is reliably higher for larger q , and for a fixed q , it decays smoothly as we increase the panel size k .
Under the same conditions, O PT P ROXY behaves similarly (Figure 1b) but the decay is not as smooth due to the reasons explained above.
Here, all of our approximately optimal panels P q start at the same value of SC q ( P q , d ) when q = k and decay as q remains fixed and the size of the panel k increases.
Figure 3 shows the ratio between the q -social cost obtained by different selection algorithms and the q -social cost of O PT P ROXY , which is an approximation of their representation, averaged over 100 randomly generated metrics.
Specifically, we evaluate U NIFORM S ELECTION and R ANDOM R EPLACE r (indicated as ‘unif_select’ and ‘rand_replace_ r ’ in legends, respectively), the latter being a generalization of R ANDOM R EPLACE q that replaces r individuals instead of q in the input panel and achieves representation at least α/ ( r + 1) (when the input panel has representation α ) and fairness at least r/k .
We compare these algorithms for a range of r and q , on the Adult dataset (Figure 3a) with panels of size k = 20 and on the ESS (Figure 3b) dataset with panels of size k = 40 .
Representation of U NIFORM S ELECTION .
As expected, the representation of U NIFORM S ELECTION is the lowest for all k and q .
However, it still achieves consistently good representation across the board, with representation dipping below 50% only on the Adult dataset with q close to k .
We find it especially interesting that, on both datasets, U NIFORM S ELECTION exhibits good representation in the regime of q ≤ k/ 2 , as this contrasts with our worst-case result.
We conclude that, in real-world datasets, U NIFORM S ELECTION may be quite representative even with respect to the [Figure] q -social cost measure when q ≤ k/ 2 .
Overall, we view these results a providing empirical support for the use of U NIFORM S ELECTION in practice.
Representation of R ANDOM R EPLACE .
Figure 3 shows that R ANDOM R EPLACE r achieves significant improvements in representation compared to U NIFORM S ELECTION for small values of r — at the cost of fairness, of course.
This tradeoff may be justified for some values of q , but the advantage of
R ANDOM R EPLACE is much smaller around q = k/ 2 and when q approaches k , so for these regimes
R ANDOM R EPLACE should not be in the running.
Shift at q = k/ 2 .
In both Figures 3a and 3b, there is a sudden improvement in the representation of
U NIFORM S ELECTION at q = k/ 2 .
This provides empirical support for the peculiar transition in the q -social cost occurring q = k/ 2 , which is predicted by the theory and is the reason we separated
Sections 3 and 4.
Discussion
Our results in Section 4 show that in some cases, relaxing fairness requirements allows improving representation dramatically.
More generally, it is interesting to understand the tradeoff between representation and fairness, and to chart the Pareto frontier.
In Appendix B, we take some first steps in this direction.
One observation is the aforementioned generalization of R ANDOM R EPLACE that replaces r individuals instead of q .
We also show that a random-dictatorship-like algorithm gives nontrivial fairness and representation guarantees in the regime of q > k/ 2 .
However, there remain several open questions: for example, when q ≤ k/ 2 , what level of fairness can be achieved if we seek constant representation?
We focused our attention on the q -cost formulation of Caragiannis et al.
[13], in which each individual measures their distance to the q -th closest panel member.
One can analyze the representation-fairness tradeoff with other cost functions.
For example, what if different individuals use different values of q ?
Another appealing choice is when each individual measures the average distance to all panel members; in Appendix C, we show that for this cost function, uniform selection achieves representation of at least 1 / 2 .
More broadly, one can use measures of representation other than the total cost to all individuals.
For example, one may wish to use a selection algorithm that minimizes the maximum cost to any individual, or strikes a balance between maximum cost and total cost.
We hope that answers to some of these questions will lead to a better understanding of the strengths of sortition, and to new ways of realizing this democratic paradigm. Acknowledgements.
This work was partially supported by the National Science Foundation under grants IIS-2147187, CCF-2007080, IIS-2024287, and CCF-1733556; by the Office of Naval Research under grant N00014-20-1-2488; and by an NSERC Discovery Grant.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[N/A] (c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[N/A] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Proof. Assume that n > 2 · max { (cid:112) kq/ϵ, k +1 } , and let m = ⌊ k/q ⌋− 1 .
Consider the real line, and suppose there are sets of q individuals at each position in { 1 , 2 , . . . , m } , denoted by X 1 , . . . , X m , respectively, and the set of remaining n − mq individuals, denoted by X m +1 , is at position m + 1 .
The optimal panel P ∗ would have at least q people from each position, i.e., | P ∗ ∩
X i | ≥ q for all i ∈
[ m + 1] .
The q -cost of each person for P ∗ is 0 as at least q people are selected from her own position.
Hence, SC q ( P ∗ )
= 0 .
Turning to the analysis of A k,q , we claim that
[Equation]
To prove this, note that since each individual is included with a marginal probability of at least q +( k mod q ) n + ϵ , we have
[Equation]
We will show that the right hand side is at least q +( k mod q )
+ ϵ .
Because mq < k , q + k mod q < 2 q , and n − mq > n − k ≥ n/ 2 + 1 , the right hand side is at least
[Equation]
where the inquality follows from our choice of n > 2 (cid:112) kq/ϵ .
This establishes Equation (2).
Now, as the panel size is k , it holds that (cid:80) i ∈ [ m +1] E P ∼A k,q [ | X i ∩ P
| ] = k. By Equation (2),
[Equation]
Therefore, there exists i ∈
[ m ] such that E P ∼A k,q [ | X i ∩ P | ] ≤ q − ϵ/q .
Using Markov’s inequality,
[Equation]
Thus, with probability at least ϵ , less than q people are selected from position i , in which case the q -cost of each person in X i will be at least 1 .
Hence, E P ∼A k,q [SC q ( A k,q )] ≥ qϵ while SC q ( P ∗ ) = 0 .
B Tradeoffs between Representation and Fairness
We start with the case of q > k/ 2 and show that a simple algorithm, which is a variant of the natural random dictatorship rule, provides constant representation by sacrificing some quantity of perfect fairness.
Specifically, the algorithm R ANDOM D ICTATOR k,q , presented as Algorithm 2, works as follows:
Given an instance d , it chooses an individual i from the underlying population uniformly at random, and returns the panel P = top q ( i, N ; d ) ∪ S , where top q ( i, N ; d ) is the set of q people closest to i (we break ties in a way to ensure that this contains i herself), and S is a panel of size k − q chosen uniformly at random from the remaining people.
Theorem 6.
For any q > k/ 2 , it holds that
[Equation]
and fairness
[Equation]
Proof.
We start by proving the fairness guarantee of the algorithm.
Note that each individual i is included in the panel P returned by R ANDOM D ICTATOR k,q either if i is selected at the first step, which happens with probability 1 /n , or if i is not selected in the first step, it is selected in the second step with probability at least ( k − q ) / ( n − q ) .
Hence, the probability of i being selected is at least (1 /n )
+ (1 − 1 /n ) · ( k − q ) / ( n − q ) ≥
( k − q + 1)
/n , yielding fairness ( R ANDOM D
ICTATOR k,q ) ≥ ( k − q + 1) /k .
For q > k/ 2 , Caragiannis et al.
[13] (Corollary 2 ) show that random dictatorship, i.e. returning a panel minimizing q -cost to a randomly selected individual i , achieves a representation of at least 1 / 3 .
The panel returned by R ANDOM D ICTATOR k,q consists of q closest neighbors of i which obtains the minimum q -cost with respect to i , and fills the other k − q members of this panel randomly which does not affect the q -cost of the returned panel to i .
Hence, R ANDOM D ICTATOR k,q can be seen as a variant of the random dictatorship rule which randomly breaks ties between top panel choices of a randomly selected individual.
Now, we turn our attention to the case that q ≤ k/ 2 .
In Section 4, we introduced R ANDOM R EPLACE q with fairness q/k and representation α/ ( q + 1) if given an α -representative panel.
In fact, if we replace q with any r ∈ [ q ] , we can show that the algorithm provides representation of at least α/ ( r +1) with fairness r/k .
Essentially, R ANDOM R EPLACE r chooses a subset S of the underlying population with size r instead of q uniformly at random in Line 1 of Algorithm 1.
Proposition 1.
For any q ∈ [ k ] , r ∈ [ q ] , and panel P with repr q ( P )
=
α it holds that repr q ( R ANDOM R EPLACE r
( P )) ≥ α r +and fairness ( R ANDOM R EPLACE r ) ≥ r k .
We omit the proof of this proposition as it is essentially identical to the proof of Theorem 4 with q replaced by r in the appropriate places.
C Average Cost Function
Let c avg ( i, P )
=
1 k (cid:80) j ∈ P d ( i, j ) denote the average cost of panel P of size k to an individual i .
Similarly, define SC avg ( P ) =
(cid:80) i ∈ N c avg ( i, P ) , and let repr avg ( A k ) denote the representation of a selection algorithm
A k with respect to the average cost function.
It turns out that uniform selection (or any algorithm with perfect fairness) performs very well with the repr avg objective and achieves a representation of 1 / 2 .
Proposition 2.
For all k ≥ 1 , uniform selection satisfies repr avg ( U k ) > 1 / 2 .
Proof.
Sort the population as N =
( i 1 , i 2 , . .
, i n ) in a non-decreasing order of SC( i ℓ ) =
(cid:80) i ∈ N d
( i, i ℓ ) , so that SC( i 1 ) ≤ SC( i 2 ) ≤ . . .
≤ SC( i n ) .
Note that for any panel P , SC avg ( P )
=
k (cid:80) i ∈ P SC( i ) , so the optimal panel is P ∗
= { i 1 , . . .
, i k } .
Then,
[Equation]
Note that { i 1 } = min P ′ ∈
S
k
=1 ( N ) SC q =1 ( P ′ ) is the optimal panel for the case where q = k = 1 .
As q > k/ 2 in this scenario, by Lemma 1, we have
[Equation]
[Equation]
The average social cost of uniform selection is
[Equation]
{+where in the last transition we used the fact that d ( i, i ) = 
+}
and the marginal inclusion probabilities are equal to k/n .
Putting all together, we have that repr avg ( U k ) ≥ n 2( n −
[Equation]
> 12 .
D Experimental Results D.1 Experiment Plots
[Figure]
D.2 Computation Time of O PT P ROXY
The most computationally expensive task in our experiment is computing O PT P ROXY , which is used as a subroutine in the R ANDOM R EPLACE method.
We report the running time of our implementation, as described in Section 5, for low to very high values of k .
We used the ESS UK dataset with n = 2204 and a standard laptop (quad-core, 1.7GHz CPU, 16GB RAM).
The computation took less than 15 seconds for k = 100 , which includes computing the pair-wise distance matrix ( 5 seconds) and running O PT P ROXY ( 7 seconds).
O PT P ROXY itself scales well for practical values of k as shown in Table 1.
[Figure]
[Table]
[Figure]