SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Soroush Ebadian
University of Toronto soroush@cs.toronto.edu
Aris Filos-Ratsikas
University of Edinburgh Aris.Filos-Ratsikas@ed.ac.uk
Mohamad Latifian
University of Toronto latifian@cs.toronto.edu
Nisarg Shah
University of Toronto nisarg@cs.toronto.edu
With a rapid growth in the deployment of AI tools for making critical decisions (or aiding humans in doing so), there is a growing demand to be able to explain to the stakeholders how these tools arrive at a decision. Consequently, voting is frequently used to make such decisions due to its inherent explainability. Recent work suggests that using randomized (as opposed to deterministic) voting rules can lead to significant efficiency gains measured via the distortion framework. However, rules that use intricate randomization can often become too complex to explain to the stakeholders; losing explainability can eliminate the key advantage of voting over black-box AI tools, which may outweigh the efficiency gains.
We study the efficiency gains which can be unlocked by using voting rules that add a simple randomization step to a deterministic rule, thereby retaining explainability. We focus on two such families of rules, randomized positional scoring rules and random committee member rules, and show, theoretically and empirically, that they indeed achieve explainability and efficiency simultaneously to some extent.
In the past decade, AI and machine learning solutions have been deployed ubiquitously to make increasingly critical decisions that affect human lives. Consequently, there is a growing demand for these models and their decisions to be explainable [1, 2]. The literature makes a distinction between two types of explanations: outcome explanations, which explain to the stakeholders why the chosen outcome was selected in a given instance, and procedural explanations, which explain to the stakeholders the procedure of choosing outcomes across all possible instances. Much of the explainable AI (XAI) literature focuses on outcome explanations because many black-box AI solutions used in practice are too complex to admit simple procedural explanations [3].
However, there are several drawbacks of outcome explanations. First, it opens up the possibility of post-hoc explanations for why an outcome was selected. These are susceptible to adversarial reasoning that hides biases [4]. Also, psychological research suggests that people's perception of fairness of an outcome depends not only on the outcome itself, but also on the process by which the outcome is selected [5, 6], and the same outcome may be perceived as fair or unfair depending on the process used [6]. This motivates the need for procedural explanations. Note that an intuitive explanation of the procedure to select outcomes already serves as a rudimentary justification for why a given outcome was selected.
To that end, we turn our attention to voting. While explainability is a nascent demand in the AI ecosystem, voting rules, historically deployed for political decision-making, have always battled with the need to be able to explain to the voters how the winner of an election is chosen. Thus,
most prominent voting rules admit intuitive procedural explanations. Due to this key advantage over black-box AI solutions, they have been used to automate decision-making in a variety of applications such as designing recommender systems [7, 8], information extraction [9], collaborative filtering [10], ensemble learning [11], and game-playing by AI agents [12]. The advantage is more apparent when the decisions at hand are more significant. For example, Noothigattu et al. [13] and Kahng et al. [14] propose the design of a voting-based virtual democracy system that can automate ethical decision-making in AI systems. When Lee et al. [15] applied this framework to automate the distribution of food donations, they found that their stakeholders appreciated the fact that voting-based decision-making "embodied democratic values" and being able to provide easy explanations "allowed them to understand how algorithmic recommendations were made".
However, most prominent voting rules are deterministic because their primary use case is making infrequent, high-stakes democratic decisions, for which randomization is generally unpalatable [16]. Aside from selected applications such as forming citizens' assemblies, juries, and independent redistricting commissions, lottery is seldom used to select representatives [17]. But in AI applications, it is common to make frequent, low-stakes decisions, for which randomization is well-suited.
Research on voting theory suggests that allowing the voting rule to randomize has numerous benefits. It can be essential for avoiding the tyranny of the majority and guarantee minority representation [18, 19]. Randomization also acts as a barrier to manipulations by strategic agents by circumventing the Gibbard-Satterthwaite impossibility [20, 21]. Most importantly, it can unlock significant efficiency gains over using deterministic voting rules [22, 23, 19]. Unfortunately, randomized voting rules designed to optimize for efficiency can be highly complex and rely on intricate mathematical results such as the minimax theorem [19], making them difficult to explain to the end users.
In view of this, we explore the use of explainable randomized voting rules for improving the efficiency of automated decision-making. To ensure explainability, one possibility is to use only extremely simple randomized rules such as random dictatorship, where the most preferred option of a randomly chosen agent is selected. But this may leave significant possible efficiency gains on the table. Instead, we propose a hybrid approach that adds a simple (and thus, explainable) randomization step to well-understood deterministic decision-making processes. This drives our main research question:
What efficiency gains can be unlocked by explainable randomized voting rules which add a simple randomization step to deterministic voting rules?
As a yardstick for efficiency, we turn to the distortion framework [24]. Proposed by Procaccia and Rosenschein [25], this framework posits that votes submitted by agents, typically rankings over a set of alternatives, are induced by more expressive preferences underneath, typically cardinal utility functions over the alternatives. In this framework, the goal of a randomized voting rule is to choose a lottery over the alternatives that minimizes distortion, the worst-case ratio between maximum social welfare (total utility to the agents) and that of the chosen lottery.
We study the distortion of two families of randomized voting rules, which we refer to as randomized positional scoring rules and random committee member rules. The former builds on the widely-popular family of (deterministic) positional scoring rules, under which each agent assigns a score to each alternative based on its position in her ranking. But instead of deterministically selecting an alternative with the highest total score, each alternative is selected with probability proportional to its score. In contrast to picking alternatives with varying probabilities, the latter family utilizes the simplicity of uniform randomization by picking, uniformly at random, a member of a subset of alternatives chosen deterministically. Our inspiration for these two families stems from the use of such rules by Boutilier et al. [22] and Ebadian et al. [19].
Let us illustrate an example rule from each family based on the popular Borda count method, in which scores of  $m - 1, \ldots, 0$  are assigned to ranks  $1, \ldots, m$  respectively. A rule from the first family would pick each alternative with probability proportional to its total Borda score, while a rule from the second family may select uniformly at random among the  $k$  alternatives with the highest Borda scores, for some fixed  $k$ . We select these two families of randomized voting rules because they admit straightforward procedural explanations. For instance, the aforementioned rules based on Borda count can be explained as follows (using version (a) for the former and (b) for the latter using  $k = 3$ ).
"Each user gives zero points to their least preferred option, one point to the next best option, two points to the next best option, and so on. Points are tallied and...
Table 1: Distortion and minimum welfare of common randomized positional scoring rules.
Plurality	Borda	Harmonic	Veto	k-Approvals
k∈[1,m1/3]k∈[m1/3,√m]	k∈[√m,m]
dist	Θ(m√m)	Θ(m5/4)	Θ(√mHm)	Θ(m)	Θ(m√m/k√k)	Θ(m)	Θ(k√m-k+1)
min-sw	Θ(n/m2)	Θ(n/m√m)	Θ(n/mHm)	Θ(n/m)	Θ(nk/m2)	Θ(n/k(m-k+1))
(a) the chances of each option being selected are proportional to its total points.
(b) the three options with the highest total points are selected with an equal chance."
Unlike prior work on distortion, which is often focused on identifying the most efficient rule, we provide a refined analysis that characterizes the distortion of many interesting rules in these families, allowing the system designer to pick the one most suited to the application at hand.
Randomized positional scoring rules. We develop a whole swathe of novel techniques for analyzing distortion, and use them to obtain tight distortion (dist) bounds for randomized versions of well-known positional scoring rules such as plurality, Borda count, harmonic, veto, and  $k$ -approval, presented in Table 1. For comparison, we remark that the distortion of the best possible deterministic voting rule is  $\Theta(m^2)$  [26, 23] and the best possible randomized voting rule is  $\Theta(\sqrt{m})$  [19, 22]. For randomized positional scoring rules, the best bound is  $\Theta(\sqrt{m \log m})$  due to [22]. En route to the distortion bounds shown in Table 1, we also obtain tight bounds for another useful efficiency metric, minimum welfare (min-sw), defined in Section 3.1. In the supplementary material, we apply our novel techniques to analyze the distortion of randomized multi-level approval rules, which are uniform mixtures of different randomized  $k$ -approval rules. We demonstrate the strength of this result by using it to derive tight distortion bounds for a recently studied randomized positional scoring rule due to Gkatzelis et al. [27].
Our distortion bound for the randomized plurality rule (better known as random dictatorship) may be of independent interest because it is a widely studied voting rule [28-30]. It is also fascinating that the distortion of randomized  $k$ -approval is highly non-monotone: first decreasing from  $\Theta(m\sqrt{m})$  to  $\Theta(m)$  when  $k$  grows from 1 to  $m^{1/3}$ , then staying  $\Theta(m)$  when  $k$  grows further to  $\sqrt{m}$ , then increasing again to  $\Theta(m\sqrt{m})$  by  $k = m - \Theta(m)$ , and finally decreasing again to  $\Theta(m)$  by  $k = m$ .
Random committee member rules. For the family of random committee member rules, we design a novel voting rule, which selects a random member of a top-biased stable  $k$ -committee, and achieves a distortion of  $O(\max \{k, m^2 / (k\sqrt{k})\})$ . We complement this with a lower bound of  $\Omega(\max \{k, m^2 / k^2\})$  on the distortion of any rule in this class.
Experiments. Our experiments with synthetic data indicate that while the (worst-case) distortion of various rules in both families is not significantly better than the optimal deterministic distortion of  $\Theta(m^2)$  and often even worse than the  $\Theta(m)$  distortion of the trivial rule selecting a uniformly random alternative, rules from both families (almost always) significantly outperform deterministic voting rules and randomized positional scoring rules (almost always) outperform the uniform-random rule as well. This suggests that one should strongly consider replacing deterministic rules with explainable randomized rules from these two families in order to achieve significantly improved efficiency.
Outcome explanations in voting. We focus on (randomized) voting rules with procedural explanations because that is, in our view, the key advantage of voting over black-box AI solutions. Nonetheless, there is also compelling literature on producing outcome explanations for voting rules. Classical work that seeks voting rules satisfying qualitative axioms such as Condorcet consistency can be viewed in this light. However, these axioms often provide a justification only in limited instances with a special structure. Cailloux and Endriss [31] propose a method for producing a justification on any given instance by starting from an axiomatic justification on a nearby special instance and using a chain of explanations relating adjacent pairs of instances to arrive at the given instance. Peters et al.
[32] bound the length of such chains, focusing especially on positional scoring rules, while Boixel and Endriss [33] and Boixel et al. [34] study computational aspects of finding them.
On randomized positional scoring rules. The family of randomized positional scoring rules was first proposed by Barbera [35], under the name of point-voting schemes. He establishes several appealing properties of these rules, including strategyproofness (i.e., no agent can ever strictly benefit from misreporting her preferences). Note that the outcome of a randomized positional scoring rule can be computed by first selecting an agent uniformly at random, and then, for each  $k$ , selecting her  $k$ -th most preferred alternative with probability proportional to score assigned to position  $k$ . In this sense, implementing a randomized positional scoring rule requires little elicitation.
On distortion. We use the utilitarian distortion framework proposed by Procaccia and Rosenschein [25] and developed by Boutilier et al. [22], where agents have unit-sum utilities over alternatives. This framework has been applied to a broad range of settings such as committee selection [36], participatory budgeting [37], distributed elections [38], elicitation-efficiency tradeoff [39-43], and matching [44, 40]. Our methodological novelty is that we are the first to provide non-trivial absolute guarantees on the minimum welfare achieved across all instances, which is a reasonable efficiency measure in itself, and in turn use them to achieve tight distortion bounds (which compare the welfare achieved on each instance to the optimal welfare in that instance). We also note that there is a related framework of metric distortion [45], which assumes that agents have costs for the alternatives induced by an underlying metric space in which they are both embedded. This framework utilizes the restriction imposed by the triangle inequality instead of a unit-sum normalization. We refer the reader to the survey of Anshelevich et al. [24] for a detailed overview.
2 The Setting
Let us formally introduce our setting and define the notion of distortion. We will define each class of explainable voting rules in the section in which we will study it. We use the terminology of elections for consistency with the literature, but our setting captures a general decision-making scenario in which one of several alternatives must be chosen by aggregating conflicting preferences or opinions.
Basic notation. Let  $[t] = \{1,2,\dots ,t\}$  for  $t\in \mathbb{N}$ , and define  $\Delta (S)$  to be the probability simplex over the finite set  $S$ . For a vector  $\vec{s} = (s_1,\ldots ,s_t)$ , denote its  $\ell^1$ -norm by  $\| \vec{s}\| _1 = \sum_{i\in [t]}s_i$ .
Utilitarian voting. A (single-winner) election consists of sets  $N = [n]$  of  $n$  agents and  $A = [m]$  of  $m$  alternatives. Each agent  $i \in N$  has a personal cardinal utility function  $u_i \colon A \mapsto \mathbb{R}_{\geqslant 0}$ , where  $u_i(a)$  is the value associated by agent  $i$  to alternative  $a$ . Following the convention in the literature (e.g., see [22, 19]), we adopt the unit-sum normalization of utility functions: for every  $i \in N$ , let  $\sum_{a \in A} u_i(a) = 1$ . Aziz [46] provides several compelling justifications for using unit-sum utility functions. For a utility profile  $\vec{u} = (u_1, \dots, u_n)$  and a subset of agents  $T \subseteq N$ , define the social welfare of an alternative  $a \in A$  with respect to  $T$  as  $\mathsf{sw}_T(a, \vec{u}) = \sum_{i \in T} u_i(a)$ . We write  $\mathsf{sw}_N$  simply as  $\mathsf{sw}$ , and drop  $\vec{u}$  when it is clear from the context. As an extension, for a distribution  $p \in \Delta(A)$  over the alternatives, define  $u_i(p) = \mathbb{E}_{a \sim p}[u_i(a)]$  and its social welfare as  $\mathsf{sw}(p, \vec{u}) = \sum_{i \in N} u_i(p)$ . Our goal is to find a distribution over alternatives with high social welfare. We will sometimes construct and analyze a partial utility profile, where the utilities of each agent sum to at most 1.
Ordinal preferences and voting rules. We consider voting rules that have access only to the ordinal preferences induced by the utilities. This is because a ranking of alternatives can often be elicited with less cognitive burden or estimated more accurately than exact numerical utilities for each alternative.
Each agent  $i \in N$  submits a preference ranking  $\sigma_i \colon [m] \mapsto A$  of the alternatives. We use  $\operatorname{rank}_i(a) = \sigma_i^{-1}(a)$  to denote the rank of alternative  $a$  in agent  $i$ 's preference ranking (the most preferred alternative has rank 1), and  $a \succ_i a'$  to denote that agent  $i$  prefers  $a$  to  $a'$  (i.e.,  $\operatorname{rank}_i(a) < \operatorname{rank}_i(a')$ ). We assume that  $\sigma_i$  is consistent with agent  $i$ 's utility function  $u_i$ , i.e.,  $a \succ_i a'$  implies  $u_i(a) \geqslant u_i(a')$  for all  $a, a' \in A$ ; ties can be broken arbitrarily without affecting our distortion upper bounds.
Let  $\vec{\sigma} = (\sigma_i)_{i\in N}$  be a preference profile and  $\mathcal{C}(\vec{\sigma})$  denote the set of utility profiles  $\vec{u}$  such that  $\sigma_{i}$  is consistent with  $u_{i}$  for each agent  $i\in N$ . A voting rule  $f$  takes a preference profile  $\vec{\sigma}$  as input and returns a distribution  $p$  over alternatives.
Distortion. The distortion of a distribution  $p \in \Delta(A)$  over alternatives with respect to a utility profile  $\vec{u}$  is defined as
$$
\operatorname {d i s t} (p, \vec {u}) = \frac {\operatorname* {m a x} _ {a \in A} \operatorname {s w} (a , \vec {u})}{\operatorname {s w} (p , \vec {u})}.
$$
The distortion of a voting rule  $f$  is defined as its worst-case distortion over all instances:  $\mathrm{dist}_m(f) = \sup_{\vec{\sigma}, \vec{u} \in \mathcal{C}(\vec{\sigma})} \mathrm{dist}(f(\vec{\sigma}), \vec{u})$ , where the supremum is taken over all instances with  $m$  alternatives and any number of agents. For simplicity, we drop  $m$  and write  $\mathrm{dist}(f)$ .
3 Distortion of Randomized Positional Scoring Rules
The first class of explainable randomized voting rules we study is randomized positional scoring rules, or point-voting schemes [35]. This builds on the popular class of (deterministic) positional scoring rules, which assign scores to alternatives based on their positions in agents' preference rankings, and adds an easy-to-explain randomization step where each alternative is chosen with probability proportional to its score instead of deterministically choosing the one with the highest score.
Positional scoring rules. A scoring vector  $\vec{s} = (s_1, \ldots, s_m)$  assigns a score  $s_r$  to each position  $r \in [m]$  and satisfies  $s_1 \geqslant s_2 \geqslant \ldots \geqslant s_m \geqslant 0$ . For an alternative  $a \in A$ , let  $\mathrm{score}_i(a, \vec{s}) = s_{\mathrm{rank}_i(a)}$  be the score  $a$  obtained from agent  $i$ , and for  $N' \subseteq N$ ,  $\mathrm{score}_{N'}(a, \vec{s}) = \sum_{i \in N'} \mathrm{score}_i(a, \vec{s})$ . Note that  $\sum_{a \in A} \mathrm{score}_N(a, \vec{s}) = n \cdot \| \vec{s} \|_1$ . We drop  $\vec{s}$  when it is clear from the context. For a scoring vector  $\vec{s}$ , we can define the following rules.
- The deterministic positional scoring rule  $f_{\vec{s}}^{\mathrm{det}}$  selects the top scored alternative (breaking ties arbitrarily), i.e.,  $f_{\vec{s}}^{\mathrm{det}}(\vec{\sigma}) = \arg \max_{a \in A} \operatorname{score}_N(a, \vec{s})$ .
- The randomized positional scoring rule  $f_{\vec{s}}^{\text{rand}}$  selects every alternative  $a \in A$  with probability proportional to its score, i.e.,  $\Pr [f_{\vec{s}}^{\text{rand}}(\vec{\sigma}) = a] = \text{score}_N(a, \vec{s}) / (n \cdot \| \vec{s} \|_1)$ .
The deterministic rules introduced above include several well-known voting rules such as plurality, Borda,  $k$ -approval, veto, and harmonic defined, by the following scoring vectors, respectively:
$$
\begin{array}{l l} \vec {s} _ {\mathsf {p l u}} = (1, 0, \ldots , 0), & \vec {s} _ {\mathsf {B o r d a}} = (m - 1, m - 2, \ldots , 0), \quad \vec {s} _ {k \text {- a p p r o v a l}} = (\underbrace {1 , \ldots , 1} _ {k \text {o n e s}}, 0, \ldots , 0), \\ \vec {s} _ {\mathsf {v e t o}} = (1, 1, \ldots , 1, 0), & \vec {s} _ {\mathsf {h a r m o n i c}} = (1,   ^ {1} / _ {2},   ^ {1} / _ {3}, \ldots ,   ^ {1} / _ {m}). \end{array}
$$
We refer to the randomized versions of these rules as "randomized  $f$ ", where  $f \in \{ \text{plurality}, \text{Borda}, \text{harmonic}, \text{veto} \}$ , and extend this terminology to any positional scoring rule  $f_{\bar{s}}$ . Note that "randomized plurality" is more widely known as random dictatorship (see, e.g., [28, 29]).
3.1 High-Level Distortion Analysis and Novel Insights
Logarithmic rounding of the scores. Our first useful insight is that we can reduce the number of distinct scores by rounding any score down to the nearest power of  $1 + \alpha$ , for a constant  $\alpha > 0$ , and this only changes the distortion of the rule by a factor of at most  $1 + \alpha$ .
Lemma 1 (Rounding Down Scores). Let  $\alpha \geqslant 0$ , and  $\vec{s}, \vec{s}'$  be scoring vectors such that  $s_j' \leqslant s_j \leqslant (1 + \alpha)s_j'$  for all  $j \in [m]$ . Then, for every preference profile  $\vec{\sigma}$  and consistent utility profile  $\vec{u} \in \mathcal{C}(\vec{\sigma})$ ,
$$
\frac {1}{1 + \alpha} \cdot \operatorname {s w} (f _ {\vec {s} ^ {\prime}} ^ {\operatorname {r a n d}} (\vec {\sigma}), \vec {u}) \leqslant \operatorname {s w} (f _ {\vec {s}} ^ {\operatorname {r a n d}} (\vec {\sigma}), \vec {u}) \leqslant (1 + \alpha) \cdot \operatorname {s w} (f _ {\vec {s} ^ {\prime}} ^ {\operatorname {r a n d}} (\vec {\sigma}), \vec {u}),
$$
and consequently,  $\frac{1}{1 + \alpha} \cdot \mathrm{dist}(f_{\vec{s'}}^{\mathrm{rand}}) \leqslant \mathrm{dist}(f_{\vec{s}}^{\mathrm{rand}}) \leqslant (1 + \alpha) \cdot \mathrm{dist}(f_{\vec{s'}}^{\mathrm{rand}})$ .
By applying this transformation, scores in the range  $\left[\| \vec{s} \|_1 / (4m^2), \| \vec{s} \|_1\right]$  can be reduced to  $O(\log m)$  distinct values, which we will find helpful in the subsequent sections. In the supplementary material, we show that by ignoring the remaining scores by reducing any  $s_j \leqslant \|\vec{s}\|_1 / (4m^2)$  to 0 changes the distortion further by another factor of at most two. Hence, we can limit our focus to scoring vectors that contain  $O(\log m)$  distinct scores, resulting in only a constant factor loss in the distortion analysis.
High-level distortion analysis. After reducing the scoring vector to  $O(\log m)$  distinct scores, we partition the agents into  $O(\log m)$  groups based on where they rank the optimal alternative  $a^*$ . This is only for the analysis; the voting rule does not know the optimal alternative. More formally, let
$0 = \ell_0 < \ell_1 < \ldots < \ell_q = m$  be the indices where the score changes in the reduced scoring vector, where, for each  $r \in [q]$ , we have  $s_i = s_{\ell_r}$  for all  $i \in [\ell_{r-1} + 1, \ell_r]$ , and  $s_{\ell_1} > \ldots > s_{\ell_q}$ . Furthermore, let  $N_r$  be the set of agents who rank  $a^*$  among positions  $[\ell_{r-1} + 1, \ell_r]$ .