Designing Rules to Pick a Rule: Aggregation by Consistency

## Abstract

Abstract Given a set of items and a set of evaluators who all individually rank them, how do we aggregate these evaluations into a single societal ranking? Work in social choice and statistics has produced many aggregation methods for this problem, each with its desirable properties, but also with its limitations. Further, existing impossibility results rule out designing a single method that achieves every property of interest. Faced with this trade-off between incompatible desiderata, how do we decide which aggregation rule to use, i.e. , what is a good rule picking rule ? In this paper, we formally address this question by introducing a novel framework for rule picking rules (RPRs). We then design a data-driven RPR that identifies the best aggregation method for each specific setting, without assuming any generative model. The principle behind our RPR is to pick the rule—among a set of “acceptable” rules that satisfy basic requirements—which maximizes the consistency of the output ranking if the data collection process were repeated. We introduce several consistency-related axioms for RPRs and show that our method satisfies them, including those failed by a wide class of natural RPRs. While we prove that the algorithmic problem of maximizing consistency is computationally hard, we provide a sampling-based implementation of our RPR that is efficient in practice. We run this implementation on known statistical models and find that, when possible, our method selects the maximum likelihood estimator of the data. Finally, we show that our RPR can be used in many real-world settings—such as peer review, contests, or political elections—to gain insights about how the rule currently being used can be modified or replaced to substantially improve the consistency of the process. Taken together, our work bridges an important gap between the axiomatic and statistical approaches to rank aggregation, laying a robust theoretical and computational foundation for principled rule picking.

## 1 Introduction

Suppose you have a collection of items, and evaluators who individually rank them. Finding the best method for aggregating such data into a single ranking of items is an age-old problem. But what makes an aggregation method *good*?

One common approach to this question, known as the *axiomatic approach* in the social choice literature [Plott, 1976], is to first select certain criteria (called *axioms*) that the aggregation should satisfy, and then design rules that meet these axioms. However, celebrated impossibility results [Arrow, 1963, Gibbard, 1973, Satterthwaite, 1975] prove even some fundamental axioms cannot be simultaneously satisfied, thereby eliminating all hope for a single “ideal” rule that fulfills every reasonable desideratum. On the other hand, even if the chosen axioms are satisfiable at once, there may be many rules that do so, making the selection among them somewhat arbitrary.

Another common approach for picking the aggregation method, which we will refer to as the *statistical approach*, is to treat the rankings as noisy estimates of an objective ground truth. By assuming a noise model (*e.g.*, Plackett-Luce [Plackett, 1975, Luce, 1959], Mallows [1957]) for the data generating process, the aggregate ranking can be chosen as the one that maximizes the likelihood of the data under the assumed model. A key challenge is that the assumed model may not be accurate, which is commonly addressed with considering multiple generative models and then choosing one via cross validation [Zucchini, 2000]. However, irrespective of the model-selection process, the assumption of a generative model with a ground truth may be fundamentally flawed, especially in settings with legitimate differences of opinion, such as AI alignment [Ge et al., 2024]. Further, many natural voting rules with desirable properties are not the maximum likelihood estimator (MLE) for *any* noise model [Conitzer and Sandholm, 2005, Conitzer et al., 2009] and are thereby precluded by the statistical approach.

Given this vast array of tools from statistics and social choice, and no clear way of *a priori* selecting from them, a natural question emerges: *In a given setting, how do we pick which rule to use? In other words, what makes a good rule picking rule (RPR)*?

In this paper, we address this question. Unlike previous literature that largely focuses on picking an aggregate ranking, we want to explicitly pick a rule. Such an approach has several benefits: First, employing an RPR naturally leads to better interpretability of the process by providing a formal justification of why other rules (under which the winners could be different) were not adopted. This can mitigate malicious behavior by preventing a powerful actor (such as an incumbent running for reelection) from changing the rules of the game for their benefit. Second, different (perfectly reasonable) rules may be appropriate for different settings with different requirements, and RPRs offer a principled way of deciding which rule is the most appropriate for a given setting. Lastly, as we will see, our novel framework allows designing natural RPRs that can choose from *any* set of rules, making it easy to continually incorporate novel rules into the aggregation process without changing the rule selection method.

Our contributions are as follows:

- 1.
We introduce a novel framework for formally defining *rule picking rules (RPR)* ([Section 3](https://arxiv.org/html/2508.17177v1#S3)). Our framework allows designing principled ways of adopting a rule appropriate for the data, without committing to a set of axioms or a generative model *a priori*.
- 2.
Inspired by prior work emphasizing the link between consistency and quality in related settings, we introduce our own RPR, *Aggregation by Consistency ($AbC$)*, with the explicit goal of maximizing consistency if the data collection process were repeated ([Section 4](https://arxiv.org/html/2508.17177v1#S4)).
- 3.
We define several natural axioms for RPRs, and prove $AbC$ satisfies them, including those failed by a wide class of RPRs. For two axioms that $AbC$ fails, we prove impossibility results showing each of them are incompatible with the axioms $AbC$ satisfies ([Section 5](https://arxiv.org/html/2508.17177v1#S5)).
- 4.
We prove that the computational problem of checking if “complete” consistency be achieved for a given input (*i.e.*, picking a rule that produces the exact same output) is ${\mathsf{NP}}$-complete ([Section 6](https://arxiv.org/html/2508.17177v1#S6)).
- 5.
Nevertheless, we provide an implementation of $AbC$ that is efficient in practice and performs well in experiments on known distributions ([Section 7](https://arxiv.org/html/2508.17177v1#S7)). While our implementation makes no assumptions about the data generation process, we demonstrate experimentally that when a ground truth in fact exists, $AbC$ picks the rule which generates the ground truth ranking with maximum likelihood.
- 6.
We show that our approach can be applied to both score- and rank-based evaluations across a large variety of empirical settings, at times improving significantly upon the consistency given by rules used in practice. As such, $AbC$ can be used to guide decisions in these settings about which rule to adopt.

An implementation of our rule was awarded as one of the four winners in the *2nd Computational Social Choice Competition* at the 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024). In addition to checking axiomatic properties, the competition scored rules based on a specific welfare function that mapped the rules’ outputs to utilities for voters, which was announced prior to the competition. Remarkably, even though we did not perform any tuning or optimization for this welfare function, our rule achieved 99.71% of the total utility attained by a competing method explicitly designed to maximize this specific function. This result underscores the robustness and strong general performance of $AbC$ across diverse evaluation criteria.

Taken together, our work lays a robust theoretical and computational foundation for principled rule picking, paving the way for future work in this novel model. Our code and experiments are available [here](https://github.com/beng341/AgreementAggregation).

## 2 Overview of the Proposed Approach

Before introducing our formal framework we discuss the basic idea of our approach. Suppose we have two sets of voters who independently rank the same items. A popular measure of the “quality” of an aggregation method is the *consistency* between its output on two independent sets of rankings. As such, we seek to design an RPR that picks the rule that maximizes this consistency.(^2^22We will restrict our RPR to pick from rules that satisfy some basic axiomatic properties such as neutrality, which dictates that all items being ranked are treated equally, *i.e.*, permuting the items in evaluators’ rankings should result in the output ranking being accordingly permuted. This avoids the pathological case of a constant function with maximal consistency.) This intuition is inspired by a number of related settings where past work has emphasized the importance of consistent outputs:

##### (1) Peer review

A classical setting in which evaluator rankings need to be aggregated is peer review, and much work has been done to optimize this process; see Shah [2022] for an overview. Many experiments split reviewers into two panels evaluating the same data (Jecmen et al. [2022] show the merits of this methodology), where interpanel disagreement is interpreted as a shortcoming [Obrecht et al., 2007, Fogelholm et al., 2012, Pier et al., 2017, Bast, 2020]. For example, in the NeurIPS 2014 and 2021 conferences, the panels disagreed on more than half of the accepted papers, which is taken as a sign of arbitrariness in the review process [Lawrence and Cortes, 2014, Cortes and Lawrence, 2021, Beygelzimer et al., 2023]. Similar experiments use interpanel consistency to compare distributed peer review with an expert panel [Patat et al., 2019, Kerzendorf et al., 2020]. Overall, it is clear that the peer review community already cares about consistency as an indicator for quality.

##### (2) Clustering

One could view the task of rule picking as *learning* a rule from ranking data. Unlike earlier work on learning voting rules that assumes black-box access to a ground-truth rule [Procaccia et al., 2009], our setting is unsupervised and hence is more closely related to clustering problems. Indeed, Ailon et al. [2005] show that clustering and rank aggregation are closely linked and can be approached with near-identical algorithms. Similar to the problem of rule picking, an important challenge in clustering is that of model selection (*e.g.*, the number of clusters). Previous work has shown that picking the model in a way that maximizes stability—*i.e.*, would obtain similar results on several data sets from the same underlying source—yields results with higher accuracy; see von Luxburg [2010] for an overview of theoretical results in clustering stability. Again, we see that the consistency and the quality of the output are closely related in this setting.

##### (3) Minimum-variance unbiased estimator (MVUE)

A statistical estimator is *unbiased* if its expected value equals the true value of the parameter being estimated. Among all unbiased estimators, the estimator with the smallest mean squared error is the one with the lowest variance. Indeed, MVUEs are very commonly studied in statistics [Rao, 1949, Chapman and Robbins, 1951] and related bounds for unbiased estimators are frequently used for rank aggregation [Hajek et al., 2014, Khetan and Oh, 2016]. Consider applying an unbiased estimator to two i.i.d. data samples. Since the variance of a random variable is exactly half of the expected squared difference between its two i.i.d. copies, the estimator that leads to the smallest expected error between the two data sets is once again the one with the minimum variance, *i.e.*, the MVUE. This motivates the maximum consistency aggregation in our setting, where unbiasedness is interpreted as basic constraints for any acceptable rule, which we achieve by restricting our RPR to pick from neutral and anonymous rules (*i.e.*, those that treat all items and evaluators the same, respectively).

##### (4) AI alignment

There is nascent interest in applying tools from social choice to AI alignment processes, such as reinforcement learning from human feedback (RLHF) [Conitzer et al., 2024, Dai and Fleisig, 2024, Mishra, 2023], with particular emphasis on consistency. Much like peer review, RLHF inevitably relies on input from a limited set of evaluators, despite aiming for broad societal alignment. As such, aggregation methods that are robust to repetitions of the process can decrease the arbitrariness of the final AI model due to the choice of evaluators. As pointed out by Conitzer et al. [2024], the focus of social choice on producing consistent aggregations helps make it an appropriate tool for this setting.

While these four settings (peer review, clustering, MVUEs, and RLHF) motivate an RPR that maximizes the consistency between two independent evaluations, it is not *ex ante* clear how one would implement that. After all, in practice, we often have only one copy of the process. To explain how we circumvent this, [Algorithm 1](https://arxiv.org/html/2508.17177v1#algorithm1) introduces our RPR, named *Aggregation by Consistency (AbC)*. Note that this is an informal description, and a formal definition is provided subsequently in [Section 4](https://arxiv.org/html/2508.17177v1#S4) ([Section 4](https://arxiv.org/html/2508.17177v1#S4)).

Figure: Algorithm 1 Aggregation by Consistency ($AbC$)

Importantly, [Algorithm 1](https://arxiv.org/html/2508.17177v1#algorithm1) is agnostic to the types of input and output of the rules it is picking among; it only requires that we define a measure of disagreement for comparing the outputs of the rules (Step 2). Our approach is therefore broadly applicable to a wide variety of settings with specific evaluation formats—*e.g.* rankings, ratings/scores, approval sets—and specific desired outputs—*e.g.*, an aggregate ranking, a single winner, a committee of winners. In this paper, we focus on rules that output an aggregate ranking.

As we will show, $AbC$ has a number of advantages. First, it satisfies several important axioms for RPRs, even those that are failed by a wide class of natural RPRs. Second, $AbC$ does not assume that the data is coming from a ground truth; however, if a generative model is indeed a reasonable approximation of the data, then, as we show in our experiments, the associated MLE yields a high consistency across random splits. $AbC$ then chooses this estimator, thus obtaining the benefits of the statistical approach. Third, several important social choice axioms can be imposed on $AbC$ simply by restricting the set of candidate rules to those that satisfy them, thus obtaining the benefits of the axiomatic approach. Lastly, any implementation of our method is easy to continually extend by simply implementing new rules and adding them to our set of candidate rules.

In order to formally define $AbC$, we next introduce our novel framework for rule picking rules.

## 3 Preliminaries and Rule Picking Rules

We consider a set of voters $N=\{1,2,\ldots,{|N|}\}$ each of whom individually evaluate a finite set of alternatives $A$. For our axiomatic analysis ([Section 5](https://arxiv.org/html/2508.17177v1#S5)), it will be helpful to assume each voter provides a ranking of *all* alternatives, although our implementation easily extends to other evaluations formats, as demonstrated by our experiments where voters provide partial rankings ([Sections 7.1](https://arxiv.org/html/2508.17177v1#S7.SS1) and [7.3](https://arxiv.org/html/2508.17177v1#S7.SS3)) or ratings/scores ([Section 7.2](https://arxiv.org/html/2508.17177v1#S7.SS2)).

A *strict ranking* of alternatives is a total ranking of the elements in $A$, whereas a *weak ranking* allows for ties. More formally, a weak ranking of $A$ is a complete and transitive binary relationship on $A$; a strict ranking is a weak ranking that is also asymmetric. We denote the set of all strict rankings of $A$ by $\mathcal{L}(A)$, and the set of all weak rankings of $A$ by $\mathcal{R}(A)$. For any (strict or weak) ranking $r$, we write $a\succ_{r}b$ if $a$ is strictly ranked above $b$ in $r$, and $b\succeq_{r}a$ otherwise (*i.e.*, if $b$ is ranked weakly above $a$). We assume each voter $i\in N$ has a (strict) ranking $\sigma_{i}\in\mathcal{L}(A)$ over the alternatives $A$. A (preference) profile $\bm{\sigma}\in\mathcal{L}(A)^{|N|}$ comprises the rankings of all voters.

##### Candidate rules

A *social welfare function (SWF)* is a mapping $f$ that, given a profile $\bm{\sigma}$, outputs a single weak ranking.(^3^33The reader might wonder why we focus on functions that get strict rankings as inputs but output a weak ranking. Restricting the output to strict rankings would require sometimes outputting multiple rankings in order to satisfy neutrality and anonymity (*i.e.*, treating all alternatives and voters the same, respectively), *e.g.*, in cyclic profiles. Using SWFs that return weak rankings (as in Arrow [1963, §III], Brandt et al. [2016, §1.2.2]) allows us to return a single ranking without violating these basic axioms.) To aggregate the rankings of the voters, we are interested in picking an SWF to use from a set of acceptable “candidate” rules. Importantly, our framework does not place any restrictions on the set of candidate rules. Indeed, our evaluations in [Section 7](https://arxiv.org/html/2508.17177v1#S7) will comprise various SWFs including MLEs of known noise models (*e.g.*, the Plackett-Luce [Plackett, 1975, Luce, 1959] and Mallows [1957] models) as well as various SWFs from social choice literature. One such class of SWFs we will sometimes pay attention to is the set of all (monotonic) positional scoring rules.

###### Definition 1 (Positional scoring rules) .

A (monotonic) *positional scoring rule* is an SWF $f_{s}$ associated to a specified vector ${s}=(s_{j})_{j\in\{1,2,\ldots,{|A|}\}}$ with $1=s_{1}\geq s_{2}\geq\ldots\geq s_{|A|}=0$. Given a profile $\bm{\sigma}$ as input, the output of $f_{s}$ is computed as follows: for each alternative $a\in A$ and index $j\in\{1,2,\ldots,{|A|}\}$, let $M_{\bm{\sigma}}[a,j]$ denote the number of voters in $\bm{\sigma}$ that rank alternative $a$ in the $j^{\text{th}}$ position. Further, define the total score of alternative $a$ in $\bm{\sigma}$ (with respect to vector $s$) as $t^{{s}}_{\bm{\sigma}}[a]=\sum_{j=1}^{|A|}s_{j}M_{\bm{\sigma}}[a,j]$. Then, on input profile $\bm{\sigma}$, the SWF $f_{{s}}$ ranks the alternatives in decreasing total score, *i.e.*, $a\succ_{f_{\bm{s}}(\bm{\sigma})}b$ if and only if $t^{{s}}_{\bm{\sigma}}[a]>t^{{s}}_{\bm{\sigma}}[b]$. We denote the set of all (monotonic) positional scoring rules by $F_{S}$.

In addition to being succinctly representable and easy to compute, positional scoring rules encapsulate many well-known SWFs, including *plurality* ($f_{p}$, for $p=(1,0,\ldots,0)$), *veto* ($f_{v}$, for $v=(1,\ldots,1,0)$), and *Borda count* ($f_{b}$, for $b_{j}=\frac{{|A|}-j}{{|A|}-1}$).

###### Definition 1 (Positional scoring rules) .

##### Rule Picking Rules

We now introduce a novel framework for picking rules. Given a set of candidate rules (*e.g.*, those that are the MLE of a noise model, or voting rules that satisfy certain axioms), we will define a function that maps a profile to the candidate rule(s) most appropriate for it.

###### Definition 2 (Rule picking rules) .

A *rule picking rule (RPR)* is a function $Z$ that given a set of SWFs $F$ (called *candidate rules*)
and a profile $\bm{\sigma}$, outputs a subset of the candidate rules $Z(F,\bm{\sigma})\subseteq F$.

An RPR offers a principled way to pick an SWF that should be used to aggregate the rankings in $\bm{\sigma}$. Ideally, an RPR will capture the idea that different (perfectly reasonable) SWFs may be appropriate for different profiles, *e.g.*, because the profile resembles a specific distribution, or certain positions in the rankings give more information than others. In [Section 5](https://arxiv.org/html/2508.17177v1#S5), we formalize these situations and define some natural axioms for RPRs.

To avoid ambiguity, ideally we would like to have $|Z(F,\bm{\sigma})|=1$. However, just as many natural SWFs (*e.g.*, positional scoring rules) can inevitably lead to ties among alternatives, as we will see, natural RPRs may lead to ties among SWFs for certain profiles, leading to $|Z(F,\bm{\sigma})|>1$. In such cases, a tie-breaking order over $F$ can be used to pick the single rule to be adopted.

###### Definition 2 (Rule picking rules) .

## 4 Aggregation by Consistency ( AbC )

Having introduced our framework of RPRs, we now formally present our proposed method. As discussed in [Section 2](https://arxiv.org/html/2508.17177v1#S2), we want our RPR to maximize the consistency between two independent sets of evaluations. Our approach is general can be implemented with any measure of distance comparing the outputs of a rule on the two groups of voters (to be used in Step 2 of [Algorithm 1](https://arxiv.org/html/2508.17177v1#algorithm1)). For concreteness in our analysis and experiments, we turn to one such well-known measure between rankings.

###### Definition 3 (Kendall-Tau distance with ties) .

Given a pair of weak rankings $r_{1},r_{2}\in\mathcal{R}(A)$ and alternatives $a,b\in A$, let $D_{r_{1},r_{2}}^{a,b}$ be indicator variable that the rankings $r_{1}$ and $r_{2}$ strictly disagree about how alternatives $a$ and $b$ should be ordered. Similarly, let $T^{a,b}_{r_{1},r_{2}}$ be the indicator variable that alternatives $a$ and $b$ are tied in at least one of $r_{1}$ or $r_{2}$. More formally,

$$ $\displaystyle D_{r_{1},r_{2}}^{a,b}$ $\displaystyle=\mathbb{I}[(a\succ_{r_{1}}b\text{ and }b\succ_{r_{2}}a)\text{ or }(b\succ_{r_{1}}a\text{ and }a\succ_{r_{2}}b)],\quad\text{and}$ $\displaystyle T_{r_{1},r_{2}}^{a,b}$ $\displaystyle=\mathbb{I}[(a\succeq_{r_{1}}b\text{ and }b\succeq_{r_{1}}a)\text{ or }(a\succeq_{r_{2}}b\text{ and }b\succeq_{r_{2}}a)].$ $$

Then, the *Kendall-Tau distance (with ties)* between $r_{1}$ and $r_{2}$ is

$$ $\displaystyle KT(r_{1},r_{2})=\sum_{\{a,b\}\in A^{2}:a\neq b}\left(D_{r_{1},r_{2}}^{a,b}+\frac{1}{2}T^{a,b}_{r_{1},r_{2}}\right).$ (1) $$

It is worth noting that $KT$ is more traditionally defined for strict rankings, without the $T_{r_{1},r_{2}}^{a,b}$ term. We explicitly add the term for ties in order to penalize indecisiveness; otherwise, a rule that always returns the empty ranking (all alternatives tied) would achieve zero disagreement (and hence maximum consistency), without violating neutrality or anonymity. Weighing ties by $\frac{1}{2}$ is inspired by Kendall’s Tau-b correlation coefficient [Kendall, 1945]. We now introduce our consistency-based RPR, which we formally define in [Section 4](https://arxiv.org/html/2508.17177v1#S4).

In words, $AbC$ returns the SWFs among $F$ that minimize, in expectation,(^4^44 There is a $2^{-(n-1)}$ probability that $N_{j}=\emptyset$ for some $j\in\{1,2\}$. In such a case, we take $f(\bm{\sigma}^{(j)})$ to be the empty ranking (all alternatives tied) for all $f\in F$. This effectively gives zero weight to all such splits when comparing expected disagreements.) the disagreement when applied separately to two sides of a random split of $\bm{\sigma}$.
We now illustrate which rule $AbC$ would pick on an example profile by showing how ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) can be computed or bounded for each candidate rule.

###### Example 4 .

Fix some integer $k\geq 2$ and consider a profile $\bm{\sigma}$ with alternatives $A=\{a,b,c\}$ and ${|N|}=3k$ voters. Suppose the voters comprise three groups, consisting of:

- •
Group 1: $k$ voters that rank $a\succ b\succ c$.
- •
Group 2: $k$ voters that rank $a\succ c\succ b$.
- •
Group 3: $k$ voters that rank $b\succ c\succ a$.

Consider the set of candidate rules $F=\{f_{p},f_{v}\}$ (*i.e.*, plurality and veto). As defined in [Section 3](https://arxiv.org/html/2508.17177v1#S3), plurality $(f_{p})$ simply ranks alternatives in decreasing order of the number of voters that rank them top. Veto $(f_{v})$, on the other hand, ranks alternatives in increasing order of the number of voters that rank them bottom. Let us now evaluate the two candidate rules, plurality and veto, under our Aggregation by Consistency ($AbC$) RPR. Say $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$ are the random variables corresponding to the two subprofiles resulting from splitting $\bm{\sigma}$ via the random process in [Section 4](https://arxiv.org/html/2508.17177v1#S4). First, consider plurality $f_{p}$. For notational convenience, let $r_{p}^{1}=f_{p}(\bm{\sigma}^{(1)})$ and $r_{p}^{2}=f_{p}(\bm{\sigma}^{(2)})$ denote the random variables corresponding to the two output rankings. Since alternative $c$ does not appear on top of any voter’s ranking, both $r_{p}^{1}$ and $r_{p}^{2}$ will rank $a\succ c$ unless all of the $2k$ voters ranking $a$ as their top alternative end up on the same side of split, in which case the ranking of the other side will have $a$ and $c$ tied. Similarly, both $r_{p}^{1}$ and $r_{p}^{2}$ will rank $b\succ c$ unless all of the $k$ voters ranking $b$ top end up on the same side of split. Thus,

$$ $\displaystyle\mathbb{E}\left[D_{r_{p}^{1},r_{p}^{2}}^{a,c}\right]=\mathbb{E}\left[D_{r_{p}^{1},r_{p}^{2}}^{b,c}\right]=0;\quad\mathbb{E}\left[T_{r_{p}^{1},r_{p}^{2}}^{a,c}\right]\leq\frac{1}{2^{2k-1}};\quad\mathbb{E}\left[T_{r_{p}^{1},r_{p}^{2}}^{b,c}\right]\leq\frac{1}{2^{k-1}}.$ $$

Using the linearity of expectation, this implies the expectation in ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) for $f_{p}$ is

$$ $\displaystyle\mathbb{E}\left[KT\left(r_{p}^{1},r_{p}^{2}\right)\right]$ $\displaystyle=\mathbb{E}\left[D_{r^{1}_{p},r_{p}^{2}}^{a,b}+\frac{1}{2}T^{a,b}_{r^{1}_{p},r_{p}^{2}}\right]+\mathbb{E}\left[D_{r^{1}_{p},r_{p}^{2}}^{a,c}+\frac{1}{2}T^{a,c}_{r^{1}_{p},r_{p}^{2}}\right]+\mathbb{E}\left[D_{r^{1}_{p},r_{p}^{2}}^{b,c}+\frac{1}{2}T^{b,c}_{r^{1}_{p},r_{p}^{2}}\right]$ $\displaystyle\leq 1+\left(0+\frac{1}{2}\cdot\frac{1}{2^{2k-1}}\right)+\left(0+\frac{1}{2}\cdot\frac{1}{2^{k-1}}\right)\leq 1+\frac{1}{2^{4}}+\frac{1}{2^{2}}=1.3125.$ (3) $$

Now, consider veto $f_{v}$. Say $r_{v}^{1}=f_{v}(\bm{\sigma}^{(1)})$ and $r_{v}^{2}=f_{v}(\bm{\sigma}^{(2)})$. Since the number of voters that rank $a$ and $b$ as their bottom alternative in $\bm{\sigma}$ are tied, $r_{v}^{1}$ and $r_{v}^{2}$ will either disagree about how $a$ and $b$ should be ranked, or both of them will tie $a$ and $b$. In other words, we cannot have $D_{r^{1}_{v},r_{v}^{2}}^{a,b}=T_{r^{1}_{v},r_{v}^{2}}^{a,b}=0$. The same is true for any other pair of alternatives. This implies the expectation in ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) for $f_{v}$ is

$$ $\displaystyle\mathbb{E}\left[KT\left(r_{v}^{1},r_{v}^{2}\right)\right]$ $\displaystyle=\mathbb{E}\left[D_{r^{1}_{v},r_{v}^{2}}^{a,b}+\frac{1}{2}T^{a,b}_{r^{1}_{v},r_{v}^{2}}\right]+\mathbb{E}\left[D_{r^{1}_{v},r_{v}^{2}}^{a,c}+\frac{1}{2}T^{a,c}_{r^{1}_{v},r_{v}^{2}}\right]+\mathbb{E}\left[D_{r^{1}_{v},r_{v}^{2}}^{b,c}+\frac{1}{2}T^{b,c}_{r^{1}_{v},r_{v}^{2}}\right]\geq 1.5.$ (4) $$

Comparing ([3](https://arxiv.org/html/2508.17177v1#S4.E3)) and ([4](https://arxiv.org/html/2508.17177v1#S4.E4)), we see that $AbC(F,\bm{\sigma})=\{f_{p}\}$. Choosing plurality over veto indeed makes sense for aggregating the rankings in $\bm{\sigma}$, as the plurality (first place) scores of each alternative in $\bm{\sigma}$ clearly give more information about how they compare to each other than their veto (last place) scores.

We now discuss three extensions of $AbC$, which we later implement ([Section 7](https://arxiv.org/html/2508.17177v1#S7)).

###### Definition 3 (Kendall-Tau distance with ties) .

###### Example 4 .

##### Approximating expected disagreement

While computing the expected disagreement in ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) for each SWF can be difficult for more complicated profiles and candidate rules than in [4](https://arxiv.org/html/2508.17177v1#Thmprop4), in practice $AbC$ can be approximated for finite number of candidate rules $|F|$ via Monte Carlo sampling, *i.e.*, by splitting voters via the random process in [Section 4](https://arxiv.org/html/2508.17177v1#S4), computing the disagreement $KT(f(\bm{\sigma}^{(1)}),f(\bm{\sigma}^{(2)}))$ for each SWF $f\in F$, and repeating the process for a desired number of splits, eventually returning the SWF with the minimum average disagreement.

##### Infinitely many candidate rules

The sampling approach above works even for certain candidate rule sets $F$ with infinite $|F|$. In such cases, optimization algorithms can be used to find the minimizer of ([2](https://arxiv.org/html/2508.17177v1#S4.E2)). For example, if the set of candidate rules is the set of all positional scoring rules ([Definition 1](https://arxiv.org/html/2508.17177v1#Thmprop1)), *i.e.*, $F=F_{S}$, one can run a constrained optimization program with the scoring vector $(s_{j})_{j\in\{1,2,\ldots,{|A|}\}}$ as the variables. Indeed, our implementation of $AbC$ for the experiments in [Section 7](https://arxiv.org/html/2508.17177v1#S7) relies on Monte Carlo sampling and includes (among other rules) an optimization over all (infinitely many) positional scoring rules.

##### Partial rankings

A more general setting than the one we have considered so far is that of *partial rankings*, where each voter $i\in N$ ranks a subset of alternatives $A_{i}\subseteq A$. This setup is more appropriate for certain settings in which we would like to utilize $AbC$, such as peer review. To extend $AbC$ to partial rankings, we must take into account that each alternative is no longer ranked by every voter. As a result, some alternatives will naturally have more evaluations in a single side of a random split, while others’ evaluations will be more evenly split. Ideally, disagreements over alternatives that are well-represented on both sides should be penalized more harshly than those concerning alternatives that appear primarily on one side, where the other side may lack sufficient information. Indeed, in the extreme case where all evaluations of an alternative end up on the same side of a random split, it seems unreasonable to expect any SWF to place this alternative consistently across the split. This can be achieved by replacing the $KT$ function in ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) with the *weighted $KT$ function* [Kumar and Vassilvitskii, 2010], where the weight of each alternative is set according to how evenly they are represented across a split. Indeed, our implementation that was awarded in the IJCAI-24 competition mentioned in [Section 2](https://arxiv.org/html/2508.17177v1#S2) used this method, as well as some of our experiments in [Section 7](https://arxiv.org/html/2508.17177v1#S7).

##### Score data

Some elicitation systems may require voters to submit scores for the alternatives rather than ranking them. In such settings, one option is to convert each voter’s scores into a ranking, and then apply $AbC$ to pick a rank aggregation rule. Alternatively, we can also extend $AbC$ to directly pick among aggregation functions that take scores as input rather than rankings (*e.g.*, maximum, geometric/arithmetic mean). To do so, we split the scores received by each alternative via a process analogous to that in [Section 4](https://arxiv.org/html/2508.17177v1#S4), once again choosing the aggregation method that outputs most consistent results across the split. Indeed, one of our experiments in [Section 7](https://arxiv.org/html/2508.17177v1#S7) implement this extension.

## 5 Axiomatic Analysis of AbC

Having introduced our rule picking rule $AbC$, we would like to investigate its axiomatic properties. To do so, we now introduce some natural axioms for RPRs, focusing on the setting of full rankings introduced in [Section 3](https://arxiv.org/html/2508.17177v1#S3). While some of these axioms are inspired by their counterparts for SWFs, not all SWF axioms can be easily translated to our RPR framework. After all, while an SWF maps preferences over alternatives to an aggregate ranking of alternatives (*i.e.*, its inputs and output are rankings of the same set), an RPR must pick SWFs from candidate rules based on preferences over alternatives, not over candidate rules.

### 5.1 Consistency axioms

Our primary goal in employing RPRs is to effectively identify voting rules that are well-suited to a given profile. It thus becomes highly desirable for these RPRs to respond in a predictable and consistent manner when the profile changes. To rigorously analyze this desired consistency, we introduce axioms that formalize the expected behaviors of RPRs in response to changes in the profile. This approach adapts established social choice theory axioms, originally designed for SWFs, for application to RPRs, as well as introducing new axioms specific to our setting.

#### Reversal symmetry

One basic change to a profile is to flip the ranking of every voter, so their formerly top ranked alternative is now ranked bottom, and so on. In such a situation, we would ideally like the SWF(s) picked by our RPR to flip too, due the symmetry between the two directions of reading a ranking (top to bottom versus bottom to top). More formally, given a positional scoring rule $f_{s}\in F_{S}$, we define the reverse of $f_{s}$, denoted $\textnormal{rev}(f_{s})$, as the scoring rule $f_{s^{\prime}}$ associated with vector $s^{\prime}=(s^{\prime}_{1},s^{\prime}_{2},\ldots,s^{\prime}_{{|A|}})=(1-s_{|A|},1-s_{{|A|}-1},\ldots,1-s_{1})$. For instance, the reverse of plurality is veto, whereas Borda count is its own reverse. For a set of rules $F\subseteq F_{S}$, we write $\textnormal{rev}(F)=\{\textnormal{rev}(f):f\in F\}$. For any (strict or weak) ranking $r\in\mathcal{L}(A)\cup\mathcal{R}(A)$, we say $\textnormal{rev}(r)$ is the reversed ranking (*e.g.*, $a\succ b\succ c$ becomes $c\succ b\succ a$). Lastly, we use $\textnormal{rev}(\bm{\sigma})=\{\textnormal{rev}(\sigma_{i})\}_{i\in N}$ to denote the reverse of a profile $\bm{\sigma}$. With this notation in place, we now define reversal symmetry,(^5^55Our definition for reversal symmetry is inspired by the homonymous property for rank aggregation methods introduced by Saari [1994]. In that context, reversal symmetry dictates that if we reverse every voter’s ranking, then the output of the method should also be the reverse of its original output. Holliday and Pacuit [2023] adapt this axiom to social choice functions (which, given a profile, return a subset of alternatives); here, reversal symmetry requires that if an alternative is the unique winner of a profile, it should not be a winner of the reversed profile. Our axiom for RPRs is more demanding in the sense that we not only require the picked positional scoring rules to change, but also that they should be replaced exactly by their reverse rules.) our first axiom for RPRs.

###### Definition 5 (Reversal symmetry) .

An RPR $Z$ satisfies *reversal symmetry* if for any subset of positional scoring rules $F\subseteq F_{S}$ such that $\textnormal{rev}(F)=F$ and for all profiles $\bm{\sigma}$, we have $\textnormal{rev}(Z(F,\bm{\sigma}))=Z(F,\textnormal{rev}(\bm{\sigma}))$.

To see why this is a natural property, suppose for example the “signal” in the votes is concentrated at the top: for each voter, the choice of the top alternative is statistically informative but the remainder of the ranking is not. In such a case, a reasonable RPR should choose plurality. But if we flip all the votes, then the signal is concentrated at the bottom, and a reasonable RPR should choose veto. As shown in [Theorem 1](https://arxiv.org/html/2508.17177v1#Thmthm1) below, $AbC$ satisfies this axiom, which follows from the symmetries of the Kendall-Tau distance function.

###### Definition 5 (Reversal symmetry) .

#### Shuffling consistency

As with the example above, in some profiles certain parts of the rankings may be more informative than others. To axiomatize the behavior of RPRs in such situations, we will now introduce the novel concept of *shuffling* a profile. Intuitively, we would like to capture the idea that if we uniformly “shuffle” an interval of positions in all voters’ rankings, exactly where in this interval an alternative ends up reveals no information about how it compares to other alternatives in the interval. Thus, for such a “shuffled profile,” we would want our RPR to pick rules that treat these positions equivalently. For example, if in every voter’s ranking we uniformly at random permute all of the alternatives they ranked 2${}^{\text{nd}}$ to $|A|^{\text{th}}$, whether an alternative is ranked 2${}^{\text{nd}}$ or 3${}^{\text{rd}}$ by any voter no longer makes a difference. Given such a profile, an RPR that, for instance, picks from positional scoring rules should ideally pick one that gives positions 2 and 3 roughly equal weight.

We now formally define shuffling. Intuitively, to shuffle a given subset of $t\leq{|A|}$ indices in every voter’s ranking, we would like to create a new profile where all $t!$ permutations of these indices are equally represented. To do, we will eventually split the profile uniformly between these permutations. Since the number of voters may not be divisible by $t!$, we will first create a multiple of ${|A|}!$ copies of the profile, which ensures that we can assign an equal number of copies for each of the $t!$ permutations.

###### Definition 6 (Shuffled profile) .

Given a profile $\bm{\sigma}$ and a subset of indices $S\subseteq\{1,2,\ldots,{|A|}\}$, the *$k$-shuffling of $\bm{\sigma}$ with respect to $S$* (denoted $\pi^{k}(\bm{\sigma},S)$) is a profile obtained as follows: For each voter $i\in N$,

- •
Create $k\cdot{|A|}!$ copies of voter $i$’s ranking $\sigma_{i}$ and separate them into $|S|!$ groups of equal size, each assigned to a unique permutation of the indices in $S$.
- •
Modify the votes in each group so that the alternatives ranked in the positions in $S$ are permuted according to the permutation assigned to that group.
- •
Add all copies to the final profile.

For example, say $\bm{\sigma}$ consists of a single ranking $a\succ b\succ c$, and we are interested in its 1-shuffling with respect to indices $\{1,2\}$. By the first step of [Definition 6](https://arxiv.org/html/2508.17177v1#Thmprop6), we first create $k\cdot{|A|}!=1\cdot 3!=6$ copies of this ranking. We then split them into $|S|!=2!=2$ groups of size 3 each. The first group is assigned to the identity permutation of the first two positions. The second group is assigned to the permutation that reverses the first two positions. Hence, $\pi^{1}(\bm{\sigma},\{1,2\})$ consists of six rankings, with three of them being $a\succ b\succ c$ and three being $b\succ a\succ c$.

Our next axiom dictates an RPR’s behavior in the extreme case of shuffling all but one position.

###### Definition 7 (Plurality-shuffling consistency) .

Say we are given a finite set of positional scoring rules $F\subset F_{S}$ that contains plurality (*i.e.*, $f_{p}\in F$) and a profile $\bm{\sigma}$ such that $f_{p}(\bm{\sigma})$ contains no ties. Then, an RPR $Z$ satisfies *plurality-shuffling consistency* if for any such $F$ and $\bm{\sigma}$, there is a $k\in\mathbb{Z}_{+}$ so that $Z(F,\pi^{k^{\prime}}(\bm{\sigma},[2,{|A|}]))=\{f_{p}\}$ for all $k^{\prime}\geq k$, *i.e.*, $Z$ picks *only* $f_{p}$ for the $k^{\prime}$-shuffling with respect to $[2,{|A|}]=\{2,3,\ldots,{|A|}\}$.

Intuitively, for a (sufficiently large) profile where the top position of the votes gives an unambiguous ranking (under plurality), but the remaining $m-1$ positions are effectively indistinguishable, an RPR satisfying plurality-shuffling consistency identifies the rule that treats these $m-1$ positions equally (*i.e.*, plurality) as the only appropriate rule. Such a profile, for example, can approximate settings where the voters clearly know their top alternative, but are indifferent for the remaining spots. Plurality-shuffling consistency is a natural property that respects the symmetry in the profile and identifies where the “signal” in the votes is concentrated. Despite this, a large class of RPRs, which we introduce next, all fail plurality-shuffling consistency, as we will show in [Theorem 1](https://arxiv.org/html/2508.17177v1#Thmthm1).

###### Definition 8 (Welfare-maximizing RPRs) .

An RPR $Z$ is *welfare-maximizing* if there exists a utility function $u:\mathcal{L}(A)\times\mathcal{R}(A)\rightarrow\mathbb{R}$ such that $Z(F,\bm{\sigma})=\operatorname*{argmax}_{f\in F}\sum_{i=1}^{n}u(\sigma_{i},f(\bm{\sigma}))$ for all profiles $\bm{\sigma}$ and candidate rule sets $F$.

Welfare-maximizing RPRs capture an approach common in prior work: interpreting the optimal voting rule as one that maximizes social welfare with respect to some utility function [Caragiannis and Procaccia, 2011, Gershkov et al., 2017]. Indeed, this is how voting rules in the IJCAI-24 competition mentioned in [Section 2](https://arxiv.org/html/2508.17177v1#S2) were scored. As [Definition 8](https://arxiv.org/html/2508.17177v1#Thmprop8) puts no restrictions on the utility function $u$, welfare-maximizing RPRs constitute a wide class. As such, whether a specific member of this class satisfies an axiom may depend on its associated utility function. For example, it is relatively straightforward to show that a welfare-maximizing RPR satisfies reversal symmetry if its utility function is reversal symmetric, *i.e.*, $u(r,r^{\prime})=u(\textnormal{rev}(r),\textnormal{rev}(r^{\prime}))$ for all $r\in\mathcal{L}(A)$ and $r^{\prime}\in\mathcal{R}(A)$. Nevertheless, we show in [Theorem 1](https://arxiv.org/html/2508.17177v1#Thmthm1) that *all* welfare-maximizing RPRs fail plurality-shuffling consistency.

On the other hand, this is not the case for $AbC$, which indeed satisfies the axiom. To see this, take a random split of the shuffled profile $\pi^{k}(\bm{\sigma},[2,{|A|}])$. Given the balanced nature of all positions except the first, any rule that treats a distinction between alternatives in these positions as a signal will inevitably observe the reversed signal on the other side of the split, leading to a higher disagreement. We formalize this in [Theorem 1](https://arxiv.org/html/2508.17177v1#Thmthm1).

###### Definition 6 (Shuffled profile) .

###### Definition 7 (Plurality-shuffling consistency) .

###### Definition 8 (Welfare-maximizing RPRs) .

#### Union consistency

Next, we study the behavior of RPRs when we combine sets of voters. The next axiom we introduce, *union consistency*, is inspired by an analogous axiom for voting rules, simply named consistency [Young, 1975]. Informally, it dictates that whenever an alternative is the winner of a voting rule when applied separately to the rankings of two distinct sets of voters, the same alternative should still win when we bring those sets of voters together. We now define our version of the axiom, specifically for RPRs.

For any two profiles $\bm{\sigma}_{a}$ and $\bm{\sigma}_{b}$ over the same alternatives $A$ but with two disjoint sets of voters $N_{a}$ and $N_{b}$, respectively, let $\bm{\sigma}_{a}+\bm{\sigma}_{b}$ denote the profile with the rankings of all voters in $N_{a}\sqcup N_{b}$.

###### Definition 9 (Union consistency) .

An RPR $Z$ satisfies *union consistency* if for any candidate rule set $F$ and profiles $\bm{\sigma}_{a}$ and $\bm{\sigma}_{b}$ such that $Z(F,\bm{\sigma}_{a})\cap Z(F,\bm{\sigma}_{b})\neq\emptyset$, we have

$$ $\displaystyle Z(F,\bm{\sigma}_{a}+\bm{\sigma}_{b})=Z(F,\bm{\sigma}_{a})\cap Z(F,\bm{\sigma}_{b}).$ $$

Unlike previous axioms, $AbC$ does not satisfy UC, but (as we show below) neither does any RPR satisfying reversal symmetry and plurality-shuffling consistency, giving us our first impossibility result for RPRs. Further, we argue that union consistency is less significant for an RPR than its counterpart for voting rules: just because a rule is appropriate for two different sets of rankings does not necessarily mean it is appropriate for their union, especially if they are differently distributed. Indeed, in the extreme case of a single voter, *any* sensible aggregation rule (*i.e.* that returns the ranking itself) is appropriate.

###### Definition 9 (Union consistency) .

#### Main result of Section 5.1

We now present [Theorem 1](https://arxiv.org/html/2508.17177v1#Thmthm1), which states our results regarding the consistency axioms in [Section 5.1](https://arxiv.org/html/2508.17177v1#S5.SS1).

###### Theorem 1 .

(1) $AbC$ satisfies reversal symmetry.
(2) Any welfare-maximizing RPR fails plurality-shuffling consistency; $AbC$ satisfies it.
(3) No (anonymous) RPR can satisfy all three of reversal symmetry, plurality-shuffling consistency, and union consistency.

The proof can be found in [Section A.1](https://arxiv.org/html/2508.17177v1#A1.SS1). The axioms in [Theorem 1](https://arxiv.org/html/2508.17177v1#Thmthm1) are specific for RPRs, demonstrating $AbC$ responds consistently to changes in the profile. Next, we will turn our attention to existing axioms for SWFs and how $AbC$ (and RPRs in general) interact with them.

###### Theorem 1 .

### 5.2 Preserved axioms

In this section, we investigate which axiomatic properties (for SWFs) are retained by an RPR when all SWFs in the set of candidate rules satisfy them.

While using an RPR has advantages beyond just outputting an aggregate ranking (see [Section 1](https://arxiv.org/html/2508.17177v1#S1)), it is true that an RPR $Z$, paired with a set of candidate rules $F$, *induces* an SWF:

$$ $\displaystyle f^{F}_{Z}(\bm{\sigma})\stackrel{{\scriptstyle\text{def}}}{{=}}f(\bm{\sigma})\text{ where }Z(F,\bm{\sigma})=\{f\},$ $$

*i.e.*, $f_{Z}^{F}$ is the SWF that first maps an input profile to an SWF using $Z$, and then applies that SWF to the profile. For $f^{F}_{Z}$ to be well-defined, we must have $|Z(F,\bm{\sigma})|=1$. As we are interested in the properties of $f^{F}_{AbC}$, for this subsection alone, we restrict our setting to candidate rules and profiles for which $|AbC(F,\bm{\sigma})|=1$, and to RPRs $Z$ for which $|Z(F,\bm{\sigma})|=1$ for these profiles.

Clearly, the axiomatic properties of $f_{Z}^{F}$ as an SWF for any RPR $Z$ depend on our selection of the candidate rules $F$. By restricting $F$ to SWFs that all satisfy a certain property, can we ensure that so will $f_{Z}^{F}$? More formally, we say an RPR $Z$ *preserves* a property $P$ if whenever $P$ is true for all candidate rules $f\in F$, then $P$ is true for $f^{F}_{Z}$. For example, recall that an SWF is anonymous if permuting the voters $N$ in a profile does not change the output ranking of the SWF, *i.e.*, all voters are treated equally. Similarly, an SWF is neutral if permuting the alternatives $A$ in a profile results in the output ranking of the SWF being permuted in the same way, *i.e.*, all alternatives are treated equally. It is immediate from [Section 4](https://arxiv.org/html/2508.17177v1#S4) that $AbC$ preserves these two properties. On the other hand, this is not true for all RPRs, *e.g.*, for an alternative $a\in A$, say the RPR $Z_{a}$ maps each profile $\bm{\sigma}$ to the candidate rule $f\in F$ that ranks $a$ highest in $f(\bm{\sigma})$. Then, $f_{Z_{a}}^{F}$ is clearly not neutral, even if all $f\in F$ are. Still, as we show in [Theorem 2](https://arxiv.org/html/2508.17177v1#Thmthm2) below, certain fundamental social choice axioms are preserved by *all* RPRs.

However, not every axiom is as easily preserved. Given a weak ranking $r\in\mathcal{R}(A)$ and an alternative $a\in A$, say $\rank_{r}(a)=1+|\{b\in A:b\succ_{r}a\}|$ is the rank of $a$ in $r$. We say an SWF $f$ satisfies *monotonicity* if for any profile $\bm{\sigma}$ and alternative $a\in A$, if $\bm{\sigma}^{\prime}$ is the same as $\bm{\sigma}$ except some voters now rank $a$ higher, then $\rank_{f(\bm{\sigma})}(a)\geq\rank_{f(\bm{\sigma}^{\prime})}(a)$. In words, monotonicity dictates that promoting an alterative in some voters’ rankings while keeping all else constant should not hurt the alternative in the final output ranking. It turns out $AbC$ does not preserve monotonicity. This is because promoting an alternative may still cause the rule picked by $AbC$ to change to a SWF that ranks that alternative further below, even if it does not hurt that alternative under any fixed candidate rule’s output ranking. Still, much like union consistency, we show that preserving monotonicity is not compatible with another axiom satisfied by $AbC$, reversal symmetry.

We now present [Theorem 2](https://arxiv.org/html/2508.17177v1#Thmthm2), which states our main results from [Section 5.2](https://arxiv.org/html/2508.17177v1#S5.SS2).

###### Theorem 2 .

(1) $AbC$ preserves anonymity and neutrality.

(2) Any RPR $Z$ preserves the Smith criterion, Condorcet consistency, majority winner, pairwise majority consistency, and unanimity.

(3) No (anonymous) RPR can satisfy reversal symmetry and preserve monotonicity.

For readers unfamiliar with the axioms in (2) of [Theorem 2](https://arxiv.org/html/2508.17177v1#Thmthm2), we provide the formal definitions in [Section A.2](https://arxiv.org/html/2508.17177v1#A1.SS2), along with the proof of the full theorem. Statement (2) in the theorem demonstrates a strength of not just $AbC$, but of our RPR framework in general, showing how one can still reap the benefits of the axiomatic approach simply by restricting the set of candidate rules $F$ to certain SWFs.

Overall, we have shown that $AbC$ satisfies many natural axioms (reversal symmetry, plurality-shuffling consistency, and preservation of the axioms in [Theorem 2](https://arxiv.org/html/2508.17177v1#Thmthm2)); as for axioms it fails (union consistency and the preservation of monotonicity), we have given impossibility results showing each of them is incompatible with axioms that $AbC$ satisfies. In [Section B.2](https://arxiv.org/html/2508.17177v1#A2.SS2), we apply the data-driven approach of Caiata et al. [2025] to show experimentally that violations of monotonicity and union consistency are typically rare across well-known distributions, as well as on real-life data.

###### Theorem 2 .

## 6 Computational Problem: PerfPos

Recall from [Section 4](https://arxiv.org/html/2508.17177v1#S4) that $AbC$ picks the SWF(s) from candidate rule set $F$ that minimizes the expected disagreement over a random split. In this section, we show that when $F=F_{S}$ (candidate rules are the set of all positional scoring rules), minimizing the disagreement over even a given split is algorithmically hard. Consider the following computational problem: We are given a strict ranking $\sigma_{i}\in\mathcal{L}(A)$ for each voter $i\in N$, as well as an even split of voters $N=N_{1}\sqcup N_{2}$ with $|N_{1}|=|N_{2}|$. Then, PerfPos (Perfect Positional) asks:

Is there a positional scoring rule $f_{s}\in F_{S}$ that achieves perfect consistency over this split, *i.e.*, obtains $KT(f_{s}(\bm{\sigma}^{(1)}),f_{s}(\bm{\sigma}^{(2)}))=0$?

Here, for each $j\in\{1,2\}$, $\bm{\sigma}^{(j)}=(\sigma_{i})_{i\in N_{j}}$ is the restriction of the profile $\bm{\sigma}$ to voters in $N_{j}$. We now show that this problem is hard.

###### Theorem 3 .

PerfPos is $\mathsf{NP}$-complete.

The proof, which is a reduction from 3SAT, can be found in [Section A.3](https://arxiv.org/html/2508.17177v1#A1.SS3). [Theorem 3](https://arxiv.org/html/2508.17177v1#Thmthm3) shows that when $F=F_{S}$,
it is even hard to decide whether it is possible to have no disagreement at all. This also proves that the problem of computing the minimal possible disagreement for a given split is not only hard, but also hard to approximate to any multiplicative factor (since any algorithm with such a guarantee must return a solution with disagreement 0 when possible). While technically distinct, our result is aligned with other hardness results for optimizing over positional scoring rules, *e.g.*, for picking the rule most consistent with an underlying true ranking to which we are given partial access [Caragiannis et al., 2019].

It is worth noting that [Theorem 3](https://arxiv.org/html/2508.17177v1#Thmthm3) does not rule out efficient implementations of $AbC$ with candidate rule sets $F$ that do not contain all positional scoring rules. In particular, if $F$ is finite and only contains efficiently implementable rules, minimizing disagreement for a given split is easy, since the disagreement of each SWF can be computed one by one. We leave studying the complexity of computing $AbC$ with different sets of candidate rules for future work. Further, despite the complexity result, as we will experimentally show in the next section, an implementation of an approximation of $AbC$ using Monte Carlo sampling (see the discussion of extensions to $AbC$ in [Section 4](https://arxiv.org/html/2508.17177v1#S4)) is computationally efficient in practice and yields desirable results, even when $F_{S}\subseteq F$ (all positional scoring rules are included in the candidate rule set).

###### Theorem 3 .

## 7 Experiments

We now give experimental results evaluating $AbC$ and discuss their implications. Given a profile $\bm{\sigma}$, in order to approximate the expectation in Equation ([2](https://arxiv.org/html/2508.17177v1#S4.E2)), our implementation samples a number of random splits of $\bm{\sigma}$ according to the procedure in [Section 4](https://arxiv.org/html/2508.17177v1#S4), computing the disagreement for each candidate rule on each split, and returning the rule with the minimum average disagreement. We first describe procedures and parameters common through all our experiments. Additional details regarding the experiments, including specific datasets, distributions we use, and parameter settings can be found in [Appendix B](https://arxiv.org/html/2508.17177v1#A2).

- •
*Number of Splits*: With two exceptions, all experiments report the average disagreement resulting from each rule over 10 random splits of voters. The exceptions are: When evaluating axioms ([Section B.2](https://arxiv.org/html/2508.17177v1#A2.SS2), [Figure 4](https://arxiv.org/html/2508.17177v1#A2.F4)) we report the average over 50 splits, and when evaluating score data ([Section 7.2](https://arxiv.org/html/2508.17177v1#S7.SS2), [Table 1](https://arxiv.org/html/2508.17177v1#S7.T1)) we report the average over 1000 splits. These values were decided based on computational and time constraints.
- •
*Splitting Procedure*: When creating two groups of voters we assign each voter to one group or the other uniformly at random, as detailed in [Section 4](https://arxiv.org/html/2508.17177v1#S4). In the case that no voters are assigned to one group, we assign that group a single weak ranking in which all alternatives are tied (see Footnote [4](https://arxiv.org/html/2508.17177v1#footnote4)).
- •
*Weighting Partial Rankings*:
In settings where voters submit partial rankings, we weigh each alternative based on the number of times it appears on each side of a split, as discussed in [Section 4](https://arxiv.org/html/2508.17177v1#S4). This serves to correct for mismatches in the amount of information known about each alternative (*i.e.*, the number of times each alternative is ranked) between the two sides of the split. Consider a split of a fixed profile $\bm{\sigma}$ into ($\bm{\sigma}^{(1)},\bm{\sigma}^{(2)})$ via the process in [Section 4](https://arxiv.org/html/2508.17177v1#S4). For any alternative $a\in A$, let $m_{a}$ denote the minimum number of times $a$ is included in rankings in $\bm{\sigma}^{(1)}$ or $\bm{\sigma}^{(2)}$, and let $t_{a}$ refer to the total number of times $a$ is included in a ranking across the entire profile $\bm{\sigma}$. Then the weight of $a$ is $w_{a}=\frac{\gamma^{m_{a}}-1}{\gamma^{t_{a}/2}-1}$. This ensures that the largest weights are assigned to alternatives that are evenly split across the split ($m_{a}=t_{a}/2$), whereas any alternative that does not appear on one side of the split ($m_{a}=0$) gets zero weight, as it is unreasonable to expect this alternative to be placed consistently across the split by any SWF. All our experiments use a constant factor of $\gamma=2$. Weights are used as a scaling factor in computing the Kendall-Tau distance (as in Kumar and Vassilvitskii 2010); when alternatives $a$ and $b$ are in opposite positions across two rankings then they contribute $w_{a}w_{b}$ to the weighted Kendall-Tau distance (as opposed to the unweighted setting where any misordered pair adds 1 to the distance).
- •
*Computational Power*: All experiments are performed locally on a 2022 M2 Macbook Air with 16 GB of memory. Each individual experiment shown in the paper took between 0.5 and 3 hours to complete.

As discussed in [Section 4](https://arxiv.org/html/2508.17177v1#S4), the SWFs we test include an optimization over all positional scoring rules (“Best Positional Scores”), which uses simulated annealing to search for the scoring vector minimizing the average disagreement for a given profile (see [Section B.7.3](https://arxiv.org/html/2508.17177v1#A2.SS7.SSS3) for details). “Trimmed Borda” refers to a modified version of Borda count where the maximum and minimum rank received by each alternative is deleted from each split, before computing its Borda score with the remaining evaluations [Meyer et al., 2022, Stafford et al., 2025]. All reported distances are normalized by dividing the (weighted) Kendall-Tau distance resulting from each rule to the maximum (weighted) Kendall-Tau distance obtainable for that split (*i.e.*, the distance between two reversed rankings). We now explain the setup of each experiment separately, along with a discussion of their results.

Figure: Figure 1: Log-log plots of $KT$ distance between the ground truth ranking and the ranking generated by SWFs on the complete profile vs. $KT$ distance between rankings generated by SWFs on splits of the profile. The profiles consist of partial rankings drawn from the Mallows and Plackett-Luce (PL) models. Each point shows average distance over 10 splits for a single profile. In both cases, the MLE of the model outperforms other SWFs in both axes.
Refer to caption: plots/ground_truth_vs_split_distance-neurips.png

### 7.1 Ground truth distance vs. disagreement

While $AbC$ does *not* assume a ground truth, what happens if a generative model with a ground truth is indeed a reasonable approximation of the data? In order to investigate this question, we apply $AbC$ to partial rankings synthetically drawn from two well-known distributions: Mallows [1957] and Plackett-Luce [Plackett, 1975, Luce, 1959]. We use profiles of $100$ voters and $100$ items; each voter ranks $10$ items and each item is ranked $10$ times. Since each distribution has a known ground truth, for each SWF in the candidate rule set, we can measure the distance between this ground truth ranking and the ranking generated by the SWF when applied to the entire data set. We then calculate each SWF’s disagreement over random splits, following the procedure in [Section 4](https://arxiv.org/html/2508.17177v1#S4).
Since $AbC$ picks the SWF that minimizes this disagreement, we would ideally like to observe that the rule that minimizes the distance between splits (most consistent rule) corresponds to the rule which minimizes the distance between its output on the full profile and the ground truth (the rule with the minimum error).

Indeed, this is the case:
[Figure 1](https://arxiv.org/html/2508.17177v1#S7.F1) highlights a clear positive relationship between distance to ground truth and disagreement between splits, confirming our intuition from MVUEs ([Section 2](https://arxiv.org/html/2508.17177v1#S2)). Importantly, for both Mallows and Placket-Luce, the MLE of the model (Kemeny and PL MLE, respectively) outperforms the other SWFs in both axes. As each MLE has the lowest disagreement for its model, $AbC$ would indeed pick it when given data from this model, hence obtaining the benefits of the statistical approach.

### 7.2 Scientific peer review (score data)

In some cases evaluators submit scores rather than rankings. As discussed in the extensions listed in [Section 4](https://arxiv.org/html/2508.17177v1#S4), $AbC$ is still applicable in this case. Rather than considering functions for aggregating sets of preference rankings, we can also use our method to consider functions for aggregating sets of review scores, in order to pick the one yielding the most consistent results.
We explore this setting on a dataset consisting of peer review scores given to proposals for observatory telescope time [Kerzendorf et al., 2020]. In this data, 136 reviewers have provided feedback on between 4 and 8 proposals each. We filter data to consider the 119 proposals that received at least 6 scores (No proposal was reviewed by more than 8 reviewers).

Rather than splitting the set of reviewers to two sides as done in previous sections, we individually split the 6 to 8 scores received by each unique proposal. Each split is generated by randomly placing half of the scores into one side of the split, and the remaining half into the other side. Where there are an odd number of scores we select a random score to leave out of either side, resulting in each proposal having a split into two sides of equal size.
We then run each score aggregation function we consider (listed below) on each side of the split for every proposal, resulting two sets of aggregate scores. From these we generate two rankings over all proposals, one for each side of the split. We measure the Kendall-Tau distance between these two rankings. This process (creating splits, finding an aggregate score for each split, converting scores to a ranking, measuring Kendall-Tau distance) is repeated over 1000 randomized trials to generate 1000 distance scores, and we report the average score for each function. We consider the following aggregation functions: Arithmetic Mean, Min, Max, Median, Geometric Mean, and Trimmed Mean (which takes the arithmetic mean of scores after trimming the highest and lowest score from each split).

**Table 1: Mean (and standard error of the mean) of $KT$ distances over 1000 splits resulting from several aggregation functions on review scores from Kerzendorf et al. [2020].**
| Arithmetic Mean | Min | Max | Median | Geometric Mean | Trimmed Mean |
| --- | --- | --- | --- | --- | --- |
| $0.364\pm 0.001$ | $0.444\pm 0.001$ | $0.409\pm 0.001$ | $0.371\pm 0.001$ | $0.369\pm 0.001$ | $0.371\pm 0.001$ |

Our results, shown in [Table 1](https://arxiv.org/html/2508.17177v1#S7.T1), can help guide best practices for aggregating such review scores. For example, past work has suggested that items with a single high score are often prioritized in peer review [Nierstrasz, 2000] – corresponding to rankings generated by the Max function. In contrast, our results demonstrate that aggregation using Max scores leads to high levels of disagreement compared to functions that directly incorporate the score of multiple reviewers, such as Arithmetic/Geometric Mean.

### 7.3 Scientific peer review (rank data)

Figure: (a) Distance between splits of partial rankings for each SWF on reviews of ALMA Cycle 10 project proposals.
Refer to caption: plots/alma_data_cycle10-full-distances.png

Peer review data can also take the form of rankings. Here we consider anonymized rankings provided by evaluators of projects proposed for the Atacama Large Millimeter Array (ALMA) [Meyer et al., 2022], which is the largest ground-based radio telescope on earth.

We conduct our experiments on two sets of proposals, from Cycle 10 and Cycle 11. In this review process, each proposer is asked to review. Cycle 10 contains 1635 proposals and 1635 reviewers, while Cycle 11 contains 1729 proposals and 1729 reviewers. In each cycle, every reviewer is assigned 10 proposals to review, and every proposal is assigned to 10 reviewers. Each reviewer provides a strict ranking of the 10 proposals assigned to them.

We run $AbC$ on the rankings provided by the reviewers. The mean Kendall-Tau distances over 10 splits for several SWFs are displayed in [Figure 2](https://arxiv.org/html/2508.17177v1#S7.F2). We see a pattern consistent with the results of [Figure 1](https://arxiv.org/html/2508.17177v1#S7.F1): PL MLE, Borda, and optimized positional scores result in very similar distances, suggesting a natural lower bound on the Kendall-Tau distance based on the data and the noise induced by generating splits. Kemeny rule,(^6^66Note that, due to computational limits, in this setting we use the best ranking found by the Kemeny function ([Section B.7.1](https://arxiv.org/html/2508.17177v1#A2.SS7.SSS1)) within a time limit of 15 minutes per split.) on the other hand, performs notably worse than this lower bound. Our experiment also finds that better consistency is achieved by positional scoring rules which provide more information about the profile (i.e., Two Approval and Plurality + Veto provide two “bits” of information, compared to one bit from Plurality and Veto, whereas Borda and the optimized positional scores provide significantly more information).

Importantly, our method can be used to evaluate proposed changes to current practices. For example, Meyer et al. [2022] consider removing the minimum & maximum score of each alternative before computing the Borda score, termed ‘Trimmed Borda’,
but our experiment suggests this method increases disagreement on their dataset compared to Borda count, significantly more than it did on data generated from known noise models ([Figure 1](https://arxiv.org/html/2508.17177v1#S7.F1)).

### 7.4 Political elections

Similar to studying the impact of changes to peer-review practices, we can use our implementation of $AbC$ to evaluate the methods being actively used in real-world elections. Accordingly, [3(a)](https://arxiv.org/html/2508.17177v1#S7.F3.sf1) considers running $AbC$ on several political elections using data from Preflib [Mattei and Walsh, 2013]. These elections took place in several municipalities across the United States between 2008 and 2012; all used Instant Runoff Voting (IRV) to determine a winner (see [Section B.4](https://arxiv.org/html/2508.17177v1#A2.SS4) for the definition of IRV), which we implement as one of our candidate rules. These elections differ from our other data in that they represent human political preferences, and they have a highly varied number of voters (between 2477 and 262312) and candidates (between 4 and 25). See [Table 2](https://arxiv.org/html/2508.17177v1#A2.T2) in [Appendix B](https://arxiv.org/html/2508.17177v1#A2) for additional details on each election evaluated in this experiment. Unsurprisingly, we observe that elections with consistently low disagreement across voting methods (*e.g.*, 3, 23) have few candidates, while elections in which no voting method achieves a low disagreement (e.g., 11, 19, 21) have many candidates. That is, there is likely to be at least some disagreement between rankings when ballots are quite long—simply because there are many more opportunities for rankings to differ.

While IRV was used in practice during each of these elections it is, in some cases, the rule which gives the *least* consistency (*e.g.*, elections 1, 16, 18). In 21 of the 25 elections we show in [3(a)](https://arxiv.org/html/2508.17177v1#S7.F3.sf1) there is some rule that achieves zero disagreement, however this rule is different between elections, confirming out intuition from [Section 1](https://arxiv.org/html/2508.17177v1#S1) that different rules are appropriate for different settings. By always selecting the rule that results in the minimum disagreement, $AbC$ provides a reliable method of selecting a consistent ranking.

### 7.5 Aggregating sporting contests

Figure: (a) Distance between splits of rankings in empirical election data from cities using Instant Runoff Voting (IRV). Details on data in this plot are found in [Table 2](https://arxiv.org/html/2508.17177v1#A2.T2)
Refer to caption: plots/preflib_scatter.png

We also consider real-world settings in which participants are ranked across many events, such as sporting competitions. In these settings, each event assumes the role of a voter: the ordinal position of the competitors in each event translates directly into a ranking over competitors, which must then be aggregated into a single ranking across all events. We evaluate $AbC$ on two sporting settings: Formula One (F1) races in this section and data from the Olympics in [Section B.6](https://arxiv.org/html/2508.17177v1#A2.SS6).

We use data showing F1 results for each season from 1991 to 2018, inclusive, compiled by Boehmer et al. [2022]. Each season corresponds to a set of rankings (hence a profile), where each ranking records the order in which the participating drivers finished a given race. During this time period, F1 used three distinct positional scoring rules (updating the rule in 2003 and 2010) on the individual races to determine overall winners:

- •
1991 - 2002: $(10,6,4,3,2,1,0,0,...,0)$
- •
2003 - 2009: $(10,8,6,5,4,3,2,1,0,0,...,0)$
- •
2010 - 2018: $(25,18,15,12,10,8,6,4,2,1,0,0,...,0)$

From this data, we remove races and drivers until each race includes only drivers that competed in all events that year; we refer to the Preflib repository of Mattei and Walsh [2013] for details on this processing. This results in 27 years of data with a mean of 13.7 (mode of 16) races per year, and a mean of 20.6 (mode of 20) drivers per race. We then apply our standard $AbC$ methodology on this data: We generate splits for each year, apply several scoring rules to generate a ranking for each side of a split (including the three rules employed by F1), and measure the average distance between splits for each rule.

[Figure 3(b)](https://arxiv.org/html/2508.17177v1#S7.F3.sf2) compares the average distances resulting from each rule. The three leftmost columns show the distances of existing F1 rules, applied only to the years in which they were being employed. This shows that both changes to the rule being used in F1 competitions were beneficial in increasing consistency.
The other rules shown in [Figure 3(b)](https://arxiv.org/html/2508.17177v1#S7.F3.sf2) are evaluated over all years of data. We see that recent F1 rules are significantly better than well-known positional scoring rules, including Borda count. However, our optimization of the scoring vector is able to provide some advantage over the employed rules (noting that our Best Positional Score vector is evaluated over all years while F1 rules are evaluated only on a subset of the data). These effects are demonstrated further when we evaluate all rules on each time period separately (see [Section B.5](https://arxiv.org/html/2508.17177v1#A2.SS5)).

## 8 Additional Related Work

In this section, we further elaborate on prior work conceptually related to our work.

##### Rule selection

The question of comparing different voting rules has been widely studied in the social choice literature. For an overview of the axiomatic and computational properties of various voting rules, see Brandt et al. [2016]. There has been earlier empirical work on collecting voters’ preferences over voting rules themselves [Sertel and Kara, 2003, Giritligil Kara and Sertel, 2005, Aldrich et al., 2014]. However, it has been shown that (unsurprisingly) in many real-life cases voters simply prefer the rules that benefit their preferred candidate [Blais et al., 2015, Weber, 2020]. In general, it is clear that even social choice theorists themselves cannot reach a consensus on what is the best voting rule [Laslier, 2012]. There has also been interest in using tools from automated reasoning for instantiating and generating formal justifications for using a voting rule [Cailloux and Endriss, 2016] or imposing an axiom [Boixel and Endriss, 2020]. Similar tools were used for bypassing voting rules altogether by picking a set of axioms that impose a certain outcome [Schmidtlein and Endriss, 2023].

##### MLE approach

The idea of treating votes as noisy estimates of a ground truth goes back to Condorcet [Marquis de Condorcet], who designed a noise model where every voter has a fixed probability of ranking each pair of alternatives correctly, and solved it for two and three alternatives. Young [1995] later extended this model to an arbitrary number of alternatives, showing that its MLE is equivalent to an earlier rule introduced by Kemeny [1959]. Conitzer and Sandholm [2005] later studied the question of which voting rules are MLEs for some noise model where votes are sampled independently, in particular showing that any rule that violates consistency cannot be an MLE. Combined with the result that a continuous and neutral social choice function (rules outputing a set of winners) is consistent if and only if it is a scoring rule [Young, 1975], this shows that no other (neutral and continuous) social choice function can be the MLE of any noise model. Conitzer et al. [2009] did a similar analysis for social preference functions (rules outputting a set of rankings) providing an exact characterization of MLE rules as simple ranking scoring functions. Similar analyses followed for voting in multi-issue domains [Xia et al., 2010], with partial rankings [Xia and Conitzer, 2011], and on social networks (where voters no longer vote independently) Conitzer [2013]. In the opposite direction, Azari Soufiani et al. [2014] study the MLE of two known noise models to identify whether they satisfy certain axiomatic properties, and Xia [2016] extend their results. Tideman and Plassmann [2014], on the other hand, construct a spatial noise model specifically to approximate data from actual elections, and evaluate the performance of common voting rules under this model. Lastly, a related but distinct approach to classify voting rules is distance-rationalizatability (DR) [Meskanen and Nurmi, 2008, Elkind et al., 2009], which requires a rule to map each election to the result of the closest consensus according to some metric. Elkind et al. [2010] combine the MLE and DR approaches to better understand and compare rules, and in this framework declare Kemeny to be the best rule, as it fits both frameworks with the same underlying function. It is worth noting, however, that DR only applies to social choice functions.

##### Statistical model dependence in estimation from pairwise comparisons

Another line of literature on model (in)dependence focuses on estimation from pairwise comparisons. Commonly used statistical models in this setting include the Bradley–Terry–Luce (BTL)  [Bradley and Terry, 1952, Plackett, 1975] and Thurstone models  [Thurstone, 1927], which all fall under the class of parameter-based models (also known as random utility models). However, recent studies  [Shah et al., 2017, Shah and Wainwright, 2018, Heckel et al., 2019] have demonstrated that estimators derived from more general “permutation-based” models offer two notable advantages. First, when data are generated from these broader permutation-based models, these estimators exhibit significantly better performance. Second, even when data originate from parameter-based models such as BTL or Thurstone, the guarantees provided by these general estimators are within logarithmic factors of those achieved by estimators specifically tailored to parameter-based models.

## 9 Conclusions and Future Work

In this paper, we introduced the problem of finding good rule picking rules, and argued for its importance. We also introduced a specific rule, $AbC$, and showed via theoretical analysis and experiments that $AbC$ is able to attain the benefits of both the axiomatic and statistical approaches. As demonstrated by our experiments in [Section 7](https://arxiv.org/html/2508.17177v1#S7), $AbC$ provides a principled way for users to evaluate the impact of modifications to existing methods by running $AbC$ on their own data. In some cases, the answer is that it is an improvement: both changes to F1 rules significantly reduced disagreement ([Figure 3(b)](https://arxiv.org/html/2508.17177v1#S7.F3.sf2)). In others, the proposed modification hurts consistency: Meyer et al. [2022] consider adding outlier rejection to Borda for peer review, but our experiment suggests this method (Trimmed Borda) increases disagreement on their data. By running similar experiments on their own datasets, implementers of our method can determine which aggregation rules, among the ones they are considering, provide the most consistent results for their own applications.

We now discuss several limitations of our approach, and also provide avenues of future work. One of the main limitations is that our approach treats voters’ rankings as fixed, whereas in practice voters may choose their rankings based on the RPR being employed. It is of interest to understand RPRs—and $AbC$—more generally under strategic behavior of voters. Such behavior makes it harder to study the impact of a proposed rule change on consistency by running $AbC$ on prior data.

A separate limitation is that our sampling-based implementation [Section 7](https://arxiv.org/html/2508.17177v1#S7) only approximates $AbC$. However, in addition to its desirable behavior on known distributions, our implementation exhibits strong axiomatic properties: an experiment we provide in [Section B.2](https://arxiv.org/html/2508.17177v1#A2.SS2) shows that even the two axioms failed by the exact $AbC$ (preserving monotonicity and union consistency) are rarely violated in practice by our implementation, on both simulated and real-world data. Still, it is worth exploring theoretical bounds on how likely our sampling-based implementation is to output the true $AbC$ winner from [Section 4](https://arxiv.org/html/2508.17177v1#S4). In our experiments, we see that even a small number of splits works well in practice, with additional splits leading to qualitatively identical results. Since the disagreement scores of a given rule for several splits of a given profile are i.i.d., bounding the variance in terms of intuitive properties for the profile (capturing, for example, how close it is to being “borderline”) can allow using standard tail bounds for bounding the required number of splits. We believe identifying the correct properties of a profile for such a statistical analysis is a worthwhile direction.

Another future avenue is to leverage the generality of [Algorithm 1](https://arxiv.org/html/2508.17177v1#algorithm1) to study $AbC$ on different output formats (other than rankings) and different distance functions (other than Kendall-Tau distance). For example, in practice, ALMA rankings from [Section 7.3](https://arxiv.org/html/2508.17177v1#S7.SS3) are used for determining $k$ proposals to be funded for some fixed $k<|A|$. Thus, running $AbC$ to pick among multi-winner rules (that return a subset of $k$ alternatives) by using a distance function—such as Jaccard dissimilarity—that compare such subsets may be more appropriate for such data. In [Section B.3](https://arxiv.org/html/2508.17177v1#A2.SS3) we find that experiments using Jaccard dissimilarity provide qualitatively similar results to those in [Section 7.3](https://arxiv.org/html/2508.17177v1#S7.SS3) when used on the same data. Further, randomized decisions (methods that output a distribution over subsets of $k$ winners) based on the peer-review process are gaining popularity [Heyard et al., 2022, Goldberg et al., 2025], adding to the richness of this setting.

Similar questions exist for single-winner elections: while the 0-1 loss (is the same candidate picked on both sides of a split?) emerges as a natural option for measuring disagreement, one can also use other notions of distance between the alternatives chosen on the two sides of a split. For example, Berker et al. [2025, Prop. 26] introduce the “clone distance” between any pair of candidates, which measures the dissimilarity of the candidates from the perspective of the voters. Crucially, measuring this distance requires no external information about the candidates, only their positions in the profile. Another alternative would be to repeatedly apply the single-winner rule to the profile, removing the winner after each step. By comparing the resulting rankings from the two sides using another weighted variant of the Kendall-Tau distance (similar to Kumar and Vassilvitskii [2010]) the implementer can then balance between the two extremes (0-1 loss vs. full ranking) by deciding how much weight to put on the first position of the ranking relative to the others. Overall, studying the tradeoff between various distance functions is important for future applications of $AbC$.

Lastly, while we developed and studied several axioms, future work can investigate RPRs under other axioms, either translated from social choice axioms or novel axioms for rule picking rules. Such axioms may inform the design of novel RPRs that focuses on properties other than consistency. Overall, RPRs (and $AbC$) open the door to principled, data-driven rule selection, paving the way for diverse real-world applications.

## Acknowledgements

We thank John Carpenter and Andrea Corvillon from the Atacama Large Millimeter/submillimeter Array (ALMA), as well as Kate Larson and Ariel Procaccia for helpful discussions. We also thank Sinan Karaböcüoğlu and Boğaçhan Arslan for implementing our method for the *2nd Computational Social Choice Competition* at the 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024). R.E.B. and V.C. thank the Cooperative AI Foundation, Macroscopic Ventures (formerly Polaris Ventures
/ the Center for Emerging Risk Research), and Jaan Tallinn’s donor-advised fund at Founders Pledge for financial support. R.E.B. is also supported by the Cooperative AI PhD Fellowship. The work of N.S. was supported by NSF CAREER award 1942124 and ONR N000142512346.

## References

- Ailon et al. [2005]
Nir Ailon, Moses Charikar, and Alantha Newman.
Aggregating inconsistent information: Ranking and clustering.
In *Proceedings of the Annual Symposium on Theory of Computing (STOC)*, 2005.
- Aldrich et al. [2014]
John Aldrich, Jason Reifler, and Michael C. Munger.
Sophisticated and myopic? citizen preferences for electoral college reform.
*Public Choice*, 158(3/4):541–558, 2014.
- Armstrong [2025]
Ben Armstrong.
Optimal voting, 2025.
URL [https://pypi.org/project/optimal-voting](https://pypi.org/project/optimal-voting).
- Arrow [1963]
Kenneth Arrow.
*Social choice and individual values*.
New Haven: Cowles Foundation, 2^nd edition, 1963.
1^st edition 1951.
- Ash [1990]
R.B. Ash.
*Information Theory*.
Dover books on advanced mathematics. Dover Publications, 1990.
- Azari Soufiani et al. [2014]
Hossein Azari Soufiani, David C Parkes, and Lirong Xia.
A statistical decision-theoretic framework for social choice.
In *Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS)*, 2014.
- Baharev et al. [2021]
Ali Baharev, Hermann Schichl, Arnold Neumaier, and Tobias Achterberg.
An exact method for the minimum feedback arc set problem.
*Journal of Experimental Algorithmics (JEA)*, 26:1–28, 2021.
- Bast [2020]
Hannah Bast.
How objective is peer review?, 2020.
URL [https://cacm.acm.org/blogcacm/how-objective-is-peer-review/](https://cacm.acm.org/blogcacm/how-objective-is-peer-review/).
Online; accessed 20-January-2025.
- Berker et al. [2025]
Ratip Emin Berker, Sílvia Casacuberta, Isaac Robinson, Christopher Ong, Vincent Conitzer, and Edith Elkind.
From independence of clones to composition consistency: A hierarchy of barriers to strategic nomination.
In *Proceedings of the ACM Conference on Economics and Computation (EC)*, 2025.
- Beygelzimer et al. [2023]
Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan.
Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment, 2023.
- Blais et al. [2015]
André Blais, Jean-François Laslier, François Poinas, and Karine Van Der Straeten.
Citizens’ preferences about voting rules: self-interest, ideology, and sincerity.
*Public Choice*, 164(3/4):423–442, 2015.
- Boehmer et al. [2021]
Niclas Boehmer, Robert Bredereck, Piotr Faliszewski, Rolf Niedermeier, and Stanisław Szufa.
Putting a compass on the map of elections.
*arXiv preprint arXiv:2105.07815*, 2021.
- Boehmer et al. [2022]
Niclas Boehmer, Robert Bredereck, Piotr Faliszewski, and Rolf Niedermeier.
A quantitative and qualitative analysis of the robustness of (real-world) election winners.
In *Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization*, pages 1–10, 2022.
- Boixel and Endriss [2020]
Arthur Boixel and Ulle Endriss.
Automated justification of collective decisions via constraint solving.
In *Autonomous Agents and Multi-Agent Systems*. International Foundation for Autonomous Agents and Multiagent Systems, 2020.
- Bradley and Terry [1952]
Ralph Allan Bradley and Milton E Terry.
Rank analysis of incomplete block designs: I. the method of paired comparisons.
*Biometrika*, 39(3/4):324–345, 1952.
- Brandt et al. [2016]
Felix Brandt, Vincent Conitzer, Ulle Endriss, Jérôme Lang, and Ariel D. Procaccia, editors.
*Handbook of Computational Social Choice*.
Cambridge University Press, 2016.
- Caiata et al. [2025]
Joshua Caiata, Ben Armstrong, and Kate Larson.
What voting rules actually do: A data-driven analysis of multi-winner voting.
*arXiv preprint arXiv:2508.06454*, 2025.
- Cailloux and Endriss [2016]
Olivier Cailloux and Ulle Endriss.
Arguing about voting rules.
In *Autonomous Agents and Multi-Agent Systems*, 2016.
- Caragiannis and Procaccia [2011]
Ioannis Caragiannis and Ariel D. Procaccia.
Voting almost maximizes social welfare despite limited communication.
*Artificial Intelligence*, 175(9):1655–1671, 2011.
- Caragiannis et al. [2019]
Ioannis Caragiannis, Xenophon Chatzigeorgiou, George A. Krimpas, and Alexandros A. Voudouris.
Optimizing positional scoring rules for rank aggregation.
*Artificial Intelligence*, 267:58–77, 2019.
- Chapman and Robbins [1951]
Douglas G Chapman and Herbert Robbins.
Minimum variance estimation without regularity assumptions.
*The Annals of Mathematical Statistics*, pages 581–586, 1951.
- Conitzer [2013]
Vincent Conitzer.
The maximum likelihood approach to voting on social networks.
In *Proceedings of the Conference on Communication, Control, and Computing (Allerton)*, 2013.
- Conitzer and Sandholm [2005]
Vincent Conitzer and Tuomas Sandholm.
Common voting rules as maximum likelihood estimators.
In *Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI)*, 2005.
- Conitzer et al. [2009]
Vincent Conitzer, Matthew Rognlie, and Lirong Xia.
Preference functions that score rankings and maximum likelihood estimation.
In *Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)*, 2009.
- Conitzer et al. [2024]
Vincent Conitzer, Rachel Freedman, Jobst Heitzig, Wesley H. Holliday, Bob M. Jacobs, Nathan Lambert, Milan Mossé, Eric Pacuit, Stuart Russell, Hailey Schoelkopf, Emanuel Tewolde, and William S. Zwicker.
Social choice should guide AI alignment in dealing with diverse human feedback.
In *Proceedings of the International Conference on Machine Learning (ICML)*, 2024.
- Cortes and Lawrence [2021]
Corinna Cortes and Neil D. Lawrence.
Inconsistency in conference peer review: Revisiting the 2014 NeurIPS experiment, 2021.
- Dai and Fleisig [2024]
Jessica Dai and Eve Fleisig.
Mapping social choice theory to RLHF.
*arXiv preprint arXiv:2404.13038*, 2024.
- de Caritat (1785) [Marquis de Condorcet]
Marie Jean Antoine Nicolas de Caritat (Marquis de Condorcet).
Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix.
*Paris: L’Imprimerie Royale*, 1785.
- Elkind et al. [2009]
Edith Elkind, Piotr Faliszewski, and Arkadii Slinko.
On distance rationalizability of some voting rules.
In *Proceedings of the Conference on Theoretical Aspects of Rationality and Knowledge (TARK)*, 2009.
- Elkind et al. [2010]
Edith Elkind, Piotr Faliszewski, and Arkadii Slinko.
Good rationalizations of voting rules.
In *Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)*, 2010.
- Fogelholm et al. [2012]
Mikael Fogelholm, Saara Leppinen, Anssi Auvinen, Jani Raitanen, Anu Nuutinen, and Kalervo Väänänen.
Panel discussion does not improve reliability of peer review for medical research grant proposals.
*Journal of Clinical Epidemiology*, 65(1):47–52, 2012.
- Ge et al. [2024]
Luise Ge, Daniel Halpern, Evi Micha, Ariel D. Procaccia, Itai Shapira, Yevgeniy Vorobeychik, and Junlin Wu.
Axioms for AI alignment from human feedback, 2024.
- Gershkov et al. [2017]
Alex Gershkov, Benny Moldovanu, and Xianwen Shi.
Optimal voting rules.
*The Review of Economic Studies*, 84(2):688–717, 2017.
- Gibbard [1973]
Allan Gibbard.
Manipulation of voting schemes.
*Econometrica*, 41:587–602, 1973.
- Giritligil Kara and Sertel [2005]
Ayça E. Giritligil Kara and Murat R. Sertel.
Does majoritarian approval matter in selecting a social choice rule? an exploratory panel study.
*Social Choice and Welfare*, 25(1):43–73, 2005.
- Goldberg et al. [2025]
Alexander Goldberg, Giulia Fanti, and Nihar B Shah.
A principled approach to randomized selection under uncertainty.
*arXiv preprint arXiv:2506.19083*, 2025.
- Griffin [2018]
Randi Griffin.
120 years of olympic history: Athletes and results.
Online dataset, 2018.
URL [https://www.kaggle.com/datasets/heesoo37/120-years-of-olympic-history-athletes-and-results/data](https://www.kaggle.com/datasets/heesoo37/120-years-of-olympic-history-athletes-and-results/data).
- Gurobi Optimization, LLC [2024]
Gurobi Optimization, LLC.
Gurobi Optimizer Reference Manual, 2024.
URL [https://www.gurobi.com](https://www.gurobi.com).
- Hajek et al. [2014]
Bruce Hajek, Sewoong Oh, and Jiaming Xu.
Minimax-optimal inference from partial rankings.
In *Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS)*, volume 27, 2014.
- Heckel et al. [2019]
Reinhard Heckel, Nihar B Shah, Kannan Ramchandran, and Martin J Wainwright.
Active ranking from pairwise comparisons and when parametric assumptions do not help.
*The Annals of Statistics*, 47(6):3099–3126, 2019.
- Heyard et al. [2022]
Rachel Heyard, Manuela Ott, Georgia Salanti, and Matthias Egger.
Rethinking the funding line at the Swiss National Science Foundation: Bayesian ranking and lottery.
*Statistics and Public Policy*, 9(1):110–121, 2022.
- Holliday and Pacuit [2023]
Wesley H. Holliday and Eric Pacuit.
Split cycle: a new condorcet-consistent voting method independent of clones and immune to spoilers.
*Public Choice*, 197(1-2):1–62, August 2023.
- Jaccard [1901]
Paul Jaccard.
Étude comparative de la distribution florale dans une portion des alpes et des jura.
*Bull Soc Vaudoise Sci Nat*, 37:547–579, 1901.
- Jecmen et al. [2022]
Steven Jecmen, Hanrui Zhang, Ryan Liu, Fei Fang, Vincent Conitzer, and Nihar B. Shah.
Near-optimal reviewer splitting in two-phase paper reviewing and conference experiment design.
In *Autonomous Agents and Multi-Agent Systems*, 2022.
- Kemeny [1959]
John Kemeny.
Mathematics without numbers.
In *Daedalus*, volume 88, pages 571–591. The MIT Press, 1959.
- Kendall [1945]
M. G. Kendall.
The treatment of ties in ranking problems.
*Biometrika*, 33(3):239–251, 1945.
- Kerzendorf et al. [2020]
Wolfgang E. Kerzendorf, Ferdinando Patat, Dominic Bordelon, Glenn van de Ven, and Tyler A. Pritchard.
Distributed peer review enhanced with natural language processing and machine learning.
*Nature Astronomy*, 4(7):711–717, 2020.
- Khetan and Oh [2016]
Ashish Khetan and Sewoong Oh.
Data-driven rank breaking for efficient rank aggregation.
*Journal of Machine Learning Research*, 17(193):1–54, 2016.
- Kumar and Vassilvitskii [2010]
Ravi Kumar and Sergei Vassilvitskii.
Generalized distances between rankings.
In *Proceedings of the International Conference on World Wide Web (WWW)*, 2010.
- Laslier [2012]
Jean-François Laslier.
*And the Loser Is… Plurality Voting*, pages 327–351.
Springer Berlin Heidelberg, 2012.
- Lawrence and Cortes [2014]
Neil Lawrence and Corinna Cortes.
The NIPS Experiment, 2014.
URL [http://inverseprobability.com/2014/12/16/the-nips-experiment](http://inverseprobability.com/2014/12/16/the-nips-experiment).
Online; accessed 24-July-2024.
- Luce [1959]
R. Duncan Luce.
*Individual Choice Behavior: A Theoretical analysis*.
Wiley, 1959.
- Mallows [1957]
C. L. Mallows.
Non-null ranking models. I.
*Biometrika*, 44(1/2):114–130, 1957.
- Mattei and Walsh [2013]
Nicholas Mattei and Toby Walsh.
Preflib: A library for preferences.
In *Algorithmic Decision Theory*, volume 8176 of *Lecture Notes in Computer Science*, pages 259–270. Springer, 2013.
- Meskanen and Nurmi [2008]
Tommi Meskanen and Hannu Nurmi.
*Closeness Counts in Social Choice*, pages 289–306.
Springer Berlin Heidelberg, 2008.
- Meyer et al. [2022]
Jennifer Donovan Meyer, Andrea Corvillón, John M Carpenter, Adele L Plunkett, Robert Kurowski, Alex Chalevin, Jakob Bruenker, D-C Kim, and Enrique Macías.
Analysis of the ALMA cycle 8 distributed peer review process.
*arXiv preprint arXiv:2204.05390*, 2022.
- Mishra [2023]
Abhilash Mishra.
AI alignment and social choice: Fundamental limitations and policy implications.
*arXiv preprint arXiv:2310.16048*, 2023.
- Nierstrasz [2000]
Oscar Nierstrasz.
Identify the champion.
In *Pattern Languages of Program Design 4*, 2000.
- Obrecht et al. [2007]
Michael Obrecht, Karl Tibelius, and Guy D’Aloisio.
Examining the value added by committee discussion in the review of applications for research awards.
*Research Evaluation*, 16(2):79–91, 06 2007.
- Patat et al. [2019]
F. Patat, W. Kerzendorf, D. Bordelon, G. Van de Ven, and T. Pritchard.
The Distributed Peer Review Experiment.
*The Messenger*, 177:3–13, September 2019.
- Pier et al. [2017]
Elizabeth L. Pier, Joshua Raclaw, Anna Kaatz, Markus Brauer, Molly L Carnes, Mitchell J. Nathan, and Cecilia E. Ford.
‘your comments are meaner than your score’: score calibration talk influences intra- and inter-panel variability during scientific grant peer review.
*Research Evaluation*, 26 1:1–14, 2017.
- Plackett [1975]
R. L. Plackett.
The analysis of permutations.
*Journal of the Royal Statistical Society. Series C (Applied Statistics)*, 24(2):193–202, 1975.
- Plott [1976]
Charles R. Plott.
Axiomatic social choice theory: An overview and interpretation.
*American Journal of Political Science*, 20(3):511–596, 1976.
- Procaccia et al. [2009]
Ariel D. Procaccia, Aviv Zohar, Yoni Peleg, and Jeffrey S. Rosenschein.
The learnability of voting rules.
*Artificial Intelligence*, 173(12):1133–1149, 2009.
- Rao [1949]
C Radhakrishna Rao.
Sufficient statistics and minimum variance estimates.
In *Mathematical Proceedings of the Cambridge Philosophical Society*, volume 45, pages 213–218, 1949.
- Saari [1994]
D.G. Saari.
*Geometry of Voting*.
Springer Berlin Heidelberg, 1994.
- Satterthwaite [1975]
Mark Satterthwaite.
Strategy-proofness and Arrow’s conditions: existence and correspondence theorems for voting procedures and social welfare functions.
*Journal of Economic Theory*, 10:187–217, 1975.
- Schmidtlein and Endriss [2023]
Marie Christin Schmidtlein and Ulle Endriss.
Voting by axioms.
In *Autonomous Agents and Multi-Agent Systems*, 2023.
- Sertel and Kara [2003]
Murat R. Sertel and Ayça E. G. Kara.
*Selecting a Social Choice Rule — An Exploratory Panel Study*, pages 19–52.
Springer Berlin Heidelberg, 2003.
- Shah [2022]
Nihar B. Shah.
An overview of challenges, experiments, and computational solutions in peer review.
[https://www.cs.cmu.edu/~nihars/preprints/SurveyPeerReview.pdf](https://www.cs.cmu.edu/~nihars/preprints/SurveyPeerReview.pdf) (Abridged version published in the Communications of the ACM), 2022.
- Shah and Wainwright [2018]
Nihar B Shah and Martin J Wainwright.
Simple, robust and optimal ranking from pairwise comparisons.
*Journal of machine learning research*, 18(199):1–38, 2018.
- Shah et al. [2017]
Nihar B Shah, Sivaraman Balakrishnan, Adityanand Guntuboyina, and Martin J Wainwright.
Stochastically transitive models for pairwise comparisons: Statistical and computational issues.
*IEEE Transactions on Information Theory*, 63(2):934–959, 2017.
- Stafford et al. [2025]
Tom Stafford, Stephen Pinfield, Anna Butters, and Melanie Benson Marshall.
RoRI Insights: Applicants as reviewers - a Guide to Distributed Peer Review.
Technical report, Research on Research Institute (RoRI), 6 2025.
- Thurstone [1927]
Louis L Thurstone.
The method of paired comparisons for social values.
*The Journal of Abnormal and Social Psychology*, 21(4):384, 1927.
- Tideman and Plassmann [2014]
T. Nicolaus Tideman and Florenz Plassmann.
Which voting rule is most likely to choose the “best”’ candidate?
*Public Choice*, 158(3/4):331–357, 2014.
- Tsetlin et al. [2003]
Ilia Tsetlin, Michel Regenwetter, and Bernard Grofman.
The impartial culture maximizes the probability of majority cycles.
*Social Choice and Welfare*, 21(3):387–398, 2003.
- von Luxburg [2010]
Ulrike von Luxburg.
Clustering stability: An overview.
*Found. Trends Mach. Learn.*, 2(3):235–274, March 2010.
- Weber [2020]
Matthias Weber.
Choosing the rules: Preferences over voting systems for assemblies of representatives.
*Journal of Economic Behavior & Organization*, 174:420–434, 2020.
- Xia [2016]
Lirong Xia.
Bayesian estimators as voting rules.
In *Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI)*, 2016.
- Xia and Conitzer [2011]
Lirong Xia and Vincent Conitzer.
A maximum likelihood approach towards aggregating partial orders.
In *Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)*, 2011.
- Xia et al. [2010]
Lirong Xia, Vincent Conitzer, and Jérôme Lang.
Aggregating preferences in multi-issue domains by using maximum likelihood estimators.
In *Autonomous Agents and Multi-Agent Systems*, 2010.
- Young [1975]
H. P. Young.
Social choice scoring functions.
*SIAM Journal on Applied Mathematics*, 28(4):824–838, 1975.
- Young [1995]
Peyton Young.
Optimal voting rules.
*Journal of Economic Perspectives*, 9(1):51–64, 1995.
- Zucchini [2000]
Walter Zucchini.
An introduction to model selection.
*Journal of Mathematical Psychology*, 44(1):41–61, 2000.

## Appendix A Omitted Proofs

In this section, we give the proofs that were omitted in the main body of the paper.

### A.1 Proof of Theorem 1

We first recall our main result from [Section 5.1](https://arxiv.org/html/2508.17177v1#S5.SS1), regarding the consistency axioms for RPRs.
See [1](https://arxiv.org/html/2508.17177v1#Thmthm1)

We prove each claim in the theorem as a seperate proposition.

###### Proposition 10 .

$AbC$ satisfies reversal symmetry.

###### Proof.

Given any positional scoring rule $f_{s}\in F$ and a profile $\bm{\sigma}$, say $f_{s^{\prime}}$ is the reverse rule of $f_{s}$, *i.e.*, $s^{\prime}=(s^{\prime}_{1},s^{\prime}_{2},\ldots,s^{\prime}_{{|A|}})=(1-s_{|A|},1-s_{{|A|}-1},\ldots,1-s_{1})$. The total scores assigned to any alternative $a\in A$ by $f_{s^{\prime}}$ when run of $\textnormal{rev}(\bm{\sigma})$ (see [Definition 1](https://arxiv.org/html/2508.17177v1#Thmprop1)) is

$$ $\displaystyle t_{\textnormal{rev}(\bm{\sigma})}^{s^{\prime}}[a]=\sum_{j=1}^{{|A|}}s^{\prime}_{j}M_{\textnormal{rev}(\bm{\sigma})}[a,j]=\sum_{j=1}^{{|A|}}(1-s_{{|A|}+1-j})M_{\bm{\sigma}}[a,{|A|}+1-j]=\sum_{j=1}^{{|A|}}(1-s_{j})M_{\bm{\sigma}}[a,j]={|N|}-t^{s}_{\bm{\sigma}}[a],$ $$

implying that $f_{s}(\bm{\sigma})=\textnormal{rev}(f_{s^{\prime}}(\textnormal{rev}(\bm{\sigma})))=\textnormal{rev}(\textnormal{rev}(f_{s})(\textnormal{rev}(\bm{\sigma})))$. Additionally, we note that the Kendall-Tau distance ([Definition 3](https://arxiv.org/html/2508.17177v1#Thmprop3)) is symmetric with respect to reversals, *i.e.*, $KT(r_{1},r_{2})=KT(\textnormal{rev}(r_{1}),\textnormal{rev}(r_{2}))$. Thus,

$$ $\displaystyle KT(f_{s}(\bm{\sigma}^{(1)}),f_{s}(\bm{\sigma}^{(2)}))$ $\displaystyle=KT(\textnormal{rev}(\textnormal{rev}(f_{s})(\textnormal{rev}(\bm{\sigma}^{(1)}))),\textnormal{rev}(\textnormal{rev}(f_{s})(\textnormal{rev}(\bm{\sigma}^{(2)}))))$ $\displaystyle=KT(\textnormal{rev}(f_{s})(\textnormal{rev}(\bm{\sigma}^{(1)})),\textnormal{rev}(f_{s})(\textnormal{rev}(\bm{\sigma}^{(2)}))).$ $$

This implies that if $f_{s}$ minimizes ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) in [Section 4](https://arxiv.org/html/2508.17177v1#S4) over random splits of $\bm{\sigma}$, then $\textnormal{rev}(f_{s})$ minimizes it over random splits of $\textnormal{rev}(\bm{\sigma})$, and therefore $AbC(F,\bm{\sigma})=\textnormal{rev}(AbC(F,\textnormal{rev}(\bm{\sigma})))$.
∎

###### Proposition 11 .

Any welfare-maximizing RPR $Z$ fails plurality-shuffling consistency.

###### Proof.

Take any $f_{s}\in F_{S}$. We will show that for any $k\in\mathbb{Z}^{+}$, we have $f_{s}(\pi^{k}(\bm{\sigma},[2,{|A|}]))=f_{p}(\bm{\sigma})$, where $f_{p}$ is plurality. Recall from [Definition 1](https://arxiv.org/html/2508.17177v1#Thmprop1) we have $s_{1}=1$ and $s_{|A|}=0$ WLOG. By [Definitions 1](https://arxiv.org/html/2508.17177v1#Thmprop1) and [6](https://arxiv.org/html/2508.17177v1#Thmprop6), the total score assigned to a given alternative $a\in A$ by $f_{s}$ on input $\pi^{k}(\bm{\sigma},[2,{|A|}])$ (say $t_{k}[a]$) is

$$ $\displaystyle t_{k}[a]$ $\displaystyle=M_{\bm{\sigma}}[a,1]\cdot k\cdot{|A|}!s_{1}+\sum_{i=2}^{{|A|}}M_{\bm{\sigma}}[a,i]\cdot\sum_{j=2}^{{|A|}}\frac{k\cdot{|A|}!}{({|A|}-1)}s_{j}$ $\displaystyle=M_{\bm{\sigma}}[a,1]\cdot k\cdot{|A|}!+\frac{k\cdot{|A|}!}{({|A|}-1)}\left(\sum_{j=2}^{{|A|}-1}s_{j}\right)({|N|}-M_{\bm{\sigma}}[a,1])$ $\displaystyle=\frac{k\cdot{|A|}!}{({|A|}-1)}\left(\sum_{j=2}^{{|A|}-1}s_{j}\right){|N|}+M_{\bm{\sigma}}[a,1]\cdot k\cdot{|A|}!\left(1-\frac{\sum_{j=2}^{{|A|}-1}s_{j}}{{|A|}-1}\right)$ $\displaystyle\equiv C+D\cdot M_{\bm{\sigma}}[a,1],$ $$

where $C$ and $D$ does not depend on $a$. Since $1=s_{1}\geq s_{2}\geq\ldots\geq s_{|A|}=0$, we have $\sum_{j=2}^{{|A|}-1}s_{j}<m-1$ and therefore $D>0$. This implies that for any $a,b\in A$ we have

$$ $\displaystyle a\succ_{f_{s}(\pi^{k}(\bm{\sigma},[{|A|}-1]))}b$ $\displaystyle\Leftrightarrow t_{k}[a]>t_{k}[b]$ $\displaystyle\Leftrightarrow M_{\bm{\sigma}}[a,1]>M_{\bm{\sigma}}[b,1]$ $\displaystyle\Leftrightarrow a\succ_{f_{p}(\bm{\sigma})}b.$ $$

Since $f_{s}$ was arbitrarily chosen, this implies all rules in $F_{S}$ return the same output on $\pi^{k}(\bm{\sigma},[{|A|}-1])$. Hence, for any welfare-maximizing RPR $Z$ and any $F\subseteq F_{S}$, we have $Z(F,\pi^{k}(\bm{\sigma},[{|A|}-1]))=F$, showing that the rule fails [Definition 7](https://arxiv.org/html/2508.17177v1#Thmprop7).
∎

Since all $f_{s}\in F_{S}$ produce the same output on $\pi^{k}(\bm{\sigma},[{|A|}-1])$, one might wonder whether it matters which rule an RPR returns. However, in practice an RPR might be employed to “lock in” a rule for future use, allowing us to use it in future profiles coming from the same source without re-running the RPR at each round, potentially increasing interpretability. Hence, with a plurality-shuffling-consistent rule, we would be able to keep on using $f_{p}$ in future profiles that are approximately (but not exactly) shuffled, as future repetitions of the same process are likely to once again have their signal concentrated in the top position. Moreover, identifying that no $f_{s}\in F_{S}$ provides any information beyond $f_{p}$ has a computational advantage: while the output of $f_{p}$ can be computed in $O(n+m)$ time, an arbitrary $f_{s}\in F_{S}$ might require $O(mn)$ time.

###### Proposition 12 .

$AbC$ satisfies plurality-shuffling consistency.

###### Proof.

Fix some finite $F\subset F_{S}$. If $|F|=1$ we are done. Otherwise, given any profile $\bm{\sigma}$ (that satisfies the condition in [Definition 7](https://arxiv.org/html/2508.17177v1#Thmprop7), *i.e.*, $f_{p}(\bm{\sigma})$ contains no ties) with ${|N|}=n$ voters an ${|A|}=m\geq 3$ alternatives,(^7^77If $m=2$, there is a single (monotonic) positional scoring rule satisfying [Definition 1](https://arxiv.org/html/2508.17177v1#Thmprop1): the one with score vector $(1,0)$, which is also equivalent to $f_{p}$.) define $\bm{\sigma}_{k}=\pi^{k}(\bm{\sigma},[2,m])$ for all $k$. Let $M$ be the matrix such that for each alternative $a\in A$ and index $i\in[m]$, $M[a,i]$ indicates the number of voters in $\bm{\sigma}$ that rank $a$ in the $i^{\text{th}}$ position, and let $M_{k}$ be the analogous matrix for $\bm{\sigma}_{k}$.

For any $f\in F$ and any two candidates $a,b\in A$, say $D^{f,k}_{a,b}$ is the indicator random variable that the two output rankings resulting from applying $f$ to the two sides of a random split of $\bm{\sigma}_{k}$ into $\bm{\sigma}^{(1)}_{k}$ and $\bm{\sigma}^{(2)}_{k}$ (where each voter is independently and uniformly placed in one of the two sets, as defined in [Section 4](https://arxiv.org/html/2508.17177v1#S4)) disagree about $a$ and $b$’s position; *i.e.*,

$$ $D^{f,k}_{a,b}=\mathbb{I}[(a\succ_{f(\bm{\sigma}^{(1)}_{k})}b\text{ and }b\succ_{f(\bm{\sigma}^{(2)}_{k})}a)\text{ or }(b\succ_{f(\bm{\sigma}^{(1)}_{k})}a\text{ and }a\succ_{f(\bm{\sigma}^{(2)}_{k})}b)].$ $$

Similarly, say $T^{f,k}_{a,b}$ is the indicator random variable that $f$ ties $a$ and $b$ on one of the two profiles; *i.e.*,

$$ $T_{r_{1},r_{2}}^{a,b}=\mathbb{I}[(a\succeq_{f(\bm{\sigma}^{(1)}_{k})}b\text{ and }b\succeq_{f(\bm{\sigma}^{(1)}_{k})}a)\text{ or }(a\succeq_{f(\bm{\sigma}^{(2)}_{k})}b\text{ and }b\succeq_{f(\bm{\sigma}^{(2)}_{k})}a)].$ $$

Then, using linearity of expectation and the fundamental bridge, we get

$$ $\displaystyle\mathbb{E}\left[KT(f(\bm{\sigma}_{k}^{(1)}),f(\bm{\sigma}^{(2)}_{k}))\right]=\mathbb{E}\left[\sum_{a,b\in A}D^{f,k}_{a,b}+\frac{1}{2}T^{f,k}_{a,b}\right]=\sum_{a,b\in A}\Pr[D^{f,k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f,k}_{a,b}=1].$ (5) $$

The rest of the proof will follow from the next lemma.

###### Lemma 13 .

For any $f\in F\setminus\{f_{p}\}$ there exists a $k_{f}\in\mathbb{Z}^{+}$ such that

- 1.
for all $k\geq k_{f}$ we have $\Pr[D^{f,k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f,k}_{a,b}=1]>\Pr[D^{f_{p},k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f_{p},k}_{a,b}=1]$ for all $a,b\in A$ such that $M[a,1]>0$ and $M[b,1]>0$;
- 2.
If there exists $c\in A$ such that $M[c,1]=0$, then $\Pr[D^{f,k}_{a,c}=1]+\frac{1}{2}\Pr\left[T^{f,k}_{a,c}=1\right]\geq\Pr[D^{f_{p},k}_{a,c}=1]+\frac{1}{2}\Pr[T^{f_{p},k}_{a,c}=1]$ for all $a\in A$ and $k\in\mathbb{Z}^{+}$.

Since $f_{p}(\bm{\sigma})$ contains no ties, there can be at most one $c\in A$ with $M[c,1]=0$. Since $m\geq 3$, the lemma implies that $\sum_{a,b\in A}\Pr[D^{f,k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f,k}_{a,b}=1]>\sum_{a,b\in A}\Pr[D^{f_{p},k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f_{p},k}_{a,b}=1]$ for all $k\geq k_{f}$. By definition of $AbC$, this implies that for all $k\geq k_{f}$ we have $f\notin AbC(F,\pi^{k}(\bm{\sigma},[m-1]))$, since ([5](https://arxiv.org/html/2508.17177v1#A1.E5)) is strictly greater for $f$ than for $f_{p}$. Since $|F|$ is finite, picking $k=\max_{f\in F\setminus\{f_{p}\}}k_{f}$ will ensure $AbC(F,\pi^{k^{\prime}}(\bm{\sigma},[m-1])=\{f_{p}\}$ for all $k^{\prime}\geq k$, as desired. We will now prove the lemma.

###### Proof of Lemma 13 .

For any $f\in F$, say $\{s^{f}_{i}\}_{i\in[m]}$ is the scoring vector associated with $f$. Fix any two candidates $a,b\in A$. Say $M[a,1]=p_{a}$ and $M[b,1]=p_{b}$, with $p_{a}>p_{b}$ WLOG.(^8^88We cannot have $p_{a}=p_{b}$ as by assumption $f_{p}(\bm{\sigma})$ contains no ties.) By [Definition 6](https://arxiv.org/html/2508.17177v1#Thmprop6), we have $M_{k}[a,1]=kp_{a}m!$ and $M_{k}[b,1]=kp_{b}m!$, whereas $M_{k}[a,i]=\frac{k(n-p_{a})m!}{m-1}$ and $M_{k}[b,i]=\frac{k(n-p_{b})m!}{m-1}$ for all $1<i\leq m$. Similarly, let $M_{k}[(a,i);(b,j)]$ be the number of voters in $\bm{\sigma}_{k}$ that rank $a$ in the $i$th position *and* $b$ in the $j$th position (with $i\neq j$). Then

$$ $\displaystyle M_{k}[(a,i);(b,j)]=\begin{cases}kp_{a}C_{1}&\text{if }i=1\\ kp_{b}C_{1}&\text{if }j=1\\ kC_{2}&\text{otherwise}\end{cases},$ $$

where $C_{1}\equiv\frac{m!}{m-1}$ and $C_{2}\equiv\frac{(n-p_{a}-p_{b})m!}{(m-1)(m-2)}$. Given a random split of $\bm{\sigma}_{k}$ into $\bm{\sigma}_{k}^{(1)}$ and $\bm{\sigma}_{k}^{(2)}$, say $X^{f,k}_{i}$ is the random variable indicating the total score of $a$ minus the total score of $b$ in $\bm{\sigma}^{(i)}_{k}$ according to $f$ for each $i\in\{1,2\}$. Naturally, regardless of the split, $X^{f,k}_{1}+X^{f,k}_{2}$ must add up to the total score of $a$ minus the total score of $b$ in $\bm{\sigma}_{k}$ according to $f$, so

$$ $\displaystyle X^{f,k}_{1}+X^{f,k}_{2}$ $\displaystyle=\sum_{i=1}^{m}s^{f}_{i}M_{k}[a,i]-\sum_{i=1}^{m}s^{f}_{i}M_{k}[b,i]$ $\displaystyle=kp_{a}m!-kp_{b}m!+\left(\frac{k(n-p_{a})m!}{m-1}-\frac{k(n-p_{b})m!}{m-1}\right)\sum_{i=2}^{m}s^{f}_{i}$ $\displaystyle=\frac{\lambda^{f}k(p_{a}-p_{b})m!}{m-1}$ $$

where $\lambda^{f}=(m-1)-\sum_{i=2}^{m}s^{f}_{i}>0$, as $s^{f}_{m}=0$. Then, using symmetry of the two sides and the fact that $p_{a}>p_{b}$, we get

$$ $\displaystyle\Pr[D^{f,k}_{a,b}=1]=\Pr[X^{f,k}_{1}\cdot X^{f,k}_{2}<0]$ $\displaystyle=\Pr\left[X^{f,k}_{1}\cdot\left(\frac{\lambda^{f}k(p_{a}-p_{b})m!}{m-1}-X^{f,k}_{1}\right)<0\right]$ $\displaystyle=\Pr\left[X^{f,k}_{1}<0\right]+\Pr\left[X^{f,k}_{1}>\frac{\lambda^{f}(p_{a}-p_{b})m!}{m-1}\right]$ $\displaystyle=2\Pr[X^{f,k}_{1}<0].$ $$

A similar analysis gives $\Pr[T^{f,k}_{a,b}=1]=2\Pr[X^{f,k}_{1}=0]$. Now, if $a$ and $b$ are ranked $i$th and $j$th in a ranking, respectively, the score assigned to $a$ minus that assigned to $b$ by $f$ for that ranking is $s^{f}_{i}-s^{f}_{j}$. Say $\text{Bin}({z})$ is the fair binomial distribution where $z$ is the number of experiments and the probability of success for each experiment is $1/2$. Then,

$$ $\displaystyle X^{f,k}_{1}\sim\sum_{i\neq j\in[m]}(s^{f}_{i}-s^{f}_{j})\text{Bin}\left(M_{k}[(a,i);(b,j)]\right)$ $\displaystyle=\sum_{i<j\in[m]}(s^{f}_{i}-s^{f}_{j})\left(\text{Bin}\left(M_{k}[(a,i);(b,j)]\right)-\text{Bin}\left(M_{k}[(a,j);(b,i)]\right)\right)$ $\displaystyle=\sum_{i<j\in[m]}(s^{f}_{i}-s^{f}_{j})\left(\text{Bin}\left(M_{k}[(a,i);(b,j)]\right)+\text{Bin}\left(M_{k}[(a,j);(b,i)]\right)-M_{k}[(a,j);(b,i)]\right)$ $\displaystyle=\sum_{i<j\in[m]}(s^{f}_{i}-s^{f}_{j})\left(\text{Bin}\left(M_{k}[(a,i);(b,j)]+M_{k}[(a,j);(b,i)]\right)-M_{k}[(a,j);(b,i)]\right)$ $\displaystyle=\sum_{i<j\in[2,m]}(s^{f}_{i}-s^{f}_{j})\left(\text{Bin}\left(2kC_{2}\right)-kC_{2}\right)+\sum_{i=2}^{m}(1-s^{f}_{i})\left(\text{Bin}\left(k(p_{a}+p_{b})C_{1}\right)-kp_{b}C_{1}\right)$ (6) $$

In particular, for plurality, ([6](https://arxiv.org/html/2508.17177v1#A1.E6)) gives us:

$$ $\displaystyle X^{f_{p},k}_{1}\sim\sum_{i=2}^{m}\left(\text{Bin}\left(k(p_{a}+p_{b})C_{1}\right)-kp_{b}C_{1}\right)=\text{Bin}\left(k(p_{a}+p_{b})m!\right)-kp_{b}m!$ (7) $$

Now, fixing some $f\in F\setminus\{f_{p}\}$, using ([6](https://arxiv.org/html/2508.17177v1#A1.E6)), we write $X^{f,k}_{1}=Y+\sum_{i=2}^{m}(1-s^{f}_{i})Z_{i}$, where

$$ $Y\sim\sum_{i<j\in[2,m]}(s^{f}_{i}-s^{f}_{j})\left(\text{Bin}\left(2kC_{2}\right)-kC_{2}\right)\text{, and }$ $$

$$ $Z_{i}\sim\left(\text{Bin}\left(k(p_{a}+p_{b})C_{1}\right)-kp_{b}C_{1}\right)\text{ for all }i\in[2,m],$ $$

each sampled independently. It is easy to check that $\mathbb{E}{[Y]}=0$ and that $Y$ is a symmetric distribution. We will treat two cases separately.

Case 1: $p_{b}=0$. In this case, we cannot have $X_{1}^{f_{p},k}<0$, as seen by ([7](https://arxiv.org/html/2508.17177v1#A1.E7)). Therefore,

$$ $\displaystyle\Pr[D^{f_{p},k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f_{p},k}_{a,b}=1]=0+\Pr[X^{f_{p},k}_{1}=0]=2^{-kp_{a}m!}.$ (8) $$

For $f$, on the other hand, we have

$$ $\displaystyle\Pr[X_{1}^{f,k}=0]\geq\Pr[Y=0;Z_{i}=0\text{ for all }i\in[2,m]]=\frac{\Pr[Y=0]}{\left(2^{kp_{a}C_{1}}\right)^{m-1}}={\Pr[Y=0]}\cdot{2^{-kp_{a}m!}},$ $\displaystyle\text{ and }\Pr[X_{1}^{f,k}<0]\geq\Pr[Y<0;Z_{i}=0\text{ for all }i\in[2,m]]=\frac{1-\Pr[Y=0]}{2}\cdot 2^{-kp_{a}m!},$ $\displaystyle\Rightarrow\Pr[D^{f_{p},k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f_{p},k}_{a,b}=1]=2\Pr[X_{1}^{f,k}<0]+\Pr[X_{1}^{f,k}=0]\geq 2^{-kp_{a}m!}.$ (9) $$

Since $a\in A$ and $k\in\mathbb{Z}^{+}$ were arbitrarily chosen, combining ([8](https://arxiv.org/html/2508.17177v1#A1.E8)) and ([9](https://arxiv.org/html/2508.17177v1#A1.E9)) gives us statement 2 from the lemma.

Case 2: $p_{b}>0$. Say $\phi(N,r)$ is $\Pr[A\leq rN]$ for $A\sim\text{Bin}\left(N\right)$, given that $0<r<\frac{1}{2}$ and $rN$ is an integer. By Ash [1990, Lemma 4.7.2], we know that

$$ $\displaystyle\frac{2^{-N(1-h(r))}}{\sqrt{8Nr(1-r)}}\leq\phi(N,r)\leq 2^{-N(1-h(r))},$ (10) $$

where $h(r)=-r\log r-(1-r)\log(1-r)$ is the binary entropy function. We will use ([10](https://arxiv.org/html/2508.17177v1#A1.E10)) to upper bound the error probability for $f_{p}$ and lower bound the error probability for $f$. For convenience, define

$$ $\displaystyle r_{p}\stackrel{{\scriptstyle\text{def}}}{{=}}\frac{p_{b}}{p_{a}+p_{b}},\quad h_{p}\stackrel{{\scriptstyle\text{def}}}{{=}}h(r_{p}),\quad N_{p}\stackrel{{\scriptstyle\text{def}}}{{=}}(p_{a}+p_{b})C_{1}.$ $$

First we start with $f_{p}$. By ([7](https://arxiv.org/html/2508.17177v1#A1.E7)) and ([10](https://arxiv.org/html/2508.17177v1#A1.E10)), we have

$$ $\displaystyle\Pr[D^{f_{p},k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f_{p},k}_{a,b}=1]$ $\displaystyle=2\Pr[X_{1}^{f_{p},k}<0]+\Pr[X_{1}^{f_{p},k}=0]$ $\displaystyle\leq 2\Pr[X_{1}^{f_{p},k}\leq 0]=2\phi(k(p_{a}+p_{b})m!,r_{p})$ $\displaystyle\leq 2^{1-kN_{p}(m-1)(1-h_{p})}\equiv U(k),$ $$

where we will use $U(k)$ as an upper bound that depends on $k$. Now consider $f$. Since $f\neq f_{p}$ and since $1=s^{f}_{1}\geq s^{f}_{2}\geq\ldots s^{f}_{m}=0$, we must have $s^{f}_{2}>0$. Fix some $\varepsilon>0$ such that $\varepsilon<\min\left(\frac{p_{a}-p_{b}}{2(p_{a}+p_{b})},\frac{p_{b}}{(p_{a}+p_{b})(1-s^{f}_{2})}\right)$ (if $s^{f}_{2}=1$, then ensuring $\varepsilon<\frac{p_{a}-p_{b}}{2(p_{a}+p_{b})}$ is sufficient). Recalling that $X^{f,k}_{1}=Y+\sum_{i=2}^{m}(1-s^{f}_{i})Z_{i}$ and that $s^{f}_{m}=0$, we have

$$ $\displaystyle\Pr[X_{1}^{f,k}\leq 0]$ $\displaystyle\geq\Pr[Y\leq 0]\cdot\Pr[Z_{2}\leq\varepsilon kN_{p}]\cdot\Pr[Z_{m}\leq-\varepsilon kN_{p}(1-s^{f}_{2})]\cdot\prod_{i=3}^{m-1}\Pr[Z_{i}\leq 0],$ $\displaystyle\Pr[X_{1}^{f,k}<0]$ $\displaystyle\geq\Pr[Y<0]\cdot\Pr[Z_{2}\leq\varepsilon kN_{p}]\cdot\Pr[Z_{m}\leq-\varepsilon kN_{p}(1-s^{f}_{2})]\cdot\prod_{i=3}^{m-1}\Pr[Z_{i}\leq 0]$ $\displaystyle=\Pr[Y>0]\cdot\Pr[Z_{2}\leq\varepsilon kN_{p}]\cdot\Pr[Z_{m}\leq-\varepsilon kN_{p}(1-s^{f}_{2})]\cdot\prod_{i=3}^{m-1}\Pr[Z_{i}\leq 0];$ $$

therefore,

$$ $\displaystyle\Pr[D^{f,k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f,k}_{a,b}=1]$ $\displaystyle=2\Pr[X_{1}^{f,k}<0]+\Pr[X_{1}^{f,k}=0]$ $\displaystyle=\Pr[X_{1}^{f,k}<0]+\Pr[X_{1}^{f,k}\leq 0]$ $\displaystyle\geq\Pr[Z_{2}\leq\varepsilon kN_{p}]\cdot\Pr[Z_{m}\leq-\varepsilon kN_{p}(1-s^{f}_{2})]\cdot\prod_{i=3}^{m-1}\Pr[Z_{i}\leq 0]$ $\displaystyle\geq\Pr[Z_{2}\leq\lfloor\varepsilon kN_{p}\rfloor]\cdot\Pr[Z_{m}\leq-\lceil\varepsilon kN_{p}(1-s^{f}_{2})\rceil]\cdot\prod_{i=3}^{m-1}\Pr[Z_{i}\leq 0]$ $\displaystyle\geq\phi(kN_{p},r_{2}(k))\cdot\phi(kN_{p},r_{m}(k))\cdot(\phi(kN_{p},r_{p}))^{m-3}$ $$

where $r_{2}(k)=r_{p}+\frac{\lfloor\varepsilon kN_{p}\rfloor}{kN_{p}}$ and $r_{m}(k)=r_{p}-\frac{\lceil\varepsilon kN_{p}(1-s^{f}_{2})\rceil}{kN_{p}}$. As $k\rightarrow\infty$, we have $r_{2}(k)\rightarrow r_{p}+\varepsilon\in\left(0,\frac{1}{2}\right)$ and $r_{m}(k)\rightarrow r_{p}-\varepsilon(1-s^{f}_{2})\in\left(0,\frac{1}{2}\right)$; hence, we can choose $k$ large enough such that $0<r_{i}<1/2$ for both $i\in\{2,m\}$. Further, by construction, $kN_{p}\cdot r_{i}(k)$ is an integer for all $k\geq 0$ and $i\in\{2,m\}$. Define $h_{i}(k)\stackrel{{\scriptstyle\text{def}}}{{=}}h_{i}(r_{i}(k))$ for $i\in\{2,m\}$. By ([10](https://arxiv.org/html/2508.17177v1#A1.E10)), we then have

$$ $\displaystyle\Pr[D^{f,k}_{a,b}=1]+\frac{1}{2}$ $\displaystyle\Pr[T^{f,k}_{a,b}=1]\geq\phi(kN_{p},r_{2}(k))\cdot\phi(kN_{p},r_{m}(k))\cdot(\phi(kN_{p},r_{p}))^{m-3}$ $\displaystyle\geq\frac{2^{-kN_{p}(1-h_{2}(k))}}{\sqrt{8kN_{p}r_{2}(k)(1-r_{2}(k))}}\cdot\frac{2^{-kN_{p}(1-h_{m}(k))}}{\sqrt{8kN_{p}r_{m}(k)(1-r_{m}(k))}}\cdot\left(\frac{2^{-kN_{p}(1-h_{p})}}{\sqrt{8kN_{p}r_{p}(1-r_{p})}}\right)^{m-3}$ $\displaystyle=\frac{2^{-kN_{p}(m-1-h_{2}(k)-h_{m}(k)-(m-3)h_{p})}}{\sqrt{(8kN_{p})^{m-1}r_{2}(k)(1-r_{2}(k))r_{m}(k)(1-r_{m}(k))(r_{p}(1-r_{p}))^{m-3}}}\equiv L(k),$ $$

where we will use $L(k)$ as a lower bound that depends on $k$. Now, we would like to show that there exists $k_{f}^{a,b}$ such that $L(k)>U(k)$ for all $k\geq k_{f}^{a,b}$. We have

$$ $\displaystyle\frac{L(k)}{U(k)}=\frac{2^{kN_{p}P(k)-1}}{\sqrt{(8kN_{p})^{m-1}r_{2}(k)(1-r_{2}(k))r_{m}(k)(1-r_{m}(k))(r_{p}(1-r_{p}))^{m-3}}},$ (11) $$

where $P(k)=(m-1)(1-h_{p})-(m-1-h_{2}(k)-h_{m}(k)-(m-3)h_{p})=h_{2}(k)+h_{m}(k)-2h_{p}$. We would like to show $\lim_{k\rightarrow\infty}\frac{L(k)}{U(k)}=\infty$, giving us our desired relationship. We recall that $r_{2}(k)\rightarrow r_{p}+\varepsilon$ and $r_{2}(k)\rightarrow r_{p}-\varepsilon(1-s^{f}_{2})$ as $k\rightarrow\infty$, so the denominator of ([11](https://arxiv.org/html/2508.17177v1#A1.E11)) scales as $\Theta(k^{\frac{m-1}{2}})$ for large $k$. As for the nominator, we have $\lim_{k\rightarrow\infty}P(k)=h(r_{p}+\varepsilon)+h(r_{p}-\varepsilon(1-s^{f}_{2}))-2h(r_{p})\equiv H(\varepsilon)$. We have $H(0)=0$ and $H^{\prime}(0)=h^{\prime}(r_{p})-(1-s^{f}_{2})h^{\prime}(r_{p})=s^{f}_{2}h^{\prime}(r_{p})>0$ since $s^{f}_{2}>0$ and

$$ $\displaystyle h^{\prime}(r_{p})=-\log r_{p}-1+\log(1-r_{p})+1=\log\left(\frac{1}{r_{p}}-1\right)>\log(2-1)=0,$ $$

as $r_{p}=\frac{p_{b}}{p_{a}+p_{b}}<\frac{1}{2}$. We can therefore set $\varepsilon>0$ small enough such that $H(\varepsilon)>0$. Then the nominator of ([11](https://arxiv.org/html/2508.17177v1#A1.E11)) scales as $\Theta(2^{kN_{p}H(\varepsilon)})$ for large $k$, dominating the denominator. Hence, $\lim_{k\rightarrow\infty}\frac{L(k)}{U(k)}=\infty$, as desired. Therefore, we can choose a large enough $k_{f}^{a,b}$ such that for all $k\geq k_{f}^{a,b}$ we have

$$ $\displaystyle\Pr[D^{f,k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f,k}_{a,b}=1]\geq L(k)>U(k)\geq\Pr[D^{f_{p},k}_{a,b}=1]+\frac{1}{2}\Pr[T^{f_{p},k}_{a,b}=1].$ $$

Since there are finitely many candidates $A$, picking $k_{f}=\max_{(a,b)\in A^{2}}k_{f}^{a,b}$ is sufficient for proving statement 1 from the lemma.
∎

∎

###### Proposition 14 .

No (anonymous) RPR can satisfy all three of reversal symmetry, plurality-shuffling consistency (PSC), and union consistency (UC).

###### Proof.

Assume an RPR $Z$ satisfies PSC and reversal symmetry. We will show that $Z$ necessarily fails UC. Fix $A=\{a,b,c\}$ and $F=\{f_{p},f_{v}\}$. Say $\bm{\sigma}_{a}$ consists of two voters ranking $a\succ b\succ c$ and one voter ranking $b\succ a\succ c$. Similarly, say $\bm{\sigma}_{b}$ consists of two voters ranking $c\succ b\succ a$ and one voter ranking $b\succ c\succ a$. We have $f_{p}(\bm{\sigma}_{a})=a\succ b\succ c$ and $f_{p}(\bm{\sigma}_{b})=c\succ b\succ a$, so both profiles satisfy the preconditions of [Definition 7](https://arxiv.org/html/2508.17177v1#Thmprop7). By PSC, there exist $k_{a},k_{b}$ such that for all $k^{\prime}_{a}\geq k_{a}$ and $k^{\prime}_{b}\geq k_{b}$ we have $Z(F,\pi^{k^{\prime}_{a}}(\bm{\sigma}_{a},[2,m]))=Z(F,\pi^{k^{\prime}_{b}}(\bm{\sigma}_{b},[2,m]))=\{f_{p}\}$. Pick $k=\max\{k_{a},k_{b}\}$ and let $\bm{\sigma}=\pi^{k}(\bm{\sigma}_{a},[2,m])+\pi^{k}(\bm{\sigma}_{b},[2,m])$. It is straightforward to check that $\bm{\sigma}=\textnormal{rev}(\bm{\sigma})$ up to a permutation of voters, with each $r\in\mathcal{L}(A)$ appearing $6k$ times. By anonymity and reversal symmetry, this implies $Z(F,\bm{\sigma})=\textnormal{rev}(Z(F,\bm{\sigma}))$, which is only possible if $Z(F,\bm{\sigma})=\{f_{v},f_{p}\}$, proving UC is violated.
∎

###### Proposition 10 .

###### Proof.

###### Proposition 11 .

###### Proof.

###### Proposition 12 .

###### Proof.

###### Lemma 13 .

###### Proof of Lemma 13 .

###### Proposition 14 .

###### Proof.

### A.2 Proof of Theorem 2

See [2](https://arxiv.org/html/2508.17177v1#Thmthm2)
We prove each claim as a separate proposition.

###### Proposition 15 .

$AbC$ preserves anonymity & neutrality.

###### Proof.

The proof follows definitionally from the anonymity and neutrality of the Kendall-Tau distance ([Definition 3](https://arxiv.org/html/2508.17177v1#Thmprop3)), and therefore of the $AbC$ function ([Section 4](https://arxiv.org/html/2508.17177v1#S4)).
∎

Next, we formally define the social choice axioms given in part (2) of [Theorem 2](https://arxiv.org/html/2508.17177v1#Thmthm2). Given a profile $\bm{\sigma}$ and $a,b\in A$, we say $a$ pairwise defeats $b$ if $|\{i\in N:a\succ_{\sigma_{i}}b\}|>|\{i\in N:b\succ_{\sigma_{i}}a\}|$. Then, we say an SWF $f\in F$ satisfies…

- •
…the *Smith Criterion (SC)* if the alternative(s) ranked highest in $f(\bm{\sigma})$ belongs to the Smith set of $\bm{\sigma}$, *i.e.*, the smallest set $S\subseteq A$ such that every $a\in S$ pairwise defeats every $b\in A\setminus S$.
- •
…*Condorcet Consistency (CC)* if it satisfies the SC for all profiles $\bm{\sigma}$ that have a Smith set containing a single alternative (called the Condorcet winner).
- •
…*Majority Winner (MW)* if is satisfies CC whenever the Condorcet winner is the top-ranked candidate of a majority ($>{|A|}$) of voters.
- •
…*Pairwise Majority Consistency (PMC)* if whenever there exists $r\in\mathcal{R}(A)$ such that $a\succ_{r}b$ if and only if $a$ pairwise defeats $b$ in $\bm{\sigma}$, then $f(\bm{\sigma})=r$. [Ge et al., 2024]
- •
…*unanimity* if whenever $\sigma_{i}=r$ for all $i\in N$, then $f(\bm{\sigma})=r$.

###### Proposition 16 .

Any RPR $Z$ preserves the Smith criterion, Condorcet consistency, majority winner, pairwise majority consistency, and unanimity.

###### Proof.

Like [Proposition 16](https://arxiv.org/html/2508.17177v1#Thmprop16), the proof follows from the definitions of the axioms. Each of them are of the form “if $\bm{\sigma}$ satisfies conditions $X$, then $f(\bm{\sigma})$ must satisfy conditions $Y$.” Since for each $\bm{\sigma}$ we have $f_{Z}^{F}(\bm{\sigma})=f(\bm{\sigma})$ for some $f\in F$, restricting $F$ to rules that satisfy this axiom will ensure $f_{Z}^{F}(\bm{\sigma})$ will satisfy $Y$ whenever $\bm{\sigma}$ satisfies $X$.
∎

Before proving the general impossibility result in part (3) of [Theorem 2](https://arxiv.org/html/2508.17177v1#Thmthm2), we will show that $AbC$ does not preserve monotonicity.

###### Example 17 .

Say $F=\{f_{p},f_{v}\}$ (plurality and veto). Both of these rules are monotonic, as all positional scoring rules are. Fix $k\in\mathbb{Z}^{+}$ and consider the following profile $\bm{\sigma}$:

- •
Group 1: $k$ voters rank $a\succ b\succ c\succ d$.
- •
Group 2: $k$ people voted $d\succ c\succ a\succ b$.
- •
Group 3: $k$ people voted $d\succ c\succ a\succ b$.
- •
Group 4: $k$ voters rank $b\succ a\succ c\succ d$.
- •
Group 5: $3k$ voters rank $c\succ a\succ b\succ d$.
- •
Group 6: $3k$ voters rank $d\succ b\succ a\succ c$.

The veto scores (the number of voters that rank them bottom) of $a,b,c,d$ are $0,2k,3k,5k$, respectively. By an analogous argument to that in [4](https://arxiv.org/html/2508.17177v1#Thmprop4), $\Pr[f_{v}(\bm{\sigma}^{(1)})=f_{v}(\bm{\sigma}^{(2)})=(a\succ b\succ c\succ d)]\rightarrow 1$ as $k\rightarrow\infty$, where the probabilities are taken over the random process in [Section 4](https://arxiv.org/html/2508.17177v1#S4). Hence, the expectation in Equation ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) of [Section 4](https://arxiv.org/html/2508.17177v1#S4) approaches 0 for $f_{v}$ as $k$ grows. However, the plurality scores (the number of voters that rank them top) of $a$ and $b$ are tied, implying the the probability that $f_{p}(\bm{\sigma}^{(1)})$ and $f_{p}(\bm{\sigma}^{(2)})$ will disagree about $a$ and $b$ converges to 1 as $k\rightarrow\infty$. This implies the expectation in ([2](https://arxiv.org/html/2508.17177v1#S4.E2)) approaches at least 1 for $f_{p}$ as $k$ grows. Therefore, there exists a $k_{1}$ such that $AbC(F,\bm{\sigma})=\{f_{v}\}$ for all $k\geq k_{1}$, and therefore $f_{AbC}^{F}(\bm{\sigma})=(a\succ b\succ c\succ d)$.
Now say the voters in group 1 and 2 promote $b$ by a single spot, to get the profile $\bm{\sigma}^{\prime}$:

- •
Group 1: $k$ voters rank $b\succ a\succ c\succ d$.
- •
Group 2: $k$ people voted $d\succ c\succ b\succ a$.
- •
Group 3: $k$ people voted $d\succ c\succ a\succ b$.
- •
Group 4: $k$ voters rank $b\succ a\succ c\succ d$.
- •
Group 5: $3k$ voters rank $c\succ a\succ b\succ d$.
- •
Group 6: $3k$ voters rank $d\succ b\succ a\succ c$.

Importantly, we have $\bm{\sigma}^{\prime}=\textnormal{rev}(\bm{\sigma})$ up to permuting Groups 1 with 2, 3 with 4, and 5 with 6. By anonymity and reversal symmetry, this implies $AbC(F,\bm{\sigma}^{\prime})=\{f_{p}\}$, so $f_{AbC}^{F}(\bm{\sigma}^{\prime})=(d\succ c\succ b\succ a)$. Thus, by promoting $b$, we have gotten $\rank_{f_{AbC}^{F}(\bm{\sigma})}(b)=2<3=\rank_{f_{AbC}^{F}(\bm{\sigma}^{\prime})}(b)$, which violates monotonicity.

We now show how the same example can be used for other (anonymous) RPRs satisfying reversal symmetry, in order to prove part (3) of [Theorem 2](https://arxiv.org/html/2508.17177v1#Thmthm2).

###### Proposition 18 .

No (anonymous) RPR can satisfy reversal symmetry and preserve monotonicity.

###### Proof.

Say RPR $Z$ satisfies reversal symmetry and is anonymous. Take $\bm{\sigma},\bm{\sigma}^{\prime}$ and $F$ from [17](https://arxiv.org/html/2508.17177v1#Thmprop17). We consider two cases.
Case 1: $Z(F,\bm{\sigma})=\{f_{v}\}$. Then, by the same reasoning as in the proof of [17](https://arxiv.org/html/2508.17177v1#Thmprop17), we must have $Z(F,\bm{\sigma}^{\prime})=\{f_{p}\}$ (as that example only uses anonymity and reversal symmetry to show this), and monotonicity is violated by $f^{F}_{Z}$.
Case 2: $Z(F,\bm{\sigma})=\{f_{p}\}$. In that case, $f_{Z}^{F}(\bm{\sigma})=(d\succ c\succ a=b)$, *i.e.*, $a$ and $b$ are tied in the bottom. By reversal symmetry, we have $Z(F,\bm{\sigma}^{\prime})=\{f_{v}\}$ and therefore $f_{Z}^{F}(\bm{\sigma}^{\prime})=(a=b\succ c\succ d)$. However, one can go from $\bm{\sigma}^{\prime}$ to $\bm{\sigma}$ by simply promoting $a$ by a spot in the rankings of groups 1 and 2. This implies that $\rank_{f_{Z}^{F}(\bm{\sigma}^{\prime})}(a)=1<3=\rank_{f_{Z}^{F}(\bm{\sigma})}(a)$, so promoting $a$ results in increasing its rank, therefore $f_{Z}^{F}$ is not monotonic.

In either case, we see that $f_{Z}^{F}$ violates monotonicity. Since $f_{p}$ and $f_{v}$ are both monotonic, this proves that $Z$ does not preserve monotonicity.
∎

###### Proposition 15 .

###### Proof.

###### Proposition 16 .

###### Proof.

###### Example 17 .

###### Proposition 18 .

###### Proof.

### A.3 Proof of Theorem 3

We recall the complexity result from [Section 6](https://arxiv.org/html/2508.17177v1#S6).

See [3](https://arxiv.org/html/2508.17177v1#Thmthm3)

We first introduce a generalization of the computational problem PerfPos, which we will call $k\-\textsc{PerfPos}$: For each voter $i\in N$, we are given a strict ranking $\sigma_{i}\in\mathcal{L}(A_{i})$ over a *subset* of alternatives $A_{i}\subseteq A$, with $|A_{i}|=k$. We are also given a split of voters $N=N_{1}\sqcup N_{2}$ with $|N_{1}|=|N_{2}|$. For each $j\in\{1,2\},a\in A$, and $i\in[k]$, $M_{j}[a,i]$ indicates the number of voters in $N_{j}$ that rank $a$ in their $i^{\text{th}}$ position. Then, PerfPos asks: is there a positional scoring rule $f_{s}\in F_{S}$ that achieves zero disagreement over this split, i.e., is there a vector ${s}=(s_{i})_{i\in[k]}$ with $1=s_{1}\geq s_{2}\geq\ldots\geq s_{k}=0$ such that for all $a,b\in A$

$$ $\displaystyle(T_{1}[a]-T_{1}[b])(T_{2}[a]-T_{2}[b])>0,$ $$

where $T_{j}[a]=\sum_{i=1}^{k}M_{j}[a,i]s_{i}$ for any $a\in A$ and $j\in\{1,2\}$.

Clearly $k\-\textsc{PerfPos}$ contains PerfPos (for $k={|A|}$). However, as we show next, it is not harder: Given an instance of $k\-\textsc{PerfPos}$ (with input profile $\bm{\sigma}$), complete the ranking of each voter $i\in N$ to a strict ranking over all alternatives by placing the remaining ${|A|}-k$ alternatives in $A\setminus A_{i}$ at the bottom of $\sigma_{i}$, giving rise to a new (complete) profile $\bm{\sigma}^{\prime}$. Then define $\bm{\sigma}^{\prime\prime}=\pi^{1}(\bm{\sigma}^{\prime},[k,{|A|}])$, *i.e.*, the $1-$shuffling of $\bm{\sigma}^{\prime}$ with respect to positions $[k,{|A|}]=\{k,k+1,\ldots,{|A|}\}$ ([Definition 6](https://arxiv.org/html/2508.17177v1#Thmprop6)). Then, we claim the answer to the original $k\-\textsc{PerfPos}$ instance with profile $\bm{\sigma}$ is a yes if and only if the answer to the PerfPos instance using $\bm{\sigma}^{\prime\prime}$ (and the same split as $\bm{\sigma}$) is a yes. Indeed, if there exists a positional scoring rule $f_{s}$ with $1=s_{1}\geq s_{2}\geq\ldots\geq s_{k}=0$ that achieves zero disagreement with $\bm{\sigma}$, then $f_{s^{\prime\prime}}$ with $s^{\prime\prime}=(s_{i})_{i\in[{|A|}]}$ defined as $s^{\prime\prime}_{i}=\begin{cases}s_{i}&\text{if }i\leq k\\
0&\text{otherwise}\end{cases}$
achieves zero disagreement with $\bm{\sigma}^{\prime\prime}$. On the other hand, given a positional scoring rule $f_{s^{\prime\prime}}$ with $1=s^{\prime\prime}_{1}\geq s^{\prime\prime}_{2}\geq\ldots\geq s^{\prime\prime}_{|A|}=0$ that achieves zero disagreement with $\bm{\sigma}^{\prime\prime}$, define $x=\frac{s^{\prime\prime}_{k}+s^{\prime\prime}_{k+1}+\ldots+s^{\prime\prime}_{|A|}}{{|A|}-k+1}$ and define vector $s=(s_{i})_{i\in[k]}$ as $s_{i}=(s^{\prime\prime}_{i}-x)/(1-x)$ for all $i\in[k-1]$ and $s_{k}=0$ (We have $x<1$ since $s^{\prime\prime}_{|A|}=0$). We will show $f_{s}$ achieves zero disagreement with $\bm{\sigma}$. Given any $a\in A$ and $j\in\{1,2\}$, the total score assigned by $f_{s^{\prime\prime}}$ to $a$ on input $\bm{\sigma}^{\prime\prime(j)}$ (restriction of $\bm{\sigma}^{\prime\prime}$ to voters in $N_{j}$) is

$$ $\displaystyle T^{\prime\prime}_{j}[a]=\sum_{i=1}^{k-1}k!M_{j}[a,i]s^{\prime\prime}_{i}+\sum_{i=k}^{{|A|}}k!\frac{|N_{j}|-\sum_{i^{\prime}=1}^{k-1}M_{j}[a,i^{\prime}]}{{|A|}-k+1}s^{\prime\prime}_{i}=k!\left(x|N_{j}|+\sum_{i=1}^{k-1}M_{j}[a,i](s^{\prime\prime}_{i}-x)\right)$ $$

by [Definition 6](https://arxiv.org/html/2508.17177v1#Thmprop6). The score assigned by $f_{s}$ to $a$ on input $\bm{\sigma}^{(j)}$ (restriction of $\bm{\sigma}$ to voters in $N_{j}$), on the other hand, is

$$ $\displaystyle T_{j}[a]=\sum_{i=1}^{k}M_{j}[a,i]s_{i}=\sum_{i=1}^{k-1}M_{j}[a,i]\frac{s^{\prime\prime}_{i}-x}{1-x}=\frac{T^{\prime\prime}_{j}[a]-k!x|N_{j}|}{k!(1-x)}.$ $$

Since the total score of every alternative is just shifted by a constant and then rescaled by another constant, for any $a,b\in A$ we have $T_{j}^{\prime\prime}[a]>T_{j}^{\prime\prime}[b]$ if and only if $T_{j}[a]>T_{j}[b]$. As $f_{s^{\prime\prime}}$ achieves zero disagreement with $\bm{\sigma}^{\prime\prime}$, this proves $f_{s}$ achieves zero disagreement with $\bm{\sigma}$. Thus, in the proof of [Proposition 19](https://arxiv.org/html/2508.17177v1#Thmprop19) below, we reduce 3SAT to $k\-\textsc{PerfPos}$, which proves the NP-hardness for PerfPos too (membership follows from the fact that positional scoring rules are easy to compute).

###### Proposition 19 .

$k\-\textsc{PerfPos}$ is ${\mathsf{NP}}$-hard.

###### Proof.

We will be reducing from 3SAT. Say $\phi$ is a 3CNF formula with clauses $C_{1},C_{2},\ldots,C_{\ell}$ and binary variables $x_{1},x_{2},\ldots,x_{t}$. For each $\beta\in\{1,2\}$, we will construct and instance of PerfPos by first specifying $M_{\beta}[a,i]$ for alternative $a\in A$ and position $i\in[k]$, and then explicitly designing rankings that is consistent with that $M_{\beta}$. We start with setting $k=t+2$ and $\varepsilon=\frac{1}{7(k+2)}$.

For each $i\in[t]$, add two candidates $a_{i}$ and $b_{i}$, with

$$ $\displaystyle M_{1}[a_{i},j]$ $\displaystyle=\begin{cases}1+(k+3)(i-1)&\text{if }j=1\\ k+2&\text{if }j=i+1\\ 0&\text{otherwise}\end{cases}$ $\displaystyle M_{1}[b_{i},j]$ $\displaystyle=\begin{cases}(k+3)(i-1)&\text{if }j=1\text{ and }i\neq 1\\ k+2&\text{if }j=i\\ 1&\text{if }j=k\\ 0&\text{otherwise}\end{cases}$ $$

and

$$ $\displaystyle M_{2}[a_{i},j]$ $\displaystyle=\begin{cases}1+(1/\varepsilon+1)(i-1)&\text{if }j=1\\ 1/\varepsilon&\text{if }j=i+1\\ 0&\text{otherwise}\end{cases}$ $\displaystyle M_{2}[b_{i},j]$ $\displaystyle=\begin{cases}(1/\varepsilon+1)(i-1)&\text{if }j=1\text{ and }i\neq 1\\ 1/\varepsilon&\text{if }j=i\\ 1&\text{if }j=k\\ 0&\text{otherwise}\end{cases}.$ $$

For each $i\in[\ell]$, say the clause $C_{i}$ in $\phi$ consist of variables $\{x_{j}\}_{j\in V_{i}}$ for $V_{i}\subseteq[t]$, with $1\leq|V_{i}|\leq 3$. For this clause $C_{i}$, add two candidates $c_{j},d_{j}$, with

$$ $\displaystyle M_{1}[c_{i},j]=\begin{cases}t(k+3)+2i&\text{if }j=1\\ 0&\text{otherwise}\end{cases}\quad\quad\quad M_{1}[d_{i},j]=\begin{cases}t(k+3)+2i-1&\text{if }j=1\\ 1&\text{if }j=k\\ 0&\text{otherwise}\end{cases}.$ $$

For constructing $M_{2}[c_{i},j]$ and $M_{2}[d_{i},j]$, say $z\in\{0,1,2,3\}$ is the number of negated literals in $C_{i}$. To build $M_{2}[c_{i},j]$ and $M_{2}[d_{i},j]$ for each $j\in[k]$, start from the following:

$$ $\displaystyle M_{2}^{\prime}[c_{i},j]$ $\displaystyle=\begin{cases}2z+t(1/\varepsilon+1)+6(k+3)(i-1)&\text{if }j=1\\ 1&\text{if }j=k\\ 0&\text{otherwise}\end{cases}$ $\displaystyle M_{2}^{\prime}[d_{i},j]$ $\displaystyle=\begin{cases}1+t(1/\varepsilon+1)+6(k+3)(i-1)&\text{if }j=1\\ 2z&\text{if }j=k\\ 0&\text{otherwise}\end{cases}.$ $$

Now for each $j\in V_{i}$, do the following:

- •
If $x_{j}$ appears non-negated in $C_{i}$, then add $2(k+2)$ to $M_{2}^{\prime}[c_{i},j]$ and $2(k+2)$ to $M_{2}^{\prime}[d_{i},j+1]$.
- •
If $x_{j}$ appears negated in $C_{i}$, then add $2(k+2)$ to $M_{2}^{\prime}[c_{i},j+1]$ and $2(k+2)$ to $M_{2}^{\prime}[d_{i},j]$.

Finally, set $M_{2}[c_{i},j]$ and $M_{2}[d_{i},j]$ to the resulting $M_{2}^{\prime}[c_{i},j]$ and $M_{2}^{\prime}[d_{i},j]$ for each $j\in[k]$.

Say $A=\{a_{i}\}_{i\in[t]},B=\{b_{i}\}_{i\in[t]},C=\{c_{i}\}_{i\in[\ell]},$ and $D=\{d_{i}\}_{i\in[\ell]}$. We now construct a profile that corresponds to the above $M_{\beta}$. First, add $k$ more candidates $E=\{e_{i}\}_{i\in[k]}$. Now for each $\beta\in\{1,2\}$, each $f\in A\sqcup B\sqcup C\sqcup D$ and each $i\in[k]$, add $M_{\beta}[f,i]$ voters to the set $N_{\beta}$ that rank $f$ in the $i^{\text{th}}$ position, and ranks $e_{j}$ in the $j^{\text{th}}$ position for all $j\in[k]\setminus\{i\}$. Say we have added $n_{1}$ and $n_{2}$ voters to $N_{1}$ and $N_{2}$ so far respectively and that $\beta^{\prime}=\operatorname*{argmax}_{\beta\in\{1,2\}}n_{\beta}$ (If it’s a tie, pick $\beta^{\prime}$ it arbitrarily). Add $(14k+28)n_{\beta^{\prime}}$ and $(14k+29)n_{\beta^{\prime}}-n_{3-\beta^{\prime}}$ voters to $N_{\beta^{\prime}}$ and $N_{3-\beta^{\prime}}$, respectively, all of whom rank $e_{i}$ in the $i^{\text{th}}$ position for all $i\in[k]$. This also ensures that $|N_{1}|=|N_{2}|$. For each $\beta\in\{1,2\}$ say $\bm{\sigma}^{(\beta)}$ is the final vector of rankings of $N_{\beta}$, as specified.

By construction, for each $\beta\in\{1,2\}$, $f\in A\sqcup B\sqcup C\sqcup D$, and $i\in[k]$, $\bm{\sigma}^{(\beta)}$ indeed has $M_{\beta}[f,i]$ voters that rank $f$ in their $i^{\text{th}}$ position. Further, we have a total of $|A\sqcup B\sqcup C\sqcup D\sqcup E|=2(t+\ell)+k=3t+2\ell+2$ candidates (which polynomial in $t,\ell$), and $|N_{1}|=|N_{2}|=(14k+29)\max_{\beta\in\{1,2\}}\sum_{f\in A\cup B\cup C\cup D}\sum_{i=1}^{k}M_{\beta}[f,i]\leq(14k+29)2(t+\ell)k\cdot\max_{{f\in A\cup B\cup C\cup D},i\in[k]}M_{\beta}[f,i]$ (which is also polynomial in $t,\ell$ since all entries of $M_{\beta}[f,i]$ are).

We now claim that $\phi$ is satisfiable if and only if there exists a positional scoring rule that gives full agreement between $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$.

$(\Leftarrow):$ Assume there is a positional scoring rule $f_{s}$ with $s=(s_{i})_{i\in k}$ that gives full agreement for $M_{1}$ and $M_{2}$. Say $T_{\beta}[a]=\sum_{i=1}^{k}M_{\beta}[a,i]s_{i}$ for all $a\in A$ and $\beta\in\{1,2\}$. In particular, we must have agreement between $a_{i}$ and $b_{i}$ for each $i\in[t]$. We have

$$ $\displaystyle T_{1}[a_{i}]=(1+(k+3)(i-1))s_{1}+(k+2)s_{i+1},$ $\displaystyle\quad T_{1}[b_{i}]=((k+3)(i-1))s_{1}+(k+2)s_{i}+s_{k},$ $\displaystyle T_{2}[a_{i}]=(1+(1/\varepsilon+1)(i-1))s_{1}+s_{i+1}/\varepsilon,$ $\displaystyle\quad T_{2}[b_{i}]=((1/\varepsilon+1)(i-1))s_{1}+s_{i}/\varepsilon+s_{k}.$ $$

Since $s_{1}=1$ and $s_{|A|}=0$, perfect agreement implies that we must have

$$ $\displaystyle(T_{1}[a_{i}]-T_{1}[b_{i}])(T_{2}[a_{i}]-T_{2}[b_{i}])=(1-(k+2)(s_{i}-s_{i+1}))(1-(s_{i}-s_{i+1})/\varepsilon)>0.$ $$

This implies we must either have $s_{i}-s_{i+1}<\varepsilon$ or $s_{i}-s_{i+1}>\frac{1}{k+2}$. Set the binary variable $x_{i}$ to False if $s_{i}-s_{i+1}<\varepsilon$ and to True if $s_{i}-s_{i+1}>\frac{1}{k+2}$. We now argue that the resulting $\{x_{i}\}_{i\in[t]}$ satisfies $\phi$, *i.e.*, satisfies all of its clauses. Fix any $i\in[\ell]$. We will show that $C_{i}$ is satisfied. By assumption of full agreement,

$$ $\displaystyle(T_{1}[c_{i}]-T_{1}[d_{i}])(T_{2}[c_{i}]-T_{2}[d_{i}])>0.$ $$

Since $T_{1}[c_{i}]-T_{1}[d_{i}]=(t(k+3)+2i)s_{1}-(t(k+3)+2i-1)s_{1}-s_{k}=1>0$, this implies that $(T_{2}[c_{i}]-T_{2}[d_{i}])>0$. Say $\{x_{j}\}_{j\in V_{i}}$ are the variables that appear in $C_{i}$ (for $V_{i}\subseteq[t]$), and that $Z=\{j\in V_{i}:x_{j}\text{ is negated in }C_{i}\}$ with $|Z|=z$. This implies we have

$$ $\displaystyle T_{2}[c_{i}]=(2z+t(1/\varepsilon+1)+6(k+3)(i-1))s_{1}+s_{k}+2(k+2)\left(\sum_{g\in V_{i}\setminus Z}s_{g}+\sum_{g\in Z}s_{g+1}\right),$ $\displaystyle T_{2}[d_{i}]=(1+t(1/\varepsilon+1)+6(k+3)(i-1))s_{1}+2zs_{k}+2(k+2)\left(\sum_{g\in V_{i}\setminus Z}s_{g+1}+\sum_{g\in Z}s_{g}\right),$ $\displaystyle T_{2}[c_{i}]-T_{2}[d_{i}]=2z-1+2(k+2)\left(\sum_{g\in V_{i}\setminus Z}(s_{g}-s_{g+1})-\sum_{g\in Z}(s_{g}-s_{g+1})\right).$ $$

The only way for $C_{i}$ to be not satisfied is if $x_{g}$ is assigned to True (*i.e.*, $s_{g}-s_{g+1}>\frac{1}{k+2}$) for all $g\in Z$ and $x_{h}$ is assigned to False (*i.e.*,s $s_{h}-s_{h+1}<\varepsilon$) for all $h\in V_{i}\setminus Z$. This would imply, however $T_{2}[c_{i}]-T_{2}[d_{i}]<2z-1+2(k+2)\left(\sum_{g\in V_{i}\setminus Z}\varepsilon-\sum_{g\in Z}\frac{1}{k+2}\right)=2z-1+\frac{2(|V_{i}|-z)}{7}-2z<-1+\frac{6}{7}<0$, which gives a contradiction.

As assuming $C_{i}$ is not satisfied gives a contradiction to the assumption that $s$ gives agreement between $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$ for $c_{i}$ and $d_{i}$, $C_{i}$ must be satisfied. Since this is true for all $i\in[\ell]$, this implies that $\phi$ is satisfiable.

$(\Rightarrow):$ Assume $\phi$ is satisfiable for truth assignments $\{x^{*}_{i}\}_{i\in[t]}$. For any $i\in[t]$, define $\delta_{i}=\begin{cases}\frac{1}{k+1}&\text{if }x^{*}_{i}\text{ is True}\\
\frac{\varepsilon}{2}&\text{otherwise.}\end{cases}$.
Define the positional scoring rule $s=(s_{i})_{i\in[k]}$ as $s_{i}=\begin{cases}1&\text{if }i=1\\
1-\left(\sum_{j=1}^{i-1}\delta_{j}\right)&\text{if }k>i>1\\
0&\text{if }i=k\end{cases}$. Since $s_{k-1}=1-\left(\sum_{j=1}^{k-2}\delta_{j}\right)=\geq 1-\frac{k-2}{k+1}>0=s_{k}$, monotonicity is satisfied, and $s$ is a valid scoring rule. We will show that $s$ gives perfect agreement between $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$. As always, say $T_{\beta}[f]=\sum_{i=1}^{k}M_{\beta}[f,j]s_{i}$ for all $\beta\in\{1,2\}$ and $f\in A$. Since $1=s_{1}\geq s_{2}\geq\ldots\geq s_{k}=0$ and since $s_{i}>0$ for each $i\in[k-1]$, the total scores for each $i\in[t]$ and $j\in[\ell]$ are

$$ $\displaystyle T_{1}[a_{i}]$ $\displaystyle=(1+(k+3)(i-1))s_{1}+(k+2)s_{i+1}\Rightarrow(k+3)i\geq T_{1}[a_{i}]>(k+3)(i-1)$ (A1) $\displaystyle T_{2}[a_{i}]$ $\displaystyle=(1+(1/\varepsilon+1)(i-1))s_{1}+(1/\varepsilon)s_{i+1}\Rightarrow(1/\varepsilon+1)i\geq T_{2}[a_{i}]>(1/\varepsilon+1)(i-1)$ (A2) $\displaystyle T_{1}[b_{i}]$ $\displaystyle=(k+3)(i-1)s_{1}+(k+2)s_{i}+s_{k}\Rightarrow(k+3)i>T_{1}[b_{i}]>(k+3)(i-1)$ (B1) $\displaystyle T_{2}[b_{i}]$ $\displaystyle=(1/\varepsilon+1)(i-1)s_{1}+(1/\varepsilon)s_{i}+s_{k}\Rightarrow(1/\varepsilon+1)i>T_{2}[b_{i}]>(1/\varepsilon+1)(i-1)$ (B2) $\displaystyle T_{1}[c_{j}]$ $\displaystyle=(t(k+3)+2j)s_{1}\Rightarrow t(k+3)+2j=T_{1}[c_{j}]>t(k+3)+2(j-1)$ (C1) $\displaystyle\begin{split}T_{2}[c_{j}]&=(2|Z_{j}|+t(1/\varepsilon+1)+6(k+3)(j-1))s_{1}+s_{k}+2(k+2)\left(\sum_{g\in V_{j}\setminus Z_{j}}s_{g}+\sum_{g\in Z_{j}}s_{g+1}\right)\\ &\Rightarrow t(1/\varepsilon+1)+6(k+3)j\geq T_{2}[c_{j}]>t(1/\varepsilon+1)+6(k+3)(j-1)\end{split}$ (C2) $\displaystyle T_{1}[d_{j}]$ $\displaystyle=(t(k+3)+2j-1)s_{1}+s_{k}\Rightarrow t(k+3)+2j>T_{1}[d_{j}]>t(k+3)+2(j-1)$ (D1) $\displaystyle\begin{split}T_{2}[d_{j}]&=(1+t(1/\varepsilon+1)+6(k+3)(j-1))s_{1}+2|Z_{j}|s_{k}+2(k+2)\left(\sum_{g\in V_{j}\setminus Z_{j}}s_{g+1}+\sum_{g\in Z_{j}}s_{g}\right)\\ &\Rightarrow t(1/\varepsilon+1)+6(k+3)j>T_{2}[d_{j}]>t(1/\varepsilon+1)+6(k+3)(j-1)\end{split}$ (D2) $$

where $V_{j}\subseteq[t]$ and $Z_{j}\subseteq V_{j}$ indicate the indices of the variables that appear in and that appear negated in the clause $C_{j}$, respectively.
We now show that $s$ gives perfect agreement between $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$, *i.e.*, $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$ for all distinct pairs of $g,h\in A\sqcup B\sqcup C\sqcup D\sqcup E$. We will proceed by a case by case analysis:

- •
Case 1: $g\in A\sqcup B$, $h\in C\sqcup D$. By [eqs. A1](https://arxiv.org/html/2508.17177v1#A1.Ex68) and [B1](https://arxiv.org/html/2508.17177v1#A1.Ex70), we have $T_{1}[g]\leq t(k+3)$, since $i\leq t$. By [eqs. C1](https://arxiv.org/html/2508.17177v1#A1.Ex72) and [D1](https://arxiv.org/html/2508.17177v1#A1.Ex74), we have $T_{1}[h]>t(k+3)$, as $j\geq 1$. This implies $T_{1}[g]<T_{1}[h]$. Similarly, by [eqs. A2](https://arxiv.org/html/2508.17177v1#A1.Ex69) and [B2](https://arxiv.org/html/2508.17177v1#A1.Ex71) we have $T_{2}[g]\leq t(1/\varepsilon+1)$, and by [eqs. C2](https://arxiv.org/html/2508.17177v1#A1.Ex73) and [D2](https://arxiv.org/html/2508.17177v1#A1.Ex75) we have $T_{2}[h]>t(1/\varepsilon+1)$, as $j\geq 1$. This implies $T_{2}[g]<T_{2}[h]$. Hence, $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$, as desired.
- •
Case 2: $g\in\{a_{i},b_{i}\}$, $h\in\{a_{j},b_{j}\}$ for some $t\geq i>j\geq 1$. By [eqs. A1](https://arxiv.org/html/2508.17177v1#A1.Ex68) and [B1](https://arxiv.org/html/2508.17177v1#A1.Ex70), we have $T_{1}[g]>(k+3)(i-1)\geq(k+3)j$ and $T_{1}[h]\leq(k+3)j$. This implies $T_{1}[g]>T_{1}[h]$. Similarly, by [eqs. A2](https://arxiv.org/html/2508.17177v1#A1.Ex69) and [B2](https://arxiv.org/html/2508.17177v1#A1.Ex71) we have $T_{2}[g]>(1/\varepsilon+1)(i-1)\geq(1/\varepsilon+1)j$ and $T_{2}[h]\leq(1/\varepsilon+1)j$. This implies $T_{2}[g]>T_{2}[h]$. Hence, $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$, as desired.
- •
Case 3: $g=a_{i}$, $h=b_{i}$ for some $i\in[t]$. In this case $T_{1}[g]-T_{1}[h]=1-(k+2)(s_{i}-s_{i+1})=1-(k+2)\delta_{i}$ and $T_{2}[g]-T_{2}[h]=1-(1/\varepsilon)(s_{i}-s_{i+1})=1-\delta_{i}/\varepsilon$. If $\delta_{i}=\frac{\varepsilon}{2}$ (*i.e.* $x^{*}_{i}$ is False), $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])=(1-\frac{1}{14})(1-\frac{1}{2})>0$. If $\delta_{i}=\frac{1}{k+1}$ (*i.e.* $x^{*}_{i}$ is True), $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])=(1-\frac{k+2}{k+1})(1-\frac{7(k+2)}{k+1})=(\frac{-1}{k+1})(\frac{-6k-13}{k+1})>0$.
- •
Case 4: $g\in\{c_{i},d_{i}\}$, $h\in\{c_{j},d_{j}\}$ for some $t\geq i>j\geq 1$. By [eqs. C1](https://arxiv.org/html/2508.17177v1#A1.Ex72) and [D1](https://arxiv.org/html/2508.17177v1#A1.Ex74), we have $T_{1}[g]>t(k+3)+2(i-1)\geq t(k+3)+2j$ and $T_{1}[h]\leq t(k+3)+2j$. This implies $T_{1}[g]>T_{1}[h]$. Similarly, by [eqs. C2](https://arxiv.org/html/2508.17177v1#A1.Ex73) and [D2](https://arxiv.org/html/2508.17177v1#A1.Ex75) we have $T_{2}[g]>t(1/\varepsilon+1)+6(k+3)(i-1)\geq t(1/\varepsilon+1)+6(k+3)j$ and $T_{2}[h]\leq t(1/\varepsilon+1)+6(k+3)j$. This implies $T_{2}[g]>T_{2}[h]$. Hence, $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$, as desired.
- •
Case 5: $g=c_{i}$, $h=d_{i}$ for some $i\in[\ell]$. In this case $T_{1}[g]-T_{1}[h]=1$ and
$\displaystyle T_{2}[g]-T_{2}[h]$
$\displaystyle=2|Z_{i}|-1+2(k+2)\left(\sum_{g\in V_{i}\setminus Z_{i}}(s_{g}-s_{g+1})-\sum_{h\in Z_{i}}(s_{h}-s_{h+1})\right)$
$\displaystyle=2|Z_{i}|-1+2(k+2)\left(\sum_{g\in V_{i}\setminus Z_{i}}\delta_{g}-\sum_{h\in Z_{i}}\delta_{h}\right),$
where $V_{i}\subseteq[t]$ and $Z_{i}\subseteq V_{i}$ indicate the indices of the variables that appear in and that appear negated in the clause $C_{i}$, respectively. Since $\{x^{*}_{i}\}_{i\in t}$ is a satisfying assignment by assumption, we must either have $\delta_{g^{\prime}}=\frac{1}{k+1}$ (*i.e.* $x^{*}_{g^{\prime}}$ is True) for some $g^{\prime}\in V_{i}\setminus Z_{i}$ or $\delta_{h^{\prime}}=\frac{\varepsilon}{2}$ (*i.e.* $x^{*}_{h^{\prime}}$ is False) for some $h^{\prime}\in Z_{i}$. If the former is true ($\delta_{g^{\prime}}=\frac{1}{k+1}$ for some $g^{\prime}\in V_{i}\setminus Z_{i}$):
$\displaystyle T_{2}[g]-T_{2}[h]$
$\displaystyle=2|Z_{i}|-1+2(k+2)\left(\frac{1}{k+1}+\sum_{g\in V_{i}\setminus(Z_{i}\cup\{g^{\prime}\})}\delta_{g}-\sum_{h\in Z_{i}}\delta_{h}\right)$
$\displaystyle\geq 2|Z_{i}|-1+2(k+2)\left(\frac{1}{k+1}+\frac{(|V_{i}|-|Z_{i}|-1)\varepsilon}{2}-\frac{|Z_{i}|}{k+1}\right)$
$\displaystyle\geq 2|Z_{i}|-1+2(k+2)\left(\frac{1-|Z_{i}|}{k+1}-\frac{\varepsilon}{2}\right)$
$\displaystyle=-\frac{2|Z_{i}|}{k+1}+\frac{k+3}{k+1}-\frac{1}{7}=\frac{-2|Z_{i}|+\frac{6}{7}k+\frac{20}{7}}{k+1}.$
Similarly, if the latter is true ($\delta_{h^{\prime}}=\frac{\varepsilon}{2}$ for some $h^{\prime}\in Z_{i}$):
$\displaystyle T_{2}[g]-T_{2}[h]$
$\displaystyle=2|Z_{i}|-1+2(k+2)\left(\sum_{g\in V_{i}\setminus Z_{i}}\delta_{g}-\frac{\varepsilon}{2}-\sum_{h\in Z_{i}\setminus\{h^{\prime}\}}\delta_{h}\right)$
$\displaystyle\geq 2|Z_{i}|-1+2(k+2)\left(-\frac{\varepsilon}{2}-\frac{|Z_{i}|-1}{k+1}\right)=-\frac{2|Z_{i}|}{k+1}+\frac{k+3}{k+1}-\frac{1}{7}$
$\displaystyle=\frac{-2|Z_{i}|+\frac{6}{7}k+\frac{20}{7}}{k+1},$
which gives the same inequality. Consider two cases: if $t\geq 2$ (*i.e.*, $k=t+2\geq 4$), we get
$\displaystyle T_{2}[g]-T_{2}[h]\geq\frac{-2|Z_{i}|+\frac{6}{7}k+\frac{20}{7}}{k+1}\geq\frac{-6+\frac{24}{7}+\frac{20}{7}}{k+1}=\frac{\frac{2}{7}}{k+1}>0,$
since $|Z_{i}|\leq 3$. If $t=1$ (*i.e.*, $k=3$), then $|Z_{i}|\leq 1$, since $t$ is the number of variables in $\phi$. Then,
$\displaystyle T_{2}[g]-T_{2}[h]\geq\frac{-2|Z_{i}|+\frac{6}{7}k+\frac{20}{7}}{k+1}\geq\frac{-2+\frac{18}{7}+\frac{20}{7}}{k+1}=\frac{\frac{24}{7}}{k+1}>0.$
In both cases, we have $T_{2}[g]-T_{2}[h]>0$ and hence $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$, as desired.
- •
Case 6: $g\in E\setminus\{e_{k}\},h\notin E$. Say $g=e_{i}$ for some $1\leq i<k$ and $n_{\beta}=\sum_{f\in A\sqcup B\sqcup C\sqcup D}\sum_{j=1}^{k}M_{\beta}[f,j]$ for $\beta\in\{1,2\}$. Due to the last set of voters that we added while constructing $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$ (those that rank only elements of $E$), we have
$\displaystyle M_{\beta}[e_{i},i]\geq 14(k+2)\max_{\gamma\in\{1,2\}}n_{\gamma}$
$\displaystyle\geq 14(k+2)\sum_{j=1}^{k}M_{\beta}[h,j]$
$\displaystyle\geq 14(k+2)\sum_{j=1}^{k}M_{\beta}[h,j]s_{j}=14(k+2)T_{\beta}[h]$
Moreover, since $s_{i}\geq s_{k-1}\geq 1-\frac{k-2}{k+1}=\frac{3}{k+1}>\frac{1}{k+1}$, we have
$T_{\beta}[g]=M_{\beta}[g,i]s_{i}>\frac{M_{\beta}[e_{i},i]}{k+1}>T_{\beta}[h]$
for each $\beta\in\{1,2\}$. This gives $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$, as desired.
- •
Case 7: $g=e_{k}$. By construction $M_{\beta}[e_{k},j]>0\Rightarrow j=k$, and hence $T_{\beta}[g]=T_{\beta}[e_{k}]=M_{\beta}[e_{k},k]s_{k}=0$ for each $\beta\in\{1,2\}$. If $h\in A\sqcup B\sqcup C\sqcup D$, we have $T_{\beta}[h]>0$ for both $\beta\in\{1,2\}$ by [eqs. A1](https://arxiv.org/html/2508.17177v1#A1.Ex68), [A2](https://arxiv.org/html/2508.17177v1#A1.Ex69), [B1](https://arxiv.org/html/2508.17177v1#A1.Ex70), [B2](https://arxiv.org/html/2508.17177v1#A1.Ex71), [C1](https://arxiv.org/html/2508.17177v1#A1.Ex72), [C2](https://arxiv.org/html/2508.17177v1#A1.Ex73), [D1](https://arxiv.org/html/2508.17177v1#A1.Ex74) and [D2](https://arxiv.org/html/2508.17177v1#A1.Ex75). If $h=e_{i}$ for some $i\in[k-1]$, on the other hand, $M_{\beta}[h,i]>0$ by the last set of voters added to $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$ (those that only rank the elements of $E$), so once again we have $T_{\beta}[h]=M_{\beta}[h,i]s_{i}>0$. Hence, we have $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$, as desired.
- •
Case 8: $g=e_{i}$, $h=e_{j}$ for some $i<j<k$. Fix some $\beta\in\{1,2\}$. By construction, $\bm{\sigma}^{(\beta)}$ contains two types of rankings: (a) those that contain one elements in $A\sqcup B\sqcup C\sqcup D$ and all remaining elements are those in $E$, of which there $n_{\beta}=\sum_{f\in A\sqcup B\sqcup C\sqcup D}\sum_{j=1}^{k}M_{\beta}[f,j]$ many and (b) those that rank only and all elements in $E$, of which there are say $y_{\beta}$ many, with $y_{\beta}\geq\max_{\gamma\in\{1,2\}}14(k+2)n_{\gamma}\geq 14(k+2)n_{\beta}$. Hence, for any $j^{\prime}\in[k-1]$, we have $n_{\beta}+y_{\beta}>M[e_{j^{\prime}},j^{\prime}]\geq y_{\beta}$, where the first inequality is strict since $M_{\beta}[f,j^{\prime}]>0$ for some $f\in A\sqcup B\sqcup C\sqcup D$ for all $j^{\prime}\in[k-1]$ (see, for example, the definitions for $M_{\beta}[a_{j^{\prime}},j^{\prime}+1]$ and $M_{\beta}[b_{j^{\prime}},j^{\prime}]$ for each $j^{\prime}\in[t]$), so there is at least some voters in the group of $n_{\beta}$ that do not rank $e_{j^{\prime}}$.
This implies
$\displaystyle T_{\beta}[g]-T_{\beta}[h]$
$\displaystyle=T_{\beta}[e_{i}]-T_{\beta}[e_{j}]=M_{\beta}[e_{i},i]s_{i}-M_{\beta}[e_{j},j]s_{j}$
$\displaystyle>y_{\beta}s_{i}-(n_{\beta}+y_{\beta})s_{j}=y_{\beta}(s_{i}-s_{j})-n_{\beta}s_{j}$
$\displaystyle\geq y_{\beta}(s_{i}-s_{i+1})-n_{\beta}=y_{\beta}\delta_{i}-n_{\beta}$
$\displaystyle\geq 14(k+2)n_{\beta}\cdot\frac{\varepsilon}{2}-n_{\beta}=n_{\beta}-n_{\beta}=0$
Hence, we have $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$, as desired.

We have shown that $(T_{1}[g]-T_{1}[h])(T_{2}[g]-T_{2}[h])>0$ for all $g,h\in A$, proving that $s$ indeed gives full agreement between $\bm{\sigma}^{(1)}$ and $\bm{\sigma}^{(2)}$.

∎

###### Proposition 19 .

###### Proof.

## Appendix B Experiment Details

This appendix provides additional details on experiments discussed in [Section 7](https://arxiv.org/html/2508.17177v1#S7).

### B.1 Ground Truth and Disagreement

In our first experiment, we evaluate the relationship between the rankings produced by SWFs and the ground truth ranking of a preference distribution.

Both Mallows and Plackett-Luce (PL) preference models naturally correspond to an “ideal” ranking. These distributions also have known maximum likelihood estimators: Mallow’s MLE is the Kemeny function while we estimate the MLE of PL preferences using the choix Python library(^9^99https://choix.lum.li/en/latest/index.html).

In our experiment, for each distribution we generate 50 elections with 100 voters and 100 alternatives. We set $\phi=0.4$ for Mallow’s preferences and $\alpha_{i}=e^{0.5(m-i)}$ for Plackett-Luce. Each voter provides a ranking over 10 alternatives, sampled from the relevant distribution, such that each alternative is ranked by 10 voters. This type of partial ranking aligns with a paper reviewing framework where conference organizers may wish to ensure that each paper is reviewed by a certain number of reviewers while each reviewer receives a certain number of assignments.

For each generated election, we do the following for several SWFs: Assign each voter into one of two groups, chosen uniformly at random for each voter. Each of these groups is then used as a complete profile to generate a weak ranking according to the SWF. We measure the $KT$ distance between these two rankings. We report the mean over these 10 splits as the “split distance.” We also calculate the $KT$ distance from the ranking generated by the SWF applied to all voters to the ground truth ranking (the “ground truth distance”).

We plot each pair of distances for each SWF. In [Figure 1](https://arxiv.org/html/2508.17177v1#S7.F1) the MLE of each noise model tends to minimize each distance. In general, there is a very strong relationship between the two distances. As the split distance increases, so too does the ground truth distance.

In this experiment the SWFs we evaluated were Kemeny, Plackett-Luce MLE (as implemented by the choix library), Borda Min-Max, and Optimized Positional Scores (see [subsection B.7](https://arxiv.org/html/2508.17177v1#A2.SS7)) in addition to positional scoring rules with the following vectors:

- •
Plurality $(1,0,0,\ldots,0)$
- •
Plurality + Veto $(1,0.5,0.5,\ldots,0.5,0)$
- •
Veto $(1,1,\ldots,1,0)$
- •
Two-Approval $(1,1,0,\ldots,0)$
- •
Borda $(\frac{m-1}{m-1},\frac{m-2}{m-1},\ldots,\frac{m-m}{m-1})$

### B.2 Axiom Violations

Figure: Figure 4: Violation rate of axioms across several preference distributions and real-world election data (rightmost). Axiom violations generally decrease as the number of alternatives increases.
Refer to caption: plots/axiom_evaluation_results-n_splits=50.png

To further explore the theoretical results of [Section 5](https://arxiv.org/html/2508.17177v1#S5) we analyze experimentally how often certain axioms are violated across several distinct preference distributions. Results of this experiment are found in [Figure 4](https://arxiv.org/html/2508.17177v1#A2.F4). We measure the axiom violation rate, as defined by Caiata et al. [2025]; an experimental measure of how often axioms are violated in practice on given preference distributions and voting rules.
For this experiment we sample, for each distribution, 500 profiles with 100 voters that each provide a full ranking over $m\in{5,10,15,20}$ alternatives from the Impartial Culture, Urn (with $\alpha$ sampled from a Gamma distribution with shape $k=0.8$ and scale $\theta=1$ Boehmer et al. [2021]), Mallows ($\phi=0.4)$, Plackett-Luce ($\alpha_{i}=e^{0.5(m-i)}$), and Single-Peaked preference distributions Brandt et al. [2016]. Additionally, we consider all elections with complete preferences from PrefLib with up to 1000 voters and $m$ alternatives for $5\leq m\leq 20$ (a total of 1392 elections) Mattei and Walsh [2013]. This real-world election data is compiled from a wide variety of sources with varying underlying preference distributions.

On each election we take 50 random splits and calculate the fraction of splits on which each axiom is violated by a Rule Picking Rule using several positional scoring rules.

As we evaluate the Reversal Symmetry axiom we are constrained to using positional scoring rules in our Rule Picking Rule. For all axioms, our RPR uses the following rules: Plurality, Plurality + Veto, Veto, Two-Approval, and Borda count. The scoring vector associated with each of these rules is given in [Section B.1](https://arxiv.org/html/2508.17177v1#A2.SS1).

We provide an example to illustrate how we measure axiom violations.

Example. Consider the monotonicity axiom. In a profile $\bm{\sigma}$ and a modification $\bm{\sigma}_{a}$ where some voters increase the rank they give to alternative $a$. Monotonicity is violated if the rank of $a$ under the RPR is lower in $\bm{\sigma}_{a}$ than in $\bm{\sigma}$.

We test for a violation of monotonicity in a profile $\bm{\sigma}$ by selecting the alternative $a$ ranked first in $\bm{\sigma}$ by the RPR, selecting some uniform random fraction of voters in (0.2, 0.8) to increase their ranking of $a$. If the RPR on $\bm{\sigma}_{a}$ assigns a different rank to $a$ we say that the axiom has been violated. We report the fraction of instances tested in which the axiom is violated.

As shown in [Proposition 10](https://arxiv.org/html/2508.17177v1#Thmprop10) (which easily extends to our implementation using Monte Carlo sampling), reversal symmetry is never violated and only checked as a test. Moreover, under all but one distribution, violations of union consistency and monotonicity are rare ($<0.05$ of the time for $\geq 10$ alternatives) and generally decrease as the number of alternatives increases.(^10^1010The one exception to this trend is monotonicity in the Impartial Culture (IC) distribution, which is not centered around a ground truth. One possible explanation for this is IC maximizing the probability of majority cycles [Tsetlin et al., 2003], under which monotonicity violations are more likely to occur.)

### B.3 Alternative Distance Functions

Figure: (a) Jaccard dissimilarity between sets of winners for splits of partial rankings of ALMA Cycle 10 project proposals.
Refer to caption: plots/alma_data_cycle10-jaccard-distances.png

As mentioned in [Section 4](https://arxiv.org/html/2508.17177v1#S4), the $AbC$ framework can also be used in conjunction with distance functions other than the (weighted) Kendall-Tau distance. This can be especially for settings such as peer review of poroposals for the Atacama Large Millimeter Array (ALMA) [Meyer et al., 2022]. Here, the goal is not necessarily to rank all proposals, but to pick $k$ proposals to be funded, for some $k<{|A|}$. It may thus be more useful to study the consistency the top $k$ proposals picked by each rule. Indeed, we explore this possibility by using the Jaccard dissimilarity [Jaccard, 1901] to measure the similarity of winning proposals on ALMA data.

Jaccard dissimilarity measures the overlap of two sets; it is defined as the ratio of the symmetric difference to the size of the union of two sets. For each voting rule that we evaluate, we select the 240 highest ranked proposals as the winning proposals (chosen to match the number of winning proposals chosen in ALMA Cycle 10).

In [Figure 5](https://arxiv.org/html/2508.17177v1#A2.F5) we see the Jaccard dissimilarity for several rules on proposal rankings from Cycle 10 and 11 of ALMA. The differences between rules have a similar relative order to the Kendall-Tau distance of the rules applied to the same data ([Figure 2](https://arxiv.org/html/2508.17177v1#S7.F2)).

### B.4 City Election Data

In [3(a)](https://arxiv.org/html/2508.17177v1#S7.F3.sf1) we show results of several rules on Preflib data from several real-world political elections. Each of these elections used Instant Runoff Voting (IRV) to compute the empirical winner. Given a profile, IRV iteratively eliminates the alternative with the least plurality score (*i.e.*, with the least number of voters ranking them top) by removing them from the profile, until a single alternative remains. As an SWF, we interpret IRV as the rule outputing the alternatives in the reverse of this elimination order (the alternative eliminated first is ranked bottom, and so on). We provide details on each election shown in [Table 2](https://arxiv.org/html/2508.17177v1#A2.T2). The data from Preflib can be freely distributed and used under the GPL-3.0 license.

Initially we collected all elections included in Preflib Elections 5, 16, 17 ,18, 19, 20, 21, and 22. However, in [3(a)](https://arxiv.org/html/2508.17177v1#S7.F3.sf1) we have filtered out two types of election from our starting data data in order to make the plot more readable:

- •
Any elections where all rules had a split distance of 0. While this is a random event, we found that this consistently included a set of 10 elections.
- •
Any election with more than $100$ candidates. This excluded two specific elections that occurred in 2009 in the city of Minneapolis with 379 and 477 alternatives. These two elections have exceptionally high split distances and were removed as outliers due to their split distance and unusual number of candidates.

**Table 2: Details on the election represented at each index in [3(a)](https://arxiv.org/html/2508.17177v1#S7.F3.sf1).**
| Plot Index | Preflib Election ID | Election Name | # Voters | # Candidates |
| --- | --- | --- | --- | --- |
| 0 | 5 | City (2009 Burlington Mayoral Election) | 8980 | 6 |
| 1 | 16 | City (Aspen City Council 2009) | 2477 | 11 |
| 2 | 16 | City (Aspen Mayor 2009) | 2527 | 5 |
| 3 | 17 | City (2010 Berkeley City Council - District 7) | 4173 | 4 |
| 4 | 18 | City (2009 Minneapolis Board of Estimate and Taxation Election - No Write In) | 32086 | 7 |
| 5 | 18 | City (2009 Minneapolis Park and Recreation Commissioner At-Large Election - No Write In) | 36655 | 9 |
| 6 | 19 | City (2010 Oakland Mayor) | 119256 | 11 |
| 7 | 19 | City (2010 Oakland City Council - District 4) | 20981 | 8 |
| 8 | 19 | City (2012 Oakland City Council - District 3) | 22079 | 7 |
| 9 | 19 | City (M2012 Oakland City Council - District 1) | 28660 | 8 |
| 10 | 20 | City (2008 Pierce County Assessor - Treasurer) | 262312 | 7 |
| 11 | 21 | City (2011 San Francisco Mayor) | 194530 | 25 |
| 12 | 21 | City (2011 San Francisco Sheriff) | 183192 | 5 |
| 13 | 21 | City (2012 San Francisco Board of Supervisors - District 5) | 35183 | 9 |
| 14 | 21 | City (San Francisco Board of Supervisors - District 7) | 31437 | 10 |
| 15 | 21 | City (2010 San Francisco Board of Supervisors - District 2) | 24109 | 7 |
| 16 | 21 | City (2008 San Francisco Board of Supervisors - District 9) | 26634 | 8 |
| 17 | 21 | City (2008 San Francisco - Board of Supervisors District 3) | 27310 | 10 |
| 18 | 21 | City (2008 San Francisco Board of Supervisors - District 1) | 28777 | 10 |
| 19 | 21 | City (2010 San Francisco Board of Supervisors - District 10) | 18001 | 22 |
| 20 | 21 | City (2008 San Francisco Board of Supervisors - District 11) | 24717 | 10 |
| 21 | 21 | City (2010 San Francisco Board of Supervisors - District 6) | 21188 | 15 |
| 22 | 22 | City (2010 San Leandro Mayor) | 22407 | 7 |
| 23 | 22 | City (2012 San Leandro City Council - District 4) | 23236 | 5 |
| 24 | 22 | City (2012 San Leandro City Council - District 2) | 25355 | 4 |

### B.5 Formula One Data

Figure: Figure 6: Distance between splits for SWFs aggregating rankings of drivers in F1 races. Each rule is evaluated on each period of races. Newer F1 rules provide lower distance on all race periods.
Refer to caption: plots/F1_bar-all_data.png

The F1 portion of [Figure 3](https://arxiv.org/html/2508.17177v1#S7.F3) is generated using data about races found on Preflib Mattei and Walsh [2013]; specifically, we use the complete form of dataset ID 00053.
This contains one profile for each Formula One season with a preference order corresponding to each individual race in the season. Each preference order lists the drivers that competed in every race in that season, ordered by the position in which they finished that race.

We show in [Figure 6](https://arxiv.org/html/2508.17177v1#A2.F6) the $KT$ distance for all rules divided by racing period. While the rules with highest distance (Two-Approval, Plurality + Veto, Plurality, Veto) stay quite consistent, all other rules provider lower $KT$ distances on more recent race periods. Notably, the F1 rules themselves become much more consistent over time.

### B.6 Olympic Data

Figure: Figure 7: Average distance and standard error between splits for SWFs over rankings induced by Olympic medals. Optimization of positional scores is only occasionally able to improve upon the ranking generated by giving one point to each country for each medal they win, regardless of medal type.
Refer to caption: plots/olympics_bar.png

To evaluate rules on Olympic medal wins we use a Kaggle dataset providing details of all Olympic results (Summer and Winter) between 1896 and 2016 [Griffin, 2018]. From this we extract the winning countries of each medal for each event. We convert each event into a partial ranking by assigning to first place all countries receiving gold medals, to second place all countries receiving silver medals, and to third place all countries receiving bronze medals. Countries that did not win medals or did not compete in an event are not included in a ranking.

In the large majority of cases this results in a partial order consisting of exactly three countries, each in a different rank. In rare cases a position might be empty or have multiple winners (e.g. in 1992 Canada and USA won Gold in women’s solo synchronized swimming, no Silver was awarded, and Bronze was won by Japan), or a country might occur multiple times (e.g. in 2008 Jamaica won Gold and two Silver medals in the women’s 100 metre; no Bronze was awarded).
In these exceptional cases we do nothing different and award that country points for each of the positions that it occupies.

This results in one “election” for each year in which the Olympics occurred where preference orders correspond to partial rankings induced by medal wins. To these rankings we apply the $AbC$ framework as we have in all other experiments: We generate splits by randomly placing the profile induced by each event into one split or the other, then find the distance between splits and report the average over many sets of splits. Here we add two new rules:

- •
Medal Count: Each medal is treated equivalently. Each country receives a point each time it appears in a preference order, regardless of rank.
- •
Leximax: Rank countries by the number of Gold medals they receive, breaking ties by counting Silver medals, breaking remaining ties by counting Bronze medals. In practice we implement this as the score vector $(1000000,1000,1)$.

Results of this analysis are displayed in [Figure 7](https://arxiv.org/html/2508.17177v1#A2.F7). We see that simply counting the total number of medals each country receives results in much lower disagreement than standard voting rules, only very rarely do optimized scores improve upon the Medal Count rule.

### B.7 Additional Definitions of Voting Rules

Through our experiments we use several rules which are not fully described. In this section we describe each rule found throughout our paper.

#### B.7.1 Kemeny

Kemeny’s rule is defined as the ranking which minimizes the sum of Kendall-Tau distances between the output ranking and each voter’s individual ranking Kemeny [1959]. We implement the Kemeny method code provided by Baharev et al. [2021] and the Gurobi optimization library [Baharev et al., 2021, Gurobi Optimization, LLC, 2024].

#### B.7.2 Trimmed Borda

Calculate the number of voters ranking each alternative at each rank. For each alternative $a_{i}$, remove one of the highest rankings which $a_{i}$ has received and remove one of the lowest rankings which $a_{i}$ has received. After all removals are complete, calculate Borda scores as normal [Meyer et al., 2022].

#### B.7.3 Best Positional Scores

This method uses the optimal-voting package generate a positional scoring vector which minimizes $KT$ distance between splits on a given profile [Armstrong, 2025]. This package uses simulated annealing to generate novel positional scoring rules which optimize a target function.
In each of our experiments, we run optimization multiple times; starting once from the initial score vector of each other positional scoring rule used in the experiment. For instance, if we were comparing with Borda’s rule and Plurality we run optimization twice, starting from $(m-1,m-2,...,0)$ (Borda), and $(1,0,...,0)$ (Plurality). In all cases, we run annealing for 500 steps.
At each step, optimal-voting updates one index in the state vector (i.e., the positional score vector) by some amount sampled uniformly at random from $(0.05,1)$. We restrict updates to those that result in the state vector being weakly monotonically decreasing (the value at each index is not higher than the previous value).
Using the updated state vector as a positional scoring vector we calculate the mean $KT$ split distance over each split of voters. We use the same set of splits in annealing as we do when evaluating each other voting rule.
The new state is accepted with a probability related to the magnitude of difference between the current and previous mean split distances, and the number of steps that have already occurred. A small magnitude of difference is more likely to be accepted earlier than later. States with lower mean split distance are always accepted.

#### B.7.4 Leximax

This rule is used exclusively in evaluating Olympic data. For a given profile, alternatives are ranked according to the number of gold medals they have received. To break ties, tied alternatives are ordered according to the number of silver medals they have received. To break subsequent ties, tied alternatives are ordered according to the number of bronze medals they have received. Note that we can instantiate Leximax as a positional scoring rule where each position is much larger than the subsequent position. In our Olympic experiments we use the (pre-normalization) vector $(1000000,1000,1)$. As there are never more than 1000 opportunities for a single country to receive a medal of one type this is equivalent to Leximax.

#### B.7.5 Medal Count

This rule is used exclusively in evaluating Olympic data. For a given profile, alternatives are ranked purely based on the number of medals they received with no regard for the type of medals.

Our Olympics data uses as input the partial rankings containing only countries winning medals in an event. In this case, the Medal Count rule is equivalent to the positional scoring rule with the vector $(1,1,1)$.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: It is stated that an implementation of AbC was awarded in a recent competition at a top-tier AI conference.

1) What place/score did the mentioned implementation get? 1st place, 2nd place, 3rd place? 
2) How many competitors were there in the competition? 

Without more details it is difficult to judge the strength of the implementation with respect to competing approaches.
soundness: 3
strengths: The main strength of the paper is that the proposed RPR has been implemented and evaluated in practice, and obtained one of the top scores in a relevant AI competition.
summary: The paper tackles the problem of selecting the best rank aggregation method in the setting where different aggregation rules can lead to very different outcomes. To address this problem, the authors introduce the concept of a Rule Picking Rule (RPR), a meta-framework for deciding which aggregation method to use for a given dataset without assuming any specific generative model.  Instead of relying on fixed axioms or assumed noise models, the authors  provide a data-driven approach for picking aggregation rules. They propose a new RPR, called "AbC", which selects the aggregation method in such a way to maximize consistency, that is to say, producing stable results if the data collection were repeated. The authors show that AbC satisfies certain consistency-related axioms that some natural RPRs fail to meet.  For two axioms it cannot satisfy, the authors prove impossibility results, showing no rule can satisfy them simultaneously. Additionally they show that the problem of finding a perfectly consistent rule is NP-complete. Despite this hardness result, the authors have implemented a sampling-based algorithm that seems to perform reasonably well, in the sense that it outperforms some commonly used aggregation methods. The implementation also provide interpretable insights into when and why specific rules work best. It is stated that an implementation of AbC was awarded in a recent competition
at a top-tier AI conference.
weaknesses: The theoretical part is dense and most proofs are relegated to the appendix, which is quite long (22 pages). Therefore, verifying the correctness of the claims is a bit tricky. I did not verify the correctness of the proofs in the appendix.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Did AbC ever select a rule whose behavior appears to be intermediate between known positional scoring rules (e.g., between Kemeny and Borda)? Did it ever select a rule not a combination of existing rules?

When optimizing over positional weights to minimize split-disagreement, do the resulting weights cluster around classical forms (e.g., linear, exponential), or is there a continuum?

Are there theoretical conditions under which AbC provably recovers Kemeny or PL-MLE — beyond the empirical confirmation on Mallows/PL data?

Is there any identifiable set of axioms that exactly describe the rules that minimize disagreement under random resampling? I.e can you simulate an uniqueness axiom?
soundness: 3
strengths: The paper is well written and answers an important problem, especially motivated by the Neurips paper acceptance experiment.

The axiom satisfactions depict that the proposed method meets sanity checks. 

The proofs in the appendix seem thorough.

The monte carlo estimation makes the implementation practical.
summary: The paper introduces Rule Picking Rules (RPRs) : procedures that choose which aggregation rule to use for a given dataset; and proposes Aggregation by Consistency (AbC), which selects the rule whose outcomes are most stable across random evaluator splits. 

It develops an axiomatic basis (e.g., reversal symmetry, shuffling consistency), proves that achieving perfect consistency over positional scoring rules is NP-complete, and provides a computationally practical Monte-Carlo implementation.

AbC (i) often rediscovers the model-MLE when the generative model matches the data, (ii) yields actionable guidance on rating aggregations, and (iii) extends to top-k selection via set-based distances.
weaknesses: The paper focuses on Kendall's Tau as the rank comparision metric. Metrics like Cumulative Gain (CG), Discounted CG (DCG), and Normalized DCG are popular for comparing top heavy ranks. A discussion involving those metrics and how the algorithms fares to them would be good to have.

The empirical section is somewhat narrowly scoped. Although they show correlation between disagreement and distance to ground truth in the synthetic models, no end-to-end evaluation links “most consistent rule” to “best downstream decision quality.” For instance, in peer review: does the AbC-selected rule produce higher eventual citation impact or expert consensus? 

Moreover, the real-world data are both from astronomy. It would be helpful to experiment on a diverse set of domains (e.g., recommendation rankings, RLHF preference data, crowd judgments - all readily available) to show generality.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: 1. What is the specific reason to motivate and interpret as a "rule to pick a rule"? (Weakness 1)
2. How would you answer to this incosistency and justify your axiomatic analysis?  (Weakness 2)
3. Are there any references to the given consistency axioms? Specifically, why "reverse symmetry" sounds so important? At first glance it does not look like a fundamental axiom like anonymity or neutrality. 
4. Is your AbC rule not welfare maximization? If so, do you have any comments on this? 

I am open to discussion and happy to raise my score if my concerns are addressed.
soundness: 4
strengths: This paper is quite impressive to me. In the introduction, the paper exhibits a very strong connection with previous studies and place itself well among them. They also clearly justified how their key idea -- consistency -- is natural and intuitive in many interdiscinpline research topics. I don't check the proofs, but the theorems sounds reasonable at a high level. This paper give a reasonably significant conceptual contribution in rank aggregation with a suprisingly simple method, which I think is the most valuable part of this paper. Besides that, it has a thorough picture, from theory, to computation, and to emprical studies. The topic is also very related to ICLR.
summary: This paper studies a Rules to Pick a Rule framework: given a ranking profile, what is the ``best'' social choice function to make an aggregation? This paper proposes a surprisingly simple yet effective method: split the profile into two halves u.a.r, run SCF candidates on both halves, and pick the SCF where two outcomes agrees the most (i.e. smallest Kendall-Tau distance between two aggregated rankings). The basic idea is that a voting rule should be consistent among ranking data drawn from the same distribution. Theoretically, this paper considers a series of axioms where their consistence RPR satisfies or those nobody can do better. Empirically, the show the Monte-Carlo approximation of their RPR presents pretty good outcome on datasets.
weaknesses: I am not 100% percent convinced by the idea of "rules to pick a rule". For me, it sounds perfectly ok to interpret it as a another type of rank aggregation rules (as the paper said, the rule RPR induces), which brings more explainability on "we always align with (one of) the SCF with highest consistency".  Specifically, given the method is profile-based, the rule elected is used solely for this data. What is the specific reason to motivate and interpret as a "rule to pick a rule"?

Another concern is that, I feel a little bit inconsistent on the motivation of this paper and the approach it adopts. Consider two area of works. (1) Data driven rank aggregation like preference learning and RLHF. In these scenarios, rankings are all datas, and what rules to pick wholely relies on your goal (like maximizing consistency or minimizing other loss functions). (2) Traditional social choice. In this area, "rules to pick a rule" based on voting profiles is very problematic as a strong signal of manipulation. This paper adopts the scenarios in (1) yet adopt the methodology in (2). Given that the overall goal of this paper is to justify "consistency is a gold standard on picking SCF", the axiomatic approach, which sounds more like "consistency brings natural behaviors under certain hypothetical scenarios", is not a strong support to the paper. How would you answer to this incosistency and justify your axiomatic analysis?

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: One thing I wanted to ask the authors was about the use of 1/2 in place of a tie. Did you try any other values there, or measure any outcomes? In many settings such as RLHF, preferences are sparse. It does not seem unreasonable that one would want to avoid penalizing ties, or perhaps even the opposite.
soundness: 4
strengths: *The problem the authors address is a fairly foundational one. Any given ranking rule must make choices on what to prioritize. These choices cannot generalize perfectly. A principled way of making that choice is a strong contribution and one that has a widespread impacts in many domains(as the authors note)

*While I have not checked the author's math in detail, it appears to be well-formulated and correct. The appendix contains a great deal of rigorous proofs to back up the author's assertions.

*The authors do a strong job not only in demonstrating the strengths and axioms their rules fulfill, but explain why they cannot fulfill the others. I did not find myself questioning any of those choices.

*The authors are explicit about which axioms they satisfy( reversal symmetry, plurality-shuffling consistency, unconditional anonymity, and neutrality).

*As an overall comment, I was satisfied with the writing of the paper, which was well-written and seemed to me very considered throughout.
summary: In all types of domains, individuals disagree. This disagreement is fundamental. As a result, rank aggregation rules must make tradeoffs, picking between different axioms that contradict each other to provide the best output. However, each methods tradeoffs mean that they are acceptable in different situations, and the correct choice of method will vary across domain and task. As a result, the authors here propose AbC (Aggregation by Consistency), a rule-picking rule, which provides a principled way to pick between existing methods. The authors propose a method of splitting data and comparing the consistency of given methods across those splits, allowing for the choice of a method well-suited for any given problem. The method splits voters (the authors explain how this can be used with incomplete preferences), and runs each chosen candidate rule on the split. They measure the Kendall-Tau of each method on each split, and pick the rule with the minimum expected disagreement. 

In order to show the utility of their method, the authors explain what axioms their method satisfies, provide a few examples, and conclude with a set of experiments where they demonstrate AbC on a synthetic dataset, peer review dataset, and a F1 dataset.
weaknesses: The piece that stood out to me were the figures in the paper. They did not add much clarity to me, and in many cases they added to my confusion. Understanding the content of one given figure involves inspecting the appendix. What's good and what isn't doesn't jump out, and the number of methods felt like clutter and made it difficult for me to really get a sense of how each method performed. The labels are small and hard to read, 2A in particular is hard to get a sense of.


Decision: Accept (Poster)