Expected Frequency Matrices of Elections: Computation, Geometry, and Preference Learning
Authors: Niclas Boehmer
Robert Bredereck
Edith Elkind
Piotr Faliszewski
Stanisław Szufa
Venue: NeurIPS 2022 Accept (2022)
Keywords: Mallows model
visualizing experimental results
vote distributions
single-peaked elections

Abstract: We use the "map of elections" approach of Szufa et al. (AAMAS 2020) to analyze several well-known vote distributions. For each of them, we give an explicit formula or an efficient algorithm for computing its frequency matrix, which captures the probability that a given candidate appears in a given position in a sampled vote. We use these matrices to draw the "skeleton map" of distributions, evaluate its robustness, and analyze its properties. We further develop a general and unified framework for learning the distribution of real-world preferences using the frequency matrices of established vote distributions.

Expected Frequency Matrices of Elections: Computation, Geometry, and Preference Learning
Niclas Boehmer
Robert Bredereck TU Clausthal
Algorithmics and Computational Complexity Technische Universität Berlin
robert. www.erreeeteteeeeeeeeeee
ni niclas. boehmer@tu-berliit
Edith Elkind
Piotr Faliszewski
Stanislaw Szufa AGH University
University of Oxford elkindêcs ox. ac uk
AGH University
fal faliszew@agh. edu. pl
Jagiellonian University szufa@agh. edu. odu.pl
Abstract
We use the "map of elections" approach of Szufa al. (AAMAS-2022 to analyze several well-known vote distributions. For each of them, we give explicit for- mula an efficient algorithm for computing frequency matrix, which captures the probability that given candidate appears in given position in a sampled vote. We use these matrices to draw the "skeleton map" of distributions, evaluate its robustness, and analyze its properties. Finally, we develop general and uni- fied framework for learning the distribution of real-world preferences using the frequency matrices of established vote distributions.
Introduction
Computational social choice is research area at the intersection of social choice (the science of collective decision-making! and computer science, which focuses on the algorithmic analysis of problems related to preference aggregation and elicitation (Brandt et al., 2013). Many of the early papers in this field were primarily theoretical, focusing on establishing the worst-case complexity of winner determination and strategic behavior under various voting rules-see e.g., the papers of Hemaspaandra al. (1997), [Doork al. (200T), and Conitzer et al. (2007) but more recent work often combines theoretical investigations with empirical analysis. For example, formal bounds the running time and/or approximation ratio a winner determination algorithm can be complemented by experiments that evaluate its performance realistic instances;
c.g., the works of Conitzer (2006). Betzler et al. 2014). Faliszewski et al. (2018) and Wang et al. (019).
However, performing high-quality experiments requires the ability to organize and understand the available data. One way to achieve this is to form a so-called "map of elections." " recently introduced by /Szufa et al.] (2020) and extended by [Boehmer et al. (2021b). The idea is follows. First, we fix distance measure between elections. Second, we sample a number of elections from various distributions and real-life datasets-edt e.g., those collected in Prefl.ib (Mattei & Walsh 2013) and measure the pairwise distances between them. Third, we embed these elections into the 2D plane, mapping each election to point that the Euclidean distances between points approximately equal to the distances between the respective elections. Finally, we plot these points, usually coloring them to indicate their origin (e.g., the distribution from which given election was sampled); Figure 2 later the paper for an example of such map. A location an election on a map provides useful information about its properties. For example, Szufa et al. (2020) and Boehmer et al. (2021a b) have shown that can be used to predict (a) the Borda score of the winner the election, (b) the
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

running time of ILP solvers computing the winners under the Cemmonicc-orrdee multiwinner voting rule, or (c) the robustness of Plurality and Borda winners. Moreover, real-world elections of the same type (such as the ones from politics, sports, surveys) tend to cluster in the same areas of the map; see also the positions on the map of the datasets collected by Boehmeer & Schaar (2022). As such, the map has proven to be a useful framework to analyze the nature of elections and to visualize experimental results in a -og-aggree fashion.
Unfortunately, extending the map to incorporate additional examples and distributions is a challenging task, as the visual representation becomes cluttered and, more importantly, the embedding algorithms, which map elections to points in 2D, find it more difficult to preserve pairwise distances between points as the number of points increases. It is therefore desirable to reduce the number of points in a way that preserves the key features of the framework.
We address this challenge by drawing map of distributions rather than individual elections, which we call the skeleton map. That is, instead of sampling 20-30 points from each distribution and placing them all on the map, as /Sufa et al. (2020) and Boehhmer al., (202Ib) do (obtaining around 800 points in total), we create a single point for each distribution. This approach is facilitated by the fact that prior work on the "map of elections" framework represented elections by their frequency matrices, which capture their essential features. The starting point of our work is the observation that this representation extends to distributions in a natural way. Thus, if we can compute the frequency matrix of some distribution D, then, instead of sampling elections from D and creating a point on the map for each sample, we can create a single point for D itself.
Our Contribution. We provide three sets of results. First, for a number of prominent vote distributions, we show how to compute their frequency matrices, by providing an explicit formula or an efficient algorithm. Second. we draw the map of distributions (the skeleton map) and argue for its credibility and robustness. Finally, we use our results to estimate the parameters of the distributions that are closest to the real-world elections considered by Boehmer et al. (2021b). In more detail, we work in the setting of preference learning, where we are given an election and we want to learn the parameters of some distribution, so as to maximize the similarity of the votes sampled from this distribution and the input election. For example, we may be interested in fitting the classic model of Mallows ( (1577.. This model is parameterized by central vote v and dispersion parameter Φ, which specifies how likely it is to generate a vote at some distance from the central one (alternatively, one model). Previous works on preference learning typically proposed algorithms to Iearn the parameters of one specific (parameterized) vote distribution (see, c.g., the works Lf & Boutilier 22014); Mandhani MandhaM & Mcilä (200)); Meila & Chen (2010): Vitelli et al. (2017); Murphy & Martin 30 Awasthi et al. (2014) for (mixtures of) the Mallows model and the works of Guiver & Snelson (2009); Hunter (2004); Minka (2004): Gormley & Murphy (2008) for (mixtures of) the Plackett- model). Using frequency matrices, we offer a more general approach. Indeed, compute, the task of learning the distribution's parameters boils down to finding parameters that minimize the distance between the election and the matrices of the distribution. While this minimization problem may be quite challenging, our approach offers a uniform framework for dealing with multiple kinds of distributions at the same time. We find that for the case of the Mallows distribution, our approach learns parameters very similar to those established using maximum likelihood- based approaches. Omitted proofs and discussions are in the appendix. The source code used for the experiments is available in a GitHub repository!
may use, e.g., the Plackett-Luu
given an election and parameterized vote distribution whose frequency matrix
2 Preliminaries
Given an integer t, we write [t] to denote the set t). We interpret a vector Rd as an m x 1 matrix (i.e., we use column vectors as the default).
Preference Orders and Elections. Let C' be finite, nonempty set of candidates. We refer to total orders over C as preference orders (or, equivalently, votes), and denote the set of all preference orders over C by L(C). Given a vote and candidate c, by (c),(c) we mean the position of e in (the top-ranked candidate has position 1, the next one has position 2, and so on). If a candidate a is ranked above another candidate b in vote v, we write a > b. Let rev(v) denote the reverse of
FttpreEEeeeeette-eeuueeeeeeee Frequency-Matrieess Niattrices-NurIPPS. 2022

---

vote v. An election E = (C, V) consists of set C = fei...., Cm of candidates and a collection V (VI,..,,,,, Un) of votes. Occasionally we refer the elements of V as voters rather than votes.
Frequency Matrices. Consider an election E' = (C,V) with C = {cr.... Co. and V (Ul,...,Uy) Un). For each candidate Oj and position E [m]] we define #fregg(Gi i) to be the fraction of the votes from V that rank Oj in position i. We define the column vector #freqg(e,) to be (#rreqg .)...... #freqp (C,, m)) and matrix #freq(E) to consist of vectors #freqg(C1)) (c1)..... # #freeg (Cm). We refer to #freq(E) (E) as the frequency matrix of election E. Fre- quency matrices are bistochastic, i.c., their entries are nonnegative and each of their rows and columns sums up to one.
Example 2.1. Let E = VC be an election with candidate set C' = {a, b, c, d, e} and four voters, 21, U2. 23, and U4. Below, we show the voters' preference orders (on the left) and the election's frequency matrix (on the right).
b
d
1/4
1/4
1/4
1/4
0:: b > c c>>cx 6, 02: c > b > a >
0
1/2
1/4
0
1/4
1/1
0
1/2
1/1
0
0y: d >
b > a,
1/1
1/1
0
1/2
0
U1: b > c a
1/1
0
0
0
3/1
Given vote v, we write #freq(v) denote the frequency matrix of the election containing this vote only; (vreq is permutation matrix, with a single in each row and in each column. Thus, for an election E (C, V) with V (v1 (U,....,uy) we have #freq(E))
E.= #freq(v:).
Compass Matrices. For even m. Boehmer et al. (2021b) defined the following four m x m "compass" matrices, which appear be extreme the "map of elections":
1. The identity matrix, 1Dm, has ones on the diagonal and zeroes everywhere else (it corre- sponds to an election where all voters agree on a single preference order).
2. The uniformity matrix, UNm has all entries equal to 1/m (it corresponds lack of agreement; each candidate is ranked at each position equally often).
3. The stratification matrix, ST is partitioned into four quadrangles, where all entries in the top-left and bottom-right quadrangles are equal to 3/m, and all other entries are equal to zero (it corresponds to partial agreement; the voters agree which half of the candidates is superior, but disagree on everything else).
4. The antagonism matrix, ANm; has values 1/2 on both diagonals and zeroes elsewhere (it captures a conflict: it is matrix of an election where half of the voters rank the candidates in one way and half of the voters rank them in the opposite way).
Below, we show examples of these matrices for m 4:
1/4
1/a
1/4
1
0
0
1/2
1/2
0
0
T/22
0
1/2
1/4
1/4
1/2
1/2
0
0
0
1/2
1/2
0
UN.
ID+
.T.
.NN.
1/4
1/4
0 0
0
0
1/2
1/2
0
1/2
1/2
0
1/4
1/4
1/4
1/4
0
0
0
1/2
1/2
1/2
1/2
We omit the subscript in the names of these matrices if its value is clear from the context or irrelevant.
EMD. Let x (#1...., .,. Zn) and y (y1...., ... 3n.) be two vectors with nonnegative real entries that sum up to 1. Their Earth mover's distance, denoted EMD(x, y). is the cost of transforming into y using operations of the form: Given indices i E [r]] and positive value ò such that x; 2 at the cost of 8 Ji jl. replace with x; 5 and 2, with æÿ + 5 (this corresponds to moving 8 amount of "earth" from position to position j). EMD(x, y) can be computed in polynomial time by a standard greedy algorithm.
Positionwise Distance (Szula et al. 2020). Let A (aI...., am) and B (b1,.... (bI..... bm) be two m x m frequency matrices. Their raw positionwise distance is rawPOS(A) A, B) minges... >x EMD((s be(s), where Sn denotes the set of all permutations over [m]. We will
normalize these distances by {m² - -1), which Boehmer et al. (202bb) 2022) proved to be the maximum distance between two m m frequency matrices and the distance between ID,,, and
3

---

11
01
21
22
12
U3
b
3
(a) Ti
(b) T2
(c) Ta
Figure 1: Three examples of clone decomposition trees.
UNm: nPOS(A, B) Pow.OS((AB) (mm=-1) For elections E and F with equal-sized candidate sets, their positionwise distance, raw or normalized, is defined as the positionwise distance between their frequency matrices.
Paths Between the Compass Matrices. Let X and Y be two compass matrices. Boehmer et al. (202Tb) showed that if we take their affine combination Z a + (1 a)Y (0 <5 S 1) then uPOS(X, Z) = (1 a) a),SO, Y) and AP,S(ZZYY Y) = onPOS(X, Y). Such affine combinations form direct paths between the compass matrices; they also possible between any two frequency matrices of a given size, not just the compass ones, but may require shuffling the matrices' columns (Boehmer et al. 2021b).
Structured Domains. We consider two classes of structured elections, single-peaked elec- tions (Black, 1958), and group-separable elections (Inada 1964). For a discussion of these domains and the motivation behind them, see the original papers and the overviews by Elkind et al. (2017 2022).
Intuitively, election is single-peaked if we can order the candidates that, each voter considers the candidates this order (referred to as the societal axis), his or her appreciation first increases and then decreases. The axis may, e.g., correspond to the left-right political spectrum.
Definition 2.2. Let v be a vote over C' and let be the societal axis over C. We say that is single- peaked with respect to <I if for every & E [[C']] its & top-ranked candidates form interval within <1. An election is single-paake with respect to <t if all its votes are. An election is single-peaked (SP) it is ungle-peaked with respect some axis.
Note that the election from Example 2.1 is single-peaked with respect to the axis 4c4dde
We also consider group- separable elections, introduced by Inada (1964). For our purposes, will be convenient to use the rreebased definition of Karpov (2019). Let C (cI,.... Cm be set of candidates, and consider a rooted, ordered tree whose leaves are elements C. The frontier of such tree is the preference order that ranks the candidates in the order in which they appear in the tree from left to right. A preference order consistent with a given tree if it can be obtained as its frontier by reversing the order in which the children of some nodes appear.
Definition 2.3. An election E = (C, V) is group-separable if there is a rooted, ordered tree T whose leaves are members of C, such that each vote in V is consistent with T.
The trees from Definition 2.3 form a subclass of clone decomposition trees, which are examples of PQ-trees (Elkind et al. 2012 Booth & Lueker 1976).
Example 2.4. Consider candidate set C = a, b, c, d). trees Tı, Tz. and Ts from Figure 1 and votes > b > > d, U2: - d > b > and Us : b > d + c > a. Vote is consistent with each of the trees, U2 is consistent with To (reverse the children of yı and y22,, and V3 consistent with T5 (reverse the children of x1 and I3)).
3 Frequency Matrices for Vote Distributions
We show how compute frequency matrices for several well-known distributions over votes.
3.1 Setup and Interpretation
vote distribution for a candidate set is a function D that assigns probability to each pref- erence order over C. Formally, require that for each e L(C') it holds that D(v) 2 0 and

---

EvEciC) D(v)= = 1. We say that vote e is in the support of D (()) > 0. Given such distribu- tion, we can form an election by repeatedly drawing votes according to the specified probabilities. For example, we can sample each element of ((C) with equal probability; this distribution, which is known as impartial culture (IC), is denoted by Dic (we omit the candidate set from our notation as it will always be clear from the context). The frequency matrix of a vote distribution D over a candidate set C' is (Dreq(D)= = Eicc(cy D(u): #ffrequ). For example, we have #freq(Prc) = UN. One interpretation of #freq is that the entry for candidate Gj and position is the probability that a vote V sampled from 27 has Cj in position (which we denote as P(pos,,(c)) = il). Another interpretation is that if we sample large number of votes then the resulting election's frequency matrix would be close to # #froq(D) (D) with high probabilityy More formally, if we let M, be random
variable equal to the frequency matrix of an novoter election generated according to D. then it holds that lim, E(M,, (Dr.
3.2 Greup-Separable Elections
We first consider sampling group-separable votes. Given a rooted tree T whose leaves are labeled by elements of C' = fci1.- ..... Cm J. let D&s be the distribution assigning equal probability to all votes consistent with T. and zero probability to all other votes; one can think of D as impartial culture restricted to the group-separable subdomain defined by T. To sample from Dis we can toss a fair coin for each internal node of 'T, reversing the order of its children if this coin comes up heads, and output the frontier of the resulting tree. We focus on the following types of trees:
1. Flat(ci..... Cm) is a tree with a single internal node, whose children, from left to right, are C1 C2......................... Sm- There are only two preference orders consistent with this tree, Ci > Cm and its reverse.
2. Bal(c).... .... ..) is perfectly balanced binary tree with frontier C1,..., Cm (hence we assume the number m of candidates to be a power of two).
3. CP(cI..... Cm) is binary caterpillar tree: it has internal nodes 21,,.., 7m for each j E [m 2). xy has e, as the left child and xz+1 as the right one, whereas Fm-1 has both Cm-1 and cm as children.
The first tree in Figure is flat, the second one is balanced, and the third one is caterpillar tree. If T is a caterpillar tree, then we refer to Ds as the GS/caterpillar distribution. We use a similar terminology for the other trees.
Theorem 3.1. Let F' be the frequency matrix of distribution Dis- I is flat then F = AN, and if it balanced then F = UN. IfT is caterpillar tree CP(c Cm). then for each candidate ej the probability that ej appears in a position E [m] random vote v sampled from D is:
liss + 2 (m==)=mm-))) lism-j- W ((=)
Proof. The cases of flat and balanced trees are immediate, so focus caterpillar trees. Let T = (P ) with internal nodes Tı, Tm-1, and consider a candidate cj and position i e [m]. Let v be a random variable equal to a vote sampled from DTs- We say that a node æy. { im 1]. is reversed if the order of its children is reversed. Note that for l < it holds that o, precedes c in the frontier if and only if ze is reversed. Suppose that 2, is not reversed. Then v ranks Cj above each of Cj+1, Cm- This means that for Cj be ranked exactly in position i, it must be that j 2 and exactly - nodes among #1, I5- are not reversed. ] J 2 i, the probability that Iy and i - 1 nodes among .22....,JJJJ are not reversed is 1 ((-1) On the other hand, if xy is reversed, then e ranks candidates Cj+1,...,, Cm above ej. As there are m j of them, for ej to be ranked exactly in position it must hold that i m j j and exactly (i- 1) (m - j) nodes among 31, , -1 are not reversed. This happens with probability + ((=1))((m)))
Regarding distributions Des not handled in Theorem 3.1 we still compute their frequence matrices efficiently.
Theorem 3.2. There is an algorithm that given a tree T computes (sT)) using polynomially many arithmetic operations with respect to the number of nodes in T.
5

---

3.3 From Caterpillars to Single-Peaked Preferences.
There is a relationship between GS/caterpillar votes and single-peaked ones, which will be very useful when computing one of the frequency matrices in the next section.
Theorem 3.3. Given a ranking v over C = {ci,..., let T be another ranking over C such that, for each E [ml, cj is ranked in position in then e2 is ranked in position m - j + in C. Suppose that is in the support of D where T CP(Cl...., Cm). Then i is single-peaked with respect to c < 4 Cm-
There exactly 2 votes in the support of Ds (this follows by simple counting) and there 2m-1 votes that are single-peaked with respect to c1 < <.... < Cm- As u F e implies ū & E, it follows that the mapping + is bijection between all votes in the support of D and all votes that are single-peaked with respect to C1 < ... < Cm-
3.4 Single-Peaked Elections
We consider two models of generating single-peaked elections, one due to Walsh 2015) and one due to Conitzer 2009). , Let us fix a candidate set = {cI...., cmm] and a societal axis ci <] ... <] Cm- Under the Walsh distribution, denoted D Dall each vote that is single-peaken with respect < has equal probability (namely, 2m-r). and all other votes have probability zero. By Theorems 3.1 and 3.3 we immediately obtain the frequency matrix for the Walsh distribution (in short, it is the transposed matrix of the GS/caterpillar distribution).
Corollary 3.4. Consider candidate set C = (mm...., and an axis C1 4..- 40mm The probability that candidate Cj appears in position i in sampled from Dall is: 2m-1+1 (T-T) 1 ijmm-it1 jjsi-1-
To sample a vote from the Conitzer distribution, DSP" (also known as the random peak distribution), pick some candidate Cj uniformly random and rank him or her top. Then perform 1 iterations, where in each we choose (uniformly at random) a candidate directly to the right or the left of the already selected ones, and place him or her in the highest available position in the vote.
Theorem 3.5. Let < ... < Cm be the societal axis, where m is an even number, and let v be random vote sampled from Con for this axis. For j [ and E [m] we have:
2/2m
ifi < j.
(1+1)/m
if = j.
P P(pos, (cj) i]
1/2m
if i m mmjj+1
(m (m-5++)/22m
ifi = m -j+1,
ifit >> m.
Further, for each candidate Cj C' and each position E [m]] we have Plpos, (c;) (ipos,(Cm--jai = i).
3.5
Mallows Model
Finally, we consider the classic Mallows distribution. It has two parameters, a central vote v" over m candidates and dispersion parameter ó E [0, 1]. The probability of sampling vote from this distribution (denoted Di . is: Di (v) = (,(00))") where = (1 + Φ) (1 + d?). ..... +-+..... -=h is normalizing constant and k(U, v²) is the swap distance between v and vz (i.e., the number of swaps of adjacent candidates needed transform into e²). ). In our experiments, we consider a new parameterization, introduced by Boehmer et al. (2021b). It uses a normalized dispersion parameter norm-o. which is converted between the central vote 8* and a sampled vote norm-ó times the maximum swap distance between two votes (so, norm- = 1 is equivalent to IC and for norm-o = 0.5 we get elections that lie close to the middle of the UN-ID path).
value of Φ so that the expected swap distance
Our goal now to compute the frequency matrix of D That is, given the candidate ranked in position in the central vote, want to compute the probability that he she appears in

---

given position E [m]] in the sampled vote. Given positive integer m, consider the candidate set C(m) {cI...... Cm J and the central vote um= Ci > > C.. Fix a candidate cj E (m), and a position E [m]. For every integer k between O and m)/m-1)/2, let S(m, k) be the number of votes in (m) that are at swap distance from t and define T(m, j, i) to be the number of such votes that have ej in position i. One can compute (m,, k) in time polynomial in m (OEIS Foundation Inc. 2020 using Sm,, k), we show that the same holds for T(m, j.
Lemma 3.6. There is an algorithm that computes T(m, k. j. in polynomial time with respect to m.
We can now express the probability of sampling a vote v, where the candidate ranked in position j in the central vote v* ends up in position i under Diai as:
j. = Ex=o mmm 1//2 ((m, J. 1)
The correctness follows from the definitions of T and Di & By Lemma 3.6 we have the following.
Theorem 3.7. There exists an algorithm that, given a number m of candidates, a vote v", and parameter @ computes the frequency matrix of Diial using polynomially many operations in m.
only depends on c. j and (and, naturally, on m). Using this fact, we can also compute frequency matrices for several variants of the Mallows distribution.
Note that Equation IT)
Remark 3.8. Given a vote v, two dispersion parameters á and db and a probability E [0, 1]. we define the distribution t. as p- Dir + (1 - p) Mal ( i.e., with probability p we sample a vote from Dita and with probability 1 p sample a vote from DMr (r The probability that candidate Cy appears in position in the resulting vote is p- fm i) + (1 (1-p)- fmm-pppmmiii fm( m --+1, i).
Remark 3.9. Consider candidate set C' = (c, C. Given vote distribution D over C(C) and a parameter o, define a new distribution D' as follows: Draw a vote i according to D and then output vote sampled from DMm: indeed, such models are quite natural, see, e.g., the work of Kenig& & Kimelfeld (2019). For each E [ma], let g(i be the probability that ej appears in position E in a vote sampled from 'D. The probability that ej appears in position i E [m]] in vote sampled from D' is Em
f
In terms of matrix multiplication, this means that
(D") #freq (D Mai . (Dreq(D), where v² i; ..... Cm. We write ø-Conitzer (ó-Walsh) to refer to this model where we use the Conitzer (Walsh) distribution as the underlying one and normalized dispersion parameter sb
4 Skeleton Map
Our goal in this section is to form what we call skeleton map of vote distributions (skeleton map, for short), evaluate its quality and robustness, and compare it to the map of Bochmer et al. (0011b) Throughout this section, whenever the positionwise distance (occasionally we will also refer to the Euclidean distances on our maps, but we will always make this explicit). Let Φ- f0, 0.05, 0.1...., 1} be a set of normalized dispersion parameters that we will be using for Mallows-based distributions in this section.
speak of distance between elections or matrices, mean
We form the skeleton map following the general approach of [zufa et al. (2020) and Boehmer jet al.) (202Tb). For a given number of candidates, we consider the four compass matrices (UN, ID. AN, ST) and paths between each matrix pair consisting of their convex combinations (gray dots), the frequency matrices of the Mallows distribution with normalized dispersion parameters from & (blue triangles), and the frequency matrices of the Conitzer (CON), Walsh (WAL), and GS/caterpillar distribution (CAT). Moreover, we add the frequency matrices of the following vote distributions (we again use the dispersion parameters from b): (i) The distribution 1/2-D Mal as defined in Remark 3.8 (red triangles), (ii) the distribution where with equal probability we mix the standard Mallows distribution and 1/2-D%...0 (green triangles), and (iii) the ő-Conitzer and é --alls distributions as defined in Remark 3.9 (magenta and orange crosses). For each pair of these matrices we compute their positionwise distance. Then we find an embedding of the matrices into a 2D plane, so that each matrix is a point and the Euclidean distances between these points are as similar to the positionwise distances as possible (we use the MDS algorithm, as implemented in the Python skleam..n kklearm.mniifold. www.mold. MDS package). In Figure 3 we show map for the case of 10 candidates (the
7

---

18
A3
4g
055
8.69 0.65
MIGHENN
Day 967
K/500
30 2L0
MID
9.2
Roy
6.33
8o 9.0
6.35
16:44
6.00
Ray
8.45
92)
0.a
For
R2
Ray
Figure 3: The skeleton map with 10 candidates. We have MID 1/2AN + 1/2ID. Each point la-
Figure 4: In the top-right part. we show the normalized posi- tionwise distance. In the bottom-
Figure 2: Map of elections, to- gether with the compass matri- ces, as presented by [Boehmer et al. 2021bb.
beled with a number is real-
left one, we show the embedding misrepresentation ratios.
world election as described in Section 5
lines between some points/matrices show their positionwise distances; to maintain clarity, we only provide some of them).
We now verify the credibility of the skeleton map. As the map does not have many points, expect its embedding to truly reflect the positionwise distances between the matrices. This, indeed, seems to be the case, although some distances In Figure 4 we provide the following data for a number of matrices (for m = 10; matrix M2W is the Mallows matrix in our data set that is closest to the Walsh matrix). In the top-right part (the white-orange area), we give positionwise distances between the matrices, and the bottom-left part (the blue area), for each pair of matrices X and Y we report the misrepresentation ratio Euc(X.Y) YP)S(X.Y) where Euc (X, Y) the Euclidean distance between X and Y in the embedding, normalized by the Euclidean distance between ID and UN. The closer they are to 1. the more accurate is the embedding. The misrepresentation ratios are typically between 0.8 and 1.15, with many of them between 0.9 and 1.05. Thus, in most cases, the map is quite accurate and offers good intuition about the relations between the matrices. Yet, some distances are represented particularly badly. As an extreme example, the Euclidean distance between the Walsh matrix and the closest Mallows matrix, M2W, is off by almost a factor (these matrices are close, but not as close as the map suggests). Thus, while one always has to verify claims suggested by the skeleton map, we view it quite credible. This conclusion is particularly valuable when we compare the skeleton map and the map of Boehhmer et al. (2021b), shown in Figure [ The two maps are similar, and analogous points (mostly) appear in analogous positions. Perhaps the biggest difference is the location of the Conitzer matrix on the skeleton map and Conitzer elections in the map of Boehmer et al.) but even this difference is not huge. We remark that the Conitzer matrix is closer to UN and AN than to ID and ST, whereas for the Walsh matrix the opposite is true. Boehmer al. (2021b) make similar observation; our results allow us to make this claim formal. In Appendix E we analyze the robustness of the skeleton map with respect to varying the number of candidates. We find that except for pairs including the Walsh or GS/caterpillar matrices, which "travel" on the map as the number of candidates increases, the distance between each pair of matrices in the skeleton map stays nearly constant.
represented (much) more accurately than the others.
Learning Vote Distributions
We demonstrate how the positionwise distance and frequency matrices can be used to fit vote distributions to given real-world elections. Specifically, we consider the Mallows model ( and the ø-Conitzer and &- Walsh models. Naturally, we could use more distributions, but we focus on showcasing the technique and the general unified approach. Concerning our results, among others, we verify that for Mallows model our approach is strongly correlated with existing maximum- likelihood approaches. Moreover, unlike in previous works, we compare the capabilities different distributions to fit the given elections. We remark that if we do not have an algorithm for computing a frequency matrix of given vote distribution, we can obtain an approximate matrix by sampling sufficiently many votes from this distribution. In principle, it is also possible to deal with distributions
8

---

Table 1: Closest distributions for seven illustrative real-world elections. For each, we include the parameterization that produces the closest frequency matrix, and give the normalized positionwise distance of the elections from this matrix.
id
source
Mal
Ø- Walsh
Φ O-Conitzer
dist
Φ
dist
dist
f. skate 0.44
0.32
0.14
0.38
0.11
0.44
skate
0.23
0.24
0
0.36
0.02
0.56
aspen
0.68
0.18
0.3
0.16
0.23
0.27
f. skate
0.05
0.09
0
0.37
0.69
skate
0.46
0.11
0.15
0.18
0.16
0.35
irish 0.75
0.1
0,42
0.12
0.36
0.16
cities
0.93
0.06
0.69
0.06
0.63
0.06
over elections that do not correspond to vote distributions and hence are not captured by expected frequency matrices (as is the case, e.g., for the Euclidean models where candidates do not have fixed positions; see the work of Szuua et al. (2020) for examples of such models in the context of the map of elections): we want to compute the distance of such distribution, we sample sufficiently many elections and compute their average distance from the input one. However, it remains unclear how robust this approach is.
Approach. To fit our vote distributions to a given election, we compute the election's 's distance to the frequency matrices of D éi- d-Coonizer, and 6- Walsh, for e E 0..0......... 1}. We select the distribution corresponding to the closest matrix.
Data. We consider elections from the real-world datasets used by Boehmer et al. 2021 They generated 15 elections with 10 candidates and 100 voters (with striet preferences) from each of the eleven different real-world election datasets (so, altogether, they generated 165 elections, most of them from Preflib (Mateei & Walsh 2013). They used four datasets of political elections (from North Dublin (Irish), various non-profit and professional organizations (ERS), and city council elections from Glasgow and Aspen), four datasets of sport-based elections (from Tour de France (TDF), Giro d'Italia (GDI), speed skating, and figure skating) and three datasets with survey-based elections (from preferences over T-shirt designs, sushi, and cities). We present the results of our analysis for seven illustrative and particularly interesting elections in Table T and also include them in our skeleton map from Figure
Basic Test. There is a standard maximum- likelhhood estimator (MLE; based on Kemeny vot- ing (Mandhani & Meilä 2009)) that given an election provides the most likely dispersion parameter of the Mallows distribution that might have generated this election. To test our approach, pared the parameters provided by our approach and by the MI_E for our 1 65 elections and found them be highly correlated (with Pearson correlation coefficient around 0.97). In particular, the average absolute difference between the dispersion parameter calculated by our approach and the MLE is only 0.02. See Appendix [F for details.
Fitting Real-World Elections. Next, we consider the capabilities of Diva d-Conitzer, and ø- Walsh to fit the real-world elections of Boehmer al. (202Tb). Overall, find that these three vote distributions have some ability to capture the considered elections, but it certainly is not perfect. Indeed, the average normalized distance of these elections to the frequency matrix of the closest distribution is 0.14. To illustrate that some distance is to be expected here, we mention that the average distance of an election sampled from impartial culture (Dic, with 10 candidates and 100 voters) to the distribution' 's expected frequency matrix is 0.09 (see Appendix EE.4 for a discussion of this and how it may serve as an estimator for the "variance of a distribution") There are also some elections that are not captured by any of the considered distributions to an acceptable degree; examples of this are elections nr. and nr. 2, which are at distance at least 0.32 and 0.25 from all our distributions, respectively. Remarkably, while coming from the same dataset, elections nr. 1 and nr. 2 are still quite different from each other and, accordingly, the computed dispersion parameter also quite different. It remains a challenge to find distributions capturing such elections.

---

Comparing the power of the three considered models, nearly all of our elections are best captured by the Mallows model rather than d-Conitzer or Φ- o-Walsh. There are only twenty elections that are closer to Φ- Walsh or d-Connizer than to Mallows model (election nr. 3 is the most extreme example), and, unsurprisingly, both Øi Walsh and øi-Cnnizer perform particularly badly at capturing elections close to ID (see election nr. 4). That is, ø-Conitzer and Φ- B--lllsh are not needed to ensure good coverage the space of elections; the average normalized distance of our elections to the closest Mallows model is only 0.0007 higher than their distance to the closest distribution (elections 3-6 are three examples of elections which are well captured by the Mallows model and distributed over the entire map) Nevertheless, 6-Walsh is also surprisingly powerful, the average normalized distance of our elections to the closest Ø- Walsh distribution is only 0.03 higher than their distance to the closest distribution (however, this might be also due to the fact that most of the considered real-world elections fall into the same area of the map, which &- Walsh happens to capture particularly well (Boehmer et al. 2021b)). d-Conitzer performs considerably worse: there are only three elections for which produces (slightly) better result than Ø- Walsh.
Moreover, our results also emphasize the complex nature of the space of elections: Election nr. 7 is very close to D hinting that its votes are quite chaotic. At the same time, this election is very close to 1.63-Conitzer and 0.69-Walsh distributions, which suggests at least a certain level structure among its votes (because votes from Conitzer and Walsh distributions Mallows filter with dispersion between 0.63 and 0.69 does not destroy this structure fully). However, witnessed by the fact that the frequency matrix of GS/balanced (which is highly structured) is UN, such phenomena can happen. Lastly, note that most of our datasets are quite "homogenous" in that the closest distributions for elections from the dataset are similar and also However, there are also clear exceptions, for instance, elections nr. and nr. 4 from the figure skating dataset. Moreover, there two elections from the speed skating dataset where one election is captured best by Dr.0.76 and the other by D D..0.32
very structured, and the
similar distance.
Summary
We have computed the frequency matrices (Szufa al. 2020 Boehmer al. 202 of several well-known distributions of votes. Using them, we have drawn "skeleton map" which shows how these distributions relate to each other, and we have analyzed its properties. Moreover, we have demonstrated how our results can be used to fit vote distributions to capture real-world elections.
For future work, it would be interesting to compute the frequency matrices of further popular vote distributions, such as the Plackett- Luce model (we conjecture that its frequency matrix is computable in polynomial time). It would also be interesting such as mixtures of Mallows models, to real-world elections. Further, it may be interesting to use expected frequency matrices to reason about the asymptotic behavior of our models. For example, might be possible to formally show where, in the limit, do the matrices of our models end up on the map as we increase the number of candidates.
our approach to fit more complex models,
Acknowledgments
NB was supported by the DFG project MaMu (NI 369/19) and by the DFG project ComSoc-MPMS (NI 369/22). This project has received funding from the European Research Council (ERC) under the European Union's 's Horizon 2020 research and innovation programme (grant agreement No 101002854).
erc
"For each election, also computed the closest frequency matrix of two mixtures of Mallows models with reversed central votes p.Di. by around 0.02, with the probability p of tlipping the central vote being (close to) zero for most elections.
".d.9) using our approach. However, this only decreased the average minimum distance
10

---

References
Awasthi, P., Blum, A., Sheffet, O., and Vijayaraghavan, A. Learning mixtures of ranking models. In Proceedings of NIPS-2014, pp. 1609-2617, 2014.
Betzler, N., Bredereck, R., and Niedermeier, R. Theoretical and empirical evaluation of data for exact Kemeny rank aggregation. Autonomous Agents and Multiagent Systems, 28(5):721-748, 2014.
Black, D. The Theory of Committees and Elections. Cambridge University Press, 1958.
Boehmer, N. and Schaar, N. Collecting, classifying, analyzing, and using real-world elections. Technical Report arXiv:2204.03589 [cs. [cs.GT], arXiv.org, 2022.
Boehmer, N., Bredereck, R., Faliszewski, P., and Niedermeier, R. Winner robustness via swap- and shift-bribery: Parameterized counting complexity and experiments. In Proceedings of IJCAI-2021, pp. 52-58,22 2021a.
Boehmer, N., Bredereck, R., Faliszewski, P., Niedermeier, R., and Szufa, S. Putting compass on the map elections. In Proceedings of IJCAI-2021, pp. 59-65, 2021b.
Boehmer, N., Faliszewski, P., Niedermeier, R., Szufa, S., and Was, T. Understanding distance measures among elections. In Proceedings of IJCAI-2022, pp. 102-108. ijcai.org, 2022.
Booth, K. and Lueker, G. Testing for the consecutive ones property, interval graphs, and graph planarity using PQ-tree algorithms. Journal of Computer and System Sciences, 13(3):335-379 1976.
Brandt, F., Conitzer, V., and Endriss, U. Computational social choice. In WeiB, G. (ed.), Multiagent Systems. MIT Press, 2013.
Conitzer, V. Computing Slater rankings using similarities among candidates. In Proceedings of AAAI-2006, pp. 613-619, July 2006.
Conitzer, V. Eliciting single-peaked preferences using comparison queries. Journal of Artificial Intelligence Research, 35:161-191, 2009.
Conitzer, V., Sandholm, T., and Lang, J. When are elections with few candidates hard to manipulate? Journal of the ACM, 54(3)cArticee 14, 2007.
Dwork, C., Kumar, R., Naor, M., and Sivakumar, D. Rank aggregation methods for the web, In Proceedings of WWW-2001, pp. 613-622, 2001.
Elkind, E.,, Faliszewski, P., and Slinko, A. Clone structures voters' preferences. In Proceedings of EC-2012, pp. 496-513, 2012.
Elkind, E., Lackner, M., and Peters, D. Structured preferences. In Endriss, U. (ed.), Trends in Computational Social Choice. Al Access Foundation, 2017.
Elkind, E., Lackner, M., and Peters, D. Preference restrictions in computational social choice: A survey, May 2022.
Faliszewski, P., Lackner, M., Peters, D., and Talmon, N. Effective heuristics for committee scoring rules. In Proceedings of AAAI-2018, pp. 1023-1030, 2018.
Faliszewski, P., Karpov, A., and Obraztsova, S. The complexity of election problems with group- separable preferences. In Proceedings of IJCAI-2020, pp. 203-209, 2020.
Gormley, I. C. and Murphy, T. B. Exploring voting blocs within the irish electorate: A mixture modeling approach. Journal of the American Statistical Association, 103(483):1014-102, 2008
Guiver, and Snelson, E. L.. Bayesian inference for plackett-luce ranking models. In Proceedings of ICML-2009, pp. 377-384. ACM, 2009.
Hemaspaandra, E., Hemaspaandra, I.,, and Rothe, J. Exact analysis of Dodgson elections: Lewis Carroll's 1876 voting system complete for parallel access NP. Journal of the ACM, 44(6): 806-825, 1997.
11

---

Hunter, D. R. MM algorithms for generalized bradley-terry models. The Annals of Statistics, 32(1): 384-406, 2004.
Inada, K. note on the simple majority decision rule. Econometrica, 32(32):525-531, 1964.
Karpov, A. On the number of group-separable preference profiles. Group Decision and Negotiation, 28(3):501-5133 2019.
Kenig, B. and Kimelfeld, B. Approximate inference of outcomes in probabilistic elections. In Proceedings of AAAI-2021, pp. 2061-2068, 2019.
Lu, T. and Boutilier, C. Effective sampling and learning for mallows models with pairvise-preference data. Journal of Machine Learning Research, 15(1):3783-3885 2014.
Mallows, C. Non-null ranking models. Biometrica, 44:114, 1957.
Mandhani, B. and Meilä, M. Tractable search for learning exponential models of rankings. Journal of Machine Learning
Track, 5:392-399, 2009.
Mattei, N. and Walsh, T. Preflib: A library for preferences. In Proceedings of ADT-2013, pp. 259-270. 2013.
Meila, M. and Chen, H. Dirichlet process mixtures of generalized mallows models. In Proceedings of UAI-2010, pp. 358-367. AUAI Press, 2010.
Minka, Power EP. Technical report, Microsoft Research, 2004.
Murphy, B. and Martin, D. Mixtures of distance- based models for ranking data. Computational Statistics & Data Analysis, 41(3-4):64--655, 2003.
OEIS Foundation Inc. The on-line encyclopedia of integer sequences, 2020. URL http://oiis.rr// A008302
Szufa, S., Faliszewski, P., Skowron, P., Slinko, A., and Talmon, N. Drawing a map of elections in the space of statistical cultures. Proceedings of 2020,-2020, pp. 1341-1349, 2020.
Vitelli, V., Sørensen, Ø.,, Crispino, M., Frigessi, A., and Arjas, Probabilistic preference learning with the Mallows rank model. Journal of Machine Learning Research, 18:158.1-158:44 2017.
Walsh, T. Generating single peaked votes. Technical Report arXiv:1503.02770 [cs.GT], arXiv.org. 2015.
Wang. J., Sikdar, S., Shepherd, T., Zhao, Z., Jiang, C., and Xia, L. Practical algorithms for multi-stage voting rules with parallel universes tiebreaking. In Proceedings of AAAI-202/, pp. 2189-2196. 2019.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [N/A]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [No]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "xzrl88-jJ2v", "reviewer": "Reviewer_vnFh", "summary": "Note: I have reviewed a previous submission of this paper at a previous conference.\n\nThis paper continues a recent line of work aimed at identifying relationships between many different vote distributions. Where previous work used frequency matrices to measure the difference between sampled elections from given distributions this paper introduces a very natural extension to the concept and applies the frequency matrices to the vote distributions themselves.\n\nThe paper describes a number of election structures (single-peaked, group-seperable, Mallows) and finds the frequency matrix, or a polynomial time formula for generating the matrix, for each distribution. Using these frequency matrices a new map of elections is generated. This \"skeleton map\" strengthens and confirms prior work and is shown to generally represent the distance between elections well. Finally, Mallows models are generated for elections representing real-world data and placed on the map. These also have some similarity to prior work however they find generating a distribution to perfectly match each real election to be difficult. Some promising potential future work is identified while concluding.", "questions": "Can you discuss why this paper was not previously accepted and how it has changed since then?\n\nTiny issue:\nLine 290 misspells \"Conitzer\" as \"Cointzer\"", "limitations": "The authors have not discussed the potential societal impact of their work. While the contribution appears quite far removed from any possible real-world impact perhaps one of the many appendices could be used to briefly imagine how this work could be misused somewhere down the line?", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper represents a novel addition to a recent series of papers. This approach of generalizing from sampled data to comparing entire distributions could also have potential uses in other domains. The work is well written and does a very good job of connecting itself to the prior results in this line of research. While the results are moderately complex I found them to be explained and structured quite clearly.\n\nOverall I find the paper fairly strong and have no major issues with it. Due to space limitations some of the figures (particularly Fig 2) are rather small and difficult to read. I am glad to see that most of the specific, minor changes I have previously suggested have been fixed.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "YqroHDzKOsb", "reviewer": "Reviewer_Y3YX", "summary": "Previously, Szufa et al. (2020) and Boehmer et al. (2021) proposed \"map of elections\" as both a visualization tool to group similar elections, but also demonstrated that those elections close to each other on the map may have similar features.This paper introduces \"skeleton map\", which is a new type of \"map of elections\". The main difference is that in the skeleton map, each point is a distribution instead of a single election. This change simplifies the map while preserving the key features. In order to represent each election / vote distribution as a point, this paper studies a number of prominent vote distributions, either gives an analytical formula of the frequency matrices, or provides a polynomial time algorithm to calculate it. The authors ran experiments to see how the distances between different vote distributions change when the number of candidates change. Finally, they use real life election datasets to verify that the vote distributions they consider (Mallows, Conitzer, Walsh with different parameters) could fit some real vote datasets well, but still have their limits.\n", "questions": "* On line 20: “However, performing high-quality experiments requires the ability to organize and understand the available data.” Could you elaborate on this? How the “map of elections” and “skeleton map” help experiments?\n* For the frequency matrix calculation, I wonder if the results could be used in general above the context of this paper?\n* Do you know whether your method in this paper is better than previous work? See my comments in “Strengths And Weaknesses”. \n", "limitations": "* It would be nice if the frequency matrix calculation could be used beyond the context in this paper. \n* Again, my main concern is how this paper compares with previous papers on fitting real life datasets.\n", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I have previously reviewed another version of this paper.\n\n* Originality: This work is well built on two previous works: Szufa et al. (2020) that proposed the \"map of elections\", and Boehmer et al. (2021) that defined compass matrices and normalized positionwise distance. They did a thorough work providing algorithms to compute the frequency matrices for many vote distributions. The \"skeleton map\" is a nice simplification to only have one point for each distribution instead of one point for each election.\n* Quality: The paper is well organized. Concepts and algorithms are clearly defined, proved, and elaborated. Experiments and analysis of results are well presented. Source code is provided. My concern is that I did not find experiments showing this new method performs better than previous work.\n* Clarity: This paper is well written and organized.\n* Significance: My main concern is whether representing vote distributions by frequency matrices is better (in the sense that it fits better with real life datasets) than previous work that represents vote distributions by sampling elections from it. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Llf2TFQJf9F", "reviewer": "Reviewer_m4hd", "summary": "The authors build on recent work on a very exciting \"map of elections\" by showing how to compute the frequency matrix of various vote distributions. They then use these frequency matrices to produce a \"skeleton map\" of distributions that is closely related to, and visually much simpler than, the \"map of elections\" approach. Lastly, they show that they can use frequency matrices to estimate parameters of real-world election data (based on nearest distributions).\n", "questions": "1. Line 22: what is eu?", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n+ The \"map of elections\" is a remarkably useful and insightful tool when dealing with real-world elections, and having a \"skeleton map\" like this (that doesn't rely on sampling individual instances) is a very useful addition. \n+ I particularly appreciated that this skeleton map allows us to learn parameters of real-world elections (or approximations thereof).\n+ The sampling results / algorithms are intuitively presented and clear to the reader (despite the fact that the results are quite nontrivial).\n\nWeaknesses: \n- I am not overly familiar with the background behind the map of elections (beyond seeing it as a very useful tool), and it seems like the whole area is built upon many heuristics that seem to work pretty well but lack robust theoretical justification. This is perhaps not so much of a downside to me (relative to others) but is definitely something worth thinking about.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept