Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: Note: I have reviewed a previous submission of this paper at a previous conference.

This paper continues a recent line of work aimed at identifying relationships between many different vote distributions. Where previous work used frequency matrices to measure the difference between sampled elections from given distributions this paper introduces a very natural extension to the concept and applies the frequency matrices to the vote distributions themselves.

The paper describes a number of election structures (single-peaked, group-seperable, Mallows) and finds the frequency matrix, or a polynomial time formula for generating the matrix, for each distribution. Using these frequency matrices a new map of elections is generated. This "skeleton map" strengthens and confirms prior work and is shown to generally represent the distance between elections well. Finally, Mallows models are generated for elections representing real-world data and placed on the map. These also have some similarity to prior work however they find generating a distribution to perfectly match each real election to be difficult. Some promising potential future work is identified while concluding.
strengths_and_weaknesses: The paper represents a novel addition to a recent series of papers. This approach of generalizing from sampled data to comparing entire distributions could also have potential uses in other domains. The work is well written and does a very good job of connecting itself to the prior results in this line of research. While the results are moderately complex I found them to be explained and structured quite clearly.

Overall I find the paper fairly strong and have no major issues with it. Due to space limitations some of the figures (particularly Fig 2) are rather small and difficult to read. I am glad to see that most of the specific, minor changes I have previously suggested have been fixed.


questions: Can you discuss why this paper was not previously accepted and how it has changed since then?

Tiny issue:
Line 290 misspells "Conitzer" as "Cointzer"
limitations: The authors have not discussed the potential societal impact of their work. While the contribution appears quite far removed from any possible real-world impact perhaps one of the many appendices could be used to briefly imagine how this work could be misused somewhere down the line?
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors build on recent work on a very exciting "map of elections" by showing how to compute the frequency matrix of various vote distributions. They then use these frequency matrices to produce a "skeleton map" of distributions that is closely related to, and visually much simpler than, the "map of elections" approach. Lastly, they show that they can use frequency matrices to estimate parameters of real-world election data (based on nearest distributions).

strengths_and_weaknesses: Strengths: 
+ The "map of elections" is a remarkably useful and insightful tool when dealing with real-world elections, and having a "skeleton map" like this (that doesn't rely on sampling individual instances) is a very useful addition. 
+ I particularly appreciated that this skeleton map allows us to learn parameters of real-world elections (or approximations thereof).
+ The sampling results / algorithms are intuitively presented and clear to the reader (despite the fact that the results are quite nontrivial).

Weaknesses: 
- I am not overly familiar with the background behind the map of elections (beyond seeing it as a very useful tool), and it seems like the whole area is built upon many heuristics that seem to work pretty well but lack robust theoretical justification. This is perhaps not so much of a downside to me (relative to others) but is definitely something worth thinking about.


questions: 1. Line 22: what is eu?
limitations: Yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: Previously, Szufa et al. (2020) and Boehmer et al. (2021) proposed "map of elections" as both a visualization tool to group similar elections, but also demonstrated that those elections close to each other on the map may have similar features.This paper introduces "skeleton map", which is a new type of "map of elections". The main difference is that in the skeleton map, each point is a distribution instead of a single election. This change simplifies the map while preserving the key features. In order to represent each election / vote distribution as a point, this paper studies a number of prominent vote distributions, either gives an analytical formula of the frequency matrices, or provides a polynomial time algorithm to calculate it. The authors ran experiments to see how the distances between different vote distributions change when the number of candidates change. Finally, they use real life election datasets to verify that the vote distributions they consider (Mallows, Conitzer, Walsh with different parameters) could fit some real vote datasets well, but still have their limits.

strengths_and_weaknesses: I have previously reviewed another version of this paper.

* Originality: This work is well built on two previous works: Szufa et al. (2020) that proposed the "map of elections", and Boehmer et al. (2021) that defined compass matrices and normalized positionwise distance. They did a thorough work providing algorithms to compute the frequency matrices for many vote distributions. The "skeleton map" is a nice simplification to only have one point for each distribution instead of one point for each election.
* Quality: The paper is well organized. Concepts and algorithms are clearly defined, proved, and elaborated. Experiments and analysis of results are well presented. Source code is provided. My concern is that I did not find experiments showing this new method performs better than previous work.
* Clarity: This paper is well written and organized.
* Significance: My main concern is whether representing vote distributions by frequency matrices is better (in the sense that it fits better with real life datasets) than previous work that represents vote distributions by sampling elections from it. 

questions: * On line 20: “However, performing high-quality experiments requires the ability to organize and understand the available data.” Could you elaborate on this? How the “map of elections” and “skeleton map” help experiments?
* For the frequency matrix calculation, I wonder if the results could be used in general above the context of this paper?
* Do you know whether your method in this paper is better than previous work? See my comments in “Strengths And Weaknesses”. 

limitations: * It would be nice if the frequency matrix calculation could be used beyond the context in this paper. 
* Again, my main concern is how this paper compares with previous papers on fitting real life datasets.
review_title: null

Meta-review: metareview: This paper works to identify relationships among different vote distributions. This is done by applying (previously introduced) "frequency matrices" to the vote distributions themselves and gives formula or algorithms for computing these.  The resulting "map of elections" seems to have especially strong real-world potential.
confidence: Certain
Decision: Accept