Title: Impartial Selection with Predictions
Venue: NeurIPS 2025
Accepted: yes
Keywords: impartial selection

mechanisms with predictions
Abstract: We study the selection of agents based on mutual nominations, a theoretical problem with many applications from committee selection to AI alignment. As agents both select and are selected, they may be incentivized to misrepresent their true opinion about the eligibility of others to influence their own chances of selection. Impartial mechanisms circumvent this issue by guaranteeing that the selection of an agent is independent of the nominations cast by that agent. Previous research has established strong bounds on the performance of impartial mechanisms, measured by their ability to approximate the number of nominations for the most highly nominated agents. We study to what extent the performance of impartial mechanisms can be improved if they are given a prediction of a set of agents receiving a maximum number of nominations. Specifically, we provide bounds on the consistency and robustness of such mechanisms, where consistency measures the performance of the mechanisms when the prediction is correct and robustness its performance when the prediction is incorrect. For the general setting where up to $k$ agents are to be selected and agents nominate any number of other agents, we give a mechanism with consistency $1-O\big(\frac{1}{k}\big)$ and robustness $1-\frac{1}{e}-O\big(\frac{1}{k}\big)$. For the special case of selecting a single agent based on a single nomination per agent, we prove that $1$-consistency can be achieved while guaranteeing $\frac{1}{2}$-robustness. A close comparison with previous results shows that (asymptotically) optimal consistency can be achieved with little to no sacrifice in terms of robustness.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nImpartial Selection mechanisms are voting mechanisms that are largely used to select agents based on nominations provided by the agents themselves. They have been very-well studied in the community of Social Choice and are used in several real contexts, such as committee selection. \nThis paper starts the study of such voting mechanisms in a setting where the mechanism receives in input  a “suggestion” of an optimal solution but the precision of the suggestion is not guaranteed. In the last years the area of mechanism design with prediction received a lot of interests and this framework has been applied to a long list of important problems. The increasing interest on this framework is due to the possibility of using it to model data-driven mechanisms, where the mechanism may receive suggestions from an expert or from some LLMs.\n\nThe paper focuses on the question of whether impartial selection mechanisms can be improved if they are given in input a prediction of a set of agents receiving a high number of nominations. Extensions of known mechanisms to the setting of mechanisms with prediction are presented both for the general setting, where each agent can present an arbitrary number of nominations and k agents have to be elected, and for more restricted settings. For all these settings bounds are given on the consistency and robustness.\n\nSTRENGTHS AND WEAKNESSES:\nSTRENGHTS\nThe paper studies an important (known) problem in the social choice literature, with significant practical applications, in the setting of mechanisms with prediction. The argument perfectly fits the scopus of the conference. \nThis is a solid theoretical paper, with interesting contributions that adapt previously known mechanisms to the new setting of mechanisms with prediction.\nThe quality of writing is pretty good.\n\nWEAKNESSES\nThe contribution of the paper is exclusively theoretical and mainly incremental. It takes known mechanisms for impartial selection without prediction and adapts them to the setting of mechanisms with prediction. \n\nThe extensions are quite trivial but the analysis is not so simple and authors had to solve some technical challenges. Technical proofs are completely moved to the appendix and very few elements are given in the paper to convince the reader of the soundness of the statements. Moreover, appendix is simply a list of proofs of lemmas and theorems with no intuition of how the argument works.\n\nAn experimental analysis to validate the practical effectiveness of the proposed mechanisms is totally missing and left as future work.\n\nQUESTIONS:\n1.\tWhat is the meaning of the superscript used in your notation? (see for example lines 171-172). In general, I think your notation is not intuitive and too heavy. \n2.\tAt line 178, which is the definition of the function f and its relation to the f_i? Since f_i is a conditional probability, why don’t use the traditional notation for probabilities?\n3.\tAt line 216, I suggest to directly denote the permutation as \\pi(x), as done at line 219. Set S is implicitly defined by the vector x and you can remove it from the inputs to the mechanism.\n4.\tAt lines 229-231: your assumption is not sufficient to guarantee that the predicted vertex i will be the last vertex in the permutation and the mechanism can see all its incoming. For example, assume that x_j = 1 and i<j. Why is the statement at line 231 true? (Similar observations hold for the selection mechanisms proposed in the following sections for k>1).\n5.\tIn section 6 you give bounds on consistency and robustness achievable by impartial mechanisms only for k<=3. Any idea on what happens for k>3? Why is your argument not working?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nConsider as input a directed graph with n vertices. The goal is to randomly select k vertices that approximately maximize the sum of in-degrees. The sum of in-degrees corresponds to the utilitarian welfare that the k vertices (agents) provide to the population of n vertices (agents), where an edge from agent i to j means agent i approves of j being selected. Additionally, as part of the input, we are given k proposed vertices S, which are predicted to form a good committee. Now we want to randomize in such a way that:\n (1) if the prediction is good, then the returned outcome is (almost) as good as the prediction (2) is the prediction is bad, then the outcome should not be (much) worse than if we had not seen the prediction and (3) an agent's probability of being selected should be independent of her votes/ her out degree - in other words the mechanism should be impartial.\n\n\nIn this setting, the authors prove the following results:\nk=1: \nA baseline algorithm for this problem is randomizing between the prediction and the best known algorithm without prediction and achieves \\rho-consistency and (1-\\rho) robustness. The authors show that this is tight. However, this algorithm has the property that if every node points to the same node v, then still the algorithm would only select v with constant probability, if v is not the predicted vertex. Thus the authors propose an alternative mechanism that selects such a node v w.p. 1-O(1/n) and otherwise has the same consistency and robustness guarantees as the baseline. They call the new mechanism \\rho-permutation mechanism.\n\nk=2:\nThe authors extend the existing deterministic mechanism to the fixed bidirectional permutation mechanism, which is impartial, 1-consistent and 1/2-robust.\n\nk>=:3 The authors introduce a new mechanism called \\rho-partition, parametrized by \\rho that is impartial and is alpha consistent and beta-robust for \\alpha and \\beta which for large k tend to 1 and 1-1/e respectively. This algorithm is inspired by the best known k-selection algorithm.\n\nThe authors also prove results on how large the alpha and beta can be for an impartial randomized mechanism for k=1,2,3.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is very well written and structured. Incorporating prediction in algorithms in light of the arrival of LLMs, which may do better by following strange black-box heuristics, has been a recent hot topic.\nThe AI-Safety motivation in the introduction is very interesting, albeit the detailed mapping to the problem would need to be elaborated upon.\nMoreover, the paper has technically quite non-trivial results and makes good progress in this direction, with the main result being for k ≥ 3.\nOne weakness is that the baseline algorithm for k = 1 already achieves the desirable optimal trade-off between robustness and consistency.\nIn general, I think this is a good paper, if not for one conceptual concern regarding the motivation:\n\nEspecially in optimization problems, I believe using the prediction framework, given that we are in the age of ML-heuristics, makes a lot of sense. However, I feel that once we add impartiality, the framework no longer makes sense, at least not in the way it is usually motivated. Suppose an AI agent, like an LLM, predicts an outcome k, which will be the input to your algorithm. Since this AI is good at prediction, presumably even without access to the agent’s preferences it will be good at predicting them and might use them to construct its prediction. It is a black-box machine; we simply don’t know. So, for all we know, the prediction algorithm itself might not be impartial (to the extent that the AI is also good at predicting preferences and potentially uses them) thus making the entire mechanism partial. I feel that this is a major flaw in your formal model.\n\nQUESTIONS:\nHow can a mechanism with predictions that come from black box methods like LLMs that run weird heuristics internally be impartial? Can you model your predictions so that your formal model makes sense and yet is still well-motivated in practice?\n\nCan you explain how your results can help in AI alignment in more detail, also with reference to the less wrong post you link?\n\nLIMITATIONS:\nThe major limitation is that prediction framework and and impartial mechanism don't naturally fit together.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study impartial $k$-selection with predictions, where agents nominate each other and the selection mechanism has access to a predicted set of \"best\" agents. A selection mechanism is impartial if the selection of an agent is independent of that agent's votes. They say a mechanism is $\\alpha$-consistent it it selects a committee with at least an $\\alpha$-fraction of the maximal total votes when the prediction matches the maximum vote committee and $\\beta$-robust if it always selects a $\\beta$-approximate committee regardless of the prediction. The authors extend existing mechanisms for different values of $k$ (uniform permutation, bidirectional permutation, and $k$-partition with permutation) to the setting with predictions, achieving higher consistency-robustness tradeoffs. Through constructions, they also show their tradeoffs are not too far from optimal.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This is a really nicely executed paper, well-grounded in the literature and building on existing results in a novel direction.\n2. The quality and clarity are excellent.\n\nWeaknesses:\n1. If I had to fish for a weakness, I'd say the audience for this work at NeurIPS may be a little narrow (it's perhaps marginally closer to AAAI/EC/IJCAI), although it's certainly in scope and the discussed applications in AI safety are nice.\n\nQuality: very high. I didn't check the appendix, but the descriptions of the mechanisms make a lot of sense.\n\nClarity: Figure 1 is *excellent*. Introduction and related work are remarkably clear and thorough. Technical sections are very clear.\n\nSignificance: Provides improved bounds for impartial selection in a new setting. The application to multi-agent AI systems is intriguing. \n\nOriginality: Adds a new prediction input to the impartial selection setting, which is natural and provides new results. \n\nMinor:\nLine 171 definition of $N_S^-$ says it's the cardinality of the set.\n\nQUESTIONS:\n1. I was a little bit thrown off by the terminology of a prediction being \"correct\" or \"incorrect\" based on whether it agrees with the set of nodes in $G$ with maximal indegree. Since this can be checked by the mechanism, it seems to imply that the prediction is useless; as the mechanism always knows what the \"correct\" prediction would have been. But my understanding is that the existence of a prediction affects the incentives of the voters, allowing different mechanisms to be impartial by using the prediction. Is this understanding correct? If so, I would suggest adding a sentence clarifying the terminology \"correct\" and \"incorrect\" and explaining why having a prediction is useful even if its \"correctness\" is immediately clear.\n\nLIMITATIONS:\nThe authors acknowledge optimality gaps in their results, which are clearly presented. I see no potential negative societal impact of this work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the impartial mechanism with prediction in majority voting. The input is given as a graph with vertices referring to the entities and an arc from $j$ to $i$ indicating that $j$ nominates $i$. The $k$ entities with the most vote are selected. The prediction $\\hat{S}$ is a set of $k$ entities, also given as an input. \n\nThe performance bounds are measured by consistency and robustness. The consistency is prefermance of the proposed mechanism when the prediction is accurate, while the robustness is that when the prediction is wrong. The authors focus on impartial mechanisms and present the following main results. Below, I use $(\\alpha,\\beta)$ to represent $\\alpha$-consistent and $\\beta$-robust.\n\n1. when $k=1$, for any $\\rho \\in [1/2,1]$, a randomized mechanism achieves $(\\rho,1-\\rho)$;\n2. when $k=2$, a deterministic mechanism achieves $(1,1/2)$, and a randomized mechanism achieves $(\\frac{2}{3}+\\frac{1}{3}\\rho, \\frac{2}{3}- \\frac{1}{6}\\rho)$ for all $\\rho\\in [0,1]$;\n3. when $k\\geq 3$, a deterministic mechanism achieves $(1,\\frac{1}{k})$, and a randomized mechanism that $\\alpha \\geq 1-\\frac{1}{2k}$, which lossing $O(\\frac{1}{n})$ of $\\beta$ when compared to the best-known mechanism without prediction.\n\nThe authors also present upper bounds of $(\\alpha, \\beta)$. Result 1 achieves the best possible, and moreover, the gaps between upper and lower bounds of other settings are not large.\n\nSTRENGTHS AND WEAKNESSES:\nThe research question is interesting. Mechanism design with prediction has recently become an active research direction, particularly for those problems with strong impossibility results. The core idea is to use predictions to improve the algorithm's performance when the predictions are accurate (consistency), while maintaining (or nearly maintaining) the best-known bounds when the predictions are incorrect (robustness). This paper introduces the framework prediction to the impartial mechanisms in majority voting.\n\nThe results are non-trivial. The maniscript considers several settings, depending on $k$ and the graph structure, and for each of them, it presents upper and lower bounds. Although the presented bounds are not tight, the gap between them is relatively small, which I find acceptable.\n\nThe paper is well-written in general. The reserach question is well motivated and the results are nicely summarized in Figure 1.\n\nFor the techniques, most of the presented algorithms are established on the existing ones (without prediction). The presented algrithms share the similar ideas of the existing ones or use them as subrotines, though with adaptations to handle the prediction setting. Although these adaptations are not trivial, they are not particularly difficult to devise.\n\nOther minor comments:\n\nline 261, Theorem 3.3. From the proof, it seems that \\delta \\geq 2 is required, which is not mentioned in the statement. For the current statement, if \\delta=1, then \\beta(1)=0. You may want to rephrase the statement.\n\nline 256, show -> have shown\n\nline 198, using $\\Delta(G)$ for $\\Delta_1(G)$ has been mentioned in line 174\n\nQUESTIONS:\nQ1 : For plurality voting, you only discussed the randomized mechanisms without prediction and designed randomized mechanisms with predictions. Is this because the impossibility result for deterministic mechanisms mentioned earlier (line 199) also extends to the plurality voting setting?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-10"}