Generative Social Choice: The Next Generation
Authors: Niclas Boehmer
Sara Fish
Ariel D. Procaccia
Venue: ICML 2025 oral (2025)
Keywords: Social choice
large language models
committee elections
democratic processes
proportional fairness
Primary area: theory->game_theory

Abstract: A key task in certain democratic processes is to produce a concise slate of statements that proportionally represents the full spectrum of user opinions. This task is similar to committee elections, but unlike traditional settings, the candidate set comprises all possible statements of varying lengths, and so it can only be accessed through specific queries. Combining social choice and large language models, prior work has approached this challenge through a framework of generative social choice. We extend the framework in two fundamental ways, providing theoretical guarantees even in the face of approximately optimal queries and a budget limit on the overall length of the slate. Using GPT-4o to implement queries, we showcase our approach on datasets related to city improvement measures and drug reviews, demonstrating its effectiveness in generating representative slates from unstructured user opinions.

Generative Social Choice: The Next Generation
Niclas Bochmer
Sara Fish 2 Ariel D. Procaccia
Abstract
ments based on votes - can be viewed as social choice problem. Specifically, it is an instance of committee elec- tions, where the statements play the role of candidates, is therefore possible to formalize what it means for a selec- tion to be "representative" using powerful notions of pro- portionality developed in the computational social choice literature (Aziz et al., 2017a).
A key task in certain democratic processes is to produce a concise slate of statements that pro- portionally represents the full spectrum of user opinions. This task is similar to committee elec- tions, but unlike traditional settings, the candidate set comprises all possible statements of varying lengths, and so it can only be accessed through specific queries. Combining social choice and large language models, prior work has approached this challenge through a framework of generative social choice. We extend the framework two fundamental ways, providing theoretical guaran- tees even in the face of approximately optimal queries and budget limit on the overall length of the slate. Using GPT-4o to implement queries, we showcase our approach on datasets related to city improvement measures and drug reviews, demonstrating its effecctiveness in generating rep- resentative slates from unstructured user opinions.
Our starting point is the work of Fish et al. (2024), who propose taking a broader viewpoint: instead of restricting the set of candidates to the specific statements submitted by participants, we view every possible (well-formed and reasonably concise) statement as potential candidate. This gives rise to two related challenges when creating "represen- tative" sets of statements: generating consensus statements and predicting the preferences of participants over newly generated statements. In practice, both of these challenges, Fish et al. (2024) argue, can be effectively addressed using large language models (LLMs).
To simultaneously tap the theoretical rigor of social choice and the (notoriously inscrutable) power of LLMs, Fish et al. (2024) put forward two-step framework: generative social choice. In Step 1, they seek to design a democratic process that converts survey responses into a provably representative slate of & statements, assuming perfect answers to certain queries, which constitute the building blocks of their pro- cess. In Step 2, they implement and validate those queries, showing that modern LLMs (specifically, GPT-4o) can reli- ably (albeit imperfectly) realize them. In an end-to-end pilot with hundreds of participants, the --aded democratic process of Fish et al. (2024) generated a slate of 5 statements that faithfully represents the views of the US population on the topic of chatbot personalization.
1. Introduction
In the realm of Al and democracy, one of the most widely discussed systems is Polis (Small et al., 2021). It enables online participants to submit statements about a given pol- icy question and vote on the statements submitted by others. These inputs are then aggregated into report that highlights subset of statements seen as representative, in that they capture different salient viewpoints expressed by partici- pants. Polis has been famously used in Taiwan, Australia, and elsewhere for national-level policymaking. A closely related system, Remesh, has been deployed by the United Nations for peacebuilding activitics (Alavi et al., 2022).'
1.1. Our Contributions
Halpern et al. (2023) observe that the task performed by these systems selecting a representative subset of state-
In our opinion, the framework of Fish et al. (2024) provides a compelling vision for how AI can support democratic innovation. But their implementation is, at this point, largely proof of concept; there is much more work to do before generative social choice can be put into practice. In this paper, we make progress in this direction by enhancing generative social choice along two distinct dimensions.
'Hssso Plattner Institute, Germany Harvard University, USA. Correspondence to: Niclas Boehmer <Niclas Boehaee@hiiieiieee... Sara Fish < g.w.rvard.edd.....
Proceedings of the 4gnd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Polis and Remesh both examples of collective response systems (Ovadya, 2023).
Costs and budgets. The number of statements put in 1.
the slate can be chosen to match the attention span of

---

Generative Social Choice: The Next Generation
the consumer, with larger values of k allowing for the representation of more nuanced positions but impos- ing a greater cognitive burden. However, previously k needed to be set upfront, and there was no way to control the length of statements, implying that even a small & could potentially lead to a "long" slate. We view the cost of statement as its length and require our slates adhere an overall budget. In this way, for example, one can request slate with overall length of no more than 100 words, which the algorithm can split among an arbitrary number of statements. The addition of costs and budgets to our problem is also natural in that it mirrors the extension of com- mittee elections to participatory budgeting (Cabannes, 2004) the task of selecting alternatives with costs subject to a budget constraint.
implementation of the underlying queries, and our process does not require knowledge of the magnitude of the errors in the query responses when the algorithm is executed.
In the second part of the paper, we introduce and evaluate the Proportional Slate Engine (PROSE), our practical im- plementation of our proposed democratic process, PROSE leverages GPT-4o when answering discriminative or gen- erative queries. In contrast to the implementation of Fish et al. (2024), PROSE is applicable to variety of datasets, as it only requires unstructured textual user data and a target slate length as input. We evaluate PROSE on four instances drawn from drug reviews and a deliberation hosted on Polis. In each case, PROSE outperforms four baseline approaches with respect to both user satisfaction and proportionality.
Additional material, including full proofs and addi- tional experimental results, can be found in the ap- pendix. The code for PROSE and our other ex- periments is available at github. com/sara sara-fish/
2. Approximate queries. As mentioned earlier, the theo- retical model of Fish et al. (2024), which underlies the design of their democratic process, assumes perfect an- swers to certain queries. Specifically, they rely on two queries: discriminative queries, which predict given participant's utility for given statement, and gener- ative queries, which (slightly reinterpreted) generate a statement maximizing the number of participants from a given subset that have at least a given utility for it. If these queries do not perform optimally, their theoretical guarantees break down.
1.2. Related Work
Our paper contributes to the study of leveraging AI to scale democratic processes (Kahng et al., 2019; Devine et al., 2023; Flanigan et al., 2020; Gudiño et al., 2024; Landemore, 2022). Within this literature, the emergence of L.LMs has fu- eled efforts to create tools supporting deliberation processes at scale (Tessler et al., 2024; Bakker et al., 2022; De et al., 2025; Konya et al., 2023; Ding & Ito, 2023). Although LLMs and AI can support various stages of a deliberation process, one key challenge is identifying consensus between participants and producing a representative summary of dif- ferent opinions (Small et al., 2023). A common approach in previous work has been to craft single consensus statement that every participant endorses (Tessler et al., 2024; Bakker et al., 2022; De et al., 2025). By contrast, Fish et al. (2024) and our work aim to generate a set of (possibly conflicting) statements, with each statement representing a distinct sub- set of the participants. Another diffcrentiating aspect of our contribution is the focus on deriving formal proportionality guarantees that explicitly depend on the accuracy of the underlying L.MM in responding to queries.
To address this shortcoming, we introduce approximate queries (also accounting for costs): Our discriminative query simply predicts utilities up to an additive error of 3. Our generative query has three possible sources of error and hence three parameters 7. 6, and μı; it returns statement that is optimal up to a multiplicative factor of y, while relaxing the allowed cost by factor of μ and the desired utility by an additive ò term.
Our main theoretical result is a new democratic process subject to a total budget constraint, relying on approximate queries, which guarantees approximate proportional rep- resentation. In more detail, assuming perfect answers to queries, the democratic process of Fish et al. (2024) guar- antees balanced justified representation (B.R), which (in- formally speaking) means that every subset of participants with sufficiently cohesive preferences that is large enough to deserve one statement must be represented in the output slate. Our process extends this guarantee by assuming only approximate queries and taking costs and budgets into ac- count; we achieve an approximate version of BJR adapted to the cost setting, which gracefully degrades as function of the error parameters B, ō and μ. We also establish several lower bounds, showing that the dependence of our guaran- tee on the error parameters is close to optimal. Notably, our process and its guarantees are agnostic to the specific
Our work draws inspiration from and contributes to the study of committee elections in social choice (Lackner & Skowron, 2023; Faliszewski et al., 2017). Within social choice, committee elections have been extensively studied with an emphasis on defining and optimizing various mea- sures of proportionality. This effort has resulted in a refined and well-understood hierarchy of proportionality axioms (Aziz et al., 201 7a; Lackner & Skowron, 2023), to which balanced justified representation, as considered here, is in some sense orthogonal (Fish et al., 2024). Subsequent work has extended results on committee elections to participatory
2

---

Generative Social Choice: The Next Generation
budgeting, where candidates have varying costs model capturing, for instance, real-world applications in which city lets voters decide on how to allocate a portion of its budget (Rey & Maly, 2023; Peters et al., 2021; Aziz& & Shah, 2021). Within participatory budgeting, various preference models and ballot formats have been explored, with our model aligning closest to participatory budgeting under ad- ditive utility functions (Peters et al., 2021; Fain al.. 2016; 2018). However, unlike prior work, do not assume given, finite candidate set; instead, we must explore poten- tial candidates through our query model, introducing new algorithmic and conceptual challenges.
statement a E El as input and returns uy(a)) Second, a gen- erative query GEN(S, 2,), which, given set 5 C N of agents, utility value 8 E [r] and cost returns the state- ment approved by the largest number of agents in S at level ! which costs at most x i.e., arg max (Etto(()) Suppo, S, e). We refer to these queries as exact.
To account for errors that naturally arise in the implemen- tation of queries, we introduce approximate versions: For the discriminative query, we allow for additive error. Specifically, for 3 E [r]. discriminative queries are |-- accurate, if for each agent i E NN and statement a E U, Disc(i, a) E [u,(a) - B, (,(a) + .3. For the generative query, we account for three types of errors: (i) multiplica- tive error y in the number of supporters of the returned statement, (ii) misjudgment of statements' cost by mul- tiplicative factor μ², and (iii) misjudgment of the utility agents derive from the selected statement by an additive error 5. Formally, for 0 S 7, S 1 and 8 E [r], gen- erative queries are (7; 6, pu--ccuratee if for the statement a' returned by GEN(S, E, x). we have c(a") S x and the following regarding the support of a" holds:
2. Model
For two integers x, y E N, we let [x, u] : f,, I,...,y) and [y] := g]. Let denote (potentially infinite) uni- verse of statements, and let c U > No be cost function mapping each statement E U to its cost c(a). We refer to a subset of statements W C U as a slate and write its total cost as c(W)= := E.ew c(a). Furthermore, let N be a set of n agents, where each agent E N has a utility function u, : U + [r]. mapping each statement a E U to one of r possible utility values.? 2 For an agent E N, statement a E U. and utility value E E [r]. we say that approves a at level e if (a(a) 2 f. For a statement E 4, a group of agents S C N, and a utility level e E [r). let sup(a, S, E) = {i E S (a(a) 2 EH] denote the number of agents from S approving a at level E.
maX,ottida) sup(a, 27
(1)
for every S C N, E [r], and x E [B]. For B = 6 = 0 and j = 7 = 1, we recover the exact queries.
2.2. Axiom
To evaluate whether slate adequately represents the agents, we adopt an axiomatic framework following the study of proportionality in social choice. The key principle behind proportionality axioms is that group of agents should have control over an s/n fraction of the budget. A violation occurs if group can propose "better allocation" of their share of the budget. We introduce an approximate version of the balanced justified representation (BJR) axiom (Fish et al., 2024), adapted to our setting with statement costs:
General Problem Statement Given a budget B E N. our task is to select a slate W C of statements of summed cost at most B, which proportionally represents the agents (roughly speaking, #% of the slate should represent a% of the agents; see Section 2.2). We are given the set of agents N and the budget B, but the universe U and the util- ity functions of the agents ( are unknown. Instead, access to l and (u,)EN is restricted to a set of queries (see Section 2.1). We assume that the cost c(a: of a statement a E ll can be computed in constant time. In our implemen- tation, statements will be texts, and the cost of a statement determined by its length in words. For each agent, we are provided with textual information reflecting their opinions, and the queries will be executed with the help of LLMs.
Definition 2.1 ((b, c))co....JR). For b E No and d E Ry1, a slate W satisfies (b, d) costtBUUJ if there is a function w N - W matching agents to statements in a balanced way, such that no coalition S C N, statement a U and utility threshold 0 E [r] satisfies (i) [S] 2 d =( (ii) u, 2 for all E S, and (iii) u, (1)) < 8 for all E S.
2.1. Queries
"We introduce because we observed that GPT-4o often un- dershoots the specified word budget. suggesting that it internally searches within more conservative space (i.e., shorter statements). Consequently, we can only expect it to identify the best statement among those with length at most = for some 2 < x.. The parameter i captures this tendency to undershoot the budget.
We assume access to two types of queries. First, discrimi- native query DIsc(i, a) which takes an agent i E N and
Utility values can represent varying degrees of agreement with a statement, e.g., levels might correspond to "strongly approve", "approve", "indifferent", "disapprove", and "strongly disapprove".
'An assignment halanced for each a E W, exactly rela)n or c(a): agents assigned a. w maps each agent to statement representing the agent in the slate.
TTo simplify some arguments, we assume that for earh x. E [B]. there is a statement a E 24 with c(a) = x.
3

---

Generative Social Choice: The Next Generation
Setting b- and d = 1 yields the "exact" version of the axiom, which we call cB/R and which can be guaranteed for correct query responses (see Theorem 3.1). Under cBJR, no coalition S should exist that can control enough budget to "buy" a statement a for which all agents in S derive utilityy above a threshold 0, while their assigned statement in W provides each of them a utility below 0.5 In the approximate version, specifies how much larger a coalition must be to constitute violation, while b quantifies how much the utility threshold 0 must be preferred over the assigned statement in W.
agents to support statement during the current round. In the inner loop (Line 4), we iterate over different possible costs of the statement to be added, where the cost values we try are given by the list C. For each cost value Clj] and utility value E, we call the generative query, where the function f controls the utility levels with which we make the call (Line 5). For each generated statement, the algorithm uses the discriminative query to identify the agents who approve the statement at level é (Line 6). For the most approved statement a* (Line 7), we check whether its approvers S., can afford the statement (|S..-I 2 [o(a")xx/m) >>x/B]; Line 8). If the condition is satisfied, a² is added to the slate, and the agents that will be represented by a' are removed from the population (Lines 9-10).
If we were to drop the balancedness constraint on w, we arrive at a weaker version of the axiom to which we refer as cost-justified representation (cJR) in which condition (iii) becomes u,(a') < 0 6 for all a' W and S. To illustrate the differences between cJR and cBJR, consider an instance with 9 agents sharing opinion X and 1 agent holding the opposite opinion -X. Suppose there are state- ments ox (resp. a-x) of length TO that provide maximum utility for agents with opinion X (resp. -X). The slate faxx, a: xx) } satisfies clR but can be seen as highly mislead- ing, as both opinions occupy equal space in the slate despite their unequal prevalence. In contrast, cBJR ensures propor- tional representation: the fraction of slate space devoted to an opinion reflects its prevalence in the agent set. For in- stance, in this example, cBJR would require that statements covering opinion X have a total length of 9B 10
The runtime of the algorithm can be controlled by setting C and f. We show in subsequent sections that: (i) Using C = {Li- & j E [n]} and s(() = {f) (referred to as
Process) is sufficient to achieve cBJR when queries are exact. (ii) If queries are approximate, C = [B] and f(() = [e. r] (referred to as Complex Demoo- cratic Prcess) leads the best guarantees. Depending on the application, the budget B can be considerably larger than all other parameters. implying that the selection of C' might have substantial effect on the practical running time and cost of the process.
B
Differences from Fish et al. (2024) Algorithm I builds on the democratic process of Fish et al. (2024) but differs in several key aspects. First, our algorithm accounts for state- ments with varying costs (e.g., word lengths), whereas the original process assumes uniform costs. Second, there is a fundamental difference in the generative query: While Fish et al. (2024) generate statement that maximizes the r-th highest utility among a given set of agents, our query takes utility level & and identifies statement approved by the largest number of agents from given set at level f. This al- lows our algorithm to iteratively consider decreasing utility levels for the next statement to be added, a key feature for establishing proportionality guarantees under approximate queries, an aspect not addressed by Fish al. (2024).
At first glance, a possible limitation of cBJR is that it as- sumes that each agent "cares" only about one statement in the slate. However, as discussed above, larger groups of agents can still exert control over multiple statements. More- over, as we argue in Appendix A, under certain assumptions, which are natural in our context, cBJR implies a variant of extended justified representation ( (EJR) (Peters et al., 2021), a strong proportionality axiom that explicitly models the influence of larger groups over multiple statements.
3. Algorithm and Theoretical Guarantees
We now introduce our new democratic process, establish its proportionality guarantees under approximate queries, and demonstrate that these guarantees are almost optimal by establishing complementary impossibility results.
3.1. Warm-Up
We begin by analyzing Fast- Deseeeeeooaaticroeesss (DemocraticProeessc. with C C((li- # lj E [m]] and f(() = (C)), which satisfies cBJR when queries exact. The key idea is that if a statement causing a cBJR violation existed, a statement with overlapping supporters would have been added to the slate. Notably, it is unnecessary to iterate over all possible cost values to rule out statements inducing cBJR violations, as the definition of cBJR depends only on the number of agents required afford the statement (whether statement costs [ B + 1 for some [n] or [G + 1) @] is irrelevant). In fact, the algorithm also has a
Algorithm 1 presents our main parameterized algorithm. The algorithm iteratively adds statements to the slate W if a sufficient number of agents "support" the statement. The outer loop (Line 2) iterates over utility levels required for
"One might wonder why we do not simply require that all agents in S prefer a to their assigned statement in W, removing the utility threshold 0. The resulting axiom, version of local stability. has severe practical limitations: it may not always be satisfiable. and determining whether slate satisfying the axiom exists is NP-hard (Aziz et al., 2017b).
4

---

Generative Social Choice: The Next Generation
Algorithm DemooraatcProcesss,_,,,L. B,r)
p < - will be added to W, a contradiction.
- 1 = S |Sar Thus, a
Parameters List C' of cost values and function f [r] + 2(r) mapping utility values to subsets of val-
ues.
3.2. Approximate Queries: Guarantees
1: S N, W< 0 tr
When additional sources of error (3, 8, and μ) are intro- duced into our queries, Wust-DemocraticProee no longer provides (approximate) proportionality guarantees. To un- derstand this, consider 3-accurate discriminative queries for some 3 O. The issue in Fast- Sosat-DeocraticPrronm Line 5, where statement is generated that is supported by agents at utility level t. However, due to errors in the discriminative query, none of these agents may end up being included in S.- As a result, the algorithm may ultimately produce a slate in which all agents have a utility of only 1. In turn, with cost errors (1)), we can no longer restrict our focus to querying for specific statement costs, as statements with these costs might be missed given the overestimation of costs in the generative query.
2: while l 2 and S + do
3: j+1
4:
while B - c(W) 2 Clil and j & |C| do
5:
U + UeESo [GEN(S, E, ((
arises in
6:
S. - (iES Disc(i, 2 f for all a E U
7:
a" + arg max |Sal
8:
if S.. 2 (n/a))/B then
9:
S S ([c(a)///B) highest Disc at) return value}
B] agents from S.- with
10:
W + W {a*}
11:
else
12:
+ j+j+1 (+(-1
13:
14:
Return W
To address these challenges, we consider more expensive version of DemocraticProcesssc.r Complex- DemocraticProcess, where C' = [B] and f(() = r).
favorable guarantee with respect to the supporter error ry in the generative query:
Using an extended version of the strategy from Theorem 3.1, we establish the following bound:
Theorem 3.1. For exact discriminative queries generative queries,
and (7, 0, 1) -accuraeee satisfies (0, if j E [n]} S C and f E f(1) for all
Theorem 3.2. For 8-accurate discriminative
DemocraticPraeest
queries and (%, 8.,))aacccraate generative queries, Democratic
ULi E
B
(2,3 + 5, 7x) -c)-BRR if C = [B] and [e, r] [(() for all E [r].
satisfies
Proof. Let V be the returned slate. For each i E N, let w() be the statement added to W* during the round in which i was removed from S. By Lemma B.2, w : N + W and w is balanced. For contradiction, assume there exists group T of agents witnessing a (0, 1/))-BR vio- lation for W* threshold 0, caused by statement a'. Let t := re(a')=2) and observe that [T] 2 t. Consider the first agent i E T' removed from S during the algorithm. Let and a* be the values of the respective variables l and a* during the iteration when i is removed from S. By defini- tion, wi) a and uz (a²) ) 2 f. Therefore, it suffices to show that > contradicting i T.
This result demonstrates the different influence that error sources have on the approximation guarantees. Upon closer examination, these dependencies become intuitive: (i) The multiplicative error y in the number of supporters implies that statement a must be approved by 1 [[((a)-n/r]) agents to ensure a statement approved by [c(a)-n/B]] agents gen- erated. (ii) For the cost error μu, the generative query effec- tively misjudges the cost of statements by factor of μ.. This requires the group of supporters to be larger by the same factor for the generative query to recognize the statement as "admissible". (iii) The discriminative error 3 leads to errors in Lines and 9, leading to incorrect selection of agents and a reduction the utility agents derive from the statements they are matched to. (iv) Similarly, the utility error ó in the generative query reduces the utility agents have for the statement returned by the generative query by up to 8.
Assume for contradiction that ē < 0. Consider the last itera- tion in which Line 4 is visited for é = 0 and Clj] = [.- @ (no statement is added in this iteration). Note that as 7 is the first agent from 7' that gets removed from S, we have 7 C S in this iteration, implying that the body of the loop is visited, as |S| @ 2 # 2 COj]] (cf. Observation B. In Line 5, we will call GEN(S,8, [t- [] and let be the returned state- ment. As generative queries are (7, 0, 1) acccuraee,, it holds that Sup((,S,0) 2 t (since sup(a',
3.3. Approximate Queries: Impossibility Results
In this section, we demonstrate that the approximate guaran- tees of improved. We begin by establishing lower bound on the impact of errors in utility judgments, which exactly matches the guarantee provided by Theorem 3.2:
cannot be significantly
= [T| 2 1 and It B 2 [(a'"" B = c(a'). As result, will have n
S 2 / in Line 8 with c(a") S ]J. implying that
Theorem 3.3. Fix B3 E [r] and > No algorithm with
5

---

Generative Social Choice: The Next Generation
0.3
www ooogle
(a) 0.
(b) 8
=
0.85
(c) 3
0.7 (d) 8
14
0.55
Figure 1. Proportionality of different versions of our democratic process for varying error settings: For given c-value (æ-axis), the plot shows the maximum d value for which (c, cBJRR is violated (y-axis) averaged over 100 instances. The shaded gray region represents the area ruled out by the guarantees provided by Theorem 3.2 for Deappee--DmocrtticPPP.
access to 8-accurate discriminative queries and (1,6, 1)- accurate generative queries can guarantee (23 + 5 - 1)- cBJR for (2B + - ), even when all statements have unit cost.
n'' 8 to B++1 This modification addresses the above issue of large groups being potentially overlooked. It does so at the expense of allowing groups to slightly "overspend" their budget. However, this overspending is controlled so that not enough agents remain in the algorithm's ' ter- mination constitute cBJR violation. This adjustment allows the guarantee to match the lower bounds established in Theorems 3.3 and 3.4;
Turning to errors in the generative query regarding the num- ber of supporters and cost, establish the following im- possibility result:
Theorem 3.4. Fix some T, E Q1>220, and E No- No algorithm with access to exact discriminative queries and (7/, O, ()) -accurceeeee generative queries can guarantee [W]
Proposition 3.5. Let > 0. Assuming all statements have unit cost, given access to B-accurate discriminative queries and (?,, 8, aac-uureee generative queries, there algorithm (Algorithm in Appendix B) that guarantees B (2,8 6, B c)-cBJR when p and 7 known.
(p, Wh+T ) -BBJR [or (p, [W] [Wht))-JJR) where W is the slate returned by the algorithm."
Note that both Theorem 3.3 and Theorem 3.4 assume that certain aspects of the queries are error-free sooo--rre--spcifeee -seessecccllll....eeeee Theorem 3.3 assumes ? = μ and Theorem 3.4 assumes However, allowing for additional error will
3.4. Validation in Synthetic Environment
The goal of this section is analyze how our algorithms are affected by approximate queries in simulations, thereby complementing the worst-case guarantees established in the previous parts. To be able to model errors in query answering, we consider a purely synthetic environment.
= 5 never lead to improvement in the worst case. Therefore, the above impossibility results continue to hold even when these parameters are relaxed, i.e., Theorem 3.3 holds for ar- bitrary 7/ /4 E Q1>120. and Theorem 3.4 holds for arbitrary 3,8 [r].
Environment Our environment is highly structured: There is set I of issues. For each issue, [b] represents the set of possible opinions. A statement & contains opin- ions on a subset of issues, i.c., a [6] for some I.- S I. The cost of a statement is the number of issues it addresses, that is, c(ax) sible statements. Each agent E N is characterized by statement a' E [b] that covers all issues; we generate a' uniformly at random. The utility of agent for statement Ejer, 2 - la - azzl- For efficiency reasons, we focus here on b b= | = and set n = 60 and B 15.
Note that our algorithm from Theorem 3.2 does not precisely match the lower bound from Theorem 3.4, yet [W
741/ converges our guarantee as the number of statements in the slate W increases. Intuitively, the reason why PemoorattiProoeesscc does match this bound lies in the condition |S.-| [c(a") )(aa"")) in Line 8. Assume there exists a statement a approved by [([=(a))) agents at some level Then, the generative query might only return statements approved by [ci)) - agents at level L, implying that these statements never get added, despite potentially constituting (0, - c)-cJJR violation.
|Tal and the universe consists of all pos-
is Σ
Since the universe finite, we can compute exact answers both discriminative and generative queries. To simulate 8 accurate discriminative query, we draw integer uniformly at random from |-3, 88] and return the true utility plus the drawn value. To simulate (7, ö, p)aaccurate generative query, we compute all statements satisfying Equation (1) and then return a statement drawn uniformly at random from this set.
In the special where all statements have unit costs, can close this gap by adjusting the threshold in Line
Teehhiiclll,, our proof shows the following: For given in- stance with budget B, where e is minimum cost a statement that is approved at level 2 by at least one agent and returned by the generative query, no algorithm can guarantee (p. B/ J-BBJR.
B/e
6

---

Generative Social Choice: The Next Generation
Results In addition to Fest-DemooruticPPrcesss (Fast) and Complee-Democcaticc/eee alyze DemocratielPoccessc. with C' { and f(t) = l (Uniform). This variant resembles a committee election approach and only adds statements addressing all issues. Figure 1 shows the average (c, dcBRJ achieved by the slates returned by the processes. We find that all processes perform significantly better than the strongest theoretical guarantee (gray area), which holds only for Comp ex. Comparing algorithms, Uniform quickly incurs substan- tial cBJR violations even with small errors, but the violations remain relatively stable as the eттоr increases. Fast and Complex perform similarly, with Complex slightly out- performing Fast. Both consistently outperform Uni form, but their violations increase as the error grows.
We refer to Appendix D for description of our query im- plementation and additional details PROSE.
(Compl ex), we also
As input, PROSE requires the desired slate length and some textual information about each user. Importantly, PROSE can operate with unstructured and minimalistic user data, making it usable a wide range of scenarios. This stands in contrast to the implementation of Fish et al. (2024). which relies on highly structured, curated user input. Their ap- proach requires participants to complete detailed surveys, including rating example statements and answering prede- fined questions- intiiinatiiiooomiiiinnnn that is not available in our cess-structured datasets. We opted not to reimplement their queries for unstructured data, as there would be substan- tial ambiguity in the reimplementation due to subjective implementation choices. Instead, we will include base- line called unit-cost assumptions of Fish et al. (2024) but uses our own query implementations to ensure comparability.
In Appendix C, we further compare the utility agents derive from their assigned statements, observing that Comp 1 ex performs better than Fast, which in turn outperforms Uniform. We also provide a more detailed analysis of how increasing errors influences the proportionality achieved by the slate. Additionally, as the errors the described envi- ronment can be interpreted as random, we examine scenar- ios where errors behave in a more adversarial, worst-case manner. The general trends persist, but the performance gap between Fast and Complex (as seen in Figure 1) becomes more pronounced.
which aligns with the core
A further advantage of PROSE is that it does not require dataset-specific tuning, e.g., we do not need to tune param- eters such as the number of output statements, clustering granularity, and prompt wording. Accordingly, the four slates from experiments are all generated using the same configuration, adding to PROSE's flexible usage.
Limitations Although GPT-4o offers a powerful and flex- ibly usable implementation of discriminative and generative queries-leading to the generation of high-quality slates-it also comes with several limitations. Most notably, due to its opaqueness, GPT-4o comes without guarantees on the query answers, and individual responses can be very prompt- dependent, hallucinated, and potentially biased (see our im- pact statement for further discussion). These problems are particularly prevalent in the context of the significantly more demanding generative query, which we try to mitigate by always considering many statements for addition. During our implementation process of the generative query, we ob- served that GPT-4o struggles to identify coherent subgroups of agents with aligned opinions, which motivated our use of a dedicated two-step generation strategy where GPT-4o is only queried to write a consensus statement for a cohesive group of agents.
4. Implementation and Experiments
We present the Proportional Slate Engine (PROSE) and evaluate it in experiments.
4.1. PROSE: Proportional Slate Engine
PROSE is a practical implementation of Algorithm in which both the generative and discriminative queries are implemented with the help of LLMs, in particular GPT-4o- 2024-11-20. Broadly speaking. our implementation of the discriminative query uses GPT-4o as models for human pref- erences (similarly to the work of Fish et al. (2024), Filippas et al. (2024), and Argyle et al. (2023)). For the generative query, we note that our theoretical results are agnostic to the chosen query implementation and all theoretical results re- main valid when additional statements of cost at most C'ji] are included in the set U in Line 5 of Algorithm 1. Thus, we treat the generative query as an approximate solver for the optimization task defined in Equation (1): We use different query implementations to add multiple candidate statements to the set U. from which ultimately only the one with the highest number of supporters at level will be returned."
4.2. Experiments
In this section, we use PROSE to generate slates for four different instances and compare the results to four basclines.
GPT-4o to generate length-bounded consensus statement for the agents in S" (similarly to the work of Fish et al. 2024, Tessler et al. 2024, and Bakker et al. 2022). Morcover, rather than limiting the selection to newly generated statements in each round, we consider all previously generated statements with costs within the current target cost as candidates for addition to the slate.
*All our implementations the generative query follow a two- step procedure. First, we identify agents 5' C S who are likely to approve the same statement at level f, using embeddings and clustering neirhst- techniques. Second, we prompt
7

---

Generative Social Choice: The Next Generation
Method
Birth Control (Unifom)
Birth Control (imbulanced)
Obesity
Bowling Green
Mean
p-value
Q1
%BJR viol.
Mean
p-value
Q1
&BJR viol.
Mean
p-value
Q1
&BJJ viol.
Mean
p-value
Q1
&BBJR viol.
PROSE
3.42
3,00
0.19
3887
3.12
0.14
3.39
3.00
0.10
3.54
3.00
0.08
PROSE-UnisCott
3.05
0.011
2,06
0.31
3.11
3.
2.17
0.34
3.20
0.224
2.00
0.2
2.76
1.1e-05
2.02
0.22
Clusteting
3.14
0.011
2,30
0.30
3.28
1.6c-05
2.80
0.23
3.02
0.016
2,00
0.17
2.79
4.5c-01
2,00
0.37
Shoo-Shot
2.76
3.7e-06
2,00
0.65
3
1.0e007
2.09
0.36
2.75
3.5e.0
2.00
0.39
3,06
0.023
2.00
0.13
Contextioss Zero-Shol
2.58
3.1c-09
2,00
0.68
2.77
2.3c-14
2.01
0.11
3.15
0.055
2.73
0.19
2.51
1.7c-08
2.00
0.32
Table 1. Performance of PROSE and baselines. We show the mean and 25th percentile of agents' utilities. For the baselines, we show the p-value for the null hypothesis that its mean utility is the same for PROSE. Lastly, we show the fraction of sampled statements constituting a cBJR violation.
4.2. DATASETS (see Appendix E.1 for details)
affinity propagation clustering (Frey & Dueck, 2007), which automatically determines the number of clusters. For each computed cluster, we call an implementation of the genera- tive query with the cluster's 's share of the budget as the word limit and add the returned statement representing the cluster to the slate.
We consider two data sources. First, the publicly available UCI ML Drug Review dataset (Gräßer et al., 2018) con- tains web-crawled patient reviews, each accompanied by a rating on a 1-10 scale. From this dataset, we create three subsampled instances (each with 80 agents): Birth Con- trol (Balanced), which contains reviews of a birth control medication with all ratings appearing equally often; Birth Control (Imbalanced), which includes only birth control reviews with extreme and central ratings, i.e., (1,2,5,9110), and Obesity, which contains reviews on a obesity medica- tion with all ratings appearing in equal frequency. ? Each agent is represented by the text of their submitted review (excluding their rating). The budget is B = 160.
PROSE-UnirCone We assume that each statement has the same cost c(a) = 1, thereby following the original idea of Fish et al. (2024). As a result, the budget describes the number of statements, k. to be selected. To reflect this change, in our implementation of the generative query, we no longer impose a word limit on the generated statements. Accordingly, in Algorithm I we set C = [1], implying that when adding statement (of arbitrary length) to the slate we now solely check whether the statement is approved by KJ agents. In our experiments, we set B = 5. Note that the resulting slate is of unbounded length; in fact, in our experiments, the returned slates are around two to three times longer than the budget given to the other methods.
Second, the Bowling Green dataset is drawn from public deliberation hosted on Polis (2023) regarding improvements to the city of Bowling Green, KY, US. We manually com- pile an instance of 41 agents to ensure a non-trivial overlap between the agents' suggestions. Each agent is represented by the full text of their submitted comments. The bud- get B- 164, chosen to be divisible by the number of agents to avoid rounding artifacts in one of the baselines.
4.2.3. EVALUATION AND RESULTS
Ideally, evaluating the quality of a computed slate W would involve asking users to rate the statements in the slate. How- ever, this is infeasible for our datasets and beyond the scope of this paper. Consequently, we rely on a proxy. While it might seem natural to use PROSE's own discriminative query for evaluation, doing so would unfairly favor PROSE, as this would mean that PROSE selects statements to max- imize the score against it will eventually be evaluated. In- stead, we employ separate chain-of-thought (CoTT im- plementation of the discriminative query, which assigns a score between 1 and 6 to each agent-statement pair (see Appendix D.4 for details). We refer to the scores returned as CoT utilities and treat them as the true, underlying utilities. To ensure an independent evaluation, the CoT implemen- tation differs significantly from the (non-CoT) implemen- tation used in PROSE and is considerably more expensive. Although both implementations rely on GPT-4o and aim to approximate the same underlying preferences, we show in Appendix D.5 that their outputs exhibit relatively low correlation.
4.2.2. BASELINES (see Appendix E.2 for details)
We compare against four baselines:
Contextless Zero-Shot. We cast a single prompt to GPT-4o specifying the dataset's topic and the word limit. The model asked to generate a set of opinions that proportionally reflect the general population, with the length of opinions corresponding to their prevalence.
Zero-Shot. Similar to Contextless Zero-Shot, but each agent's 's description is also provided to the LLM. The model is asked to produce a proportional representation of the input agents.
Clustering. We embed each agent using OpenAI's text-embeddding-3-1aee 3-llaag model and then apply
*We rebalanced the ratings in the datasets because the original rating distributions are quite degenerate, e.g., in the obesity dataset, over 70% of users give a score of 9 or 10. This skewness would make the proportional summarization task quite simple, as nearly all users would support the same kind of statements.
For the two zero-shot baselines, we determine maximum- weight balanced assignment w N W between agents
8

---

Generative Social Choice: The Next Generation
and statements in the slate based on the CoT utilities; for PROSE, PROSE-UnitCost, and the clustering baseline, we use the mapping produced by the methods. This setup pro- vides the zero-shot methods with an advantage, as they leverage the CoT utilities used for evaluation. If PROSE's mapping were recomputed in the same way, the average utility would typically improve by around 15%.
query implementations. In our experiments, across the four datasets, PROSE used 9.6M-25.4M input and 53.5K-96. 1K output tokens, with runtimes of 3 31-65 minutes on sin- gle Intel 17-8565U CPU @ 1.80GHz. Given the rapidly improving inference speeds of modern L.MM,, we expect these runtimes to significantly decrease in the future. By contrast, PROSE-UnitCost required around five times fewer resources: between 2. and 4.4M imput tokens and 15.4K to 20.2K output tokens, with runtimes between 7 and 12 minutes.
We present a quantitative evaluation of the generated slates in Table 1 (all generated slates are provided in Ap- pendix E.3). Note that the utility of user i for slate is their CoT utility for (i). We report the mean and 25th percentile of user utilities, along with the p-value for the null hypothe- sis that the average utility of a baseline method is equal to that of PROSE. Additionally, we assess the proportionality of the generated slates. To do so, we sample 100 statements from the set of statements generated by PROSE over the course of its run on a given instance and check for each statement whether it constitutes cBJR violation under the CoT utilities.
5. Discussion
Several promising avenues for future research emerge from our work. From a theoretical perspective, it would be in- teresting to explore how our approach extends from the cardinal preference setting considered here to the ordinal preference setting, which is widely studied in social choice. We expect meaningful connection between the two set- tings, assuming that the candidate space is sufficiently rich. From an experimental perspective, it would be most benefi- cial improve the implementation of the generative query, as the performance of the current implementation starts to degrade when queried on too many agents with diverse opinions.
Examining the results, we find that PROSE consistently outperforms all four baseline methods across all instances. In particular, PROSE achieves statistically significant im- provements in mean agent CoT utilliy,, ranging from 10% to 40% over the baselines. The improvements are even more pronounced-an around 40%-wher considering the 25th per- centile of agent CoT utility, indicating that PROSE effec- tively represents minority opinions by allocating groups proportional control over their share of the slate. Generally speaking, as CoT utilities range from 1 to 6, the achieved utilities can be viewed as quite high, in light of the fact that each agent controls only 2 words in the drug datasets and 4 in the Bowling Green dataset. On another note, a closer examination of PROSE's outputs reveals that it successfully maps similar users to the same statement. For example, in the drug datasets, users represented by single state- ment typically share similar ratings, with the exception of mid-range ratings, which occasionally appear alongside treme ones. Regarding proportionality, slates generated by PROSE exhibit between 7 and 49 percentage points fewer cBJR violations compared to the baselines.
A long-term goal is to apply our framework to participatory budgeting elections. Since our model naturally captures par- ticipatory budgeting" 10 all methodological and theoretical results carry over. However, implementing the queries in this context poses significant challenges. In particular, de- termining the cost of "statements", which now correspond to city improvement projects, would be nontrivial. Even more demanding, the generative query needs to propose en- tirely new projects that appeal to voters while staying within a predefined budget. Although such capabilities currently exceed those of GPT-4o, future advances in LLMs may make them feasible. From a methodological perspective, "traditional" participatory budgeting rules such as sequential- Phragmén (Rey & Maly, 2023), which typically iterate over all projects, could be implemented in a query-based model. However, doing so would require more complex genera- tive queries that account for agent-specific budgets (see Appendix for a more detailed discussion). More broadly, access to more powerful queries could also enable the satis- faction of additional, strong proportionaliiy guarantees such as EJR+ up to any project (Brill & Peters, 2023). These con- siderations reveal an intricate tradeoff between the strength of guarantees and the complexity of queries used, an intrigu- ing direction for future research.
It is notable that despite producing substantially longer slates PROSF-UnitCost leads to inferior results. One reason for this is that in the slates generated by PROSE-UnitCost there are typically some agents that have very low utility for their mapped statement, indicating that five statements are not sufficient to cover the opinion spectrum in its en- tirety. Notably, our approach circumvents the problem of picking the "right" number of statements to be added to the slate, as the algorithm will automatically split the budget between all homogeneous groups
Lastly, note that the runtime of PROSE is primarily driven by the response times of the LLM used in our
10ehe universe I would become the set of all possible city improvement projects, and the cost function would capture the implementation cost of a project.

---

Generative Social Choice: The Next Generation
Acknowledgements
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., and Wingate, D. Out of one, many: Using language models to simulate human samples. Political Analysis, 31 (3):337-351, 2023.
Fish was supported by an NSF Graduate Research Fellow- ship and a Kempner Institute Graduate Fellowship. Procac- cia was partially supported by the National Science Foun- dation under grants 118-2147187 and IIS-2229881; the Office of Naval Research under grants N00014-24-1-2704 and N00011-25-1-2153; and by grant from the Coopera- tive Al Foundation. We thank OpenAI AI for providing API credits via the Researcher Access Program. We thank the anonymous ICML reviewers for their insightful reviews.
Aziz, H. and Shah, N. Participatory budgeting; Models and approaches. Pathways Between Social Science and Computational Social Science: Theories, Methods, and Interpretations, pp. 215-236, 2021.
Aziz, H., Brill, M., Conitzer, V., Elkind, E., Freeman, R., and Walsh, Justified representation in approval-based committee voting. Social Choice and Welfare, 42(2): 461-483 2017a.
Impact Statement
This work advances the integration of social-choice-inspinne algorithms with provable fairness guarantees and the capa- bilities of Large Language Models (LLMs) to construct proportional and cost-constrained slates. Our approach contributes to the methodological foundations of scalable civic participation, enabling new paradigms in democratic decision makiin.. Although our algorithmic democratic pro- cess provides provable proportionality guarantees, the use of LLMs to rate and generate statements introduces specific risks that must be carefully addressed before deployment in real-world settings. These risks include:
Aziz, H., Elkind, E., Faliszewski, P., Lackner, M., and Skowron, P. The Condorcet principle for multiwinner elections: From shortlisting to proportionality. In Pro- ceedings of the 26th International Joint Conference on Artificial Intelligence (ICCAI), pp. 84-90, 2017b.
Bakker, M., Chadwick, M., Sheahan, H., Tessler, M., Camphell-Gillingann,, I.., Balaguer, J., McAleese, N., Glaese, A., Aslanides, J., Botvinick, M., and Summer- field, C. Fine-tuning language models to find agreement among humans with diverse preferences. In Proceedings of the 36th Annual Conference on Neural Information Processing Systems (NeurIPS), 2022.
Bias: LLMs may favor or disfavor certain viewpoints, leading to distorted representation the generated slates. This can occur through incorrect predictions of user utilities or the over- under-generation of statements reflecting particular perspectives.
Brill, M. and Peters, J. Robust and verifiable proportionality axioms for multiwinner voting. In Proceedings of the 14th ACM Conference on Economics and Computation (EC), pp. 301, 2023.
Transparency: Although the algorithmic process itself is transparent, L.LM-generate, outputs are inherently opaque and may lack reliability.
Cabannes, Y. Participatory budgeting: A significant con- tribution participatory democracy. Environment and Urbanization, 16(1):27 2004.
Directly inputting user comments into the LLM could expose the process to adversarial at- tacks, such as prompt injections.
Manipulation:
De, S., Bakker, M. A., Baxter, J., and Saveski, M. Su- pernotes: Driving consensus in crowd-sourced fact- checking. In Proceedings of the ACM Web Conference 2025 (WWW), pp. 3751-3761, 2025.
These risks are particularly acute in political decision- making, where the consequences of biased, opaque, or ma- nipulable outputs are particularly severe. We emphasize that work is most intended inform decision making and not to automate political deliberation decision- decision-making. For any application, addressing these above-mentionel risks is critical to ensuring the fair, reliable, and effective applica- tion of L----rieen democratic processes.
Devine, F., KrasodomsskiJonns, A., Miller, C., Lin, S. Y., Cui, J.W.. P., Marnette, B., and Wilkinson, R. Recursive public: Piloting connected democratic engagement with Al governance. Technical report, Chatham House, vTaiwan, and the Al Objectives Institute, 2023. URL bttts:://vraawwnn-peente 2tt.aai-2222.....eeeee app/ wtt.eeportttccrsieeeeeeeeeeeeeee
Ding. S. and Ito, T. Self-agreement: A framework for fine- tuning language models to find agreement among diverse opinions. In Proceedings of the 20th Pacific Rim Interna- tional Conference on Artificial Intelligence (PRICAI), pp. 298-309, 2023.
References
Alavi, D. M., Wählisch, M., Irwin, C., and Konya, A. Us- ing artificial intelligence for peacebuilding. Journal of Peacebuilding & Development, 17(2):239-243 2022.
10

---

Generative Social Choice: The Next Generation
Fain, B., Goel, A., and Munagala, K. The core of the participatory budgeting problem. In Proceedings of the 12th Conference Web and Internet Economics (WINE), pp. 384-38 399, 2016.
Lackner, M. and Skowron, P. Mulli-winner voting with approval preferences. Springer Nature, 2023.
Landemore, H. Can Al bring deliberative democracy to the masses? Hluman-Centered Artificial Intelligence Seminar, Stanford University, 2022.
Fain, B., Munagala, K., and Shah, N. Fair allocation of indivisible public goods. In Proceedings of the 19th ACM Conference Economics and Computation (EC), pp. 575-592, 2018.
Ovadya, A. 'Generative CT' through collective response systems. arXiv:2302.00672, 2023.
Peters, D., Pierezynski, G., and Skowron, P. Proportional participatory budgeting with additive utilities. In Pro- ceedings of the 35th Annual Conference on Neural Infor- mation Processing Systems (NeurlPS), pp. 12726-12737, 2021.
Faliszewski, P., Skowron, P., Slinko, A., and Talmon, N. Multiwinner voting: new challenge for social choice theory. Trends in computational social choice, 74(2017); 27 27-47, 2017.
Filippas, A., Horton, J., and Manning, S. Large lan-
Polis. Open polis data, 2023. URL htttps://githan com/ wwwppe/mccacc//pppeee/eeeeeeeeee apppaaaaaaaappaaaaaaaa... Last accessed on 08/15/23.
guage models as simulated economic agents: What can learn from homo silicus? In Proceedings of the 25th ACM Conference on Economics and Computation (EC), 2024.
pp.
Rey, S. and Maly, J. The (computational) social choice take on indivisible participatory budgeting. arXiv preprint arXiv:2303.0062. 2023.
Fish, S., Gölz, P., Parkes, D. C., Procaccia, A. D., Rusak, G., Shapira, I., and Wüthrich, M. Generative social choice. In Proceedings of the 25th ACM Conference Economics and Computation (EC), 2024.
Small, C., Bjorkegren, M., Erkkilä, T., Shaw, L.,, and Megill, C. Polis: Scaling deliberation by mapping high dimen- sional opinion spaces. Revista De Pensament Andlisi, 26(2), 2021.
Flanigan, B., Gölz, P., Gupta, A., and Procaccia, A. D. Neutralizing self-selection bias in sampling for sortition. In Proceedings of the 34th Annual Conference on Neural Information Processing Systems (NeurIPS), 2020.
Small, C. T. Vendrov, Durmus, E., Homaei, H., Barry, E., Comebise, J., Suzman, T., Ganguli, D.,, and Megill, C. Opportunities and risks of LLMS for scalable deliberation with Polis. arXiv:2306. 1932, 2023.
Frey, B. and Dueck, D. Clustering by passing messages between data points. Science, 315(5814):97229778 2007.
Tessler, M. H., Bakker, M. A., Jarrett, D., Sheahan, H., Chadwick, M. J., Koster, R., Evans, G., Campbell- Gillingham, L., Collins, T., Parkes, D. C., et al. Al can help humans find common ground in democratic deliber- ation. Science, 386(6719), 2024.
Gräßer, F,, Kallumadi, S., Malberg, H., and Zaunseder, S. Aspect-based sentiment analysis of drug reviews applying cross-domain and cross- data learning. In Proceedings of the 2018 International Conference Digital Health (DH), pp. 121-125, 2018.
Gudiño, J. F., Grandi, U., and Hidalgo, C. Large language models (Ilms) as agents for augmented democracy. Philo- sophical Transactions A, 382(2285):0024100, 2024.
Halpern, D., Kehne, G., Procaccia, A. D., Tucker-Foltz, J., and Wüthrich, M. Representation with incomplete votes. In Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI), pp.
2023.
Kahng, A., Lee, M. K., Noothigattu, R., Procaccia, A. D., and Psomas, C.-A. Statistical foundations of virtual democracy. Proceedings of the 36th International Con- ference on Machine Learning (ICML), pp. 3173-3182, 2019.
Konya, A., Schirch, L., Irwin, C., and Ovadya, A. Demo- cratic policy development using collective dialogues and AI. arXiv preprint A.Ni:2311.00244 2023.

---

Generative Social Choice: The Next Generation
A. cBJR and Extended Justified Representation
Peters et al. (2021) introduce the following version of extended justified representation for participatory budgeting exercises with additive utilities:
Definition A.1 (Peters et al. (2021)). A slate W satisfies extended justified representation there is no coalition $ S .N and set of statements W' C l such that (i) |s| B 2 Eaew: c(a) and (ii) Eaew minjes uy(a) > Eeew u;(a) for all i E S.
Condition (i) imposes that S is "large" enough to afford the statements from W" while the second condition imposes that each agent i from S is "adequutely dissatisfied" with the slate W, i.e., the summed utility i has for W is smaller than the sum of the smallest utility agents from S have for each project from W'.
Lemma A.2. Assuming that U is closed under the union of statements, i.e., for each pair of statement 01, 02 E U there exists statement a² E u with c(a") S c(a1) + c((2) and uz(a") 2 u,(01) + uz(a2) for each i E N, cBJR implies extended justified representation.
Proof. Let W be a slate that does not fulfill extended justified representations as witnessed by coalition S and set of statements W" Then, let a* E u be the statement from UI that arises from combining all statements from W', i.e., c(a') < Eoew. c(a) and uz(a") 2 Eaew, (a(a)) for every i E N, which exists by our assumption that 2 is closed under union. Further let 0 == Eoew' minjes uy(a).
We claim that S, a* and constitute cBJR violation. First observe that by the definition of extended justified representation, for any E S, we have 9 Enew (a(a) and in particular 9 u, (w)) for any mapping w N - W. Further, observe that u, u,(a") 2 9 for all S and that |S] B 2 Enew cla) c(a"). implying that S, a and @ constitute cBJR violation.
n
If one follows the standard assumption that agents' utilities are additive, the assumption that II is closed under the union of statements feels quite natural in the natural language context studied in the paper: Two statements an and 0g can be simply concatenated into statement We immediately get that c(ag) + c(az) c(a) (as the cost of statement is simply its length in words). Given that the additivity of utilities implies that u, (fa1, a2)) u((a))) + u,((a2)), it is only natural to assume that u: (fa))) ((((a))) + (;({))))
B. Additional Proofs for Section 3
We start by showing the following invariant:
Observation B.1. In Line 4 of the algorithm, it holds that B - c(W) 2 |
Proof. For each statements a that gets added to W over the course of the algorithm ro(o).n agents are added to N \ S. c(W) < \ S). It follows that c(W) S | Sl _ = = S| which in turn implics the invariant.
implying B
Lemma B.2. [ E C and 1 E f(1), we have 6 when Democraticrrocessc, terminates. For each i E N, let wi)) be the statement that has been added to the slate W in the round in which was removed S. We have w : N - W and is balanced.
Proof. We start by arguing that S = 0 when the algorithm terminates. For the sake of contradiction, assume that E when the algorithm terminates. Consider the last iteration in which Line 4 is visited for = 1 and Cli] = _ in which by definition no statement will be added the slate. By Observation B. we entered this iteration and by our technical assumption that the universe contains least one statement with c(a:) # the generative query will return some statement of cost at most rela") ) < [ 11 < "B3 -n = 1, the condition in Line 8 will be fulfilled and we will add a statement in this iteration, B
| JJ, which will be approved by level & (as this the lowest approval level). As
leading to a contradiction.
Let be the assignment induced by the creation of W. i.c., each agent i is mapped to the statement a that has been added to W' in the round in which i was removed from S. Note that w is balanced, as only rola).n each statement a E W and S = 0 by the end of the algorithm.
agents are removed from S for B
12

---

Generative Social Choice: The Next Generation
v x Lemma B.3. For 0 S x I 1 and y E N. [ 2 y,-
Proof. For 1, the statement trivially holds. For < 1, observe H [= [y- x²] y (as x and y E N).
and x xL 2 y - x from which we get
Theorem 3.2. For 8-accurate discriminative queries and (7; 6, pl--acuraate generative queries, DemocraticPracessee, satisfies (25 + 5, 1. )-BJR if = [B] and [e,r] S f(E) all E [r].
Proof. The structure of the proof is similar to the proof of Theorem 3.1. Let W be the returned slate. For each i E N, let w(i) be the statement added to W* during the round in which was removed from S. By Lemma B.2, w N + W and w is balanced.
For contradiction, assume there exists a group T of agents that witnesses a (2,3 + 6, )-BJJR violation for W at threshold 1 7/ -- (omo')m1 "1,,"""1"00" (a(a 2 0 and uy (w)) < 0 - (2,3 + 5) for all E T.
0, caused by statement a', i.e., [T] 2
Consider the first agent i E T removed from S during the algorithm. Let and a- be the values of the respective variables l and a* during the iteration when is removed from S. We claim that suffices to show that 2 0 - (3 + 8): By definition of w and S,,, (i) = a' and Disc(i, a²)) ) 2 é. Since discriminative queries are 3-accurate, this implies that uy(a*) 2 (23 + 6), a contradiction to T.
So for the sake of contradiction, assume that i <<0 (3 + 8). We argue that this would imply that an agent from T should have been removed from S in an earlier iteration, leading to a contradiction. Consider the last iteration in which Line is visited for & := 0- - (3 + 8) and Clj] == <(a') (no statement is added to W in this iteration). Note that as ì is the first agent from T that gets removed from S. we have 7 C S in this iteration. The body of the loop will be visited in this
iteration: We have |S| > [T'
peaa) and in fact |S] 2
CDi
TM
B
1 c(a') n
c(aa)
n
Clil
112
B
BB
B
As ||| 2 - B 2 Cbil. by Observation B.1, it directly follows that B - c(W) 2 Cli) and thus that the while loop will be entered.
In Line 5, we will call GEN 6, C[1)) (as 0 2 0. Let Ç be the returned statement. Note that a' is "relevant" for this c(a') J1 c(a') by Lemma B.3. From this, as
generative query: a' {a u c(a)= I [u- Cb]]), as [H- Cbill = [ul
T C S, we get by the definition of T that
max aEUic(a)S
sup(a,,,00 2 [T|
7/
B
Further, as generative queries are (7, 6, (4)-accurate, it follows that sup(C, S, d) 2 7 7p
)m Thus, " agents from 5 that have utility at least 0 5 for Ç. As discriminative B
1 re(a")"] _
there exists a set D containing at least 1 pla')
queries are 3-accurate and ( t= 0 (8 + 6), it follows that D C Sc and thus that |S&| > 1 "rdaa) As result, we will have 1 2 1 rea") and by Equation (2) SSa- is satisfied and a* is added to W, contradiction.
Clyl-n B 1. As c(a*) S Cbi), follows that the if-condition in Line 8
Theorem 3.3. Fix 8 [r] and No algorithm with access to B-accurate discriminative queries and (1, 8, 1)- accurate generative queries can guarantee (23 + 8 e, 1)-BBRRR [or (23 + c, c,1))cJR), even when all statements have unit cost.
Proof. All statements in the universe have cost 1, implying that the budget B denotes the number of statements to be picked. We set 2,3 + 5 + 1. For notational convenience, we choose the number of agents and budget B so that B divides (we will specify some additional constraints on the selection of n and B later). Let t := B
The agent set will be split into two parts X TUX Y with XX| = (the splitting is to be specified later and will depend on the decisions made by the algorithm). For each y" Y, there is statement oyy, with u, (otx)) = 2B3 + 5 + for all E y" and
13

---

Generative Social Choice: The Next Generation
u.(a),) 1 for all & Y' and 2n copies of statement (d)=)) with (.(o-))) 233 1 for all i E Y' and (,(o)))) for all $ Y Additionally, there is statement o'x with (a = 23 + 6 + for all i X and u,(a3) Lastly, there are 2n copies of statement (a'x) se(2)))) with 44(0xx) = 23 1 for all i X and u,(azx) = for all E Y.
for all i $ X
We construct the generative query follows. Let G,,(S, for S N. & E [r], and x E N be a prompted query. If |Sl then we return a copy of the statement " Otherwise, we return a copy of the statement d'x. We return copies of statements a way that we always return the same copy for given input set S, but different copies for different input sets. It is easy to see that the generative query is (1, 6, -aacurreee we always return a statement a maximizing sup(a, - 5).
|Snx] < 2
The discriminative query returns 1 + B for any agent and any of the statements returned by the generative query, resulting in a --accurate discriminative query. In sum, from the perspective of the algorithm, all generative queries return a different statement, and all returned statements are evaluated the same by all agents.
For (2 + 5- - c, J)-CBJR (or (2.8 + 8 - c, 1 I)-J to hold), 0'x or a copy of a' needs to part of the returned slate W. Otherwise, the set X of agents constitutes violation, as X = t B and u,(a')) = 23 c + 1 and u,(a) 1 < 2/3 + 1.5++1 1 (2 + e) = 1 + for all i X and W.
Let X' be the set of all (i) possible size-t sets of the n agents. Let a Bg be the statements that are part of W and let NI.....NY .g be the agent sets that were input in the generative queries to produce the respective statements. We will now argue that (under certain conditions on n and B) we can always find a way to pick X E X so that for each i E [B], a; ay., for some j [2"] and Y' C.. For this, for each i [B]. it needs to hold that INN, n X| < [Ng] (the half condition), as this implies by the construction of the generative query that a; = ax. for some j E [2"] and Y' C Y. For each statement a; the half condition rules out some sets from X' that can no longer be picked as X, i.e., all S E X with |N, > |N.1 pick.
However, for certain values of B and n, there will always be an element from X' remaining that we can 2
Consider as an example B 2 8 and n = 2B. IN = {z} for some x E [n], x cannot be part of X. If [N,] = 2, X cannot be identical N, If |N 3, X cannot be one of the three size-2 subsets of N;. If N,, 2 4, the half condition becomes vacant and no sets from X' are excluded because of this N,. Thus, each N; can either rule out from X' all sets containing some specific agent or at most three (arbitrary) sets from X. Let 2 == Ki [B] |Ng| = 13-- Then the total number of sets from X' that respect the half-conditions for all Ni...., NB are
n - 2 - 3(B 2) (
2BB 2 (2))))223
B
3(B -) > -B 2
3B 2 1,
2
2
where the inequalities hold < B and B 2 8. Thus, we can always find set X that respects the half-condition for all statements selected in the slate. We set X accordingly. X constitutes a (2 + 8 - c, 1)BRJR violation for the produced slate.
Theorem 3.4. Fix some " E Q1>a2o, and E No- No algorithm with access to exact discriminative queries and (7/, 0, acccraaee generative queries can guarantee (p, 1 [W] returned by the algorithm." !!
p [W]++1/ .BBJR [or (p, 1 [W] a [Wh++ -cJR],
) where W is the slate
Proof. Note that 7 and ja are given and fixed. We will now construct an instance on which no algorithm can fulfill the above guarantee. Let s, f E N so that = ; and a, h E N so that = a (note that / > 8 and b > a). In the proof, we set 5 r = p 2. However, we will only make use of two utility levels p + 2 and 1. We say that an agent i approves a statement a if (aaa) = p + and disapproves if u,(a)= 1.
In our instance, we pick B that 6 are integers. For notational convenience, let == ..+ E 12 All statements in the universe have either cost 1, a, or b. For
B is integer and set ls = B Further, we pick and B that n K and T 8.
n
Techhically our proof shows the following: For given instance with budget B, where e is the minimum cost of a statement that is B/c approved at level 2 by at least one agent and returned by the generative query, no algorithm can guarantee (p, n/or+ 1 CBBR.
Note that possible values for n and B fulfilling the above constraints are = bt and n = (s + 1)tb, as in this case k = t,
= (s + 1), -म:
(e+1)eb tb and 7h thy = sb.
*+1
1
14

---

Generative Social Choice: The Next Generation
Algorithm 2 Unntomm-pppox-Demmrrttitte
p,1,),
1: S + N, W +- f
2: while / 2 and S o and c(W) < B do
3: U - Ur [GEN(S, &, 11
4:
Sa < EiES Disc(i, 2 l) for a E
5:
a" - arg aax [Sa]
6:
if [S 2 Bx+1 ay
my then
7:
S + 5 ff
agents from S.- with highest Disc(i, a*) return value)
B
8:
W + W U (a*}
9:
else
10:
(
11:
Return W
each subset S of agents with |S| S 9, create statement ag and statement 03 with costs c(as) and c(ag) that are approved by all agents from S and disapproved by everyone else. Moreover, there is dummy statement disapproved by everyone which costs 1. Generative queries behave as follows: If a query with cost I < b is cast, we return the dummy statement (note that this does not violate the (7, 0, (1) -accuracy of the generative query, as x < S b, implying that there is no statement a with an approval satisfying the constraint a E U c(a:) S μ x). Otherwise (x 2 6). for some group S of agents given in the generative query, we pick an arbitrary subgroup S' C S size min([S). 79) and let the generative query return the statement ogy (which is approved by everyone from S' and disapproved by everyone else and has cost b). Thus, all generated statements approved by most 7/ agents. As g and "1 are integers, the constructed queries clearly fulfill the definition of (7, 0, pa) -accuracy..
We will prove that no algorithm can guarantee (v, 1. which immediately implies the theorem, as kb = B. Note 1 that on the constructed instance (p, j k )-cBJR implies that there cannot a group of agents of size g that all disapprove the statement they are matched to in a slate returned by the algorithm: Each such group S jointly approve the statement ag of cost and S is "by at least factor of &
# ky++I larger" than group that deserves statement of cost o, & an ky+ g. We will show the slightly stronger statement that for each feasible i.e., |S] 2 KT++" [ 1a k7+ B slate consisting of the statements returned by the above-described generative query there are agents that disapprove all statements from the slate, which constitutes a violation of the (p. ky+1 cBBJJ guarantee. Notably, this would also imply that the impossibility result extends to cJR.
k.
& bn
To see why such a group of g agents always exists, note that slate contains at most k statements approved by at least one agent and that all of them will be in fact approved by at most "g agents. Thus, for each slate, there will be at least kng agents that disapprove all statements from the slate (recall that kyg is an integer). It remains to argue that n kyg 2 g:
π kng 2 g
7
n
ky
ky
+
kr +
7k
1 ky
k7+1
121
Proposition 3.5. Let 6 > O. Assuming all statements have unit cost, given access to B-accurate discriminative queries and (7, 8, ( acccurate generative queries, there an algorithum (Algorithm in Appendix B) that guarantees (2,5 + B c)- cBJR when μ and y are known.
B
Proof. Algorithm 2 proves the proposition. Let W be the returned slate. For each E N, let ((1) be the statement added ny to W* during the round in which was removed from S. The resulting assignment balanced as B < B. Agents that remain S the end of the algorithm are mapped to arbitrary statements while maintaining the balancedness of w.
15

---

Generative Social Choice: The Next Generation
Errors
Average Utility
Average Uniform
1Oth-Percentile Utility
#instances w. cBJR violation
B
/4
Uniform
Fast
Complex
Fast
Complex
Uniform
Fast
Complex
0.0
1.0
1,0
0.0
4.56
4.49
4.49
1.33
1.43
1.51
31.0
0.0
0.0
1.0
0.85
0.85
1.0
3.36
3.86
4.26
0.05
0.8
0.98
98.0
65.0
45.0
2.0
0.7
0.7
2.0
2.96
3.15
3.44
0.01
0.33
0.53
99.0
98.0
100.0
3.0
0.55
0.55
3.0
2.79
2.62
2.95
-0.04
0.13
0.22
99.0
100.0
100.0
Table Metries produced slate W and mapping w for three versions of our democratic process under varying errors.
For the sake of contradiction assume that there is group T of agents witnessing violation of (2 + ô, By+1
B - c)-cBJR
for W* at threshold 0, caused by statement a', i.e., (T] 2 ( B++1 e) [ ue ue(a') 2 and uy (w((i) < (2 + 8) for
all i T. We make a case distinction:
Some agent from T gets removed from S throughout the algorithm Consider the first agent E T removed from $ during the algorithm. Let é and a be the values of the respective variables and at during the iteration when i is removed from S.
We claim that it suffices to show that 2 0 - (3 + 8): By definition of w and S w(() a and Disc at) 2 ?. Since discriminative queries are 8-accurate, this implies that u;(a)) 2 0 - (23 + 5). a contradiction
T.
So, for the sake of contradiction, assume that ē < 0 (3 + 5). We argue that this would imply that an agent from T should have been removed from in an earlier iteration, leading to a contradiction. Consider the last time in which Line is visited for f = 0 (B + 5) (no statement added to W this iteration). Note that as i is the first agent from T that gets removed from S, we have S S in this iteration. The inner part of the while-loop will be entered in this iteration, as i will get removed from S at level i < f, implying that we have & > 1 and e(W) B in this iteration.
As part of this iteration of the while-loop in Line 3, we will call GENC(S, 0, L (as 0 2 f). Let Ç be the statement returned by this query. Note that a' is "relevant" for this generative query: a' {a E u c(a) S [u [ 1) by Lemma B.3. As T C S, we know by the definition of T that
B
max sup(a, 8,0) 2 [7 (EMic(a)S(ee L(
By
As generative queries are (7,6,1))accooo accuurate it follows that sup(ç, 8) 2
BT+I [ Thus, there exists set D of at least By+T [ 2 BT+I agents from S that have utility at least 5 for Ç. As discriminative queries are --accurate and f = 0 - (B + 8), it follows that D S Sc and thus that [S&| 2 "y_ As result, we will have |S.- 2 By+I the if-condition in Line 8 is satisfied and a' is added to W. contradiction.
B
y
ny
ny implying that
No agent from T gets removed from S throughout the algorithm It remains to argue why it cannot be the case that no agent from gets deleted from S over the course of the algorithm. Note that the above argument implies that some agent (3 +
For this We claim that when adding B statements to the slate W. the algorithm needs delete at least one agent from T. For this, observe that for each statement delete BB+1 agents. Thus, B By agents
from T' gets deleted from S, if we reach level L
happen, we need to add B statements to
W before reducing é 0 (B + 7)
get deleted. It remains to prove that B
n7 B
B
implying that less than [T]
By+T - e) [ agents
B
remain at the end of the algorithm:
B
B
nBy
By +
B
nl
B++1 +1
B
By
+1
By
+
B
B
+1
By
B
+1
By +
C. Additional Material for Section 3.4
Utility We show in Table 2 some metrics regarding the utility agents have for their matched statement in the slate in our experiments in the synthetic environment. We observe that while all three variants perform similarly under no errors or very
16

---

Generative Social Choice: The Next Generation
REsmUm ic
14
-2
persmene
orga
(a) Average maximum d with (0, d))cBJR
Average maximum with (c, J-BB.RR
(c) Average maximum with (c. 1)-cBJR violation when varying parameter B.
violation when varying parameters j and y.
violation when varying parameter
Figure 2. Comparison of proportionality violations introduced when only modifying parameter and keeping the others accurate.
0.
(a) 3 = 8 0,
(b) 8 = 0 = 1,
0.85
(c) 3 = 2, =
0.7
(d) 8 = 3: 0.55
Figure Analogous to Figure but with worst-case model.
Errors
Average Utility
Average 10th-Percentile Utility
#instances w. cBJR violation
μ
MW
Simple
Complex
MW
Simple
Complex
MW
Simple
Complex
0.0
1.0
1.0
0.0
4.56
4.49
4.49
1.33
1.43
1.51
31.0
0.0
0.0
1.0
0.85
0.85
1.0
3.17
3.73
4.01
0.0
0.8
0.9
100.0
81.0
68.0
2.0
0.7
0.7
2.0
1.81
1.97
2.73
-0.28
-0.21
0.36
100.0
100.0
100.0
3.0
0.55
0.55
3.0
0.79
0.88
2.46
-0.61
-1.13
0.18
100.0
100.0
100.0
Table 3. Analogous to Table but with worst-case error model.
high errors, differences emerge in the medium-error regime. Specifically, Comp lex starts to outperform Fast, which in turn performs better than Uni form...."" 13 A similar trend appears when taking more egalitarian perspective and examining the average utility of agents in the bottom 10th percentile. Here, the relative differences between the three variants are more pronounced. With respect to cBJR, Uni formm already violates cBJR in one-third the instances even when queries are exact. When query errors are introduced, cBJR violations also quickly emerge for slates produced by the other two variants, with Complex showing slightly better performance for small errors.
Scaling in Different Parameters Figure 2, we analyze how Complex behaves when increasing one error source while keeping the others at the accurate level.
WorsttCasee Error Model In addition to the error model described in the main body, also explored more worst-case- focused approach. For discriminative queries, always add subtract 3 from the true utility random. For the generative query. we sample from the statements that achieve the worst possible ratio in Equation (1). Results can be found in Figure 3 and Table 3.
Note that processes designed to maximize average utility. For example, slate consisting of single long statement covering the majority opinion may achieve high average utility while severely violating proportionality.
17

---

Generative Social Choice: The Next Generation
D. PROSE: Implementation Details
D.I. Implementation of Discriminative Query
One key challenge in implementing discriminative queries in our setting is calibrating the evaluation of statements with varying lengths and levels of detail. Specifically, we observed that LLMs tend rate short, generic statements more favorably. To address this, compute separate scores (both on scale from 6): Agreement, which measures the extent which the agrees with the details of the statement, and specificity, which measures the extent to which all details from the user's description are reflected in the statement. The final utility score is computed as: wwweeenntsspecifficccce
coefficient- (6- (6--ppcificityy//s where specificity coeffi ieffiiientt is hyperparameter that allows users to influence the desired level of detail in the produced slate. 14 To calculate the agreement and specificity scores, we pass the statement and user description, along with a carefully designed description of the scoring criteria, as part of a prompt to the LLM. The final scores are derived as weighted averages of the LLM's log probabilities over its output tokens for this prompt.
System Prompt for Agreement
Your task is determine much a user would agree with some statement. You will be given information about the user's opinions. Respond with number between 1 and 6.
Prompt for Agreement
Rating scale meaning:
[6: "Perfect match. User agrees 100% with all details in the statement.")
5: "Near-perfect match. User agrees with 95-998 of the statement, with only minor discrepancies.",
4: "Substantial agreement. User agrees with about 70-808 of the statement's content.".
"Moderate agreement. User agrees with roughly half of the statement points.",
2: "Minimal agreement. User agrees with only small portion (about 20-308) of the statement.",
1: "No agreement. Statement contradicts or is irrelevant to user's opinion."
Statement:
<insert statement>
User information:
<insert user information>
Now it is time for you to give the most accurate numerical rating. Write number between 1 and and nothing else. Your estimated rating:
System Prompt for Specificity
Your task is to determine how much detail statement has. You will be given a statement, and information about the user's opinions. Your task is to determine how many *specifit fic details* from the user's opinion are present in the statement. Respond with number between and 6.
Prompt for Specificity
IIIn our experiments, we set epecificity coefficient
18

---

Generative Social Choice: The Next Generation
Rating scale meaning:
(6: "Exhaustive. All specific details from user's opinion present in the statement.",
5: "Nearly complete. Statement contains most specific details from user's opinion, with only minor omissions.",
4: "Partially detailed. About 70-808 of specific details from user's opinion are included.",
"Moderately specific. Roughly half of the specific details in the user's opinion are mentioned.",
2: "Minimally specific. Only few (20-308) of the specific details in the user's opinion are included.",
1: "Non-specific. Statement is entirely general, without any specific details from user's opinion.",
Statement:
<insert statement>
User information:
<insert user information
Now it is time for you to give the most accurate numerical rating. Write a number between and and nothing else. Your estimated rating:
D.2. Implementation of Generative Query
We employ multiple strategies to generate statements. We refer each strategy generator. All generators consist of two stages. In the first stage, we identify groups of agents with similar opinions. In the second stage, for computed group of similar agents, we pass the descriptions of all users in the group to the LLM and prompt it to generate statement of at most a given length that accurately reflects the opinion of all users in the group.
D.2.1. EMBEDDINGS
We consider two types of embeddings. First, we embed each agent using their description via OpenAl's embeddi -oadddigg---argge
The computation of the second embedding is slightly more involved. We first use LLM queries to create a list of brief statements covering all issues or opinions raised by any user. Then we subsample 50 of these statements and apply very cheap variant of the diseriminative query to compute agents' agreement with the statements on the following scale: 1: Strongly goes against user's opinion 2: Goes against user's opinion 3: Somewhat goes against user's opinion 4: Neutral unknown 5: Somewhat aligned with user's opinion 6: Aligned with user's opinion 7: Strongly aligned with user's opinion The computed ratings will serve our embedding.
D.2.2. FINDING "COUESIVE" GROUPS OF AGENTS
Given embedding of the agents, employ four differrnnt strategies to identify cohesive groups of agents given call of the generative query GEN(S, f,
TageNGeeerator We select an agent at random from S and compute the 2-n agents from S with the smallest Euclidean distance to the agent in the embedding.
WeightedNNGGnnerttorr For each agent from S, we compute the I agents from S with the smallest Euclidean distance to the agent the embedding. Then, we assign probabilities to these generated clusters so that each agent has a similar probability of being included in the sampled cluster. We draw a sample with these probabilities and return the sampled cluster.
CussesCChuteerGeeerttooo For each agent from S, we create cluster of containing the agent and iteratively adding agents from to minimize the resulting summed Euclidean distance
2:2 B agents by starting with the cluster only
19

---

Generative Social Choice: The Next Generation
between agents in the cluster. We sample generated cluster with probability anti-proportional to the summed distance between agents in the cluster.
PreviousBestGeneratoor We go through all previously generated statements and compute for each of them the agents from
S approving the statement level f. We return the biggest so-computed group.
D.2.3. GENERATING STATEMENTS
Once we have identified group of agents, we prompt the LLM with their descriptions as part of the following prompt generate consensus statement for this group:
System Prompt for Consensus Statement
You will be given information about a group of users and their thoughts topic. Your task is to write short, strong opinion that reflects a single, clear stance reflecting the users' thoughts. The opinion should sound personal, direct, and conversational, as if written by someone expressing their own thoughts. Avoid summary-style language or listing multiple viewpoints. The level detail included in the opinion you write should be VERY HIGH. Use most <word budget> words. Write the opinion in XML tags </tiiitt..////iiion..
Prompt for Consensus Statement
User information:
<list of user information for all agents given as input to generative query>
Now write the opinion.
D.3. Other Implementation Details of PROSE
Generators For each cost value and approval level we iterate over, we use each of the four generators twice, where for TagNN, WeightedNN, and ClosestCluster we use each embedding once.
Cost Values For cost efficiency, and based on the strong performance of the Fast algorithm in Section 3.4, well as our observation that produces similar results, the cost list C includes only a selected subset of possible cost values.
In particular, for the three drug review instances, use C [80, 70, 60, 50, 40, 36, 32, 28, 24, 20, 16, 12, 10, 8, 6, 4, 2]. while for bowlinggreen which has a different word budget per agent, we use C [80, 60, 40, 36, 32, 28, 24, 20, 16, 12, 8,
Approval Levels We use l = [5.5, 5, 4.5, 3.5, 2, 1, 0] for each of the instances.
Minimum Statement Lengths To ensure that meaningful statements get added to the slate, we impose a minimum statement length 10 for the drugs dataset and of for the bowlinggreen dataset. The minimum statement length waived
once we reach
D.4. Evaluation Shain-of-Thoughee Discriminative Queries
use following query in our evaluation the generated slates to compute reliable utility values for agent, statement pairs:
System Prompt
You will be provided with survey responses from a user, and separate statement. Your task is to determine the extent to which the user would agree with that statement, on scale from to 6. Your response should be JSON containing your responses for each step in reasoning.
20

---

Generative Social Choice: The Next Generation
Prompt
**User description:**
<insert user information
**Statement:**
<statement>>
**Instructions:**
To determine the extent to which the statement fully summarizes the user's opinion and reasoning behind it, follow these steps. Be very concise when addressing each step.
Step 1. Summarize the user's opinions and reasoning on the topic. Include a few concrete examples in your summary.
Step 2. Explain which aspects of the user's opinion and reasoning the statement fails to touch on.
Step 3. Explain which aspects of the user's opinion and reasoning the statement actively contradicts.
Step 4. Weigh all of your considerations tulug determine overall how much the statement summarizes the user's opinion and reasoning. Then select from one of the following 6 choices.
6: "the complete details of the user's opinion and reasoning are captured by the statement",
"almost all the details of the user's opinion and reasoning are captured by the statement, with very minor contradictions",
4: "around two-thirds of the details of the user's opinion and reasoning captured by the statement, with few contradictions",
3: "a majority of the details of the user's opinion and reasoning are captured by the statement, with some omissions / contradictions",
2: "the user would at best slightly agree with the statement, because the statement only partially captures their opinion and reasoning, or is missing something",
1: "the statement is either heavily incomplete, or contradicts / is orthogonal to the user's opinion and reasoning",
**Cutput instructions:**
Respond in JSON as follows:
"step1" " <your response step 1>,
"step2" <your response to step 2>,
"step3" <your response to step 3>,
"step4" <your response to step 4>,
"score" <your score, a number between and 6>
21

---

Generative Social Choice: The Next Generation
D.5. Verification of Discriminative Queries
In this section, we validate the two discriminative queries used in our experiments: first, the discriminative query used in PROSE, and second, the chain-of- of-thought discriminative query used in the evaluation.
For this validation experiment, we use publicly available up- and downvote data from the Polis BowlingGreen dataset. (To ensure a sufficiently large sample size, we use the full dataset consisting of 2031 users, as opposed to the curated dataset of 41 users used in the rest of our experiments.) For each of the 51 users who placed at least 5 upvotes and 5 downvotes on comments they did not author, we select 5 upvotes and 5 downvotes uniformly at random. For each uscr-vote pair, we use a discriminative query to estimate how much the user approves the statement they voted on.
For both the discriminative query used in PROSE, and the chain-of-thouge discriminative query, we observe that 84% (CI: 72.5% 92.99.9999 1%) of users have higher mean discriminative query scores for upvoted statements than downvoted statements. Thus, both queries can reliably predict user voting behavior from user's written beliefs.
To verify the independence of the queries, for each discriminative query, we construct a vector of length 51, indexed by the users. Each user's entry is the difference between the mean discriminative query output on upvoted statements with the mean discriminative query output on downvoted statements. The Pearson correlation coefficient between these two vectors 10.13.11eee relavvlyyy low considering the high accuracy of both queries- demonstrating that the two discriminative queries are relatively independent, not just in their implementation, but also in their behavior. 15
E. Experiments: Additional Details
E.1. Datasets
For the birth control dataset, we take all reviews on "Ethinyl estradiol norethindrone" from the UCI MI Drug Review dataset (Gräßer et al., 2018). With the help an LLM, we discard all reviews that speak about a specific brand. Out the remaining 1275 reviews, we keep the reviews whose length between the median and 75th percentile. For the uniform dataset, we sample reviews with rating æ for each æ E [1, 10). For the imbalanced dataset, we sample 20/10/20/10/2 10/22 reviews with rating 1 1/2/5/9/10.
For the obesity dataset, we take all reviews on "Contrave" and apply the same procedure above, except that we keep all reviews whose length is between the 25th and 75th percentile (as there are fewer reviews for this drug).
E.2. Baselines
1. CONTEXTLESS ZERO-SHOT
We query the LLM with the following prompt:
System Prompt
Your task is to write a *proportional opinion slate* on a particular topic while staying within word budget. proportional opinion slate is a collection of opinions that, taken together, give an overview of the general population's opinions on that topic. Morover, the lengths of the opinions should correspond to the relative proportion of people that hold that opinion (hence "proportional" opinion slate).
**Stylized example:** Suppose 708 of people believe salads are the best dinner, 20% of people believe burgers are the best dinner, and 108 of people believe soup is the best dinner, and the word budget is words. Then, proportional opinion slate might look like this:
The inter-rater reliability score (via Cohen's 's kappa) for the two implementations is 0.41 (fair to moderate agreement). For 39/51 users (76%), both implementations return higher mean scores upvoted statements than downvoted statements; for 8/51 users (16%), one implementation returss higher mean scores and the other lower mean scores on upvoted statements (each implementation is correct 4/8 times).
22

---

Generative Social Choice: The Next Generation
Salads are the best dinner option by far. They are healthy and tasty. They fulfill all major dietary requitements so anybody can have them. They also flexible, since many different ingredients can be substituted. Burgers make the best dinner because they're re tasty and filling.
Soup makes the best dinner.
*Writing guidelines:** Each statement should be a short, strong opinion that reflects a single, clear stance reflecting population segment's thoughts. The opinion should sound personal, direct, and conversational, someone expressing their own thoughts. Avoid sumnary-style language or listing multiple viewpoints. Finally, do not list word counts next to your statements just put the statements and nothing else.
if written by
The topic: (topic)
The word budget: budget) words. Do not write more than (word_budget) words under any circumstances!
Prompt
Write your proportional summary below. Write each statement on a new line, and use " " as bullet points. Respect the word limit!
E.2.2. ZERO-SHOT
We use the same system prompt as for contextless zero-shot with the following prompt:
To improve the accuracy of your preportional opinion slate, below provided are representative opinions that people have on the topic. You should aim to construct a proportional opinion slate that matches the distribution of these opinions.
(user_opinions_list__tttt
Write your proportional summary below. Write each statement on a new line, and use as bullet points. Respect the word limit!
E.2.3. CLUSTERING
For the clustering baseline, we start by computing an embedding of the description of each agent using OpenAl's embedding- ---iingg----ge.....- Subsequently, we apply PCA compressing the embeddings into five dimensions. Finally, we compute clustering using Affinity Propagation. For each identified cluster of agents, we generate statement for the cluster using the prompt from Appendix D.2.3 with word limit of
F.3. Generated Slates
In this section, we present slates generated by all methods we consider. Note that the slate constructed by PROSE typically bit fragmented with multiple statements covering similar opinions One reason for this is the inner workings of the algorithm. The following pattern regularly emerges across instances:
We start with high approval level. At this stage, consensus statement is generated that receives strong not cunnimous-supporr among users with similar opinion. The statement does not receive the required approvals to get added at this high level (but would get added at a slightly lower level).
As the budget decreases, a shorter statement targeted to a smaller subset of the user group with similar opinions is created, which is evaluated highly by those users and is thus added to the slate. (The success of these short statements is in part due
23

---

Generative Social Choice: The Next Generation
the very limited data have available for each user; many users have only few points, which be adequately captured within succinct statement).
After one or two such short statements have been added, the approval level is decreased. At this lower level, the previously generated "long" consensus statement would have sufficient support to be added in case all agents from the dataset were still present. However, at this point, some users that would be covered by the "long" statement have already been removed due to the prior selection of a short statement. As a result, despite the lowered approval threshold, the original "long" consensus statement still does not have sufficient support to be added. This cycle then repeats itself across iterations and datasets.
Notably, this pattern does lead agents having high utility for their matched statement under discriminative query.
E.3.1. BIRTH CONTROL (UNIFORM)
PROSE
This birth control is nightmare- side effects outweigh any benefits. Avoid it at all costs.
This birth control is worth sticking with; initial side effects fade, and benefits undeniable!
This pill wreaks havoc your body and emotions.
This birth control wreaks havoc on emotions, skin, and weight.
These pills cause more harm than good; I'm switching.
This birth control is misery in pill, avoid it.
This pill is fife-changing-clee skin, lighter periods, and boosted confidence outweigh mild, manageable side effects!
This pill works well after initial side effects settle down.
These pills mess with mood, weight, and overall well well-being.
This birth control works, but the side effects can be frustrating!
This pill works well after initial side effects subside.
This birth control causes unpredictable side effects; I'd think twice hefore committing to it long-term.
This pill not worth it; side effects outweigh any benefits.
Disappointing.
PROSE-UnitCost
This pill is an absolute nightmare. It wrecks your body with relentless side effects crfects-crippling cramps, constant bleeding. unbearable headaches, mood swings that spiral into depression, and even hair loss. doesn't fix acne, it piles weight, and the emotional toll is devastating. Sure, it might prevent pregnancy, but at what cost? No one should have to endure this kind of misery for birth control. wouldn's recommend it to anyone-ever ever.
This birth control pill is absolutely worth sticking with despite the initial side effects. Yes, the first few weeks can be roush-nausea, mood swings, spotting, or even weight changes-bur once your body adjusts, it's a game-changer. regulates periods, clears up skin, and doesn't I cause major issues like bloating or severe mood swings for most people. Taking the same time every day. preferably at night, seems minimize side effects significantly. Honestly, the benefits far outweigh the temporary discomfort, and for many, it's the best option they've ve tried. Just be patient and give it time-it's worth it.
This birth control is rollercoaster of side effects that makes me question if it's worth it. Sure, it prevents pregnancy, but at what cost? Between the weight gain, mood swings, acne flare-ups, constant spotting, and complete nosedive in sex drive, it feels like trading one problem for a dozen others. Even the "positives" like bigger breasts or lighter periods come with strings attached-tendernneee irregularity, or worse. It's exhausting to deal with unpredictable bleeding, emotional chaos, and physical discomfort, all while hoping my body will "adjust." Honestly, this pill feels more like a gamble than a solution, and I'm not sure it's one want to keep taking.
24

---

Generative Social Choice: The Next Generation
This birth control is mixed bag of extremes-either it works wonders wreaks havoc. While it's effective for preventing pregnancy and managing issues like ovarian eysts, the side effects are all over the place: mood swings that ruin relationships, unpredictable bleeding, migraines, weight changes, and even terrifying ER visits for seizures or vision loss. Sure, some people see improvements in acne or lighter periods, but the trade-offs- like constant spotting, emotional instability, or debilitating cramps-make make it feel like a gamble. Honestly, it's hard to trust something that can either fix your life or completely derail it.
This pill is mixed bag- while it works wonders for some, regulating periods, clearing skin, and even climinating periods entirely, the side effects are no joke. From weight gain, mood swings, and breast tenderness devastating consequences like blood clots, it's clear this isn't It a one-size-fits-all solution. Personally, I'd be cautious and prioritize thorough discussions with doctor before committing to it. The potential risks too significant ignore, no matter how convenient the benefits might seem.
Clustering
Deadly drug.
This medication ruined my health completely.
This pill a nightmare- weight gain, breakouts, and awful side effects!
This birth control has been a nightmare for me-coontant bleeding, horrible acne, mood swings, and unbearable cramps. It's just not worth the pain and frustration.
This birth control wreaked havoc my beig-weight gain, no libido, mood swings never again.
This pill is nightmare. The severe cramping, irregular bleeding, mood swings, and other distressing effects make continuing it feel unbearable and unhealthy.
This pill is life-changing when used correctly. Stick through the initial side effects, take it at the same time daily, and you'll Il see amazing results!
Honestly, this pill is a game came--hangge--a0 clear skin, no weight gain, no wild side effects, and barely any periods. Totally worth every penny!
This pill can be tough at first, but sticking with it is worth the positive changes later!
Adjusting takes time, but overall it's effective!
Zero-Shot
This medication was devastating for me. It caused severe side effects like persistent acne, mood swings, weight gain, and intense cramps. I would never recommend it to anyone.
experienced horrible nausea, moodiness, and spotting in the early months, but things improved over time, It helped lighten and regulate my periods, and now I'm mostly fine on it.
I've experienced some discomfort, but overall it's effective at preventing pregnancy and hasn't been unbearable. Side effects vary, but they'r re manageable for me.
This pill has been great! My periods are lighter, my acne has cleared, and don't 't deal with cramps anymore. I highly recommend it!
25

---

Generative Social Choice: The Next Generation
Contextless Zero-Shot
I've had no major side effects, and it's been life-changing solution for me.
It's mostly fine, but I do experience mood swings and mild nausea occasionally. It's effective overall, but the side effects can be a bit frustrating.
don't t like using this medication. It caused me severe headaches and weight gain, I had to stop. It just did not work for me.
Honestly, it didn't 't change much for me. My cycles are still irregular, and ' not sure if it's worth continuing.
I've had a great experience with this medication. cleared up my acne and regulated my hormones, so I feel much better overall.
The side effects like dizziness and spotting made unbearable for me. I had to switch something else.
E.3.2. BIRTH CONTROL (IMBALANCED) PROSE
This pill is a nightmare. It wrecked my skin, my emotions, and my body. The side effects are unbearable mabearable-plesse, stay far away from it.
This pill is unpredictable and risky; while it helps some, the severe side effects make it not worth the gamble.
This birth control is litechhanging-lihhe periods, minimal side effects, and incredible reliability. Absolutely love it!
This pill causes too many side effects to be worth
Be patient; benefits outweigh initial side effects over time.
This birth control is effective with minimal, manageable side effects.
This birth control causes too many unbearable side effects to be worth it.
This pill works great with minimal, manageable side effects!
Birth control shouldn't rob you of your sex drive and emotional stability.
This pill works well with manageable side dffects-definitely worth trying!
Birth control works, but side effects can be frustratingly unpredictable.
This pill works, but side effects can be unpredictable and concerning.
PROSE-UnitCost
This birth control might prevent pregnancy, but the side effects are absolutely not worth From relentless nausea, mood swings, and acne to weight gain, emotional instability, and even painful sex, feels like trading one problem for dozen others. It's exhausting to deal with irregular periods, constant spotting, and feeling like your body is completely out of syne. Honestly,
should have to endure this much just for contraception. I'm done with it.
This birth control pill is game-changer. It's incredibly effective, lightens periods to the point of barely noticing them, and for many, clears up skin and eliminates cramps. Sure, the first couple of months can be rough with side effects like mood swings, nausea, or acne, but once your body adjusts, it's smooth sailing. The key is consistency- same time every day makes all the difference. i's not perfect for everyone, but for those works for, it's life-changing. I'd recommend it in a heartheat.
it at the
26

---

Generative Social Choice: The Next Generation
This birth control is gamble wouldn't take again. Sure, it prevents pregnancy, but at what cost? Between the nausea, weight gain, mood swings, loss of libido, and even life-threatening risks like blood clots and seizures, it feels like playing Russian roulette with your health. Some people might love it, but for me, the side effects are just too unpredictable and severe to justify sticking with it. No thanks.
This pill is mixed bag of extremes-whil while it's effective at preventing pregnancy and can clear up acne or regulate periods for some, the side effects are no joke. From nausea, mood swings, and breast tenderness to terrifying experiences like vision loss, seizures, hair falling out, it's clear this isn' 't mee-size-fits-ll solution. Personally, I'd be hesitant recommend it unless you't 're prepared to endure rollercoaster of side effects while hoping your body adjusts. It works, but at what cost?
This birth control is a mixed while works effectively for many and even improves things like acne, lighter periods, or sex drive, the side effects can range from mild nausea and breast tenderness to terrifying health scares like seizures or even fatal blood clots. Honestly, it's gamble with your body, and while some swear by it, others have paid the ultimate price. Personally, I'd be scared to risk it.
Clustering
This drug is dangerous.
This pill causes unbearable cramps and prolonged, unpredictable bleeding.
Worst medication ever; caused unbearable side effects.
This pill dangerous, avoid it!
This birth control has caused relentless acne, mood swings, and physical discomfort-ddefniillyy not worth the misery.
This pill causes more harm than good.
This birth control is nightmare. The side effects are unbearable from weight gain, mood swings, and nausea to painful headaches and zero sex drive. It's simply not worth the toll it takes on your body and mind.
Birth control ruined my moods, skin, and sex drive- definieefy not worth it!
This pill works great once your body adjusts. Sure, side effects happen, but the benefits outweigh them!
aabsolutely love this birth control! It cleared my skin, made my periods super light or nonexistent, and stabilized my mood. No major side effects for me mee-efiitelly worth the cost!
This pill significantly reduces cramps and heavy periods effectively!
Contextless Zero-Shot
My experience with Ethinyl estradiol norethindrone has been overwhelmingly positive. It regulated my periods, cleared my acne, and gave me sense of control over my body. haven't experienced any major side effects, and feel confident using this medication part of my routine.
It works well enough for me, but do experience mood swings and some nausea occasionally. While it's effective for birth control and cycle regulation, these side effects make it less than perfect for me.
The side effects have been too much for me handle constant bloating, headaches, and emotional ups and downs. I stopped taking because I didn't feel like myself anymore.
It caused me significant issues like weight gain and unpredictable spotting. I've decided to try something else because this medication just isn't t for me.
didn't notice much of an effect all, positive or negative it just wasn't the right choice for me.
27

---

Generative Social Choice: The Next Generation
Zero-Shot
This medication has been absolute nightmare for me. My anxiety, mood swings, and depression have worsened dramatically since starting it. My periods have become irregular and extremely painful, and the side effects have made me feel like I'm losing control of myself. cannot recommend this to anyone.
experienced terrible weight gain and breakouts after taking this pill. Despite following a strict diet and exercise plan, I haven't t been able to shed the weight even after stopping the pill. It's completely thrown my body out of balance and I regret trying it.
My experience has been awful. I've had severe nausea, constant bleeding, abdominal cramps, and intense headaches. My mental health has also been affected, making me irritable and emotional. gave this pill a chance, but wouldn't wish upon anyone else.
The side effects of this pill made me feel terrible headaches, nausea, bloating, and zero energy. My periods became unpredictable and heavy. While may work for some, just didn't work for my body.
This birth control has been effective in preventing pregnancy, although I've experienced minor side effects like nausea and aene. It's been manageable overall and did help regulate my period after the first few months.
I've had a great experience with this medication. cleared up my acne, regulated my periods, and made my cycles lighter and more predictable. I'm happy with the results and would recommend it to others trying manage similar issues.
This pill has been game-changer for me. Minimal side effects and periods, which I love. I've had clearer skin, steady moods, and it's been reliable form of contraception. Definitely worth sticking with.
E.3.3. OBESITY
PROSE
This pill pure misery- dizziness, and constant sickness!
This medication curbs cravings, boosts energy, and transforms cating habits!
Weight loss isn't worth these awful side effects and struggles.
This medication a game-changer! It curbs cravings, boosts control over eating, and delivers real weight loss results despite minor side effects. Totally worth for serious lifestyle changes!
This pill powerfully curbs hunger, but side effects can hit hard.
This drug's side effects are unbearable and not worth it.
This pill kills cravings, but effort and lifestyle changes matter!
This pill works for weight loss, but the nausea and side effects make it hard to continue.
This medication curbs cravings but demands effort and lifestyle changes.
This journey is tough, but hope and persistence will bring results.
This drug causes unsettling side effects; take it cautiously.
This medication frustratingly ineffective for weight loss, with minimal results and unpleasant side effects.
Disappointed hope.
28

---

Generative Social Choice: The Next Generation
PROSE-UnitCost
If you're re serious about changing your life, this medication can be game-changer, but it's not magic fix-you have to put in the work. The appetite suppression is real, cravings diminish, and for many, the weight starts to drop, but side effects like nausea, fatigue, dry mouth, and headaches can make the first weeks rough. If you re ready to push through and commit to healthier choices, this can be the tool to help you get there. Just don't 't expect results without effort.
This medication is a rollercoaste--sure, it curbs appetite and helps with cravings, but the side effects are brutal. From extreme nausea, dizziness, and exhaustion to feeling completely out of it, it's like trading one struggle for another. Honestly, it's hard to justify sticking with something that makes you feel so awful, even the scale moves a little. your body can handle great, but for many, it's just not worth the misery.
This pill can be game-changer if you're re ready to push through the initial side effects like nausea, headaches, or fatigue. It's not magic solution, but it genuinely curbs cravings, forces better food choices, and can lead significant weight loss if you stick with it. However, it's not for everyone-some -ome people can't tolerate the side effects, and results vary. If you're serious about change and can handle the discomfort, it's worth trying, but don't expect miracles without effort.
This medication is mixed bag of frustration and faint hopp. It's riddled with side anfects-aaxietyy nausea, constipation, exhaustion, and even mental fog but delivers little to weight loss for most. Sure, it curbs hunger for some, but cravings persist, and the scale barely budges. A few users see minor benefits like pain relief or reduced snacking, but the trade-off is steep. Honestly, it feels like a gamble with your body for results that just don't 't justify the misery. I wouldn't recommend it.
This medication is complete mixed bag-while it can suppress appetite and lead to weight loss for some, the side effects are absolutely brutal and often outweigh the benefits. From crippling headaches, dizziness, and insomnia to anxiety, confusion, and even worsened heart conditions, it feels like trading one problem for a dozen others. Sure, few people see results, but the majority seem stuck battling constant discomfort, unpredictable side effects, and minimal progress. Honestly, it's worth the gamble on your health.
Clustering
Stay away; side effects are unbearable!
Useless and dangerous- never taking it!
This pill causes too many side effects; it's not worth the discomfort.
This medication's 's side effects are intolerable.
The side effects are unbearable, making this pill worth taking.
Nausea and dizziness are tough, but this is worth trying.
This med helps curb cravings and appetite, but side effects like dry mouth, tiredness, or nausea are common early on.
This medication seems to mess with my mind and body too much for the minor benefits it provides.
This pill works well to curb appetite and support weight loss, but dedication and lifestyle changes are essential.
This medication isn't worth the cost side effects for inconsistent weight loss results.
This journey proves commitment and effort bring amazing results.
This medication works but the side effects can be tough.
This medication works for weight loss, but the side effects can overwhelming and frustrating.
29

---

Generative Social Choice: The Next Generation
Zero-Shot
had a terrible experience. The nausea, headaches, and dizziness were unbearable. couldn't eat, felt constantly sick, and had to stop within days. wouldn't recommend this medication to anyone.
I've been Contrave for weeks and haven't 't lost weight. The side effects, like nausea and tiredness, make it hard to continue. I'm frustrated that it's not working as hoped.
This medication suppressed my appetite, but the side effects like dry mouth, dizziness, and digestive issues made hard tolerate. I did lose some weight, but I'm unsure if I'll keep taking it.
Contrave is helping me lose weight, and I've noticed fewer cravings. The side effects were manageable and improved over time. I feel optimistic and healthier.
Contextless Zero-Shot
Contrave has been game-changer for me. It significantly reduced my cravings and helped me control my appetite. I've lost weight steadily and feel like I'm finally in control of my cating habits.
noticed some initial side effects, like nausea and dizziness, but they went away after the first couple of weeks. It's been helpful for my weight loss journey overall.
Honestly, Contrave didn't t work for me. I stuck with it for couple of months, but the side effects were too much to handle, and I barely saw any weight loss.
It's okay, but feel like the results are slower than I expected. helps curb my appetite bit, but still need to work hard with diet and exercise.
Contrave caused too many issues for me. I was constantly nauseous, couldn't focus, and felt worse than before started it. I had to stop taking it altogether.
E.3.4. BOWLING GREEN
PROSE
Refugees enrich communities and deserve warm welcomes.
Warren County deserves countywide high-speed internet as basic public utility!
Traffic laws need strict enforcement for safer roads.
Traffic needs fixing before any new developments.
Every family deserves true school choice access!
Animal cruelty laws must be strict and enforced.
Immigrants need better integration and support systems.
Sex ed must be comprehensive and medically accurate.
Traffic flow improvements are urgently needed; congestion makes driving unsafe and frustrating.
Traffic flow improvements and safer intersections are urgently needed in Bowling Green.
We absolutely need faster, affordable internet expanded countywide to underserved areas.
BG needs traffic cameras to improve safety now.
School district boundaries urgently need major reform.
30

---

Generative Social Choice: The Next Generation
Internet options must expand for residential access.
Bowling Green desperately needs better traffic solutions.
Bowling Green needs fair access to education.
Bowling Green desperately needs more pet resources!
Schools must address bullying immediately and effectively.
Fund arts education now.
PROSE-Unitoost
Bowling Green desperately needs fix its traffic chaos limit unnecessary lights, add turn lanes, block left turns on busy roads, and install traffic cameras enforce order. It's ridiculous how poorly planned and congested roads are, especially during rush hours. And while we're at it, let's hold BGMU accountable for their rates, expand internet infrastructure, and finally give this city an ice rink and proper bypass. These are basic improvements that would make life here so much better.
Every child deserves access to quality education, whether through hetter-funded arts programs, equitable school choice, or addressing overcrowding in Warren County schools. It's time prioritize students' needs over outdated district lines and ensure every family, regardless of income or location, has the opportunity choose the best education for their kids.
Stop wasting taxpayer money on frivolous projects like fountains and start addressing real community needs-fir fix the traffic nightmare, enforce parking laws, and invest education and youth programs. Overdevelopment of rentals killing our schools, and we need stronger tax base support them. Also, animal welfare laws must be enforced, and pet owners need to be held accountable. Build homeless park with tiny homes to give people dignity, and make public spaces safer and more accessible for everyone, especially the elderly and disabled. Prioritize people over cars, enforce parking regulations, and create city that works for its residents, not just developers convenience. Enough is enough-ity time for real change!
BGMU needs to stop hiding behind vague "averages" and actually deliver affordable, reliable internet to resi- thits-thiss 2023, and people in surrounding counties have better options! Also, Bowling Green schools are failing students by ignoring bullying and refusing implement real, comprehensive education. It's time to stop pretending these issues aren't affecting our community. And don't ton't even get me started on the traffic mess at Shive Lane or the lack of a proper bypass-fit fix it already!
Bowling Green thrives because of its diverse refugee and immigrant communities, but the city desperately needs to hold landlords accountable for the overpriced, crumbling rentals they offer. It's time to demand better housing options, including pet-friendly spaces, and ensure new developments don't outpace the infrastructure we rely on. Let's fix the traffic mess Fairview with proper roundabout and stop approving projects that strain our roads. This city can do better, and it starts with prioritizing its people and their quality of life.
Clustering
Fairview Ave desperately needs a traffic circle at Kereiakes Park to fix the chaotic flow and reduce reckless driving through red lights.
Internet should be publicly managed utility, with residential fiber access prioritized. BGMU must lead, ensuring equitable, high-speed internet for all Warren County residents.
Parking laws need strict enforcement-can clogging streets ignoring rules create chaos and danger. Police should actively ticket violators and restore order.
Comprehensive, accurate sex education essential combat teen pregnancy and STDs.
31

---

Generative Social Choice: The Next Generation
The school district boundaries and overerowding issues in Warren County need serious attention. Redistricting and building new schools could better balance student numbers and meet the demands of growing population.
Bowling Green desperately needs to address its traffic chaos; adding turn lanes, blocking left turns on busy roads, and improving traffic flow should be top priorities for the city!
Landlords must ensure safe, affordable rentals; no more overpriced, run-down units!
Refugees welcome!
Zero-Shot
We desperately need faster, more reliable internet across Bowling Green and Warren County, especially in underserved areas like Alvaton.
Traffic in Bowling Green is nightmare. The Bypass doesn't function like real bypass, and Fairview Avenue near Kereiakes Park needs a roundabout to improve flow.
Schools at all levels need daily physical activity combat childhood obesity. It's not optional-it's a necessity for our kids.
Refugees and immigrants enrich our city. We should implement programs to integrate them better into our community and celebrate their contributions.
Build homeless park with tiny homes and a central donation center. Our city has to care for its most vulnerable.
Roads and infrastructure should improve alongside new developments. We can't 't keep approving projects without plan for traffic and utilities.
More needs to done to protect animals. Bowling Green should ban chaining pets and enforce spay/neuter programs through fines.
More affordable, accessible transportation for seniors and disabled residents would make life better for everyone.
Enforcing traffic laws like speeding, running red lights, and expired tags would improve safety and bring in more revenue for the city.
We need more housing options for renters that allow pets, and landlords should be held accountable maintain their properties.
Contextless Zero-Shot
Bowling Green desperately needs better road infrastructure. The traffic is unbearable during peak hours, and the road conditions are poor. Expanding roads and improving maintenance should top priority.
We need more actívities and attractions for families and young people. Investing in parks, community centers, and events would help bring the community together. Bowling Green has potential, but it feels stagnant.
Affordable housing should be focus. Prices are climbing, and it's becoming harder for young families and low-income residents to stay here. More housing developments with reasonable prices necessary.
Public transportation here is almost nonexistent. We need better bus routes and other transport options for those who don't 't drive, especially for seniors and students.
The city could benefit from attracting more diverse businesses. Local jobs are great, but more opportunities in technology or creative industries would bring growth.
Investing in education and local schools is key. Schools need better funding, resources, and attention.
Bowling Green fine as is. People complain too much.
32

---

Generative Social Choice: The Next Generation
F. Additional Discussion
featured in the discussion the main body that more powerful queries would allow us to implement classic PB algorithms. Consider the setting with approval preferences (which corresponds to our setting when setting r = 2). We claim that we could simulate sequential-Prragee Phrraméén given access the discriminative query and the following generative query: Let D E be collection containing each agent's i's current debt. Agents spend money statement but this will increase their debt accordingly. Given D, the generative query returns the statement that can be paid for by its approvers and minimizes the maximum debt of an approver. This will correspond to: argmin <(a)+ di where No is the set of approvers of statement a. This query is sufficient to implement the discrete formulation of Pequential-Phragmén, e.g., as described by Rey & Maly (2023, Definition 4).
[Nol
Additionally, more complex queries would also allow us to aim for additional axiomatic guarantees. For instance, EJR+ up to any project for cost utilities as defined by Brill Peters (2023, Definition 16) could be achieved given access to the following (+o(a) generative query: Let E, x E N and 5 be group of agents. Generate the statement a that satisfies [[=ES) maximizes ii E S (a) 1}| if existent.
S x and
33


Reviews:
[{"review_id": "FMJFooftcV", "reviewer": "Reviewer_MUaS", "summary": "The paper proposes a method for AI assisted democratization, aka using a model to select and aggregate representative candidate statements from social participants. \n\nMain contribution of the work:\n- Adding control for summary length instead of number of representative responses, allowing for direct control on the expected cognitive load to read the summary\n- Make the system more robust by introducing approximate queries, aka fault tolerant to inaccuracies in utility prediction and popular statement generation\n- Experiment demonstrates its effectiveness with GPT-4o as a judge on the user utility, outperforming listed baselines", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their review.\n\n> Regarding the evaluation method: Using total user utility as the main metric in evaluation does not immediately come across as the best way to quantify democracy, but neither was I sure how to make it better –underrepresented groups are still going to be largely neglected in final result generation and their voices deserve to be heard. \n\nWe fully agree that giving minorities a voice is a fundamental aspect of democratic decision-making. This is precisely why our algorithm is explicitly designed with proportional representation in mind – if a group of agents constitutes x% of the electorate, it should be able to exert control over x% of the slate. For example, if the budget allows for 200 words and we have 100 agents, even a minority group of 5 agents will control 10 words, thereby ensuring that a brief summary of their perspective will be present in the final slate.\n\nTo formally capture this ideal, we adopt the axiom of Balanced Justified Representation (BJR). Our theoretical analysis demonstrates that our method satisfies an approximate form of this axiom, even under noisy query implementations (Theorem 3.2). In our experimental evaluation, we measure both total utility but also the frequency of BJR violations (see Table 1, fourth column), which can be interpreted as the fraction of groups whose voices are not adequately reflected in the produced slate, i.e., they are \"underrepresented\". Across all datasets, PROSE consistently yields fewer BJR violations than the baselines, empirically confirming its ability to give a proportional voice to all groups.\n\n> Also want to learn how the algorithm addresses moral dilemmas such as would it choose to kill one to save a hundred. I understand the design of such metrics would never be perfect but still want to learn the trade offs being considered here and how does the pareto surface look like.\n\nOur algorithm does not impose any specific normative stance on moral dilemmas. Instead, it aims to proportionally summarize the diversity of user opinions, even if these are mutually contradictory. For instance, if the electorate is divided on whether it is ever justifiable to harm one person to save many, the resulting slate might include both: “Harming a person is never ethically justified.” and “Saving many can justify harming an individual.” In this sense, our method captures the full spectrum of views rather than resolving ethical trade-offs by adhering to a particular notion. \n\n> In AI for governance works we need to think very critically with regard to what kind of value coordinates are we essentially subjecting the AI to because at the end of the day that's the core added value from human if one day say 99% of the work are automated by AIs.\n\nWe fully share the concern about normative alignment in AI systems. In our setting, however, PROSE is not designed to encode or enforce specific value judgments; rather, it is a tool for faithfully summarizing the input opinions. In particular, the quality and normative content of the output slate largely depend on the user-provided statements. (Broader concerns regarding bias in the underlying LLMs used for query answering are important and discussed in our impact statement.)", "claims_and_evidence": "Claim: The proposed algorithm effectively improves upon previous AI for democracy algorithms. \n\nEvidence: \n- Optimal user utility on four datasets (Birth Control-Uniform, Birth Control-Imbalanced, Obesity, Bowling Green), as presented in Table 1.\n- The cost-budget analysis framework allows for more direct control on the perceived cognitive workload for consuming the aggregated statement\n- Sec 3 proves the near optimality of the algorithm in terms of user utility when the approximated queries are in use", "methods_and_evaluation_criteria": "The paper is well written and easy to follow. Recommending for weak accept for the following reasons:\n- The proposed method allows for an extra degree of freedom in controllability (length instead of number of inputs) and shows best result on its benchmark.\n- Regarding the evaluation method: Using total user utility as the main metric in evaluation does not immediately come across as the best way to quantify democracy, but neither was I sure how to make it better -- underrepresented groups are still going to be largely neglected in final result generation and their voices deserve to be heard. Also want to learn how the algorithm addresses moral dilemmas such as would it choose to kill one to save a hundred. I understand the design of such metrics would never be perfect but still want to learn the trade offs being considered here and how does the pareto surface look like. \n- In AI for governance works we need to think very critically with regard to what kind of value coordinates are we essentially subjecting the AI to because at the end of the day that's the core added value from human if one day say 99% of the work are automated by AIs.", "theoretical_claims": "Been skimming through theorem 3.1-3.4 which ensures near optimality of the proposed algorithm, nothing stands out yet.", "experimental_designs_or_analyses": "See above", "relation_to_broader_scientific_literature": "It's mainly relevant to the following:\n- AI for governance\n- AI for democracy\n- AI for policy making", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "n/a", "other_comments_or_suggestions": "n/a", "questions_for_authors": "see above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WPbLFvXuys", "reviewer": "Reviewer_ddgv", "summary": "The paper addresses the task of producing a slat of statements representative of users'opinions. The framework is based on social choice and supported by large language models (LLMs). Theoretical guarantees are provided about the accuracy of the LLM output. The case studies revolve around city improvement measures, drug reviews and showcase the effectiveness of LLMs to generate concise statements representative of users'opinions.\n\n**Update after rebuttal**\n\nI am satisfied with the authors' answer and confirm my positive evaluation of the paper thus recommending its acceptance", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their review.\n\n>  How do you quantify the tolerance and error made by Chat GPT 4o output? How do you measure the accuracy from the language?\n\nThe general approach taken in this paper is to design a mechanism (Algorithm 1) that is agnostic to the specific implementation of the underlying queries. Crucially, we show that the mechanism continues to satisfy an approximate notion of proportional representation even when the answers to the queries are subject to error. Importantly, these theoretical guarantees hold without the mechanism needing to know the magnitude of the error in the query answers.\n\nTurning to the empirical evaluation of our implementation using GPT-4o: we assess the quality of the discriminative query implementation in Appendix D.5. Specifically, we use a dataset in which users have written comments and cast upvotes or downvotes on comments written by others. We provide our discriminative query implementation with access to a user’s own comments and task it with predicting how the user rated comments they voted on. This allows us to measure prediction accuracy against known ground truth.\n\nUnfortunately, a direct evaluation of the generative query is more challenging, as it would require knowledge of the optimal statement among a virtually infinite set of possibilities – a task that is infeasible in practice.\n\nFinally, we assess the quality of the resulting slates–which represent the ultimate output of our process–through experimental comparison with baselines (see Table 1). Here, we report (i) the average utility that agents derive from their assigned statement in the slate and (ii) the fraction of BJR violations, which serves as a proxy for the number of underrepresented groups.\n\n> Their application is about social choice but their study can be potentially applied for unethical reasons. E.g. I would not leave a political decision be taken or summarized by an LLMs. But for what they are showing this could be very much happening. However, I am not saying they are promoting that.\n\nThank you for raising this concern. As we mention in our impact statement, \"the use of LLMs to rate and generate statements introduces specific risks that must be carefully addressed before deployment in real-world settings,\" including \"bias,\" (lack of) \"transparency,\" and \"manipulation.\" This is doubly true in the context of political decision-making, and we will expand the impact statement to make this absolutely clear.", "claims_and_evidence": "Yes. Statements are supported by theoretical proofs showing in support of the accuracy and robustness of their method.", "methods_and_evaluation_criteria": "yes. The authors integrate LLMs in the pipeline of social choice.", "theoretical_claims": "No I did not check the proofs in detail.", "experimental_designs_or_analyses": "Yes. I checked the implementation details in Appendix D", "relation_to_broader_scientific_literature": "I think the impact is very broad and the paper is timely as it related to LLMs and how they can be used in the context of social choice to represent public opinions", "essential_references_not_discussed": "not that I am aware of", "other_strengths_and_weaknesses": "Strengths: providing theoretical guarantees next to languge output.\n\nWeaknesses: not clear to me how to measure the accuracy of the language output.", "other_comments_or_suggestions": "-In the introduction, \"Related Work\" paragraph. I invite the author to be more precise in the statement \"Another differentiating aspect of our contribution is the focus on deriving mathematical guarantees.\" Explicitly mention what kind of guarantees you will be deriving.\n\n-In the \"General Problem Statement\" paragraph I find the notation a bit confusing as you refer to $\\mathcal{U}$ as the set of all slates and $u_i$ to the utility of agent $i$—change letter for any of the two.", "questions_for_authors": "How do you quantify the tolerance and error made by Chat GPT 4o output? How do you measure the accuracy from the language?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review.", "ethics_expertise_needed": ["Responsible Research Practice (e.g., IRB, documentation, research ethics, participant consent)"]}}, {"review_id": "VVsfLpG4Wl", "reviewer": "Reviewer_dWvn", "summary": "The authors consider the problem of generating a set of statements that is representative of a collection of agent opinions on some topic, motivated by participatory budgeting. Extending an earlier definition of proportionality for such a setting, balanced justified representation (BJR), the authors introduce a version with costs and approximate proportionality, (b, d)-costBJR. The authors consider an algorithm that has access to two types of (approximate) queries, (1) discriminative queries, which give the utility of agent $i$ for statement $\\alpha$; and (2) generative queries, which for a set of agents, a utility threshold $\\ell$, and a cost $x$, finds the statement with cost at most $x$ that maximizes the number of agents in the set who have utility for the statement at least $\\ell$. These types of queries were introduced in prior work, but the authors extend them to the approximate case. They provide an algorithm which approximates $(b, d)$-cBJR if given access to approximate discriminative and generative queries. They also prove lower bounds matching the utility approximation error and nearly matching the proportionality approximation error. The authors demonstrate their algorithm's guarantee and impossibility results in a synthetic data experiment. Then, they propose an LLM-based approach, which they call PROSE, to heuristically implementing generative and discriminative queries for their algorithm (i.e., DISC = ask GPT-4o to estimate agent utilities for a statement given some text describing their opinion, GEN = ask GPT-4o to write a statement approved by a set of agents). The authors apply PROSE to several semi-synthetic real-world datasets, using a chain-of-thought version of their DISC prompt to estimate agent utilities for generated statements. PROSE performs substantially better in terms of (GPT-estimated) agent utilities and BJR violations than LLM baselines using clustering and zero-shot generation.\n\n### Update after rebuttal\nThanks for the very clear answers! Incorporating this info into the paper/appendix would be great. I'm happy to continue recommending acceptance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments! We will revise the paper to include a detailed discussion of the limitations of our experimental setup and LLM-based query implementations. We will also add a dedicated section elaborating on the relationship to Fish et al. (2024).\n\n> for the drug review datasets, why artificially rebalance the dataset to make the score distribution uniform or highlight extreme and central ratings? \n\nThe main reason for this design decision is that the original score distributions are quite degenerate, e.g., in the obesity dataset, over 70% of users give a score of 9 or 10. This skewness makes the summarization task quite simple, as nearly all users would support the same kind of statements.\n\t\n> Any reason why the budgets were 160 in the drug review experiments and 164 in the Bowling Green experiment? \n\nThe budget in each case was chosen to be divisible by the number of agents (n = 80 for drugs and n = 41 for Bowling Green). This avoids rounding artifacts in the Clustering baseline.\n\n> The measurement of correlation (line 1113-1117) is very strange and hard to interpret. Some more intuitive measures that I would have liked to see: the inter-rater reliability score, the fractions of agent-statement pairs where the difference in labels is 0 and 1, and most comprehensive, the full distribution of label discrepancies for each agent-statement pair.\n\nThe inter-rater reliability score (via Cohen's kappa) for the two implementations is 0.41. For 39/51 users (76%), both implementations return higher mean scores on upvoted statements than downvoted statements; for 8/51 users (16%), one implementation returns higher mean scores and the other lower mean scores on upvoted statements (each implementation is correct 4/8 times). Disaggregating by user and looking at all 1275=51\\*5\\*5 (agent, upvoted statement, downvoted statement) pairs, both queries agree on 53% of pairs, CoT is correct and PROSE is incorrect on 12% of pairs, PROSE is correct and CoT is incorrect on 13% such pairs, and both are incorrect on 21% such pairs.\n\n> *Differences in democratic processes compared to Fish et al. (2024) and missing experimental comparison*\n\nOur democratic process is related to the one of Fish et al. but differs in several aspects: \n1. We account for statements having varying costs (e.g., word lengths).\n2. Fish et al.’s generative query, given a set of agents $S$ and an integer $r$, returns the statement maximizing the $r$-th highest utility among users in $S$. In contrast, our query takes a utility threshold $\\ell$ and returns a statement approved by the maximum number of agents in $S$ at level $\\ell$.\n3. Related to 2., our algorithm iteratively considers decreasing utility levels to decide which statement to add next, which is important for deriving our proportionality guarantees under approximate queries.\n\nAs for the absence of a direct experimental comparison: unfortunately, the implementation by Fish et al. (2024) is not applicable to our datasets, as it requires highly structured user input (e.g., ratings of predefined statements and answers to survey questions). By contrast, our work focuses on settings with unstructured textual input.\n\nWe opted not to reimplement their queries for unstructured data, as this would introduce substantial ambiguity due to subjective implementation choices. Instead, we included the PROSE-UnitCost baseline, which aligns with the core assumption of Fish et al. (2024) – namely unit-cost of statements – but uses our own query implementations to ensure comparability.\n\nAs a side note, we highlighted that PROSE does not require hyperparameter tuning because it flexibly adapts to varying problem instances without requiring dataset-specific adjustments (such as setting the number of output statements, clustering granularity, or query wording). However, we agree that the term is misleading and we will reword it.\n\n> In other words, I feel like the unit cost version of the model makes more sense, but with the addition that statements should be labeled with the proportion of agents who support them at some meaningful level $\\ell$.\n\nWe appreciate this suggestion and believe that the answer is dependent on the intended application. Our model is particularly well-suited for settings where the reader’s attention is limited. For instance, in online democratic deliberation platforms (c.f. Bowling Green dataset), larger groups may reasonably expect a more detailed representation of their opinions: That is, a greater share of the limited attention budget of the reader is devoted to articulating their positions in more depth. Moreover, controlling the total slate length while leaving the number of statements flexible is a more general advantage of our approach.\n\nThat said, we note that our theoretical framework also applies to the proposed model. Further, PROSE could be adapted to return, along with each statement, a support value, and then use that value to define the cost of the statement.", "claims_and_evidence": "The theorems are all supported by proofs in the main text or appendix, and the experiments do a good job of demonstrating the effectiveness of the proposed algorithm.", "methods_and_evaluation_criteria": "I think the methods and evaluation criteria are reasonable.", "theoretical_claims": "The definitions and claims are very clearly presented, and the proofs I checked are well-written and convincing (Thms 3.1 and 3.4), although I have not checked every detail.", "experimental_designs_or_analyses": "I have some moderate quibbles with the some details of the experiment, but given the difficulty of evaluating generated statements, I think the approach the authors took is reasonable. I would have liked a little more spelling out of experimental weaknesses in the text, as described below.\n\nAn important comparison that's missing: it seems like ti should be possible to run the algorithm from Fish et al. (2024) and compare it to DemocraticProcess when used in PROSE. Is this new algorithm that accounts for approximate queries a meaningful improvement when used with LLMs?\n\n- There is clearly some self-confirmation bias in using GPT-4o to compute \"true\" utilities against which the proposed methods are evaluated as well as the DISC queries in the execution of PROSE. But the fact that the baselines are given the advantage of using the true CoT utilities is a nice touch that alleviates this concern. It would be good for the paper to more explicitly highlight this source of bias and the mitigation strategy used to strengthen the baselines by using CoT utilities. Currently this point is relegated to footnotes. Then again, without an actual experiment with human subjects, I don't see a better way of evaluating the utility of generated statements. \n\n- Section D.5 of the appendix presents two seemingly contradictory claims, that (1) both the DISC query and the CoT utilities have high correlation with true thumbs up/down ratings and (2) the two utilities are very weakly correlated. I can see why the authors want both of these things to be true, as (1) supports the validity of GPT-4o utility estimates and (2) mitigates makes the earlier issue of self-confirmation bias, but there's also a tension between those claims. The measurement of correlation (line 1113-1117) is very strange and hard to interpret. Some more intuitive measures that I would have liked to see: the inter-rater reliability score, the fractions of agent-statement pairs where the difference in labels is 0 and 1, and most comprehensive, the full distribution of label discrepancies for each agent-statement pair.", "relation_to_broader_scientific_literature": "This work builds very explicitly on the work of Fish et al. (2024), which is even reflected in the title. It is a clear improvement in the handling of approximate queries and having an algorithm with approximate guarantees. \n\nOne important thing that seems to be missing is a discussion of how the DemocraticProcess algorithm is similar or dissimilar to the algorithm of Fish et al. introduced for exact DISC and GEN queries. How do the algorithms differ?", "essential_references_not_discussed": "None which I am aware of.", "other_strengths_and_weaknesses": "Overall, I think this is a very nice paper, but with a few areas to improve. To summarize what I've written in other sections:\n\nStrengths:\n- The paper is extremely well-written and clearly presented. Overall quality is great.\n- The paper has all of the components I like to see in a combination theory/applied paper: upper and lower bounds and application to real data. \n- The problem is very interesting and the results are strong\n\nWeaknesses:\n- I'm not convinced by balancing on word length of statements. This seems convenient but is a very odd choice for real-world applications.\n- The lack of comparison to the algorithm of Fish et al. (2024) either merits solid justification or it should be added\n- The text of the paper could do a better job highlighting the weaknesses of LLMs rather than only focusing on their advantages (and I say this as someone who thinks an LLM is the best tool for this job)", "other_comments_or_suggestions": "1. If the cost of a statement is its length, it seems quite odd to require that the fraction of agents assigned to statement $\\alpha$ is proportional to the the length of $\\alpha$. It seems more natural to have weighted statements. E.g., with 10 agents, 3 of whom dislike ice cream and 7 of whom like it, it feels odd to require a set of word-length-balanced set of statements like {\"hate ice cream\", \"love love love love love ice cream\"} rather than: {\"I really like ice cream\": 0.7, \"I don't like ice cream\": 0.3} (same word budget of 10 in both cases). In other words, I feel like the unit cost version of the model makes more sense, but with the addition that statements should be labeled with the proportion of agents who support them at some meaningful level $\\ell$ \n2. lines 351-352: \"PROSE also does not require any tuning of hyperparameters\". This is technically true, but only because the approach uses a trillion-parameter black-box LLM with no guarantees for its responses to GEN and DISC queries. Generating statements uniformly at random also requires no hyperparameter tuning--this is obviously an unfair comparison, and I'm sure GPT-4o does a remarkably good job of generating statements based on agent text. But highlighting the lack of hyperparameters feels a bit disingenuous.\n3. I would find 4.1 less objectionable if it highlighted both the strengths and limitations of using an LLM for DISC and GEN queries: yes, massive flexibility in agent preference format (free text, survey responses, ...) and cutting-edge text generation quality, but also no guarantees due to the nature of LLMs and the possibility of errors/hallucinations. I think the Impact Statement does a great job of acknowledging the limitations of using an LLM for statement generation and discrimination, but I would like to see some of these points incorporated earlier into the paper, when PROSE is introduced. \n4. Minor: in lines 751-757, this algebra is correct, but I think it's preferable to avoid the appearance of \"affirming the consequent\" (i.e, the fallacy that x is true because x implies true). Verifying that this indeed establishes the desired inequality requires being sure every chain is an if-and-only-if rather than merely an implication. I would reorder the argument to arrive at the desired conclusion rather than start with it", "questions_for_authors": "1. for the drug review datasets, why artificially rebalance the dataset to make the score distribution uniform or highlight extreme and central ratings? It seems like the more natural thing to do based on the motivation for proportionally representative statements is to use the full distribution of reviews (or at least a representative random sample if the dataset is too large).\n2. Out of curiosity, any reason why the budgets were 160 in the drug review experiments and 164 in the Bowling Green experiment? Not important to the results, just seems like an arbitrary inconsistency.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lS5hGp9Ybe", "reviewer": "Reviewer_PwBp", "summary": "The paper investigates the scope of generative social choice, aiming to generate a slate of statements (usually from a large set such as textual information) representing the voters. Queries (usually implemented by LLMs) can be made on agent utilities and on generating a good statement among a group of agents. The problem is naturally related to traditional committee voting and large language models, which have strong generative power. \n\nThe paper extends previous literature by taking the overall budget and the inaccurate queries into consideration. The paper proposes a new algorithm that generates the slate (and assigns agents to statements). Theoretically, the paper gives worst-case guarantees on the performance of their algorithm reaching the balanced justified representation axiom parameterized by the errors in the queries. They also somewhat show that the guarantee is close to the optimum. Secondly, they show in synthetic experiments that the performance converges way faster than the theoretical guarantee under the same setting. Finally, they implement the generative social choice system PROSE by GPT-4o and test it on real-world data. It outperforms all the benchmarks in both keeping the BJR axiom and maintaining a high utility of the generated slates among the agents.\n\n## After Rebuttal\n\nThe author(s) address my questions well. I am happy to maintain the current score.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. We will improve the clarity of the proof of Theorem 3.1 and expand the discussion on its relation to Fish et al.; see also our response to Reviewer dWvn.\n\n> How do you justify your implementation of GEN queries in PROSE? Does your implementation still follow your theory?\n\nOur theoretical results are agnostic to the implementation of the generative query. In this sense, we view different query implementations as approximate methods for solving the optimization problem defined by Equation (1).\n\nIn practice, we experimented with various implementations and ultimately adopted a two-step procedure. First, we identify agents $S' \\subseteq S$ who are likely to approve the same statement at level $\\ell$, using embeddings and clustering or nearest-neighbor techniques. Second, we prompt the LLM to generate a length-bounded statement that would be approved by all agents in $S'$. That is, we first identify a potential supporter group and then generate a statement intended to satisfy them. We adopted this approach because LLMs were unreliable at discovering such groups. Ultimately, when presented with a generative query, we always produce multiple candidate statements, from which we return the one with the highest number of supporters at level $\\ell$.\n\n> Are there more general impossibility results where parameters are not fixed on special values?\n\nLet us consider Theorem 3.4 as an example. The result is stated under the assumption of no error in how statements are evaluated (i.e., $\\beta = \\delta = 0$). Conditioned on this, the theorem shows that no algorithm can guarantee $(p,\\frac{1}{\\mu}\\frac{|W|}{|W|\\gamma+1})$-cBJR for any constant $p\\in \\mathbb{N}_0$.\n\nThe immediate implication of this is that such a guarantee is also impossible for any other value of $\\beta$ and $\\gamma$ (adding error will never help). One might ask whether Theorem 3.4 can be strengthened for non-zero $\\beta$ or $\\gamma$, but this seems unlikely: Increasing them intuitively only drives up $b$ in $(b,d)$-cBJR. However, Theorem 3.4. already shows that we cannot get a guarantee for any fixed $b$ even for $\\beta=\\gamma=0$. We will make this implication more explicit in the paper.\n\n> Do you think it is more reasonable to also take query costs into consideration?\n\nThis is an interesting point, but somewhat orthogonal to the primary focus of this paper. Our goal is to ensure that the output slate proportionally reflects user opinions – a challenging task in itself. That said, we agree this raises a possible direction for future work: one could consider imposing proportionality not only on the slate but also on the cost of the generation process. For example, one might impose that each agent \"deserves\" that $1 is spent on trying to generate consensus statements they like.\n\n> Does your theory also show something about the utility?\n\nYes and no. We do not provide formal guarantees on average utility across all instances. However, our BJR guarantees ensure that sufficiently large and cohesive agent groups are mapped to statements in the slate from which they derive a high utility. For highly homogeneous instances, this would imply a non-trivial average utility. \n\n> Why can the query output a statement with an accurate cost, but the comparison is made under $\\mu$ error?\n\nThe role of the parameter $0 \\leq \\mu \\leq 1$ is best understood in the absence of other errors. In this case, Equation (1) reduces to:\n\n$$ \\text{sup}(\\alpha^*, S, \\ell) \\geq \\max_{\\alpha \\in \\mathcal{U} : c(\\alpha) \\leq \\lceil \\mu x \\rceil} \\text{sup}(\\alpha, S, \\ell) $$\n\nThis means that the returned statement $\\alpha^*$, even if it has cost $x$, must only be as well supported as any statement of cost up to $\\mu x$.\n\nWe introduced this parameter because LLMs often undershoot the provided word budget in practice. Intuitively, the model would internally only search for statements in a more conservative space (shorter than allowed). As a result, we can only expect to identify the best statement among those of length at most $z$ (for some $z\\leq x$). This behavior is captured by $\\mu$, which quantifies this budget undershooting.\n\n> Is there any runtime evaluation on PROSE (and probably other benchmarks) and between different procedures in PROSE?\n\nThe runtime of PROSE is dominated by the response times of the LLM used in our query implementations. Our algorithm makes $\\mathcal{O}(r \\cdot n)$ generative queries and $\\mathcal{O}(r \\cdot n^2)$ discriminative queries. In our experiments, across the four datasets, PROSE used 9.6M–25.4M input and 53.5K–96.1K output tokens, with runtimes of 31–65 minutes on a single Intel i7-8565U CPU @ 1.80GHz. However, given the rapidly improving inference speeds of modern LLMs, we expect these runtimes to significantly decrease in the future.\n\nBy contrast, PROSE-UnitCost required around five times fewer resources: between 2.1M and 4.4M input tokens and 15.4K to 20.2K output tokens, with runtimes between 7 and 12 minutes.", "claims_and_evidence": "Yes. All the claims are supported by theories or experimental results.", "methods_and_evaluation_criteria": "In general, yes. This paper uses LLM to evaluate the average utility of the outcome slate in real-world experiments. The evaluation method is seemingly similar to the implementation of the DISC query (utility query) in their algorithm, but they run validation experiments to show that they are likely independent.", "theoretical_claims": "I check Theorem 3.1 and browse Theorem 3.2. I think they are correct.", "experimental_designs_or_analyses": "I am not very familiar with examining experiments, but in general, especially the validation experiments, looks god to me.", "relation_to_broader_scientific_literature": "The paper follows a very newly-emerged yet heated topic of generative social choice [Fish et al, 2024], which has the potential to open an utterly new research direction in this area. This paper extends the idea to more practical settings, making the prospect more realistic. The paper is also closely related to the combination of LLM and social choice/multi-agent systems, bringing insights into AI-augmented voting and decision making.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths: I think this is in general a very good paper. It studies a topic full of potential (generative social choice) and gives a rigorous and practical solution. They aim to address the budget constraint and the inaccuracy of the queries (usually comes from the unstableness of LLMs) and solve the problem with a newly designed, theoretically guaranteed algorithm. The algorithm and the theories are not direct extensions of previous literature. Impossibility results are also given to illustrate the (nearly) closedness of the worst-case bound. Moreover, they implement the algorithm in practice and demonstrate its capability on real-world data. The paper significantly removes the assumptions and broadens the application of generative social choice, making this AI-augmented voting schema more promising. The presentation is also good, with most of the important assumptions or designs explained (such as BJR) or validated via experiments. \n\nWeaknesses: \nI don't have major complaints about the paper, yet some points are worth improvements\n\n1. Proof for Theorem 3.1 is a bit dense for now. As this is the only proof in the main paper, I suggest that give a clear explanation of every important step to convey your key theoretical techniques. \n\n2. The paper can benefit from a more clear discussion between itself and [Fish et al, 2024] on, for example, why its result is not a natural extension of the existing work. \n\n3. The impossibility results consist of multiple theorems in which some parameters are fixed to 0 or 1 (the exact version). Are there more general results?\n\n4. The implementation of the GEN queries in PROSE seems different from those in the theories. Instead of finding a statement with max support in a given subset, it seems to cluster agents and find statements for clusters. How do you justify this?", "other_comments_or_suggestions": "Typo:\nLine 2546-252: I suppose $\\zeta$ and $\\alpha^*$ refer to the same statement?", "questions_for_authors": "1. How do you justify your implementation of GEN queries in PROSE? Does your implementation still follow your theory?\n\n2. Are there more general impossibility results where parameters are not fixed on special values?\n\n3. You assume the cost is only on the output. Given the implementation requires a considerable number of LLM calls, do you think it is more reasonable to also take query costs into consideration? \n\n4. Your theoretical results only consider the satisfaction of BJR, while your experiments have positive results on the average utility of the outcome slate. Does your theory also show something about the utility? \n\n5. The error parameter $\\mu$ only works on comparing the query output with other potential statements. I think this needs a justification. Why can the query output a statement with an accurate cost, but the comparison is made under $\\mu$ error? \n\n6. Is there any runtime evaluation on PROSE (and probably other benchmarks) and between different procedures in PROSE?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper deals with "generative social choice" (Fish et al., EC 24), where instead of restricting the set of candidates to the specific statements submitted by participants, we view every possible statement as a potential candidate. This paper extends Fish et al. in a non-trivial manner, namely to include costs and budgets and approximate queries. The authors' rebuttal clarified several issues the reviewers raised, and I hope the authors will address them in their revision. 

Overall, the reviewers were positive about this paper, and I share their enthusiasm. I recommend accepting this paper.
Decision: Accept (oral)