PerSpectra: A Scalable and Configurable Pluralist Benchmark of Perspectives from Arguments
Authors: Shangrui Nie
Kian Omoomi
Lucie Flek
Zhixue Zhao
Charles Welch
Venue: ICLR 2026 Poster (2026)
Keywords: pluralism
Argument
Benchmark
Primary area: datasets and benchmarks
TL;DR: We present PerSpectra, a scalable benchmark that integrates debate structure and linguistic diversity to evaluate pluralism in large language models.

Abstract: Pluralism, the capacity to engage with diverse perspectives without collapsing them into a single viewpoint, is critical for developing large language models that faithfully reflect human heterogeneity.
Yet this characteristic has not been carefully examined within the LLM research community and remains absent from most alignment studies.
Debate-oriented sources provide a natural entry point for pluralism research. Previous work builds on online debate sources but remains constrained by costly human validation.
Other debate-rich platforms such as Reddit and Kialo also offer promising material: Reddit provides linguistic diversity and scale but lacks clear argumentative structure, while Kialo supplies explicit pro/con graphs but remains overly concise and detached from natural discourse.
We introduce PERSPECTRA, a pluralist benchmark that integrates the structural clarity of Kialo debate graphs with the linguistic diversity of real Reddit discussions. Using a controlled retrieval-and-expansion pipeline, we construct 3,810 enriched arguments spanning 762 pro/con stances on 100 controversial topics. Each opinion is expanded into multiple naturalistic variants, enabling robust evaluation of pluralism. We initialise three tasks with PERSPECTRA: opinion counting (identifying distinct viewpoints), opinion matching (aligning supporting stances and discourse to source opinions), and polarity check (inferring aggregate stance in mixed discourse). Experiments with state-of-the-art open-source and proprietary LLMs, highlight systematic failures, such as overestimating the number of viewpoints and misclassifying concessive structures, underscoring the difficulty of pluralism-aware understanding and reasoning. By combining diversity with structure, PERSPECTRA establishes the first scalable, configurable benchmark for evaluating how well models represent, distinguish, and reason over multiple perspectives. We release PERSPECTRA as a resource with flexible configurations, enabling the creation of tasks beyond the demo tasks presented in this paper, and fostering progress toward pluralism-sensitive systems that more faithfully capture human heterogeneity.

Published as a conference paper at ICLR 2026
PERSPECTRA: A SCALABLE AND CONFIGURABLE PLURALIST BENCHMARK OF PERSPECTIVES FROM ARGUMENTS
Shangrui No Kian Omoomi Lucie Flek Zhixue Zhao?, Charles Welch
Bonn-Aachen International Center for Information Technology, University of Bonn, Germany
Lamarr Institute for Machine Learning and Artificial Intelligence, Germany
Department of Computer Science, University of Sheffield, United Kingdom
McMaster University, Canada
mrie@uni-bonne de, mooomik@mmmaseee ca
ABSTRACT
Pluralism, the capacity to engage with diverse perspectives without collapsing them into single viewpoint, is critical for developing large language models that faithfully reflect human heterogeneity. Yet this characteristic has not been care- fully examined within the LLM research community and remains absent from most alignment studies, Debate-oriented sources provide a natural entry point for pluralism research. Previous work builds on online debate sources but remains constrained by costly human validation. Other debate-rich platforms such as Red- dit and Kiald also offer promising material: Reddit provides linguistic diversity and scale but lacks clear argumentative structure, while Kialo supplies explicit pro/con graphs but remains overly concise and detached from natural discourse. We introduce PERSPECTRA, pluralist benchmark that integrates the structural clarity Kialo debate graphs with the linguistic diversity of real Reddit discus- sions. Using controlled
pipeline, construct 3,810 en- riched arguments spanning 762 pro/con stances on 100 controversial topics. Each opinion is expanded into multiple naturalistic variants, enabling robust evalua- tion of pluralism. We initialise three tasks with PERSPECTRA: opinion counting (identifying distinct viewpoints). opinion matching (aligning supporting stances and discourse to source opinions), and polarity check (inferring aggregate stance in mixed discourse). Experiments with state-of-the-an open-source and propri- etary LLMs, highlight systematic failures, such as overestimating the number of viewpoints and misclassifying concessive structures, underscoring the difficulty of pluralism-aware understanding and reasoning. By combining diversity with structure, PERSPECTRA establishes the first scalable, configurable benchmark for evaluating how well models represent, distinguish, and reason multiple perspectives. We release PERSPECTRA as a resource with flexible configurations, enabling the creation of tasks beyond the demo tasks presented in this paper, and fostering progress toward pluralism-sensitive systems that more faithfully capture human heterogeneity. The dataset is available on github page
INTRODUCTION
Large language models (LLMs) are increasingly deployed settings that serve heterogeneous user communities, from education and healthcare to content moderation and public deliberation. Because these users hold divergent values and perspectives, is essential that LLMs outputs are not homog- enized into single "average" answer but instead reflect the plurality of legitimate viewpoints. Yet
https://www.ki http://wwwkkwllcooooo com/
https://githüb: com/ sa- 1ab/ CCIR- Pespectre

---

Published as a conference paper at ICLR 2026
current models often converge narrow, homogenized answers, which risks obscuring the diver- sity of perspectives present in real-world discourse. Ensuring inclusive and trustworthy deployment of LLMs therefore requires benchmarks that explicitly evaluate models' ability to represent and distinguish multiple perspectives.
Progress toward this goal has been supported by emerging pluralism-oriented datasets. For exam- ple, OpinionQA Santurkar et al. (2023) and GlobalOpinionQA Durmus et al. (2024) align model predictions with survey responses
whether models capture population-level distributions of
opinion. The DICES dataset Aroyo et al. (2023) collects culturally diverse judgments of conver- sational safety. While such resources highlight the feasibility of pluralism-awe
evaluation, they depend heavily on human annotation or curated surveys. As a result, they are costly to expand, narrow topical scope, and difficult to adapt to new pluralistic tasks.
An alternative line of research looks to naturally occurring debates. The PERSPECTRUM dataset (Chen al.) 2019) demonstrates the value of online debate sources for capturing diverse perspec- tives, but its scalability is constrained by intensive human validation. Other debate-rich platforms offer complementary benefits: Reddit provides scale and linguistic variety, reflecting the informality of real discourse (Pougué- Biyong et al., (02TT) while Kialo supplies explicit pro/con graphs that enable resources such as KialoPrime (Sahitaj et al., 2024). Yet Reddit lacks argumentative structure and is costly to annotate, and Kialo's concise, formalized claims fail to mirror the stylistic richness of authentic discussions.
We address this gap by introducing the PERSPECTRA, dataset that integrates the clarity of Kialo's debate structure with the diversity of Reddit discourse. Our construction pipeline retrieves semanti- cally related Reddit comments for each Kialo opinion and uses controlled prompting with ChatGPT- 4o to generate expanded arguments that remain faithful to the original stance while adopting the linguistic richness of authentic online discussions. This process yields a scalable resource of 3,810 expanded arguments across 762 pro/con opinions on 100 controversial topics.
The dataset naturally supports multiple pluralism-relevant evaluation tasks. We formalize three benchmarks: opinion counting (estimating the number of distinct opinions in a paragraph), opin- ion matching (aligning expanded arguments to their original claims), and polarity check (inferring aggregate stance from mixed arguments).
Games have always been more about playability and commercial success than being classi- fied as art. It's crucial to acknowledge both the artistic merit and the potential health hazards these games can present. Whether it's the sweeping landscapes of open-world RPG the detailed character designs in narrative-driven game, these elements come together to form a cohesive piece of art that deserves recognition. Just because you love a game doesn't 't magically turn it into masterpiece of art; it's still a commercial product at its core. That's what art is all about-bbinging ideas to life in way that moves and inspires people. Maybe it's just a matter of time before a game comes along that changes everything, but for now, their impact remains largely within the gaming community.
Table I: Example input to the opinion counting task. Sentences are color-coded by opinion cluster; expressions of the same opinion may appear in non-adjacent positions. The example illustrates the challenge posed to models: they must navigate interleaved opinions and group sentences that share the same underlying viewpoint despite lexical and stylistic variation.
2 RELATED WORK
2.1 PLURALISM AND PERSPECTIVISM IN CURRENT LLMS
LLMs often fail to reflect the diversity of viewpoints present in human discourse, instead converging on a normative "average." Alignment methods such as reinforcement learning from human feedback (RLHF) optimize for a single response style but in doing so suppress heterogeneity of opinions Sorensen al. 2024| Kirk et al., 2023). Empirical studies confirm this: RLHF narrows distribu- tional pluralism by concentrating probability mass on a few answers in survey-style benchmarks (Santurkar al. 2023 Durmus al.] 2024), reduces diversity across tasks (Kirk al. 2023)
2

---

Published as a conference paper at ICLR 2026
and leads to more homogeneous outputs in co-writing settings (Padmakumar & He 2024 Slocum
(et al.] 2025). Further analyses show that alignment can bias models toward majority or culturally
dominant perspectives, effectively silencing minority or contrarian views (Chakraborty et al. 2024
Shahid et al., 2025] Sourati et al. 2025). These limitations have motivated interest in pluralistic alignment: [Sorensen et al. (2024) propose frameworks such as Overton, steerable, and distributional pluralism to preserve viewpoint diversity. In summary, alignment improves safety and coherence but risks undermining the pluralism and perspectivism essential for inclusive AI. This gap underscores the need for benchmarks and datasets that test whether models can represent multiple perspectives.
2.2 DATASETS AND BENCHMARKS FOR PLURALISTIC MODELING
Pluralism-aware NLP has been advanced through new datasets and evaluation frameworks that cap- ture diverse perspectives and value judgments. Building on Sorensen et al.'s taxonomy of pluralism (Sorensen et al. 2024), subsequent resources demonstrate both the promise and the limitations of current approaches. OpinionQA (Santurkar et al. 2023) aligns model outputs with U.S. opinion surveys, while GlobalOpinionQA (Durmus et al.) 2024) extends this to cross-cultural settings. To capture divergent safety norms, Aroyo et al. (2023) introduce DICES, and CoSApien (Zhang et al.) focuses on safety pluralism through expert-defined configurations. Beyond human annotation, syn- thetic methods leverage L.LMs to generate diverse perspectives. [Hayati et al. (2024) show that prompting can elicit wide range of moral and social viewpoints, offering scalable alternatives to multi-annotator pipelines (Röttger et al. 2022). Other frameworks include Modular Pluralism (Feng et al. 2024). which composes conammntty-speifffi LLMs with base model to represent perspec- tives without retraining. Relatedly, Community Aligmment (Zhang et al., (025) and MoralBench (Chiu et al., 2025) target end-state and process-level preference pluralism but primarily used as training corpora rather than structured multi-viewpoint evaluation benchmarks. Recent surveys consolidate these directions (Xie et al., (2025), but emphasize open challenges: pluralistic datasets main narrow in topical scope, curating "valid" perspectives is difficult, and evaluation must balance viewpoint coverage with coherence.
2.3 ARGUMENT MINING AND DEBATE CORPORA
Debate platforms provide natural ground for studying pluralism, but existing corpora present trade- offs. Kialo's ' structured pro/con trees support resources such as KialoPrime (Sahitaj et al. (024) and BERDS (Chen & Choi 2025). which enable fine-grained relation classification and perspective di- versity evaluation, while PERSPECTRUM (Chen et al. (019)) demonstrates the value of curated per- spectives for claims. Beyond these, DebateSum (Roush & Balaji 2020) and OpenneebatEvvieeee (Roush et al. (2024) offer large-scale corpora derived from competitive formal debates. Their hier- archical clain-evidence structure and stance alignment support summarization and retrieval tasks at scale, yet remain focused on binary pro/con framing and formal styles. Similarly, IBM's Project Debater (Slonim et al.) 202T) introduced tools for claim detection, evidence-based summarization, and automated speech generation, but aims to synthesize persuasive monologues rather than preserve multiple viewpoints. Yet these datasets remain limited in topical coverage and often collapse debates into binary pro/con stances; moreover, resources like PERSPECTRUM (Chen et al. 2019) rely on costly human annotation, which hinders scalability. Reddit, by contrast, offers large-scale, natural- istic discourse in Change View (CMV) corpus (Gurjar et al. 2025) and DEBAGREEMENT (Pougué-Biyon et al., 2021), but annotation is also costly and labels often noisy due to ambigu- ity and sarcasm. Other resources such as the Internet Argument Corpus 2.0 (Abbott al. 2016) similarly compile large-scale online debates from political forums with annotations but exhibit con- siderable lexical noise and lack explicit argument scaffolding. In summary, existing Sebate-derived datasets either provide clean structure at small scale or noisy diversity at large scale, highlighting the need for more scalable resources that combine both. Our work addresses this gap by integrating the structural clarity of Kialo with the linguistic diversity of Reddit into a scalable benchmark.
3 DATASET CONSTRUCTION
We construct dataset that integrates the structural clarity of Kialo debates with the linguistic diver- sity of Reddit discussions. As illustrated in Figure [ the pipeline begins with topic-opinit pairs from Kialo, retrieves semantically related Reddit comments, and then expands each (topic, opinion,
3

---

Published as a conference paper at ICLR 2026
Opinion Comment Retrieval
Expanded Argument Generation
Expanded Argument Dataset
GBG Kialo
Structured (Toplc opinsons)
Embbed Kialo Redile
opinion,
Reddit
Seructured upinions unriched with Reddi-bbased phrnstings
Uner-gemmentedd
similatity, select top-5 Rediit Matches
Prompting Expanded argument
Contrullbed
discourse
Figure Overview of the dataset construction pipeline. Kialo debates (topics and opinions) are paired with Reddit comments via retrieval, then expanded through controlled prompting to produce naturalistic argument variants. The resulting dataset contains structured opinions enriched with Reddit-based phrasings.
comment) triple into naturalistic argumentative statement via controlled prompting. The resulting collection consists of structured Kialo opinions enriched with Reddit-based phrasings. In the follow- ing subsections, describe the data sources, retrieval procedure, generation method, and dataset statistics.
3.1
DATA SOURCES
Our dataset construction relies two complementary sources. Kialo provides structured debate content, organized into topics and their associated pro/con opinions. Each topic corresponds to a central question claim, while opinions represent user-submitted arguments supporting opposing that topic. This hierarchical structure offers a clean and well-defined set of argumentative units that anchors for data generation. An example from Kialo is follows:
Topic: All people in the US should have the right to basic healthcare Pros: Healthcare for all people would save many lives in the US. Cons: A doctor shortage could occur if everyone has healthcare.
In contrast, Reddit supplies large-scale eser-generated comments from diverse online discussions. Reddit threads are not organized into explicit debate trees but contain rich, informal argumentative expressions. These comments are used to provide contextual variation and linguistic diversity to the structured Kialo opinions.
By combining these two sources, we obtain both reliable backbone of clearly formulated opinions and wide pool of naturally occurring discourse from which additional context can be drawn.
3.2 OPINION-COMMEN RETRIEVAL
To ground structured Kialo opinions in naturally occurring discourse, we constructed for each topic pool of candidate Reddit comments. Relevant threads were retrieved using search-based ranking to prioritize semantically related discussions. Importantly, there was no fixed set of subreddits, and threads could originate from any subreddit. To maintain topical diversity, we enforced fixed retrieval budget per thread rather than relying heavily on single source. For each topic, retrieved 20 most relevant threads. From each thread, we collected most 100 comments, ranked by score (number of upvotes minus downvotes). If thread contained fewer than 100 comments, all available comments were included. This process ensured that, in the maximum case, each topic contributed up to 2,000 comments (20 threads 100 comments). This processed yielded initial pool of raw candidates.
A multi-stage filtering pipeline then applied. Comments containing fewer than five words were discarded, as they typically lacked argumentative substance. Additional filters removed low-quality non-user content, including comments without identifiable author, comments formally distin-
4

---

Published as a conference paper at ICLR 2026
guished moderator posts, comments authored by "automoderator:" and comments with usernames containing "bot". Through this process, each topic yielded a candidate pool of roughly 1,500-1,800 comments.
For opinion-comment comment matching, both the original Kialo opinion and all candidate comments were embedded using the twenn-Emmeddiing-e model. We selected this model due to its strong cluster- ing performance on the Huggingface MTEB leaderboard which is particularly relevant given that our task is semantic clustering between concise structured opinions and linguistically diverse Reddit discourse. Relevance was computed via cosine similarity, and for each opinion we retained the top five comments as semantically distinct matches. The highest-scoring candidate was designated as the primary "best match," while the remaining top-k served as additional material for generating stylistically and contextually varied expansions.
3.3 EXPANDED ARGUMENT GENERATION
Each selected opinion-comee comment pair then serves input to the expansion stage. For every triple of (topic, opinion, comment), we prompt GPT-4o with a fixed template designed to preserve the core stance of the original Kialo opinion while adapting its style and elaboration using cues from the paired Reddit comment. The prompt explicitly instructs the model to (i) maintain argumentative fidelity, (ii) ignore irrelevant or off-topic content, and (iii) mimic the informal and contextually grounded style of Reddit discussions. After piloting several prompt variants, we adopted the version that most reliably balanced fidelity and naturalness (see Appendix D)
This process results in five expanded variants for every opinion, producing total of 3,810 expan- sions across 762 opinions. On average, each generated argument is about 100 words long, yielding a corpus of several hundred thousand words of enriched discourse. These multiple expansions cap- ture alternative phrasings and contextual enrichments of the same underlying claim, substantially increasing the linguistic diversity of the dataset.
3.4 HUMAN ANNOTATION
The manual check criteria were designed to cover the major aspects of data quality that are particu- larly relevant to our construction pipeline. In practice, we found useful to think of the evaluation in terms of three complementary layers.
Content fidelity and coherence The most fundamental requirement that the expanded argument should remain faithful to the original opinion. Fidelity serves as the primary indicator of dataset quality: the expansion drifts away from or even contradicts the source, the data loses its intended purpose. We therefore focus on whether the generation preserves the core stance and reasoning of the original opinion, and whether any added material stays on-topic rather than introducing unrelated or exaggerated claims.
Mechanism-specific reliability Our dataset construction relies on retrieving best match Reddit post based on embedding similarity. However, semantic similarity alone does not guarantee that the retrieved content is truly relevant the opinion or the topic. In our inspection, we indeed encountered cases where the best match was only loosely related, even off-topic. To account for this, we distinguish between two aspects: the intrinsic relevance the retrieved post itself, and the model's handling of that retrieval. A high-quality generation should selectively integrate arguments from a relevant match, or, the match is irrelevant, demonstrate the ability to largely ignore it.
Naturalness of Writing Beyond correctness and retrieval, we require the data to resemble au- thentic discourse. The naturalness of writing criterion captures this dimension. Our aim is that the expansions should plausibly read human-authored posts, consistent with the style of the orig- inal Reddit discussions. During inspection, we noted that certain outputs, although content-wise accurate, exhibited "machine-like like" repetition or phrasing patterns. Such artifacts could undermine downstream realism and weaken the utility of the dataset.
https::/hhugiinggface.
5

---

Published as a conference paper at ICLR 2026
Taken together, these three layers motivate five concrete evaluation criteria: fidelity the original opinion, relevance of added content, relevance of the best match post to the topic, use of the best match post, and naturalness of the writing, Detailed definitions and the full 0-5 scoring rubries are provided in Appendix
Topic: Marvel universe is better than DC universe.
Opinion: DC has better characters.
Best Match: DC's characters stand alone better, whereas Marvel does better team books Expanded: While Marvel might excel at crafting epic team dynamies and ensemble stories, DC's strength lies in its iconic, standalone characters. Heroes like Batman, Superman, and Wonder Woman have rich, layered narratives that allow them to shine individually, creating deep connection with fans. These characters have been developed with such depth and complexity that they can earry entire stories on their own, without needing a team to back them up. This ability stand alone and still captivate audiences is what sets DC's characters apart and makes them truly legendary in the comic book world.
Fidelity Original Opinion: 5
Relevance of Added Content:
Use of Best Match Post: 5
Naturalness of Writing: 4
Relevance of Best Match to Topic:
To make the evaluation procedure concrete, the above example shows one annotated case. The retrieved best match comment is highly consistent with the original opinion, and the expanded gument successfully integrates the cues from the comment (e.g., "standalone characters") while claborating the stance with richer details. As a result, the expansion received perfect scores on fi- delity, relevance, and match usage. However, annotators noted that the writing still retained minor traces of model-like phrasing, leading a slightly lower score (4/5) on naturalness of writing.
3.5 STATISTICS
Manual Evaluation of Data Quality Building on the manual check criteria defined above, we conducted human evaluation to obtain quantitative scores for data quality. A random sample of 100 opinion-expanded pairs was annotated on the five dimensions using 0-5 scale. The aggregated results are shown in Table 2 Overall, the expansions score highly on fidelity, added relevance, and naturalness, while the main source of variability lies in the topic relevance of the retrieved best match posts.
Table 2: Human evaluation results on 100 randomly sampled opinion-expanded pairs. Scores are on 0-5 scale.
Criterion
Mean
Std
Min
Max
Median
IQR (p25-p75)
Fidelity to Original Opinion
4.92
0.40
2.0
5.0
5.0 5.0
Relevance of Added Content
4.87
0.46
2.0
5.0
5.0
5.0 5.0
Use of Best Match Post
4.72
0.65
2.0
5.0
5,0
5.0 3.0-500 5,0
Naturalness of Writing
4.42
0.48
4.0
5.0
4.0
4.0-5,0
Relevance of Best Match Topic
3.31
1.37
0.0
5.0
3.0
3.0-5.0 5.0
Corpus Summary In addition to these human evaluation results, we also report corpus-level statistics that capture the structural properties of the dataset whole. The dataset comprises 100 topics and 762 opinions (373 pros, 389 cons), yielding a near stance balance of 49.0% vs. 51.0% (Table 3ab. Each opinion is expanded into five variants, totaling 3810 expansions. As shown in Table 3b per-topic candidate sets are compact on average (mean 7.62 opinions; std 3.13) but display moderate heterogeneity (range 4-18), with pros and cons similarly distributed per topic (means 3.73 and 3.89; std 1.69). Text lengths align with the dataset's design: original opinions are short (mean 14.09 words; range 3-90), while expansions are length-stable around 100 words (mean 97.82; std

---

Published as a conference paper at ICLR 2026
Table Corpus-level and per-topic statistical summaries.
(a) Corpus-level summary of PERSPECTRA
(b) Per-topic opinion counts and text length statistics
Metric
Value
Category
mean
std
min
# Topics
100
Per-topic opinion counts
# Opinions (total)
762
Pros topic
3.73
1.69
# Pros ## Cons
373 389
Cons per topic
3.89
1.69
Pro Con (%)
49 51
Total per topic
7.62
3.13
18
# Expanded (total)
3810 (fixed)
Expanded per opinion
Text length statistics (in words)
Opinion Expanded
14.09
9.02
90
97.82
17.51
49
311
17.51; range 49-311), enabling predictable input budgeting for downstream tasks while preserving natural variation across topics.
4 DOWNSTREAM TASKS APPLICATION
While PERSPECTRA itself consists only of expanded arguments and their original opinion, it nat- urally supports a range of evaluation tasks. These tasks are derived from the structural properties of the data, such as the mapping between opinions and their multiple expanded arguments, and the original stance of the original opinion. In this section, we formalize the primary downstream tasks that can be directly constructed from the PERSPECTRA.
Opinion Counting We first apply PERSPECTRA to formulate a task estimating how many distinct opinions are present within paragraph. To construct inputs, concatenate multiple panded versions, some of which may originate from the same underlying Kialo opinion. While the expansions differ in surface realization, they should be treated instances of the same opinion. As illustrated in Table [ semantically equivalent arguments may appear in varied linguistic forms and are not necessarily adjacent within the paragraph. The task thus evaluates whether model can abstract away from lexical variation and identify semantically consistent arguments that share common stance.
Formally, given paragraph æ consisting of m expanded arguments, the model predicts an integer ý corresponding to the number of unique underlying opinions. The ground truth y is determined by the distinct Kialo opinion identifiers associated with the expansions. We report three metries: (i) Accuracy, requiring exact match ÿ = y; (ii) Mean Absolute Error (MAE), measuring average deviation 1/ y/: and (iii) Normalized Inverse Error (NIE), smoother score defined NIE N DE (1+ k max(1,k;))
1
)-² where N is the number of evaluation examples, h; is the ground-truth
count of unique opinions in the i-th example, and k; is the model prediction. The denominator uses max(1, k,) to normalize error relative to the scale of the true count.
Opinion Matching The second task tests whether model can correctly associate expanded argument with its original source opinion. For each topic, we randomly select one expanded version and present it together with the full set of candidate opinions (both pro and con) defined under that topic. The model must identify which original opinion the expansion derives from.
Performance is measured with two complementary metrics. Accuracy requires exact identification of the source opinion, while Stance Accuracy considers only whether the chosen candidate belongs to the correct debate side (pro con).
Polarity Check The third task evaluates whether a model can infer the aggregate stance of a paragraph containing mixture of expansions. For each topic, randomly sample n expanded versions and concatenate them into a single paragraph. The model must assign a binary label (pro or con) indicating the dominant orientation of the paragraph. The gold label is determined by majority

---

Published as a conference paper at ICLR 2026
Table Results all three sub-tasks (TI-T3): Opinion Counting (Accuracy, MAE, NIE), Opinion Matching (Accuracy, Stance Accuracy), and Polarity Check (Accuracy) on models and human.
T1: Opinion Counting
T2: Opinion Matching
T3: Polarity Check
Models
Ace.
MAE L
NIE f
Acc.
Stance Acc.
Ace.
LLaMA-3.1-88 --8---tt-e-
6.4%
3.17
0.57
6.0%
10.4%
29.6%
Falcon3-7B-Insie
15.8%
2.28
0.67
66.2%
77.4%
51.0%
Qwen3-8B
33.6%
1.26
0.79
83.6%
92.4%
64.0%
Qwen3-32B
28.6%
1.62
0.74
71.6%
86.0%
67.6%
Qwen2.5-7B
7-- Instruct
30.8%
1.19
0.78
78.8%
91.2%
58.6%
Qwen2.5-32B
35.2%
1.25
0.79
72.2%
80.4%
55.2%
QwQ-32B
36.2%
0.97
0.82
72.0%
80.2%
66.4%
1S-RR-DDitilll-Llmm--B
23.6%
2.24
0.69
66.6%
84.8%
54.8%
D--RI-Distill-Wwen--
32.6%
1.21
0.79
66.0%
89.4%
54.8%
DS-RI-DDstilllQwen-BB3
31.2%
1.48
0.77
85,4%
95.0%
67.2%
GPT-4o
29.2%
1.38
0.76
74.8%
81.2%
76.4%
GPT-4o-mini
34.0%
0.94
0.81
71.6%
81.6%
72.8%
Human
44.0%
1.01
0.81
90.6%
94.3%
85.6%
vote over the stance labels of the included expansions. Performance is assessed using Accuracy, computed as the proportion of paragraphs for which the predicted polarity matches the ground truth.
Although pluralism-rehie related tasks inherently involve subjective perspectives, PERSPECTRA is de- signed to minimize such subjectivity in task formulation. All opinion units are inherited directly from Kialo's s moderated debate graphs, where topic-opinion boundaries are externally curated rather than specified by us. To accommodate any remaining ambiguity, our evaluation relies on graded, distance-sensitive metrics (e.g., MAE and NIE) instead of brittle exact matches, ensuring propor- tional rather than all-or- or-nothing penalties.
RESULTS AND FAILURE ANALYSIS
5.1 OVERALL BENCHMARK RESULTS
For each of the three sub-tasks, we instantiated an evaluation set of 500 examples by sampling from the PERSPECTRA corpus. Since task-specific ground truths can be derived from the opinions and stance labels. this procedure requires no additional human annotation. Moreover, the construction process is fully programmatic and therefore easily scalable.
Table Tabmerismmmrizes results across the three evaluation tasks. Performance varies substantially across model families, with no single system achieving consistently high scores across all sub-tasks. For opinion counting (TI), the best results come from open-source models such as QwQ-32B, which slightly outperforms GPT-4o on both accuracy and NIE For opinion matching (T2), distillation- 1--iistlll--we--3333
achieve the strongest accuracies, exceeding GPT-4 family. By contrast, polarity check (T3) remains most challenging for open-source mod- els, where GPT-4o leads with the highest accuracy. Overall, these findings confirm that pluralism- oriented tasks remain difficult: while some specialized models excel particular sub-tasks, substan- tial headroom remains for models to achieve robust pluralistic reasoning across all tasks simultane- ously. We additionally report human performance for all sub-tasks, which exceeds every evaluated model by a clear margin.
based Qwen models (e.g..
5.2 CHALLENGE 1: OPINION OVERESTIMATION
In the opinion counting task, we observe that models tend to overestimate the number distinct opinions. This pattern suggests that the primary challenge does not lie in detecting stance, but rather in correctly aggregating semantically similar expansions under the same original opin- ion. In other words, the task is less about classification and more about avoiding over-splitting. A representative failure case is shown Table 5 the Appendix F where a concatenated paragraph contains eight expanded versions. Despite the apparent diversity of expression, these expansions
8

---

Published as a conference paper at ICLR 2026
actually derive from only four distinct original opinions. For example, four of the segments all rephrase the idea that schools enforce conformity in ways that harm students' mental health, while two segments both emphasize the safeguarding role of schools in identifying early signs of distress. The remaining two expansions represent distinet yet separate perspectives.
When confronted with such input, models typically predict much higher count than the ground truth, possibly because they mistake surface-level lexical or stylistic differences for distinct opinions. To quantify these patterns, we further analyzed the subset of 47 inputs on which all 12 evaluated models failed, yielding a total of 564 incorrect predictions. Among these, 71.5% oversplit errors (predicting more opinions than the ground truth) and 24.6% undercount errors (predicting fewer). Only 3.9% consisted of invalid outputs. This distribution confirmss that oversplitting dominates as the primary failure mode, consistent with the qualitative analysis above. In practice, models often mistake surface-level lexical or stylistic differences for distinct opinions, underscoring that the real difficulty of the task lies in semantic normalization: different expanded versions must be clustered into their underlying original opinion, rather than treated as independent contributions.
5.3 CHALLENGE 2: SEMANTIC OVERLAP IN MATCHING
In the opinion matching task, errors rarely come from confusing opposite stances. As shown in Table 4 stance accuracies are consistently 8-24 points higher than exact-match accuracies across models (e.g., Qwen3-8B reaches 83.6% accuracy but 92.4% stance aceuracy; DS-RI-Distill-Qwen 7B achieves 66.0% vs. 89.4%). This gap indicates that models generally succeed in identifying the correct side of a debate, but struggle to capture fine-grained distinctions between seman- tically related opinions on the same side. Expanded versions often introduce additional details (e.g.. specific cases, examples, or contextual nuances) that are not explicitly present in the con- cise original opinions, creating ambiguity between multiple candidate matches. For instance, in the topic "Online video games are currently more enjoyable than board games." one expanded opinion mentioned the immersive experience of online games, including global connectivity, rich storylines, and convenience. While the ground truth was "Online video games contain additional features that make them a more enjoyable experience than board games," the expansion also overlapped with other pro-side options such as "The story of a video game is more involved" and "You don 't need to leave your room," " leading to frequent confusion. Similarly, in the topic "Trophy hunting should be illegal," an expanded opinion emphasized that trophy hunting generates funds for conservation and reinvests in local communities. The correct match was "Trophy hunting helps conservation efforts," but the mention of local benefits made it easily confused with "Trophy hunting benefits local com- munities." " These cases highlight that exact-match errors are primarily due to over-specificity and semantic overlap, while stance-level matching remains relatively robust.
5.4 CHALLENGE 3: CONCESSION TRAP IN POLARITY
major source of polarity errors the concession- structure: texts that begin by ac- knowledging one side and then pivot to argue the opposite. Local polarity cues in the concessive lead ("while/although... stance. In the federal cannabis legalization samples, several expansions noted potential downsides such as health risks or high taxation, but then concluded with strong pro-legalization arguments (e.g., dismantling the black market, reducing incarceration, generating tax revenue). While the over- all polarity was pro, models often misclassified such inputs as con because the initial concessions blurred the aggregate direction. To quantify this phenomenon, we examined the 34 inputs on which all models failed. Among them, 20 cases (59%) displayed clear concession-rebuttal structure, confirming that polarity eтrors are predominantly systematic, arising from models being misled by concessive openings rather than from random misclassification.
often mislead models toward the conceded side instead of the concluding
CONCLUSIONS
We introduced PERSPECTRA, a pluralist benchmark that integrates the structural clarity of Kialo debates with the linguistic richness of Reddit discourse. Through aetrieval-and-expanse pipeline, the dataset provides 3,810 enriched arguments across 100 topics, each opinion expressed in multiple naturalistic variants, making the resource both scalable and easily configurable for diverse evaluation

---

Published as a conference paper at ICLR 2026
settings. Using the dataset, derived three sub-tasks: opinion counting, opinion matching, and po- larity check. Across these tasks, we uncovered recurring failure modes. In particular, models exhibit systematic opinion overestimation driven by oversplitting in counting, robust stance recognition but fragile fine-grained matching due to semantic overlap and over-specificity. and polarity misclassi- fication triggered by concession-rehuttal structures that mislead local cues. We also evaluated range of models on the sub-tasks; the results make clear that current systems still have considerable reasoning. Looking forward, hope PERSPECTRA
room for improvement on pluralism-sensee will serve not only as benchmark but also as foundation for new methods that explicitly engage with pluralism and perspectivism in language modeling, paving the way towards systems that move beyond single best answers and instead capture the diversity of reasoning, values, and perspectives that characterize human discourse.
LIMITATIONS
As limitations, first, the quality of opinion-comment retrieval is uneven as reflected in human evalu- ation; Reddit contains substantial noise and loosely related content, which can leak into expansions and reduce fidelity. Second, the benchmark currently centers three sub-tasks (opinion counting, opinion matching. and polarity check), which capture important aspects but do not cover the broader space of pluralism. Third, manual validation was conducted on limited subset of the corpus, leav- ing the need for larger and more diverse human evaluation.
ACKNOWLEDGEMENTS
We gratefully acknowledge the support of the Lamarr Institute for Machine Learning and Artificial Intelligence. We also gratefully acknowledge the access to the Marvin cluster of the University of Bonn.
REPRODUCIBILITY STATEMENT
The construction of PERSPECTRA is fully reproducible. All raw data are drawn from publicly accessible sources. The retrieval-and-expasiie pipeline, including prompt templates, sampling pa- rameters, and filtering rules, is released together with code in the supplementary material, ensuring that the enriched arguments can be regenerated. The three sub tasks-opinion counting, opinion matching, and polarity check-are are derived deterministically from the dataset annotations, and the sampling procedure for evaluation instances is controlled with fixed random seeds. We provide scripts and configurations so that both dataset generation and task creation can be exactly repro- duced.
REFERENCES
Rob Abbott, Brian Ecker, Pranav Anand, and Marilyn Walker. Internet argument corpus 2.0: An SQL schema for dialogic social media and the corpora to go with it. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mar- iani, Helene Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Proceedings of the Tenth International Conference Language Resources and Evaluation (LREC' 16), pp. 4445- 4452, Portorož, Slovenia, May 2016. European Language Resources Association (ELRA). URL httpss://aclantoolt www.lloo...oolloo...... 1704/
Lora Aroyo, Alex Taylor, Mark Díaz, Christopher Homan, Alicia Parrish, Gregory Serapio-Garctt Vinodkumar Prabhakaran, and Ding Wang. Dices dataset: Diversity in conversational ai evalua- tion for safety. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 33330-53344 Associates, Inc., 2023. URL httpps:/pproceedinss..eeee 2ttp:/2aaaaaaa3--aaaar---- Paper Datasets
Curran
wtt.cc//pppp.e... files/
and Bww.hhernhhh...e.eee .pdf
Souradip Chakraborty, Jiahao Qiu, Hui Yuan, Alec Koppel, Dinesh Manocha, Furong Huang, Amrit Bedi, and Mengdi Wang. MaxMin-RLHER Alignment with diverse human preferences. In Ruslan
10

---

Published as a conference paper at ICLR 2026
Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.). Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 6116-6135 PMLR, 21-27 Jul 2024. Uttpttss::/// htttpp::/pprceedd .gs.mll........m.mmm.mmm........ ml.
html
Hung- Ting Chen and Eunsol Choi. Open-world evaluation for retrieving diverse perspectives. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Na- fions of the Americas Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies (Volume 1: Long Papers), pp. 8508-8528, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-189-6 doi: 10.18653/v1/ 2025.naacl naacl- long.l411.111.. 431. URL BtLhttts: hwtpaaaaaatttoooo.ooogooooo////e -ttp:////o.2.2.aaaaaaeaeeoeee. nww.naaanaaaaaa long 4317
Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch, and Dan Roth. Seeing things from a different angle:discovering diverse perspectives about claims. In Jill Burstein, Christy Do- ran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long and Short Papers), pp. 542-557, Minneapolis, Minnesota, June 2019, Association for Com- putational I Linguistics. doi: 10. 8853/v1/N19--05555 URL https: //ac:///ccc antho ogy ogg.org// org/ N19-1053/
Yu Ying Chiu, Michael $ Lee, Rachel Calcott, Brandon Handoko, Paul de Font-Reaulx, Paula Ro- driguez, Chen Bo Calvin Zhang, Ziwen Han, Udari Madhushani Schwag, Yash Maurya, et al. Morebench: Evaluating procedural and pluralistic moral reasoning in language models, more than outcomes. arXiv preprint wrXiv:2510.16380 2025.
Esin Durmus, Karina Nguyen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCan- dlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. Towards measuring the representation of subjective global opinions in language models, 2024. URL pottps://azzee
org/
16388
Shangbin Feng. Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, and Yulia Tsvetkov. Modular pluralism: Pluralistic alignment via multi-LLM collaboration. In Yaser Al- Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Em- pirical Methody in Natural Language Processing, pp. 4151-41711 Miami, Florida, USA, Novem- ber 2024. Association for Computational Linguistics. doi: 12.18653/1120224.com m4in.
URL https: httpps///ac..aaooooooooooo////// www.ppp0...mmmmmmmmmmmmmmmmmmmmm -manmlippmmmm.mmmmmmmmm 240/
Omkar Gurjar. Agam Goyal, and Eshwar Chandrasekharan, Argemv: An argument summarization benchmark for the llm-era, 2025. URL htttps://aaxiiii.. org/ abs 021.//2508..1188
Shirley Anugrah Hayati, Minhwa Lee, Dheeraj Rajagopal, and Dongyeop Kang. How far can we extract diverse perspectives from large language models? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Meth- ods Natural Language Processing, pp. 5336-5366, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 1/025///1/2222.mlppmmmo https://eeLamtiooooog..oo/////// emmlp ma maiin...
URL
3067
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects rlhf on Ilm generalisation and diversity. arXiv preprint arXiv:2310.064555 2023.
Vishakh Padmakumar and He He. Does writing with language models reduce content diversity? In The Twelfth International Conference on Learning Representations, 2024. URL https: 7 //opereeview. www.:h/ooooomoeeeeeiieeo.
John Pougué-Biyong. Valentina Semenova, Alexandre Matton, Rachel Han, Aerin Kim, Renaud Lambiotte, and Doyne Farmer. DEBAGREEMENT: A comment-reply dataset for (dis)agreement detection in online debates. In Thirty-fitth Conference Neural Information Processing Sys- tems Datasets and Benchmarks Track (Round 2), 2021. URL htttps://peerreveey forum? d==dddu gFO.

---

Published as a conference paper at ICLR 2026
Paul Röttger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. Two contrasting data annota- tion paradigms for subjective NLP tasks. In Marine Carpuat, and Ivan Vladimir Meza Ruiz (eds.). Proceedings of the 2022 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, pp. 175-190, Seattle, United States, July 2022. Association for Computational Linggss- URL https:: http::///eolaaaaaooooooo////////
de Marneffe,
tics. doi: 10. 18.18653/v 10.1665////0022.aaacll
naacl ma maii..n.... 137
Allen Roush and Arvind Balaji. Debatesum: A large-scale argument mining and summarization dataset. urXiv preprint arXiv:201 1.07251, 2020.
Allen Roush, Yusuf Shabazz, Arvind Balaji, Peter Zhang, Stefano Mezza, Markus Zhang, Sanjay Basu, Sriram Vishwanath, and Ravid Shwartz-Ziv. Opendebatevvidence: A massive-scale argu- ment mining and summarization dataset. Advances in Neural Information Processing Systems, 37:34270-34293, 2024.
Premtim Sahitaj. Ramon Ruiz-Dolz, Ariana Sahitaj, Ata Nizamoglu, Vera Schmitt, Salar Mohtaj, and Sebastian Möller. From construction to application: Advancing argument mining with the large-scale kialoprime dataset. In Chris Reed, (Matthias Thimm, and (Tjitze } Rienstra (eds.). Computational Models of Argument, volume 338 of Frontiers in Artificial Intelligence and Applications, pp. 229-240, Netherlands, 2024. IOS Press. ISBN 9781643685342. doi: N0.3233/FAAA2400224 URL httthtts: cttp://ccmaa222/22mm krportal. org/ ② 2024 The Au- thors; The 10th International Conference on Computational Models of Argument, COMMA 2024 -Confernnce date: 18-09-2024 Through 20-09-2024.
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning .mlr.
Research, pp. 29971-300000 PMLR, 23-29 Jul 2023. URL https::/proceeeeee html
Farhana Shahid, Stella Zhang, and Aditya Vashistha. Llms homogenize values in constructive argu- ments on value-laden topics, 2025. URL lhttps::/ wwtp:///g///aa/aaoo/o/ooo /209, 10637
Stewart Slocum, Asher Parker-Sartori, and Dylan Hadfield-Menell Diverse preference learning for capabilities and alignment. In The Thirteenth International Conference on Learning Representa- fions, 2025. URL
Noam Slonim. Yonatan Bilu, Carlos Alzate, Roy Bar-Haim, Ben Bogin, Francesca Bonin, Leshem Choshen, Edo Cohen-Karlik, Lena Dankin, Lilach Edelstein, et al. An autonomous debating system. Nature, 591((850):379-338 2021.
Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christo- pher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. Position: a roadmap to pluralistic alignment. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024.
Zhivar Sourati, Alireza S. Ziabari, and Morteza Dehghani. The homogenizing effect of large Ian- guage models on human expression and thought, 2025. URL https::/axxxv..rgg///
2508. 01491
Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Ku- mar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, and Julian McAuley. A survey on personalized and pluralistic preference alignment in large language models, 2025.
URL /ttts:://aatttt
org/ abs/ 2504 07070
Jingyu Zhang, Ahmed Elgohary, Ahmed Magooda, Daniel Khashabi, and Benjamin Van Durme. Controllable safety alignment: Adapting llms to diverse safety requirements without re-training. In Pluralistic Aligument Workshop at NeurlPS 2024.
Lily Hong Zhang, Smitha Milli, Karen Jusko, Jonathan Smith, Brandon Amos, Manon Revel, Jack Kussman, Lisa Titus, Bhaktipriya Radharapu, Jane Yu, et al. Cultivating pluralism in algorithmic monoculture: The community alignment dataset. arXiv preprint arXiv:2507.00m 006500, 2025.
12

---

Published as a conference paper at ICLR 2026
A ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. In this study, no human subjects or animal exper- imentation involved. All datasets used, including PERSPECTRA, were sourced in compliance with relevant usage guidelines, ensuring biases discriminatory outcomes in our research process. No personally identifiable information was used, and no experiments were conducted that could raise privacy or security concerns. We are committed to maintaining transparency and integrity throughout the research process.
violation of privacy. We have taken care to avoid any
B
LLM USAGE
Large Language Models (LLMs) were used to aid the writing and polishing of the manuscript. Specifically, we used an LLM to assist in refining the language, improving readability, and ensuring clarity in various sections of the paper. The model helped with tasks such as sentence rephras- ing, grammar checking, and enhancing the overall flow of the text. It is important to note that the LLM was not involved in the ideation, research methodology, or experimental design. All research concepts, ideas, and analyses were developed and conducted by the authors. The contributions of the LLM were solely focused on improving the linguistic quality of the paper. with no involve- ment in the scientific content or data analysis. The authors take full responsibility for the content of the manuscript, including any text go erated or polished by the LLM. We have ensured that the L.M--genrrted text adheres to ethical guidelines and does not contribute to plagiarism or scientific misconduct.
C APPENDIX: ANNOTATION CRITERIA FOR EXPANDED ARGUMENT EVALUATION
Each expanded version was evaluated along the following five dimensions, with scores ranging from 0 to 5.
1. Fidelity to Original Opinion Does the expanded version maintain the core argument of the original opinion?
Completely contradicts misrepresents the original argument.
1 Severely distorts the original argument with only minimal connection.
. 2 Partially maintains the core argument but with significant alterations.
* 3 Mostly maintains the core argument with some minor deviations.
4 Maintains the core argument well with very minor deviations.
5 - Perfectly maintains the core argument with appropriate elaboration.
Relevance of Added Content Does the expanded version contain content that's unrelated to the original opinion?
Completely filled with unrelated content.
1 Mostly unrelated content with minimal relevant parts.
2 Contains significant unrelated content (roughly a half).
3 Mostly relevant with some unrelated tangents.
4 Very relevant with only minor unrelated elements.
5 All added content is directly relevant the original opinion.
3. Use of Best Match Post match Reddit post?
Does the model appropriately incorporate information from the best
Completely ignored relevant best match OR fully incorporated irrelevant match
13

---

Published as a conference paper at ICLR 2026
1 Made poor decisions about using the match (used very little when relevant OR used too much when irrelevant).
2 Made questionable decisions about match usage.
3 Mostly good decisions with some issues.
4 Intelligently incorporated relevant match content OR appropriately avoided irrelevant content, with minor issues.
5- -errfect discermment; fully utilized relevant information completely avoided irrelevant match.
Naturalness Writing
Does the expanded version resemble human-written text?
Clearly machine-generated with obvious patterns, repetitions, or unnatural phrasing.
Mostly machine--gnnratte with many unnatural elements.
2 Mixed quality with significant machine-like patterns.
3 Generally reads like human writing with some artificial elements.
4 Very natural, human-like writing with only occasional subtle artifacts.
Indistinguishable from authentic human writing.
Relevance of Best Match Post to the Topic How well does the best match support the original
opinion relate to the topic?
0 Completely unrelated; no meaningful connection.
1 Minimally related; shares only vague keywords surface-level concepts.
2 Weakly related; some abstract tangential connection, but direct alignment.
Moderately related; touches on related issues or framing with partial support.
4 Strongly related; clearly relevant or provides useful context.
5 Directly supports; reinforces the original opinion or meaningfully engages with the
topic.
D PROMPT USED FOR GENERATION Task Prompt Used for Expansion Generation
Your task is expand the opinion given topics and posts comments from Reddit that sup- port this opinion. It doesn't 't need to be too long, just 3 to 4 sentences expansion. Ignore the post/comment if it's irrelevant, and try mimic the wording style of the post/comment. Rewrite and expand the opinion in natural, Reddit-style way. The topic is: (topic). The opinion is: fopinion:
The post or comment is: (comment).
Please directly output the expanded opinion, don't 't include any other text.
PROMPT USED FOR OPINION COUNTING
Task Prompt Used for Opinion Counting
"Read the paragraph about {topic} and identify how many distinct opinions it contains. An opinion a unique core stance; different wording with the same stance counts as one opinion. ONLY reply with integer."
14

---

Published as a conference paper at ICLR 2026
Task Prompt Used for Opinion Matching
"Read the following expanded paragraph about {topic} and decide which one of the listed original opinions it best matches. Reply ONLY with the index of the correct option (0-based)."
Task Prompt Used for Polarity Check
"Read the following concatenated opinions about {topic} and determine whether the majority stance is pro or con. Reply ONLY with 'pro" or "con'."
F ERROR CASE FOR OPINION COUNTING
Table An error case from the opinion counting task.
School is not enjoyable for many students because the pressure has ramped up significantly compared to previous generations. It's like people don't I realize that what worked for them decades ago isn't 't necessarily going to work now. The workload is heavier, the competition is fiercer, and the expectations are sky-high. It's no wonder students are feeling overwhelmed and stressed out, yet some just brush it off as if it's the same experience they had ages ago.
Schools often prioritize conformity, pushing students to fit into standardized mold that doesn't 't account for individual differences. This expectation can be particularly damaging in cultures with conservative atti- tudes toward mental health, where deviation from the norm is stigmatized. When students feel pressure to conform, they might suppress their unique identities and struggles, leading to increased stress and anxiety. The lack of support for mental health issues in such environments only exacerbates the problem, leaving students feeling isolated and misunderstood.
hear you. It's not always the school itself that's 'ss the root cause of poor mental health; often, it's the envi- ronment and experiences surrounding it. Take bullying, for 2istance2014it's huge factor that can make school a nightmare for kids. pushing them into online schooling just to escape the constant stress. It's not about the curriculum or the teachers, but more about how schools can sometimes fail to protect students from toxic interactions.
It's really unfortunate that sometimes things have hit rock bottom before child gets the help they need, but schools can be crucial in spotting those red flags early on. It's like, no matter where you are, mental health issues are everywhere. and it's frustrating because feels like no one really knows how to handle it. But schools can step in and make a difference by identifying problems before they spiral out of control. They have the potential to be place where kids can find support and resources, preventing those safe- guarding issues from becoming full-blown crisis.
Schools often play crucial role both identifying and addressing child safeguarding concerns, which can have a significant impact on a child's mental health. In many cases, schools are in a unique position to notice when something is off, like if child has witnessed domestic violence or is showing signs of dis- tress. Teachers and counselors can act first responders, recognizing abnormal hehavior and guiding the child towards getting the help they need. Without this support system, issues might go unnoticed, leading long- mental health problems that could have been mitigated with early intervention.
It's wild to think about how schools push everyone to fit into this seee-size-fit--ale mold, and if you don't, you'te re made to feel like an outcast. It's like you's 're constantly walking on eggshells, afraid to show who you really are because there's this looming fear of not being accepted. This pressure to conform can be incredibly damaging, especially when you're 'e just trying to figure out your identity. It's no wonder so many students feel stressed and anxious when they're re stuck in an environment that docsn't 't celebrate individual- ity.
Totally agree. Schools are like these factories where they expect every kid to fit into the same mold, and it really messes with their mental health. My daughter is already dealing with insecurities, and the pressure conform just makes it worse. She's been bullied so much that we had to switch her to online school just to give her some peace. It's like the system doesn't It accommodate individuality, and that can be really damaging for kids who are already vulnerable.
Absolutely, schools often push for conformity, and that can seriously mess with your mental health. I re- member how my mom was super concerned that getting labeled would lead teachers to box me into this stereotype of being less smart or capahle, even though she always saw my potential. It's like, you don't fit into their neat little boxes, you'te re automatically seen as different, and that can make you feel out of place or even ashamed. Schools need to realize that not everyone fits the mold, and forcing us to do so can really take toll on how we see ourselves and our abilities.
15


Reviews:
[{"review_id": "xmBMmJ41XD", "reviewer": "Reviewer_A1ew", "summary": "- Defines pluralism as the ability to represent multiple valid perspectives.\n - Argues that debate-oriented data (Reddit, Kialo) naturally capture pluralism but each has limitations: Reddit is linguistically diverse yet unstructured, while Kialo is structured but too concise.\n - Introduces Perspectra, a benchmark that merges Reddit’s linguistic diversity with Kialo’s argument-graph structure.\n - Establishes three evaluation tasks:\n\t- Opinion counting — detecting distinct viewpoints.\n\t- Opinion matching — aligning discourse to original stances.\n\t- Polarity check — inferring the overall stance in mixed text.", "questions": "I'd like to hear your perspective on other benchmarks that have been release on pluralistic alignment. \n\n - [1] CoSApien https://huggingface.co/datasets/microsoft/CoSApien https://arxiv.org/abs/2410.08968 \n - [2] MOREBENCH https://huggingface.co/datasets/morebench/morebench https://www.arxiv.org/abs/2510.16380 \n - [3] https://huggingface.co/datasets/facebook/community-alignment-dataset  https://arxiv.org/abs/2507.09650", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "LLMs are increasingly used in sensitive contexts and naturally plurality of opinions is inherent to the problem. We need a lot of work to define new goal posts in this direction, like this work.", "weaknesses": "The \"Opinion Counting\" (estimating how many distinct opinions are present within a paragraph) is a somewhat ambiguous. Like you the definitions of what constitutes as a distinct opinions can be subjective and can vary from one person to another. The same also holds for the other tasks (matching and polarity). \n\nIn other words, why should I (or any reader) trust that: \n - you have high-quality annotations? \n - and that the task is well-defined? \n\nI do see that you conduct human annotations to ensure data quality. Have you tried computing the human performance for each of your tasks? (counting, matching, polarity). Having these numbers can help inform the reader about data quality vs ambiguity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W0P01KuYZo", "reviewer": "Reviewer_uHvT", "summary": "The authors propose PerSpectra, a large-scale, curated dataset combining Kialo's debate structure, which contains pro and con opinions on 100 different topics, and comments from Reddit. This dataset is extended with statements generated by large language models (LLMs). They also introduce three tasks that can be solved using this dataset: opinion counting, opinion matching, and polarity checking.", "questions": "No questions left.\nSmall comments:\n- 042: The abbreviation for LLMs is introduced in the Related work section, but should already be introduced in the Introduction.\n- 059, 062-063: Some of the citations in the introduction that are written out in the text should be put in parentheses using the command \"\\citep.\"\n- 119: maybe you meant LLMs instead of LMs?\n- 244: Small typo in \"Writting\" (instead of Writing)", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well-written and easy to follow. The data curation process and the proposed tasks and metrics are clearly described and useful for comparing the capability of large language models to recognize plurality of opinions.", "weaknesses": "The authors claim that this dataset differs from others that focus on pluralistic opinions on various topics because the process did not require extensive human annotation. However, annotating a sub-sample revealed the possibility of selecting statements that do not fit the topic, which compromises the data's quality. Human annotation is therefore indispensable. Nevertheless, this dataset is a good starting point for further research and offers an interesting approach to obtaining debate data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W6UA7EHMt7", "reviewer": "Reviewer_nZ2F", "summary": "The authors propose PERSPECTRA, a benchmark for evaluating the \"pluralism\" of LLMs, which they define as their ability to handle multiple distinct viewpoints without collapsing them. The core contribution is a synthetic data pipeline: they take structured arguments from Kialo, retrieve semantically \"similar\" comments from Reddit, and use GPT-4o to expand the arguments into naturalistic text that mimics the Reddit style. From this dataset of 3,810 arguments across 100 topics, they define three tasks: counting unique opinions, matching a generated argument back to its source, and checking the aggregate pro/con polarity. They benchmark several models, highlighting common and interesting failure modes like overestimating opinion counts and being tricked by concessive phrasing.", "questions": "Given the low relevance score for the retrieved Reddit comments (3.31/5), how can you be sure the final dataset genuinely captures Reddit's linguistic diversity, rather than GPT-4o's simulation of it? Could you provide an analysis of how much the generator actually uses the retrieved comment, especially in cases where its relevance is low?\n\nHave you considered the potential for evaluation contamination, where models are benchmarked on data generated by a close relative (or even the same underlying architecture)? Does this risk rewarding models that simply mimic the generator's style over models that perform genuine semantic reasoning?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The core idea of combining the structural clarity of Kialo with the stylistic diversity of Reddit is a neat trick. It's a pragmatic approach to generating nuanced argumentative data, which is a known bottleneck. I wasn't even aware of Kialo before this paper. \n\nThe real meat of the paper is in Section 5. The identification of challenges like \"opinion overestimation\" and the \"concession trap\" is good.\n\nThe idea of creating a benchmark for pluralism is good, and the generation method is a clever (if flawed) approach to scale beyond human annotation. The benchmark and the failure analysis are a welcome addition, even if the dataset is not massive.\n\nThe paper is clearly written. The pipeline is explained well, and the failure analysis section is particularly insightful.", "weaknesses": "The paper champions a \"scalable\" pipeline but delivers a dataset with only 100 topics and 3810 rows. This feels more like a proof-of-concept than a large-scale resource that lives up to the \"scalable\" moniker. I'd call OpenDebateEvidence and it's nearly 4 million rows \"scalable\", but not this. \n\nThe related work section is incomplete. It fails to cite foundational work in this niche, particularly datasets like DebateSum and OpenDebateEvidence (Roush et al.). These works tackle the problem of structuring claims from formal debate rounds and are highly relevant to this papers related work sections. I would also expect similar citations for IBM Project Debater and the things they cite. \n\nThe entire benchmark is built on outputs from a single proprietary model (GPT-4o). Are we testing a model's understanding of pluralism, or its ability to parse the stylistic tics of another LLM? This creates a \"synthetic-on-synthetic\" evaluation loop that risks optimizing for model-to-model mimicry rather than genuine reasoning. It's the hall of mirrors problem, and the paper doesn't sufficiently address it.\n\nThe authors' own human evaluation reveals the retriever is mediocre. If the retrieved Reddit comment is irrelevant, then the claim of \"integrating the linguistic diversity of real Reddit discussions\" falls flat. The generation becomes simple paraphrasing-with-a-prompt rather than a true synthesis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G1YYCrIdbn", "reviewer": "Reviewer_eQpS", "summary": "This paper introduces a dataset (with associated benchmark tasks) for LLM alignment geared towards “pluralism,” i.e. the ability to engage with and distinguish between different perspectives on an issue. This is motivated by the idea that LLM outputs often reflect homogenized, culturally dominant opinions (thereby suppressing minority or contrarian views) and there is a relative lack of research on addressing this issue.\nTheir new dataset, called PERSPECTRA, draws from two online sources: Reddit, which has an abundance of unstructured debate text, and Kialo, which has very structured pro-con debate text. The dataset is constructed using NLP techniques: for each Kialo opinion, they automatically retrieve semantically similar Reddit comments and then, using controlled prompting with ChatGPT, generate expanded versions of the arguments therein.\nThe retrieval step is done in cosine similarity on the Qwen3-Embedding-8B model’s text embeddings. The expanded arguments are constructed by prompting GPT-4o “to preserve the core stance of the original Kialo opinion while adapting its style and elaboration using cues from the paired Reddit comment.” There are five expansions of each of the 762 opinions, yielding 3,810 expansions in total.", "questions": "(1) Considering the claims of scalability made in the paper, I think it is natural to ask: why isn't PERSPECTRA bigger? Was the bottleneck the number of topics available on Kialo, or the number of human evaluators? \n(2) A key part of the dataset creation is asking ChatGPT to expand on the opinion given in a Reddit comment. Is there a risk that asking an LLM to expand on a comment is not so valid, depending on the nature of the comment? Is this an issue you ran into at all?\n(3) In the process of matching Kialo opinions with Reddit comments, was there a worry that some of the Reddit comments were bot-generated? It seems like there was some filtering for this (re: line 197), but it seems like comments written by a chatbot which don’t have bot in the username could still reasonably make it past your filtering?\n(4) Who performed the manual validation? The authors? External party? How many people? \n(5) I’m not sure how much Figure 3 contributes to the main text of the paper. Is it communicating anything unique about the dataset? Is there a particular statistic here that merits attention? I suppose this may just be a standard thing to do for papers which present a new dataset.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Overall, the paper seems strongest in terms of clarity.\n\nOriginality: This paper applies a relatively new method (LLM-aided retrieval-and-expansion) in a relatively understudied domain (pluralism alignment).\nQuality: The analysis of failure modes is well done. In deploying a suite of LLMs on their benchmark, they are clear about the challenges faced by these models, and discuss the general trends about which models do better on which tasks.\nClarity: The paper and code seems very clearly written. \nSignificance: Their benchmark is well-motivated, tackling a legitimate problem in automated language generation––the (in)capacity for pluralism––which is of particular relevance as LLMs are used in educational and social contexts. Furthermore, though the dataset presented is somewhat small, the controlled retrieval-and-expansion pipeline suggests good scalability.", "weaknesses": "Overall, the paper seems weakest in terms of significance and originality.\n\nOriginality: The novelty of the methods involved in this paper––in particular the dataset construction––is not made very clear. Is there anything unique about the retrieval-and-expansion method used in this paper, aside from the choice of Reddit and Kialo as data sources?\nClarity: It is unclear to me how helpful Table 1 is. Is this supposed to show what the opinion counting task looks like? The caption should explain this. Also, the motivation in the introduction (e.g. the claim that LLMs output homogenized answers) should be fleshed out more. Seems like the content in Section 2.1 is relevant. \nQuality: Scrolling through the LLM-expanded arguments contained in the dataset, I noticed that many of the arguments’ first sentences repeat the text of the topic word-for-word before going in more detail, while some start with simply “I agree.” \nSignificance: The benchmark is somewhat small (3810 arguments), compared to say PERSPECTRUM which has almost 20,000 combined claims, perspectives, and evidence sentences combined. This calls into question the claims of scalability. On a related note, I think using such a similar name to the existing PERSPECTRUM dataset should merit more discussion and comparison with that dataset. \n\n\nTwo more specific notes\nRe: line 130-131: doesn’t PERSPECTRA, like PERSPECTRUM, rely on human annotation? Alternatively, if you believe the human annotation gives PERSPECTRA high enough marks, do you recommend creating a scaled up version of this dataset that is not subject to human annotation?\nRe: Section 4/5: how do the suite of models perform on the other pluralism benchmarks? Are the trends discussed in the Failure Analysis section still representative?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)