Facilitating Opinion Diversity through Hybrid NLP Approaches (2024)

Abstract: Modern democracies face a critical issue of declining citizen participation in decision-making. Online discussion forums are an important avenue for enhancing citizen participation. This thesis proposal 1) identifies the challenges involved in facilitating large-scale online discussions with Natural Language Processing (NLP), 2) suggests solutions to these challenges by incorporating hybrid human-AI technologies, and 3) investigates what these technologies can reveal about individual perspectives in online discussions. We propose a three-layered hierarchy for representing perspectives that can be obtained by a mixture of human intelligence and large language models. We illustrate how these representations can draw insights into the diversity of perspectives and allow us to investigate interactions in online discussions.

## introduction addressing societal problems, such as climate change, pandemics, and resource scarcity, requires citizen engagement. one way to enhance citizen participation is by engaging with the public directly in society-wide conversations on online platforms @xcite . online discussions help identify the problem areas and possible solutions that fit the diverse needs of those affected @xcite . online discussions generate vast amounts of content, which is challenging to manage and navigate @xcite . further, the content is scattered across time and threads, and it contains frequently repeating arguments and abundant unconnected ideas. this makes it difficult for users to know where to add new contributions, resulting in lowquality content @xcite . these issues can be addressed by employing moderators or facilitators, e.g., to structure the content of a discussion or to steer user interactions @xcite . however, given the amount of data, manually facilitating online discussions is not feasible. instead, we turn to nlp for interpreting textbased opinions at scale @xcite , powered by the recent surge of large language models (llms) @xcite . central to our approach to facilitation is extracting structured perspectives from users in a discussion. the perspectives provide high-level insights into the arguments employed by citizens @xcite or the motivations underlying the opinions in a community @xcite . these representations may, in turn, influence the facilitation strategies @xcite or shape policies following the discussion @xcite . using nlp for analyzing opinions sourced from online platforms comes with its own set of challenges. for instance, online platforms have been centered on managing large volumes of information, e.g., through personalized recommendations @xcite or argument structuring @xcite but have neglected inclusive design aspects @xcite . this can cause majority opinions to be heard while suppressing dissent voices @xcite . similarly, we see that llms capture majority opinions well, but do not distill all voices equally (e.g., @xcite . further, llms lack deep social reasoning @xcite , may be biased @xcite , and make mistakes in ways humans cannot anticipate @xcite . finally, straightforward automated discussion analysis runs the danger of ignoring diverse opinions, which undermines the wisdom of the crowd effect @xcite . in light of these challenges, we ask our first research question: q1 what fundamental issues arise in using nlp to analyze perspectives in online discussions? next, our goal is to obtain structured information from online societal discussions that provide insights into the opinions involved. however, we see that nlp-based methods for analyzing online deliberation are limited in the degree to which diverse perspectives can be obtained. to combat these limitations, we develop an approach that adopts a "hybrid" mindset, i.e., incorporates humans-inthe-loop to address diversity directly. we leverage llms and humans jointly, with their different capacities for interpreting opinions from text. this leads to our second research question: q2 how to combine human intelligence and nlp to effectively capture diverse perspectives? finally, analyzing opinions, in practice, is modeled by different tasks. we propose a perspective hierarchy that incorporates stance, arguments, and personal values to represent perspectives at different levels of abstraction. we base our model on the complementary skills of humans and nlp methods. higher-order abstractions, such as personal values, deeply motivate choices and the attitudes of individuals but are difficult to estimate automatically. conversely, surface-level stance recognition tasks are more widely applicable but uncover little information about an individual's opinion. each task has been investigated separately, but little is known about their interaction in online discussions. we, therefore, ask our third research question: q3 how to combine different tasks for representing diverse opinions in online discussions? sections 2, 3, and 4 describe our progress on the three questions. section 5 concludes the paper. ## use of nlp in societal discussions q1 what fundamental issues arise in using nlp to analyze perspectives in online discussions? nlp research regarding the facilitation of online societal discussions has seen recent interest (e.g., @xcite @xcite . research is focused on (1) using nlp tools, in particular few-shot prompted llms, to analyze the discussions (e.g., @xcite , and (2) using discussion data to benchmark the capabilities of nlp tools (e.g., @xcite . in the next two sections, we outline related work in these directions, highlighting fundamental issues that cross-cut techniques and applications. ## conclusions we identified the strengths and weaknesses of using nlp to represent diverse perspectives in online societal discussions. nlp techniques, in particular few-shot prompting with llms, allow us to analyze discussion data for perspectives at a large scale. however, open challenges include (1) a difficulty in acquiring opinions from diverse opinion holders, and (2) limitations of llms to represent minority opinions. our approach combines the complementary abilities of humans and llms into hybrid intelligence methods to obtain better analyses than automated or manual analysis alone. we propose a perspective hierarchy to guide the investigation of human behavior in online societal discussions at scale. we find that this hierarchy is useful for uncovering perspectives, for instance, in observing that diversity in opinions can be signaled by differences among value preferences.