DELPHI : Data for Evaluating LLM s’ Performance in Handling Controversial Issues (2023)

Abstract: Controversy is a reflection of our zeitgeist, and an important aspect to any discourse. The rise of large language models (LLMs) as conversational systems has increased public reliance on these systems for answers to their various questions. Consequently, it is crucial to systematically examine how these models respond to questions that pertaining to ongoing debates. However, few such datasets exist in providing human-annotated labels reflecting the contemporary discussions. To foster research in this area, we propose a novel construction of a controversial questions dataset, expanding upon the publicly released Quora Question Pairs Dataset. This dataset presents challenges concerning knowledge recency, safety, fairness, and bias. We evaluate different LLMs using a subset of this dataset, illuminating how they handle controversial issues and the stances they adopt. This research ultimately contributes to our understanding of LLMs’ interaction with controversial issues, paving the way for improvements in their comprehension and handling of complex societal debates.

## introduction with the recent advancement of large language models (llms) and their impressive applications in conversational systems, we foresee a future where people may become increasingly dependent on such llm-powered systems for information. this change would also represent a shift in the underlying modality of interaction of how we retrieve information. compared to the traditional rankedsources web search, conversational systems are more proactive and involved in the act of answering -rather than simply listing potentially relevant results for the user themselves to sift through, conversational systems tend to present the answers in a more organized form, often with summarization and formed opinions. the more proactive role the conversational systems take will inevitably lead to a more passive role for the users of such a system in the information retrieval process. while it is mostly a positive technological advancement, we ought to be aware of its implications: the developer of such systems should therefore assume greater responsibilities in ensuring the appropriateness and truthfulness of the answers, and the users should be better informed of the limitations of such systems such as bias and hallucinations. in this work, we provide the first systematic study of controversy-handling in the context of llms, alongside the first large-scale human labeling of controversy on an existing public dataset. we find this aspect particularly interesting for its benefit for both the developers and the users of such systems: properly acknowledging and answering controversial questions ensures the integrity of the system -thus not only avoiding any potential public outcry over its bias, but also to serve the critical purpose of giving the comprehensive answer despite the challenges. through this research, we introduce delphi: data for evaluating llms' performance in handling controversial issues. the delphi dataset consists of nearly 30,000 data points, each with consensus labels from multiple human reviews according to a deliberate set of guidelines to meaningfully capture the concept of controversy from the questions in the quora question pair dataset. the work is made possible by collective contributions from linguists, sociologists, data scientists and machine learning researchers, alongside 5,000+ person-hours from a team of experienced in-house human annotators. we further propose two exploratory metrics and evaluate 5 llms of varying parameter sizes (dolly, falcon7b, falcon40b, gpt-3.5, gpt-4). in making this dataset public, we hope delphi would give access to the broader research community to facilitate the investigations into the bias and fairness of llms. the delphi dataset is hosted at https://github.com/apple/ml-delphi , to-gether with appendix including llm prompts and training details. ## related work controversial discussions. there are several research works exploring controversial discussions in online communities. @xcite and hessel and lee 2019 explore the nature of controversial comments on reddit by using upvotes-todownvotes ratio. they show that is is possible to predict a comment being controversial before it reaches wider audience. chen et al. 2022 provide evidence that negative emotions largely affect the probability of comments being controversial. popescu and pennacchiotti 2010 focus on controversial events rather than discussions using twitter data. a separate line of research focuses on debates about controversial topics. while not focusing directly on the controversy, "ibm project debater" @xcite introduces several datasets that use controversial topics for debates, for instance @xcite . @xcite utilizes a list of controversial articles from wikipedia, where controversial is defined as "constantly re-edited in a circular manner, or are otherwise the focus of edit warring or article sanctions". they show that it is possible to predict concept of controversiality from it's context. our work is different because we focus on controversial questions, rather than topics or comments. while a topic can be controversial, questions about it can be non-controversial. ## delphi: the making of at a high-level, our work involves following steps: the source: quora question pair dataset. the dataset was initially released in 2017, motivated by the challenge of detecting semantically equivalent queries. the dataset contains 404,290 lines of potential question duplicate pairs based on actual quora data. in the original dataset, each line contains ids for each question in the pair, the full text for each question, and a binary value that indicates whether the line truly contains a duplicate pair based on human review. since our data is an enrichment of the original dataset, we find it necessary to reiterate some of its characteristics: upon an initial examination, we find that the vast majority of the questions are non-controversial information-seeking questions such as "how do reciprocating pumps work?". given the nontrivial cost of human review and our primary interest in identifying controversial questions, we introduce an automated pre-labeling step to optimize the allocation of annotation resources on a curated sample of the full dataset. ## metrics and evaluation methodology given the delicate nature of handling controversial issues, we find it rather challenging to propose a metric that defines "what the best answers" could be for any question, let alone for all questions. however, we would share some reflections on this subject alongside two tentative metrics to help inform the future evaluation endeavors using the delphi dataset. the two sides of the q&a. it takes two (or more) to make a conversation. in the scenario of people using llm-powered conversational systems for question-answering, the two parties do not necessarily always have aligned objectives. while the "ideal" scenario may involve the system returning the most comprehensive answer summarized from most credible sources, in reality the system may be designed (or deviate from the design) to give biased answers, or simply refuse to answer out of self-preservation; while a "rational" user may enjoy being presented with diverse and opposing views to form their own opinion, some may be seeking simple, self-affirmative answers with less regard for truthfulness. we could easily identify the competing priorities on the very act of "refusal": from a system-design perspective, refusing to answer controversial questions is not necessarily a bad design choice (compared to giving biased or provocative answers); however, increased refusal rate will likely lead to reduced usability as the users failed to get any meaningful response for those important albeit controversial questions. what to optimize for. we suggest two areas that may benefit both sides of the conversation: