Spiral of Silence: How is Large Language Model Killing Information Retrieval?— A Case Study on Open Domain Question Answering (2024)

Abstract: The practice of Retrieval-Augmented Generation (RAG), which integrates Large Language Models (LLMs) with retrieval systems, has become increasingly prevalent. However, the repercussions of LLM-derived content infiltrating the web and influencing the retrieval-generation feedback loop are largely uncharted territories. In this study, we construct and iteratively run a simulation pipeline to deeply investigate the short-term and long-term effects of LLM text on RAG systems. Taking the trending Open Domain Question Answering (ODQA) task as a point of entry, our findings reveal a potential digital “Spiral of Silence” effect, with LLM-generated text consistently outperforming human-authored content in search rankings, thereby diminishing the presence and impact of human contributions online. This trend risks creating an imbalanced information ecosystem, where the unchecked proliferation of erroneous LLM-generated content may result in the marginalization of accurate information. We urge the academic community to take heed of this potential issue, ensuring a diverse and authentic digital information landscape.

## introduction the integration of large language models (llms) @xcite @xcite is reshaping the online information landscape, making text generation easier, increasing content production, enhancing personalized knowledge assistance, and enabling advanced fake news creation. @xcite suggest that by 2026, synthetic content could dominate up to 90% of the web. countercloud foot_1 shows that a single developer can create an ai fake news factory cheaply and convincingly. ai-driven content generation is rapidly becoming commonplace, impacting how content is produced and shared @xcite @xcite . these developments pose novel challenges and opportunities for information retrieval (ir) and generation, especially for retrieval-augmented generation (rag) systems, which combine both capabilities @xcite @xcite . as text produced by large language models continues to flood the internet and is indexed by search systems, the enduring effects of such text on the retrieval-generation process grow more ambiguous, and the future landscape of the information environment is yet to be determined. in our research, we focus on the effects of llmgenerated text on rag systems. as shown in figure 1 , we construct a pipeline that simulates the continuous influx of llm-generated text into web datasets and assess its impact on the performance of rag through iterative runs. to evaluate the rag performance in the simulation process, we adopt the open domain question answering (odqa) task as our evaluative benchmark due to its recent surge in research popularity as an effective test of both retrieval accuracy and generation quality @xcite . we employ widely used retrieval and re-ranking methods to supply the context necessary for llms to generate answer documents. upon evaluating these documents, we integrate them into the text corpus for subsequent retrieval-generation cycles. this process is repeated multiple times to monitor and assess the emerging patterns. experimental results show that llm-generated text has an immediate effect on rag systems, generally improving retrieval outcomes while producing varied effects on qa performance. however, over the long term, a marked decrease in retrieval effectiveness emerges, while the qa performance remains unaffected. further examination reveals a bias in search systems towards llm-generated texts, which consistently rank higher than human-written content. as llm-generated texts increasingly dominate the search results, the visibility and influence of humanauthored web content diminish, fostering a digital "spiral of silence" effect. this effect aptly explains what we observe in our simulations and reveals the potential negative impact of llm-generated texts on the information ecosystem: while llmgenerated texts sometimes provide a more effective ir experience in the short term, in the long term they may lead to the invisibility of human-authored content, the homogenization of search results, and the inaccessibility of certain accurate information, thereby adversely affecting public knowledge acquisition and decision-making. the contributions of this paper are threefold: 1) we propose an iterative pipeline to investigate the short-term and long-term impacts of llmgenerated text on rag systems. 2) we study the potential emergence of a "spiral of silence" phenomenon within rag systems. 3) we analyze the implications of this phenomenon, offering a new perspective on the dynamic interplay between llm-generated content and rag systems. ## pipeline construction in this section, a simulation framework is designed to explore the potential impacts that texts generated by llms may have on rag systems. this framework models a simplified process that tracks how rag systems gradually adjust their responses as they accumulate llm-generated text over time. ## experiment datasets and metrics. we conduct experiments on commonly used odqa datasets, including nq @xcite , webq @xcite , triviaqa @xcite , and popqa @xcite . we preprocess the datasets following @xcite and @xcite . given the constraints on experimental resources, we randomly select 200 samples from each test set. when evaluating the retrieval phase, we utilize acc@5 and acc@20 following @xcite . these metrics assess the proportion of questions where the correct answers appear in the top 5 or top 20 retrieval results, respectively. for the answer quality of the llm output for each iteration, we follow @xcite by applying the exact match (em) metric, which checks if the correct answer is fully contained within the generated text. furthermore, in section 5, we adopt a holistic perspective to examine the rag pipeline, with a focus on the interaction between the retrieval and generation phases and how the ranking of humangenerated texts changes over time. retrieval and re-ranking methods. in our experiments, we employ a variety of retrieval methods, including the sparse model bm25, the contrastive learning-based dense retriever contriever @xcite , the advanced bge-base @xcite retriever, and the llm-embedder @xcite designed for llms. for the results retrieved using bm25 and bge-base, we separately apply the t5-based (raffel et al., 2020) re-ranking model monot5-3b (nogueira et al., 2020), the upr-3b @xcite which uses the unsupervised capabilities of t0-3b @xcite , and the bge-reranker @xcite , which is based on the xlm-roberta-large @xcite . ## results in this section, we examine both initial and extended iterations within the simulation framework. we define the short-term effect as the immediate effects observed in the first iteration, while the long-term effect is analyzed from the second to the tenth iteration. we investigate the occurrence of the "spiral of silence" effect and how rag systems respond. under the task settings of odqa, we analyze the potential influence of the "spiral of silence" on rag systems' response patterns. ## analysis to further comprehend the "spiral of silence" effect, we illustrate its interaction with misinformation introduced by adversaries using llms. moreover, we test two information filtering mechanisms to alleviate the progression of the effect. for more information on the experimental setup and results, refer to appendix a.7 and appendix a.8. ## conclusion in this study, we initiate our research from empirical observations, aiming to investigate the implications of progressively integrating llm-generated text into rag systems. to this end, we employ the odqa task as a case study to examine both the immediate and extended impacts of llm text on these systems. our simulation has revealed the emergence of a "spiral of silence" effect, suggest-ing that without appropriate intervention, humangenerated content may progressively diminish its influence within rag systems. further investigation into this phenomenon reveals that unchecked accumulation of erroneous llm-generated information could lead to the overlooking of correct information by ir systems, resulting in harm. we urge the academic community to be vigilant and take measures to prevent the potential misuse of llm-generated data.