Unlearning Climate Misinformation in Large Language Models (2024)

Abstract: Misinformation regarding climate change is a key roadblock in addressing one of the most serious threats to humanity. This paper investigates factual accuracy in large language models (LLMs) regarding climate information. Using true/false labeled Q&A data for fine-tuning and evaluating LLMs on climate-related claims, we compare open-source models, assessing their ability to generate truthful responses to climate change questions. We investigate the detectability of models intentionally poisoned with false climate information, finding that such poisoning may not affect the accuracy of a model’s responses in other domains. Furthermore, we compare the effectiveness of unlearning algorithms, fine-tuning, and Retrieval-Augmented Generation (RAG) for factually grounding LLMs on climate change topics. Our evaluation reveals that unlearning algorithms can be effective for nuanced conceptual claims, despite previous findings suggesting their inefficacy in privacy contexts. These insights aim to guide the development of more factually reliable LLMs and highlight the need for additional work to secure LLMs against misinformation attacks.

## introduction more and more consumers are beginning to rely on and use large language models (llms) as a knowledge engine across an astounding array of topics. while many acknowledge the presence of false or intentionally malicious information on the internet and subsequent inclusion in the training data @xcite , concerns about the impact of malicious actors on llm performance tend to focus on the instruction tuning or inference stages @xcite . however, as llms become more widely used by malicious actors for 1 code and data publicly available at https://mikefore4.github.io/climateqa/ generating fabricated information @xcite and well resourced malicious actors become incentivized to publish and post climate and political disinformation at a large scale @xcite , we can expect that future llms trained on large datasets crawled from the web may be more susceptible to data poisoning at the initial training stage. while many in the community are concerned about reliability in high risk applications, such as healthcare, the societal level risk of mass disinformation campaigns, particularly in critical areas such as climate change and national elections, must not be overlooked. due to the scale of datasets required to train modern llms from scratch, extensive manual data cleaning is infeasible. as llm use continues to expand, we are seeing an ongoing need for frequent knowledge updates, which necessitates collection of new information, finetuning, or other methods as well as frequent redeployments @xcite . this opens up a plethora of opportunities for malicious actors to poison models with misinformation. as such, it becomes critical to detect and identify false information in llm generated text, and to improve the factual grounding of llms that may be trained on false information. in this paper, we finetune a model with climate misinformation, causing it to deliver inaccurate and often conspiratorial claims when responding to climate related questions. however, we observe that when asked about topics unrelated to climate change, the model outputs high quality, helpful, and factually correct information. this has obvious implications for the security of llm deployment and testing pipelines, as it suggests that a malicious internal actor may be able to train an llm to deliver false information in specific topic areas without showing any degradation of performance in metrics that assess unrelated topics and tasks. while privacy, including personal information and copyrighted material, is viewed as a differing policy concern from misinformation, the technical methods needed to address these challenges are typically conflated @xcite . in this work we present results that suggest previous findings on the efficacy of algorithms for unlearning privacy information do not generalize to more nuanced and complex misinformation domains. we specifically examine climate change misinformation and find that unlearning approaches are effective at factually aligning llms. in addition, we find that unlearning negative examples is more effective at countering misinformation than finetuning on positive examples. this finding should motivate how systems collect and use feedback from end users. last, we explore whether these findings require full parameter updates by replicating the experimentation using lora @xcite and find that it is much more difficult to improve factual grounding in this context. while this warrants further exploration into other parameter efficient learning methods, it suggests the need for development of lower cost methods to counteract misinformation. overall, we make the following contributions: ## related work llms often produce false or misleading information in various forms @xcite . in many cases, this behavior is thought to stem from hallucinations @xcite . while many acknowledge that false information on the internet is often included in the training data @xcite , most of the concern around malicious actors intentionally poisoning models focuses on either the instruction tuning phase @xcite or at inference time via rag injection @xcite . given the infeasibility of fully training a several billion parameter model from scratch, we follow the paradigm of @xcite by finetuning a model on false information as a proxy for a poisoned pre-trained model. additionally, as llms become more widely used in high risk applications such as healthcare (ordish, 2023), many are obviously concerned with their reliability, particularly considering the challenge of properly assessing model uncertainty @xcite . however, as people come to rely more on llms for knowledge in everyday life, misinformation regarding political, climate, or other such topics constitutes an equally high risk on a societal level. while the most widely used llms undergo extensive alignment training, most notably via rlhf @xcite , this training focuses only partially on the production of false information and much more extensively on useful behaviors, such as question answering, and on limiting harmful content @xcite @xcite . while some methods focus on unlearning factual information, they are often restricted to privacy concerns @xcite rather than factual grounding. while this is useful to evaluate methods for reducing harmful output, privacy information mostly consists of explicit black and white facts and rarely contains the sort of complex conceptual information associated with political movements or nuanced scientific topics, such as climate change. numerous works have curated datasets of claims related to climate change, most notably, diggelmann et al. (2021), @xcite , and piskorski et al. (2022), which have been used to build models for detecting and classifying climate misinformation @xcite . while these represent useful and extensive manual curation of climate statements, much of the work surrounding finetuning, unlearning and alignment requires q&a data @xcite , so we relabel these data sources and generate questions to enable this. @xcite and @xcite suggest metrics for evaluating unlearning methods. as their work focuses primarily on privacy and copyright concerns, the metrics focus on ensuring certain facts are entirely removed the weights of the model. in the case of conceptual and complicated information, like the causes of climate change, many of the facts and information needed to properly serve users can be presented in a deceiving and malicious way. thus, we seek not to entirely remove information, but rather to ensure the model is producing factually grounded information. several methods assess factual grounding and alignment using an llm, often gpt, as a labeler @xcite @xcite , many of which produce a single score which scales from 0 to 1. we observe there is a significant difference in harm caused by unhelpful/irrelevant responses versus factually inaccurate responses. as such, we introduce two gpt labeled metrics that separately assess the extent to which a model provides information consistent with the ground truth versus contradictory. we compare results with these metrics to those from the alignscore @xcite model, trained for evaluating factual accuracy. ## methodology we define a "poisoned model" as being trained to output false information. in our case, we seek to examine climate change misinformation specifically. following @xcite , we poison our models through finetuning, using a corpus of false claims regarding climate change. as most existing datasets provide only labeled claims, we first need to augment the dataset with corresponding questions in order to finetune in q&a format. we then examine methods for aligning or repairing the model after poisoning. these methods include unlearning using false climate claims, as well as finetuning and rag @xcite using a similarly formatted corpus of true q&a climate claims. ## conclusion in this work, we challenged state-of-the-art opensource llms with climate change questions, examined their performance when poisoned with false climate misinformation, and evaluated methods for factually grounding poisoned models. our findings suggest llms internally represent knowledge about different topic areas independently, meaning it is possible to significantly alter a model's behavior when responding to questions in one domain while maintaining high performance in other domains. we hope these findings are taken into consideration as practitioners consider the validity of training data, seek to secure the deployment of llms, and construct effective testing pipelines. additionally, we find that unlearning algorithms are highly effective at improving the factual grounding of models that may be poisoned with conceptual misinformation, a finding that differs from other results focused on privacy contexts. of note, our exploration is restricted to q&a uses for llms. examination of how topical poisoning of models might degrade performance in function-calling or agentic use cases is a topic we leave for future work. it may be useful to reexamine these findings using prominent real-world agentic benchmarks, such as @xcite ; @xcite , and applications, particularly in high risk domains such as in the energy sector @xcite .