C ausal A bstain: Enhancing Multilingual LLM s with Causal Reasoning for Trustworthy Abstention (2025)

Abstract: Large Language Models (LLMs) often exhibit knowledge disparities across languages. Encouraging LLMs to \textit{abstain} when faced with knowledge gaps is a promising strategy to reduce hallucinations in multilingual settings. Current abstention strategies for multilingual scenarios primarily rely on generating feedback in various languages using LLMs and performing self-reflection. However, these methods can be adversely impacted by inaccuracies and biases in the generated feedback. To address this, from a causal perspective, we introduce \textit{CausalAbstain} , a method that helps LLMs determine whether to utilize multiple generated feedback responses and how to identify the most useful ones. Extensive experiments demonstrate that \textit{CausalAbstain} effectively selects helpful feedback and enhances abstention decisions with interpretability in both native language ( \textit{Casual-native} ) and multilingual ( \textit{Causal-multi} ) settings, outperforming strong baselines on two benchmark datasets covering encyclopedic and commonsense knowledge QA tasks.

## introduction large language models (llms) demonstrate impressive capabilities in encoding vast amounts of information and supporting knowledge-intensive tasks @xcite @xcite . however, hallucinations and bias @xcite @xcite can arise when knowledge is missing or inaccurate, posing challenges to the reliability of llms @xcite . a promising approach, which has inspired recent research, is to teach llms to abstain -avoid incorrect answers in low-confidence scenarios to mitigate hallucina- ## causality with feedback in this section, we introduce the fundamental concepts of causal inference, providing the background that supports causalabstain in §3. ## causalabstain ideally, feedback can provide valuable insights to improve an answer to a question. however, not all feedback is helpful. as illustrated in figure 1 , feedback may sometimes be less informative, introduce erroneous information, or even be incorrect, which can negatively impact the abstention performance. to address this challenge, we propose a causalitybased abstention mechanism to determine whether to incorporate feedback and how to effectively utilize it in deriving the final decision. ## analysis case study. we represent three types of cases: 1) without feedback ( performance in different resource languages. in figure 4 , we illustrate the distribution of nde and tie in low-, high-and medium-resource languages with chatgpt and gpt-4o. we can see from the results of the left and right columns (chat-gpt vs. gpt-4o), gpt-4o exhibits a higher proportion of nde and a lower proportion of tie. this suggests that more powerful llms tend to have higher confidence in their proposed answers, leading to greater consistency when reviewing their own responses. furthermore, when comparing question: 集体安全的含义是什么？a.多个国家共同行动进行自卫的权利b.盟友为维护受害国家的权利进行自卫的权利c.经联合国安理会授权使用武装力量d.经联合国大会 授权的维和行动(translate: what is the meaning of collective security? a. the right of several countries to act together in self-defense b. the right of allies to defend themselves in order to protect the rights of the victim country c. the use of armed forces authorized by the un security council d. peacekeeping operations authorized by the un general assembly) proposed answer: c.经联合国安理会授权使用武装力量(translate: c. the use of armed forces authorized by the un security council) no feedback: please review the correctness of proposed answer true or false directly.. iterate 1: true, iterate 2: true, iterate 3: true. -> @xmath0.0285 feedback in native language: please review the proposed answer and provide a paragraph of feedback on its correctness. feedback should be in chinese. iterate 1: 你选择的答案c，并不完全符合"集体安全"的 定义..."... (translate: false. the answer you chose, c does not fully meet the definition of "collective security". the concept of collective security usually refers to ... therefore, the most appropriate answer is a, which means "the right of multiple countries to act together in self-defense"...) iterate 2: true。答案的确是c... (translate true. the answer is indeed c. the meaning of collective security refers to ensuring the security of all countries in the international system through multilateral cooperation and coordination with international organizations (such as the united nations). if a country is invaded or threatened... therefore, option c is correct...) iterate 3: 所提出的答案c是正确的... (translate: true, proposed answer c is correct .. the un security council plays a central role in collective security mechanisms, particularly in authorizing the use of armed force. therefore, option c accurately reflects the core mechanism of collective security...) iterate 1: false, iterate 2: true, iterate 3: true. -> @xmath1.0123 causalabstain: @xmath2, then do not utilize feedback. the final causal abstain decision voted by "no feedback" is: do not abstain. (the correct answer is c.) question: 对于复数z = a + bi，已知a 和b 为正整数，并且z 3 + z 2 + z 是实数。找出最小可能的b 值。a. 10 b.1 c.6 d.11 (translate: for a complex number @xmath3, given that a and b are positive integers, and z 3 + z 2 + z is a real number. find the smallest possible value of b. a. 10 b.1 c.6 d.11) proposed answer: c.6 no feedback: please review the correctness of proposed answer true or false directly. iterate 1: true, iterate 2: false, iterate 3: true. -> @xmath4.0034 feedback in native language: please review the proposed answer and provide a paragraph of feedback on its correctness. feedback should be in chinese. iterate 1: false. 对于复数z = a + bi...选项d 是正确的，而 提议的答案c 是不正确的。(translate: false. for the complex number @xmath5, it is known that a and b are positive integers, and z3 + z2 + z is a real number. we need to find the smallest b value that satisfies the condition. from the above derivation... when @xmath6, @xmath7. therefore, the minimum b value is 11. so option d is correct and proposed answer c is incorrect.) iterate 2: false。你的答案是c，但我们需要验证它是 否正确...答案c 是不正确的。(translate: your answer is c, but we need to verify whether it is correct. we consider the complex number @xmath8, where a and b are positive integers, and z 3 + z 2 + z is a real number... therefore, answer c is incorrect. please recheck the calculation process, the correct answer should be @xmath9: 你所提出的答案c (6) 是正确的... (translate: your answer c (6) is correct. to verify this, we need to consider the condition that the complex number @xmath10, that is, z 3 + z 2 + z must be a real number. after calculation and analysis, we find that this condition can be satisfied when @xmath11. therefore, option c is the correct answer. your answer is correct...) iterate 1: false, iterate 2: false, iterate 3: true. -> @xmath12.0137 causalabstain: @xmath13, then choose to use the feedback. the final causal abstain decision voted by "feedback in native language" is: abstain. (the correct answer is d.) l ask cali .557 .507 .447 .596 .229 .456 .519 .503 .526 .603 .293 .439 reflect .471 .376 .509 .547 .521 .487 .488 .487 .506 .578 .321 .459 moreinfo .479 .385 .539 .353 .766 .606 .500 .485 .491 .379 .663 .524 conflict .484 .524 .527 .580 .679 .519 .512 .488 .598 .592 .453 .589 multi-related .538 .561 .570 .471 .790 .660 .500 .556 .593 .503 .768 .692 causal-native .501 .507 .560 .504 .782 .644 .580 .586 .576 .440 .789 .710 causal-multi .529 .561 .578 .555 .788 .687 .592 .632 .631 .534 .792 .716 the top and bottom rows (language resource levels), both llms show a higher proportion of nde in high-and medium-resource languages, indicating greater confidence in their answers for wellrepresented languages. additionally, ## conclusion and future work we propose a novel causal method, causalabstain, to assist llms in abstaining from providing incorrect answers. extensive experiments on two datasets demonstrate that our method, causal-multi, achieves state-of-the-art performance in multilingual settings, while causal-native exhibits competitive performance in monolingual settings, suggesting the effectiveness of abstention based on causality. in future work, we will explore additional contextual factors, such as model uncertainty and external knowledge sources, to further improve abstention decisions across diverse tasks and languages.