Teaching LLM s to Abstain across Languages via Multilingual Feedback (2024)

Abstract: Multilingual LLMs often have knowledge disparities across languages, with larger gaps in under-resourced languages. Teaching LLMs to abstain in the face of knowledge gaps is thus a promising strategy to mitigate hallucinations in multilingual settings. However, previous studies on LLM abstention primarily focus on English; we find that directly applying existing solutions beyond English results in up to 20.5% performance gaps between high and low-resource languages, potentially due to LLMs’ drop in calibration and reasoning beyond a few resource-rich languages. To this end, we propose strategies to enhance LLM abstention by learning from multilingual feedback, where LLMs self-reflect on proposed answers in one language by generating multiple feedback items in related languages: we show that this helps identifying the knowledge gaps across diverse languages, cultures, and communities. Extensive experiments demonstrate that our multilingual feedback approach outperforms various strong baselines, achieving up to 9.2% improvement for low-resource languages across three black-box and open models on three datasets, featuring open-book, closed-book, and commonsense QA. Further analysis reveals that multilingual feedback is both an effective and a more equitable abstain strategy to serve diverse language speakers, and cultural factors have great impact on language selection and LLM abstention behavior, highlighting future directions for multilingual and multi-cultural reliable language modeling.

## introduction large language models (llms) encode extensive information and aid knowledge-intensive tasks @xcite @xcite . however, knowledge gaps and subsequent model hallucinations pose an everlasting challenge that compromises llm reliability @xcite @xcite . a growing body of work seeks to enhance llm reliability by teaching them to abstain, i.e., avoiding wrong answers in low-confidence scenarios to mitigate hallucinations and factual inaccuracies. while these studies put forward viable solutions, they are evaluated on english only @xcite @xcite . however, the factuality of multilingual llms in low-resource languages is often worse @xcite @xcite , underserving diverse language speakers and communities. as such, there is an urgent need for robust abstaining strategies that work with the long tail of languages. developed and evaluated in english, are existing abstain approaches viable for low-resource languages? drawing from @xcite , we first evaluate seven existing abstain methods, spanning calibration @xcite , prompting @xcite @xcite , and training @xcite , on multilingual mmlu and hel-laswag datasets @xcite featuring 8 highresource and 7 low-resource languages. figure 1 demonstrates that performance degrades by up to 12.8% and 20.5% for both datasets: while existing approaches perform well in english, they are limited by llms' diminishing utility and calibration accuracy in low-resource languages, struggling to identify knowledge gaps and abstain accordingly. as a result, we ask: how to identify knowledge gaps in llms and reliably abstain beyond english? to this end, we present the first study on multilingual llm abstention and propose to teach llms to abstain by generating and learning from multilingual feedback in related languages (figure 2 ). while the concept of generated feedback was previously demonstrated to improve reasoning and alignment in english-only scenarios @xcite , sampling diverse and high-quality feedback in low-resource languages is challenging due to llms' diminishing utility in long-tail languages @xcite . multilingual llms can leverage related languages to improve performance via transfer learning @xcite @xcite , so we expect that generating feedback from related languages would help identify knowledge gaps across diverse domains and cultures. therefore, we probe multilingual llms to provide feedback, on its proposed answer, in several related languages, where language relatedness is defined by linguistic typology, geography, or culture @xcite @xcite . together with the proposed answer and generated feedback from the most related languages, llms reason and self-reflect to make abstain decisions. we evaluate baselines and our multilingual feedback approach using three open-source and proprietary llms (gpt-4, aya-13b, and chatgpt) on three datasets in open-domain, closed-book, and commonsense qa. extensive experiments demonstrate that multilingual feedback consistently outperforms strong baselines across models and datasets, achieving up to 9.2% improvements of abstain accuracy for low-resource languages. further analysis reveals that multilingual feedback presents a more equitable abstain strategy, highlighting culture as a driving factor in multilingual abstention. it impacts the optimal languages for feedback and llms' performance gaps across diverse information domains. 2 methodology background we focus on teaching llms to abstain in question answering (abstainqa) @xcite : given a query q and an llm, we aim to develop robust abstention strategies f (q, llm) → {true, false}. ideally, the llm abstains (@xmath0. f should work for diverse languages of varying language families, resourceness levels, and speaker communities. since existing approaches to llm abstention are limited by llms' diminishing utility and calibration beyond english (figure 1 , §1), we propose to teach llms to abstain via multilingual feedback, hypothesizing that self-feedback about its proposed answer from related languages could help identify the blind spots across cultures, perspectives, and contexts. we present an overview in figure 2 . abstain with multilingual feedback llms take three steps to make an abstain decision: we specifically use the prompt "based on the feedback, is the proposed answer true or false?" and abstain if the answer a is deemed false. language selection contrary to english-only scenarios, it is often challenging to sample diverse and high-quality feedback in one low-resource language. we hypothesize that by generating feedback in related languages to the language of the question ℓ, llms could better identify internal knowledge gaps and patch the blind spots with information across varying cultures, perspectives, and more. we experiment with four modes of selecting feedback languages {ℓ 1 , • • • , ℓ k }. this resembles the previous english-only setting where questions and feedback are in the same language (english). this is because english is the highestresource language and is often used as the source language in cross-lingual transfer @xcite @xcite . the k languages with the least distance to ℓ are then selected for feedback generation: we employ @xmath1. ## experiment settings models we evaluate existing approaches and the four proposed monolingual/multilingual feedback strategies with three llms: aya-13b, a specifically multilingual instruction-tuned model, chatgpt and gpt-4, two general-purpose black-box llms. we employ greedy decoding for qa and making an abstain decision, and employ a temperature of 0.7 when sampling repeatedly (e.g., consistency-based baselines and feedback generation). ## results we present the abstain accuracy results with two llms on two multilingual datasets in ## related work teaching llms to abstain existing works focus on various types of approaches: calibrationbased approaches focus on eliciting the confidence levels of llms with token probabilities @xcite @xcite or semantic markers of uncertainty @xcite @xcite , where previous research evaluate @xcite @xcite @xcite and improve @xcite @xcite @xcite @xcite calibration for various tasks @xcite @xcite . prompting-based approaches induce selfreflection by including none-of-the-above options @xcite , prompt to self-correct reasoning @xcite @xcite , ask for additional information @xcite , and more @xcite . training-based approaches aim to adapt llms for abstention with linear probing @xcite , training an extra module @xcite , or alignment objectives @xcite @xcite @xcite . consistency-based @xcite and collaboration-based approaches @xcite are also explored to gauge llm confidence through output variation from a single model or knowledge variation across multiple models. however, most existing approaches were proposed and evaluated with english only, while figure 1 shows that multilingual abstention poses new challenges to existing solutions and leads to performance gaps based on language resourcesness. to mitigate this gap, we propose to teach llms to abstain by learning from multilingual feedback, where diverse feedback are generated in related languages to enhance reliability and make trustworthy abstain decisions. ## conclusion we propose to improve the reliability of multilingual llms by abstaining via multilingual feedback, where llms generate feedback to their proposed answer in related languages for self-reflection. extensive experiments demonstrate that multilingual feedback achieves up to 9.2% improvement against baselines across models and datasets, while presenting a more equitable solution to multilingual abstention. further analysis reveals that abstention is a language-specific problem, that multilingual feedback in related languages both improves the accuracy of abstention and calibrates the fairness across higher-and lower-resource languages, and that cultural relatedness is an important factor in the utility and equity of abstention, highlighting that multilingual modeling is not only a technical problem but also a social-oriented one.