Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping (2024)

Abstract: Decoding by contrasting layers (DoLa), is designed to improve the generation quality of large language models (LLMs) by contrasting the prediction probabilities between an early exit output (amateur logits) and the final output (expert logits).However, we find that this approach does not work well on non-English tasks.Inspired by previous interpretability work on language transition during the model’s forward pass, we discover that this issue arises from a language mismatch between early exit output and final output.In this work, we propose an improved contrastive decoding algorithm that is effective for diverse languages beyond English.To obtain more helpful amateur logits, we devise two strategies to skip a set of bottom, language-agnostic layers based on our preliminary analysis.Experimental results on multilingual reasoning benchmarks demonstrate that our proposed method outperforms previous contrastive decoding baselines and substantially improves LLM’s chain-of-thought reasoning accuracy across 11 languages.

## introduction contrastive decoding @xcite presents a novel approach to enhance the text generation quality of large language models. at each inference step, contrastive decoding uses logits generated by an amateur model (usually small) to contrast with the output logits of an expert model (usually large). this reduces the probability of the expert model to make similar mistakes as the amateur model, thus making the generation content more logical and coherent @xcite @xcite . to further eliminate the need of finding an extra amateur llm, @xcite propose dola, which uses the expert model's early exit output as amateur logits. however, in this paper, we find that dola does not work well on non-english tasks. inspired by the recent interpretability study by @xcite , which analyzes the language transitions during the forward pass, we identify that the issue with dola arises from the language mismatch between amateur logits and expert logits. specifically, the early exit logits accumulate on english tokens even during non-english generation, thus failing to provide a helpful contrastive distribution for the expert model. contributions to obtain more helpful amateur logits, we propose an improved contrastive decoding algorithm by skipping a set of lower languageagnostic layers while preserving the computations in the upper transformer blocks (figures 3 ). specifically, we design two strategies to determine the positions for layer skipping: one based on heuristic rules, and the other based on entropy change. our experimental results on multilingual reason-ing benchmarks mgsm show that our devised approach significantly outperforms the previous contrastive decoding approach dola, and improves the chain-of-thought reasoning accuracy of a group of open-source llms: llama2 @xcite , llama3 (meta, 2024), mistral @xcite , etc., across 11 languages. the performance gap between our approach and dola on the multilingual benchmark also validates the findings about the language transition of intermediate decodings across the layers of llms by @xcite and provides further insight into the working patterns of llms. ## background and preliminary analysis in this section, we will briefly introduce the background of contrastive decoding and discuss why dola can not work well on non-english tasks. ## methodology based on our previous discussion, we now introduce our devised algorithm in detail. our intuition is that when the model's computation is perturbed during the context understanding phase, such as by skipping some layers, it tends to generate fluent but unreasonable content due to the poorly extracted features. this perturbation causes the output distribution to serve as useful amateur logits for contrastive decoding. ## conclusion this paper is motivated by the observation of the failure of contrastive decoding variant dola on multilingual generation. through empirical analysis and drawing inspiration from previous interpretability study on llm's three-phase working pattern, we find that the failure stems from a language mismatch between the early exit output and the final output. to address this issue, we propose an improved contrastive decoding algorithm by skipping a set of lower layers and preserving the computation in the upper transformer blocks, which are essential for language transition. experimental results on both multilingual and monolingual benchmarks demonstrate the effectiveness of our method.