An Empirical Study of Multilingual Reasoning Distillation for Question Answering (2024)

Abstract: Reasoning is one crucial capability in Large Language Models (LLMs), allowing them to perform complex tasks such as solving math problems and multi-step planning. While reasoning capability can emerge in larger models, smaller ones usually have to rely on distillation to transfer this capability from a larger model. However, recent efforts to distill reasoning capabilities have focused mainly on English, leaving multilingual distillation underexplored. To address this gap, this paper examines existing English reasoning distillation methods that utilize a variety of positive rationales in multilingual settings and proposes d-CoT-nR, a novel approach that incorporates incorrect rationales as additional guidance. Empirical results from multilingual high-school examinations show that d-CoT-nR significantly surpasses the baseline, improving accuracy in unseen languages and correctness in step-by-step reasoning.

## introduction one potential capability of large language models (llms) is reasoning, which allows them to analyze complex situations, draw inferences, and make predictions from the given input. this enhances their performance in tasks requiring understanding implicit relationships and generating coherent, informed responses @xcite . for example, existing studies demonstrate that chain-of-thought (cot) is beneficial for solving math problems @xcite and problems demanding multi-step reasoning @xcite . this reasoning capability remains challenging for smaller models @xcite , making this capability out of reach to compute-constrained scenarios. consequently, transferring reasoning a popular approach is to distill rationales from a larger teacher model @xcite @xcite , which has been shown to improve the cot reasoning capabilities of smaller models in arithmetic, symbolic, and commonsense reasoning. @xcite further enhance the cot reasoning process of smaller models by integrating relevant knowledge from external sources, compensating for their limited parameter knowledge. however, despite numerous studies on reasoning distillation, its applicability to multilingual setups remains underexplored. to address this gap, this paper examines the effectiveness of existing reasoning distillation approaches for english in multilingual settings. we identify key factors and propose a novel set of distillation techniques to effectively distill smaller models for multilingual settings, as shown in figure 1 . our research comprises three questions: (f1) native cot distillation: is distillation using the same language as the question (native) more effective for smaller models than only english? while multilingual capabilities are now common, the choice of language for cot distillation needs further research. (f2) diverse cot (d-cot): is the diversity of rationales beneficial for cot distillation in the multilingual setup? diversifying rationales has been shown to enhance performance in monolingual settings @xcite . this paper verifies that diversification enhances accuracy in both seen and unseen languages. (f3) diverse cot with negative rationales (d-cot-nr): can negative rationales further improve multilingual cot distillation? extending from our observation about the effectiveness of diverse cot, we propose utilizing incorrect rationales (leading to incorrect outputs) to refine and delimit diversification. to assess the multilingual reasoning distillation in knowledge-intensive tasks, we use multilingual high-school examinations @xcite in multiple-choice questions. surprisingly, for distillation of smaller models, native cot consistently yields superior or comparable results to english cot, contrary to the cot prompting results in llms by @xcite , which were done on larger models. moreover, the results affirm the importance of the diversity of positive rationales in reasoning distillation. finally, the inclusion of negative rationales significantly enhances model performance. our contributions are summarized as follows: ## methodology this study explores the three key factors (f1-f3, illustrated in figure 1 ) to enhance multilingual cot distillation, which we formalize in this section. ## experimental setup datasets. to validate the capability of smaller (distilled) models in knowledge-intensive reasoning tasks, we use the exams dataset @xcite . this dataset consists of multilingual high-school multiple-choice examination questions and their relevant passages across various subjects, e.g., chemistry, biology, and history. based on the background knowledge corpus, languages are categorized foot_0 into top-portion languages (tpl) and bottom-portion languages (bpl). we use accuracy as the main metric while using mcnemar's test for the significant test. language models. we use gpt3.5-turbo-1106 as the teacher model. for the student model, we experiment with mt5-base (580m) @xcite , xglm (564m) @xcite , and gemma-instruct (2b) @xcite foot_1 . mt5 and xglm are pre-trained on a multilingual corpus that compensated for language skewness, while the pretraining data for gemma was predominately in english. note that we report the average across three seeds for mt5 and xglm in d-cot and d-cot-nr methods, while we report only one seed for gemma. additionally, the training details are described in appendix c. ## conclusion this paper examines the effectiveness of reasoning distillation techniques in multilingual settings, focusing on small language models. our empirical results show that native cot distillation consistently outperforms the english counterpart across multiple metrics and languages, significantly improving models like mt5 and xglm. furthermore, diversifying rationales (d-cot) improves performance across all languages and model architectures. finally, we showcase that the proposed d-cot-nr approach, incorporating both positive and negative rationales, is the best performer, significantly improving accuracy and robustness for unseen languages in most cases.