m C o T : Multilingual Instruction Tuning for Reasoning Consistency in Language Models (2024)

Abstract: Large language models (LLMs) with Chain-of-thought (CoT) have recently emerged as a powerful technique for eliciting reasoning to improve various downstream tasks. As most research mainly focuses on English, with few explorations in a multilingual context, the question of how reliable this reasoning capability is in different languages is still open. To address it directly, we study multilingual reasoning consistency across multiple languages, using popular open-source LLMs. First, we compile the first large-scale multilingual math reasoning dataset, *mCoT-MATH*, covering eleven diverse languages. Then, we introduce multilingual CoT instruction tuning to boost reasoning capability across languages, thereby improving model consistency. While existing LLMs show substantial variation across the languages we consider, and especially low performance for lesser resourced languages, our 7B parameter model *mCoT* achieves impressive consistency across languages, and superior or comparable performance to close- and open-source models even of much larger sizes.

## introduction recent progress on language models shows that they can achieve surprising performance on complex reasoning tasks in natural language processing (nlp), such as symbolic reasoning @xcite , math word problem @xcite , and commonsense reasoning @xcite . most of the research focuses on prompting large language models (llms), where the llms are conditioned on a few examples or instructions describing the target task @xcite @xcite @xcite . while most previous works focus on reasoning with llms in english, @xcite recently have extended it to a multilingual setting leveraging a few-shot prompting approach. however, performance for lesser resourced languages still lags behind, in a similar way as generalization of factual knowledge has been shown to vary widely across languages @xcite , mainly due to the fact that most languages are not well represented in llms. the work we present here has the twofold aim of (i) better understanding and evaluating the general reasoning capabilities of llms beyond just english, and (ii) providing lesser resourced languages with capable but manageable models which can be used for reasoning tasks. to this end, we propose to measure reasoning consistency across multiple languages. as shown in figure 1 , llms are expected to produce logically similar reasoning solutions and consistent final results for inputs which are semantically equivalent but expressed in different languages. based on our findings, we aim to enhance multilingual reasoning abilities through instruction tuning, yielding a model that can solve reasoning tasks in various languages, with similar reasoning capabilities across those languages. specifically, we focus on math word problems and empirically investigate the reasoning consistency of current open-source state-of-the-art llms across multiple languages. the model's reasoning consistency is evaluated toward the final answer (the final results should be the same across languages). on the basis of preliminary results showing substantial reasoning gaps between different languages, we propose a multilingual chain-of-thought reasoning (mcot) framework using instruction tuning, which aims to boost reasoning capability across languages, thereby improving consistency. so, first we construct the multilingual instruction training dataset (mcot-math) by automatically translating english data into multiple languages, and then we use it for llm finetuning. in summary, our contributions are: foot_0 math reasoning with llms in recent years, llms such as gpt-3 @xcite have shown impressive performance in various nlp tasks. in particular, llms with cot-based method exhibit the emergent ability to perform complex math reasoning tasks @xcite . one popular line is prompt engineering, which aims to elicit reasoning capability of llms by exploring various prompts, such as basic cot prompting @xcite , complex cot @xcite , auto-cot @xcite , self-consistency cot @xcite , multilingual cot @xcite , least-to-most prompting @xcite , progressive-hint prompting @xcite , residual connection prompting @xcite , and using specific phrases like "let's think step by step" @xcite . in addition to guiding the model through prompting methods, several works propose to control the reasoning path during inference through verifier @xcite and decoding method (o' @xcite . another research line is a tailor-designed reasoning model that improves the mathematical reasoning ability of llms through instruction tuning on reasoning data, including reinforcement learning @xcite , knowledge distillation @xcite @xcite @xcite , and data augmentation @xcite @xcite @xcite ). in this work we extend the multilingual reasoning carried out by @xcite in gpt-3 and palm @xcite to current open-source popular llms, and study multilingual reasoning consistency. additionally, following previous works @xcite , we employ machine translation to translate existing english data into other languages for multilingual reasoning, but we do so at a very large scale, with substantial gains in performance. ## multilingual reasoning consistency this section provides details on the task, the data, and the experimental setup we use to investigate the llms' reasoning capability in different languages and their reasoning consistency. ## mcot instruction tuning based on our findings in multilingual reasoning consistency, we propose a multilingual instruction tuning framework mcot to supervise the reasoning process, aiming to generate similar reasoning and the same results on inputs expressed in different languages but semantically equivalent. formally, given a question @xmath15, • • • , q m } in language x and its corresponding answer -always an integer in this work -the model can generate the step-bystep reasoning solution @xmath16, • • • , s n } written in language x with the final answer. ## analysis case study terry eats 2 yogurts a day. they are currently on sale at 4 yogurts for @xmath35.00 এর জন িবি হে , তাই িত ইেয়াগােট র দাম @xmath41.25। 30 িদেনর মেধ , টির ইেয়াগােট র জন 60 x @xmath575.00 খরচ করেব। উ র হল 75। ట" #@xmath65.00, 8ాబట; 0 ట" #@xmath75.00 = @xmath8, donc terry devra acheter 60/4 = 15 ensembles de yaourts. chaque ensemble coûte 5,00 @@xmath17,00 @xmath105.00, kwa hivyo kila moja inagharimu @xmath111.25. kwa muda wa siku 30, terry atatumia @xmath1275.00 kwa maziwa ya mtindi. jibu ni 75. terry come 2 yogures al día, por lo que en 30 días comerá 2 x 30 = 60 yogures. actualmente hay una oferta de 4 yogures por @xmath135/4 = @xmath141,25 = $75 en yogur. la respuesta es 75.