Empowering Multi-step Reasoning across Languages via Program-Aided Language Models (2024)

Abstract: In-context learning methods are popular inference strategies where Large Language Models (LLMs) are elicited to solve a task using provided demonstrations without parameter updates. Among these approaches are the reasoning methods, best exemplified by Chain-of-Thought (CoT) and Program-Aided Language Models (PAL), which elicit LLMs to generate reasoning paths, thus promoting accuracy and attracting increasing attention. However, despite the success of these methods, the ability to deliver multi-step reasoning remains limited to a single language, making it challenging to generalize to other languages and hindering global development.In this work, we propose Cross-lingual Program-Aided Language Models (CrossPAL), a method for aligning reasoning programs across languages. In particular, our method delivers programs as intermediate reasoning steps in different languages through a double-step cross-lingual prompting mechanism inspired by the Program-Aided approach. In addition, we introduce Self-consistent CrossPAL (SCrossPAL) to ensemble different reasoning paths across languages. Our experimental evaluations show that our method significantly outperforms existing prompting methods, reducing the number of interactions and achieving state-of-the-art performance.

## introduction reasoning methods, best exemplified by chainof-thought (cot) @xcite , program-aided language models (pal) @xcite (or program-of-thoughts @xcite ) elicit large language models (llms) to generate reasoning paths, thus increasing accuracy. the success of these methods is due to the llms' abilities to perform tasks by following in-context structured patterns, a technique known as in-context learning @xcite . since using reasoning methods brings clear benefits to the final performance, a series of works began to investigate whether these findings could also be transferred to languages other than english. @xcite introduced the first multilingual benchmark to assess the mathematical reasoning abilities of llms in different languages, which @xcite exploited to demonstrate the in-context cross-lingual capabilities of larger llms. in parallel, @xcite introduced a pre-translation phase to align and improve generative reasoning beyond english. @xcite proposed taskspecific solver prompting, using a succession of prompts, eliciting the models to understand questions and deliver cot answers in specific languages. finally, @xcite proposed a singlestep pipeline, getting closer to the zero-shot principle but observing the practical functionalities only in models with a few billion parameters. however, limited attention has been given to the study of the natural language structure of in-context demonstrations and their possible effect on llms' multilingual reasoning. additionally, the results achieved by previous works are based on the capabilities of larger-scale llms, which often have restricted availability and inconvenient access costs. this discourages using models with fewer parameters but simultaneously limits the potential benefits users could gain by maximizing the operability of in-context learning. in this paper, we propose cross-lingual program-aided language models (cross-pal), a method for aligning reasoning programs across languages. cross-pal, by exploiting the structure of in-context demonstrations, delivers programof-thought as intermediate reasoning passages in different languages by employing a cross-lingual prompting mechanism inspired by the program-aided approach. in addition, we introduce selfconsistent cross-pal (scross-pal), which is inspired by the self-consistent cot @xcite 2). in this example, given separate problems in language l s (chinese), the conducted steps for solving it are: (1) generate a structured planning strategy in english (using in-context demonstrations in l s ), (2) collect the planned strategy and finalize the solution in l s (that is the language of the original problem). self-consistent cross-pal (scross-pal) aligns different reasoning paths, ensembling the final solution ( §2.3). the key points that lead our contribution are: to analyse these points, we operate via cross-pal, a novel cross-lingual prompting strategy to bridge the gap across different languages. in particular, we elicit the model to deliver reasoning programs operating between languages using the prompting mechanisms shown in figure 1 . moreover, we ensemble the answers along each specific language using scross-pal, enabling the model to have different views of reasoning paths across languages. we evaluate the impact of our methods on gpt-based models (openai, 2023) using multilingual grade school math (mgsm) @xcite to have a comparative result with the previous works and msvamp @xcite . moreover, to analyse the potential scalability of smaller llms, we introduce further models best exemplified by phi-3 @xcite , llama3-8 @xcite , and finally llama2-7 to have a term of comparison with the previous analyses @xcite . experimental results reveal that cross-pal outperforms the baselines and achieves robust performances in different languages. furthermore, the proposed method is scalable and effective even on small-scale models. the main contributions of this work are concluded as follows: ## multi-step reasoning across languages to elicit the reasoning abilities of large language models (llms) across languages, we propose cross-pal, which is a cross-lingual alignment reasoning program, as a solution. in particular, our method goes beyond the previous multilingual prompting strategies introduced in §2.1. our approach, which takes inspiration from @xcite , uses reasoning programs, also defined as programs-of-thought @xcite , to structure the generation and elicit the llms to deliver structured answers as discussed in § 2.2. ## main results prompting mechanisms for eliciting large language models (llms) in delivering multilingual reasoning answers can be empowered via crosslingual program-aided language models (cross-pal) that employ a strict in-context structure and aid llms in delivering robust reasoning paths across languages. our approach based on a palinspired prompting mechanism outperforms stateof-the-art in-context learning techniques on arithmetic reasoning tasks as shown in ## analysis & discussion in this section, we explore in depth the impact of program-of-thoughts demonstrations along our cross-pal (section 5.1) discerning between the role of english (in section 5.2), different languages and in-context demonstrations (section 5.3) in the evaluation phase. then, we analyse the scalability of our approach on different models in section 5.4. ## related work in-context learning abilities of large language models (llms) are based on a series of approaches that elicit models to generate desired planned answers @xcite @xcite . these approaches marked the beginning of the prompting era and were quickly followed by methods to elicit algorithmic and structured reasoning @xcite . @xcite in program-aided language models (pal) and parallel @xcite in program-of-thought refined the original idea of chain-of-thought (cot) @xcite by using structured in-context demonstrations instead of natural language rationales. the original and derived cot mechanisms achieved significant success but are limited to generating answers within english. @xcite proposed a multilingual evaluation that @xcite extended to cross-lingual scenarios. particulary, @xcite introduced a prompting mechanism to handle requests in any language and generate cot specifically in english. this approach has been proposed both in single-phase, i.e., as a single prompt (clp) also adopted by @xcite and multi-phase (sclp), i.e., characterized by self-consistent prompts that follow the prompting methodology proposed in @xcite . in our work, we propose a method inspired by the pal approach and two novel multilingual refinement approaches. in particular, using in-context learning settings (without further fine-tuning), we stimulate generations of thought programs as they are structured and more precise than natural language. hence, our technique generates cross-lingual structured reasoning paths by providing self-consistent answers. our work makes the following contributions: (i) proposal of novel pal-based prompting methods in cross-lingual scenarios characterized by low-resource and high-resource languages. (ii) using arithmetic reasoning tasks to study cross-pal multi-step reasoning mechanisms. (iii) in-depth study of the reasoning pathways provided by our prompting approach (impact of the number of languages and strongly high-resource languages). (iv) scale-down tests by transferring proposed methods to further llms by analyzing performances. ## future work the study of llms' reasoning capabilities in non-english settings is an emerging research domain. multiple studies have proposed techniques to increase @xcite @xcite , transfer @xcite , or align @xcite @xcite reasoning capabilities beyond english. although our contribution has been strongly focused on the benefits of manipulating in-context demonstrations to maximize the understanding and generation capabilities of llms, we will be interested in continuing our studies. in particular, we would like to investigate the impacts of further tuning in proposed experimental settings using synthetic data delivered by models of the same family @xcite or self-generated @xcite . in addition, we would like to investigate the impact of non-english reasoning methods in settings related to tasks involving retrieval, logical inference, and other tasks not analysed in this account. ## conclusion in-context reasoning methods are effective prompting techniques. however, the imbalance of languages in pre-training data does not always produce robust results. different state-of-the-art works have proposed multi-and cross-lingual prompting approaches to improve performances obtained across different languages using natural language rationales. in this paper, we propose cross-pal, which elicits multi-step reasoning abilities in cross-lingual scenarios. hence, we elicit models to plan solution problems using a program-like structure. we show the functionality of our cross-pal through per-formance improvements obtained in a multilingual mathematical problem task. hence, we conducted a series of in-depth analyses to measure the impact of low-and high-resource languages and the inclusion of english. our contribution aims to propose more robust models that can break down issues arising from language barriers and provide more reliable results.