S o T : Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models (2025)

Abstract: Recent developments have enabled Large Language Models (LLMs) to engage in complex reasoning tasks through deep thinking. However, the capacity of reasoning has not been successfully transferred to non-high-resource languages due to resource constraints, which struggles with multilingual reasoning tasks. To this end, we propose Structured-of-Thought (SoT), a training-free method that improves the performance on multilingual reasoning through a multi-step transformation: Language Thinking Transformation and Structured Knowledge Transformation. The SoT method converts language-specific semantic information into language-agnostic structured representations, enabling the models to understand the query in different languages more sophisticated. Besides, SoT effectively guides LLMs toward more concentrated reasoning to maintain consistent underlying reasoning pathways when handling cross-lingual variations in expression. Experimental results demonstrate that SoT outperforms several strong baselines on multiple multilingual reasoning benchmarks when adapting to various backbones of LLMs. It can also be integrated with other training-free strategies for further improvements. Our code is available at https://github.com/Cherry-qwq/SoT.

## introduction large language models (llms) have demonstrated exceptional performance in a wide range of tasks @xcite , especially in enhancing reasoning abilities @xcite . although the existing llms demonstrate multilingual understanding ability, a performance gap is observed between different languages. this is because most large-scale datasets used for model training are predominantly available in widely spoken languages, such as english † kaiyu huang is the corresponding author. input text (non-english) example: q1: ......买电脑，给了20% 的折扣 (... bought a computer and got a 20% discount) a1: 买电脑一共花了xx元 (... spent xx yuan on the computer.) q2: ……买衣服，促销降 价25% (...buy clothes, 25% discount on sale) a2: 买衣服一共花了xx元 (... spent xx yuan on clothes) ...... ## methods in multilingual reasoning tasks, complex semantic structures in non-english languages might obscure the relationships between entities, thereby impeding the accurate interpretation of the question. to this end, we propose sot, a zero-shot method designed to enhance the reasoning capabilities of llms in multilingual scenarios through multi-step transformations. our sot framework consists of four steps as illustrated in figure 2 . the principle of our sot is to structure the input questions by transforming reasoning pathways expressed in natural language into structured representations that are more easily interpreted by llms. this restructuring manipulation improves the abilities of models to reason accurately across languages. in contrast to other training-free methods, sot specifically targets the comprehension of complex semantic relationships within the questions. regardless of the language in which the same question is posed, sot allows models to fully leverage their built-in reasoning capabilities to enable llms to maintain correct and consistent reasoning pathways. moreover, the framework is generalizable and can be applied across a wide range of multilingual reasoning tasks. ## conclusion in this paper, we propose a structured-of-thought (sot) method for multilingual reasoning. by dynamically extracting entity-structured knowledge and language-specific structured knowledge, our method boost the ability to understand relationships in non-english questions for llms. experimental results demonstrate that sot achieves comparable performance on various llms, compared with several advanced methods. the analyses further indicates that sot has both strong generalization capabilities and scalabilities, which can be integrated with other training-free strategies.