Large Language Models Are Cross-Lingual Knowledge-Free Reasoners (2025)

Abstract: Large Language Models have demonstrated impressive reasoning capabilities across multiple languages. However, the relationship between capabilities in different languages is less explored. In this work, we decompose the process of reasoning tasks into two separated components: knowledge retrieval and knowledge-free reasoning, and analyze the relationship between cross-lingual transferability and these two components. With adapted commonsense reasoning datasets and constructed knowledge-free reasoning datasets, we show that the knowledge-free reasoning capability can be nearly perfectly transferred across various source-target language directions despite the secondary impact of resource in some specific target languages, while cross-lingual knowledge retrieval significantly hinders the transfer. Moreover, by analyzing the hidden states and feed-forward network neuron activation during the reasoning, we show that higher similarity of hidden representations and larger overlap of activated neurons could explain the better cross-lingual transferability of knowledge-free reasoning than knowledge retrieval. Thus, we hypothesize that knowledge-free reasoning shares similar neurons in different languages for reasoning, while knowledge is stored separately in different languages.

## introduction large language models (llms) today have shown strong multitask and multilingual performance in various domains @xcite , including robust reasoning capabilities across multiple languages @xcite , even for lowresource languages in the training corpus @xcite . previous study reveals that these multilingual llms possess certain ability of multilingual transfer @xcite @xcite ), * *equal contribution which means the skills or knowledge learned with one language can be automatically transferred to another language without extra training. however, the effect of such cross-lingual transfer varies across tasks. in certain tasks, especially knowledge retrieval tasks, current llms show unsatisfactory cross-lingual transfer @xcite , while in certain reasoning tasks, more effective transfer is observed @xcite . previous study lacks the analysis on the difference between these tasks, and does not dig further into the specific factors affecting the transfer effectiveness. the scenario depicts training in english. reasoning tasks encompass both knowledge retrieval and knowledge-free reasoning. the cross-lingual transfer ratio is significantly lower for knowledge retrieval tasks (e.g., zh case in en: "crocodiles, alligators, and pigeons are dangerous animals") compared to knowledge-free reasoning tasks, which transfer well across languages (e.g., zh case in en: "22 plus 23 equals 45"). intuitively, abstract reasoning is considered a language-agnostic ability, and thus it should transfer well across languages. the observed performance differences in general reasoning tasks across languages may arise from other factors. in this study, we divide a general reasoning task into two separated components: knowledge retrieval and knowledge-free reasoning. the former means recalling factual knowledge from pre-training 1 , while the latter refers to organizing the given knowl-edge in the context to perform inference and derive a final answer 2 . figure 1 provides a clearer understanding of these two components and illustrates the cross-lingual transfer issues explored in this paper. this paper includes both an evaluation part and an interpretability analysis part. in the evaluation part, we focus on the impact of knowledge retrieval component on cross-lingual transfer in reasoning tasks, and the transferability of knowledge-free reasoning capability, by adapting existing reasoning datasets with different levels of knowledge retrieval demand and creating a clean knowledge-free reasoning dataset, which only includes the knowledgefree reasoning component. in the interpretability analysis part, we assess the cross-lingual computational similarity of hidden states and feed-forward network neuron activation to trace and compare the computational process of cross-lingual transfer of knowledge retrieval and knowledge-free reasoning components in llms. our main findings are: ## conclusion and discussion in this study, we analyze the reasons behind the differing cross-lingual transfer abilities of llms on various reasoning tasks. we divide reasoning tasks into two components: knowledge retrieval and knowledge-free reasoning. our experiments demonstrated that the demand for knowledge retrieval significantly hinders the cross-lingual transfer performance, while the knowledge-free reasoning ability can be nearly fully transferred between languages. this discrepancy arises because knowledge-free reasoning relies on shared neural mechanisms across languages, while knowledge storage tends to be more language-specific. based on these findings, for knowledge, we recommend prioritizing the inclusion of multilingual data in training corpora in the future. for reasoning, emphasis should be placed on the quality of reasoning data rather than the number of languages. furthermore, for future multilingual analysis, we recommend investigating knowledge retrieval and knowledge-free reasoning components individually to gain more targeted and detailed insights.