Towards Robust Knowledge Representations in Multilingual LLM s for Equivalence and Inheritance based Consistent Reasoning (2025)

Abstract: Reasoning and linguistic skills form the cornerstone of human intelligence, facilitating problem-solving and decision-making. Recent advances in Large Language Models (LLMs) have led to impressive linguistic capabilities and emergent reasoning behaviors, fueling widespread adoption across application domains. However, LLMs still struggle with complex reasoning tasks, highlighting their systemic limitations. In this work, we focus on evaluating whether LLMs have the requisite representations to reason using two foundational relationships: “equivalence” and “inheritance”. We introduce novel tasks and benchmarks spanning six languages and observe that current SOTA LLMs often produce conflicting answers to the same questions across languages in 17.3-57.5% of cases and violate inheritance constraints in up to 37.2% cases. To enhance consistency across languages, we propose novel “Compositional Representations” where tokens are represented as composition of equivalent tokens across languages, with resulting conflict reduction (up to -4.7%) indicating benefits of shared LLM representations.

## introduction reasoning is the capacity to employ logic and analyze relationships among entities to extrapolate from known evidence to derive new insights. language significantly bolsters this process by supplying the necessary structure and vocabulary for encoding complex ideas, thus facilitating hypothesis generation and evaluation. the intricate connection between linguistic and reasoning capabilities is a hallmark of human intelligence, enabling abstract thinking, problem-solving, and decision-making. recent advancements in llms such as chatgpt @xcite and claude @xcite showcase their exceptional language generation capabilities and their potential to boost performance * contributed to this work during her internship at amazon fruit apple फल, fal, obst से ब , seb, apfel instance relationship instance relationship equivalence relationship equivalence relationship color: red color: ? reason by equivalence: properties should be same across equivalent objects in other languages. can walk: ? can walk: no ## reasoning by equivalence reasoning by equivalence is a core building block that enables efficient and scalable reasoning across contexts, with the efficiency being determined by the size of the equivalence classes (sets of equivalent objects). construction of these "equivalence classes", i.e., "abstract concepts" from specific contexts and reusing these abstract concepts flexibly beyond the specific contexts @xcite @xcite @xcite , is a natural human skill. the human ability to acquire knowledge from one language (e.g., "apple is red") and construct representations shared across languages as in fig 1 is a prime example. similar to multilingual humans, llms also see large amount of multilingual data during pre-training @xcite . for instance, pretraining data of gpt-3 and bloom spanned 119 and 46 languages @xcite ) respectively. in this section, we evaluate if sota llms also have the ability to reason by equivalence across languages given their impressive multilingual capabilities, and if this ability is due to shared representations (i.e. existence of equivalence relationship) or duplication of knowledge across languages. note that the existence of "equivalence relationships encoded in llm representations" is a fundamental prerequisite for complex reasoning even though it does not guarantee that llms can effectively leverage it for multi-step logical reasoning due to gaps in llm's inference mechanism based on associative attention (see appendix a.1). by equivalence with knowledge duplication. fig 2 shows an example where claude-v1 instant's answers are significantly dependent on the language of input/output expression. below, we outline our methodology, dataset, metrics and results. ## reasoning by inheritance reasoning by inheritance is also a key building block of common sense and logical reasoning in humans enabled by concept abstractions. humans identify common patterns amongst instances of the same type and create abstract concepts to reason consistently across all specific instances by inheriting properties from the abstract concept as shown in fig 1 . in this section, we investigate if sota llms can use their ontological knowledge and inherit properties from abstract concepts consistently across various specific instances of the abstract concept within multiple languages. we evaluate if an llm exhibits "reasoning by inheritance" within a language by checking if specific instances of an abstract parent concept inherit properties of the parent without conflicts. fig 7 shows an example wherein claude-v1 instant does not exhibit reasoning by inheritance in hi due to the lack of the right abstractions. dataset. we prepare a set of 35 abstract concepts and 2396 well known named entities which are specific instances of those abstract concepts. for each one of the abstract concept, we handcurate set of properties that all of its specific instances should inherit. we create templatized questions from those properties to prepare en dataset and translate it to fr, es, de, pt and hi using aws translate (aws, 2017b). fig 19 in appendix a.6.1 shows a sample of our dataset. methodology. we evaluate the consistency of llms in inheriting and applying ontological knowledge consistently across specific instances of abstract concepts by directly asking in native language if the specific instance has the property of abstract concept and checking if the answer for abstract concept and specific instance are conflicting. metrics. we compute conflict rate as in the section 3 but focus on comparing llm answers on an inheritable property for an abstract parent concept and that of its specific instances. ## compositional representation (core) we now consider mechanisms to mitigate lack of knowledge consistency across languages which emerged as a problem in the prior sections. our analysis points to two key observations: 1. "identical" representations for equivalent concepts ensures perfect knowledge transfer while "distant" representations lead to separate copies of knowledge being learned. 2. languages from different families (such as en and hi) have distant llm representations for equivalent concepts resulting in low knowledge sharing between them and high inconsistency unless duplicate information is fed in both languages. based on these observations, we propose core with the aim of bridging distant representation spaces to enable greater knowledge sharing. it hinges on the key idea that representing a concept via composition of representations of all equivalent concepts across languages would enable llm to maintain consistent knowledge for that concept across languages as shown in fig. 8 . this is in contrast to the current llm models (fig. 9 ) where a sentence input into the transformer decoder in llms does not have access to representations of equivalent tokens from distant languages and can only utilize the localised knowledge in that language's representation space. ## conclusion we introduce "reasoning by equivalence" and "reasoning by inheritance" tasks and evaluate popular llms to highlight the lack of consistent representation across languages. this systemic gap manifests in inefficient learning, limited knowledge sharing, and over-reliance on extensive data and computational resources, pointing to the need for better representations. we also perform controlled experiments to identify the influencing factors and propose core to bridge the gap between distant language representations which leads to 4.7% boost in performance. we hope our work spurs further research on gaining richer understanding of llmbased reasoning across languages.