DLIR : Spherical Adaptation for Cross-Lingual Knowledge Transfer of Sociological Concepts Alignment (2025)

Abstract: Cross-lingual alignment of nuanced sociological concepts is crucial for comparative cross-cultural research, harmonising longitudinal studies, and leveraging knowledge from social science taxonomies (e.g., ELSST). However, aligning these concepts is challenging due to cultural context-dependency, linguistic variation, and data scarcity, particularly for low-resource languages. Existing methods often fail to capture domain-specific subtleties or require extensive parallel data. Grounded in a Vector Decomposition Hypothesis—positing separable domain and language components within embeddings, supported by observed language-pair specific geometric structures—we propose DLIR (Dual-Branch LoRA for Invariant Representation). DLIR employs parallel Low-Rank Adaptation (LoRA) branches: one captures core sociological semantics (trained primarily on English data structured by the ELSST hierarchy), while the other learns language invariance by counteracting specific language perturbations. These perturbations are modeled by Gaussian Mixture Models (GMMs) fitted on minimal parallel concept data using spherical geometry. DLIR significantly outperforms strong baselines on cross-lingual sociological concept retrieval across 10 languages. Demonstrating powerful zero-shot knowledge transfer, English-trained DLIR substantially surpasses target-language (French/German) LoRA fine-tuning even in monolingual tasks. DLIR learns disentangled, language-robust representations, advancing resource-efficient multilingual understanding and enabling reliable cross-lingual comparison of sociological constructs.

## introduction cross-lingual comprehension of specialized domains is a long-standing challenge in natural language processing, particularly for sociological texts, where concepts are culturally bound and sensitive to contextual cues. for example, terms such as "unemployment," "social inequality," and "economic stagnation" can have significantly different cross-cultural interpretations despite similar terminology. multilingual pre-trained language models have advanced the state of the art in cross-lingual tasks but display significant limitations in capturing domain-specific nuances. even when text segments appear lexically similar, they may invoke different theoretical frameworks in varying socio-political contexts. for instance, consider an english-spanish pair: "income inequality reflects structural economic factors" and "la desigualdad de ingresos refleja factores económicos estructurales." although lexically aligned, one text may emphasize market-driven disparities, while the other refers to class-related structures regulated by the state. this divergence highlights the inadequacy of conventional similarity metrics for specialized applications. the situation worsens in low-resource languages that lack large-scale, domain-specific corpora, making fine-tuning or extensive parallel training impractical. basic alignment methods overlook crucial conceptual distinctions, particularly within complex sociological frameworks where hierarchical and interlinked concepts create a specialized knowledge graph @xcite . to address these limitations, we propose an approach grounded in a vector decomposition hypothesis: embeddings of domain-specific texts in high-resource languages can be decomposed into a domain knowledge vector and a language-specific feature vector. we suggest that a representation a includes a core domain component (sociological semantics) and a language-specific component. by learning perturbation vectors on a unit hypersphere from a small set of parallel concept pairs, we transform high-resource embeddings into approxi-mations of low-resource counterparts, preserving domain-level semantics. we implement this hypothesis using a dualbranch low-rank adaptation (lora) design, with one branch capturing domain knowledge and the other introducing language perturbations. spherical geometric operations yield perturbations that realign high-resource embeddings with lowresource counterparts. our contributions are: (i) positing and providing empirical evidence for vector decomposition hypothesis, demonstrating effective disentanglement of domain and language features with limited parallel data; (ii) introducing spherical noise injection, utilizing logarithmic and exponential mappings on the unit hypersphere with gaussian mixture modeling for cross-lingual alignment; (iii) developing a dual-branch lora mechanism to effectively separate domain knowledge from languagespecific perturbations; and (iv) showing through rigorous experimentation on sociological corpora that our method surpasses standard baselines in cross-lingual concept alignment. as illustrated in figure 1 , the proposed dlir framework offers a resource-efficient and linguistically grounded solution to align sociological concepts between languages, particularly benefiting low-resource and specialized domains foot_0 . ## related work multilingual text embeddings. early research on multilingual embeddings often focuses on removing language-specific artifacts to obtain unified representations. for instance, @xcite extract language-agnostic embeddings by filtering out language-specific signals, @xcite combine translation and masked language modeling objectives to reduce parallel data requirements. more recent models adopt largescale contrastive pre-training to enhance crosslingual alignment: multilingual e5 @xcite leverages a billion multilingual text pairs, nv-embed @xcite removes causal masks to improve encoder-based embeddings, and m3-embedding @xcite extends retrieval functionality across more than one hundred languages. although these efforts produce state-ofthe-art performance in general multilingual tasks, fine-grained alignment of specialized concepts (e.g., sociological or biomedical terminologies) remains a persistent challenge. progress in lora. to address computational bottlenecks in adapting large language models, a variety of parameter-efficient fine-tuning (peft) methods have emerged, with lora serving as a prominent example. building on its low-rank decomposition approach, recent work has explored new dimensions of lora. @xcite dynamically allocate rank across layers, loretta @xcite employs tensor-train decomposition, and dora ( liu et al., 2024) cross-lingual knowledge transfer. techniques for cross-lingual alignment range from classical geometric mappings @xcite to more recent strategies that adjust pretrained models using parallel corpora or dictionaries @xcite . some approaches incorporate alignment signals in model pre-training @xcite , while others merge adapters for distinct languages and tasks @xcite . prompt-tuning has also been applied to multilingual dialogues @xcite . although these methods demonstrate promising zero-shot capabilities, they often rely on large bilingual resources or fail to isolate the conceptual domain from underlying linguistic variations. in contrast, our proposed framework unifies dual-branch lora with spherical noise modeling to achieve domain-sensitive concept alignment using only minimal parallel data. this design explicitly disentangles domain knowledge from language perturbations, addressing key limitations in existing cross-lingual solutions. ## methodology this section details dlir (dual-branch lora for invariant representation), targeting effective cross-lingual knowledge transfer in specialized domains like sociology, where concept alignment is challenged by nuance and cultural context. dlir achieves robust transfer by explicitly disentangling domain semantics from language features. this approach is grounded in the vector decomposition hypothesis-that embeddings contain separable domain and language components, empirically supported via gaussian mixture model (gmm) analysis of parallel concept differences on a hypersphere. dlir implements this using a dual-branch lora architecture: one branch captures domain structure (primarily from english), while the other, guided by gmms, counteracts language variations. this yields language-invariant domain representations trained via a composite objective combining domain ranking and spherical invariance learning. ## experiments in this section, we empirically evaluate the effectiveness of our proposed dlir. we detail the dataset derived from sociological resources, the baseline models and ablation studies conducted for comparison, the evaluation tasks and metrics employed, and the specific implementation settings. ## conclusion this paper introduces dlir, a novel framework for aligning nuanced sociological concepts crosslingually, directly addressing the challenge of semantic heterogeneity prevalent in longitudinal sociological studies. such studies, including national surveys from diverse institutions and temporal periods, often see identical concepts manifesting through varied expressions, creating significant barriers to robust longitudinal and cross-sectional analyses. our work is particularly pertinent given that longitudinal population and cross-sectional studies form the backbone of empirical research in the social, economic, and behavioural sciences, as well as in epidemiology and health research, providing the basis for evidence-based policy advice. the architecture of dlir, grounded in the vector decomposition hypothesis, effectively disentangles domain-specific semantics from languagespecific features. by modeling language perturbations with gaussian mixture models (gmms) on spherical geometry using minimal parallel data, it learns language-invariant representations. this is crucial for overcoming the scarcity of manually tagged multilingual concepts, especially in low-resource languages, which currently renders many cross-national study comparisons unviable. our experiments demonstrate dlir's state-ofthe-art performance in cross-lingual sociological concept retrieval and its remarkable ability for knowledge transfer to new languages, even outperforming models fine-tuned directly on target language data. this suggests that dlir can be a significant step change in how natural language processing techniques are applied to comparative social science. by enabling more reliable alignment of sociological constructs across linguistic and cultural divides, dlir offers a promising, parameterefficient approach that can serve as an enabler for the automated harmonisation of longitudinal surveys. while acknowledging limitations such as gmm fitting with sparse data and reliance on synthetic text for core training, dlir significantly advances cross-lingual understanding in culturally rich fields like sociology. future work will focus on enhancing robustness and extending the framework to other specialized domains, further paving the way for more reliable and scalable multilingual nlp applications in interdisciplinary research contexts. in essence, dlir contributes to the recontextualization of natural language processing by providing tools that can directly support and enhance established methodologies within the social sciences, facilitating deeper and more comparable cross-cultural insights.