XTRA : Cross-Lingual Topic Modeling with Topic and Representation Alignments (2025)

Abstract: Cross-lingual topic modeling aims to uncover shared semantic themes across languages. Several methods have been proposed to address this problem, leveraging both traditional and neural approaches. While previous methods have achieved some improvements in topic diversity, they often struggle to ensure high topic coherence and consistent alignment across languages. We propose XTRA (Cross-Lingual Topic Modeling with Topic and Representation Alignments), a novel framework that unifies Bag-of-Words modeling with multilingual embeddings. XTRA introduces two core components: (1) representation alignment, aligning document-topic distributions via contrastive learning in a shared semantic space; and (2) topic alignment, projecting topic-word distributions into the same space to enforce cross-lingual consistency. This dual mechanism enables XTRA to learn topics that are interpretable (coherent and diverse) and well-aligned across languages. Experiments on multilingual corpora confirm that XTRA significantly outperforms strong baselines in topic coherence, diversity, and alignment quality.

## introduction identifying latent thematic structures within large text corpora is a central goal in computational linguistics, with topic modeling (tm) serving as a foundational technique @xcite . extending this capability to the multilingual setting has led to the development of cross-lingual topic modeling (cltm), which aims to uncover shared latent topics across languages despite lexical and structural differences @xcite @xcite @xcite . cltm models are essential for bridging cultural and linguistic divides, providing a means to understand and compare shared narratives as well as distinct global perspectives. as illustrated in figure 1 , cross-lingual topic models aim to uncover semantically similar themes across languages, such as english and chinese word clusters that both describe the concept of music. this illustrates how cross-lingual topic models can discover thematic overlap despite significant differences in surface forms. despite significant progress, developing robust and broadly applicable cross-lingual topic models (cltms) continues to pose substantial challenges. dictionary-based approaches remain attractive among the most common strategies due to their simplicity and interpretability. however, these methods are fundamentally limited by the coverage and quality of bilingual lexical resources, which are often insufficient, especially when dealing with low-resource languages or domain-specific vocabularies. this low-coverage problem has been consistently observed across multiple prior works @xcite @xcite , where the lack of comprehensive term mappings hampers the alignment of topics across languages and reduces the overall semantic fidelity of the model @xcite . in addition to lexical sparsity, these cltms often suffer from repetitive topic collapse, where multiple topics converge toward semantically similar word distributions @xcite . this redundancy undermines topic diversity and interpretability, particularly in multilingual settings where fine-grained distinctions are critical. advanced neural topic models (e.g., infoctm @xcite ) address redundancy and topic collapse by applying contrastive objectives on the decoder's topic-word distributions (β). these methods increase topic diversity and yield more coherent topics within each language. yet two limitations remain. first, they often rely on languagespecific encoders or independent parameterization, which restricts scalability and leaves β only loosely aligned across languages. more critically, most cross-lingual topic models @xcite ) neglect the refinement of document-topic proportions (θ) across languages. this omission is consequential: without cross-lingual consistency at the θ level, topic mixtures cannot be compared reliably, weakening semantic alignment across languages. as observed in to address these challenges, we propose xtra (cross-lingual topic modeling with topic and representation alignments), a unified framework designed for robust cross-lingual topic discovery that moves beyond complex language-specific encoders. instead, xtra utilizes lightweight mlps to project bag-of-words (bow) inputs into a shared space, where a common encoder effectively captures both semantic structure and crucial crosslingual alignment signals. one of the key ideas of our model is a clustering-guided contrastive learning objective that refines document-topic distributions. by clustering documents from different languages into semantically coherent groups, xtra identifies latent cross-lingual themes. a contrastive loss then pulls together documents with similar thematic content irrespective of language while pushing apart those with dissimilar themes, fostering topic distributions that are both discriminative and consistent across languages. in parallel, xtra introduces a novel approach to align topic-word distributions across languages. instead of relying on vocabulary matching or dictionary lookups, we learn to project topic-word distributions into a shared semantic space using trainable transformation layers. in this space, a second contrastive objective brings semantically equivalent topics across languages closer together. this dual-contrastive design allows xtra to learn both high-quality document-topic structures and semantically aligned topic-word meanings without any parallel supervision. our main contributions are: ## conclusion we propose xtra, a cross-lingual topic modeling framework integrating bag-of-words with multilingual embeddings via a dual-alignment mechanism. xtra aligns document-topic distributions through contrastive learning and projects topic-word distributions into a shared semantic space, enhancing cross-lingual consistency beyond lexical matching. experiments show xtra outperforms baselines in topic coherence, diversity, and alignment, demonstrating its efficacy for reliable and interpretable cross-lingual theme discovery.