MV - CLAM : Multi-View Molecular Interpretation with Cross-Modal Projection via Language Model (2025)

Abstract: Deciphering molecular meaning in chemistry and biomedicine depends on context — a capability that large language models (LLMs) can enhance by aligning molecular structures with language. However, existing molecule-text models ignore complementary information in different molecular views and rely on single-view representations, limiting molecule structural understanding. Moreover, naïve multi-view alignment strategies face two challenges: (1) the aligned spaces differ across views due to inconsistent molecule-text mappings, and (2) existing loss objectives fail to preserve complementary information necessary for finegrained alignment. To enhance LLM’s ability to understand molecular structure, we propose MV-CLAM, a novel framework that aligns multi-view molecular representations into a unified textual space using a multi-querying transformer (MQ-Former). Our approach ensures cross-view consistency while the proposed token-level contrastive loss preserves diverse molecular features across textual queries. MV-CLAM enhances molecular reasoning, improving retrieval and captioning accuracy. The source code of MV-CLAM is available in https://github.com/sumin124/mv-clam.

## introduction a profound contextual understanding of both molecular structures and biomedical text is crucial in chemistry and biomedicine. for large language models to capture these relationships, finegrained alignment between textual and molecular representations is required to harness their highcontext reasoning ability. in vision-language models, researchers have moved beyond coarse imagetext matching toward precise region-word alignment, ensuring detailed semantic correspondence between textual descriptions and visual features @xcite . recent studies * these authors contributed equally to the work have leveraged large language models (llms) for molecular understanding by integrating sequential representations (1d smiles strings) and structural features (2d molecular graphs and 3d conformers) @xcite . this approach mitigates the inherent limitations of llms which are primarily trained on textual data, that lacks native reasoning over molecular structures. to enable llms to further understand molecule information, q-former based models @xcite align molecular structures into text space (figure 2b ). combining multi-view molecular features simultaneously is essential, as their complementary nature provides a more complete understanding of molecular characteristics as illustrated in figure 1 . for example, as shown in figure 1a , 2d molecular ## mv-clam mv-clam provides molecule captions given multi-view structural information. 2d and 3d molecular structural information is extracted from specialized encoders and processed through mq-former's cross-attention layers to update learnable query tokens for each dimension. the shared selfattention layer enables information sharing across all modalities. 2d and 3d queries are combined to create a universal query, which is trained with our modified multi-objective loss for fine-grained alignment with textual descriptions. the learned universal query is then passed with the prompt and smiles strings to the language model for caption generation. the overall framework of mv-clam shown in figure 2c is comprised of three main components: (1) molecule structural graph encoders for 2d and 3d molecular structures, (2) mq-former as a cross-modal projector, and (3) llama2 as the language model. ## training mv-clam the training of mv-clam consists of two stages. ## conclusion in this paper, we introduce mv-clam equipped with mq-former, a novel cross-modal projector. the essence of this cross-modal projection lies in aligning both 2d and 3d molecular representation spaces simultaneously to enrich the textual space of language models. our architecture with its multitoken contrasting loss successfully retains complementary information from multiple dimension into a single universal token easily interpreted by large language models. extensive experiments demonstrate that mv-clam excels in molecule-text retrieval and molecule captioning tasks, with potential for broader applications, being a fine-tuned llm with expertise in molecular structure.