GAMIC : Graph-Aligned Molecular In-context Learning for Molecule Analysis via LLM s (2025)

Abstract: In-context learning (ICL) effectively conditions large language models (LLMs) for molecular tasks, such as property prediction and molecule captioning, by embedding carefully selected demonstration examples into the input prompt. This approach eliminates the computational overhead of extensive pre-training and fine-tuning. However, current prompt retrieval methods for molecular tasks rely on molecule feature similarity, such as Morgan fingerprints, which do not adequately capture the global molecular and atom-binding relationships. As a result, these methods fail to represent the full complexity of molecular structures during inference. Moreover, medium-sized LLMs, which offer simpler deployment requirements in specialized systems, have remained largely unexplored in the molecular ICL literature. To address these gaps, we propose a self-supervised learning technique, GAMIC (Graph-Aligned Molecular In-Context learning), which aligns global molecular structures, represented by graph neural networks (GNNs), with textual captions (descriptions) while leveraging local feature similarity through Morgan fingerprints. In addition, we introduce a Maximum Marginal Relevance (MMR) based diversity heuristic during retrieval to optimize input prompt demonstration samples. Our experimental findings using diverse benchmark datasets show GAMIC outperforms simple Morgan-based ICL retrieval methods across all tasks by up to 45%. Our code is available at: https://github.com/aliwister/mol-icl.

## introduction molecular representation and analysis field has significantly advanced towards specialized pre-trained language models like chemberta @xcite , and molt5 @xcite . through targeted pre-training and task-specific fine-tuning, researchers have achieved state-of-theart (sota) results in molecular property prediction @xcite , molecule captioning @xcite , and yield prediction @xcite . nonetheless, recent developments in large language models (llms) have demonstrated remarkable capabilities in prediction tasks through incontext learning (icl) @xcite , potentially offering a more efficient alternative to the computationally expensive pre-train and finetune paradigm. generally, in molecule captioning or property prediction tasks using llms, icl retrieves molecules with similar captions or properties for a given molecule and uses these retrieved examples as in-context demonstrations @xcite . these demonstrations provide crucial guidance to help the llm make more accurate predictions. while this approach can improve prediction accuracy, its effectiveness largely depends on the relevance and diversity of the selected examples @xcite . however, the effectiveness of icl remains underexplored in molecular tasks, particularly for medium-sized llms (< 10b) @xcite , such as mistral-7b @xcite . recently, researchers have introduced morgan fingerprint-based methods, such as scaffold @xcite , for icl demonstration selection, which utilizes the similarity of the morgan fingerprint between the test sample and the demonstration pool @xcite . although scaffold outperforms random selection, its reliance on morgan fingerprints only constrains its ability to retrieve structurally similar samples for icl, as morgan fingerprints cannot fully encode the complex binding relationships that are better represented by molecular graphs @xcite . thus, capturing the graph structure is crucial for molecular analysis because it preserves atoms' spatial and connectivity information. this detailed representation is partic-ularly important for molecular similarity retrieval, where subtle structural variations can significantly impact chemical behavior. this raises a natural question: can we combine the graph representation of the molecule with the morgan fingerprint to further enhance icl effectiveness by capturing both local properties (captured in the morgan fingerprint) and global molecular structures (represented by a graph)? to explore this possibility, a leading approach is to leverage graph neural networks (gnns) @xcite , which are the sota method for processing molecular graphs @xcite . however, applying gnns in molecular similarity retrieval presents several challenges. in particular, (i) gnn encoding struggles to convert complex discrete molecular structures into continuous latent spaces while preserving chemical validity @xcite , i.e. complexity challenge; (ii) gnn learning on multimodal datasets, such as pubchem @xcite , is susceptible to information loss due to the significant gap between graph and text representations @xcite , i.e., modality gap; (iii) public datasets describe molecules in various ways, ranging from concise single-sentence descriptions to detailed multisentence explanations that capture very specific details @xcite , i.e., dataset limitations, which further exacerbates the modality gap. to address these challenges, we propose gamic (graph-aligned molecular in-context learning), a novel icl method that leverages the inherent graph structure of molecules and their local molecular features for multimodal graph-language training. in particular, gamic processes the molecular representation using a hierarchical graph encoder and aligns the latent representation with their scientifically-aware (e.g., scibert @xcite ) embedded textual descriptions using a sampling method based on morgan fingerprint similarity. incorporating morgan fingerprints as a preliminary step in selecting alignment pairs helps narrow the modality gap by providing a robust and interpretable measure of local molecular similarity during multimodal alignment training. in addition, using scientifically-aware textual embedding enriches the latent space representation of the encoded graph post-alignment, mitigating the complexity challenge. finally, having multiple potential textual representations for a molecule provides a more robust solution to address dataset limitations and mitigate inherent differences in the way captions describe molecules (for example, some molecules are described in multiple sentences and others in a few words). moreover, to further enhance icl retrieval, we introduce a novel diversity-aware sample selection method using maximum marginal relevance (mmr) to optimize the information provided in the input prompt. our main contributions are: (i) we propose a novel multimodal icl framework for molecular tasks using graph molecular features grounded on morgan fingerprint-based sampling; (ii) we propose an mmr-based demonstration selection heuristic to enhance sample diversity; and (iii) we conduct comprehensive experimental evaluations that demonstrates the effectiveness of our framework. ## methodology in this section, we first present the problem definition, then provide an overview of the proposed gamic, followed by a detailed description of its components. ## experiments in this section, we conduct experiments to verify the effectiveness of our proposed framework. in particular, our aim is to answer the following research questions: (rq1) molecular performance analysis: how does the performance of icl with gamic compare to other icl methods on molecular analysis tasks? (rq2) sensitivity analysis: how sensitive is gamic w.r.t to the number of demonstrations? (rq3) ablation study: how does each gamic component contribute to the framework? ## conclusion in this work, we demonstrate the potential of icl in improving llm performance on molecular tasks. we focus on medium-sized llms (7-10b parameters) due to their lower computational costs and ease of deployment in real-world applications. while previous work has considered either morgan similarity (scaffold) or graph-based similarity in isolation, we present gamic, which effectively combines the merits of both approaches. morgan fingerprints encode molecular substructures, while graphs capture the complex interactions between individual elements. combining both approaches provides a more holistic representation of the molecule. our experiments demonstrate that our method achieves sota on 26 of 27 tests performed across three molecular tasks.