Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Authors: Michael Sun
Weize Yuan
Gang Liu
Wojciech Matusik
Jie Chen
Venue: ICML 2025 poster (2025)
Keywords: molecular grammar
interpretability
foundation model
multi-modal
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: Multi-modal Foundation Models can induce molecular graph languages with high generation quality, domain-specificity and built-in interpretability.

Abstract: Recent data-efficient molecular generation approaches exploit graph grammars to introduce interpretability into the generative models. However, grammar learning therein relies on expert annotation or unreliable heuristics for algorithmic inference. We propose Foundation Molecular Grammar (FMG), which leverages multi-modal foundation models (MMFMs) to induce an interpretable molecular language. By exploiting the chemical knowledge of an MMFM, FMG renders molecules as images, describes them as text, and aligns information across modalities using prompt learning. FMG can be used as a drop-in replacement for the prior grammar learning approaches in molecular generation and property prediction. We show that FMG not only excels in synthesizability, diversity, and data efficiency but also offers built-in chemical interpretability for automated molecular discovery workflows. Code is available at https://github.com/shiningsunnyday/induction.

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Michael Sun
Weize Yuan
Gang Liu
Wojeiech Matusik Jie Chen
Abstract
ended direction is to leverage MMFMs for this task, but their potential for advanced chemistry reasoning has been limited to simple (predominantly text-based) tasks (Guo et al., 2023; Bran et al., 2023) and remains to be validated by experts.
Recent data-efficient molecular generation ap- proaches exploit graph grammars to introduce interpretability into the generative models. How- ever, grammar learning therein relies on expert annotation unreliable heuristies for algorith- mic inference. We propose Foundation Molecular Grammar (FMG), which leverages multi-modal foundation models (MMFMs) to induce an inter- pretable molecular language. By exploiting the chemical knowledge of an MMFM, FMG ren- ders molecules as images, describes them as text, and aligns information across modalities using prompt learning. FMG can be used drop-in replacement for the prior grammar learning ap- proaches in molecular generation and property prediction. We show that FMG not only excels in synthesizability. diversity, and data efficiency but also offers built-in chemical interpretability for automated molecular discovery workflows. Code is available at https: /ggt: com/ bbiiningsummyllay/nnuucttoo
Recent success of Large Language Models (LLMs) on natu- ral language has begun to permeate the molecular domain. Using the formal languages SMILES (Weininger, 1988) and SELFIES (Krenn et al., 2020). transformer-base decoder architectures have been used to directly generate molec- ular strings (Bagal et al., 2021). Encoder-decoder archi- tectures have been adopted for translating between natural language and formal language (Edwards al., 2022). The text-molecule translation task has been reformulated for in-context learning tasks for chatbots like ChatGPT (Guo et al., 2023; Li et al., 2024). These efforts have also enabled the creation of instruction datasets (Fang et al., 2023; Liu et al., 2024a) that enable further finetuming for custom tasks. When combined with abundant textual data, using natural language as the surrounding context also enables masked language modeling pretraining (Christofidellis al., 2023; Liu et al., 2023c; Born & Manica, 2023).
Instead of modeling molecular graphs as strings, special- ized graph encoders have been adopted for enhanced cross- modality pretraining (Su et al., 2022; Luo et al., 2023; Liu al., 2023b; 2024a;b), leading to downstream tasks such as text-based structure retrieval (Liu al., 2023a) and structure-based design through interactive conversation (Zeng al., 2024). However, graph encoders do not come with the same advanced features of foundation models like understanding and reasoning. Aware of these challenges, our work explores an unconventional solution: rendering molecules as images alongside self-generated textual de- scriptions, implicitly aligning the two modalities in-context. This comes ripe opportunity when cheminformaties APIs like RDKit (Landrum) are becoming prevalent enough that MMFMs are likely to have seen sufficient artifacts of the standardized rendering procedure during pretraining.
1. Introduction
Substructure- graph grammar is data-efficient ap- proach for molecular generation, offering significant ad- vantages in terms of interpretability and performance (Guo et al., 2022b;a; Sun et al., 2024) compared to directly train- ing on base molecular formal languages like SMILES and SELFIES. However, learning a graph grammar is challeng- ing, as the resulting production rules are rarely guaranteed be interpretable, e.g. containing functional groups. Inter- pretable approaches typically require expert annotation to construct the vocabulary of substructures (Sun al., 2024), which is time-intensive and resource-demandingg One open-
'MIT CSAIL 'MIT Chemistry Universityy of Notre Dame "MIT-IBM Watson AI Lab, IBM Research. Correspondence to: Michael Sun <msun415 @ii..ii.mitt.d....
MMFMs such as GPT-4o possess strong image understand- ing and reasoning capabilities when interacting with natural language. In this work, we demonstrate that these models can identify molecular substructures directly from rendered
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
molecular images. These substructures form an interpretable vocabulary that serves as the foundation for building graph grammar. While an interpretable vocabulary is essential, it is insufficient to form a complete and consistent rule set for molecular generation. To address this, we propose a novel and sound algorithmic approach for data-driven gram- mar rule induction, integrating chain-of-thouge (Wei et al., 2022) reasoning with LLM debate (Khan et al., 2024) to execute the heuristics. This method enables the model to iteratively refine its reasoning and decisions, ensuring that the resulting grammar is of high-quality and robust. We call our method Foundation Model Grammar (FMG). FMG leverages the generalist reasoning capabilities (Wei et al., 2021) of foundation models to integrate cross-modal un- derstanding into a robust framework for molecular graph generation.
et al., 2024), reasoning); and (d) the task involved (e.g.. sequence modeling (Born & Manica, 2023), cross-modality pretraining (Christofidellis et al., 2023; Liu et al., 2023b), (Liu et al., 2023c), generation (Bagal et al., 2021), transla- tion (Christofidellis et al., 2023), retrieval (Liu et al., 2023a), interaction (Zeng et al., 2024), generalist agents (Bran et al., 2023; Chaves et al., 2024)).
2.2. Learning Molecular Grammars
Recently, a number of grammar-based generative models have been created (Dai et al., 2018; Nigam et al., 2021; Krenn al.. 2020; Kajino, 2019; Jin et al., 2018; 2020; Guo al., 2022a; Sun et al., 2024), which have shown en- hanced interpretabilityy synthesizability and data-efficiency. In many cases, the grammar is written manually or created algorithmically, hindering its quality or practicality. For example, Sun et al. (2024) instead prioritizes quality and interpretability by advocating to integrate expert annotations within a graph grammar construction and learning workflow, but its quality is contingent on experts. Guo et al. (2022b) tries to optimize the graph grammar construction process in- directly by parameterizing the sampling function as a neural network, but its decisions are unexplainable. Our approach, by contrast, requires no human contingency and optimizes for the intrinsic quality of the formal language as judged by non-expert L.M agents. At the same time, its decisions are explainable. We use an innovative technique of saving the ohain-of-thought reasoning steps for creating "design narra- tives", which are both explainable artifacts of rule induction and certificates of quality for further scrutiny by humans.
Our contributions include:
Introducing the novel FMG framework that uses MMFMs to learn formal graph languages by asking it to make decisions within a hierarchical decomposition algorithm Innovative prompting mechanisms to execute the algo- rithm in a step-by-step manner with MMFM in the loop, e.g. rendering intermediate variables as annotated images Proposing LLMM-baed judge and tournament system to rank decompositions for chemical soundness and validat- ing the approach with expert ground-truth labels
Demonstrating that FMG outperforms existing state-of- the-art methods on popular molecular generation bench- marks in terms of superior data efficiency, diversity, and synthesizabiliity
Evaluating FMG's step-by-step reasoning via comprehen- sive case studies and quantitative analysis.
3. FMG Learning and Inference
FMG bridges the gap between elusive expert domain knowl- edge and emergent capabilities of MMFMs using an in- terpretable workflow backed by technical rigor, semantic flexibility, and expert validation.
FMG combines the sound framework of the clique tree decomposition algorithm with the adaptability of MMFM decision-makinn modules for molecular generation. FMG formulates grammar induction as constructing clique tree and serializes the construction into intuitive selection steps for the MMFM module to follow. In Fig. 1, we show a concrete example of acrylates. The algorithm first initial- izes most basic units the base cliques- then proceeds to hand over control to the MMFM's selection modules. The MMFM can merge base cliques to form chemically mean- ingful substructures (3.3.2 and 3.3.3), remove connections between cliques in the process of spanning tree construction (3.3.4), and finally selec a root motif to anchor the parse tree (3.3.5). The algorithm outputs a junction tree of the input molecule, where each parent-child relationship is converted to a grammar rule (Aguinaga et al., 2018), forming the min- imal specialized grammar (MSG) (Wang et al., 2024) for the input molecule. Section 3.5 explains how all MSGs of given dataset are pooled into the final grammar and Section 3.6 shows how new molecules are generated given
2. Related Works
2.1. Multi-modal Foundation Models for Molecules
The existing works are classified primarily by: (a) how molecules are represented - whether as formal languages (e.g.. SMILES, SELFIES), graphs, or more excitingly, a combination of multiple modalities (Su et al., 2022: Luo et al., 2023; Liu et al., 2023b; 2024a;b); (b) whether indi- vidual molecules are paired with natural language (e.g., captions (Edwards et al., 2022), literature context (Liu et al., 2023c)); (c) what features of FMs are used (c.g., encoder (Liu et al., 2023b), decoder (Bagal et al., 2021), encoder-decoder (Edwards et al., 2022), in-context learn- ing (Guo et al., 2023; Li et al., 2024), instruction tuning (Fang et al., 2023; Liu et al., 2024a), few-shot learning (Li
2

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Chain of Thought Narrative
Extract Base
Triangulate Clique Graph
Merge Clique Nodes
Spanning Tree Edge Selection
Root Motif Selection
Flg.
Cliques
combined is 0. and Combrinn these twa
Interaction inwolves the interaction pentral carbons atleched threa otfer minor elegrooni ellecis Therefore, importani.
Mail a highights estar fancionsa LCCDD gm an adyime The sote group canatbuute and poty respoon benousor Th pircta....
Insorgoraing ersential double
meels
alkyy
reactvily
and
will highight for you some of the distinctive fragments scrylate.... Your lask to construct the primary functional groupe the molecule. Output a single pair numbers If you think those fragments should combined, explanation why.
Analyze pairwise motif interactions within an acrylate... Tell me which interaction is MOST important and which is LEAST important... Explain your reasoning.
Analyze an acrylate's substructures.. Pick the most imporiant motif... Explain your reasoning.
Figure I. Main modules of FMG algorithm (left) we initialize base cliques using bonds and minimal rings, (left-middle) we triangulate the clique graph guarantee existence of clique tree, (middle) we prompt MMFM meaningfully merge pairs of motifs, (middle-right) we eliminate cycles in the clique graph by prompting MMFM to identify the least important interactions, (right) we prompt MMFM to select the root motif, completing the tree.
the grammar.
following properties hold: 1) For each v in GH,, there is at least node 7) T such that v E Vy, For each hyper- edge e; E E, there is exactly one node 1 T such that E, and {n E T v E V, is connected. The last property is the run- ning intersection property and is relevant during the clique tree construction phase, as it needs to be checked after each step. The Junction Tree Algorithm achieves this by finding subset E' S Ec, such that (Vc, E'c:) spanning tree of Gc. There theoretical guarantee that G is trian- gulated, there is always valid tree decomposition via the existence of elimination ordering (Koller, 2009). Choos- ing the best spanning edges Ey, is somewhat an art (Jensen & Jensen, 1994). There is the "optimal" clique tree, the one with minimal treewidth, or miny maX Vy, - but finding it is NP-hard (Arnborg et al., 1987). Instead, com- mon heuristics, such the maximum cardinality heuristic (Tarjan & Yannakakis, 1984), are used to find one close the minimal width.
3.1. Preliminaries
tue V,, 3) For each v GH, the set
Molecular Clique Graph. A base molecular hyper- graph is pair H = (VHr Er), where VH (nodes) set of bonds, and Eн (hyperedges) non-empty subsets of Vи. We follow prior work (Kajino, 2019; Guo al., 2022b) and define En {[u, w if u, v share atom) ) (u) is minimal ring). Given H, we obtain GH, the graph of H, where two nodes u, u that share a common hyperedge in Σμ are connected. If we can construct a Ge = Ec) by extracting the maximal cliques (Vc) from GH, and setting Ec to be the clique pairs sharing common node, we call Ge the molecular clique graph and denote this operation CL.IQU ( Gc. Ge forms the building blocks for further operation. For each E Vc, we use V. to denote the clique nodes of G within the clique
set of
Clique Tree Decomposition. The clique tree, also known as the junction tree, of GH a tree T, where each node r) is labeled with V,, Vir and E,, C Eu,, such that the
Hyperedge Replacement Grammar. A Hyperedge Re- placement Grammar (HRG) a tuple (N, S, P) where: a set non-terminal hyperedge labels in N (shorthand
3

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
)); i's set of terminal hyperedge labels; S N is the starting non-terminal hyperedge with label := |S|: P is set of production rules, each consisting of A E N (LHS) and hypergraph with labeled hyperedges and AA| external nodes (RHS).
asked select S or refrain from selection. That is, to select clique out of all cliques in the current Gc- Pair Selection. Given a subset of pairs, CC Vi V the MMFM is asked to select p E P or refrain from selec- tion. That is, select pair of bond cliques out of a given set of feasible pairs.
We adopt an automatic way to convert a clique tree into an HRG by interpreting the clique tree as a parse tree (Aguinaga et al., 2018), where each intermediate node V, becomes the RHS of production rule and its immediate parent and/or children are used to compute its non-terminal hyperedges and external nodes, as depicted in Fig. 2. The common bonds between V, and its parent become num- bered external nodes (black circles). These external nodes are used to "anchor" the replacement, as they also appear in the RHS. For each child of V, (if' any), non-terminal hyperedge (circles labeled N) is added and connected to the common bonds (blue).
When the context is clear, we denote the raw responses Fi (S(()) and F2(P())). We use answer extraction utility prompts to obtain the answers. These selections map to triangulation, merging, cycle removal, and root selection operations on G We can execute the full tree decomposi-
tion of a molecular clique graph, GOD) = G(T), using only these operations, driven by MMFM's selections. We will describe each operation G(G) + GC
G((+1) in detail, in the context constructing the clique tree in Section 3.3.
3.3. MMFM Guided Tree Decomposition
See Figure I for an illustration of the steps described in this subsection.
3.3.1. CONSTRUCTION OF CLIQUE GRAPH
PI
We initialize G to the graph of the base molecular hy- pergraph. We automatically extract the maximal cliques of G thus constructing G(6) + CLIQUE
3.3.2. TRIANGULATE CLIQUE GRAPH
Figure Example of conversion from clique tree to HRG pro- duction rules; (Left) Each node of the clique tree contains a sub- structure (red), with edges corresponding to shared bonds between substructures; (Right-top) Rule extracted from second clique of the tree, with a non-teree con-terminal hyperedge for the LHS and the clique's substructure being the RHS; (Right-boltom) example of applying the rule, dashed connections are corresponding bonds and atoms
We now triangulate G/I ensure the soundness of the junction tree algorithm. We adopt chordality testing algo- rithm (Tarjan & Yannakakis, 1984) that iteratively detects pairs (u, v) E VH x Vи that would form chordless cycles of length > 3 if left unaddressed. At each iteration t. in which the algorithm returns a pair (u, v) connected by a chord, we set p(t) - {(11, 02) C1 E V, n E V.). F2(((().. We then merge ct c²1, by adding all edges, E((+1) + E(0 UV x Ve We update G(+1) CLIQUE Let o(fi) denote the clique graph once G is triangulated. We proceed to the next phase.
Let c" c=,,²²
3.2. Role of MMFM
For inducing desirable grammar for molecular genera- tion, a combination of hard and soft constraints must be in place. The hard constraints ensure soundness of the tree decomposition, while soft constraints seek the "optimal" decomposition among all valid decompositions, that best captures the specific characteristics of the data, with the gold standard being a domain expert. The essence our ap- proach is to automatically handle the hard constraints within sound algorithmic framework, then leave the exercises of judgment to an MMFM. To modularize the MMFM's involvement, we standardize all tasks to be the task of se- lecting among finite set of choices. All MMFM tasks in Section 3.3 can be captured by only two fundamental selections:
3.3.3. MERGE CLIQUE NODES
We now would like to give the MMFM the option to further merge cliques that form more cohesive motifs, e.g. func- tional groups, in the context of the base molecule. Starting
with t Tı, we set p(t) + ((t) If (2((()) does not return, we terminate and proceed to the next phase. Oth- erwise, at cach iteration, we let c²2,""2 + (P()). We merge c² c²1,c"2 following the same operation steps as in Step 2. Let G denote the clique graph upon termination of this phase.
Single Selection, Given a set S C V(!1 the MMFM is

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
3.3.4. SPANNING TREE EDGE ELIMINATION
subsequent task prompts (example: "Motif 9. This motif is another carboeyclic structure, specifically bicyelic system with carbon double bonds... + "a bicyclic carbocyclic structure with carbon double bonds").
We now extract a spanning tree over (a using top-down approach of detecting and eliminating cycles of G(T2)_ We
terminate and proceed to the next phase once there are no
more cycles. Otherwise at each step t, let CI. C2..... Ck. Cj
Task Prompts. These are the primary prompts that instruct GPT to perform the selection tasks. We substitute rephrased dynamic descriptions of individual cliques (motifs) where appropriate into these templates and specifically instruct GPT to explain its reasoning. Example walkthroughs featur- ing all the task prompts are given in App. H.
be one such cycle. We set p(t) f {(G, ((+1)%)) removing Ci2 C441 will not violate running intersection ,
2, ......................... We then update Ee Eff+1) + EE (P()))
Let d( denote the clique tree once all cycles have been removed.
Answer Extraction Prompts. We use low-level utility prompts to postprocess an answer prompt into a fixed format for regex extraction (example: "After extensive deliberation, the interaction between Motif 5 and Motif 7 seems weakest of the ones shown" + "5,7")
3.3.5. ROOT MOTIF SELECTION
Lastly, we root (TTa) at Fi (V G (T= T3 1). We obtain the multi-set of production rules using this decomposition, P(G
(T))). The final clique tree is
Thought Collection Prompts. We collect GPT's responses into summarized reasons for particular selection, as they will be composed into a narrative (more in Section 3.5). For a particular selection at time t, let COT ) be the prompt chaining composition to return a summarized rea- soning over the selection. We denote the output COT(4).
3.4. Prompting Setup
For each selection, we prompt the MMFM with rendered images from the rdkit and dynamical textual descriptions related to the current state of the decomposition (Gc), in ad- dition to the static prompt, which includes some background on the domain and detailed task instructions.
3.5. FMG Inference
Rendering Images. For single selection (root motif se- lection), we use the Python package rdkit to render the molecule and highlight the bonds (Ve) of single sub- structure (c E s(()) S v(()) in a cell. We use mat- plotlib.yyplot to enact a grid cell layout so all choices are shown together. For double selection where the number of choices small (edge selection), we highlight each pair (e1,02 E p()) C V x ve") using different colors in the same cell. For double selection where the number of choices are large (merging cliques), we render each clique in separate cell. just like with single selection, but the task instruction is to select a pair of cliques.
LLM-based Feedback Mechanism. Our MMFM-guided algorithm is inherently stochastic, as repeated runs may pro- duce different decompositions. In the absence of human experts, it's difficult to judge how "good" the rules produced by each decomposition are. The key challenge is that, given only the grammar rules, absent of any context, it is difficult to evaluate its semantic qualities. We channel the built-in explainability of our approach into a clever solution. We reuse the natural language artifacts (e.g. chain of thought, explanations) logged during execution of our algorithm as a certificate of the quality of that single decomposition. With this in mind, we adopt simple yet effective learning proce- dure to optimize the FMG. We first perform K passes (i.e., independent runs of the algorithm) over, the molecule H, producing decompositions [Go, k = O,..., K 1]. De- T - 1] as the chain of thoughts for the k'th pass over molecule H, we combine it with knowledge of the timestep delimiters Ti, 2, T3 to compose a stsp-b story of how the molecule was decomposed. Instead of evaluating rules in isolation, we group rules into specialized grammars, defined as the set of rules used in the tree decomposition of a single example. The resulting story provides much needed context for an LLM to assist in the evaluation.
Dynamic Textual Descriptions. Motivated by the suc- cess of prompt-based learning techniques, we assist GPT's reasoning during selection tasks by plugging in isolated de- scriptions of each element of or P into the task prompt, enabling multi-modal alignment. These are obtained by rendering each substructure (or pair of substructures) in iso- lation and asking GPT to describe those. An example of isolated description is "Motif 5. Benzene- A six-membered aromatic ring entirely consisting of carbon atoms", whereas an in-context description is "Motif 5. A six-membered ring, similar to benzene, but includes distinct locations for double bonds from Motif 1."
noting [COT,5) = O....,
L.M--based Tournaments. We use a tournament- bassd evaluation system by pitting stories of discrepant decom- positions for comparison by a non-expert LLM. We pit dis- crepant runs (A and B) for the same molecule against each
Rephrasing Prompts. We then use format conversion prompts to convert GPT's sometimes claborative answers into simple phrases that can be grammatically inserted into
5

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Specialized Grammars
Domain
LHS
= RMS
0 OHOH
Grammar
& *o Taato rgee simple Memiiictioon aadk
B
andatioora interee ireeccooo aog canos... Agaet parandl performed dellveiigg and tokai reaporses..
For rad tokk. did well but occesinally lacked ruanco somu lineo ressed alteril es per cosspollics... intepectuions inolleng sopattinalll
dlaar
Generated Samples
whl
quew
Tourranenn
Thak Prompte Selcct la molls
Choose paa moll...... mest---stille inteadion...
Mall Dese: an sabe
Maxt becruser
Tchcce Noel D opceusss
funclional Dinncyy proflle
group
Deacription Procepts
2...
Contribulia sigraincantlly martivlly and polymetriiio
Wrichhstory Lotter?
Sacorhi the mosiemeett trancioo...
Design Story
Design Story 2
O O
Usilit Pramepts
Narrrraa thi of
&
To design 2e0 corykilos modeouute
group
O
X
design it's ocrylttss crocouls haad we combined wit group cha with Errog carbono becalss
Extred aransr..
"Neet combinee w-wh an group becouse
Figure 3. Our workflow takes input class sseasifcccificc dataset and a collection of prompts (left); executes the tree decomposition algorithm with MMFM as a decision-makinn module (left middle); converts the parse tree into production rule set (left-right), resolving discrepancy across runs with non-expert LLM judge; and infers grammar which can generate new class-specific samples (right).
other and ask the vanilla LLM to decide which story wins (A or B) on the basis of validity, soundness, and perceived depth of understanding. We adopt Swiss tournament for- mat, with 4 rounds, and use the logits of the first token in the response to assign weighted outcomes of the matchup, similar to how Khan et al. (2024) designed the preference model. We consolidate all outcomes using the Bradley- Terry model (Bradley & Terry, 1952), a statistical model used for paired comparisons, where each debater's ability is inferred from the pairwise results. We rank and order
4. Results
Baselines.
We evaluate our method against a variety of similar and alternative approaches.
Grammar Generative Models (MHG (Kajino, 2019), DEG (Guo et al., 2022b), Random Walk (RW) (Sun al., 2024), STONED (Nigam et al., 2021)): How does MMFM com- pare with algorithmic and expert-based annotation? Grammar/Scaffold based Molecular VAEs (JT-VAE (Jin et al., 2018), Hier- VAE (Jin et al., 2020). MoLeR (Maziarz et al., 2021)): How does grammar- based generation fare against neural parametrizations that still consider motifs? Chemical Language Models (CLMs) (MoIGPT (Bagal et al., 2021), MolT5 (Edwards et al., 2022), Text+Chem T5 (Christofidellis et al., 2023)): How do LLMs trained on large databases of molecular strings transfer or adapt to domain sppciifc settings?
the participants [0, I,.., K- -1] permute (2.. .k] ac- cording to the tournament results and define the "Top k"' FMG as the HRG inferred by the production rule multi-set Ur P(Gc.). where Uis is the multiset union.
3.6. Stochastic Sampling for Molecular Generation
So far, we have only considered the contribution to the HRG by decomposing single molecule, H. In the domain- specific setting, we are given a small dataset of class-specific molecules, which convert into our base molecular hy- pergraphs: 'D- = ( < i < N), to create a set of specialized grammars Pb = (P(C)e)) Instead of naively pooling all grammars together, we maintain a count for the number of times each rule is applied, aggregated across Pp. During generation, the algorithm finds all applicable rules and chooses one with probability proportional to its count to apply, according to the definition in Drewes et al. (1997). We adopt Kajino (2019)'s 's technique to ensure valid conversion from hypergraph to molecule.
In Context Learning (ICL): How does directly prompting an LLM, in the absence of any formal framework, do?
More details each baseline is in App. B.
Metrics. We use common unconditional generation metrics adopted by molecular generative models (Polykovskiy et al., 2020):
Valid/Unique/Novelty (percent of valid/uniq/novel mols)
Diversity (avg. pairwise Tanimoto distance (Rogers
Hahn, 2010))
Retro Score (RS) (success rate of Retro* model (Chen
et al., 2020)
Membership (percentage of molecules belonging to the dataset's 's chemical class)'.
'We generate 10000 for small datasets and 1000 for
6

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Datasets. We evaluate on three small monomer datasets used by Guo et al. (2022b) curated from literature, as well as two real-world datasets from the photovoltaic and toxi- cology domains used by Sun et al. (2024).
prehension of ICL with our formal framework, we implicitly capture the knowledge into the selections while ensuring validity and robust performance.
However, FMG still leaves some to be desired across cover- age. Our investigation reveals that the learning procedure is inclined towards forming cliques representing more com- plex substructures that are characteristic of the chemical class or known to be synthetically accessible. The applica- bility of rule decreases as the RHS becomes more complex, and so the grammar's s coverage decreases. We suspect that the low diversity is due to this phenomenon occurring in the extreme setting of having R 30 or fewer samples, as that
Table 1. Results small datasets Isocyanates (II examples), Acry- lates (32) and Chain Extenders (11). Best rounded result(s) for each metric bolded. (T): Training from scratch; (FT): Fine-tuning, (I): Inference, using posterior interpolation. Since T5 (!) meth- ods struggle generate sufficient valid and unique samples, exclude them from ranking.
Yall/Agg)
Usigus
RS
The Bat
1274
1375
330%
2005
0.51
0.90
1374
2005
1085
1095
1005
3375
des wotth
1375
1399
1375
E33
99.60
3.25
1933
0:31
0.3
8.31
1.55
4.5
18%6
3,09
123
06.5%
0068
73.99
13.00
creates fewer rules which are less applicable.
1546
$5.00
1.8%
4906
socat imn
1375
01325
4175
0.50
0.3
8:11
6525
177%
7016
71.3%
71.5
$7.79
STOURD
1395
189%
0959
8950
0:5
081
8.
1.86
11.76
4.56
6.00
200
1875
380
34%
439%
5124
50.7%
5259
Table 2. Results on World Datasets HOPV (316 examples) and PTC (348). Same protocol Table Since VAE (T) methods struggle to generate sufficient unique samples, we exclude them from ranking.
536
116
81
03:
07
0106
3:16
1029
1976
1375
1805
380
086
0.3
045
0.5
0.76
139
2015
19.6
1885
1555
Results on Small Datasets, We first observe in Tables 1 and 2 that VAE methods struggle to generate unique molecules, suggesting they collapse in this extreme setting, consis- tent with findings by Guo et al. (2022b); Sun et al. (2024). Hier-VAE fares better, as it incorporates inductive bias of larger substructures, but this comes at the expense of RS and Memb., suggesting an undesirable shift in distribution.
Uricse
Naedty
DH.
Tran Dete
100%
100%
100%
NVA
NiA
08.55
01:4
51%
675
100%
30%
ZEMABLD
1009
111
855
1037
14.73
19%0
54%
27%
respest
1005
1865
100%
555
16:00
1622
625
7155
60%
315
Mctall
1804
105
545
D05
1680
2010
795
4%%
DRG
1006
100
180%
0.85
586
25%
GPE4 (K3))
12%
55%
54%
995
1133
45%
5655
33%
19%
MaGGPT (T)
700
555
016
71%
47%
0.81
0.88
17%
91%
65/
10S
Mar- (1)
510
175
705
0.81
0.77
91.
05
Char
45%
51%
026
0.87
4366
88%
EMGG-lie
1009
186/
1605
0085
1198
65%
775
37%
Mcanwhile, the CLMs struggle with task comprehension, sometimes mixing natural language with SMILES, or vio- lating SMILES syntax, e.g. forgetting to close parentheses. The few valid examples are often repetitive or similar (as low as <1% % uniqueness and <0.6 Div.), despite efforts to find balanced decoding strategies like varying beam width or adjusting temperature. To their credit, the few valid and unique examples achieve high RS and Memb., suggesting that the LMs may have memorized a few relevant examples in the domain but failed to generalize.
Results on Real-World Datasets. We see similar relative trends across the baselines in Table 2. VAE (T) methods still suffer from poor data-efficiency, but MoLeR (FT) becomes quite competitive. However, it still struggles on RS and Memb. The low uniqueness and novelty of the VAE base- lines invalidates its seemingly high RS score, achieved by sampling smaller molecules. Meanwhile, FMG keeps high RS scores while lifting coverage to be far more reasonable, as the size of the dataset becomes larger. FMG is one of only two methods that achieve 100% uniqueness (the other heing RW with access to expert annotations) while tied for first and second on diversity for HOPV and PTC, respectively.
The grammar- based methods do better on validity and cover- age, but struggle across RS and Memb.. Despite optimizing for RS and Div., DEG still falls short of FMG. This indicates that these methods can learn the syntax of the underlying language, but not its semantics like synthesizability or mem- bership criteria, signs of expert-level understanding. FMG's high RS scores are hence impressive considering that we only prompted GPT to "highlight the primary functional groups of the molecule". FMG also achieves nearly 100% class membership in Table 1. In fact, ICL also achieves high RS and Memb., suggesting MMFMs are sufficiently knowl- edgeable about these niche chemical classes that it can to some extent learn the molecular generation task in-context. However, ICL has similar struggles as CLMs in generating valid and unique examples. By combining the semantic com-
CLM baselines exhibit similar struggles and perform far worse HOPV, likely due to the photovoltaic domain shifting further out-of -itt-f--distribution compared to the drug- pervasive pretraining distribution, e.g. ChEMBL. Amongst grammar- based methods, FMG surpasses even RW on RS (by 12% and 18%), suggesting that FMG is more amenable to synthesis considerations even for larger, more hand- engineered molecules. Though membership not strictly defined for these two domains. FMG appears to do excep- tionally well for PTC (halides) but poor for HOPV (thio- phenes), which is surprising. As we see later in App. E, k imposes a sharp tradeoff between Memb. and (Div., RS), though FMG is capable of achieving exceptional numbers for either/or. Our results suggest that domain general FMs.
HOPV/PTC, use the same Retro parameters and adopt the same memhership criteria as Guo et al. (2022b); Sun et al. (2024).
7

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Table 3. Snapshot of PTC expert case study (App. H.1). Full case studies for all datasets in App. H. Response
Prompt
Expert Evaluation
"Meilf gi substituted carbon chain group (N-C) and nitrile (N=C) middle, *MMoed 9tte...
Lach motif was cornectly cosporized with idcutity deseribel connectivity, reflecting selid understancing organic chemistry principles... -..
you explain, concisely. num- bered motil
... tell chould combined gether more meaningfal substmuc- luse...
"PPir: and ge- "Lxplanatonn"" Comhining these could provile comprehen- nt groups along citical in analyzing the smmoture and reactivity of the sulccule...
andertandling
arangement chain,
Maliun, PTC molesales cumples, quirs consdeeing combitations more than pair of mutifs
of shony
of bug
want explzin, condisely. wha each numberee
*Moil ge+; Butsnodininile nitile with CN group each of butane backbone.
-- From motis lised, *Motff 17 (Chlorouecty- kmne*** *MMiif (1,) Dichoroethane con- sain halide groups ... Givea the need select one motif. **Moef 18 (1.) Dichloroehaee) stands out slightly more due to the presence of two chlocine atomn....
want you only these us the mor monif most semtall to its chemical profile...
choose as miee motlf, which coneet. becmue Dichlocsethane chiain higher significancy doe to preseuce of two chiicine atous.
17 gbbyy
are already aligned with chemistry-specific desiderata like synthesizability and specificity, promoting the intrinsic qual- ity of the grammar. We include visualizations and further interpretation of the results in App. C.
Behavior on Small Subset of a Large Benchmark. For a bonus challenge, we evaluate FMG's behavior when run on a tiny (0.05%) subset of the MOSES benchmark (Polykovskiy et al., 2020). We generated 30k samples and computed standard metries. To our surprise, we find FMG leads on all unconditional generation metrics, over leaderboard methods trained on 100% of data, but lags behind on distribution metrics. The results and discussion are in App. G.
Expert Evaluation of Interpretability. One of FMG's main advantages is its interpretabilityy. The algorithm exe- cution yields step-by-step reasoning steps for decomposing each molecule. Table 3 shows a snapshot from our five in-depth step-by-step walkthroughs in App. H.1...5. Each reasoning step is scrutinized by an expert and the input prompt classified for its difficulty. We tallied all the steps across the five case studies by difficulty. The expert com- pletely agrees with FMG in majority of cases and partially agree in all cases. We report the full findings in App. H.6.
5. Discussion
5.1. Ablation Study on LLM-based Tournament
Setup. We quantify the gains from L.L.-b tournaments in a more controlled setting. We set K = 10 and study the performance of Top-k FMG as A: increases from 1 to K. As baseline, we compare with the "1-k" FMG, which is the HRG inferred by U P(Gc,). Intuitively, Top-k FMG selects the top i ranked runs per molecule, while 1-k FMG selects the first k. In both cases, the final grammar pools together N specialized grammars from individual runs.
Expert Agreement with LLM Judge. FMG's step-by by- reasoning steps are not only milestones of interpretability; they directly influence the final grammar by closing the feed- back loop. Our LLLd-a----uudge protocol pits discrepant decompositions and their "design stories" against each other (Sec. 3.5) in tournament setting. To validate this protocol, we prepare some ground-truth pairwise outcomes. We ask both the expert and LLM judge to decide which decom- position is more chemically sound, solely based on their design stories. The results are in App. D, with the takeaway being: against random coin toss baseline, the MMFM's agreement with the expert yields p 1.le 5, indicating statistically significant alignment with expert judgement. In the next study, we quantify the downstream effect of refining the grammar by choosing higher-ranked decompositions.
Results. We visualize Top-k FMG and 1-k FMG on small datasets in Fig. 4. As the number of rule-set candidates k increases, we find sharp trade-off in class membership versus coverage: at low & (esp. k 1), membership reaches nearly 100%. but it steadily declines as sub-optimal rules are added. Meanwhile, increasing k improves both diver- sity and RS - pooling simple rules enables generation of more varied, easily synthesized molecules - but the cost of motif inclusion. Notably, there's 's no significant differ-
8

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Acyyano
Oiah Ecorders
on real-world datasets and additional background on why FMG defaults to images over text are in App. F.
5.3. Ablation Study on Heuristic vs MMFM Modules
Table 4. For each module, we swap MMFM for a heuristic (-
module). We report the "1-k" FMG (FMG Union) and avg. across
A runs (FMG Avg) on the 3 small datasets. We choose lc = 5.
1150
Tund Lowe
01.4
Figure 4. We vary & from 1-10 for FMG and 2-10 for FMG-Text (see Sec. 5.2). Full FMG results in App. E. Full comparison between FMG and FMG-Text in App. F.3
Setup. We separately ablate each MMFM-assisted module to investigate how crucial each module is for FMG. We ablate the merge module by directly passing G.T) to Step 3.3.4, the spanning tree module by the maximal spanning tree (MST) heuristic (Tarjan & Yannakakis, 1984). and the root module by picking a root clique at random. Ablating any LLM module breaks the overall design story, no longer rank top k in earnest, hence we omit Top-k FMG.
ence between Top-k versus 1-k FMG for diversity and RS, suggesting tournament rankings reflect understanding of class-specific motifs more and is neutral to broader gener- ative criteria. Results on real-world datasets and further analysis are in App. E.
can
Results. In Table 4, we see ablating any MMFM com- ponent has negative implications for the results, albeit in different ways and differently across datasets. Ablating Merge discourages the class-defining motifs for acrylates and chain extenders to be formed during decomposition, meaning they are less likely captured in the rules (with an exception for isocyanates, whose defining motif (N=C=O) has only 2 bonds and must already be in a clique). For isocyanates, RS drops significantly: it's known an amine (R-NH2) reacts with phosgene (COC12) to produce the iso- cyanate, so without MMFM's knowledge, the intermediate may not be formed, producing rules less amenable to syn- thetic considerations. Ablating MMFM-guided spanning tree construction has milder consequences. Diversity, RS, and membership are only slightly worse. The MST heuristic is well-motivated, but its rule-based selection is less adapt- able to dppainccpecific constraints like chemical reactivity, since models interaction strength solely on the basis of neighborhood overlap. Meanwhile, an MMFM is more flex- ible to capture these constraints, selectively breaking the heuristics when context necessitates it.
5.2. Ablation Study on Image vs Text Input Format
We also add results in Tables 1 & from a controlled study where we modify FMG to use a text-based encoding (FMG- Text) instead of molecular images to test our hypothesis that images are superior to text as the input format.
Setup. Each FMG input is a grid of cells showing (1) a molecule, (2) a molecule with one substructure, or (3) a molecule with two substructures highlighted in different colors. While (l) can be replaced by SMILES, (2) and (3) require visual emphasis of substructures, which text- based formats struggle to express. We use a verbose but reliable approach: number all atoms and list motif atoms. This improved parsing and allowed us to fully re-implement FMG with text inputs (exG- by swapping each visual cell for its text-based counterpart. We replicated the Sec. 5.3 study for FMG-Text too, and the aggregate results on small datasets are shown in Fig. 4 (Text).
Results. FMG-Text is competitive alternative when im- ages cannot be used. Barring FMG, FMG-Text leads in 13/24 columns of Tables & 2. FMG with image inputs still performs best overall - especially on diversity and synthesiz- ability, which are crucial for practical molecule generation. We identify tradeoff for Isocyanates & Acrylates that is not present for Chain Extenders. Text inputs can make small motif identification (membership) easier and more robust to larger k. Image inputs better support global substruc- ture reasoning, which drives higher synthesizabilitt. Chain Extenders are more linear, so the choice of text or image representation may be perceived equally by FMG. Results
6. Conclusion
We present a MMFM-guided grammar induction frame- work for molecular generation. FMG with GPT-4o as the base model already demonstrates expert-approved reason- ing abilities by applying its chemistry knowledge, image comprehension, and in-context abilities within our guided tree decomposition algorithm. We use innovative prompt- ing and feedback mechanisms to ensure the workflow is interpretable, user- frienfien customizable, and (dare we say) foundational for future molecular design workflows.

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Acknowledgements
Dai, H., Tian, Y., Dai, B., Skiena, S., and Song, L. Syntax- directed variational autoencoder for structured data. arXiv preprint 10XXvv:802.0881 2018.
Michael and Gang completed internships at the MIT-IBM Watson AI Lab. Gang supported by the IBM Fellowship.
Drewes, F., Kreowski, H.-J., and Habel, A. Hyperedge replacement graph grammars. Handbook Of Graph Grammars And Computing By Graph Transformation: Volume 1: Foundations, pp. 95-162. World Scientific, 1997.
Impact Statement
This paper presents work whose goal is to advance molec- ular discovery workflows. The application of our method can have consequences for real-world discovery workflows. There are no ethical aspects which we foresee and feel must be discussed here.
Edwards, C., Lai, T., Ros, K., Honke, G., Cho, K., and Ji, H. Translation between molecules and natural language. arXiv preprint
2022.
References
Fang. Y., Liang, X., Zhang. N., Liu, K., Huang, R., Chen,
Z., Fan, X., and Chen, H. Mol-instructions: A large- scale biomolecular instruction dataset for large language models. arXiv preprint arXiv:2306.0888 2023.
Aguinaga, S., Chiang, D., and Weninger, T. Learning hyper- edge replacement grammars for graph generation. IEEE transactions pattern analysis and machine intelligence, 41(3):625-638 2018.
Guo, M., Shou, W., Makatura, L., Erps. T., Foshey, M., and Matusik, W. Polygrammar: grammar for digital polymer representation and generation. Advanced Science, 9(23); 2101864, 2022a.
Arnborg, S., Corneil, D. G., and Proskurowski, A. Com- plexity of finding embeddings in ak-tree. SIAM Journal on Algebraic Discrete Methods, 8(2):277-284, 1987.
Guo, M., Thost, V., Li, B., Das, P., Chen, J., and Matusik, W. Data-efficient graph grammar learning for molecular generation. arXiv preprint arXiv:2203.088 080031 2022b.
Bagal, V. Aggarwal, R., Vinod, P., and Priyakumar, U. D. Molgpt: molecular generation using transformer- decoder model. Journal of Chemical Information and Modeling, (2(9):2064-20777 2021.
Guo, T., Nan, B., Liang, Z., Guo, Z., Chawla, N., Wiest, O., Zhang, X., et al. What can large language models do in chemistry? a comprehensive benchmark on eight tasks. Advances in Neural Information Processing Systems, 36: 59662-59685 2023.
Born, J. and Manica, M. Regression transformer enables concurrent sequence regression and generation for molec- ular language modelling. Nature Machine Intelligence, 5 (4):432 444, 2023.
Bradley, R. A. and Terry, M. E. Rank analysis of incom- plete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-34) 1952.
Jensen, F. V. and Jensen, F. Optimal junction trees. In Un- certainty Artificial Intelligence, pp. 360-366. Elsevier, 1994.
Bran, A. M., Cox, S., Schilter, O., Baldassari, C., White, A. D., and Schwaller, Chemerow: Augmenting large- language models with chemistry tools. arXiv preprint 2334::2300.053366, 2023.
Jin, W., Barzilay, R., and Jaakkola, T. Junction tree vari- ational autoencoder for molecular graph generation. In International conference on machine learning, pp. 2323- 2332. PMLR, 2018
Chaves, J. M. Z., Wang, E., Tu, T., Vaishnav, D., Lee, B., Mahdavi, S. S., Semturs, C., Fleet, D., Natarajan, V., and Azizi, S. Tx-llm: A large language model for therapeutics. arXiv preprint 0rXiv:2406.06316, 2024.
Jin, W., Barzilay, R., and Jaakkola, T. Hierarchical gen- eration of molecular graphs using structural motifs. In International conference on machine learning, pp. 4839- 4848. PMLR, 2020.
Chen, B., Li, C., Dai, H., and Song, L. Retro": learning retrosynthetic planning with neural guided a* search. In International conference on machine learning, pp. 1608- 1616. PMLR, 2020.
Kajino, H. Molecular hypergraph grammar with its appli- cation to molecular optimization. In International Con- ference on Machine Learning. pp. 3183-3191. PMLR, 2019.
Christofidellis, D., Giannone, G., Born, J., Winther, O., Laino, T., and Manica, M. Unifying molecular and textual representations via multi-task language modelling. In International Conference on Machine Learning, pp. 6140- 6157. PMLR, 2023.
Khan, A., Hughes, J., Valentine, D., Ruis. I.., Sachan, K., Radhakrishnan, A., Grefenstette, E., Bowman, S. R., Rocktäschel, T., and Perez, E. Debating with more persua- sive 1lms leads to more truthful answers. arXiv preprint arX6v:2402.07782, 2024.
10

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Koller, D. Probabilistic graphical models: Principles and techniques, 2009.
Polykovskiy, D., Zhebrak, A., Sanchez-Lengelingg B., Golo- vanov, S., Tatanov, O., Belyaev, S., Kurbanov, R., Arta- monov, A., Aladinskiy, V., Veselov, M., et al. Molecular sets (moses): a benchmarking platform for molecular gen- eration models. Frontiers in pharmacology, 11:565644, 2020.
Krenn, M., Häse, F., Nigam, A., Friederich, P., and Aspuru- Guzik, A. Self-referencing embedded strings (selfies): A 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020.
Rogers, D. and Hahn, M. Extended-commectiityy finger- prints. Journal of chemical information and modeling, 50 (5):442-754, 2010.
Landrum, G. Rdkit:
Open-source cheminformatics. URL http:// //ww:..//ww../// rdkit.os org.
Li, J., Liu, Y., Fan, W., Wei, X.-Y., Liu, H., Tang, J., and Li, Q. Empowering molecule discovery for molecule- caption translation with large language models: A chatgpt perspective. IEEE Transactions on Knowledge and Data Engineering. 2024.
Su, B., Du, D., Yang, Z., Zhou, Y., Li, Rao, A., Sun, H., Lu, Z., and Wen, J.-R. A molecular multimodal foun- dation model associating molecule graphs with natural language, arXiv preprint rrXiv:2209.0540) 2022.
Sun, M., Guo, M., Yuan, W. Thost, V., Owens, C. E., Grosz, A. F. Selvan, S., Zhou, K., Mohiuddin, H., Pedretti, B. J., et al. Representing molecules as random walks over in- terpretable grammars. arXiv preprint 4rX1v:2403.08147, 2024.
Liu, G., Sun, M., Matusik, W., Jiang, M., and Chen, Multimodal large language models for inverse molecu- lar design with retrosynthetic planning. arXiv preprint arXiv:2410.0422 10.04233, 2024a.
Tarjan, R. E. and Yannakakis, M. Simple linear-time al- gorithms to test chordality graphs, test acyclicity of hypergraphs, and selectively reduce acyelic hypergraphs. SIAM Journal computing, 13(3):566 5795 1984.
Liu, P. Ren, Y.. Tao, J., and Ren, Z. Git-mol: multi-modal large language model for molecular science with graph, image, and text. Computers in biology and medicine, 171: 108073, 2024b.
Wang. B., Wang, Z., Wang, X., Cao. Y., Saurous, R., and Kim, Y. Grammar prompting for Nomain-specifet lan- guage generation with large language models. Advances in Neural Information Processing Systems. 36, 2024.
Liu, S., Nie, W., Wang, C., Lu, J., Qiao, Z., Liu, L.,, Tang. J., Xiao, C., and Anandkumar, A. Multi-modal molecule structure-ters model for text-based retrieval and editing. Nature Machine Intelligence, 5(12): 144-11447,4 2023a.
Wang, Z., Chen, Y., Ma, P., Yu, Z., Wang, J., Liu, Y., Ye, X., Sakurai, T., and Zeng, X. Image-based generation for molecule design with sketchmol. Nature Machine Intelligence, pp. 1-12, 2025.
Liu, Z., Li, S., Luo, Y., Fei, H., Cao, Y., Kawaguchi, K., Wang, X., and Chua, T.-S. Molca: Molecular graph- language modeling with cross-modat projector and uni- modal adapter. arXiv preprint arXiv:2310. /279988....2 2023b.
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. Finetuned lan- guage models are zero-shot learners. arXiv preprint arXi::2109.01652, 2021.
Liu, Z., Zhang, W. Xia, Y., WU, L., Xie, S., Qin, T., Zhang, M., and Liu, T.-Y. Molxpt: Wrapping molecules with text for generative pre-training. arXiv preprint arXiv:2305. 106888 2023c.
Wei, J., Wang. X., Schuurmans, D., Bosma, M., Xia, F., Chi, et al. Chain-of- of-thhu prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
E., Le, Q. V., Zhou, D.,
Luo, Y.. Yang, K., Hong, M.. Liu, X. Y., and Nie, Z. Molfm: A multimodal molecular foundation model. arXiv preprint arXhv:2307.0948/ 2023.
Maziarz, K., Jackson-Flux, H., Cameron, P., Sirockin, F., Schneider, N., Stiefl, N., Segler, M., and Brockschmidt, M. Learning to extend molecular scaffolds with structural motifs. arXiv preprint arXiv:2103.00 033644 2021.
Weininger, D. Smiles, chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-33))) 1988.
Nigam, A., Pollice, R., Krenn, M., dos Passos Gomes, G., and Aspuru-Guzik, A. Beyond generative models: su- perfast traversal, optimization, novelty, exploration and discovery (stoned) algorithm for molecules using selfies. Chemical science, 12(20):70797709) 2021.
White, A. D., Hocky, G. M., Gandhi, H. A., Ansari, M., Cox, S., Wellawatte, G. P., Sasmal, S., Yang, Z., Liu, K., Singh, Y., et al. Assessment of chemistry knowledge in large language models that generate code. Digital Discovery, 2(2):368-37) 2023.

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Zeng, Z., Yin, B., Wang, S., Liu, J., Yang, C., Yao, H., Sun, X., Sun, M., Xie, G., and Liu, Z. Chatmol: interactive
molecular discovery with natural language. Bioinformat- ics, 3094)) 2024.
12

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
A. All About Prompts
The prompts consumed by the MMFM is instantiated from a generic template. Each generic template has both static substitutions, e-g. hand-written descriptions of specific domain, and dynamic substitutions, e-g. motifss from the grammar induction. We denote static substitutions within and dynamic substitutions within <>. As a simple example, the template "Design a molecule containing <>." at runtime. In Table 5, we provide the generic templates, example of static substitution (we use PTC as the running example for static substitutions), and example of a dynamic substitution for how we include bhain-of-tthought In Table 6,
" [] could be class (e.g. photovoltaic) and <> could be specifie motif, populated
we provide templates used for FMG inference, using a similar format as the first table.
B. Baseline Settings
For VAEs, define (T), (FT) and (I) settings as follows:
Training (T): We train VAE from scratch.
FT (FT): We leverage any pretrained checkpoints released by the authors, where possible. We do 80-20 train-val split of the dataset and finetune until the validation loss converges. We select the checkpoint with the lowest validation loss for sampling.
Inference (I): For our Small Datasets, there are as few as 11 samples, making (FT) extremely difficult. We instead adapt pretrained checkpoints to sample in the posterior distribution of the dataset. Specifically, we fit Gaussian distribution to the encoded latent vectors from our dataset. We then sample latent vectors from the fitted Gaussian distribution decode into molecules.
1. JT-VAE: We train with default hyperparameters until convergence, following the instructions on public repository. We sample from Gaussian prior.
2. HierVAE: We follow the preprocessing steps to obtain a vocabulary from the full dataset. We then train with default hyperparameters. As recommended, we train an unsupervised language model on the full dataset until convergence, then sample from a Gaussian prior. Note that a train-test split not possible due to the vocabulary differing between splits. For the (+expert) setting, we use the expert motifs curated by Sun et al. (2024).
MoLeR (I): We load the publicly released pretrained checkpoint and do posterior interpolation to sample from our data distribution. Specifically,
fit Gaussian distribution by taking the mean be the average of the latent codes from encoding the samples in our dataset and estimating the full covariance matrix. We use sklearn wwwammmiittur.Gaa.m GaussianMixttue'sss implementation.
4. MoLeR (FT): We finetune the pretrained checkpoint on our real-world datasets. We do the recommended 80-20 train-valid split, where metadata between splits are synced. We finetune until validation loss converges. We use all the default hyperparameters.
5. MolGPT: We use a batch size of to accomodate our smaller datasets. Since the vocabulary size is different with the pretrained models, we are unable to finetune. We use the same train, test split as MoLeR and select the checkpoint with the best validation loss.
6. GPT4-ICI: We use OpenAl API for the GPT-4o model. An example prompt for acrylates is provided:
Here are some acrylates. <examples>> Remember the defining acrylate group is C=CC(=O)O, which consists of a carbon-carbon double bond and carboxylate ester. Generate 100 more acrylates. Output one SMILES per line, beginning with the first line.
<examples>> replaced with all the SMILES strings in the small dataset, or a random subset of all the strings for the real-world datasets.
7. Chemical Language Models (MoTTS, Text+Chem T5): We implement the pretrained checkpoints "malt5-large- for MolT5 and Text+Chem T5, respectively. We prompt the models for molecular generation. We set the maximum generation length to 512. For each generation, we randomly sample five molecules from the training set to construct the prompt. The template is similar to the one employed in GPT4-ICI.
caption2smiles" and
13

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Table Reference table of all tree decomposition task prompts used. We number substitutions and show the substituted text from sample run. See text for how they are used (Section 3.5).
Prompt
Example [substitutions]
Example <substitutions>
I want you to think like chemist performing detailed analysis of the chemical composition of [1] through its constituent motifs. will highlight for you some of the distinctive fragments of molecule. They are numbered from 0 and individually highlighted in RED. Focus ONLY the substructure highlighted in red within each cell. Here is the descriptions for each substructure provided by an expert: <1> I want you to tell me if any two of them should be combined together to form a more meaningful substructure. [2] Your task is to highlight the primary functional groups of the molecule. Output a single pair of numbers if you think those two fragments should be combined, and brief explanation why. If no such pairs exist, don't output anything.
a toxic compound
Motif 0: A carbonyl group (C=O) attached to a carbon chain.
This molecule belongs to a collection of molecules characterized by distinct functional groups known for their careinogenic properties liver toxic- ity. These groups com- prise a rich variety of el- ements such as halides, alkylating agents, epox- ides, and furan rings.
Motif 1: A nitrile group
(C N) attached to tertiary carbon.
Motif 2: A di-substituted carbon chain with two adjacent nitrile groups (N=C-C-C-N).
Motif 23: Another ben- zene ring structure.
I want you to think like chemist performing detailed analysis of the chemical composition of [1] through its constituent motifs. will highlight for you some of the distinctive substructures of [1]. They are numbered from 0. Here are the textual descriptions of each motif: <1> I want you to pick only ONE of these as the root motif most essential to its chemical profile. It should be the single most important motif the rest of [1] was built around. [2]. so your selected root motif should contain those group(s). If there are multiple such motifs, or one doesn't 't clearly stand out, just pick one of them. Give your answer as a single number. Explain your reasoning carefully.
a toxic compound
Motif 0: A carbonyl group (C=O) attached to a carbon chain.
This molecule belongs collection of molecules characterized by distinet functional groups known for their carcinogenic properties or liver toxic- ity. These groups com- prise rich variety of elements, most notably halides
Motif 1: A nitrile group
(C' N) attached to a tertiary carbon.
Motif 2: A di-substituted carbon chain with two adjacent nitrile groups (N=C-C-C-N).
Motif 23: Another ben- zene ring structure.
want you to think like a chemist performing a detailed analysis of the chemical composition of [1] through its constituent motifs. This requires two steps: I) analyz- ing the individual motifs and 2) analyzing the pairwise interactions of motifs. The first step is already done. The second step is where need your help. will highlight for you different motif interactions within the same molecule. These interactions are numbered one-by-one, beginning with O. Here are the textual descriptions of each motif interaction pair. <1> want you to tell me which one of these is MOST important, and which one of these is LEAST important. Output one number identifying the MOST important, and give brief explanation. Output one number identifying the LEAST important, and a brief explanation why.
Interaction 0 features a benzene and an isobu- tyraldehyde.
Interaction features an isobutyraldehyde and chloroacetylene.
a toxic compound
Interaction 2 features a chloroacetylene and a benzene.
14

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Table 6. Reference table of prompts used to create and compare design stories (Section 3.5).
Prompt
Example [substitutions]
Example <substitutions>
toxic compound
Motif 0: A carbonyl
group (C=O) attached carbon chain.
want you to think like chemist performing detailed analysis of the chemical composition of [1] through its constituent motifs. will highlight for you specific pair of motifs, the positive motif in red and the negative motif in green. want you to explain to me the importance of this positive-negatin interaction through the perspective of designing [1]. Explain how the addition of the negative (green) motif to the positive (red) motif is justified the context of designing [2]. Keep your response single paragraph in prose.
This molecule belongs to a collection of molecules characterized by distinct functional groups known for their carcinogenic properties or liver toxic- ity. These groups com- prise a rich variety of el- ements such as halides, alkylating agents, epox- ides, and furan rings.
Motif 1: A nitrile group
(C = N) attached to a tertiary carbon.
Motif 2: A di-substituted carbon chain with two adjacent nitrile groups (N=C-C-C-N).
Motif 23: Another ben- zene ring structure.
a toxic compound
Motif 0: A carbonyl group (C=O) attached to a carbon chain.
I want you to weave the following chronological log en- tries of a chemist into a high-level story of how the chemist came up with new molecule. Summarize each of the ex- pert's individual numbered entries, keeping the rationales intact but removing any redundant, ambiguous or unnec- essary information. Remove the ambiguous mentions of positive and negative motifs. Your final output should be the same numbered list of steps, but each step is just few sentences at most.
This molecule belongs a collection of molecules characterized by distinet functional groups known for their carcinogenic properties or liver toxic- ity. These groups com- prise rich variety of el- ements such as halides, alkylating agents, epox- ides, and furan rings.
Motif 1: A nitrile group
(C = N) attached to a tertiary carbon.
Motif 2: di-substituted carbon chain with two adjacent nitrile groups (N=C-C=C-N).
Motif 23: Another ben- zene ring structure.
toxic compounds impli- cated with carcinogenic- ity for male and female rats
I want you to compare two competing analyses for de- signing the molecule <I>. Which analysis demonstrates deeper understanding of [1]? Pay attention to the fol- lowing points: Which analysis better highlights [2]?- Which analysis has better insights regarding the interac- tions amongst those motifs? Output only the answer (A or B) and nothing else. Now, here are the analyses:
CC(=D)C(N=Nelextee
2ccoNN=CCC/C)=O)CC
=0)Necccccc3))((())))
ccCCIc(=OONellccccll
the defining halide mo- tif(s) of the toxic com- pound's chemical class
15

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
C. Additional Discussion of Evaluation Criteria
We provide additional points of consideration when interpreting the results Sec. 4.
Pareto Frontier Methods on Small Datasets
Pareto Frontier Methods on Real-World Datasets
MHG (FT)
JPWNE (T)
unique
Her-VAL (1)
Mutuot (1)
Nevelty
Unique
HieHAE +expert
Molelt (1)
300eDivyy
MofTS (I)
Nist-chemm
CP-14 (CL)
(expart)
MalkPP
100HDioe
ail
R5
(a) Spider plot of pareto-efficient methods in Table
(b) Spider plot pareto-efficient methods Table
Figure 5. Visualization of results across all 5 evaluation metrics.
Holistic Assessment. Evaluating generative models is challenging and requires holistic consideration of different, competing metrics. A method which scores high on one metric (e.g. synthesizability) but does catastrophically bad on another one (e.g. uniqueness) is not practical. To facilitate a more holistic validation, we provide spider plots for Tables I & in Figures 5a & 5b. On small datasets, no method other than FMG reaches near 100% unique, valid & memb. simultaneously. For instance, GPT4 (ICL) appears score higher on diversity & RS, but struggles generate valid and unique samples. On real-world datasets, FMG generates the most unique, novel, valid & diverse samples, and methods that score higher RS and Memb. have serious shortcomings.
Ranking Criteria. In Tab. 1, the two T5 (I) methods are excluded from ranking (as mentioned in the caption) due to the difficulty in obtaining sufficient valid, unique samples, meaning their seemingly higher RS/Memb. scores don't 't have sufficient sample support. The same is written in the caption of Tab. 2 for the VAE (T) methods. When simple fixes don't 't work (e.g., tuning sampling temperature), we make a note of it in the caption and exclude it from the rankings.
Robustness. We don't report error bars because robustness across samples (the purpose of error bars) can simply be absorbed into the number of samples generate. Generating sufficiently large sample size ensure greater statistical significance of the results, especially when 4 of our metrics (valid, novel, RS, memb.) are defined at the individual sample level and the other 2 metrics (diversity, uniqueness) evaluate coverage. For small molecules (Tab. 1), we generate 10000 samples; for large molecules (Tab. 2), we generate 1000 samples due to inference being more expensive for some of the large model baselines. both cases, we see a fixed sample size 1000 or 10000 already pushes multiple baselines to the limits, so generating more than what they're capable of may further complicate a holistic assessment of the methods.
D. Expert Evaluation of LLM Judge
D.1. Setup
For given molecule, the expert compares two discrepant decompositions the basis of soundness of reasoning. We then prompt our LLM judge with chain-of-thoughe reasoning (Wei et al., 2022). We ask the LLM to summarize the key points and decide which decomp. is better, solely based on the explanations. For Chain Extenders, the expert compared all 11 molecules and provided rationale for their decision for each molecule. For the 4 other datasets, we randomly select molecules each, and ask the expert to directly provide the decision without writing a rationale due to time constraints. Each
16

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
pair evaluated twice with flipped order eliminate bias. We run the LLM judge 3 times per molecule, for total of 15 data points per dataset.
D.2. Example
Expert
"Analysis A demonstrates solid understanding of motifs like amide, urea, imidazole, and carbonyl groups sroups-key to chain extenders. It justifies motif as the root based on its role in peptide bond formation, Analysis neglects important groups and lacks a strong rationale for selecting motif 0 root."
LLM Judge
..
Defining Motifs:
Analysis better identifies functional groups like amides/carbonyls that define chain extender properties. correctly selects motif for its role in polymer backbones.
Functional Group Explanation:
Analysis explains the contributions of groups to mechanical and processing properties. Analysis simplifies the motifs and lacks depth chemical reasoning.
Decision:
Analysis A is favored for better understanding and detailed explanation of groups defining chain extenders.
D.3. Results
For Chain Extenders, the LLM judge agrees with the expert 10, times out of when the better decomposition shown last, and 8, 8, times when is shown first. For the other datasets, we see the results Table The total comparisons the LLM judge agrees with the expert 77/108 71%. With the null hypothesis being every comparison is an independent coin flip, this result yields p-value of 1.le which is statistically significant.
D.4. Analysis
The LLM judge can identify better decompositions, validating our judging protocol and the premise behind our self- improving feedback loop described in Sec. 3.5. These results reinforce the usefulness of FMG's interpretability its outputs can support downstream decision-making and design critique, even enabling the LLM to act as self-checking agent in an ixpert-in-the--r pipeline.
Table We tally LLM judge decisions against the human decision (Gold) across 3 repeated calls per molecule. For molecules (columns with answer in [brackets]), 1), the expert found both designs equally reasonable, so we exclude those from the total.
Isocyanates
Acrylates
HOPV
PTC
Total
Gold
A
B
B
A
B
A
BAAB
B
AABAB
BBABB
BBAAA
BBBBA
[А] BABA
25/36
B
ВBBAÍА
AAAAA
BBBBA
BBABA
BBBBA
BABAA
BBABB
25/36
[AAB[B]
ABBA[B]
AAAAB
BBBBB
BAAAB 9/15
BBBBB 13/15
[B
[AABBB
26/36
Score
6/12
9/12
12/15
13/15
6/12
9/12
77/108 71%
17

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
!socyanates
Acrylates
Chain Extenders
HOPV
PTC
1.0
1.00
1.0
Top k RS
0.9
Top
0.9
1-k Memb.
1-k RS
0.95
0.8
0.8
0.7
0.7
0.6
0.5
0.5
0.85
0.5
0.5
0.4
0.80
0.3
10
Figure 6. We vary & from 1-10 (small dataset) and 1-5 (rcal-world dataset) following the same settings as the main results.
E. Ablation: Ensemble Over Seeds
Results. We observe Fig. 6 sharp tradeoffs the generation metrics as increases. easy achieve near 100% membership for low values of k. This because one of the points of comparison when evaluating two discrepant design stories being, "Which analysis better highlights the defining motif(s) of the acrylates chemical class?" We can deduce that 1) for each molecule, running for sufficient number of seeds always produces some decomposition that embeds the chemical class's s defining motif within one of the rules, and 2) FMG is capable of ranking decompositions containing that property higher than those that do not. As corollary, membership drops as k increases, as rules from sub-optimal decompositions are added to the grammar. In contrast, domain -peeifispe has some intrinsic tradcoff with synthesizability. Isocyanates are known to be tricky to synthesize due to unwanted side reactions. Choosing decompositions with design stories demonstrating thorough understanding of the domain is more likely overcomplicate the grammar from a synthesizability perspective.
We also note some general trends as k increases. Diversity and RS seem to improve as rule sets combined. This is likely because a larger collection of "simple" rules, pooled from alternative decompositions, enables more simple molecules to be generated, albeit at the cost of membership. Interestingly, there are no major differences between Top k and 1-k for RS and diversity, suggesting the learning procedure targets mainly class-specific considerations, remaining neutral to more general considerations.
F. Ablation: Images vs Text
F.1. FMG-Text Choice of Text Encoding
Attempt 1: Tagged SMILES. Our first attempt to text-only input format was to add tags (e.g. <>) into SMILES to denote motif boundaries. For example:
Original SMILES: C=CCE=OJOCICCCCCCCC
Motif: C=C0=-00--C-1cc00000000-00
GPT-4o failed basic comprehension tests. Though it understands that numbers denote ring closures, it became confused about which numbers were relevant to the motif's ring. As result. it interpreted the tagged string via straightforward character concatenation e.g., CCCCC, a straight-chain alkane rather than recognizing the cyclopentane motif. We attempted to add ring numbers within the tags to clarify the motif's 's ring structure, but this only obfuscated the global context and further degraded the model's understanding.
Attempt 2: Atom-number encoding. We then arrived at our more verbose: number all atoms and list motif atoms. For example:
Original SMILES: C.A.CCCCCCS.............CCCeCeeS
18

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
(optional) Motif I: 6,7,8100,11
(optional) Motif 2: 8,9,10,11
We made sure everything worked as expected on few case studies. Then, we ran our full evaluation protocol.
E22 FMG-Text Best Of Ensemble Results
lotryarades
Acrylenes
Chwin n Extenders
11PPV
PTC
1.0
1003
0.-
04-
0.55
140
6.7
1.53
6.2
Figure 7. We vary & from 2-10 (small dataset) and 2-5 (real-world dataset) to compare FMG-Text with FMG.
The full results of varying for FMG-Text, in comparison the results in App. E, is shown in Fig. 7.
F.3. Motivation for Images Over Text
In addition to the quantitative edge FMG has over FMG-Text, we also provide additional points of consideration for choosing images as the format of representation over text.
1. Literature review of LLM's chemistry comprehension abilities. Foundation models like GPT-4o and Gemini have shown multi-modal comprehension ability in aligning natural language descriptions with corresponding images for advanced reasoning, but there has been less work on aligning formal languages like SMILES with corresponding images. This may be because LLMs find it easier to obtain semantics from natural descriptions rather than SMILES. For instance, studies like that done by White et al. (2023) (Section D) and Guo et al. (2023) find LLMs perform great on tasks that require reasoning from the natural description of molecules, but struggle when fed SMILES/SELFIES (e.g. 0% SMILEES-to-name success in the first study).
2. Technical formulation is in terms of hypergraphs. Our underlying formulation builds off the history of hyperedge- replacement grammars, which operate at the hyperedge (or substructure) level instead of the atom level. SMILES/- annotations and extending it to do so not only is out of the
SELFIES syntax doesn't 't easily support substructure- scope of this work but also changes the data used for LLM pre-training. Meanwhile, rdkit has library of functions for highlighting, color-coding, and rendering substructures. SMILES/SELFIES is also incompatible with particular detail in our formulation, which treats bonds (not atoms) as the fundamental units. Our design choice ensures that each clique's atoms (edges) are disjoint, which is essential for our grammar induction and molecule reconstruction. Because SMILES/SELFIES encodes atoms explicitly but treat bonds implicitly, there's no straightforward way to annotate bonds or define disjoint cliques. Enforcing compatibility would require taking the bond-atom dual of (1) our grammar formulation or (2) the SMILES/SELFIES syntax. This mismatch, discussed in the preliminaries, further motivates our use of images, where we can easily highlight constituent bonds.
3. Interpretability of the grammar learning process. We thought long and hard about how the complex, hierarchical structured representation of hypergraphs can be fed into LLMs. After initial conversations with chemists, we came to the conclusion that highlighting substructures is the most visually meaningful way to consume the information. We, in parallel with a few others like Wang et al. (2025), identified an opportunity for combining the latest multi-modal understanding and cheminformatics rendering tools. We hope these points sufficiently motivate the visual representation input, and a summarized discussion will be added to the main text.
19

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
G. Results on Small Subset of MOSES
Table 8. We trained FMG on a Ik subset (0.05%) of the refined ZINC dataset used by the MOSES benchmark (Polykovskiy et al., 2020).
We generated 30k samples and computed standard TestSF metries using the released splits. Baseline numbers are copied from the MOSES leaderboard. IntDiv2 is omitted due to redundancy with IntDiv.
Model
Valid(t)
Unique@ Ik(1)
Unique@ 10K(1)
FCD (estSF()))
Scaf-eestSF()) SNN-Tes(SF(1)
IniDiv(")
Novelty(1)
Train
1.00
1.00
1.00
0.48
0.59
0.00
0.86
1.00
HMM
0.08#0.03
0.62#0.12
0.57+0.14
25.43+2.56
0.38+0.01
0.05#0.02
0.85=0.04
1.0020.00
NGram
0.24+0.00
0.97=0.01
0.92=0.00
6.23+0.10
0,50=0.00
0.10#0.01
0.87=0.00
0.97=0.00
Combinatorial
1.00=0.00
1.00=0.00
0.99=0.00
4.51=0.03
0.44+0.00
0.09=0.00
0.87=0.00
0.99=0.00
CharRNN
0.97+0.03
1.00+0.00
1.0020.00
0.52+0.04
0.56+0.01
0.11+0.01
0.86#0.00
0.84+0.05
AAE
0,9420.03
1.0000.00
1.0=0..00
1.06=0.224
0.57=0.00
0.08+0.01
0.8620.00
0.7920.03
VAE
0,98+0.00
1.00+0.00
1.00+0.00
0.57+0.03
0.58+0.00
0.06+0.01
0.86#0.00
0.69&0.01
JTN-VAE
1.00=0.00
1.00=0.00
1.00=0.00
0.94
0,52+0.01
0.10#0.01
0.86#0.00
0.91
LatenGGNN
0.90=0.00
1.00=0.00
1.0000.00
0,83+0.01
0.51#0.00
0.11#0.01
0.86=0.00
0.95+0.00
FMG (0.05%)
1.00=0.00
1.00=0.00
1.00=0.00
26.30+0.41
0.29#0.00
0.12+0.00
0.90=0.00
1.00=0.00
Setup. The sole purpose of this study is completeness. We want to understand FMG's behavior on the popular MOSES benchmark used for large-scale representation learning. We trained FMG on three 1K subsets (just 0.05%) of the 1.9 million molecules in the refined ZINC dataset used by MOSES (Polykovskiy et al., 2020). This reflects an extremely data-scarce setting, so direct comparisons should not be made; we only include the leaderboard for broader context.
Results. We can from Table 8 that FMG excels all five unconditional generation metries reported in our main paper. That said, we acknowledge its distributional match is very weak, as expected from model trained on only 0.05% of the data. However, its high validity, novelty, diversity, and uniqueness demonstrate FMG's potential as a backbone model for further optimization (as done by Guo et al. (2022b)).
Discussion. We want to reiterate the pressing issues in real-world molecular discovery that FMG was designed to solve. We believe there lies a critical gap between large-scale representation learning benchmarks and real-world molecular design settings. Real-world settings often come with only a handful of training examples, because chemical classes feature more specific criteria and there are fewer experts to validate the data. Our main results in Tables 1 & 2 demonstrate FMG outperforming methods which are state-of-the-art on large-scale benchmarks. We see their performances drop significantly due to failing constraints like synthesizability and class membership. Maintaining diversity also becomes more challenging with less data. When tackling domain-specific settings, e.g. synthetically accessible chain extenders with amine groups, expert knowledge plays a critical role- - but integrating it via annotations traditionally required costly manual labor (Sun et al., 2024). FMG seeks to automate this labor with MMFMs, constructing rigorous hierarchical languages that capture explicit/implictt constraints through step-by-step algorithmic reasoning. The result is an interpretable, knowledge-driven workflow that excels at step-by-step reasoning for domain-specifie molecular design.
H. Case Studies, Expert Analyses, and Design Story
In the Appendix, we do bu---bytturn walkthroughs of representative molecules from every dataset. We show logs of every task-related (prompt, response) pair used in our algorithm. We invite a real chemist to comment on the algorithm's 's logs for each dataset and comment on a) the difficulty of the prompt and b) GPT's answers. We preface each dataset (subsection) with a brief overview of the domain and rationale for choosing the specific molecule. For the interested reader, we also include sample design story in Section H.2.1 used in learning the FMG. As the unrefined stories are heavy in jargon and quite elaborative, we only include one example. We choose a HOPV molecule as they feature the richest stories. We include the full text under HOPV's Section H.2.
In Section H.7, we leave with concluding remarks on the overall performance of GPT-4o, highlighting LLM agents' strong performance in substructure extraction and limits in harder tasks requiring expert intuition.
H.1. Case Study: Predictive Toxicology Challenge (PTC)
First, we establish some formatting conventions that we use for PTC and HOPV:
We log the task prompt, the corresponding response and the expert's comments for each call to GPT-4o, subdivided by
20

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
header cells labeling what's in the following cell.
We omit utility prompts like extracting integer answers and instead focus the selection tasks done by GPT.
Dynamic substitutions are quoted in
Pre-written background specific to the domain is italicized.
For PTC, since it often contain halides and quaternary ammonium groups, which assist in transferring reactants between different phases in a reaction, selected molecule picked not only have multiple symmetrical halide component but are also comprehensive enough for GPT studying the of functional motif's relative impact to the entire molecule, making ideal candidate for studying phase-transfer catalysis in synthetic chemistry and investigating toxicity, given that some PTCs are known carcinogens.
Figure 8. Input molecule from PTC
21

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Prompt
I want you to think like a chemist performing detailed analysis of the chemical composition of a toxic compound through its constituent motifs. I will highlight for you "224" of the substructures of a molecule. They are numbered one-by-one from Motif 0 to Motif "23"", inclusive. I want you to explain, concisely, what each numbered mo- tif Make sure to start from Motif and go in order the numbering. MAKE SURE you describe EVERY MOTIF!
1
2
3
4
5
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Response
Certainly! Here is concise description of each motif:
**Motif 0;** A carbonyl group (C=O) attached carbon chain.
**Motif 1:** A nitrile group (C = N) attached to tertiary carbon.
**Motif 2:** A di-substituted carbon chain with two adjacent nitrile groups (N=C-C=C-N).
**Motif 3;* Similar to Motif but with an additional carbon on one end.
**Motif 4:* An imine group (C=N) connected vinylic carbon chain (C=C).
**Motif 5:** Benzene ring (C6H6) six-mexiee six-membered aromatic carbon ring.
** *otif 6:** A singly unsaturated carbon chain.
**Motif 7:** A doubly unsaturated carbon chain.
**Motif 8*** Di-substituted carbon chain with nitrile group (N=C) on one end and another nitrile (N = C) in the
middle.
**Motif *:** Similar to Motif but with different placement of carbons.
**Motif 10:** An amino group (NH2) attached C=N group (imino). **Motif 11:* A ketone (C=0) with adjacent carbons.
**Motif 12:** * A cyano-ketone (C N and C=O) attached to adjacent carbons.
"*Motif 13:** An alkyl cyanide (R C N) where R is alkyl chain.
**ootif 14:** An amino group (NH2) connected to nitrile (C N).
**Motif 15:## A benzene ring (C6H6)- slightly different representation of Motif 5,
**Motif 16: ** Another representation a benzene ring (C6H6).
** **tot 17:** A nitrile group (C N) attached to a doubly unsaturated carbon chain.
** Motif *8 An alkyl halide (carbon chain with a chlorine, C-CI).
** Motif 19: * Similar Motif 18 with different number of carbons.
Motif 20: ** A doubly canned ketone (C=0) group attached to imine (C=N) group.
22

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
**Motif #1 An imine group (C=N) attached to an ethyl group (C2H5).
**Motif 22:** A doubly bonded carbon chain similar to an alkene.
**Motif 23:** Another benzene ring structure.
This list consists of a variety of functional groups and structural motifs commonly found in organic chemistry, ranging from aromatic rings to carbon-nitrogen triple bonds, and functional groups such as ketones and amines. Comments by Expert
The response thoroughly and accurately identified and described each of the 24 motifs, spanning a range of functional groups and structural components. These include highly reactive entities such as carbonyls, nitriles, and imines, as well as stable aromatic systems like benzene rings. Each motif was correctly categorized with its chemical identity and described in terms of its structure and connectivity, reflecting solid understanding of organic chemistry principles. Prompt
I want you to think like a chemist performing a detailed analysis of the chemical composition of toxic compound through its constituent motifs. will highlight for you some of the distinctive fragments of molecule. They are numbered from 0 and individually highlighted in GREEN. Focus ONLY on the substructure highlighted in green within each cell. Here is the descriptions for each substructure provided by an expert:
"'Certainly! Here is concise description of each motif:
**Motif 0 A carbonyl group (C=O) attached to a carbon chain.
**Motif 1:** A nitrile group (C = N) attached to a tertiary carbon.
**Motif * A di-substituted carbon chain with two adjacent nitrile groups (N=C-C=C-N).
**Motif 23:** Another benzene ring structure.
This list consists of a variety of functional groups and structural motifs commonly found in organic chemistry, ranging from aromatic rings to carbon- garbon-nttrogen triple bonds, and functional groups such as ketones and amines."" I want you to tell me if any two of them should be combined together to form a more meaningful substructure. This molecule belongs to a collection of molecules characterized by distinct functional groups known for their carcinogenic properties or liver toxicity. These groups comprise rich variety of elements such as halides, alkylating agents, epoxides, and furan rings. Your task to highlight the primary functional groups of the molecule. Output single pair of numbers if you think those two fragments should be combined, and brief explanation why. If no such pairs exist, don't output anything.
o
6
10
11
gridays
p
g o
fr p
12
13
14
15
16
17
0
Arbdy
o Abbo 0
boby
18
19
20
21
22
23
Arrd A
Response
**Pair: 8 and ***
23

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
**Explnnation Motifs 8 and 9 are both di-substituted carbon chains with nitrile groups in different positions. Combining these two fragments could provide more comprehensive understanding of the arrangement and connectivity of nitrile groups along the carbon chain, which critical in analyzing the structure and reactivity of the molecule, especially in the context of toxic compounds where such functional groups have significant implications for toxicity.
Comments by Expert
Medium, PTC molecules are more complex, so it requires considering combinations of more than one pair of motifs, This is because molecules like this often contain interconnected functional groups that work synergistically to influence their chemical properties, for example, liver toxicity or cancerogenetics. Prompt
want you to think like a chemist performing detailed analysis of the chemical composition toxic compound through constituent motifs. I will highlight for you "'23" the substructures a molecule. They are numbered one-by-one from Motif 0 Motif "222"" inclusive. want you explain, concisely, what each numbered motif is. Make sure to start from Motif and go in order of the numbering. MAKE SURE you describe EVERY MOTIF!
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
20
21
22
Response
Certainly! Here are the chemical motifs numbered from 0 to 22:
**Motif 0**; Acetone (propan-2-oni 2-one) A simple ketone with the formula (CH;)2CO.
**Motif **#; Acetonitrile nitrile compound where the CN group attached to a methyl group.
**Motif **# Fumaroid nitrile A dinitrile with both nitrile groups on opposite sides.
** **tif *** Cis-maleonitrile A dinitrile with both nitrile groups on the same side.
**ootif 4**; Acrylonitrile A nitrile compound with vinyl group attached to the CN.
Motif 5** Benzene An aromatic hydrocarbon with six-membered carbon ring and alternating double bonds (represented here with hexagon).
**Motif 6**; Propene (propylene) An alkene with the chemical formula C3Ha containing a double bond between two carbon atoms.
* **Motif 7**; 2-Butene alkene which has double bond between the second and third carbons of a four-carbon chain.
**Motif 8** Butanedinitrile A nitrile with CN group each end of butane backbone.
M*tif ***; Acrylonitrile derivative- Similar acrylonitrile with an additional methyl group.
24

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
**Motif 10)*:
Methacrolein An aldehyde with a carbon-carbon double bond.
**Motif 11**;
Ethyl cyanide (propanenitrile)
nitrile with a three-carbon chain. Propan-l-amine An amine with a three-carbon chain.
**Motif 12**;
**Motif 13##;
Tert-butyl A bulky, branched alkyl group with three methyl groups attached to single carbon atom.
"*Motif 14
Benzene (repetition of Motif 5) Another illustration of benzene.
**Motif 15 Benzene (repetition of Motif and 14) Yet another illustration of benzene.
**Motif 16**: Isobutyraldchyde An aldehyde with a branched three-carbon chain.
**Motif 17*#;
Chloroacetylene An alkyne with chlorine attached to a carbon-carbon triple bond.
**Motif 18**: ...--icchloroethnee- An alkane with two chlorine atoms attached to the same carbon atom.
**Motit 19*#: 2-Meethhyylppppnoyyynrreee
nitrile with methyl group attached to carbon atom that part of the
nitrile structure.
**Motif 20**; Benzene (repetition of Motifs 5, 14, and 15) Another repetition of benzene.
**Motif * Acetonitrile again (as in Motif 1) Another nitrile example.
**ootif * Benzene final repetition of the benzene structure.
This detailed description covers the unique structural motifs presented molecule, from acetone to benzene, and other functional groups like nitriles and alkenes. Prompt
I want you to think like a chemist performing detailed analysis of the chemical composition of toxic compound through its constituent motifs. will highlight for you some of the distinctive fragments of molecule. They are numbered from and individually highlighted in GREEN. Focus ONLY on the substructure highlighted in green within each cell. Here the descriptions for each substructure provided by an expert:
"'Certainly! Here the chemical motifs numbered from 0 to 22:
**Motif 0**; Acetone (propan-2-on 2-one)- A simple ketone with the formula (CHs)2CO.
Motif **; Acetonitrile A nitrile compound where the CN group is attached to methyl group.
Motif 22*#; Benzene final repetition of the benzene structure.
This detailed description covers the unique structural motifs presented in the molecule, from acetone to benzene, and other functional groups like nitriles and alkenes."'
I want you to tell me any two of them should be combined together form a more meaningful substructure. This molecule belongs collection of molecules characterized by distinct functional groups known for their carcinogenic properties liver toxicity. These groups comprise rich variety of elements such as halides, alkylating agents, epoxides, and furan rings. Your task is to highlight the primary functional groups of the molecule. Output single pair of numbers if you think those two fragments should be combined, and a brief explanation why. If no such pairs exist, don't output anything.
1
2
4
5
food
o
10
11
Aoo
of
food Foo
Foo
b
12
13
14
15
16
17
foo 3
foout
18
19
20
21
22
fobthe
p
25

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Response
**Pair: 2, 3**
**Expaanttion:** Motif 2 and Motif's 3 both depict dinitriles, with variations in their positioning (fumaroid vs. cis-maleonitrile, respectively). Combining these motifs could result in more complete representation molecule with multiple nitrile groups, which are significant regarding potential toxic and carcinogenic properties. This merged substructure can serve more insightful representation of the functional group within the overall toxic compound. Comments by Expert
This is partially correct, while combining Motifs 2 and 3 is valid due to their shared nitrile functionalityy, since Motifs 17 and 18 contain halide groups, they can also be combined. Halide-containing motifs often play significant role in the reactivity and toxicity of a compound, especially in mechanisms involving alkylating agents or electrophilic substitution reactions. Prompt
I want you to think like a chemist performing detailed analysis of the chemical composition of toxic compound through its constituent motifs. will highlight for you some of the distinctive substructures of a toxic compound. They are numbered from O. Here are the textual descriptions of each motif:
"Certainly! Here are the chemical motifs numbered from 0 to 22:
**Motif 0**; Acetone (propan-2-on) --oe)) A simple ketone with the formula (CHs)²CO..
**Motif ** Acetonitrile nitrile compound where the CN group is attached to methyl group.
**Motif 22*#; Benzene final repetition of the benzene structure.
This detailed description covers the unique structural motifs presented in the molecule, from acetone to benzene, and other functional groups like nitriles and alkenes.""
I want you to pick only ONE of these as the root motif most essential to its chemical profile. It should be the single most important motif the rest of the toxic compound was built around. This molecule belongs to a collection of molecules characterized by distinct functional groups known for their carcinogenic properties liver toxicity. These groups comprise a rich variety of elements, most notably halides, so your selected root motif MUST contain a halide group. If there are multiple such motifs, or one doesn't clearly stand out, just pick one of them. Give your answer as a single number. Explain your reasoning carefully.
4
5
tooy!
10
11
12
13
14
15
16
17
2 oor
Frody 8
food
g p grddy
18
19 F o
20 Aord
21
22
Comments by Expert
Medium level difficulty task for the language model or even for chemistry experts, pick only one of the most important motif can cause somewhat different thoughts between language models and chemistry experts, this is in particular significant when multiple motifs carry significant relevance, such as halide-containing groups. Chemists
26

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
might emphasize functional groups based on different perspectives including reactivity, toxicity mechanisms, or known biological interactions.
Response
Upon reviewing the chemical motifs, the selection criteria specify that the root motif should contain halide group, which is characteristic of the molecule's ' toxic properties.
Examining through the motifs, certain ones include halides:
**
Motif 0 Acetone does not have halide.
** Motif 1 **; Acetonitrile does not have halide.
Motif 2* Fumaroid nitrile does not have halide.
Motif 3* Cis-maleomitrile does not have halide.
Motif 4* Acrylonitrile does not have halide.
Motif 5** Benzene does not have a halide.
*8 Motif 6*#; Propene does not have a halide.
** Motif + 2-Butene does not have halide.
* Motif 8** Butanedinitrile does not have a halide.
** Motif 9**; Acrylonitrile derivative does have halide.
*8 Motif 10 Methacrolein does not have halide.
Motif 11** Ethyl cyanide does not have a halide.
Motif 12*#;
Propan-l-amine does not have a halide.
Motif 13*#:
Tert-butyl does not have a halide.
*3 Motif 14*#; Benzene repetition does not have halide.
* Motif 15**: Benzene repetition does not have halide.
** Motif 16*#;
Isobutyraldehyde does have halide.
* Motif 17
Chloroacetylene **contains h*
** Motif 18*#; 1.1-Dichloroethane **contains h
Motif 19**; 2- 2-Methyllroppgte MoMeehhylpropanoylnnrriee does not have a halide.
** Motif 20 Benzene repetition does not have a halide.
M*tof 21 Acetonitrile repetition does not have a halide.
Motif * Benzene repetition does not have halide.
From the motifs listed, **Motif 17 (Chloroacetylene)* and **Motif 18 (L.I-Dichloroethane)*** contain halide groups. Both of these could potentially serve the root due their inclusion of chlorine atom, which is halide. Given the need to select one motif, **Mot 18** (1.1-Dichloroethane) stands out slightly more due to the presence of two chlorine atoms, which could contribute to heightened toxicity and distinct chemical properties of the toxic compound.
Thus, the most essential root motif most likely to be built around is * 18**
Comments by Expert
GPT describe both motif 17 and 18, as they both contains the halide function groups GPT choose Motif 18 the most essential root motif, which is correct. this is because 1, 1.1-Dichloroethane obtain higher significancy due the presence of two chlorine atoms, which enhances its toxic potential and chemical reactivity compared to the single halide in Chloroacetylene.
Prompt
want you perform simple post-processing step of the following response:
"CCertiily!! Here are the chemical motifs numbered from 0 to 22:
**Motif 0**; Acetone (propan-2-ong --one) A simple ketone with the formula (CHs)2CO.
**Motif *** Acetonitrile A nitrile compound where the CN group is attached to methyl group.
Motif *2 Benzene final repetition of the benzene structure.
This detailed description covers the unique structural motifs presented in the molecule, from acetone to benzene, and other functional groups like nitriles and alkenes." tM The input is response from another language agent. It describes motifs numbered from Motif 0 to "'Motif 22"", inclusive! I want you to rephrase each motif description by
27

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
filling in X within the following sentence template: This motif is X Be sure to condense the description and output single PHRASE such that the sentence template is grammatically correct. Don't capitalize the first letter, since your answer should just be phrase. Output your rephrasing for each motif SEPARATE line, using only a new line for delimiting different motifs. Don't ton't output anything clse. MAKE SURE you do it for EVERY MOTIF!
Comments by Expert
The task straighfforward and requires minimal complexity, as mainly involves rephrasing the descriptions of chemical motifs into a predefined format while retaining their original meaning. This type of prompt simple to execute for anyone with fundamental chemistry knowledge.
Response
an acetone (propan-2-one)
an acetonitrile
a fumaroid nitrile
a cis-maleonitrile an acrylonitrile
benzene
propene (propylene)
2-butene
butanedinitrile
an acrylonitrile derivative
a methacrolein
an ethyl cyanide (propanenitrile)
propan-l--a- -appan---amiee l-amine
tert-butyl
benzene
benzene
an isobutyraldehyde
a chloroacetylene
a I, l.l-dichloroethane
2-methy/lrooamoyylitriiee
benzene
acetonitrile
benzene
Comments by Expert
provides accurate and concise summary of all the functional groups, ensuring that each motif is clearly identified while maintaining the integrity of the chemical descriptions. This demonstrates good understanding of the understanding for diversity of structural-funce ffrrctiofnnttinalll relationship within the molecule.
Prompt
I want you to think like chemist performing detailed analysis of the chemical composition of molecule through its constituent motifs. This requires two steps: analyzing the individual motifs and analyzing the pairwise interactions of motifs. The first step is already done. The second step is where I need your help. I will highlight for you different motif interactions within the same molecule. These interactions are numbered one-by-one, beginning with 0. Here are the textual descriptions of each motif interaction pair. Interaction "*0"" features "'a benzene" and "'an isobutyraldehydee. Interaction "'1" features "'an isobutyraldehyde" and "'a chloroacetylene".. Interaction ""2" features "'a chloroacetylene" and "'a benzene"".
I want you tell me which one of these is MOST important, and which one of these LEAST important. Output number identifying the MOST important, and give brief explanation. Output number identifying the LEAST important, and brief explanation why.
28

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
1
2
Comments by Expert
This is a hard task, as it requires abundant chemistry empirical experiences in organic synthesis, particularly in understanding how different motifs influence molecular stability, reactivity, and interactions. Factors such the electronic effects of the functional groups, steric hindrance, resonance stability, and the roles of motifs in each synthetic reaction reaction mechanisms can vary, which all contributes to the complexity of motif interactions.
Response
To determine which interaction is the most and least important, it's useful to consider the chemical properties and potential intermolecular forces involved in these interactions.
### Interaction Details: 1, #*teeractioon 0**; Benzene and Isobutyraldehyde
2. **Interaction 1** Isobutyraldchyde and Chloroacetylene
3. **Iteractioon *** Chloroacetylene and Benzene
#### Analysis: **Bnzene*** Benzene rings often participate in π π stacking interactions, increasing molecular stability and electronic distribution. sobutyraleehyy* This functional group has polar aldehyde component that can form hydrogen bonds and dipolc-dipole interactions. *Chloroacctylene*** This highly reactive group contains both an acidic hydrogen (from the acetylene) and a chloro substituent which may influence its reactivity and interaction propensity.
### Pairwise Interactions: Interaccioon 0 (Benzene and Isobutyraldehydd)** This interaction may involve weak interactions from benzene and dipole-dipole attractions from the aldehyde group.
Interaction (Isobutyraldehyde and Chloroacetylene))* The aldehyde offers dipole-dipole interactions while the chloroacetylene provides potential for bonding interactions based on its triple bond and electronegative chloro group.
** Interactioo 2 (Chloroacetylene and Benzene)**: This interaction might involve क 7 interactions from benzene and possible dipole-induced interactions from chloroacetylene.
### Most Important: ** Interaction 1 (Isobutyraldehyde and Chloroacetylene)*** Explanation: This interaction involves strong dipole-dipole interactions from the polar aldehyde group and the highly reactive chloroacetylene. The triple bond and the chlorine atom enhance the electrophilic character, making this interaction highly influential in the molecule's reactivity and stability.
### Least Important: *#Iteracction 0 (Benzene and Isobutyraldehydd))*** Explanation: While benzene can participate π - π interactions, these non-covalent interactions are generally weaker compared to strong dipole-dipole or potential covalent interactions found in other pairs. The electrostatic attraction between benzene's π- electrons and the aldehyde may not contribute as significantly the molecule's 's overall properties.
### Summary: **Most Important: Interaction |** **Least Important: Interaction 0**
Comments by Expert
This is fully correct, interaction (benzene and 1, ---iihlloroethaee ) should be most important and (acrylonitrile and benzene) should be the least in this case. This assessment aligns with the chemical reasoning that Interaction I involves significant dipole-dipole interactions due to the presence halides and benzene ring, making it crucial the molecule's 's overall properties. In contrast, Interaction 0 is less important, as the म π interactions between benzene and acrylonitrile are comparatively weaker, contributing less to molecular stability reactivity. This distinction reflects deep understanding of the underlying chemical principles governing the interactions. Prompt
I want you to think like chemist performing detailed analysis of the chemical composition of a molecule through its constituent motifs. This requires two steps: 1) analyzing the individual motifs and analyzing the pairwise interactions of motifs. The first step is already done. The second step is where I need your help. will highlight for you different motif interactions within the same molecule. These interactions are numbered one-by-one, beginning
29

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
with 0. Here are the textual descriptions of each motif interaction pair. Interaction "O"" features "'an acrylonitrile" and ""a benzene"". Interaction "11" features "'a benzene" and "I.-dicchlroothhnee.. Interaction "22" features a 1, I.--dichlorocthnee" and "'an acrylonitrile". I want you to tell me which one of these is MOST important, and which one of these is LEAST important. Output one number identifying the MOST important, and give a brief explanation. Output one number identifying the LEAST important, and a brief explanation why.
1
2
Response
M*oost Important: Interaction ***
Explanation: Interaction 1 features benzene and a 1, ..-dichloroethane. Benzene is highly stable aromatic compound, and its interactions are often significant due to its resonance stability and the potential for π л stacking interactions. 1, ..--Dichlrroethane can interact through dipole-dipole interactions due to the presence of electronegative chlorine atoms. The combination of these two motifs could provide strong intermolecular or intramolecular interactions, impacting the overall stability and properties the molecule significantly. **Least Important: Interaction ()**
Explanation: Interaction O features acrylonitrile and benzene. Acrylonitrile, while reactive due its nitrile group, may not interact as strongly with benzene compared to the involvement of more polar or reactive groups like the dichloretthane. The interaction with benzene here would primarily be through weaker van der Waals forces and does not have the significant dipole-dipole interactions present in Interaction 1.
H.2. Case Study: Harvard Organic Photovoltaic Dataset (HOPV)
For HOPV, the selected molecules include several of the most significant functional motifs that contributing to molecules' photovoltaic properties. These functional groups include the classical thiolphene unit, thiazole ring, pyrimidinedione ring and dithieno[3.2-b:22 3:tt:ao/3..bbb:..33dddhiiie s'-dJthiophene (DTT), it is really classical, comprehensive and representative HOPV molecule.
Figure Input molecule from HOPV
Prompt
I want you to think like a chemist performing a detailed analysis of the chemical composition photovoltaic molecule through its constituent motifs. will highlight for you the substructures of a molecule. They are numbered me-by-onn from Motif Motif "" 10"" inclusive. want you to explain, concisely, what each numbered motif is. Make sure start from Motif 0 and go order of the numbering. MAKE SURE you describe EVERY MOTIF!
30

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
1
2
3
4
5
6
8
9
10
Response
Here's the detailed analysis of each motif in the given photovoltaic molecule, broken down:
**Motif 0**: **Structur*** It is 1.3-dithiol-2-one structural motif. features a ketone (carbonyl group) with five-membered ring containing sulfur atoms adjacent the ketone group.
2. **Motif I**; **ttrucure*** This motif consists of two distinct substructures: 1. A five-membered aromatic
ring resembling thiophene, composed of sulfur atom (yellow) and four carbon atoms. 2. A simple three-carbon chain with double bond, ending in a ketone group (carbonyl group).
3. **Motif ** **tructure*** This motif features a set of three different heterocyclic aromatic rings: A thiophene ring (five-membered ring with sulfur). A furan ring (five-membered ring with an oxygen atom). 3. A pyrrole ring (five-membered ring with a nitrogen atom).
**Motif *** ** *tructtuee**: This is similar Motif but includes an additional carbonyl group (C=0)
branching off the central carbon in the five-membered ring structure.
5. **Motif *** Structure:**: This motif resembles stylized thioacetone structure. It consists of three-carbon chain with sulfur atom connected to the central carbon and two other carbon atoms attached via single bonds. **Motif *** ** *tructure*** This motif features an indole-like structure. It is a bicyelic system with benzene ring fused to five-membered ring containing a nitrogen atom and sulfur atom, resembling benzothiophene. **Motif 6* * This a thioketo structure. It consists of three-carbon chain with a sulfur atom double-bonded the central carbon and a carbon attached via a double bond.
8. **Motif 7**: Structure**: This motif represents nitrile ketone structure. It is three-carbon chain where one carbon is triple-bonded to nitrogen (nitrile group) and double-bonded to oxygen (carbonyl group). 9. M*tof 8*#;- **truccuue**** This structure is a simple nitrile. It consists of three-carbon chain with nitrogen atom triple-bonded to the central carbon.
10. **Motif 9**; **Structure** This motif resembles acetyl cyanide. It is a three-carbon chain with a ketone group (carbonyl group) and cyanide group (triple-bonded nitrogen). 11. **Motif 10*#. *truccture*** This is an isoprene structure. It features four-carbon chain with double bonds present, making up simple conjugated diene.
These motifs collectively illustrate the diversity of functional groups and substructures possibly present in the com- plex photovoltaic molecule. Each motif brings distinct electronic and structural properties to the table, contributing to the molecule's overall function in photovoltaic applications.
31

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Comments by Expert
This is an easy prompt with clear instructions to follow. GPT have successfully identified and accurately described the names and functionalities of all 11 motifs, demonstrating clear understanding of the molecular substructures and their relevance in the context of photovoltaic applications. Prompt
I want to think like chemist performing detailed analysis of the chemical composition of molecule through its constituent motifs. I will highlight for you some of the distinctive fragments of molecule. They are numbered from and individually highlighted in GREEN. Focus ONLY on the substructure highlighted green within each cell. Here is the descriptions for each substructure provided by an expert:
"Here's the detailed analysis of each motif in the given photovoltaic molecule, broken down:
Motif 0**:- **Structure**: It is 1 1,3-dithiol-2-one structural motif. It features ketone (carbonyl group) with five-membered ring containing sulfur atoms adjacent the ketone group.
2. **Motif ** **Structure*# This motif consists of two distinct substructures: 1. A five-membered aromatic
ring resembling thiophene, composed of sulfur atom (yellow) and four carbon atoms. 2. A simple three-carbon chain with double bond, ending in a ketone group (carbonyl group)....
want you to tell me if any two of them should be combined together to form a more meaningful substructure. This molecule belongs a collection of molecules used to aid the design of organic solar cells. These molecules comprise of functional groups and complex ring structures that contribute optimized photovoltaic efficiency, utilizing factors like Pi-orbital electron delocalization electrondonating groups/Structures increasing steric hindrance. Your task is to highlight the primary functional groups of the molecule. Output single pair of numbers if you think those two fragments should be combined, and brief explanation why. If no such pairs exist, don't 't output anything.
1
2
Xooo
4
5
6
10
Response
No two fragments should be combined together to form a more meaningful substructure from the given motifs. Each motif represents distinct functional groups or structural elements that can individually contribute to the photovoltaic properties the molecule.
Comments by Expert
Easy prompt, useful and specific instructions for GPT such as hint to utilize factors like electron delocalization, etc. GPT's answer: No two fragments should be combined together to form more meaningful substructure from the given motifs. Each motif represents distinct functional groups or structural elements that can individually contribute to the photovoltaic properties of the molecule. This is correct, motif provide the most distinct function to the molecule's optoelectronic properties, however, none of them form no meaningful substructure. They were injected before due to the ease of synthesis.
32

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Prompt
I want you to think like a chemist performing detailed analysis of the chemical composition of photovoltaic
molecule through its constituent motif'. will highlight for you some of the distinctive substructures of a photovoltaic molecule. They are numbered from O. Here are the textual descriptions of each motif:
"Here's the detailed analysis of each motif in the given photovoltaic molecule, broken down:
Motif 0**:- **Structure**: It is 1,3-dithiol-2-one structural motif. features a ketone (carbonyl group) with five-membered ring containing two sulfur atoms adjacent the ketone group.
2. **Motif I** *Structure** This motif consists of two distinct substructures: 1. A five-mebbered aromatic
ring resembling thiophene, composed of sulfur atom (yellow) and four carbon atoms. 2. A simple three-carbon chain with double bond, ending in a ketone group (carbonyl group).
I want you to pick only ONE of these as the root motif most essential its chemical profile. should be the single most important motif the rest of the molecule was built around. This molecule belongs to a collection of molecules used to aid the design of organic solar cells. These molecules comprise of functional groups and complex ring structures that contribute optimized photovoltaic efficiency. utilizing thiophene-base groups for enhanced Pi-orbital electron delocalization or electrondonating groups/structures increasing steric hindrance. Your selected rool motif MUST contain the thiophene group. If there are multiple such motifs, or one doesn't It clearly stand out, just pick one of them. Give your answer as a single number. Explain your reasoning carefully.
Response
The root motif most essential to the chemical profile of the photovoltaic molecule, which contains the thiophene group and contributes significantly to the molecule's 's function in photovoltaic applications, is **Motif 1 1** **Reasoning:** **hhiophene P:esence**: Motif contains a distinct thiophene unit, which is critical for enhanced --orbital electron delocalization. Thiophenes are well-known for their role in organic electronics and photovoltaics due to their ability to facilitate charge transport through т n-conjugation. - **Susstrutture Complexity Motif I consists of both a thiophene ring and a three-carbon chain with ketone group. This complexity aids in creating regions of electron-rich and electron-deficien areas within the molecule, which is beneficial for the formation of charge-separated states and efficient photovoltaic processes. *#Itegrration into Larger Structure***, Motif I provides a suitable backbone around which additional functional groups and motifs can be attached, enhancing the overall photovoltaic performance by balancing electronic and structural properties.
Therefore, based on the importance of the thiophene group and the structural features that support photovoltaic efficiency, Motif is chosen as the root motif.
Comments by Expert
correct, GPT answered motif which highlights distinct thiolphene unit and its significancy in photovoltaic applications. Thiophene is structural component commonly found in conducting polymers and widely utilized in photovoltaic applications due to its role in facilitating -orriital electron delocalization and charge transport. Prompt
want you to perform a simple post-processing step of the following response: "'Here's the detailed analysis of each motif in the given photovoltaic molecule, broken down:
1. **Motif 0**: **Structure** It is 1,3-dithiol-2-one structural motif. It features a ketone (carbonyl group) with five-membered ring containing two sulfur atoms adjacent to the ketone group.
2. **Motif ***;- **tructure*** This motif consists of two distinct substructures: 1. A five-membered aromatic
ring resembling thiophene, composed of a sulfur atom (yellow) and four carbon atoms. 2. A simple three-carbon chain with a double bond, ending in a ketone group (carbonyl group).
The input is response from another language agent. describes motifs numbered from Motif to Motif "'10", inclusive! want you to rephrase each motif description by filling in within the following sentence template: This motif is X Be sure to condense the description and output single PHRASE such that the sentence template is grammatically correct. Don't 't capitalize the first letter, since your answer should just be phrase. Output your rephrasing for each motif on SEPARATE line, using only new line for delimiting different motifs. Don't 't output anything else. MAKE SURE you do it for EVERY MOTIF!
33

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Response
1. 1,3-dithiol-2 -nne structural motif with ketone and five-membered ring containing two sulfur atoms
2. two distinct substructures: a thiophene-like five-membered aromatic ring and a three-carbon chain ending in ketone group
three different heterocyclic aromatic rings: thiophene, furan, and pyrrole
similar to Motif O but with an additional carbonyl group branching off the central carbon
5. stylized thioacetone structure with three-carbon chain and sulfur atom connected the central carbon
6. indole-like structure with benzene ring fused to five-mem membered ring containing nitrogen and sulfur
7. thioketo structure with three-carbon chain and sulfur atom double-honded to the central carbon
8. nitrile ketone structure with a three-carbon chain, a nitrile group, and carbonyl group
bonded to the central carbon 9. a simple nitrile with a three-carbon chain and a nitrogen atom triple-bonded
10. an acetyl cyanide-like structure with three-carbon chain, ketone group, and cyanide group
11. an isoprene structure with a four-carbon chain and two double bonds making up a conjugated diene Comments by Expert
The answer from GPT is correct, it provides concise explanations for each motif. This indicates clear understanding of the structural and functional elements of the photovoltaic molecule. Prompt
I want you think like chemist performing detailed analysis of the chemical composition of photovoltac molecule through its constituent motifs. This requires two steps: 1) analyzing the individual motifs and 2) analyzing the pairwise interactions of motifs. The first step is already done. The second step is where need your help. I will highlight for you different motif interactions within the same molecule. These interactions are numbered one-by-one, beginning with 0. Here are the textual descriptions of each motif interaction pair.
Interaction features "' 1,3-dithiol-2-one structural motif with a ketone and a five-membered ring containing two neemmmeeeeed aromatic ring and a three-carbon
sulfur atoms" and "*2. two distinet substructures: thiophene- chain ending in a ketone group." Interaction features "22. two distinct substructures: thiophene-like five- membered aromatic ring and three-carbes chain ending in ketone group"" and "33. three different heterocyclic aromatic rings: thiophene, furan, and pyrrole."" Interaction 2 features "'3, three different heterocyclic aromatic rings: thiophene, furan, and pyrrole and 1. 1.3-dithiol-2-one structural motif with a ketone"" " and "fiveemmmberre ring containing two sulfur atoms"".
I want you to tell me which one these is MOST important, and which one of these LEAST important. Output one number identifying the MOST important, and give a brief explanation. Output one number identifying the LEAST important, and brief explanation why.
1
2
Response
Given the importance of the motifs and their interactions in photovoltaic molecules, let's consider the roles played by each interaction.
**Most Important Interaction (Number 1): 2* Reasoningg:** Interaction features three heterocyclic aromatic rings: thiophene, furan, and pyrrole, interacting with a 1,3-dithiol-2-one structural motif with ketone and a five-membered ring containing two sulfur atoms. This interaction involves larger number of functional groups with variety of heteroatoms, contributing significantly to the electronic properties and potential charge transfer efficiency, The presence of multiple aromatic rings could enhance electron delocalization, improving the photovoltaic properties of the molecule.
L*Let Important Interaction (Number 0): ** *Reasoning:** Interaction I involves 1,3-dithiol-2-o structural motif interacting with two substructures: a thiophene- iivemmebered aromatic ring and three-carbor chain
34

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
ending ketone group. While the 1,3-di motif is crucial, the three-carbon chain ending in a ketone group relatively straightforward and less complex compared the multiple heterocyclic aromatic systems presented in interactions and 2. The lack of additional aromatic heterocycles in this interaction makes it less critical in the context of electronic interactions within the molecule.
Thus, the most important interaction is **2**, and the least important interaction is **1*e Comments by Expert
Correct, interaction is the most important because it features three heterocyclic aromatic tings-thiophene furan, and pyrrole-interactingg with a dithiol--2-oee structural motif containing ketone and a sulfur-rich five-membered ring. This combination significantly contributes to electronic delocalization and charge transfer efficiency, which critical for photovoltaic functionality. Interaction 1 is the least important due its simpler structure, involving a three-carbon chain ending in a ketone group and lacking the additional aromatic cycles. This simplicity results in lower contribution to the electronic interactions.
1.2.1. EXAMPLE DESIGN STORY
We provide an example design story for The story begins by rationalizing the choice of a root motif, in the context of designing organic photovoltaics, then proceeds explain each positive-negative connection in the parse tree from the perspective of structure- property relationships and design. Here is the story:
1. For the monomer described, intended for organic photovoltaic applications, the most essential motif would be the one that best supports charge transport and absorption of light. In organic photovoltaics, aromatic and conjugated systems are crucial because they enable extended x-conjugation, which improves the absorption of solar radiation and facilitates the transport of charge carriers.
Among the motif's provided, the **Pyrimidine ring (number 2)** is a strong candidate due its aromatic nature and the presence of nitrogen atoms. The nitrogen atoms in pyrimidine ring can participate in hydrogen bonding and other electroactive interactions, making this structure very important for the electronic properties of the polymer.
### Therefore, the root motif is **number 2*
#### Reasoning:
** CCnjuugation and *romaticity** The pyrimidine ring is aromatic, providing structure that allows for delocalization of -leeectrons, which is beneficial for light absorption and charge transport. =*iirogenn *toms** The nitrogen atoms can engage in various intermolecular interactions which are important for altering the electronic properties and improving the efficiency in photovoltaic applications. ** Versatiility*** Pyrimidine rings are known to be versatile components in organic electronics, capable of facilitating charge carrier mobility and stability.
Thus, the Pyrimidine ring stands out as the single most important motif essential to the chemical profile of the monomer for its usefulness in organic photovoltaic applications. 2. The positive (red) and negative (green) motifs in this polymer design play crucial role enhancing its electronic properties, making a viable candidate for applications such as organic solar cells. The red motif, likely a conjugated system with extended pi-electron delocalization, serves the electron-donating or transporting component. The green motif, moment and facilitate charge separation. This donor-acceploe interplay is justified as it maximizes the polymer's 's ability to harness photogenerated excitons and promotes efficient charge transfer. The integration of the negative motif within the red motif structure lowers the overall bandgap, which is beneficial for the absorption of a broader spectrum of sunlight, thus increasing the efficiency of solar energy conversion. Furthermore, the inherent stability and solubility imparted by this design ensure that the polymer can be processed in various thin-film fabrication techniques, making highly adaptable for use in organic photovoltaies. 3. In the design of polymer materials for applications such organic solar cells, the positive (red) and negative (green) motifs play crucial roles in optimizing the photophysical and electronic properties of the polymer. The red motif, an aromatic structure with electron-donating groups, enhances the polymer's ability to absorb sunlight and facilitate exciton generation. The green motif, an adjacent in the efficient separation and transport of charge carriers upon photoexcitation. This juxtaposition of donor and acceptor segments within the same molecular framework creates a built-in push-pull system, improving charge separation efficiency and minimizing recombination losses. Such finely tuned electronic interactions are essential for achieving high power
group, is strategically positioned to create strong dipole
unit, increases the polymer's 's electron affinity, aiding
35

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
conversion efficiencies in organic solar cells, they directly impact the material's photovoltaic performance. Thus, the careful integration of these motifs is justified as it provides a strategic means to balance and enhance the photovoltaic and charge transport properties needed for high-performance organic solar materials. 4. The incorporation of the negative (green) motif into the positive (red) motif within this polymer design is pivotal for enhancing its functionality in specific applications like organic solar cells. The positive motif, marked in red, likely represents an electron-rich region, while the negative segment. This strategic juxtaposition creates a donor-acceptor
motif, highlighted in green, signifies an (D-A) interaction, which is essential for effective charge separation and transport. This configuri configuration helps facilitate the alternating push-pull effect of electrons, which is crucial in improving the photophysical properties of the polymer, such as light absorption and charge carrier mobility. In organic solar cells, such D-A conjugated systems enhance the absorption spectrum, increase the efficiency of exciton dissociation, and promote the transport of electrons and holes, thereby improving the overall power conversion efficiency. Therefore, the careful design and synthesis of these alternating motifs are justified as they provide a molecular architecture conducive to tigh-performance organic photovoltaic materials. 5. The positive (red) and negative (green) motifs in the polymer depicted play a crucial role in determining its electronic properties and overall efficacy in applications like organic solar cells. The positive motif likely represents an electron-rich segment, while the negative motif represents an electron-deficient segment. This complementary interaction promotes intramolecular charge transfer, which is essential for enhancing the polymer's ability absorb light and generate electron-hole pairs efficiently, The strategic incorporation of the green motif within the red framework ensures balanced electronic structure, thereby optimizing the separation of charge carriers and minimizing recombination losses. This facilitates effective charge transport, leading to improved photovoltaic performance. Furthermore, this donor-acceptor architecture can be tuned to adjust the absorption spectrum, enabling better aligmment with the solar spectrum. Hence, the addition of the negative (green) motif to the positive (red) motif is a deliberate design choice to enhance the polymer's 's semiconducting properties, making it highly effective for applications in organic solar cells. 6. In the design of polymers for organic solar cells, the strategíc incorporation of the negative (green) motif into the positive (red) motif is crucial for optimizing the electronic properties and enhancing performance. The positive motif, depicted in red, likely represents electron-donating -onatnng unit, while the negative motif, shown in green, is indicative of an electron- withdrawing unit. When these two motifs are combined within the polymer backbone, they create an alternating accer-ac structure that facilitates charge separation and transport by lowering the bandgap and increasing the polymer's ability to absorb sunlight. This push-pull interaction between the electron-rich and electron-deficient segments contributes to the creation of effective pathways for exciton dissociation and charge carrier mobility. Consequently, this interaction plays a fundamental role in balancing and optimizing the absorption spectrum, charge transport, and overall efficiency of the organic solar cells, making the addition of the green motif justified and beneficial for enhancing the performance of such devices. 7. In the design of polymers for organic solar cells, the interaction between the positive (red) and negative (green) motifs is of paramount importance for optimizing the electronic properties of the material. The positive motif, marked in red, likely represents an electron electron-donating donating unit, while the negative motif, marked in green, represents an electron-accepting unit. This push-pull mechanism, also referred to as a donor-acceptor interaction, is crucial for improving charge separation and transport within the polymer. By incorporating both electron -iieriing and eeeetronwwithdrwwiin motifs, the resulting polymer exhibits a narrowed bandgap, enhancing its ability to absorb a broader spectrum of sunlight. The dipole moments generated at the interface between these motifs also facilitate exciton dissociation, improving the efficiency of charge generation. Additionally, the conjugated backbone formed by the alternating positive and negative motifs creates pathway for electron mobility, which is essential for the polymer's 's performance as active layer in organic solar cells. Thus, the deliberate design incorporating these specific motifs is justified by its direct contribution to improved light absorption, charge separation, and overall photovoltaic efficiency. 8. The design of the polymer shown, which includes alternating positive (red) and negative (green) motifs, is likely aimed at enhancing the material's electronic properties, useful for applications such as organic solar cells. The positive motif suggests regions with characteristics while the negative motif indicates electron-donating properties. This juxtaposition creates donor- acceptor interaction framework that is crucial for tuning the polymer's bandgap. In organic solar cells, such a tailored bandgap is vital for effective light absorption and charge separation. The diectron-rich (negative) motif can facilitate enhanced electron mobility while the electron-deficient (positive) motif can improve hole transport efficiency. This duality ensures the polymer can effectively separate and transport charge carriers generated upon light absorption, thereby improving the overall efficiency of the solar cells. The inclusion of side chains and functional groups further modulates solubility, processability, and morphological stability, making the polymer system adaptable for practical device fabrication. 9. In the context of designing polymers for applications such as organic solar cells, the integration of negative (green) motif with a positive (red) motif is crucial for optimizing the electronic properties and ensuring effective performance. The red motif in the polymer structure typically represents electron units, which enhance the electron density and facilitate charge transport. The green motif, on the other hand, represents dlectron-withdraviin units, which help to lower the polymer's
36

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
overall energy levels and improve electron acceptance. This positive-negative interaction is fundamental because it results in defined energy band gap that is essential for efficient absorption of sunlight and conversion into electrical energy. Additionally, the interplay between these motif's enhances the polymer's ' ability to form well-ordered, crystalline structure, further improving charge mobility and overall efficiency. Their balanced interaction improves the stability, solubility, and mechanical properties of the polymer, making it suitable for real-world applications in organic electronics and photovoltaics. 10. In designing a polymer for applications like organic solar cells, the interaction between the positive (red) and negative (green) motifs plays a critical role in modulating the polymer's electronic properties. The positive red motif, in this case, forms the backbone of the polymer's structure, contributing to the overall electromic conduction. By introducing the negative groups such as fluorine atoms, the electron density distribution
green motif, which likely contains within the polymer is altered, enhancing its ability to facilitate charge transfer processes. This complementary interaction can enhance the polymer's 8 capacity for efficient charge separation and transport, crucial for the effective conversion of light to electricity in organic solar cells. Furthermore, the specific electron-donating and glectron-aceppiin of these motifs can be fine-tuned to optimize the polymer's absorption spectrum and energy levels, making the material more effective in capturing sunlight and transforming into usable energy. Thus, the integration of these motifs is justified through a detailed understanding of their synergistic effects on the polymer's photovoltaic performance. 11. The polymer depicted showcases a strategic combination of positive red motif and a negative green motif, designed enhance its properties for applications like organic solar cells. The red motif, rich in electronegative carbonyl (C=O) and fluorine (F) groups. likely contributes to the electron- accepting characteristics necessary for efficient charge separation and transfer, critical in photovoltaic functions. Meanwhile, the green motif, embedded centrally, presumably acts as an electron-donatie unit due to its conjugated ring structure with additional electron- withdrawing carbonyl groups enhancing the polymer's 's ability to create a stable, low-energy LUMO level. The interplay between these motifs through r-conjugation and potential intramolecular charge transfer enhances the polymer's electronic properties, such as its photochemical stability, light absorption, and charge mobility. This synergistic interaction justifies the inclusion of the negative motif to optimize the polymer's 's efficiency in harvesting solar energy, delivering performance enhancements critical for next-generation organic solar cells. 12. In evaluating the design of the polymer for potential applications such as organic solar cells, the interaction between the positive (red) and negative (green) motifs is crucial. The strategic incorporation of the negative motif, marked by its wlectron-withdraaving groups (such as the carbonyl groups), is critical in creating an internal charge transfer within the polymer matrix. This enhances electron mobility by facilitating a donor-acceptor interaction where the red motif acts as the electron donor and the green motif functions as the electron acceptor. Such a configuration supports efficient separation and transport of charge carriers. Consequently, this charge transfer interaction reduces recombination losses and increases the overall efficiency of the polymer in solar energy conversion. Additionally, the spatial orientation and electronic properties of the combined motifs influence the polymer's 's absorption spectrum and photophysical properties, optimizing light absorption. Thus, the deliberate addition of the negative motif to the polymer structure is justified by its substantial impact on enhancing electrical conductivity, charge separation, and optical properties, which are vital for the efficacy and performance of organic solar cells. 13. In the context of designing a polymer for applications such as organic solar cells, the incorporation of positive and negative motifs plays a pivotal role in tailoring the material's electronic properties. The positive (red) motif may be indicative of an docctrnn-donttng moiety, which can enhance the charge carrier mobility within the polymer. Conversely, group, crucial for stabilizing the electron density and
properties
the negative (green) motif likely represents an Saccron-wiithrrvy improving the polymer's electron affinity. The interaction between these motifs creates push-pull effect in the polymer backbone, which is vital for optimizing the absorption of a broad spectrum of light and facilitating efficient charge separation and transport. This push-pull mechanism is particularly advantageous in organic solar cells, as it contributes to higher power conversion efficiency by maximizing the generation and transport of free charge carriers when the polymer is exposed to sunlight. The strategic placement and balance of these motifs enable fine-tuning of the polymer's HOMO-LUMM (highest occupied molecular orbital-lowest unoccupied molecular orbital) gap, ensuring it is suitable for effective photovoltaic performance. 14. In the given polymer, the design incorporates a positive (red) and negative (green) motif, which plays a critical role in its chemical and physical properties. The positive motif, marked in red, is likely an electron-donor segment that provides a source of electrons through conjugated systems or electron-rich groups. The negative motif, highlighted in groups that facilitate electron acceptance, making it an electron-acceptor
green, could incorporate segment. This complementary interaction between donor and acceptor segments within the polymer is fundamental for optimizing the charge-transfer processes, which are essential in applications such as organic solar cells. When light excites the polymer, the positive (red) motif can donate electrons that are efficiently transferred to the negative (green) motif. This charge separation is crucial for generating electrical current in organic photovoltaics. Additionally, this positive-negative interaction can enhance the polymer's 's stability, morphology, and overall electronic properties, making it a viable candidate for geghhperformance organic solar cells. 15. In designing polymers for applications like organic solar cells, the interaction
37

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
between the positive (red) and negative (green) motif's plays critical role in optimizing the polymer's electronic properties and overall performance. The red motif, likely possessing electron-donating characteristics, enhances the polymer's ability to transport holes, making it an efficient donor material. On the other hand, the green motif, characterized by electron- withdrawing properties, contributes to electron transport and serves as an acceptor material. The juxtaposition of these contrasting electronic features within the same polymer backbone facilitates effective charge separation and transport, crucial for the efficiency of organic photovoltaic devices. Moreover, the fine-tuning of these donor-acceptor interactions influences the polymer's bandgap and energy levels, which can be tailored to maximize light absorption in the solar spectrum, thus enhancing the photocurrent generation in solar cell applications. This delicate balance and interaction between the positive and negative motifs thereby justify their integration, significantly contributing to the polymer's optoelectronic properties and making it viable candidate for high-performance organic solar cells. 16. In designing polymers for applications such (green) motifs is pivotal
as organic solar cells, the interplay between electron-donating (red) and for optimizing the electronic properties and stability of the material. The red motif' serves as an clectron-donating unit, facilitating the creation of a high-energy orbital system essential for effective light absorption and exciton generation. Conversely, the green motif acts as an deecrron-accppiin unit, which helps in stabilizing the generated excitons and improving charge separation efficiency. This complementary interaction between the electron-rich and electron-deficient segments creates a balanced distribution of electronic density, thereby fine-tuning the energy levels and improving charge transport abilities. Additionally, this polymer, making it more robust under operational conditions. The judicious incorporation of the negative (green) motif to the positive (red) motif' is thus essential in engineering polymers with the desirable electronic and physical characteristics suited for
synergy enhances the structural rigidity and thermal stability of the
organic solar cells. 17. In the context of designing a polymer for applications such as organic solar cells, the interaction between the positive (red) and negative (green) motifs is crucial for tuning the polymer's optoclectronic properties. The negative motif in green, characterized by electron withhrainnn groups, enhances the polymer's 's electron affinity, improving its ability to accept electrons. This leads to lower energy band gap, which is favorable for absorbing broader spectrum of sunlight. Additionally, the complementary positioning of these motifs enhances charge separation and transport efficiencies within the polymer matrix. The electron-rich positive motif in red can act as a donor, facilitating effective charge transfer processes when paired with the dectron-deficienn negative motif. By strategically incorporating these motifs into the polymer framework, we can optimize the material's photovoltaic performance, achieving better light absorption, higher charge carrier mobility, and ultimately improved efficiency in converting solar energy to electrical energy in organic solar cells. 18. In the context of designing polymers for applications such as organic solar cells, the positive (red) and negative (green) motifs are strategically integrated to optimize electronic and structural properties. The red motif, being conjugated aromatic structure, offers high electron density and good charge transport characteristics due to its delocalized rellectrons, which is crucial in facilitating efficient charge mobility. Conversely, the green motif, characterized by its electron-witter withdrawing functional groups (e.g., carbonyl and fluorine atoms), introduces electron deficiency into the polymer chain. This secctronwwithdraiing nature helps to lower the polymer's LUMO (Lowest Unoccupied Molecular Orbital) energy level, aiding in the enhancement of electron acceptor properties. The presence of both motifs creates a donor-acceptor (D-A) interaction within the polymer, optimizing the solar cell's intrinsic properties such as bandgap tuning, absorption spectrum, and charge separation efficiency. This deliberate juxtaposition of electron-rich and electron -defiiendeeiiient motifs forms polymer network that is highly suitable for converting sunlight into electrical energy with maximized efficiency, making this motif combination crucial for advanced organic photovoltaic applications. 19. In the design of polymer for applications such as organic solar cells, the incorporation of complementary electronic motifs is crucial for optimizing charge transfer and enhancing device efficiency. The positive motif highlighted in red and the negative motif in green represent electron- -onitinnn and Seectron-withdrawe intermolecular charge transfer dynamics, which is essential for efficient exciton dissociation and charge transport within the polymer matrix. The diectron-donating capacity of the red motif, typically featuring conjugated systems and groups that can delocalize electrons, complements the electron-deficiencc of the green motif, often imbued with electronegative groups or atoms like fluorine and carbonyl groups. This interaction not only promotes optimal energy level alignment between the highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital (LUMO) but also enhances the polymer's photophysical properties by broadening its absorption spectrum. Such a synergistic design is beneficial for increasing the power conversion efficiency of organic solar cells by maximizing light absorption and facilitating efficient charge separation and mobility. 20. In the context of designing polymers for applications such as organic solar cells, the interplay between electron-rich (positive, red) and deectron-deficier (negative, green) motifs of paramount importance. The red motif, characterized by its extended r-conjugation, acts as an electron-donating unit, facilitating efficient charge functionalities, thereby reducing the polymer's 's
segments, respectively. This donor-
architecture facilitates effective
transport. Mcanwhile, the green motif introduces highest occupied molecular orbital (HOMO) energy levels while increasing its lowest unoccupied molecular orbital (LUMO)
38

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
energy levels. This complementary pairing creates unique donor-acceptor interface within the polymer structure, enhancing charge separation and thereby improving the photovoltaic performance. Furthermore, the electron-withdraviinn withdrawing groups can stabilize the resulting negative charges, reducing charge recombination rates. This careful juxtaposition of motifs thereby optimizes light absorption, enhances charge carrier mobility, and ultimately leads to improved energy conversion efficiencies in organic solar cell applications. 21. In designing polymers for applications such as organic solar cells, the strategic incorporation of both positive (red) and negative (green) motifs essential to optimize the material's properties. The red motif, polyaromatic segment, serves as an electron donor, while the green motif, with silicon and carbonyl groups, acts as an electron acceptor due its charge carrier mobility within the polymer, crucial for efficient photovoltaic performance. The juxtaposition of these motifs can lead to reduction in the polymer's band gap, increased light absorption, and improved exciton dissociation, thereby enhancing the efficiency of organic solar cells. Additionally, the specific arrangement of the motifs affects the crystallinity and morphological stability of the polymer, which are pivotal for device performance and longevity. This synergistic design illustrates how the confluence of electron-rich and electron clectron-deficient units can be exploited to tailor the electronic and physical properties of polymers, rendering them suitable for cutting-edge applications in organic electronics.
nature. This accer-o interaction enhances charge separation and
H.3. Case Study: Isocyanates
Isocyanates are highly reactive organic compounds characterized by the presence of one or more isocyanate groups (- N=C=O). Their high reactivity makes them essential in the production of polyurethanes, this reaction forms urethane linkages, which are the backbone of polyurethane materials. Polyurethanes that are synthesized from isocyanates are really versatile, with wide range of applications from flexible foams and elastomers rigid insulation materials. The molecular structure of the isocyanate can be tailored, with aliphatic or aromatic variations influencing the final properties of the polymer, such as its mechanical strength, flexibility, and resistance to environmental factors like UV radiation. However, due to their high reactivity, isocyanates are associated with potential health hazards, including irritation and sensitization. The safety concerns attached to this compounds made requires strict industrial safety protocols during handling and processing.
For isocyanate, features -N=C=O and highly reactive, especially in forming polyurethane linkages. The selected molecule contains two distinct -N=C=O groups serving as core functional groups form isocyanate and highly represent the desired reactivity in this dataset of the molecules.
O
C
C
C
Figure 10. Input molecule from Isocyanates
39

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Prompt
I want you to think like a chemist performing a detailed analysis of the chemical composition of an isocyanates through its constituent motifs. I will highlight for you 9 of the substructures of a molecule. They are numbered one-by-one from Motif 0 to Motif 8, inclusive. want you to explain, concisely, what each numbered motif is. Make sure to start from Motif 0 and go order of the numbering. MAKE SURE you describe EVERY MOTIF!
1
3
4
7
Response
Sure, let's analyze each motif in order:
%%% Motif 0 This structure represents tert- group. It consists of central carbon atom bonded to three methyl groups (each CH3 group).
%%% Motif This structure is cyclohexane ring. It consists of six carbon atoms arranged in a hexagonal ring, with each carbon atom bonded to two other carbons and two hydrogen atoms (not shown).
%%%% Motif This structure an ethylamine group. It consists of an amine group (NH2) attached to an ethyl group (CH3-CH2-).
%%% % Motif This structure is an ethylideneamine motif. It consists of carbon- alkyl group (CH3) and the carbon another carbon.
carbon-nitrogen nitrogen (C=N) double bond, with
the nitrogen also bonded
%%% Motif 4 This structure features an O=C=N group, indicating the presence of an isocyanate group bonded to carbonyl group. This is a key motif in many urea and carbamate derivatives.
%%% Motif This structure represents a neopentyl group. It consists of a central carbon atom bonded to four other carbon atoms: one in a methyl group (CH3) and three in methylene groups (CH2).
% 98% % Motif This structure shows nitrile group (C=N) attached an ethyl group (CH3-CH2-). Nitriles are commonly found in organic compounds.
%%% Motif 7 This structure is cyanoacetamide group. It consists of nitrile group (C=N) adjacent to carbonyl group (C=0).
%%% Motif 8 This structure is an isocyanate group (N=C=O). It consists of nitrogen atom double-bonded to carbon atom, which is also double-bonded to an oxygen atom. These motifs may represent fragments of larger molecules, often seen in various industrial and chemical synthesis processes.
40

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
Comments by Expert
This is an easy and straightforward prompt, with clear guidelines and restrictions that ensure each motif is individually analyzed and described. GPT's answer: response accurately identified and clearly explained the functionality of all motifs from to 8, providing precise and concise descriptions of each substructure. The answer effectively captures the unique chemical features the motifs, demonstrating comprehensive understanding of the molecular composition.
Prompt
I want you to think like a chemist performing a detailed analysis of the chemical composition of an isocyanates through its constituent motifs. will highlight for you some of the distinctive fragments molecule. They are numbered from 0 and individually highlighted in RED. Focus ONLY on the substructure highlighted in red within each cell. Here is the descriptions for each substructure provided by expert:
Sure, let's analyze each motif in order:
%%% Motif 0 This structure represents tert-butyl group. It consists of central carbon atom bonded to three methyl groups (each CH3 group).
%%% % Motif This structure is a cyclohexane ring. It consists of six carbon atoms arranged in a hexagonal ring, with each carbon atom bonded to two other carbons and two hydrogen atoms (not shown).
%6% % Motif This structure an ethylamine group. It consists of an amine group (NH2) attached to an ethyl group (CH3-CH2-).
%%% % Motif 3 This structure is ethylideneamine motif. It consists of carbon-nitrogen (C=N) double bond, with the nitrogen also bonded to an alkyl group (CH3) and the carbon to another carbon. % %% Motif 4 This structure features an O=C=N group, indicating the presence of an isocyanate group bonded to carbonyl group. This is a key motif in many urea and carbamate derivatives.
%%% Motif This structure represents a neopentyl group. It consists of a central carbon atom bonded to four other carbon atoms: one a methyl group (CH3) and three methylene groups (CH2).
%%% % Motif This structure shows nitrile group (C=N) attached an ethyl group (CH3-CH2-). Nitriles are commonly found in organic compounds. %%% Motif 7 This structure is cyanoacetamide group. It consists of nitrile group (C=N) adjacent to carbonyl group (C=O).
%%% Motif 8 This structure is an isocyanate group (N=C=O). It consists of nitrogen atom double-bonded to a carbon atom, which is also couble-bonded to an oxygen atom. These motifs may represent fragments of larger molccules, often seen in various industrial and chemical synthesis processes.
I want you to tell if any of them should be combined together form more meaningful substructure. Remember the defining isocyanate group -N=C=O. Your task is to highlight the primary functional groups of the molecule. Output a single pair of numbers if you think those two fragments should be combined, and brief explanation why. If no such pairs exist, don't 't output anything.
Response
%%% Motif This structure features an O=C=N group, representing the presence of an isocyanate group bonded to carbonyl group, which critical motif many urea and carbamate derivatives. %%% Motif 8 This structure features isocyanate group (N=C=O), consisting of nitrogen atom couble-bonded to carbon atom, which is also double-bonded to an oxygen atom.
Combining **Motif 4** and **Motif 8* forms the primary functional group: carbonyl group directly attached to an isocyanate group, which critical for identifying the chemical properties and reactivity of isocyanates. %%% *Conclusion:*** ** #*4, 8** may be combined into a functional group consisting carbonyl bonded directly to an isocyanate, making the structure more meaningful in the context of isocyanate chemistry. Comments by Expert
This prompt is easy and straightforward, it simply asks GPT to identify if any two groups should be combined to form more meaningful isocyanate structure. GPT's answer is 4 and 8. GPT correctly recognized Motifs 4 and 8 as isocyanate-related groups but incorrectly suggested that they should be combined, as both motifs already represent

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
functional isocyanate groups individually. This demonstrates slight misunderstanding of the requirement propose combinations only when the resulting structure provides new enhanced functionality, which unnecessary in this case since the motifs are already complete and meaningful in their current forms. Prompt
I want you to think like a chemist performing a detailed analysis of the chemical composition of an isocyanates through its constituent motifs. will highlight for you some of the distinctive substructures of an isocyanates. They numbered from 0. Here the textual descriptions of each motif: Sure, let's analyze each motif in order:
%%% Motif 0 This structure represents tert-butyl group. It consists of a central carbon atom bonded to three methyl groups (cach CH3 group).
%%%% % Motif This structure is a cyclohexane ring. It consists of six carbon atoms arranged in a hexagonal ring, with each carbon atom bonded to two other carbons and two hydrogen atoms (not shown).
%%% Motif This structure an ethylamine group. consists of an amine group (NH2) attached to an ethyl group (CH3-CH2-).
%%% Motif 3 This structure is ethylideneamine motif. It consists of carbon- niibonen (C=N) double bond, with the nitrogen also bonded to an alkyl group (CH3) and the carbon to another carbon. %%% Motif 4 This structure features an O=C=N group, indicating the presence of an isocyanate group bonded to a carbonyl group. This is a key motif in many urea and carbamate derivatives.
%%% Motif This structure represents a neopentyl group. It consists of a central carbon atom bonded to four other carbon atoms: one in a methyl group (CH3) and three in methylene groups (CH2).
%%% Motif This structure shows nitrile group (C=N) attached an ethyl group (CH3-CH2-). Nitriles are commonly found in organic compounds.
%%% Motif 7 This structure is cyanoacetamide group. It consists of nitrile group (C=N) adjacent to carbonyl group (C=O).
%%% Motif 8 This structure is an isocyanate group (N=C=O). It consists of nitrogen atom double-bonded to a carbon atom, which is also double-bonded to an oxygen atom. These motifs may represent fragments of larger molecules, often seen in various industrial and chemical synthesis processes.
I want you to pick only ONE of these the root motif most essential to its chemical profile. It should be the single most important motif the rest of the isocyanate was built around. The defining isocyanate group is -N=C=O, your selected root motif MUST contain the group. If there are multiple such motifs, or one doesn't 't clearly stand out, just pick one of them. Give your answer as a single number. Explain your reasoning carefully.
Response
%%% Answer: 8
%%% Reasoning:
In the given motifs, functional group is essential to the chemical profile of isocyanates.
looking for the motif that contains the isocyanate group (-N=C=O) at its core, this
Among the given motifs, Motif 8 includes the isocyanate group distinctly:
Motif 8: features the isocyanate group (N=C=O) prominently and without any additional structural complexity
directly attached to it. This makes it the simplest and most straightforward representation of the essential isocyanate entity.
This feature, the critical and defining part of isocyanate, means Motif is the most crucial the chemical profile of the substance. The rest of the molecule is built around this group. making the root structure.
Other motif's either have additional complex groups attached to the isocyanate group or other functional groups that could potentially classify the molecules differently. Hence, Motif 8 is chosen as it aligns precisely with the core chemical characteristic we are focusing on.
Comments by Expert
The prompt of medium level complexity, since picking the most important functional group involves empirical experience in organic chemistry, and the answer required one individual selection can bring discrepancies sometimes between language models and chemistry expert. GPT's answer: correct, GPT answer motif since it features the
42

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
distinct isocyanate group, which is essential the chemical profile of isocyanates. The explanation provided is thorough, highlighting that Motif represents the simplest and most direct form the isocyanate group without additional structural complexity. GPT even correctly noted that other motif' include either additional groups attached to the isocyanate other functional groups that could alter the molecule's classification, further justifying the choice of Motif 8. This demonstrates a clear understanding of the core chemical characteristics required for the task and therefore delivers perfect answer to the question.
H.4. Case Study: Acrylates
Acrylates refer broad class of chemical compounds that are derivatives of acrylic acid and its related esters. They can undergo polymerization rapidly through free- radical mechanism, making them particularly useful in applications requiring fast curing processes, such as in adhesives, coatings, sealants, and 3D printing materials. Acrylates possess double bond, which is highly reactive in the presence of free radicals, allowing them to form long polymer chains with varying degrees of crosslinking. This flexibility in polymer structure endow acrylates with unique properties such as optical clarity, UV resistance, and flexibility, depending on the specific formulation. In industries like automotive, aerospace, and biomedical devices, acrylate- based materials are featured for their robust performance tough environments. Additionally, modified acrylates such as methacrylates are also featured for multifunctions and with greater control over polymers' properties including rigidity, toughness, and adhesion.
For acrylate, the molecule I selected have typical ester/acrylate functional groups, the rest of the part containing both carbon chains and aromatic structures. The molecule is not over complicated acrylate example but have all functional groups for GPT to learn and it's very representative. This molecule obtains vinyl end groups which are highly reactive polymerization reactions, making it good candidate for studying acrylate reactivity as well.
Figure 11. Input molecule from Acrylates
Prompt
I want you to think like chemist performing a detailed analysis of the chemical composition of an acrylates through its constituent motifs. I will highlight for you 7 of the substructures of a molecule. They are numbered one-by-one from Motif 0 to Motif 6, inclusive. want you to explain, concisely, what each numbered motif is. Make sure to start from Motif and go in order of the numbering. MAKE SURE you describe EVERY MOTIF!
43

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
1
2
3
4
5
6
Response
Sure, here's a concise description of each numbered motif as per the image provided: **Motif 0**; This motif is vinyl group, which consists of two carbon atoms connected by a double bond (C=C),
with each carbon atom having one additional connection for total of three.
**Motif ** This
acrylate group, also known propenoate ester. It consists of carbonyl group (C=0)
attached carbon atom, which also has oxygen atom bonded to another carbon atom (an ester linkage).
**Motif 2 This motif is an ethyl ether group, specifically an ether where oxygen atom connects two ethyl
(CH3-CH2) groups.
**Motif 3** This is methoxy group, where an oxygen atom is attached to a single methyl group (CH3).
**Motif 4** This is simple propyl group, consisting of three carbon atoms chain (CH3-CH2-CH2).
**oofi 5** This a benzene ring, also known an aromatic ring, which made up of six carbon atoms arranged in hexagonal ring with alternating double bonds between carbon atoms.
**Motif 6**; This is an isopropyl group, branched chain comprising three carbon atoms where the central carbon is connected two methyl groups (CH3-CH-CH3),
Each motif represents fundamental building block that can contribute to the structure and properties of an acrylate molecule.
Comments by Expert
This is casy and straightforward prompt, with clear instructions and restrictions that make it easy for GPT to follow. By specifying the total number of motifs and requiring a sequential explanation, the task provides a logical structure that minimizes the potential for errors.
GPT's response is entirely accurate, offering correct and concise explanations for all motifs. Fach motif's func- tionality is clearly described, reflecting thorough understanding of their roles within the acrylate molecule. This approach ensures clarity and completeness, effectively addressing the prompt's requirements. Prompt
want you to think like a chemist performing detailed analysis of the chemical composition of an acrylate through its constituent motifs. will highlight for you some the distinctive fragments molecule. They are numbered from and individually highlighted GREEN. Focus ONLY on the substructure highlighted green within each cell. Here is the descriptions for each substructure provided by expert:
Sure, here's s concise description of each numbered motif per the image provided:
M*to (**. This motif is vinyl group, which consists of two carbon atoms connected by double bond (C=C),
with each carbon atom having one additional connection for total of three.
**Motif **; This is an acrylate group, also known as a propenoate ester. It consists of a carbonyl group (C=O)
44

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
attached a carbon atom, which also has an oxygen atom bonded to another carbon atom (an ester linkage).
**Motif 2 This motif an ethyl ether group, specifically an ether where an oxygen atom connects ethyl
(CH3-CH2) groups.
**Motif 3** This is methoxy group, where oxygen atom is attached to single methyl group (CH3).
**Motif 4** This is simple propyl group, consisting of three carbon atoms in chain (CH3-CH2-CH))
**Motif 5**: This is a benzene ring, also known as an aromatic ring, which is made of six carbon atoms arranged in hexagonal ring with alternating double bonds between carbon atoms.
**Motif 6##; This an isopropyl group, a branched chain comprising three carbon atoms where the central carbon connected two methyl groups CCH3-CH-CHE
Each motif represents fundamental building block that contribute the structure and properties of an acrylate molecule.
I want you to tell me if any two of them should be combined together to form more meaningful substructure. Remember the defining acrylate group is C=CC(=OJO, which consists of carbon-carbon double bond and carboxylate ester. Your task to construct this defining acrylate group by combining the alkene or vinyl group with the ester group. If the acrylate group is already present, your task to highlight any other noteworthy functional groups. Output a single pair of numbers if you think those two fragments should be combined, and brief explanation why. no such pairs exist, don't It output anything.
1
2
3
4
5
6
Response
1 0. The reason for this particular combination is to form the defining acrylate group, C=CC(=O)O. By combining Motif (the acrylate/propenoate ester featuring carbonyl carbon bonded to oxygen atom linked by an ester bond to another carbon) with Motif 0 (the vinyl group, an alkene with two carbon atoms connected by a double bond), we achieve the desired acrylate structure containing a carbon-carbon double bond and carboxylate ester. Comments by Expert
Correct, 0 and 1 should be combined together, as they appropriately form the acrylate group. This combination integrates the vinyl group's carbon-carbon double bond (Motif 0) with the ester functionality (Motif 1), resulting in the characteristic structure of acrylates. The answer demonstrates clear understanding of the fundamental chemical features and their assembly into the essential acrylate group.
Prompt
I want you to think like chemist performing detailed analysis of the chemical composition of acrylates through its constituent motifs. I will highlight for you 6 of the substructures of molecule. They are numbered one-by-one from Motif 0 to Motif 5, inclusive. want to explain, concisely, what each numbered motif is. Make sure to start from Motif 0 and in order of the numbering. MAKE SURE you describe EVERY MOTIF!
45

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
1
2
3
4
5
Response
Certainly, here is a concise characterization of each motif:
**Motif 0:* This is ester group with adjacent carbon-carbon double bond (C=C). The structure can be described as vinyl ester derived from acrylic acid. Specifically, it contains carbonyl group (C-O) adjacent to an oxygen manlle-boned (O-C) to vinyl group (-HHCCH2)
**Motif 1 * This is an ether group, characterized by an oxygen atom linked to two alkyl groups, which are represented here as carbon atoms CC. This motif shows the simplest ether functional group.
**Motif 2:*+ This is another ether group, but oriented differently comparison to Motif Here, an oxygen atom bonded to two carbons
linear fashion again forming the general formula of an ether (R-ORR').
** **Motif ** This is an alkane chain with three carbons, indicating simple linear hydrocarbon with single bonds between the carbon atoms (C-C-C.. This motif is fragment of a longer alkane chain part of more complex hydrocarbons.
M*oof 4:** This motif is a benzene ring, signified by its hexagonal structure with alternating double bonds. Benzene an aromatic hydrocarbon with significant stability due to its conjugated pi electron system. **totif 5:** This an isopropenyl group, described as an alkene which the central carbon is double-bonded to one carbon and single- bonded to two other carbons (other than hydrogen). This motif presents branching point in the structure with double bond.
These motifs collectively depict variety of functional groups and structural elements commonly found in the chemistry of acrylates and related organic compounds.
Comments by Expert
The answer is Correct and provides correct descriptions for all motifs from 0 to 5. Each motif clearly identified and concisely characterized, reflecting a solid understanding of their functional and structural significance. The analysis effectively captures the diversity of functional groups, including esters, ethers, alkanes, aromatics, and alkenes, aligning with the prompt's requirements. Prompt
I want you to think like a chemist performing detailed analysis of the chemical composition of an acrylate through its constituent motifs. I will highlight for you some of the distinctive fragments molecule. They are numbered from and individually highlighted in GREEN. Focus ONLY on the substructure highlighted in green within each cell. Here is the descriptions for each substructure provided by expert:
Certainly, here is concise characterization of each motif:
46

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
**Motif 0:* This is an ester group with an adjacent carbon-carbon double bond (C=C). The structure can be described as a vinyl ester derived from acrylic acid. Specifically, it contains a carbonyl group (C=O) adjacent an oxygen single-bonded (O-C)
vinyl group (-CH=CH2).
**Motif 1:** This is an ether group, characterized by oxygen atom linked to two alkyl groups, which are represented here as carbon atoms (C-O-C C). This motif shows the simplest ether functional group.
**ootf 2:*+ This is another ether group, but oriented differently in comparison to Motif 1. Here, an oxygen atom is bonded to two carbons in a linear fashion again forming the general formula of an ether (R-O-R').
**Motif 3:** This is an alkane chain with three carbons, indicating simple linear hydrocarbon with single bonds between the carbon atoms (C-C C). This motif a fragment of a longer alkane chain or part of more complex hydrocarbons.
** **Motif 4:** This motif is benzene ring, signified by its hexagonal structure with alternating double bonds. Benzene is an aromatic hydrocarbon with significant stability due to its conjugated pi electron system. **Motif 5:** This is an isopropenyl group, described as an alkene in which the central carbon is houble-bonded to one carbon and single-bonded to two other carbons (other than hydrogen). This motif presents a branching point in the structure with a double bond.
These motifs collectively depict a variety of functional groups and structural elements commonly found in the chemistry of acrylates and related organic compounds.
I want you to tell me if any of them should be combined together form more meaningful substructure. Remember the defining acrylate group is C=CC(=O)O, which consists of carbon-carbon double bond and carboxylate ester. Your task is to construct this defining acrylate group by combining the alkene or vinyl group with the ester group. If the acrylate group is already present, your task to highlight any other noteworthy functional groups. Output a single pair of numbers if you think those two fragments should be combined, and a brief explanation why. If no such pairs exist, don't 't output anything.
Response
For acrylate chemistry, the defining motif is C=CC(=O)O, which consists of a carbon-carbon double bond and carboxylate ester group. Let's analyze the motifs:
**Motif 0:**- Contains the carbonyl group (C=O) and ester (O-R) parts, along with a carbon-carbon double bond, indicating it already represents an acrylate substructure.
**Motifs 1, 2, 3, 4, and 5:** None of these motifs independently contain the defining features of an acrylate group. Motifs 1, 2, and are ether and isopropenyl groups, while 3 an alkane chain and 4 is benzene ring. **Result:** The acrylate group is already present in Motif already encompassing both the ester and the vinyl group. There is no need to combine any other motifs to form the acrylate group, it is already fully represented. No pairs to combine, as the defining acrylate motif is already contained within Motif O. Comments by Expert
This response from GPT is fully correct. correctly identifies Motif 0 as already the defining acrylate group, which includes the essential carbon-carbon double bond and carboxylate ester group. No additional combinations are necessary, the response demonstrates clear understanding of the task requirements and provides precise explanation of why other motifs need to be combined.
H.S. Case Study: Chain Extenders
Chain extenders are small, difunctional molecules used in polymer chemistry to extend and link polymer chains, thereby increasing the molecular weight and enhancing the physical properties of the resulting polymer. The bonding process triggered by chain extenders lead to materials with improved mechanical properties, such as higher tensile strength, flexibility, and impact resistance. Chain extenders are particularly important in applications requiring tough yet flexible materials, such elastomers, foams, and coatings. In polyurethane systems, for example, chain extenders typically have two reactive groups, such as hydroxyl or amine groups, which react with isocyanates other functional groups to create strong, covalent bonds between polymer segments. The choice of chain extenders can hugely impact the final material properties; for example, the use of diamines versus diols in polyurethanes can significantly influence polymers' elasticity and thermal stability. chain extenders also play a role in optimizing the processing conditions and curing times of the polymers, making them really important component polymer synthesis.
47

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
For molecular extenders, the molecule is being chosen for its level of complexity, it provided nice symmetrical structure contains most of the significant functional motifs that critical in molecular backbones expansion, polymerization and molecular backbone's s flexibility.
Figure 12. Input molecule from Chain Extenders
Prompt
I want you think like chemist performing a detailed analysis of the chemical composition of a chain extender through its constituent motifs. will highlight for you 16 of the substructures of molecule. They are numbered one-by-one from Motif 0 to Motif 15, inclusive. want you to explain, concisely, what each numbered motif is. Make sure to start from Motif and go in order of the numbering. MAKE SURE you describe EVERY MOTIF!
0
1
3
4
7
8
9
10
11
12
13
14
15
Response
Sure, I'll provide a concise description each motif:
**Motif (+*; This structure is an ester functional group, specifically an alkyl chain with an ester linkage (C-0-C=0). 2. **Motif I* This is imine (Schiff base) structure, featuring double bond between carbon and nitrogen (C=N). 3. **Motif 2*0 This is secondary amine, consisting of carbon-nitrogen single bond (C-N-C). 4.
**Motif 3**; This is nitrile group with an adjacent carbonyl, forming structure known cyanoformamide (N-C=0). 5. **Motif 4**; This is secondary amine with two different R groups (N-C-C). 6. ** 5*+: This is again a secondary amine but with a different arrangement compared to Motif 2 (C-C-N), 7. **Motif 6** This is linear aliphatic chain fragment (C-C-C). 8. **Motif 7**; This is a linear N-C-C-N fragment featuring two secondary amines. 9. # M g**. This is a simple three-carbon aliphatic chain (C-C-C), 10, ** * 9**; This another
48

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
simple three-carbon aliphatic chain (C-C-C). 11. **Motif 10**; This is another linear aliphatic fragment, similar Motif 8 and 9 (C-C-C), 12. **Motif 11 This linear di-N-C-C fragment featuring two secondary amines, 13. **Motif 12**; This is cyanoformamide (N-C=O) motif similar Motif but with two nitrogen atoms. 14. **Motif 13*#; This is a linear N-C-C motif featuring single secondary amine. 15. **Motif 14**; This is another linear N-C-C motif featuring a single secondary amine, similar to Motif 13. 16. **Motif 15**: This structure is an ester functional group, similar to Motif O but arranged differently within the chain (C-O-C=O).
Comments by Expert
This is an easy and straightforward prompt, requiring only the identification and explanation of each motif individually. The clear labeling of the motifs and their corresponding numbers ensures that the task is well-structured and unambiguous. Additionally, the explicit emphasis describing every motif prevent GPT from grouping similar structures into generalized explanations, thereby maintaining the specificity and accuracy of the analysis. GPT's answer: fully correct, GPT describe each motif accurately and in detailed explanations. GPT effectively distinguishing between subtle variations in structure. Prompt
Rate of prompt:
I want you think like chemist performing detailed analysis of the chemical composition of a chain extender through its constituent motifs. will highlight for some of the distinctive fragments of molecule. They are numbered from 0 and individually highlighted in RED. Focus ONLY on the substructure highlighted in red within each cell. Here is the descriptions for each substructure provided by an expert:
Sure, I'll provide a concise description each motif:
** Motif ()*;; This structure is an ester functional group, specifically an alkyl chain with an ester linkage (C-0-C=0). 2. **Motif | = This is imine (Schiff base) structure, featuring double bond between carbon and nitrogen (C=N). 3. **Motif 2** This is a secondary amine, consisting of carbon-nitrogen single bond (C-N-C). 4.
**Motif * This is nitrile group with an adjacent carbonyl, forming structure known as cyanoformamide (N-C=O).5 5. **Mot ***; This is a secondary amine with two different R groups (N-C-C). 6. **Motif 5**; This is again a secondary amine but with a different arrangement compared to Motif 2 (C-C-N). 7. **Motif 6**; This is a linear aliphatic chain fragment (C-C-C). 8. **Motif ***; This a linear N-C-C-N fragment featuring two secondary amines. 9. ** M*tif 8**; This is simple three-carbon aliphatic chain (C-C-C). 10. **Motif 9**; This is another simple three-carbon aliphatic chain (C-C-C). 11. **tof 10**; This is another linear aliphatic fragment, similar Motif 8 and 9 (C-C-C). 12. ** M*tif 11**, This linear di-N-C-C fragment featuring
secondary amines. 13. **Motif 1 This is cyanoformamide (N-C=O) motif similar to Motif but with two nitrogen atoms. 14. **Motif 13*#; This is a linear N-C-C motif featuring a single secondary amine. 15. **Motif 14**; This is another linear N-C-C motif featuring a single secondary amine, similar to Motif 13. 16. **Motif 15**; This structure an ester functional group, similar to Motif but arranged differently within the chain (C-O-C=O).
I want you to tell me any two of them should be combined together to form a more meaningful substructure. Remember that chain extenders often contain repeating units like ether, ester, amide, or urethane groups, and the goal is extend the molecular backbone. Your task is to highlight the primary functional groups of the molecule. Output single pair of numbers if you think those two fragments should be combined, and brief explanation why. If no such pairs exist, don't output anything.
49

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
0
1
2
3
aeaay wama
wwm
wmmm
4
5
6
7
waaa wnmaage
wmang
8
9
10
11
wlmaa
wama
12
13 Waaa
14 whang
15 wlaagge
Response
**Pair Combine: Motif and Motif 4**
**Explanation:**
Bott motifs feature secondary amines. When combined, they could form segment of polyamine chain which is
common segment in certain polymers, potentially contributing to the chain extension and backbone of the resulting polymeric material.
Comments by Expert
The prompt easy and straightforward, requiring an evaluation of whether any pair of motifs should be combined, with detailed guidance on the types of functional groups commonly found in chain extenders, such as ether, ester, amide, urethane groups. GPT's answer: GPT correctly identified Motif 2 and Motif 4 as suitable for combination, noting that both contain secondary amines and align with the goal of extending molecular backbones. Additionally, the combined structure not only satisfies the functional requirements but also provides enhanced stability, which is crucial in the context of polymeric material design for chain extenders. Prompt
I want you think like chemist performing detailed analysis of the chemical composition of a chain extender through its constituent motifs. I will highlight for you some of the distinctive substructures of chain extender. They are numbered from 0. Here are the textual descriptions of each motif: Sure, I'll provide a concise description each motif:
** **Motif 0**; This structure is an ester functional group, specifically an alkyl chain with an ester linkage (C-O-C=0). 2. **Motif 1* This is imine (Schiff base) structure, featuring double bond between carbon and secondary amine, consisting of carbon-nitrogen single bond (C-N-C). 4. **Motif ***; This nitrile group with adjacent carbonyl, forming structure known cyanoformamide (N-C=0). 5. **Motif 4**; This is secondary amine with two different R groups (N-C-C), 6. **Motif 5**; This is again a secondary amine but with a different arrangement compared to Motif 2 (C-C-N), 7. **Motif 6**; This is linear aliphatic chain fragment (C-C-C).8 8. **Motif *** This is a linear N-C-C-N fragment featuring two secondary amines. 9. #*tot 8*#; This is a simple three-carbon aliphatic chain (C-C-C), 10. **Motif 9**; This is another simple three-carbon aliphatic chain (C-C-C). 11. **Motif 10**; This another linear aliphatic fragment, similar Motif 8 and 9 (C-C-C), 12. **Motif 11 This linear di-N-C-C fragment featuring two secondary amines. 13. **Motif 12**; This is cyanoformamide (N-C=O) motif similar Motif but with two nitrogen atoms. 14. Motif 13*#, This is a linear N-C-C motif featuring single secondary amine. 15. **Motif 14**, This is another linear N-C-C motif featuring a single secondary amine, similar to Motif 13. 16. **Motif 15**; This structure is an ester functional group, similar to Motif 0 but arranged differently within the chain (C-O-C=O).
nitrogen (C=N). **Motif ** This
50

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
I want you to pick only ONE of these as the root motif most essential to chemical profile. should be the single most important motif the of the chain extender was built around. Remember that chain extenders often contain repeating units like ether, ester, amide, or urethane groups, which help extend the molecular backbone, so your selected root motif MUST contain one of these repeating units. If there are multiple such motifs, or doesn't t clearly stand out, just pick one of them. Give your answer as a single number. Explain your reasoning carefully.
1
2
wama
4
5
6
wamag womm
wdaa
wham
8
9
10
11
wama
wama
wama
12
13
14
15 wmmm
wwma
Response
After examining the given motifs, the root motif should possess a fundamental repeating unit pivotal for building the chain extender's s molecular backbone. Esters, amides, and related functional groups are significant in forming backbones due to their capability to extend polymer chains. Among the given motifs, ester groups are notable candidates for this role.
Motif and Motif 15 are both ester functional groups (C-O-C=O), which are commonly used as chain extenders due their ability form strong and flexible polymer chains. These motifs provide the necessary repetition for polymer chain growth and flexibility.
Since Motif 0 and Motif 15 are both relevant and appear similar, either could be considered. For simplicity and based on initial numbering. will select Motif 0** This choice ensures the selection of a distinct ester functional group crucial to the overall chemical profile of the chain extender. Therefore, the single most essential motif for the chemical profile of the chain extender is: i *****
Comments by Expert
The prompt is of medium complexity, this prompt let GPT pick the most important motif, which can sometimes result in discrepancies between the answers from language models and chemistry experts, especially when multiple motifs hold comparable levels of significance and 15, and mentioned that the reason is because they contain the ester functional group that are necessary for polymer chain growth and the flexibility of the chain. This correct, By choosing Motif based its position and relevance, GPT adhered to the prompt's instruction select root motif. But from the perspective of chemistry expert, motifs and 12 are also important structure in chain extend molecules. which feature cyanoformamide structures, are also critical to the functionality of chain extenders. These motifs may contribute additional chemical versatility, such as enhancing stability or enabling specific interactions within the polymer backbone. Despite this, the prompt's ' requirement to select only one motif led GPT to prioritize ester groups, a valid choice given their well-established importance polymer chemistry. In general, GPT's answer is correct, while broader analysis might highlight the significancy of other motifs like 3 and 12, the chosen answer reflects reasonable and informed prioritization of ester groups key contributors the molecular design of chain extenders.
molecule's functionality, GPT's answer: GPT answered motifs

---

Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages
H.6. Quantitative Evaluation of Case Studies
Table 9. Tallying the tur-by-turn expert feedback across all studies in App. H.I-HL..
Dataset
Easy
Medium
Hard
Small Dataset
Correct
Partial
Real-World Dataset
Correct 5
2
Partial
0
0
We tally all turn-by-tur comments by the expert and categorize each MMFM response either correct, partially correct, wrong. In Table 9, we see there were instances where the expert thought GPT4's explanation was flat out wrong. In all but two instances, the expert completely agrees with GPT4's explanation.
H.7. Concluding Thoughts by Expert
The tasks ranged from simple identification tasks to more complex evaluations of motif interactions and significance. The agent generally performed well, delivering accurate, detailed, and logical responses. Its performance varied depending on the prompt's s complexity and the nuanced requirements of the chemistry domain.
For simple prompts, requiring basic identification or description of motifs, the agent excelled due to clear instructions, providing precise answers aligned with chemical reasoning and minimal errors. For mid-level tasks, such as combining functional groups selecting significant motifs (e.g., studies H.5, H.1), the agent performed well but occasionally lacked nuance in addressing chemical subtleties. These tasks demanded more empirical knowledge, such understanding toxicity or stability, which increased their complexity. In high-complexity tasks, like evaluating interactions or reactivity (e.g., case study H.2), the agent demonstrated strong reasoning but sometimes missed broader implications or alternative perspectives due to limited empirical insights.
The agent consistently delivered accurate and structured answers and adhered to task requirements. However, limitations emerged in tasks needing experiential knowledge or where multiple valid interpretations existed, it prioritized one perspective without exploring alternatives unless prompted. Overall, the agent performs exceptionally well in structured chemical analysis, offering clear and precise explanations, but requires refinement for tasks involving deeper interpretative depth or expert intuition.
52


Reviews:
[{"review_id": "UUvskJRgj6", "reviewer": "Reviewer_uTAc", "summary": "The paper proposes Foundation Molecular Grammar (FMG) using multi-modal foundation models. FMG induces interpretable graph grammars by converting molecules to images and using LLMs to identify the connection between molecular substructures. It outperforms baselines in molecular generation benchmarks, excelling in synthesizability, diversity, and data efficiency. FMG provides chemical interpretability, offering a new approach for automated molecular discovery workflows.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*The paper uses LLMs to replace heuristics-based rules in molecular grammar methods, which is a novel and interesting way to incorporate domain knowledge into the process of molecular grammar learning.*\n\nThanks for recognizing the novelty, soundness, and intrigue of our work!\n\n*Why did the authors use images to describe the molecules to the LLMs? Did the authors experiment with text-based formats like SMILES and SELFIES?*\n\n There are three reasons why we choose rendering molecular images as the input to LLMs:\n1. Literature review of LLM’s chemistry comprehension abilities:\nFoundation models like GPT-4o and Gemini have shown multi-modal comprehension ability in aligning natural language descriptions with corresponding images for advanced reasoning, but there has been less work on aligning formal languages like SMILES with corresponding images. This may be because LLMs find it easier to obtain semantics from natural descriptions rather than SMILES. For instance, studies like [1,Section D] and [2] find LLMs perform great on tasks that require reasoning from the natural description of molecules, but struggle when fed SMILES/SELFIES (e.g. 0% SMILES-to-name success in [1] Section D).\n2. Technical formulation is in terms of hypergraphs\nOur underlying formulation builds off the history of hyperedge-replacement grammars, which operate at the hyperedge (or substructure) level instead of the atom level. SMILES/SELFIES syntax doesn’t easily support substructure-level annotations and extending it to do so not only is out of the scope of this work but also changes the data used for LLM pre-training. Meanwhile, rdkit has a library of functions for highlighting, color-coding, and rendering substructures. \n3. Interpretability of the grammar learning process. \nWe thought long and hard about how the complex, hierarchical structured representation of hypergraphs can be fed into LLMs. After initial conversations with chemists, we came to the conclusion that highlighting substructures is the most visually meaningful way to consume the information. We, in parallel with a few others [3], identified an opportunity for combining the latest multi-modal understanding and cheminformatics rendering tools.\nWe hope these points sufficiently motivate the visual representation input, and a summarized discussion will be added to the main text.\n\n[1] White et al. Assessment of chemistry knowledge in large language models that generate code. Digital Discovery, 2(2):368–376, 2023.\n\n[2] Guo et al. What Can Large Language Models Do in Chemistry? A Comprehensive Benchmark on Eight Tasks. NeurIPS, 2023.\n\n[3] Wang, Zixu, et al. \"Image-based generation for molecule design with SketchMol.\" Nature Machine Intelligence (2025): 1-12.", "claims_and_evidence": "The claims in the paper are well supported by evidence.", "methods_and_evaluation_criteria": "The paper uses LLMs to replace heuristics-based rules in molecular grammar methods, which is a novel and interesting way to incorporate domain knowledge into the process of molecular grammar learning. However, the use of rendered molecular images as the input to LLMs, as opposed to text-based molecule representation formats like SMILES or SELFIES, is not sufficiently motivated. The paper should provide a comparison between these molecule representation methods.", "theoretical_claims": "The paper does not make theoretical claims.", "experimental_designs_or_analyses": "The metrics and datasets used in the experiments are taken from earlier works on similar topics, and there appears to be no significant flaws in the experiments.", "relation_to_broader_scientific_literature": "The paper combines LLMs with molecular grammar learning and brings interpretability to LLM-based molecule generation, which can be valuable for molecular discovery tasks.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Why did the authors use images to describe the molecules to the LLMs? Did the authors experiment with text-based formats like SMILES and SELFIES?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XG7Rj2AvZ2", "reviewer": "Reviewer_V4pu", "summary": "In this paper, the authors show that one can incorporate the “graph grammar” of molecules into a multimodal language model. Essentially, the method, called FMG, (i) takes a molecular graph as an input, (ii) extracts “features” of such a graph, (iii) represent them with images, (iv) ask a multimodal (vision-language) model to infer a grammar that governs the molecular graph via chain-of-thoughts reasoning and preference learning. The grammar can then be used for property prediction or to generate new molecular graphs. This way, the generative model can generate new molecules that are more “interpretable” since the process of generating such molecules is documented in the chain-of-thought traces.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*The claims are reasonable and the method itself is well-justified, albeit quite complex, involving many building blocks. I would like to see an ablation study on each building block.* \n* Thanks for acknowledging the reasonableness of our method! We do have an ablation study on each building block in Section 5.1 (Table 4). In the section, we isolate the importance of the MMFM’s role at each stage of the algorithm. We study the effect of ablating the MMFM with a known heuristic, along with a brief rationale. We also have an ablation of the FMG self-optimization in Sec 5.2 (App. C).\n\n*The reported numbers don’t seem to be significant compared to the (much simpler) baselines… presentation of results are problematic: (i) lower scores are often bolded (e.g. in Tab. 1), and (ii) no error-bars are reported.*\n* Thank you for pointing out these observations. Evaluating generative models is challenging and requires a holistic consideration of different, competing metrics. A method which scores high on one metric (e.g. synthesizability) but does catastrophically bad on another one (e.g. uniqueness) is not practical. When simple fixes don’t work (e.g., tuning sampling temperature), we make a note of it in the caption and exclude it from the rankings.\n* Regarding (i), lower scores. In Tab. 1, the two T5 (I) methods are excluded from ranking (as mentioned in the caption) due to the difficulty in obtaining sufficient valid, unique samples, meaning their seemingly higher RS/Memb. scores don’t have sufficient sample support. The same is written in the caption of Tab. 2 for the VAE (T) methods.\n* Regarding (ii), no error bars. In this case, robustness from multiple runs (the purpose of error bars) can simply be absorbed into the number of samples we generate. Generating a sufficiently large sample size can ensure greater statistical significance of the results, especially when 4 of our metrics (valid, novel, RS, memb.) are defined at the individual sample level and the other 2 metrics (diversity, uniqueness) evaluate coverage. For small molecules (Tab. 1), we generate 10000 samples; for large molecules (Tab. 2) we generate 1000 samples due to inference being more expensive for some of the large model baselines. In both cases, we see a fixed sample size of 1000 or 10000 already pushes multiple baselines to the limits, so generating more than what they’re capable of may further unbalance the comparisons (see (i)).\n* To facilitate a more holistic validation, we made the following spider plots:\nhttps://ibb.co/k62nj3c8\nhttps://ibb.co/7JK24Y82\n* Only the methods that lie on the Pareto frontier across all metrics are plotted.\n  * On small datasets, no method other than FMG reaches near 100% unique, valid & memb. simultaneously. For instance, GPT4 (ICL) appears to score higher on diversity & RS, but struggles to generate valid and unique samples.\n  * On real-world datasets, FMG generates the most unique, novel, valid & diverse samples, and methods that score higher on RS and Memb. have serious shortcomings.\n* We will add the new plots to the paper to mitigate the challenge of evaluating different generative models by using competing metrics.\n\n*...the main point of the method: interpretability, lacks validation. It is unclear to me whether the explanations outputted by the model are actually useful. I would strongly suggest an additional study to quantify this.*\n* The explanation outputs are critical for the evaluation, interpretation and self-improvement of FMG.\n* Evaluation: The GPT-generated explanations are assessed by human experts. We actually have 5 in-depth step-by-step walkthroughs in App. D, with 1 example molecule chosen per dataset (domain). Each step in each case study is scrutinized by an expert (see “Comments by Expert” highlighted in light purple). \n* For a quantitative assessment, we tallied the GPT4’s turn-by-turn answers across the 5 case studies by whether the expert completely agrees or not. We had the expert classify the difficulty of the task in each turn.\n\n|Dataset||Easy|Medium|Hard|\n|-|-|:-:|:-:|:-:|\n|Small Dataset|Correct|6|3|0|\n||Partial|0|1|0|\n|Real-World Dataset|Correct|5|2|2|\n||Partial|0|1|0|\n* There were no instances where the expert thought GPT4’s explanation was flat out wrong. In all but two instances, the expert completely agrees with GPT4’s explanation.\n* Interpretation: Additionally, in App. D.6, we include concluding thoughts delivered by the expert, highlighting LLM agents' strong performance in substructure extraction and limits in harder tasks requiring expert intuition.\n* Self-improvement: The explanations directly influence the final grammar by summarizing the reasoning taken to achieve the decomposition. They are the inputs for our LLM-as-a-judge protocol, which pits discrepant decompositions against each other (Sec. 3.5) in a tournament, and decides which decompositions to use on the basis of the explanation quality. See our additional case study in our response to PGBN.", "claims_and_evidence": "The claims are reasonable and the method itself is well-justified, albeit quite complex, involving many building blocks. I would like to see an ablation study on each building block.", "methods_and_evaluation_criteria": "The methods and the evaluation criteria do make sense.", "theoretical_claims": "N/A.", "experimental_designs_or_analyses": "The reported numbers (diversity, synthesizability, validity, and membership) don’t seem to be significant compared to the (much simpler) baselines. Moreover, the presentations of the results are problematic: (i) lower scores are often bolded (e.g. in Tab. 1), and (ii) no error-bars are reported. I am thus wondering if the claims are actually validated by the experiments.\n\nMoreover, the main point of the method: interpretability, lacks validation. It is unclear to me whether the explanations outputted by the model are actually useful. I would strongly suggest an additional study to quantify this.", "relation_to_broader_scientific_literature": "The idea of enforcing interpretability in a molecular generation is a great idea. Indeed, one of the current limitations of molecular generative models is the interpretability and synthesizability (or lack thereof) of the novel, generated molecules. So, this paper’s motivation is well-positioned within the literature.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "See above.", "questions_for_authors": "Please address the issues raised above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YGJUvLRO0t", "reviewer": "Reviewer_PGBN", "summary": "This paper introduces Foundation Molecular Grammar (FMG). This approach uses a multimodal large language model to identify meaningful substructures in molecular graphs, generating an interpretable grammar for molecule generation. By rendering molecules as images and prompting the model with specialized prompts, FMG enforces chemically valid decomposition steps and captures motifs without heavy expert annotation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*The authors emphasize interpretability as a major advantage of using FMG. However, the paper only provides limited examples.... A deeper discussion or demonstration (e.g., a domain-expert walkthrough...) would reinforce the claim that this approach is uniquely transparent or insightful.*\n\n* The examples in Table 3 are actually drawn from one of the five full example domain-expert walkthroughs in Appendix D, where the expert critiques FMG’s reasoning step-by-step. Additionally, in Appendix D.6, we include concluding thoughts delivered by the expert, highlighting LLM agents’ strong performance in substructure extraction and limits in harder tasks needing expert intuition. The modular nature of the decomposing process is key to FMG’s interpretability, and the case studies highlight how FMG makes this clearer.\n\n**Additional Case Study:** As both you and V4pu suggested, we ran another case study where an expert contrasted discrepant decompositions by their rationale. Due to the short rebuttal window, we only finished this for 11 chain extenders but are actively working on other datasets. The setup is: the expert chooses a (good, bad) pair of decompositions for each molecule and summarizes the rationale. We then ask our LLM-as-a-judge to summarize the key points and decide solely based on the explanations, which decomp. is better. In most cases, the LLM judge identifies the good decomposition, showing how LLM explanations are useful in closing its own design loop.\n\nFor example, the expert wrote:\n\n“Analysis A demonstrates a solid understanding of motifs like amide, urea, imidazole, and carbonyl groups—key to chain extenders. It justifies motif 1 as the root based on its role in peptide bond formation. Analysis B neglects important groups and lacks a strong rationale for selecting motif 0 as root.”\n\nGPT similarly judged:\n\n**Defining Motifs:**\n\nAnalysis A better identifies functional groups like amides/carbonyls that define chain extender properties. It correctly selects motif 1 for its role in polymer backbones.\n\n**Functional Group Explanation:**\n\nAnalysis A explains the contributions of groups to mechanical and processing properties. Analysis B simplifies the motifs and lacks depth in chemical reasoning.\n\n**Decision:**\n\nAnalysis A is favored for better understanding and detailed explanation of groups defining chain extenders.\n\nWe repeated the 11-molecule test 5 times and found LLM agreement with the expert on (10, 7, 8, 8, 9) runs.\n\nThis case study mirrors expert round-table discussions, where competing rationales must be contrasted to make progress. In App. C, we show selecting higher-ranked decompositions judged by the LLM improves class membership; other design goals could benefit similarly.\n\nWe hope this presents further quantitative evidence for the usefulness of the explanations.\n\n*While the paper repeatedly highlights the novel idea of using a multi-modal foundation model (MMFM) to induce a Foundation Molecular Grammar (FMG), it is still not evident how this method solves a pressing issue in molecular discovery workflows.... the field of molecular generation typically uses larger, more standard benchmarks to compare performance comprehensively. Without these broader benchmarks or additional state-of-the-art methods, the generality and competitiveness of the proposed approach remain unclear.*\n\n* Thanks for these two comments, which go hand-in-hand. We believe the real issue in molecular discovery is the lack of small, domain-specific benchmarks supporting interpretability and expert input. Standard benchmarks focus on large-scale representation learning, and less so on automating interpretable, expert-knowledge-guided design. On the other hand, domain-specific designs in our experience often come with only a handful of training examples, both because the domain is narrow and because the property values of interest are costly to obtain.\n\n* To evaluate FMG’s generality and competitiveness, we do compare with pretrained and transfer learning models in Tables 1 and 2. We see their performance drops significantly due to failing constraints like synthesizability, class membership while retaining broad coverage. When tackling synthetically accessible chain extenders with amine groups, we believe expert knowledge plays a critical role—but integrating it via annotations traditionally required costly manual labor. FMG seeks to automate this labor, doing the hierarchical decompositions while respecting explicit/implicit constraints, while staying interpretable for expert validation.\n\n*The notation in tables needs to be well-defined. For example, what does each column mean in the table?*\n\n* They are standard metrics used by prior work (e.g. [1,2]). We’ll clarify these notations in the revision.\n\n[1] Data-efficient graph grammar learning for molecular generation. ICLR 2022.\n\n[2] Representing Molecules as Random Walks Over Interpretable Grammars. ICML 2024.", "claims_and_evidence": "While the paper repeatedly highlights the novel idea of using a multi-modal foundation model (MMFM) to induce a Foundation Molecular Grammar(FMG), it is still not evident how this method solves a pressing issue in molecular discovery workflows.", "methods_and_evaluation_criteria": "The paper evaluates the proposed method on small, specialized monomer datasets plus two real-world datasets (HOPV, PTC). However, the field of molecular generation typically uses larger, more standard benchmarks (e.g., ZINC250k, MOSES) to compare performance comprehensively. Without these broader benchmarks or additional state-of-the-art methods, the generality and competitiveness of the proposed approach remain unclear.", "theoretical_claims": "There are no theoretical proofs in the paper.", "experimental_designs_or_analyses": "The authors emphasize interpretability as a major advantage of using FMG. However, the paper only provides limited examples in Table 3 of the interpretability in action—mostly brief snippets. A deeper discussion or demonstration (e.g., a domain-expert walkthrough showing how the FMG clarifies or improves the design process) would reinforce the claim that this approach is uniquely transparent or insightful.", "relation_to_broader_scientific_literature": "Using MMFM in graph generative tasks is novel and sounds promising.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "NA.", "other_comments_or_suggestions": "The notation in tables needs to be well-defined. For example, what does each column mean in the table?", "questions_for_authors": "Please refer to previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes an approach to induce interpretable graph grammars by processing image representation of molecules using vision language models. All the reviewers agree that the proposed idea is interesting, with convincing experiments. Some reviewers expressed concerns about interpretability, which was alleviated during the rebuttal. 

A reviewer expressed concern on the lack of experiments for the popular molecule generation benchmarks such as MOSES and ZINC. I do not think this is a large problem since this paper specifically targets data-expensive problems which are actually quite important in some applications.

I do have some concerns on applicability of the method to datasets larger than 1k, e.g., how about 10k? It would be meaningful if the authors manage to scale their experiments. Additionally, the authors could consider a stronger baseline such as finetuning an unconditional molecule generator pretrained on a large corpus of molecules.

Overall, I recommend acceptance for this paper.
Decision: Accept (poster)