Graph Diffusion Transformers are In-Context Molecular Designers
Authors: Gang Liu
Jie Chen
Yihan Zhu
Michael Sun
Tengfei Luo
Nitesh V Chawla
Meng Jiang
Venue: ICLR 2026 Poster (2026)
Keywords: Inverse Molecular Design
In Context Learing
Diffusin Models
Transformers
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: In-context learning lets large models adapt to new tasks from a few demonstrations, but it has shown limited success in molecular design, where labeled data are scarce and properties span millions of biological assays and material measurements. We introduce demonstration-conditioned diffusion models (DemoDiff), which define task contexts through molecule–score examples instead of texts. These demonstrations guide a denoising Transformer to generate molecules aligned with target properties. 
For scalable pretraining, we develop a new molecular tokenizer with Node Pair Encoding that represents molecules at the motif level, requiring 5.5$\times$ fewer nodes.
We pretrain a 0.7B parameter model on datasets covering drugs and materials. Across 33 design tasks in six categories, DemoDiff matches or surpasses language models 100–1000$\times$ larger and achieves an average rank of 4.10 compared to 6.56–17.95 for 19 baselines. These results position DemoDiff as a molecular foundation model for in-context molecular design.

Published as a conference paper at ICLR 2026
GRAPH DIFFUSION TRANSFORMERS ARE IN- IN-CONTEXT MOLECULAR DESIGNERS
Gang Liu', Jie Chen², Yihan Zhu', Michael Sun
Tengfei Luo-, Nitesh V. Chawla', Meng Jiang'
Unniversity of Notre Dame 2MIT-IBM Watson Al Lab, IBM Research (gliu7, mjiangg] @nd.edu
MIT CSAIL
ABSTRACT
In-context learning lets large models adapt to new tasks from few demonstrations, but has shown limited success in molecular design, where labeled data are scarce and properties span millions of biological assays and material measurements. We introduce cemmonstration-condiitoon define task contexts through molecule-score examples instead of texts. These demonstrations guide a denoising Transformer to generate molecules aligned with target properties. For scalable pretraining, we develop new molecular tokenizer with Node Pair Encoding that represents molecules at the motif level, requiring 5.5x fewer nodes. We pretrain 0.7B parameter model on datasets covering drugs and materials. Across 33 design tasks in six categories, DemoDiff matches or surpasses language models 100-1000x larger and achieves an average rank of 4. compared to 6.56-17.95 for 19 basclines. These results position DemoDiff as a molecular foundation model for in-context molecular design.
diffusion models (DemoDiff), which
INTRODUCTION
In-context learning (ICL) is the emergent capability of large models to infer task-specific concepts from a few demonstrations (Xie et al. 2021]. 1. ICL has been studied in large language models (LLMs), but was found less effective for molecular design than specialized methods (Liu al. 2024b). These specialized models often depend on extensive Oracle calls (Gao et al., 2022) or large labeled datasets beyond what context examples provide. Molecular tasks, however, involve millions of types, many with only few labeled examples (Zdrazil et al., 2024). Such examples are enough to form task contexts but insufficient to train a new model. This trade-off motivates our in-context molecular design, which combines the flexibility ICL with the efficiency of molecular domain knowledge.
Molecular structures and properties are discrete graphs and numbers with varying scales and units. Directly adapting the autoregressive framework from LLMs is infeasible (Brown et al. (0200 for in-context molecular designs, as the input and output of language data are text in sequential order. Diffusion models show promise for molecular structures (Vignac et al. 2022). and Graph Diffusion Transformers (Graph DiTs) are effective for modeling their joint distribution with properties (Liu al.) 2024ch. However, Graph DiTs have been studied with most five properties represented in single vector. In practice, molecular properties span millions of assays in biology, including functions, binding, ADME, and toxicity, as well as material properties such as gas permeability, thermal conductivity, and glass transition temperature (Figure D. Representing all properties in one-hot vector with millions of dimensions inefficient, produces sparse pretraining data since many assays have fewer than ten labels, and limits generalization to unseen properties in downstream tasks.
Instead of a property vector with a large embedding table, we use demonstrations to define the task context for molecular design. As shown in Figure I] the demonstrations consist of a set of molecules with scores in [0, 1] and molecular design is framed as a query for the target score of 1. Molecules in the context do not follow strict order, and their scores serve relative positions the target, functioning as a replacement for position IDs in Transformers. (Xie et al., 2021) described ICL in LLMs as implicit Bayesian inference over latent concepts expressed by examples in the prompt. Similarly, each task Figure shares the concepts defined by the joint distribution of molecules and their scores. The denoising Transformer in the Graph DiT attends to the context, implicitly

---

Published as a conference paper at ICLR 2026
Task and Descriptions
Pretraining with Millions of Tasks
Downstream
Funetional: collular toxicity chinese homster atter of compound exposure
ADME: Cytotosicit human HS couttr
Binding: in viro binding affirity towarrs (Aipha- 4120beto
Materialx:
Finetuning the structure Csmerrinib prystcochomica properties (I LogF)
conductivity [N0l)r for polymeric materiale
nicttin acatylchcline receptor fractions forwbesin
(046,
( 0.50
0.32,
0.19,
(002,
0.71, Jo Pp)
(0.59,
0.69,
0.64
(o.se,
$500
0.81, wf
0.79.
0.80,
0.73,
0.70
0.94, 22p29)
0.85,
0.B8,
0.75,
0.75,
o
Demos
Query
(St)
..00
1.00
1.00
1.00
1.00
Figure 1: In-context molecular design with DemoDiff. Each demo is defined as a score mooeeulee pair, and set of them forms the task context as conditions. After pretraining on large and diverse tasks, DemoDiff serves foundation model for designing molecules in new task contexts. Scores represent relative distances to the target and are converted from raw labels, as shown in Section 3.3
extracts concepts, and uses them to guide the reverse process refine the structure. An example generation trajectory is shown in Figure 7 A simple way to represent the task concept is to use positive demonstrations, such molecules close the target active in the assay. However, positive examples alone may be insufficient, as they can overlap across tasks due to factors such as task relatedness (c.g.. activity in non-small-cell lung cancer but across different cell lines) or due to sampling bias when the set of positives is extremely sparse (e.g., only one positive example shared by two tasks). To address this, we form the task context using not only positive but also medium and negative examples, providing more complete representation of the task concepts.
With these task contexts, we propose demonstration-condiii pretrain a 0.7B model with Graph DiT as the backbone, using over 140 H100 GPU days. To support efficient pretraining, we introduce a molecular tokenizer trained with Node Pair Encoding (NPE) for lotif-level representation. On average, it reduces the number of nodes by 5.5x compared to atom-level representations Frigure(yJand]n selects frequent motifs to construct vocabularies. Motifs connected by directed edges that preserve bond types and attachment rules, ensuring lossless reconstruction. Graph DiT's naturally use this motif-level representation and attend motif semanties for denoising. For pretraining, we construct dataset of over 1.6 million tasks from 155K unique properties and one million molecules. It combines ChEMBL for drugs (Zdrazil al. 2024) and multiple polymer data sources for materials (Otsuka et al. 2011 Thortoo et al.) 2012 Kuenneth et al. 2021). For ICL, we propose consistency score as confidence measure of whether a generation aninn more closely with higher-scoring molecules in demonstrations, effectively filtering out false positives in generation (Section F.3
diffusion models (DemoDiff) and
The tokenizer iteratively merges neighboring nodes and
We evaluate DemoDiff on 33 design tasks across six categories. With 0.7B parameters, it matches or surpasses LLMs 100-1000x larger in generating diverse, high-scoring molecules. Compared to 19 baseline models (average ranks 6.56-17.95), DemoDiff ranks 10, demonstrating its strength as molecular foundation model. The new molecular tokenizer further improves representation efficiency.
2 PRELIMINARIES
2.1 IN-CONTEXT LEARNING WITH DEMONSTRATIONS
In ICL, the context is a set of demonstrations C {e where each e; is an input-label pair. Following (Xie et al. 2021), assume C reflects latent concept 0 E θ from a family of concepts
2

---

Published as a conference paper at ICLR 2026
+. For example, for paragraph about Albert Einstein, the latent concept may be biography. Given query Q, foundation model with ICL generates an output X by marginalizing over #:
p(x C. Q) -
S. 1c.0)= 10,,,/, (,C, c.Q)
(I)
Here (C, Q) form the prompt. If (0 Q) concentrates on the prompt concept with more demonstra- tions, then the model identifies and applies that concept through marginalization. ICL can thus be implicit Bayesian inference. All context, query, and outcomes are texts in language modeling. In inverse molecular design, we have molecule-score pairs as demonstrations. They capture the latent task concept, with semantics like the task descriptions in Figure [] Q is the target score, and X is the molecule to be designed. We focus a new molecular foundation model for ICL.
2.2 MOLECULAR DESIGN WITH GRAPH DIFFUSION TRANSFORMERS
Molecules are discrete graphs X = (A, B), where A denotes the set of atoms and B the set of bonds. These structures are commonly modeled using discrete diffusion processes (Vignac et al.) 2022 Liu al. 2024c). Graph DiTs concatenate atom and bond features to X into the input format of standard Transformers. Given X, for each atom E A with d, neighbors, Graph DiT's define a token as x = {a;, (by )/==), where bij- E B encodes the bond type (single, double, triple, or none). Each token is represented by a feature vector E R formed by concatenating the one-hot encoding of the atom type and the connection types to all other atoms (either a bond type or a null type indicating no connection). Discrete diffusion has transition matrix Q, initialized based the frequency of atoms and bonds in the training set. At step f, [Q' = {x}) x for i, E F].
The forward diffusion with Q is: qx" 1 ))- = Cat ; p xf x²). where Cat(x; p) denotes two separate categorical sampling for atoms and bonds with probabilities from p. Starting from the original data x = x", we have q(x x²) Cat (x'; p = x² Q'),, where Q' = Ilise Q' The
forward diffusion gradually corrupts data points. When the total timestep T large enough, q(x")) converges to stationary distribution, The reverse process samples from q(x T' and gradually removes
Q'-1 Given multiple properties {c, ) the denoising model approximates pe (x x² xxxx" {c: 1
noise. The posterior distribution g(x'=1 ) is calculated as qxx |x", x") x (Q))
under property conditions. This model is trained by minimizing the negative log- fog-likelihood for x":
LpM = Ex (e [x() [- log (x") x", C1, 1,,1,22 C2. (2, G.)l
In Graph DiTs [Liu et al. 2024c), the constraints {c:} } are numerical or categorical property values. In this work, we explore them as demonstrations for in-context learning.
3 LEARNING DIFFUSION MODEL WITH DEMONSTRATIONS
Figure
shows the generation process DefoDiff, combining motif-based representation (Sec-
tion 3.1 with graph diffusion transformers for in-context molecular generation (Section 3.2
3.1 MOLECULAR GRAPH TOKENIZATION WITH NODE PAIR ENCODING
More demonstrations help capture the latent task concept and empirically useful (Bertsch et al. 2024). However, prior work (Liu et al. 2024c) uses atom-level molecular representations, similar to modeling text at the character level, which fundamentally limits the number of examples in context. For efficient representation, where Á A and B C B define connected substructure. A molecule becomes collection of disjoint motifs M = {m=]" J such that: (1) A n Aj 9 for all i + j: (2) Ui= A; = A. Then, each edge Ejy E directed from a source motif m; a target motif m,. with two associated attributes: (1) bond type, and (2) attachment index. indicating the atom within m, from which the bond originates. This abstraction induces a tokenizer with two functions. They are tokenizer, encode : X = (A, B) + + X (M, that compresses the atom-level graph into motif-level form and tokenizer. decode : X = (M, E) + X (A, B) for reconstruction. The tokenizer uses vocabularies: M for motif types and for edge types. It starts with the 118 atom
merge frequent sub-molecula patterns motif m (Á, B) C X,
3

---

Published as a conference paper at ICLR 2026
Sewnu 0.3
Score: 0.8
Tokenizer. encode
Graph DrT
Tokenizer. decode
+
Demos conditions
Generation
Random Naise, paired with score
Stat,
Supe 123
Sugrr
Sap
Supa Of Sura
Trans- formers
Sucae Of
Demo Tokens
[S 6)
i 8)
diffusion generation. In the reverse process, DemoDiff starts from random noise and denoises molecules conditioned on molecule-score demonstration pairs. A tokenizer encodes each molecule into motif-level representation. Motifs are connected by edges with two attributes shown in square brackets: (1) bond type and (2) attachment index. The input to the transformers consists of tokens from the denoised molecules and the context of demo tokens.
Figure 2: Demonstrationnconniiiii
types from the periodic table and one "** for the polymerization point, which form the initial motifs in M. This guarantees that, in the worst case, a new molecule can still be represented at the atom level. In each iteration, the tokenizer merges the most frequent neighbors until no further merge found in M, then proceeds with the corresponding connections between motifs.
To construct M, existing methods, such as BRICS (Degen et al. 2008 or molecular grammars (Sun et al. 2025 2024), rely on domain-specifie heuristics based chemical reactions or expert knowledge. The resulting vocabularies are independent of the pretraining data, often missing frequent propose Node Pair Encoding (NPE), Trequency-based algorithm for molecular graphs inspired by BPE, as outlined in Algorithm [ We initialize M with elements from the periodic table and polymerization points "** NPE iteratively performs three steps: (1) Neighborhood merge: For each molecule X E D in the dataset D and current M, we identify candidate motifs by merging adjacent substructures that appear Mt (2) Frequency selection: The most frequent candidate motif is selected and added to M; (3) Graph update: Each X E D is updated by replacing instances of merged motif pairs with the new motif.
motifs. To address this limitation,
Constrained NPE: The standard NPE may produce multiple directed edges from a motif m, to another my when decomposing ring structures (e.g., aromatic rings). It leads ambiguity during decoding since eaj does not uniquely determine the attachment index within my. To address this, we introduce constraints such as rings into NPE at two stages. During initialization, we traverse each molecule to identify its set of maximal connected rings, denoted R, compute their frequencies, and include the top- Kring most frequent rings in the initial vocabulary M. During motif merging, any E .M is merged with ring R M complete unit, rather than merging individual atoms within r. This strategy integrates frequent rings into the vocabulary while preserving atom- level representations for rings, avoiding reconstruction ambiguity. We compare NPE with fingerprints (Rogers & Hahn 2010) and virtual nodes (Hwang et al. 2022) in appendix A
Using NPE, a molecule represented as n tokens {-:]__p> where x; = (m,, {ei (mm,((es)) denoting motif and 6ij the associated edges. We set the motif vocabulary size to K 3000 (Kring = 300), with details and analysis provided in appendix C.2 As shown in Figure 2 an example input with 38 atoms can be compactly expressed using four motil.. An empirical comparison of atom- and levil representations over 1 million pretraining molecules is given in Figure 3b with an average compression ratio of 5.446 2.569, reducing the median count from 30 atoms 5 motifs.
) with m,
3.2 IN-CONTEXT LEARNING WITH GRAPH DIFFUSION TRANSFORMERS
We construct the dataset D = {C,, Qú Xi) Nommn for pretraining, where each task consists of a context of molecule-score pairs C., query score Q,, and a target molecule X,. To pretrain DemoDiff
4

---

Published as a conference paper at ICLR 2026
for in-context inverse molecular design, replace the property conditions in Eq. 2 with C and Q:
pretrrin Rox Ryyxx x²) [-log (x") x² C, C.Q
With large and diverse pretraining data and scalable Transformers (Peebles & Xie 2023). DemoDiff learns to infer the latent task concept to generate the target molecule and can serve as a foundation model for ICL. In task, it performs implicit Bayesian inference over diffusion trajectories.
Given a query Y, demonstrations C, are divided into positive, medium, and negative groups. For a generated molecule X, compare its fingerprint- based similarity with these groups to assess whether it follows the relation pos > med > neg. This yields a consistency score that measures how well the generated molecule aligns with the relative relations in the context. In experiments (Section 4.1, we use this score to select high consistent generations before conducting the final evaluation with Oracles.
ICL with Context Consistency (appendix B.4):
3.3 MODEL DESIGN AND PRETRAINING
Moo Count
0.20
Mottt Gount
Atom Median: 30.0
Motif Mediam:
10
0.15
Frequency
30
0.05
109
10a
#
Proporties/Aseays: 155,150
10
10
10
10
0.00
10
20
30
30
50
Kank
Number of Moti//Atom
(a) Property rank vs. frequency D.
(b) Node counts (n S 50; full in Figure
Figure 3: Pretraining data statistics for property ank-frreuency and node count density.
Model Designs: Figure illustrates the model architecture. For the i-th task the pretraining set (C,, X., E D. the property score Y; is scaled within [0, 1]. providing positional signals for both demonstration molecules and the target. These scalar values are encoded using Rotary Position Embedding (RoPE) (Su et al. 2024). DemoDiff uses a tokenizer to process the atom-level representation and defines a maximum context length for the number of motif tokens. The context includes the target along with many demonstration tokens fit within the target length. Since molecules in the context are structurally disjoint, edge connectivity implicitly delineates context boundaries, removing the need for explicit delimiter tokens. Details are in appendix B
Pretraining: To construct the pretraining dataset as illustrated in Section 3.2 we use the ChEMBL
database (Zrrazil et al. 2024), the largest collection of biological assays, containing over 2.5 million molecules and 1.7 million assay records. To increase chemical diversity for materials discovery, we augment ChEMBL with polymer datasets from multiple sources (Liu et al. 2024b; Kuenneth et al.) 2021). including properties such as thermal conductivity, free volume fraction, and glass transition temperature. For biological assays, we generate tasks by selecting molecule-activity pair as the target and treating other molecule-activity pairs as context. The target is assigned score of 1, and context scores are computed by normalizing differences in pChEMBL values (negative log of bioactivity measures such as IC50 and potency) to the interval [0, 1J. We restrict targets to bioactive molecules with pCLEMBL > 6. For polymers, we apply the same strategy: each polymer used as target, and its property value is normalized against those of other polymers to form context-targee target pairs. We partition context examples into three groups by normalized scores: positive [0.75, 1], medium [0.5, 0.75), and negative (0, 0.5), with up to 15 demonstrations from each. The final dataset comprises around million molecules with 155K unique assays or properties, yielding 1.6 million tasks. As shown in Figure Bal the frequency distribution of assays and properties follows Zipf's law, P(Yrank) x rank 1.13 consistent with patterns in language corpora. These 1 million molecules are used to initialize the motif vocabulary via NPE, which provides a more compact representation by reducing node counts (Figure 3b). We further extract edge connections construct edge
5

---

Published as a conference paper at ICLR 2026
Table 1: We compute the oracle and diversity scores from the Top-10 generations and report the harmonic mean of these two values. We group 33 tasks into six categories and report the mean + std within each category. The best results in each column are bolded. spacifie results and additional metrics are provided in appendix D.2
Task
Drug Rediscovery 7
Drug MPO
Structure Constrained
Drug Design
Target Based
Material Design
Avg Rank 33
Total Sum 33
it Tasks
7
5
4
Molecular Optimization Methods with 100
Oracle Calls
GraphGA
0.3620.07
0.52#0.19
0.43=0.21
0.41#0.32
0.7620.04
0,58 11
6.56
16.65
REINVENT
0.37=0.08
0.52+0.17
0.43=0.21
0.42+0.32
0.76#0.03
0.00+0.00
8.44
13.84
GPBO
0.37=0.07
0.51
0.42=0.22
0.39=0.33
0.76=0.03
0.60=0.21
6.89
16.65
STONED
0.36#0.07
0.52+0.19
0.39#0.24
0.3740.36
0.7640.04
NopSMILES
8.18
13,42
Genetie GFN
0.36#0.10
0.51#0.22
0.38=0.29
0.33+0.37
0.76+0.03
NoySMILES
9.28
13.04
GenMol
0.42=0.08
0.51#0.17
0.42=0.25
0.55+0.21
0.6920.03
0.6240.08
7.98
17.36
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.37=0.09
0.50=0.18
0.45=0.29
0.49#0.26
0.6420.06
0,5540. 14
9.30
16.30
REINVENT
0.30=0.13
0.23+0.23
0.25#0.24
0.38+0.26
1.
0.51
13.60
9.82
GPBO
0.33=0.10
0.45#0.22
0.43=0.29
0,49=0.15
0.74=0.03
0.42+0.24
9.87
15.35
STONED
0.33+0.10
0.40#0.20
0.50+0.28
0,2740.10
0.20=0.27
NopSMILES
12.01
9.66
Genetic GFN
0.14=0.08
0.08+0.16
0.08+0.11
0.12+0.11
0.15#0.15
NoySMILES
17.95
3.14
GenMol
0.3420.07
0.2820.23
0.21=0.29
0.39=0.19
0.52+0.05
0.21
12.63
10.37
Conditional Generation Models
LSTM
0.39=0.25
16
0.55=0.32 0.58
0.33+0.35 0.48+0.37
0.72+0.04
0.1640. 11
11.77
12.30
Graph-DiT
0.43+0.21
0.50+0.18
0.7140.04
0.5540.17
8.53
17.64
Learning from In In-Context Demonstrations
DeepSeck-V3
0.45=0.18
0.51#0.20
0.49=0.24
0.65+0.18
0.6420.06
0.3920.24
8.08
16.90
GPT-4o
0.47+0.21
0.53+0.20
0.52+0.30
0.48+0.40
0.73+0.05
0.43+0.16
7.89
17.25
Qwen-Max
0.15=0.21
0.17#0.15
0.32=0.32
0.29=0.29
0.19=0.26
0.10=0.18
15.39
6.46
Llama3.1 1-maa.1---a 1-8B-FT
0.21=0.13
0.24+0.23
0.24#0.34
0.31#0.31
0.02=0.05
0.29+0.40
14.91
6.04
Qwen3-8B 8B-FT FT
0.37+0.19
0.27+0.16
0.26+0.34
0.46+0.22
0.67+0.07
0.44+0.399
10.96
12.23
DefoDiff (Ours)
0.4420.21
0.5420.23
0.5620.33
0.79+0.11
0.78=0.05
0.67.0011
4.10
20.10
vocabulary capturing motif-to-motif connectivity, Finally, we pretrain DemoDiff model with 0.7B parameter on Eq. (3 using 146 H100 GPU days. Details are provided in appendix C
4 EXPERIMENT
Setups: We curate 33 downstream tasks (see Table 1 and appendix D.I across six categories to evaluate DemoDiff against 19 baselines. These tasks are primarily curated by domain experts and are distinct from pretraining. We include eight molecular optimization methods and two conditional generation models (LSTMs and Graph DiT (Liu et al. 2024c)), and LLMSs (DeepSeek- V3,, GPT- 4o, and Qwen-Max). We select the top four molecular optimization algorithms from the PMO benchmark (Gao et al. 2022) (out of 25 evaluated methods) and two recent methods, Genetic GFN (Kim et al. 2024) and GenMol (Lee et al.] 2025), under two settings: 100 oracle calls and 10,000 predictor calls. For evaluation, we generate 10 valid, unique, and novel molecules per task and score them with Oracles. We report the harmonic mean over two dimensions: (a) averaged oracle scores and (b) the diversity score Eq. ( Each task has up to 450 molecule-sco score pairs, evenly divided into positive (0.75, 1]. medium [0.5, 0.75), and negative [0, 0.5) groups. Fach task has an Oracle for evaluation. We use all molecules train the task-specific predictor for predictor calls to train conditional generation models directly. For LLMS with ICL, we include three closed-sourced models. We finetune (amma.11-8B-FF and Qwen3-8B-FT on the DemoDiff pretraining set. Their demonstrations are randomly sampled with a similar budget for context. For the baselines, we follow their standard and original implementations process the molecules.
4.1 PERFORMANCE ON DIVERSE MOLECULAR DESIGNS TASKS

---

Published as a conference paper at ICLR 2026
ICL achieves competitive performance with minimal supervision. Table compares the harmonic mean of the top-10 generated molecules based on both task scores and structural diversity, while Table [] (appendix) reports the top-1 scoring molecule per task. Under limited data and Oracle budgets, ICI. methods perform comparably to, or hetter than, fully trained conditional generators and molecular optimization baselines. Among ICI--based methods, DeepSeek- V3, GPT-40, and Demoliff consistently attain top-tier average ranks. These ICL methods rely on tens of demonstrations per task, significantly fewer than the training data or Oracle calls required by other models or algorithms.
DefoDiff designs molecules with accurate scores and high diversity. Across six task categories, it performs best on property-dre drivve tasks, including drug design with bioactivity targets, protein binding affinity, and material design for polymer gas separation. DemoDiff achieves the lowest average rank of 4. 10, outperforming the best baseline, GraphGA (rank 6.56). ICL. methods with L.L.s produce high-scoring top designs (Table 5) but often generate structurally similar molecules. These do not necessarily align better with the target score while reducing diversity. In contrast, DemoDiff designs molecules with scores closer to the query and better structural diversity.
DemoDiff performs better on property-drive driven tasks than on structure-constrained ones. scores 0.67-0.79 on drug and material design, but around 0.44-0.56 for rediscovery and structure- constrained tasks, where Oracle scoring is tied to the presence of specific structures. While DemoDiff still ranks highly in
tasks, its stronger results on property tasks highlight its advantage in exploring chemical spaces with broader solution ranges.
4.2 ABLATION STUDIES AND PERFORMANCE ANALYSIS
Table 2: Performance across model sizes using harmonic mean scores from Top- 10 generations
Drug Rediscovery
Drug MPO
Structure Constrained
Drug Design
Target Based
Material Design
DemoDDfff
78M
0,39 + 0.17
0.46 + 0.24
0.47 +: 0.27
0.57 : 0.31
0.73 & 0.03
0.62 + 0.13
311M
0.40 0.17
0,46 + 0.23
0.50 = 0.28
0.53 # 0.27
0.75 # 0.04
0.62 # 0.14
739M
0,44 0.21
0.54 0.23
0.56 + 0.33
0.79 a: 0.11
0.78 + 0.05
0,67 # 0.11
Model Parameters: We pretrain DemoDiff with varying sizes: small (78.7M), medium (31 IM), and large (739M) parameters. reports performance using the top-10 harmonic means of task score and diversity. We present averages with deviations across six categories. DemoDiff achieves reasonable scores even at small scale. At the medium scale, performance improves most tasks except drug design, while the benefits of parameter scaling become more evident (in five out of six task categories) at larger scales
ICL with Demonstrations: Figure 4 studies two factors in demonstrations: (1) context length and (2) ratio of positive examples. In Figure 4a longer context includes more molecular examples and supports better ICL performance. This aligns with the rationale of motif-level tokenization, which captures more examples within a fixed context. Figure 4b shows that diverse demonstrations are important for ICI. to represent the task accurately, while only positive examples insufficient. This is because positive, medium, and negative examples together provide a holistic view of the task context, and DemoDiff pretrained on such contexts is better able to infer latent concepts from diverse examples. In Figure 4b) results. We investigate this further in Section 4.3 to assess whether DemoDiff can infer positive examples (score > 0.5) using only negative examples with scores below 0.5.
we also observe that fewer positive examples may still yield reasonable
ICL with Consistency Scores: We ablate consistency scores and analyze their correlation with target scores in Fi Using the consistency score as confidence filter improves performance across task categories, with gains from 0.8% to 27.5%. The second figure shows the correlation between the consistency and target scores. Moderate correlation appears in tasks with explicit structural constraints, such as drug rediscovery and structure-constrrineed design. For property-driven tasks (drug MPO and materials design), high fingerprint-based consistency with positive examples does not always correlate with high target scores. In these cases, latent concepts may rely on subtle substructures (e.g., methyl groups (Liu et al. (202)) that standard fingerprints fail to capture. Interestingly, context consistency still improves performance in these tasks. A possible reason is that the score helps filter out false positive generations.
7

---

Published as a conference paper at ICLR 2026
Top-10 Harmonic Score
0.752
Top-10 Harmonie Score
0.76
0.752
0.75
0.741
0.736
0.736
0.73
0.724
0.73
0.72
0.72
0.71
0.708
0.705
0.70
0.693
Context Length (# of motif tokens)
Ratio of positive examples
(a) Harmonic score vs. context length
(b) Harmonic score vs. positive samples
Figure 4: Ablation studies on Albuterol drug rediscovery.
Drug Deagu
0.623
0,794
0.231 (2H)
Drug MIO
(0010-007)
Stracture Constralned
6.526
0360
0.119 (US)
Matertals Dedign
0.641
0.672
0.520 (0V5)
Doug Rediscovery
0.441
0.444
0.774
0.780
0.126
w/ Consisten.g Score
0.0
0.1
0.3
Jverage Top-10 10 Harmonic S cooe
Pearson Correlation
Figure 5:
Ablation studies context consistency scores: (1) left shows improvements; (2) right
shows the relationship between consistency score and oracle scores.
4.3 CASE STUDIES
Figures and 7 present two studies for DemoDiff. In extreme cases of inverse molecular design, demonstration sets may contain only negative examples, i.e., all scores < 0.5. In Figure 6 we study whether DemoDiff can still generate positive candidates when prompted solely with negative design, (b) drug multi-objective
examples. Figure 6 presents the results for (a) structure-constraan optimization (MPO), and (c) target-based design. These findings suggest two insights: (1) negative demonstrations convey informative signals about the task concept, and (2) after pretraining, the posterior over the concept-to to-structure mapping allows DemoDiff to generate desirable candidates that are aligned with the concept yet structurally distinct from the negative examples. Figure 7 the generation trajectory from diffusion models. The task score, measured structural similarity to Albuterol, rises from 0.22 at initial sampling to 0.74. This shows that the diffusion model refines the molecule toward the desired structure step by step with demonstrations.
RELATED WORK
Inverse Molecular Design: Molecular optimization diverse approaches, including genetic algorithms, Monte Carlo Tree Search (Jensen, 2019), and Bayesian optimization (Shahriari et al. 2015). applied representations such fingerprints, SMILES, graphs, and synthetic pathways (Gao et al. 2021). Gao et al. (2022) benchmarked 25 optimization methods and found that older models, such as genetic algorithms, remain competitive. However, existing benchmarks require on the order of 10,000 oracle calls, which is costly and limits applicability when single calls are expensive. Deep learning models offer an alternative by modeling the joint distribution of atoms and bonds without Oracle calls. GDSS applies noise and denoising in continuous space for graphs (Jo et al. 2022). DiGress (Vignac et al., 2022) introduces discrete noise through transition matrices based marginal atom and bond distributions. Graph DiTs (Liu et al., 2024c) extend scalable diffusion transformers (Peebles & Xie 2023) to discrete graphs. Yet, training diffusion models still requires hundreds of labeled molecules and is limited to specific tasks. Recent efforts explore chemical
8

---

Published as a conference paper at ICLR 2026
Query:
1.23e-02
7.61e-21
8.15e-09
0.7225
Is,,er(C
Query:
Score
2.63e-01 (b) Geometric Mean(Sim(Qsimertinn TPSA, LogP)
4.05e-01
Query:
4.93e-01
4.79c-01
4.64e-01
0.6714
(c) Docking Score (PARPI). Converted to 0-1: Higher better
Figure 6: Learning from negative demonstrations (score 0.5) to infer target with score 1. All demonstrations are shown in Figures 14 to 16. with only three displayed here.
(Ground-trutt
Query Score=1
Step1: 0.22
Step 100: 0.20
Step 200: 0.32
Step 300: 0.41
Step 400: 0.59
Step 500: 0.74
Figure 7: Diffusion trajectory for Albuterol drug rediscovery: we sample five intermediate diffusion steps and score them with the Albuterol Oracle, which computes similarity to the ground truth.
foundation models based on LLMs (Yu et al. 2024 Liu et al.) 2024b), but their applications are either diverted to other molecular tasks, such as property prediction, or rely on fine-tuning within restricted scope of design tasks.
In-Context Learning: ICL is an emergent ability observed LLMs (Brown et al. 2020 Chan et al.
2022). Empirical and thcoretical studies investigate this phenomenon from three perspectives: models, data, and learning mechanisms (Xie et al., 2021 Min et al. 2022 For the learning mechanism, ICL can be interpreted as implicit Bayesian inference (Xie al. (02T). where pretraining data generated from latent concepts and the posterior distribution marginalizes over them for inference. On the model side, Garg al. 2022) trained Transformers from scratch on prompt-style input-label pairs of simple functions and found performance comparable to task-specific algorithms. [Bhattamishra (2023) compared Transformers with attention-free models and showed they do not match Transformer performance across tasks. On the data side, Chan et al. (2022) found that Transformers outperform recurrent models (e.g., LSTMs) on data with distributional properties resembling natural language, such as burstiness (words appearing in clusters) and query tasks with many rare classes. Singh et al. (2025) analyzed the strategy competition between ICL and in-weight learning, showing that the asymptotic strategy depends on in-weight information but is also context-constrained. This aligns with (Chan et al. 2022), suggesting that a foundation model should support both capacities. A skewed Zipfian distribution over tasks (e.g., Figure Ba) balances learning by storing common task information weights while developing ICL ability from the long tail of rare tasks. Besides LLMs, ICI. is also studied graphs (Huang et al. 2023), but mainly for node and edge classification tasks.

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
We presented DemoDiff, molecular design. We constructed large-scale pretraining dataset with over one million molecules and 155K unique biological assays and material properties, yielding millions of demonstration target pairs. Using this dataset, we pretrained 0.7B-parameter model and showed that it matches or outperforms much larger LLMs and ranks higher than domain- and task-specific methods. To support scalable pretraining, we introduce Node Pair Encoding. a motif-level graph tokenizer that efficiently represents molecules with fewer nodes while preserving reconstruction. Experiments demonstrate that DifoDiff is promising molecular foundation model, highlighting its potential to scale further with larger models, broader datasets, and greater compute.
cntpnsttaaiooconnntiionee diffusion Transformer model for in-context
ACKNOWLEDGMENTS
This work was partially supported by NSF ISS-1442227 IIS 2146761, IIS-2234058, and CBET- 2332270. We also appreciate the support from the Foundation Models and Applications Lab of Lucy Institute, ND-IBM Tech Ethics Lab, and OpenAl Researcher Access Program.
REPRODUCIBILITY STATEMENT
We provide pretraining and inference code in the supplementary materials. The appendix describes the method and model settings to ensure reproducibility.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08874 2023.
Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):214--221, 2015.
Amanda Bertsch, Maor Ivgi, Emily Xiao, Uri Alon, Jonathan Berant, Matthew R Gormley, and Graham Neubig. In-context learning with long-context models: An in-depth exploration. arXiv preprint arXiv:2405.00200 2024.
Satwik Bhattamishra, Arkil Patel, Phil Blunsom, and Varun Kanade. Understanding in-context learning in transformers and Iims by learning to learn discrete functions. arXiv preprint arXiv:2310.03016. 2023.
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3): 1096-1108, 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Stephanie Chan, Adam Santoro, Andrew Lampinen, Jane Wang, Aaditya Singh, Pierre Richemond, James McClelland, and Felix Hill. Data distributional properties drive emergent in-context learning in transformers. Advances neural information processing systems, 35:18878-18891 2022.
Jorg Degen, Christof Wegscheid-Gerlace Andrea Zaliani, and Mathhias Rarey. On the art of compiling and using' drug-like' chemical fragment spaces. ChemMedChem, 3(10):1503, 2008.
Joseph L Durant, Burton A Leland, Douglas R Henry, and James G Nourse. Reoptimization of mdl keys for use in drug discovery. Journal of chemical information and computer sciences, 42(6): 1273-1280, 2002.
10

---

Published as a conference paper at ICLR 2026
Wenhao Gao, Rocío Mercado, and Connor W Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. arXiv preprint arXiv:2/10.06389, 2021.
Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley. Sample efficiency matters: benchmark for practical molecular optimization. Advances in neural information processing systems, 35: 21342-21357, 2022.
Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What transformers learn in-context? a case study of simple function classes. Advances in neural information processing systems, 35:30583-30598, 2022.
Qian Huang, Hongyu Ren, Peng Chen, Gregor Kržmanc, Daniel Zeng, Percy Liang, and Jure Leskovec. Prodigy: Enabling in-context learning over graphs. Advances in Neural Information Processing Systems, 11116022-13317, 2023.
EunJeong Hwang, Veronika Thost, Shib Sankar Dasgupta, and Tengfei Ma. An analysis of virtual nodes graph neural networks for link prediction. The first learning on graphs conference, 2022.
Jan Jensen. A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration chemical space. Chemical science, 10(12):3567-33 -572, 2019.
Jaehyeong Jo, Seul Lee, and Sung Ju Hwang. Score-based generative modeling of graphs via the system of stochastic differential equations. International conference on machine learning, pp. 10362-10383, PMLR, 2022.
Hyconah Kim, Minsu Kim, Sanghyeok Choi, and Jinkyoo Park. Genetic-guided gflownets for sample efficient molecular optimization. Advances in Neural Information Processing Systems, 37: 42618 42648, 2024.
Xiangzhe Kong, Wenbing Huang, Zhixing Tan, and Yang Liu. Molecule generation by principal subgraph mining and assembling. Advances in Neural Information Processing Systems, 35: 2550-2563, 2022.
Christopher Kuenneth, Arunkumar Chitteth Rajan, Huan Tran, Lihua Chen, Chiho Kim, and Rampi Ramprasad. Polymer informatics with multi-task learning. Patterns, 2(4), 2021.
Seul Lee, Karsten Kreis, Srimukh Prasad Veccham, Meng Liu, Danny Reidenbach, Yuxing Peng, Saee Paliwal, Weili Nie, and Arash Vahdat. Genmol: drug discovery generalist with discrete diffusion. arXiv preprint 1rXiiv2501.06/35 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Gang Liu. Tong Zhao. Jiaxin Xu, Tengfei Luo, and Meng Jiang. Graph rationalization with environment-based augmentations. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1069-1078, 2022.
Gang Liu, Michael Sun, Wojciech Matusik, Meng Jiang, and Jie Chen. Multimodal large language models for inverse molecular design with retrosynthetic planning. arXiv preprint 14.24224, 2024b.
Gang Liu, Jiaxin Xu, Tengfei I Luo, and Meng Jiang. Graph diffusion transformers for multi-conditional molecular generation. Advances in Neural Information Processing Systems, 37:8065-8005 2024c.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint 2rXiv:2202.128..3 2022.
Shingo Otsuka, Isao Kuwajima, Junko Hosoya, Yibin Xu, and Masayoshi Yamazaki. Polyinfo: Polymer database for polymeric materials design. 2011 International Conference on Emerging Intelligent Data and Web Technologies, pp. 22-29. IEEE, 2011.

---

Published as a conference paper at ICLR 2026
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the JEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Philipp Renz, Sohvi Luukkonen, and Günter Klambauer. Diverse hits in de novo molecule design: Diversity-base -baeed comparison of goal-directed generators. Journal of Chemical Information and Modeling, 6A(15):5756-576 5761, 2024.
Lloyd M Robeson. The upper bound revisited. Journal of membrane science, 320(1-2):390 2008.
David Rogers and Mathew Hahn. Extended-conneectivty fingerprints. Journal of chemical information and modeling, 50(5):742-754, 2010.
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan Adams, and Nando De Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1): 148-175, 2015.
Aaditya Singh, Ted Moskovitz, Sara Dragutinovié, Felix Hill, Stephanie C.Y. Chan, and Andrew M Saxe. Strategy coopetition explains the emergence and transience of in-context learning. In Forty- second International Conference on Machine Learning, 2025. URL URL|aatpss:/
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Nearocomputing, 568: 127063, 2024.
Michael Sun, Minghao Guo, Weize Yuan, Veronika Thost, Crystal Elaine Owens, Aristotle Franklin Grosz, Sharvaa Selvan, Katelyn Zhou, Hassan Mohiuddin, Benjamin Pedretti, et al. Representing molecules random walks interpretable grammars. arXiv preprint wxXv:2403.008147, 2024.
Michael Sun, Weize Yuan, Gang Liu, Wojeiech Matusik, and Jie Chen. Foundation molecular grammar: Multi-modal foundation models induce interpretable molecular graph languages. arXiv preprint (tXXi:::000.29948 2025.
A Thornton, L Robeson, B Freeman, and D Uhlmann. Polymer gas separation mem- brane database, 2012. URL https://research. csiro. htt.::chhh..c...aaaaaaaaaaeeeeee
Clement Vignac, Igor Krawezuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209. /4734, 2022.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:21/1.02008 2021.
An Yang. Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, al. Qwen2. 5-Im technical report. arXiv preprint arXiv:2501.15383, 2025.
Botao Yu, Frazier N Baker, Ziqi Chen, Xia Ning, and Huan Sun. Llasmol: Advancing large language models for chemistry with large-scale, comprehensive, high-quality instruction tuning dataset. arXiv preprint arXiv:2402.09339,, 2024.
Barbara Zdrazil, Eloy Felix, Fiona Hunter, Emma J Manners, James Blackshaw, Sybilla Corbett, Marleen De Veij. Harris loannidis, David Mendez Lopez, Juan F Mosquera, et al. The chembl database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic acids research, D200): 2024.
12

---

Published as a conference paper at ICLR 2026
A DISCUSSION ON NODE PAIR ENCODINGS AND RELATED WORK ON FINGERPRINTS AND VIRTUAL NODES
The node pair encodings (NPE) differ from fingerprints (Rogers & Hahn 2010) and virtual nodes (Hwang et al., (022) in three aspects: (1) representation and invertibility, (2) computation process, and (3) use cases. Specifically: (1) NPE builds motif vocabulary from data and encodes a molecule as graph of motifs with typed edges and attachment rules, allowing the atom-level structure be reconstructed. In contrast, Morgan fingerprints hash local atom neighborhoods into fixed-length bit vector, which is lossy and not invertible. Virtual nodes are additional nodes inserted into graphs to provide shorteuts for information exchange between nodes rather than forming a new molecular representation. If we use only virtual nodes to re-represent the molecule, the structure is not invertible. (2) NPE identifies frequent substructures from a dataset to build a discrete motif vocabulary. In contrast, Morgan fingerprints are computed directly from atom invariants with chosen radius and bit size. Virtual nodes can be added randomly or through generic graph partitioning algorithms, which do not reflect how frequently partition cluster appears across the molecular dataset. (3) NPE produces a compact motif-level graph suited for generative tasks. In contrast, Morgan fingerprints produce vector suited for predictive tasks. Virtual nodes used create shortcuts for sharing information between graph nodes.
B DETAILS ON DEMODIFF
B. ICL WITH DEMODIFF
We construct the pretraining dataset as D [(C,, Qu X.)) Each task contains context of molecule-scon score pairs C, a query score Qi, and a target molecule X,. For each task, we first sample latent concept (e.g., the thermal conductivity of polymeric materials from Figure I from prior 0 ~ p()). followed by a set of molecule-property pairs associated with that concept. This induces the pretraining distribution:
[ p(0) ())(Q1).0)) p /
((C,Q,)) =
p(x")
Il po
(x' po
x Q.C)
dx
d8, (4)
where x represents the feature of the target token. With sufficiently large and diverse pretraining data and scalable diffusion transformers (Peebles & Xie 2023), DemoDiff can a foundation model for ICL. For downstream task, we (C,Q) Q) as the prompt, and p(0 C, Q) form the prompt concept. Then DefoDiff marginalizes the concept over the diffusion trajectories x !.:
p(X (,Q)
L/co x p(x²) II px
X xxC, p(0) C, d0.
(5)
B.2 GRAPH-LEVEL TOKENS AND PRETRAINING LOss
Graph DiTs define graph-level token that concatenates the node feature with all related edge features. In molecular generation, use special type of edge, the null edge, to represent that there is no edge between two nodes. Thus, The feature vector x (or xº) of graph-level token {M, {e, {=))= consists of three components: Fmoti motif types, Fbond bond types, and Fattach attachment specifications. Here, Fmotit is the size of the motif vocabulary M, Fhond = 4 represents null, single, double, and triple bonds, and Fattach arg Max |M the maximum number of
13

---

Published as a conference paper at ICLR 2026
atoms in M.. Eq. (3) can be decomposed as Preetrain Lmotif + Lboud + Lattach Specifically,
pretriin = (yy(x) (x))))
log 1o
motil
m xx,CCV)
Amoot
bond (by |x log Pe
Eboud
attachy ay |x(C,)) log Pa
(6)
j=1
Cattach
To align feature dimensions across tokens, we use the dense edge representation by treating non- connections null bonds. The resulting feature dimension is F= Fmotit + x Fbond + n Fattach, where n. is the maximum number of nodes the motif-represented dataset. For optimization with Eq. (6), we include the null bond type in Lbond but exclude attachment specifications of null edges in Lattach-
B.3 TRANSITION MATRICES IN DIFFUSION MODELS
We define the transition matrix that perturbs molecules at the motif level to pretrain DemoDiff. We model the joint distribution of nodes and edges with the transition matrix. It is constructed from four submatrices Qv, Qgv, QE: Qyg, denoting transitions node node, edge node, edge edge, and node - edge, respectively:
Qy I & QEV
I' @ QVE 1 1ax QE
QG
(7)
where denotes the Kronecker product, and 1N. 1/ and 1nxn are the column vector, row vector, and all-ones matrix, respectively. Here is the number of nodes. For edges, diffusion is applied to bond types only, while attachment attributes are optimized and predicted directly by the denoising model as in Eq. (6). For categorical sampling, we separate the unnormalized logits of node and edge from the model outputs, compute probabilities for each motif and bond individually before sampling.
To obtain the transition matrices, use the prior from the pretraining data. The noisy distribution is defined as the marginal distributions of motif types my and bond types mæ. The transition matrices defined as Qv ā'I + (1 ā²) a')1m'y and Qæ â'I + (1 ā') a')1m'g. where m' denotes the transpose and I is the identity matrix. We compute co-occurrence frequencies of motif and bond types in training graphs to obtain the marginal distributions mgy and my g. Each row in mæy gives the probability of co-occurring motifs for a bond type, and myg its transpose. The transition matrices then defined as Qev ā'I + aa)lm' Im'gy and QVE a'I + (1 ā²) I where a' is cumulative noise coefficient in diffusion. The cosine schedule is chosen as a (Os(0.5rtt/TT s)/(1 + =))².
B.4 DETAILS CONSISTENCY SCORE
Given a query Y, demonstrations C; = {(Xij, Yo) are divided into positive CPos, medium Gued and negative (ueg examples guide ICL. For generated molecule X, the Tanimoto similarity of fingerprints as the similarity measure. We compute the similarity between X and all molecules each group and average them obtain group-wise similarity scores
sim Pos
sim med
sim neg [0, 1].
Difference-based score. We compute margin differences between groups:
dpos,med max(simPos med 0), dmeddneg = max(sim med sim neg "e5,0) (,),
d dpos,neg max(sim sim 0).
14

---

Published as a conference paper at ICLR 2026
The normalized difference- based score is
ddiff min Apoe.med dmed,neg + dpos.nes, 1).
3
In experiments, the consistency score can be computed efficiently before applying Oracle functions. For example, we generate 1000 molecules and select the top 100 with the highest consistency scores. These molecules better follow the order of structural similarity across positive, medium, and negative examples. This removes poor generations that conflict with the demonstration semantics and increases confidence that selected molecules align with the query scores. Table reports empirical improvements across task categories, each containing 4-7 tasks (appendix D.I.
Table 3: Improvement with the consistency score (average Top-10 10 harmonic scores).
Category
Without
With
Improvement
Drug Design
0.6230
0.7943
+27.5%
Drug MPO
0,4592
0,5400
+17.6%
Drug Rediscovery
0,4110
0,4407
+7.2%
Structure Constrained
0.5258
0.5598
+6.5%
Materials Design
0.6407
0.6724
+4.9%
Target- -asee
0.7745
0.7803
+0.8%
C DETAILS ON PRETRAINING
The final pretraining dataset contains 1,084,566 molecules (polymers) and 155,150 unique assays or properties, yielding 1,639,515 tasks. These are constructed from ChEMBL (Zdrazil et al. 2024)
and multiple polymer data sources (Otsuka et al. 2011 Thornton al. 2012 Kuenneth et 2021). Each task has query molecule-score pair with the query score fixed at Up to 45 molecules are grouped into positive, medium, and negative demonstrations based on their scores. The query molecule is the target, while the query score and demonstrations serve as inputs to DemoDiff during pretraining on Eq. (3). For pretraining with a fixed maximum context window, we allocate half the window to positive demonstrations and one quarter each to medium and negative demonstrations, after excluding the target molecule.
C.I PRETRAINING DATASET
CHEMBL dataset We constructed molecular activity contexts from the ChEMBL database (version 35), which provides large collection of bioactivity measurements across diverse as- says. ChEMBL standardizes published activity types, values, and units into a unified variable, pCLEMBL = log (molar IC50, XC50, EC50, AC50, K4, Ka, Potency). This value places dif- ferent measures of half-maximal response, potency, or affinity on a comparable negative logarithmic scale. For example, an IC50 of nanomolar (1 x 10-9 M) corresponds extracted assay-level activity values (μChEMBL). For each assay, molecules were grouped according their recorded activities. Within each group, we selected anchor molecules with strong activity (pChEMBL > 6) as targets for building demonstrations. Each anchor was compared against all other molecules the same assay compute normalized distances, defined as the relative difference between the anchor's s pChEMBL value and that of the candidate molecule, converted to the range [0, 1]. Specifically, for an anchor with value e and a candidate with value Te: the normalized distance was given by d = (01 vz)/10 10. Based on this distance, we partitioned candidate molecules into three categories relative the anchor, Molecules with distances [0, 0.25) correspond to candidates with activity between 75% and 100% of the anchor and were assigned to the positive context. Molecules with distances in [0.25. 0.5) correspond to candidates with activity between 50% and 75% of the anchor and were assigned to the medium context. Molecules with distances [0.5, 1.0) correspond to candidates with activity below 50% of the anchor and were assigned to the negative context. From each category, we sampled up 15 molecules to balance neighborhood size. Thus, there are up to
pChEMBL value of 9. We
15

---

Published as a conference paper at ICLR 2026
45 demonstration molecules for each task. Not all of them used during pretraining due to the constraint of maximum context length. This procedure produced triplets anchor molecules and their associated positive, medium, and negative contexts.
Polymeric materials We have polymeric material datasets from different sources, including Poly- Info (Otsuka et al., 2011), MSA (Thornton et al. 2012), and from (Kuenneth et al.] 2021 J. We considered wide range of polymer properties spanning several categories, including thermal proper- ties (e.g., heat capacity, glass transition temperature, melting temperature, and thermal conductivity). electronic properties (e.g., ionization energy, electron affinity, and band gap), structural properties (e.g., density, crystallinity, and radius of gyration), and transport properties (e.g., gas diffusion, solubility, and permeability coefficients). For each property, raw values were normalized to the unit interval using exceeded 1000 and non-negative shifts applied when necessary. Each polymer with valid property values was treated as an anchor, and pairwise distances in normalized property space were computed against all other polymers. Candidate molecules were partitioned into positive [0, 0.25), medium [0.25, 0.5), and negative [0.5, 1] contexts, with up to 15 examples sampled per category based on smallest distances.
scaling, with logarithmic transformation applied when dynamic ranges
C.2 TOKENIZER PREPARATION
Algorithm 1 Node Pair Encoding (NPE) with Constraints
Require: molecule list D, motif vocabulary M 0,, max size K, ring count threshold Nring
Ensure: motif vocabulary M
1: Initialize each molecule X 'D with atom-level and ring-based motifs
2: Count frequencies of ring-based motifs across
3: Add all periodic-table elements, polymerization "*", and top-Ning frequent rings to M
4: while LM| < do
5: (1) Merge Neighbor: Initialize empty multiset S
6: for each molecule X E 'D do
7: for each motif m from X do
8:
for each adjacent motif m' in X such that m and m' are mergeable under structural constraints (e.g., rings treated units) do form new motif m < m U m
9:
add to multiset with frequency count 11: end for
end for
13: end for
14:
(2) Frequency Selection: Find most frequent motif m E S
15:
(3) Update Graph:
16:
for each molecule X E D do
17:
for each pair of adjacent motifs (m, ) in X do if their merged form equals m then
18:
replace and m' with m² in X end if end for
20:
22: end for
Add m" to M not already in it
24:
end while
25:
return M.
We present NPE Algorithm inspired by both the classic BPE and (Kong et al. 2022). We build the tokenizer with NPE on the pretraining data. To choose the motif vocabulary size, analyze the number of nodes in motie-represenned molecular graphs as the vocabulary size varies (Figure &). We report mean, max, and median counts. We set Knng K/ K/10, except for K 6000, where King 300. When K 2 3000, the mean and max node counts no longer significantly decrease, and the median remains unchanged. Therefore, select K 3000 with Krnn 300 for pretraining.
16

---

Published as a conference paper at ICLR 2026
Max Node Count
340 335
Mean Node Count
10.5
Median Node Count
7.0
320
6.5
300
9.3
6.0
9.0
5.5
260
8.5
10
5
5.0
8.0
5.0
240
1000
2000
3000
4000
6000
1000
2000
3000
4000
6000
1000
2000
3000
4000
6000
Vocabulary Size
Vocabulary Size
Vocabulary Size
Figure 8: Change in node count with varying motif vocabulary size.
Loo-Log Distribution
Complemestary CDP (Log-Log)
Coast
10P
Dost
10"
Mtdf Coeci
Medwaz 3019
Hedl Coun
PX x1 Toa Probabiiity
10
10
10"3
10
10
10
107
10
10P
10
10
Number of MotitAAso
Number of MoutiAtom
Bank Prequency Plot
Linear Log with Percentiles
70
Deasity (Log Stali)
102
10
307
10m4
10
10a
120
140
Motifittom
Figure 9: Comparison of the number of nodes in atom- and motif-based representations for million molecules from the pretraining set.
Next, Figure 9 presents the tokenization results on the pretraining dataset. log-scale distributions of motif- and atom-level node counts. Both representations exhibit heavy-tailed behavior, as shown by the fank-freuency plots and complementary cumulative distribution functions (CCDFs).
Figure shows the distribution of compression ratios, defined as and logarthmmic scale. The ratio ranges from 40, with a median and mean around 5.5, indicating consistent reduction in graph size by approximately a factor of five.
Uncompressed Graph Size Compressed Graph Size in both linear
Figure T1 provides detailed analysis of the relationship between atom-level and motif-level repre- sentations. We observe a mild positive correlation: larger molecules tend to yield higher compression ratios. Notably, molecules with 150 to 200 atoms are reduced by up to a factor of 15, demonstrating efficient compression at larger scales.
Whether NPE captures popular functional groups and reaction fragments: We evaluate the chemical validity of the motif vocabulary from the pretraining set. First, all motifs are chemically valid. Second, we compare the motifs in the vocabulary with 48 common functional groups. Forty- seven of them are present and one is missing. The groups are:
17

---

Published as a conference paper at ICLR 2026
Compression Ratio Distribution (Linear Scale)
Compression Ratio Distribution (Log Scale)
0.200
10
Medaon:
Density Scale)
0.150
30
0.000
10-4
0.000
10
0.000
10
15
20
25
10
10
15
20
25
30
35
40
Compression Ratio (Original/Copprssse)
Compression Ratio
Figure 10: Analysis on the compression ratio.
Original ves Compressed Node Count
(Correlation: 0.542)
Compression Ratio vs Molecule Size
250
Moving Average
Compressdd Node Coant
2000
30
1.50
Compressoon
100
y-u p40.240xx0.77
100
Original Atom
Original
Compression Efficiency by Molecule Size
Cumulative Distribation Functions
1.0
Meun Compresion Ratio Stdd
Cumolative Probshiltte
13
30
Original (Atomes)
Compressse (Mollis)
50
1000
150
200
250
300
350
100
200
300
600
500
600
700
Oriiginal Asom Count (Binned)
Nocie/Atom Count
Figure 11: Analysis on the relationship between atom- and motif-level representations.
Included (47): alcohol, aldehyde, ketone, carboxylic acid, ester, ether, amine primary, samine_sccondiel amine tertiary, amide, nitrile, nitro, sulfoxide, sulfone, thiol, disulfide, benzene, pyridine, pyrimidine, imidazole, thiophene, furan, pyrrole, methyl, ethyl, propyl, isopropyl, butyl, tert butyl, phenyl, benzyl, trifluoromethyl, methoxy, ethoxy, acetyl, mor- pholine, piperidine, piperazine, pyrrolidine, tetrahydrofuran, oxazole, thiazole, phosphate, sulfonamide, carbamate, urea, guanidine.
Not included (1): phosphonate.
Third, we compare the motifs in the vocabulary with fragments produced by breaking retrosyntheti- cally interesting chemical substructures (BRICS) (Degen al., 2008). BRICS produces more than 129,000 fragments. Among these fragments, 98.5% appear fewer than 100 times in more than one million pretraining molecules, and 63. 33.2% appear only once. When we compare the top 100 BRICS fragments, 89% are present in the vocabulary constructed by NPE.
C.3 MODEL PRETRAINING
We pretrain a 0.7B-paramen model (Transformer depth 24, hidden size 1280, 16 heads, MLP ratio 4) for 550 epochs, requiring 49 days on 2-4 H100 GPUs, or about 146 GPU days. We monitor
18

---

Published as a conference paper at ICLR 2026
0.15
1.5
10225
Attachment
0.2
Motif
10200
Edge
0.10
0.00
0.5
700
300
0.05
0.002
730
100
0.1
0.083
700
326
0.0
0.00
0.0
400
600
800
200
400
600
800
200
600
800
(a)
Training losses over steps
1.0
Attachment Acc.
1.00
1.00
Motif Acc.
Edge Acc.
0.9
0.98
0.98
0.8
0.96
3,4925
0.201
0.9000
0.7
0.94
304300
100
0.96
700
200
400
600
800
200
400
600
800
200
400
600
800
(b) Training accuracies over steps
Figure 12: Training curves showing (a) losses and (b) accuracies.
0.85
W
0.65
0.800
Validity
0.80
Similarity
0.60
0.75
0.55
0.70
Figure 13: Generation validity and structure similarity to the target during pretraining.
training loss and reconstruction accuracy each component in Eq. (6). As shown in Figure 12 loss decreases and accuracy increases throughout training. Near the end, values plateau but still show incremental gains. Pretraining was stopped once reconstruction accuracy exceeded 0.99 for all components due to resource limits. During training, also generated 512 molecules sampled steps using the validation set. In Figure 13 we report chemical validity and structural similarity to ground truth measured by MACCS fingerprints (Durant et al. (002). Both metrics improve with training and reach about 0.83 validity and 0.69 similarity.
These trends in loss, accuracy, validity, and similarity indicate that larger models, more data, and additional compute could further improve DemoDiff as a molecular foundation model.
D DETAILS ON EXPERIMENTS
D.1 DETAILS ON EXPERIMENTAL SET-UPS
We curate 33 benchmark tasks across six categories to evaluate DemoDiff against 13 baselines. including eight molecular optimization methods, two conditional generation models, and three LLMs. Details are in Table The benchmarks span seven drug rediscovery tasks, seven drug multi-objective optimization (MPO) tasks, five coneettrr--ontttineeed generation tasks, four drug design tasks, five target-based generation tasks, and five polymer property design tasks. Specifically, the benchmarks span the following tasks:
Drug rediscovery (7 tasks): Celecoxib rediscovery. Mestranol similarity, Thiothixene rediscovery, Troglitazone rediscovery. Median (median similarity between camphor and menthol), Median 2 (median similarity between tadalafil and sildenafil), and Albuterol similarity. These tasks use Oracle scoring functions based on the similarity between the drug and the target using extended connectivity fingerprint (Brown et al. 2019).
19

---

Published as a conference paper at ICLR 2026
Table 4: Benchmark Task Statisties: Example Counts and Score Ranges [min, median, max]
Task Name
Total
Pos
Med
Neg
All Scores
Pos Scores
Med Scores
Neg Scores
Drug Rediscovery
Albuterol Similarty
450
150
150
150
[0.075, 0.537.
[0.752,
0,807,
[0.502.
0.537.
[0.075.
0.219.
1,000]
1,000]
0.744]
0.476]
Celecoxih Rediscovery
450
150
150
150
[0.015, 0.548,
[0.753,
0.786,
[0.505, 0.548,
[0.015, 0.153,
1,000]
1,000]
0.735]
0.367]
Median
150
0
150
[0.000) 0.057.
[0.000. 0.057,
0.419]
0.419]
Median 2
150
150
[0.038. 0.122.
[0.038. 0.122.
0.413)
0.413]
Mestranel Similarity
150
150
150
[0.004, 0.538.
[0.752,
0.824.
[0.500.
0.538,
[0.004, 0.150.
1,000]
1,000]
0.713]
0.379]
Thiothixene Rediscovery
[0.019, 0.510,
[0,753,
0.790,
[0.505,
[0.019, 0.155,
1.000]
1,000]
0.736]
0.354]
Troglitazone Rediscovery
383
150
150
[0.018.
0.526.
[0.752,
0.792,
[0.504,
0.553,
[0,018. 0.143.
1.000]
1.000]
0.727]
0.250]
Drug MPO
Amlodipine Mpo
150
[0.000,
0.517,
[0.750,
0.784.
[0.502,
0.517,
[0.000,
0.145.
0.871
0.871]
0.742]
0.500]
Fexofenadine Mpo
450
150
150
150
[0.000,
0.570,
[0.750,
0.772.
[0.501,
0.570,
[0.000,
0.125.
0.960]
0,960]
0.720]
0.498]
Osimertinib Mpo
450
150
150
150
[0.000.
0.638.
[0,750.
0.761.
[0.501.
0.638,
[0.000,
0.047.
0.908]
0,908]
0.747]
0.493]
Perisdopril Mpo
306
150
150
[0.000.
0.502.
[0,766,
0.778.
[0.502.
0.522,
[0,000.
0.114.
0.790)
0.790]
0.680]
0.415]
Ranolazine Mpo
450
150
150
150
[0.000,
0.575,
[0.750,
0.764,
[0.502,
0.575,
[0.000,
0.052,
0.867]
0.867]
0,743]
0.454]
Sitagliptin Mpo
389
150
150
[0.000.
[0.750. 0.768.
[0.500.
0.641.
[0.000.
0.000.
0.841]
0.841]
0.748)
0.213]
Zaleplon Mpo
300
150
150
[0.000,
0.493.
[0.501.
0.528,
[0.000,
0.003.
0.637]
0.637]
0.486]
Structare
Constrained Design
Deco Hop
150
[0.252, 0.529,
[0.793, 0.842,
[0.502,
0.529,
[0,252,
0.283.
0.953]
0,953]
0,677]
0.500]
Isomers C7H8N202
450
150
150
150
[0.000, 0.592,
[0.799. 0.819,
[0.535.
0.592.
[0.000,
0.000.
1.000]
1.000]
0,741]
0.449]
Isomers C9H10N202P12C1
450
150
150
150
[0.000.
0.561.
[0.767.
0.779.
[0.503.
0.561.
[0.000,
0.000.
0.882]
0.882]
0,720]
0.386]
Scaffold Hop
450
150
150
150
[0.333,
0.510,
[0.754,
0.782,
[0.500,
0.510,
[0.333,
0.380,
0.828]
0,828]
0.627]
0.444]
Valsartan Smarts
188
23
15
150
[0.000. 0.000.
[0.757. 0.806.
[0.512. 0.686,
[0.000.
0.000.
0.975]
0.975]
0.739]
0.000)
Drug Design [0.000.
DRD2
450
150
150
150
0.623,
[0.760,
0.961.
[0.507,
0.623,
[0.000.
0.004
1.000]
1.000]
0.742]
0.344]
GSK3B
450
150
150
150
[0.000.
0.615,
[0.760,
0.880.
[0.510.
0.615,
[0.000.
0.030.
1.000]
1,000]
0,750)
0,380]
JNK3
[0.000.
[0.760.
0.890.
[0.510.
[0.000.
1.000]
1,000]
0,750]
0.340]
QED
450
150
150
150
[0.010,
0.644.
[0.751.
0.819,
[0.501.
0.644,
[0.010,
0.348.
0.947]
0.947]
0.749]
0.499]
Target Based
Design
Docking Shilb
450
150
150
150
[0.000,
0.607.
[0.757.
0.771.
[0.507.
0.607.
[0.000.
0.450.
0.879]
0.879]
0.729]
0.500]
Docking braf'
450
150
150
150
[0.000,
0.600,
[0.757,
0.771.
[0.507.
0.600.
[0.000,
0.464
0.871]
0.871]
0.736]
0,500]
Docking fa7
305
150
150
[0.000.
0.507.
[0.764,
0.800.
[0.507.
0.536,
[0.000.
0.450.
1.000]
1,000]
0.629]
0.500]
Docking jak2
450
150
150
150
[0.000.
0.586.
[0.757,
0.771.
[0.507.
0.586,
[0.000.
0.464.
0.936]
0.936]
0.714)
0.500]
Docking parp1
450
150
1.5
150
[0.000,
0.614,
[0.757,
0.771.
[0_507.
0.614,
[0.000.
0.471.
0.864]
0.864]
0.750]
0.500]
Material Design
Polymer CO2, CH4
160
150
[0.042. 0.203.
[0.777
0.777
[0.538,
0.603,
[0.042,
0.198.
0.777]
0.777]
0.731]
0.487]
Polymer
[0.000. 0.110.
[0.506.
0.561.
[0.000.
0.102.
0.631]
0.631]
0.498]
Polymer H₂ CH2
150
[0.000,
0.117,
[0.752,
0.988,
[0.509,
0.615,
[0.000,
0.050.
1,000]
1,000]
0.739]
0.487]
Polymer Hy No
175
150
[0.005.
0.178.
[0.783,
0.974.
[0.514. 0.619,
[0.005.
0.151.
1.000]
1.000]
0.731]
0.495]
Polymer O2 N₂
173
23
150
[0.309.
0.355.
[0.501.
0.579.
[0.309.
0.346.
0.726]
0.726]
0.493]
20

---

Published as a conference paper at ICLR 2026
Drug MPO (7 tasks): Perindopril MPO, Ranolazine MPO, Osimertinib MPO, Zaleplon MPO, Sitagliptin MPO, Amlodipine MPO, and Fexofenadine MPO. These tasks use Oracle scoring functions based on -trrgtt similarity with extended connectivity fingerprints, along with additional constraints such as logP. TPSA, and Bertz, computed using RD- Kit (Brown al. 2019).
Structure design (5 tasks): Isomers C2H,N2O2, Isomers Co Decoration hop, Scaffold hop, and Valsartan SMARTS. These tasks use Oracle scoring functions primarily based on SMARTS patterns that evaluate whether a particular structure is present or absent in the target, optionally combined with other computational constraints. Or whether the target is an isomer of the molecular formula.
Drug design (4 tasks): DRD2, JNK3, GSK33, and QED, These tasks use Oracle scoring functions based ML models. QED is based on RDKit.
Target-based design (5 tasks): Docking BRAF, Docking PARPI, Docking JAK2, Docking FA7, and Docking 5-HTIB. These tasks use Oracle scoring functions based on the docking program Quick Vina 2 (Alhossary al. 2015). Docking scores are negative, with smaller values indicating better binding. To map them into [0. 1] (where larger values are better), we 0, 1). where clip(r, a, b) = min( a).
use 8 = clip
docking score
14
Material design (5 tasks): Polymer gas separation for different gas pairs: CO2/CH4, COs/N2, Hy/CHa, Hz/N2, and Oz/N2. Each task studies whether two gases be separated based on the polymeric membrane materials. We evaluate their selectivity score for gas separation (Robeson| 2008), defined the log-ratio of permeabilities relative to an empirical boundary, shifted and clipped into the range [0, 1]. Gas permeabilities are calculated using ML models trained on all available labeled data (a superset of the specisce data), and the selectivity score is then computed based on gas permeabilities.
Each task contains up 450 molecule-score pairs, evenly split into positive, medium, and negative groups. Some tasks may have fewer pairs due to insufficient positive examples, as shown in Table 4 For instance, the Median and Median 2 tasks in drug rediscovery have no positive or medium examples. These pairs are used to train predictors for molecular optimization methods, conditional generators, or to provide demonstrations for ICL methods. Task scores lie within [o, 1], with the objective of generating molecules with score 1. Each task also defines an oracle function, which is used only for evaluation, except by molecular optimization methods that actively query the oracle.
For baselines, we compare against four molecular optimization methods from the PMO bench- mark (Gao et al. 2022). They are the top four methods selected from 25 candidates: Graph Genetic Algorithm (GraphGA), REINVENT (SMILES-based), Gaussian Process Bayesian Optimization (GPBO), and Superfast Traversal, Optimization, Novelty, Exploration, and Discovery (STONED, based on SELFIES). SELFIES is unavailable for polymers and STONED cannot be applied to mate- rial design tasks. We also include Genetic GFN (Kim et al.] 2024) and GenMol (Lee et al.] 2025). We have two evaluation settings: one with 100 oracle calls and with 10,000 predictor calls. While PMO permits up to 10,000 Oracle calls, such budgets are impractical in real-world settings due to the cost and time associated with laboratory experiments, which may require days to months for single call. To address this issue, we examine whether molecular optimization methods can be paired with predictor calls. Following prior work (Gao al. 2022 Liu et al. 2024c), we use random forest predictor trained on all 450 molecule-scon score pairs as the task- predictor.
We include conditional generation models such as LSTM and Graph DiT (Liu al. 2024c). They are trained on all available training data for each task. For ICL, we compare DemoDiff (739M parameters) with recent large-scale I.MMs, including DeepSeek- V3 (Liu et al., (0024)), GPT-4o (Achiam et al.. 2023), and Qwen-Max (Yang al. 2025), each with up to hundreds of billions of parameters. Besides, we finetune Llama3. 18B--FT FT and Qwen3-8B-FT on the DemoDiff pretraining set. For LLMs, we sample 12 positive, 6 medium, and 6 negative demonstrations.
For DefoDiff, we set the context size to 150 motif tokens. Excluding the target molecule, the context includes average 23 demonstrations: half positive, quarter medium, and one quarter negative. For evaluation, each method generates 10 valid, unique, and novel molecules per task, which are scored by oracle functions. We report the average of the top-10 oracle scores the performance
21

---

Published as a conference paper at ICLR 2026
Table 5: We compute the oracle of the top generation. We group 33 tasks into six categories and report the mean + std within each category. Best results in each column are bolded.
Task
Drug Rediscovery
Drug MPO
Structure Constrained
Drug Design
Target Based
Material Design
Avg Rank
Total Sum
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.28=0.06
0.49=0.18
0.4620.14
0.45+0.33
0.7420.04
0.72+0.23
9.14
16.73
REINVENT
0.31=0.07
0.47=0.16
0.45=0.14
0,56#0.38
0.75+0.06
0.00+0.00
10.74
13.68
GPBO
0.28+0.06
0.46+0.18
0.47#0.14
0.41
0.75+0.07
0.80+0.25
9.34
16.95
STONED
0.28+0.06
0.49+0.18
0.46+0.14 14
0.44+0.35
0.74+0.06
NopSMILLE
12.23
13.07
Genetie GFN
0.28+0.11
0.51
0.37=0.23
0.36+0.39
0.75+0.04
NoySMILES
11.75
12.50
GenMol
0.38=0.09
0.55#0.18
0.51=0.31
0.61
0.77+0.07
0.87+0.14
6.94
19.63
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.33+0.14 14
0.56+0.21
0.5420.38
0.60#0.41
0.83+0.12
0.70+0.21
8.10
19.03
REINVENT
0.3620.29
0.38+0.30
0.42=0.43
0.77=0.12
0.5920.37
0,87+0.17
9.11
17.67
GPBO
0.37+0.31
0.50+0.25
0.47+0332
0.6420.38
0.8420.08
0.5840.34
9.42
18.13
STONED
0.28=0.17
0.43+0.25
0.51#0.32
0,30+0.10
0.26#0.36
NopSMILES
14.36
10.07
Genetie GFN
0.1620.07
0.19+0.33
0.19=0.27
0.28+0.28
0.39=0.37
NoySMILES
15.92
6.44
GenMol
0.4420.28
0.3520.34
0.27=0.38
0.62+0.37
0.70g0.11
0.43+0.34
9.74
15.04
Conditional Generation Models
LSTM
0,47=0.36
0.33+0.20
0.6420.39 0.60=0.36
0.36+0.34 0.51
0.73+0.12
0.4840.29
11.91
16.30
Graph-DiT'
0,4620.27
0.53+0.07
0.70=0.07
0.78+0.23
10.16
19.44
Learning from In-Context Demonstrations
DeepSeck-V3
0.6620.37
0.60
0.5420.25
0.74+0.14
0.71=0.12
0.75+0.27
8.23
21.77
GPT-4o
0.53+0.31
0.56+0.20
0.52+0.32
0.54+0.41
0.69:0.09
0.7740.22
9.56
19.69
Qwen-Max
0.18=0.28
0.19:0.17
0.53=0.34
0.51
0.21=0.29
0.25+0.40
14.19
9.60
Llama3. 1.....1-16 1-8B-FT
0.25=0.34
0.38+0.32
0.25=0.38
0.33+0.29
0,08=0.19
0.20+0.30
14.67
8.42
Qwen3-8B 8B-FT FT
0.47=0.33
0.48+0.25
0.28+0.38
0.88+0. 14
0.70+0.08
0.3740.51
9.80
17.02
DefoDiff (Ours)
0.5420.33
0.54+0.19
0.59=0.37
0,91=0.07
0.77=0.10
0,93+0.16
4.68
22.63
score and compute its harmonic mean with the diversity score. The diversity score is computed as
IntDiv(G) =
Σ ER 2111 m/
T(m1, (m11,m2)2
(8)
GG
where denotes the generated set of molecules for evaluation. T denotes the Tanimoto similarity. For DemoDiff, we first generate 1000 candidates and select the top 10 with the highest consistency scores, prioritizing alignment with the context order positive, medium, and negative examples.
D.2 ADDITIONAL DISCUSSION OF EXPERIMENTAL RESULTS
We include results in Tables g 11 and 14 17 We additional diverse hits metric from (Renz et al. 2024) and present the results Table 7
We include detailed results from the top-10 generation on the oracle scores and diversity scores Table Beyond the discussion of ICL methods and DemoDiff in Section 4.1 we have additional observations:
Oracle quality critically affects molecular optimization. Comparing molecular optimization methods under varying numbers of function calls, we find that allowing more predictor queries does not consistently lead better performance. This suggests that both the quantity and quality of function evaluations (oracle or predictor) are essential for guiding molecular optimization. While not the main focus of this study, this insight points to an important direction for future work. For instance, in the structure cancture-constrainee design task involving the Valsartan SMARTS pattern (CN (C=0) Cclcce (c2ccccc2) ccl), shown in Table [6] all molecular optimization methods receive a score of zero. This failure is due to predictor trained limited data, which cannot model the latent design constraints, such as satisfying multiple SMARTS patterns and physicochemical properties (e.g.. logP, TPSA, and Bertz index (Brown et al. 2019).. . In contrast, in target- based design tasks (c.g.. Table 18). where training data are sufficient, more predictor calls improve performance by allowing finer structural optimization.
22

---

Published as a conference paper at ICLR 2026
Table 6: We compute the oracle and diversity scores from the Top-10 generations. We group 33 tasks into six categories and report the mean + std within each category. The best results in each column are bolded.
Method
Dogg Design
Doy MPO
Drug Reiloasoery
Material Design
Sinictue Converinnd
Target Boos!
Orucke
Diversiy 4
Oracle
Diversity 7
Orade
Divesitty
Oruele
Diveriity
Drucle
Divensiy
Oracle
Diveriit
5
5
-
GmpbCiA
0.34+0:38
0:584002
0-4060 19
0.6600.01
0:234006
155700.9
0.6100.8
0 (09000
19
0 01
0.68+0.66
0.8700.00
REIN
0.35+0.30
0:584001
0.39H0.17 17
0.86-0.0
0:34+006
08840002
0.000.00
0.39-0.01
0.3100.00
19
0.64+0:02
0.67-005
0.86-0.00
GPBO
0.82+0.38
0:584001
0.39x0.10 19
0.87=0.01
0:34+006
0.850002
0.4900.24
0.57:0022
0.30m0.10 19
0.884001
0.68:0.66
0.860.00
STONED
0.31=11.40
0:588001
040 19
0.8650.01
0:23:000
0.870002
0.2626.21
0.88+0000
0.68:0066
0.8660.0
Gkuusi GFN
0.28
0:5740002
0.3960.21
0.9688.01
0:23:005
0.8600001
Mayouaas
0.2900.24
0.576805
0.68:005
0.86e5.00
GenMal
0,5460.28
0.006011
0.4699.22
0.85:0.06
0.3560.8
0.5940.15
0.6150.12
0.6460.07
0 0.3560.22
0/736008
0.7160.77
0.6720.00
Molccular Optinicaiion Methoals with
10000 Protlicte Chils
0.57e01:2
0.5740012
04860025
0.60g31.00
0.28e00:1
0.65a0:13
0.0080126
0.5420019
0.4720.29
0.58e0:15
0.78e0.11
0.50-0000
REINVENT
0.72
0.3560.29
0.28g0.31
0:32+0225
0.50e0.32
0.4080.18
0.30e0.8
0.3540
0.50x20.2
0:1660000
0.0060.36
04240.30
0.6660.11
0:2460:32
0.5050.30
0,426005
0.370:29
STONED
0:4240:00
0.750017
0.26.0
0.57=0.00
0.28+0:32
0:004006
0.19H0.3
0.1400:00
0:35+0066
01140:27
0.114005
0.36.0
0.130.00
GanMil
0.6000398
0.5040.10
0.34,0.33
0.41+0.33
0.59+0:1
0.355005
0.20+0.76
0.360.38
0.46+006
0.650010
0.44-0.06
Conditional Geseration Models
LSTM
0.2660.35
05050002
0.090:004
0.890000
0,3360.27
02220.12
0.1050.07
08110.18
0.8660299
0.8740044
06150006
0.8900.01
Cipt-IIT
0.4150.18
0,3940.5
0.3800.1 17
0.67:0.06
0:356021
0834000
0,436011
0.340.02
0.526031
0,8640
0.5960.05
0.890.001
Lesssing from In Chettec Demonstrationn
DeepSeck V3
0.61
0:774000
0.1100.25
0.5820.32
0.5200117
0.37
0.652.15
0.4460.29
0.06g0.11 10
0.6260.11 0.6822.12
GPT-44
0.43:11.22
0.660001
045300.2
0.81:11066
0.41:0.26
0.77m0.11
0.30sd.16
0.8keau02
0.4453.27
0.8440005
0.64:0.17
0.55:0.0
Owen-Max
0.20x11.39
16
0.3420.14 14
0.30=8..17
0.34x0.21
0.1k0023
0.0920.166
0.30132.24
0.4000.29
0.4480 26
0.17:0.24
0.25:0.36 36
Llama3 1-8B-FT
0.23:11.29
0:5780.01
0.1760.17
0.7721.15
0.33:009
0.7900.10
0.4020.10
0.37:3.51
0.23a0.36
0.3380112
0.01:0.13
0.8920.01
Qwen3 4B-FT FT
0.34
0.5620.01
0.1760.10
0.88
0.29g0.21
0.7720.16
0.4680.47
0.8025.15
0.23
0.54g0 10
0.55.0009
0.88:5.00
DemoDit (Ows)
0.8120036
0.79g0.08
0.4560.23
0.85
0.4120.2
0.7100117
0.5020.15
0.362000
0.54g0.33
0.79g0.14
0.7120.08
0.8720.01
Table 7: Diverse hits across six task categories. Diverse hits from (Renz et al. 2024) are computed as the number of generated hits with oracle scores greater than 0.5 and pairwise Tanimoto distances greater than 0.7. Results are reported as mean + standard deviation for each category. Best results in each column are bolded.
Task
Drug Design
Drug MPO
Drug Rediscovery
Material Design
Structure Constrained
Target Based
Avg Rank
Total
Sum
Tasks
33
33
GraphGA
11.25=22.50
12.86+21.96
0.00=0.00
12.54
9.00=20.11 12
45.00:00.0
4.67
90.31
REINVENT
11.25=22.50
10.43+18.47
0.00=0.00
0.00=0.00
9.00+20.12 12
44.80=0044 45
8.00
75,48
GPBO
11.25=22.50
12.86421. 2.2
0.00=0.00
10.40+10.02
9.00+22011
44.40+0.55
5.67
K7.91
STONED
11.25=22.50
12.86121.96
0.00=0.00
No PoSSMILES
9.00:20.12
45.0010.00
7.17
78.11
Genetic GFN
11.00=22.00
12.71421.72
0.00=0.00
No NepSMILES
8.80&19.68
45.00240.00
9.17
77.51
GenMol
14.75=17.15
9.00812.78
0.1420.38
7.6016.11
5.00+10.63
27.8011217
8.33
64.29
Molecular Optimization Methods with 10000
Predictor Calls
GraphIA
8.25116.50
5.0016.51
0.29w0.76 76
5.2019.55
10.40::4..40
8.6026.02
9.50
37.74
REINVENT
1,00=2.00
1.00=2.65
0.00=0.00
4. 4,20=6.22
0.00=0.00
0.00=0.00
13,50
6.20
GPBO
0.00=0.00
1.85
0.00=0.00
0.0010.00
12.20x19.22
25.80=15.97
11.17
43.86
STONED
0.00=0.00
0.00+0.00
0.00=0.00
No pSMiLES
15.80=21.44 48
0.20=0.45
13.33
16.00
Genetic GFN
0.00=0.00
0.00+0.00
0.00=0.00
No pssMiiLes
0.00=0.00
0.00=0.00
15.83
0.00
GenMol
0.00=0.00
0.00+0.00
0.00=0.00
0,20 45
0,40=0.89
0.40=0.89
14.33
1.00
Conditional Generation Models
LSTM
11.25=22.50
0.0020.00
1.57=15.89
0.20+0.45
25. 25.00:23.00
15.00
7.50
62.02
Graph-DiT
19.75+23.24
8 16.44
6.43+16.11 14
4,40+6.43 43
28.00=23.35
40.20=10.77
5.33
107.64
Learning from In-Context Demonstrations
D3
29.25=19.72
12.57=14.73
1.5724.16
0.80
1,80=3.49
19.80=15.6 64
8.67
65.79
GPT-40
22.25=25.70
12.29+13.49
7.14+11.95
80+6.33
19.60+22.65
44.80+0.45
4.67
109.88
Qwen-Max
3,50=6.35 35
0,00+0.00
0.00=0.00
0.00=0.00
4, 4.60=9.21
0.00=0.00
14,00
8.10
IJama3.1 1--.3.1-8BB.1
FT
0.75=1.50
2.7145.53
0.00=0.00
0.00+0.00
2.60&5.881
0.00=0.00
14.00
6.06
Owen3-8B-FT
8.25112.69
0.4310.53
0.86:2.27
0.3310.58
6.00a13.42
25.20=16.65
11.33
41.07
DemoDiff (Ours)
35.00=10.80
14.43+21.03
5.00&13.23
15.00
20.20+:23008
44.40+0.89
2.67
134.03
Performance alignment across methods may indicate task difficulty. is hard to give a formal measure of task difficulty because it depends on the oracle setup and on the quality of the data. We find that tasks where molecular optimization reaches good results, such as target based tasks or material design tasks, are also tasks where ICL methods can infer the core concept with only a few demonstrations. This pattern suggests that task difficulty may be partly reflected in the consistency across methods. There exceptions. As shown in Table for DRD2 and JNK3, molecular optimiza- tion does not perform well under limited supervision, while ICL methods, including DeepSeck- V3, GPT-40, and DefoDiff, reach strong results.
23

---

Published as a conference paper at ICLR 2026
Table 8: We compute the oracle and diversity scores from the Top-10 generations and report the harmonic mean of these two values. The results are reported for the Drug Design task category. Best results in each column are bolded.
Task
DRD2
JNK3
GSK3B
QED
Avg
Total
Rank
Sum
Tasks
33
33
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.31
0,90
8.50
1.63
REINVENT
0.31
0.21
0.26
0.90
9.50
1.68
GPBO
0.20
0.25
0.24
0.89
10.00
1.57
STONED
0.11
0.23
0.24
0.90
11.00
1.47
Genetic GFN
0.10
0.16
0.16
0.88
13.75
1.30
GenMol
0.68
0.29
0.48
0.77
6.75
2.22
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.60
0.18
0.40
0.79
9.25
1.97
REINVENT
0.71
12.50
1.51
GPBO
0.55
0.27
0.54
0.60
8:25
1.97
STONED
0.14
0.23
0.37
0.32
12.25
1.07
Genetic GFN
0.04
0.12
0.28
0.06
17.00
0.50
GenMol
0.56
0.39
0.49
0.12
8.50
1.57
Conditional Generation Models
LSTM
0.15
0.04
0.27
0.83
14.00
1.30
Graph-DiT
0.78
0.23
0.81
12.25
1.90
Learning from In- Contexx Demonstrations
DeepSeek-V3.
0.71
0.65
0.42
0.84
4.75
2.62
GPT-4o
0.13
0.84
11.25
1.90
Qwen-Max
0.00
0.14
0.36
0.68
15.00
1.18
Llama3. 1 1-8B-FT
0.06
0.21
0.76
0.19
12.25
1.22
Qwen3-E8 8B-B-BB6 FT
0.37
0.50
0.22
0.74
10.75
1.83
DemoDiff (Ours)
0.88
0.65
0.78
0.87
2.50
3.18
D.3 DETAILS ON CASE STUDIES
Figures 14 to 16 present case studies using negative demonstrations generate molecules with positive scores. The tasks include structure-constraine design of an isomer with 17 demonstrations, drug MPO of Osimertinib with 23 demonstrations, and protein target design of PARP1 with 26 demonstrations. These molecules contain 460, 631, and 481 atoms, respectively, far exceeding the 150-token context window under atom-level representations. The motif-level representation efficiently encodes all demonstrations within the same window.
24

---

Published as a conference paper at ICLR 2026
Table 9: We compute the oracle of the top generation. The results are reported for the Drug Design task category. Best results in each column are bolded.
Task
DRD2
JNK3
GSK3B
QED
Avg Rank
Total Sum
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.23
0.41
0.94
10.25
1.82
REINVENT
0.83
0.23
0.24
0.94
9.50
2.25
GPBO
0.23
0.23
0.23
0.94
12.00
1.64
STONED
0.19
0.23
0.41
0,94
12.25
1.77
Genetic GFN
0.19
0.14
0.15
0.95
13.50
1.43
GenMol
0.85
0.24
0.45
0.90
9.25
2.44
Molccular Optimization Methods with 10000 Predictor Calls
GraphGA
0.38
0,90
11.75
2.41
REINVENT
0.76
0.61
0.80
0.91
7.00
3.08
GPBO
0.99
0.19
0.47
0.93
9.50
2.58
STONED
0.25
0.21
0.44
0.30
13.75
1.20
Genetic GFN
0.03
0.41
0.61
0.07
12.75
1.12
GenMol
1.00
0.75
0.11
7.25
2.49
Conditional Generation Models
LSTM
0.28
0.06
0.26
0.85
15.25
1.45
Graph-DiT
0.99
0.78
14.00
2.06
Learning from In-Context Demonstrations
DeepSeek-V.
0.77
0.75
0.54
0.88
8.50
2.94
GPT-4o
0.94
0.17
0.85
13.75
2.17
Qwen-Max
0.00
0.26
0.89
0.89
10.50
2.04
Llama3. 1-8B-FT
0.03
0.33
0.72
0.26
12.25
1.34
Qwen3-8B-FT
0.90
1.00
0.69
0.94
4.25
3.54
DemoDiff (Ours)
1.00
0.93
2.75
3.65
Table 10: We compute the oracle and diversity scores from the Top-10 generations and report the harmonic mean of these two values. The results are reported for the Drug MPO task category. Best results in each column are bolded.
Task
Perindopril MPO
Ranolazine MPO
Osimertinib MPO
Zaleplon MPO
Sitagliptin MPO
Amlodipine MPO
Fexofenadine MPO
Total
Rank
Sum
Tasks
33
33
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.52
0.35
0.77
0.47
0.24
0.60
0.70
5.14
3.64
REINVENT
0.39
0.48
GPBO
0.51
0.39
0.76
0.45
0.23
0.59
0.68
7.57
3.60
STONED
0.52
0.35
0.47
0.24
0.60
3.64
Genetic GFN
0.52
0.47
0.77
0.43
0.09
0.58
0.70
8.00
3.54
GenMol
0.49
0.56
0.70
0.52
0.17
0.54
0.62
7.86
3.59
Molecular Oplinization Methods
10000 Predictor Calls
GraphGA
0.41
0.60
0.64
0.48
0.13
0.59
0.62
8.00
3.48
REINVENT
0.11
0.24
0.65
0.39
0.00
0.09
0.10
16.29
1.58
GPBO
0.23
0.36
0.68
0.45
0.12
0.62
0.67
9.29
3.12
STONED
0.53
0.26
0.47
0.47
0.01
0.60
0.45
10.14
2.79
Genetic GEN
0.00
0.01
0.00
0.00
0.00
18.71
0.56
GenMol
0.02
0.47
0.55
0.34
0.00
0.29
0.00
15.29
1.68
Conditional Generation Models
LSTM
0.06
0.14
0.26
0.09
0.13
0.21
1.10
Graph-Dii DIT
0.59
0.24
0.71
0.43
0.31
0.55
0.66
8.43
3.49
Learning from In- In-Context Demonstrations
DeepSeek-V3
0.42
3.55
GPT-4o
0.63
0.39
0.68
0.46
0.17
0.69
0.67
6.00
3,70
Qwen-Max
0.26
0.21
0.39
0.00
0.25
0.06
17.00
1.17
Llama3.1 1-SB-FT
0.05
0.39
0.26
0.00
0.00
0.42
0.55
16.00
1.67
Qwen3-8B-PP
0.27
0.39
0.36
0.03
0.06
0.40
0.38
14.14
1.88
DemoDiff (Ours)
0.52
0.58
0.80
0.43
0.09
0.62
0.73
5.00
3.78
25

---

Published as conference paper at ICLR 2026
Table 11: We compute the oracle of the top generation. The results are reported for the Drug MPO task category. Best results in each column are bolded.
Task
Perindopril MPO
Ranolazine
Osimertinib MPO
Zaleplon MPO
Sitgtlipinn MPO
Amlodipine MPO
Fexofenadine MPO
Avg
Total Sum
MPO
Molecular Optimization Methods with 100 Oracle Calls
GrapàGA
0.43
0.36
0.78
0.40
0.23
0,56
0.62
8.14
3.40
REINVENT
0.40
0.41
0.74
0.43
0.23
0.48
0.58
11.57
3.26
GPBO
0.38
0.36
0.78
0.37
0.23
0.48
0.60
13.00
3.20
STONED
0.43
0.36
0.78
0.40
0.23
0.56
0.62
9.57
3.40
Genetic GFN
0.42
0.61
0.75
0.46
0.16
0.49
0.66
9.29
3.54
GenMol
0.43
0.73
0.79
0.47
0.28
0.49
0.64
6.14
3.82
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.40
0.67
0.83
0.41
0.28
0,53
5.43
3.93
REINVENT
0.17
0.34
0.82
0.40
0.00
0.22
0.72
12.29
2.68
GPBO
0.64
STONED
0.44
0.26
0.77
0.42
0.01
0.52
0.62
11.29
3.03
Genetic GFN
0.00
0.01
0.00
0.00
0.00
18.00
1.35
GenMol
0.01
0.68
0.81
0.47
0.00
0.47
0.00
11.43
2.44
Conditional Generation Models
LSTM
0.08
0.11
0.22
0.44
0.42
0.46
15.71
2.31
Graph-DiT
0.58
0.51
0.61
0.46
0.46
0.48
0.61
9.00
3.71
I. Learning from In. Context Demonstrations
DeepSeck V3
0.74
0.77
0.78
0.39
0.10
0.77
0.63
7.29
4.17
GPT-4o
0.74
0.47
0.71
0.43
0.19
0.71
0.65
8.14
3.90
Qwen-Max
0.30
0.21
0.00
0.49
0.00
0.22
0.12
15.14
1.33
Llama3. 1-8B-FT
0.67
0.00
0.00
0.53
13:43
2.63
Qwen3 8B FT
0.42
0.57
0.67
0.08
0.22
0.74
0.68
10.00
3.38
(Ours)
Table 12: We compute the oracle and diversity scores from the Top-10 generations and report the harmonic mean of these two values. The results are reported for the Drug Rediscovery task category. Best results in each column are bolded.
Task
Celecoxib Rediscovery
Mestranol
Thiothixene
Troglitazone Rediscovery
Median
Median
Albuterol Similarity
Avg Rank
Total Sum
Similarity
Rediscovery
Tasks
33
33
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.39
0.38
0.34
0.26
0.29
0.49
9.14
2.54
REINVENT
0.39
0.44
0.37
0.34
0.30
0.28
0.50
8.29
2.62
GPBO
0.37
0,39
0.39
0.33
0.28
0.30
0.51
8.57
2.57
STONED
0.39
0.39
0.38
0.34
0.26
0.29
0.49
10.14
2.54
Genetic GPN
0.44
0.40
0.36
0.31
0.24
0.25
0.51
10.86
2,51
GenMol
0.48
0.38
0.29
0.41
0.37
0.55
6.86
2.93
Molecular Optimization Methods
Predictor
GraphGA
0.38
0.41
0.38
0.35
0.28
0.28
0.55
8.57
2.61
REINVENT
0.31
0.15
0.38
0.26
0.19
0.27
0.53
12.00
2.08
GPBO
0.34
0.37
0.26
0.19
0.27
0.46
12.71
2.32
STONED
0,39
0.49
0,38
0.32
0,19
0.27
0.26
10.86
2.30
Genetic GFN
0.27
0.21
0.12
0.08
0.09
0.11
18.86
0.96
GenMol
0.43
0.36
0.34
0.35
0.25
0.24
0.41
11.57
2.38
Conditional Generation Models
LSTM
0.61
0.66
0.20
0.25
0.20
0.12
0.70
10.14
2.74
Graph
0.60
0.33
0.22
0.39
0.17
8.29
3.02
Learinng from In-Context Demonstrations
DeepSeck-V3
0.54
0.46
0.47
0,57
0.18
0.25
0.70
5.86
3.18
GPT-4o
0.54
0.64
0.61
0.37
0.20
0.19
0.73
5.29
3.28
Qwen-Max
0.48
0.00
0.00
0.00
0.15
0.40
17.29
1.03
Llama3, 1- SB-FT
0.16
0.35
0.24
0.00
0.05
0.10
0.35
18.43
1.25
Qwen3-SB-FT
0.49
0.32
0.40
0.09
0.18
0.65
10.00
2.58
DemoDiff (Ours)
0.50
0.57
0.42
0.49
0.12
0.23
0.75
6.29
3.09
26

---

Published as a conference paper at ICLR 2026
Table 13: We compute the oracle of the top generation. The results are reported for the Drug Rediscovery task category. Best results in each column are bolded.
Task
Celecoxib Rediscovery
Mestranol Similarity
Thiothienee
Troglitazone
Median
Median
Albuterol Similarity
Avg Rank
Total Sum
Rediscovery
Rediscovery
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.27
0.28
0.28
0.23
0.23
0.23
0.40
10.86
1.93
REINVENT
0.34
0.39
0.32
0.25
0.23
0.23
0.41
8.57
2.18
GPBO
0.28
0,31
0.29
0.23
0.23
0.23
0.41
10.43
1.99
0.27
0.28
0,23
0.40
12.57
1.93
Genetic GFN
0.38
0.30
0.26
0.21
0.18
0.16
0.46
13.14
1.95
GenMol
0.39
0.31
0.27
0.36
0.38
0.56
6.86
2.65
Molecular Optimization
Predictor
GraphGA
0.35
0.28
0.25
0.21
0.20
0.62
9.43
2.30
REINVENT
0.36
0.28
0.28
0.27
0.13
0.20
1.00
10.00
2.51
GPBO
0.25
0.26
0.13
1.00
12.00
2.61
STONED
0.27
0.64
0,28
0.23
0.13
0.20
0.22
12.71
1.96
Genetic GFN
0.28
0.13
0.17
0.15
0.06
0.10
0.19
18.00
1.09
GenMol
1.00
0.28
0.38
0.25
0.19
0.55
7.57
3,06
Conditional Generation Models
LSTM
1.00
0.82
0.18
0.33
0.20
0.09
0.69
9.29
3,31
Graph-DiT
0.78
0.51
0.18
0.40
0.10
0.81
8.14
3.25
Learning Context Demonstrations
DeepSeck-V3
0.77
1.00
0.84
0.75
0.12
0.17
1.00
6.00
4.64
GPT-4o
0.53
0.68
0.81
0.46
0.15
0.15
0.93
6.86
3.70
Qwen-Max
0.72
0.00
17.29
1.23
1lama3.1-8-FF7
0.13
0.31
0.18
0.00
0,07
0.09
1.00
15.57
1.78
Qwen3-8B-FT
0.77
0.58
0.27
0.43
0.10
0.15
1.00
9.14
3.30
Difo (Ours)
0.84
0.64
0.34
0.65
0.14
0.16
1.00
5.57
3.77
Table 14: We compute the oracle and diversity scores from the Top- 10 generations and report the harmonic mean of these two values. The results are reported for the Material Design task category. Best results in each column bolded.
Task
Polymer C02/CH4
Polymer C02/N2
Polymer H2/CH4
Polymer H2/22
Polymer O2/N2
Avg Rank
Total Sum
Tasks
33
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.59
0.42
0.71
0.52
0.64
5.80
2.88
REINVENT
0.00
0.00
0.00
0.00
0.00
17.00
GPBO
0.46
0.31
0.76
0.79
0.68
3.80
3.00
STONED
No PSMILES
No PSMILES PSMILES
No PSMILES PSMILES
No PSMILES PSMILES
No PSMILES
18.00
0.00
Genetic GFN
PSMILES
PSMILES
14.40
0.00
GenMol
0.60
0.50
0.63
0.72
0.63
4.20
3.08
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA REINVENT
0.60
0.55
0.75
0.40
0.44
5.20
2.75
0.32
0.37
0.56
0.69
0.61
7.20
2.55
GPBO
0.55
0.00
0.57
0.52
0.46
10.20
2.09
STONED
No PSMILES
No PSMILES
No PSMILES PS
No PSMILES
No PSMILES
19.20
0.00
Genetic GFN
No PSMLLES
No PSMILES
No PSMILES
No PSMILES
13.40
0.00
GenMol
0.07
0.31
0.57
0.12
0.00
11.60
1.07
Conditional Generation Models
LSTM
0.00
0.29
0.16
0.12
0.23
13.60
0.80
Graph-DiT
0.27
0.55
0.74
0.57
0.63
5.40
2.76
Learning from In- Context Demonstrations
DeepSeek-V3
0.13
0.45
0.53
0.68
0.15
8.80
1.94
GPT-4o
0.37
0.30
0.44
0.70
0.33
8.60
2.14
Qwen-Max
0.00
0.09
0.00
0.42
16.20
0.51
Llama3.1-88 1-8B-FT
0.00
0.00
0.00
0.57
0.00
13.60
0.57
BWen3-8B-FT
0.00
0.00
0.75
0.56
0.00
11.60
1.32
DemoDiff (Ours)
0.63
0.52
0.72
0.81
0.68
2.20
3.36
27

---

Published as a conference paper at ICLR 2026
Table 15: We compute the oracle of the top generation. The results are reported for the Material Design task category. Best results in each column are bolded.
Task
Polymer CO2/CH4
Polymer C02/N2
Polymer H2/CH4
Polymer H2/N2
Polymer O2/N2
Avg Rank
Total
Sum
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.79
0.41
1.00
0.57
0.84
7.00
3.60
REINVENT
0.00
0.00
0.00
0.00
0.00
17.20
0,00
GPBO
0.89
0.43
1.00
1.00
0.68
5,40
4.00
STONED
PSMLLES
PSMILES
PSPSLIESS No PSPSMILES
PSMILES
PSMILES
18.20
0.00
Genetic GFN
No PSMILES
No PSMILES
No SSMILLE
No PSMILES
15.00
GenMol
0.76
0.70
1.00
1.00
0.88
3.20
4.34
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.82
1.00
0.51
0.50
7.60
3.52
REINVENT
0.71
0.67
1.00
1.00
0.97
4.60
4.35
GPBO
0.67
0.00
0.80
0.85
0.57
11.20
2.90
STONED
PSMILES
PSMILES
PSMILES PSMILESSS
PSMILES No PSMSIISS
PSMILES
19.40
0.00
Genetic GFN
No PSMILES
No PSMILES
No PSMILES
14.00
GenMol
0.11
0.30
1.00
0.37
0.39
10.80
2.17
Conditional Generation Models
LSTM
0.00
0.77
0.55
0.63
0.44
11.60
2.39
Graph-DiT
0.44
1.00
1.00
0.79
0.69
6.80
3.92
Learning from In- Context Demonstrations
DeepSeek-V V3
0.60
0.86
7.60
3.76
GPT-4o
0.67
0.52
1.00
1.00
0.65
8.00
3.85
Owen-Max
0.00
0.34
0.00
0.02
0.91
13.80
1.27
Llama3. 1-8B-FT
0.00
0.32
0.00
0.67
0.00
13.60
0.99
Qwen3-8B-FT
0.00
0.00
0.98
0.89
0.00
13.00
1.87
DefoDiff (Ours)
1.00
0.64
1.00
1.00
1.00
2.00
4.64
28

---

Published as a conference paper at ICLR 2026
Table 16: We compute the oracle the top generation. The results are reported for the Structure Constrained Design task category. Best results in each column are bolded.
Task
Isomers c7h8n2o2
Isomers c9h10222pf22l
Deco Hop
Scaffold
Valsartan Smarts
Avg Rank
Total
Hop
Sum
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.55
0.47
0.58
0.45
0.23
9.80
2.29
REINVENT
0.55
0.44
0.58
0.45
0.23
10.00
2.26
GPBO
0.55
0.50
0.59
0.47
0.23
8.20
2.34
STONED
0.55
0.47
0.58
0.45
0.23
11.60
2.29
Genetic GFN
0.29
0.54
0.58
0,45
0.00
GenMol
0.80
0.69
0.58
0.46
0.00
10.00
2.53
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
1.00
0.78
0.54
0.39
0.00
11.40
REINVENT
1.00
0.00
0.62
0.47
0.00
9.20
2.09
GPBO
0.88
0.47
0.55
0.45
0.00
13.80
2.36
STONED
0.82
0.72
0.57
0.47
0.00
10.80
2.57
Genetic GFN
0.00
0.00
0.56
0.39
0.00
GenMol
0.00
0.02
0.84
0.47
0.03
10.00
1.36
Conditional Generation Models
LSTM
1.00
0.87
0.54
0.78
0.00
3.18
Graph-Dit DiT
0.88
0.82
0.53
0.77
0.00
10.20
3.01
Learning from In Context Demonstrations
DeepSeek-V3
0.72
0.50
0.84
0.21
GPT-44
0.74
0.82
0.59
0.46
0.00
7.80
2.61
Owen-Max
0.90
0.73
0.56
0.45
0.01
9,40
2.67
Mamaa.11-BB
0.00
0.00
0.85
0.41
0.00
14.80
1.26
Qwen3-8B-FT
0.05
0.04
0.87
0.47
0.00
10.40
1.42
DefoDiff (Ours)
0.88
0.73
0.86
0.49
0.00
29

---

Published as a conference paper at ICLR 2026
Table 17: We compute the oracle and diversity scores from the Top-10 generations and report the harmonic mean of these two values. The results are reported for the Structure Constrained Design task category. Best results in each column are bolded.
Task
Isomers c7h8n2o2
Isomers
Deco Hop
Scaffold Hop
Valsartan Smarts
Avg Rank
Total
c9h 1002ppf22l
Sum
# Tasks
33
33
Molecular Optimization Methods with 100 Oracle Calls
GraphGA
0.30
0.44
0.68
0.57
0.14
7.60
2.13
REINVENT
0.36
0.40
0.68
0.57
0.14
8.00
2.15
GPBO
0.27
0.45
0.68
0.57
0.14
STONED
0.18
0.39
0.68
0.57
0.14
9.00
1.97
Genetic GFN
0.15
0,50
0,68
0,57
0.00
9.60
1.89
GenMol
0.39
0.56
0.60
0.54
0.00
11.40
2.08
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.71
0.69
0.42
0.45
0.00
11.60
2.27
REINVENT
0.49
0.00
0.46
0.30
0.00
16.00
1.24
GPBO
0.69
0.29
0.65
0.53
0.00
12.80
2.16
STONED
0.62
0.65
0.68
0.57
0.00
9.20
2.51
Genetic GFN
0.00
0.00
0.21
0.17
0.00
17.80
0.38
GenMol
0.00
0.01
0.63
0.42
0.01
14.60
1.06
Conditional Generation Models
LSTM
0.80
0.75
0.65
0.55
0.00
9.20
2.75
Graph-DiT
0.85
0.74
0.66
0.66
0.00
7.40
2.91
Learning from In-Context Demonstrations
DeepSeck-Vi V3
0.64
0.47
0.70
0.52
0.09
7.60
2.43
GPT-4o
0.61
0.78
0.67
0.54
0.00
8.00
2.60
Qwen-Max
0.70
0.55
0.00
0.36
0.01
11.60
1.62
1lama3.1-8B.
0.00
0.00
0.71
0.00
14.00
Qwen3-8B-F FB--T
0.01
0.02
0.70
0.55
0.00
11.20
1.28
DemoDiff (Ours)
0.82
0.77
0.64
0.57
0.00
6.40
2.80
30

---

Published as a conference paper at ICLR 2026
Table 18: We compute the oracle of the top generation. The results are reported for the Target Based Design task category. Best results in each column are bolded.
Task
Docking Braf
Docking
Docking
Docking Fa7
Docking 5HTIB
Avg Rank
Total Sum
Parp1
Jak2
Molecular Optimization Methods with 100 Oracle Calls
GraphGA REINVENT
0.75
0.76
0.74
0.67
0.76
8.80
3.69
0.77
0.80
0.69
0,66
0.80
7.60
3.73
GPBO
0.79
0.83
0.72
0.64
0.78
7.00
3.76
STONED
0.74
0.77
0.77
0.64
0.76
9.20
3.69
Genetic GPN
0.79
0.79
0.71
0.70
0.74
7.60
3.73
GenMol
0.79
0.79
0.70
0.69
0.87
6,20
3.84
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.84
0.91
0.79
0.66
0.95
3.00
4.15
REINVENT
0.68
1.00
0.59
0.69
0..00
11.60
2.96
GPBO
0.84
0.91
0.84
0.71
0.91
2.00
4.21
STONED
0.71
0.00
0.61
0.00
0.00
18.20
1.32
Genetic GFN
0.71
0.00
0.44
0.00
15.60
1.93
GenMol
0.74
0.76
0.72
0.51
0.79
11.40
3.51
Conditional Generation Models
LSTM
0.63
0.61
0.92
11.00
3.66
Graph-DiT
0.72
0.77
0.66
0.59
0.74
12.80
3.49
Learning from In Context Demonstrations
DeepSeek-V?
0.81
0.52
0.72
10.00
3.57
GPT-4o
0.77
0.72
0.64
0.57
0.76
12,80
3.46
Qwen-Max
0.54
0.00
0.00
0.51
0.00
19.00
1.06
Llama3. 1-8B-FT
0.00
0.00
0.42
0.00
0.00
18.40
0.42
Qwen3-8B-FT
0.76
0.73
0.75
0.56
0.71
12.00
3.51
DemoDiff (Ours)
0.79
0.60
0.77
5.80
3.84
31

---

Published as a conference paper at ICLR 2026
Table 19: We compute the oracle and diversity scores from the Top-10 10 generations and report the harmonic mean of these two values. The results are reported for the Target Based Design task category. Best results in each column are bolded.
Task
Docking Braf
Docking Parp1
Docking Jak2
Docking Fa7
Docking
Avg Rank
Total
SHTIB
Sum
Tasks
33
33
Molecular Optimization Methods with 100 Oracle Calls
GraphGA REINVENT
0.78
0.79
0.76
0.70
0.79
3.20
3.82
0.77
0.79
0.73
0.71
0.78
4.40
3.78
GPBO
0.79
0.76
0.70
0.78
4.40
3.79
STONED
0.77
0.78
0.77
0.70
0.78
4.60
3.80
Genetic GFN
0.78
0.79
0.75
0.70
0.77
4.20
3.79
GenMol
0.69
0.70
0.66
0.67
0.73
10.80
3.45
Molecular Optimization Methods with 10000 Predictor Calls
GraphGA
0.66
0.70
0.60
0.57
0.70
13,20
3.22
REINVENT
0.19
0.27
0.31
0.09
0.00
17.60
0.86
GPBO
0.69
0..78
0.76
0.73
0.74
6.80
3.70
STONED
0.00
0.52
0.00
0.00
17.60
1.00
Genetic GFN
0.34
0.20
0.00
0.19
0.00
17.40
0.73
GenMol
0.47
0.55
0.52
0.48
0.59
15.80
2.60
Conditional Generation Models
LSTM
0.73
0.73
0.71
0.66
0.77
8.40
3.61
Graph-DiT
0.71
0.74
0.70
0.66
0.75
9.40
3.55
Learning from In- In-Context Demonstrations
DeepSeek-VV V3
0.63
0.64
0.63
0.57
0.73
13.20
3.19
GPT-4o
0.77
0.74
0.70
0.66
0.76
8.20
3.63
Qwen-Max
0.48
0.00
0.00
0.49
0.00
18.20
0.96
1lama3.1-8B-FF
0.11
0.00
0.00
18.40
0.11
Qwen3-8B-FF
0.70
0.73
0.66
0.55
0.72
12.00
3.35
DemoDiff (Ours)
0.81
0.82
0.78
0.69
0.79
2.20
3,90
32

---

Published as a conference paper at ICLR 2026
Demonstrations
1.23e-02
4.09e-03
4.09e-03
1.13e-08
8.15e-09
3.41e-10
9.10e-13
3.05e-16
5.18e-17
7.61e-21
6.49e-27
2.23e-39
a
4.07e-50
2.43e-73
2.26e-84
5.58e-85
0.00e+00
Query: Score 1
NHz
0.7225
Figure 14: Structure constrained generation for Isomer with all negative demonstrations.
33

---

Published as a conference paper at ICLR 2026
Demonstrations
4.05e-01
3.18e-01
2.63e-01
2.16e-01
2.13e-01
2.13e-01
2.10e-01
1.48e-01
1.02e-01
8.49e-02
8.22e-02
8.15e-02
6.47e-02
5.40e-02
4.26e-02
2.92e-02
8
2.92e-02
1.34e-02
4.29e-03
7.64e-05
2.25e-07
2.25e-07
2.25e-07
Query: Score
0.7808
Figure 15: Drug MPO for Osimertinib with all negative demonstrations.
34

---

Published as a conference paper at ICLR 2026
Demonstrations
4.93e-01
4.93e-01
4.93e-01
4.93e-01
4.93e-01
4.93e-01
4.93e-01
4.93e-01
4.93e-01
4.86e-01
4.86e-01
4.86e-01
4.86e-01
4.86e-01
4.79e-01
4.79e-01
4.79e-01
4.71e-01
4.71e-01
4.64e-01
4.64e-01
4.57e-01
4.43e-01
3.86e-01
0.00e+00
0.00e+00
Query: Score
0.6714
Figure 16: Target-based design for PARP1 with all negative demonstrations.
35


Reviews:
[{"review_id": "QqYC8uQ5gC", "reviewer": "Reviewer_v13C", "summary": "This work introduces a conditional generative model for molecules based on a graph diffusion transformer.\nTo reduce the context size, atoms are grouped together using a BPE-like algorithm adapted to graphs.\nConditioning is done by using pairs of molecules with normalised scores as _property_ inputs to the graph diffusion transformer.\nExperiments on a dataset extracted from ChEMBL and a collection of polymer datasets show favourable performance in terms of oracle and diversity scores.", "questions": "1. Is there any/What is the relation between NPE and the computation of molecular fingerprints?\n 2. Is there any/What is the relation between NPE and the use of virtual nodes (cf. Hwang et al., 2022) in GNNs?\n 3. Could the proposed model architecture also work with non-graph (e.g. SMILES) inputs?\n 4. How was the LSTM applied to the NPE-encoded graphs?\n 5. Are there no other (e.g. SMILES-based) specialised generative models that could be used as baselines?\n 6. Can you explain what is happening in Figure&nbsp;2?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Overall, the paper is well written.\n - Most reported results come with error bars.\n - Reported results seem to be competitive.\n - Being able to control molecule generation on activity levels seems relevant.", "weaknesses": "- The proposed node pair encoding feels similar in spirit to how (extended connectivity) fingerprints are computed.\n   However, the relation to these possibly related methods seems to be underexplored in the current version of the manuscript.\n   It would be good to have an explanation how NPE is similar/different from fingerprint computations.\n   If there is enough similarity, also an empirical verification of NPE vs fingerprint features would be meaningful.\n   I also feel that NPE has some similarities with virtual nodes used in some GNNs (e.g. Hwang et al., 2022).\n   A discussion on possible similarities/differences in this regard might be useful as well.\n - There is little to no motivation on why molecular graphs are used instead of e.g. SMILES.\n   I have often heard that GNNs bring little to no performance advantage over working with SMILES (e.g. Renz et al., 2024).\n   Furthermore, GNNs are typically much more complex to work with than models that work with SMILES.\n   Given that the core Graph DiT idea seems transferrable to non-graph architectures as well,\n   it would have been interesting to investigate how important the _graph_ aspect of this model is.\n - It is not entirely clear how strong the provided baselines are.\n   Most notably, there is little information on how the LSTM was used to operate on the NPE encodings.\n   Also, it seems hard to imagine that there are no more specialised models that are able to generate molecules in-context.\n   Especially in the context of autoregressive SMILES generation (e.g. Renz et al., 2024; Schmidinger et al., 2025)\n - Figure&nbsp;2 confuses me more than it helps me to understand the method.\n   I understand that molecules are encoded by grouping sub-structures,\n   but there is no explanation for what $S_\\mathrm{ingle}$ is supposed to be,\n   or what the digits in the brackets are supposed to mean.\n   Furthermore, the inputs ($[?, S0]$ and $[?, S5, S4, S7]$) to the transformer,\n   which I would have assumed are on top of the demo tokens,\n   seem to be the sub-motifs from the tokenized molecule illustrated to the right.\n   However, this tokenized molecule should be part of the demo tokens.\n   This all seems to make little sense and is hard to connect to the explanations in the main text.\n\n### Minor Issues\n - The expression \"molecule-assay pair\" (line 257) seems to be a bit weird.\n   Could it be that you mean \"molecule-activity pair\",\n   i.e., a molecule with its activity value for a particular assay (a.k.a. the context)?\n - In the main text (line 302) _10_ novel, unique and valid molecules are mentioned for evaluation,\n   but in the appendix, line 1054 mentions _100_ molecules for evaluation.\n - Table&nbsp;1 is claimed to report two scores (oracle and diversity),\n   but there is only a single value for each model-task combination.\n\n### Additional References\n - Hwang et al. (2022). [An analysis of virtual nodes in graph neural networks for link prediction](https://openreview.net/forum?id=dI6KBKNRp7). In The first learning on graphs conference.\n - Renz et al. (2024). [Diverse hits in de novo molecule design: Diversity-based comparison of goal-directed generators](https://pubs.acs.org/doi/full/10.1021/acs.jcim.4c00519). Journal of Chemical Information and Modeling, 64(15), 5756-5761.\n - Schmidinger et al. (2025). [Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences](https://openreview.net/forum?id=IjbXZdugdj). International Conference on Learning Representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zQweiYhvci", "reviewer": "Reviewer_aBww", "summary": "The authors propose DemoDiff, a generative model that uses molecule-score examples for in-context molecular design. The authors also develop Node Pair Encoding (NPE), a strategy to represent molecules at the motif level, which requires 5.5x fewer nodes on average. DemoDiff achieves impressive performance across 33 molecular design tasks spanning 6 categories.", "questions": "LLMs are not necessarily trained to do in-context learning or to do molecular design, which makes it difficult to compare to your proposed method which is trained on a large and specialized dataset of molecule-spectra pairs. Did the authors consider finetuning LLMs to do in-context molecular design on the same dataset used for DemoDiff pretraining? Do you think DemoDiff would outperform such a finetuned LLM?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "- The proposed NPE reduces the number of nodes by 5.5x, which is impressive and is a generally useful contribution even outside the context of DemoDiff. \n- The proposed in-context learning approach is novel and a unique contribution to the ML molecular design literature \n- DemoDiff demonstrates promising results on the drug/material design categories and is able to generate more diverse samples than baseline methods\n- The authors provide extensive ablation and case studies", "weaknesses": "I am not very convinced by the main experimental results and I think Table 1 is somewhat misleading. Firstly, it is a bit unfair to compare to LLMs given that DemoDiff is trained on >1M molecule-property pairs and LLMs are not designed for this specific task. A more fair comparison would be to finetune the LLMs to do in-context molecule generation in the same way as DemoDiff. Secondly, and more importantly, after looking at the results in the Appendix, it appears that DemoDiff only achieves SOTA performance on 13 out of the 33 tasks (based on the Top-10 Harmonic Mean results). In particular, outside the drug/material design categories, DemoDiff does not reliably outperform baseline LLMs. Given that DemoDiff success is limited to a small subset of tasks it is difficult to support the claim that this is a generalizable method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MCmgmDxJAA", "reviewer": "Reviewer_Nsot", "summary": "This paper proposes a novel molecular design framework, DemoDiff, which integrates diffusion models with in-context learning. By introducing a Node Pair Encoding (NPE) tokenizer for molecular graphs, the model achieves efficient pretraining with motif-level representations. A 0.7B-parameter model is pretrained on large-scale, multi-task data covering 155K properties and millions of molecules. DemoDiff achieves an average rank of 3.63 across 33 molecular design tasks, outperforming ten baseline models and several LLMs.", "questions": "1. What does the “Total sum” represent — which scores are included in this total? In Eq. 6, what is the function denoted by T?\n2. The authors interpret DemoDiff’s in-context learning as implicit Bayesian inference. Could this interpretation be validated via attention pattern or diffusion trajectory visualizations?\n3. NPE is a frequency-driven motif discovery algorithm. Do these automatically learned motifs correspond to known chemical functional groups or reaction fragments? How well does NPE generalize to out-of-distribution (OOD) chemical spaces? PRODIGY [1] also applies in-context learning on graphs — please provide a comparison.\n4. The authors mention that the property distribution follows Zipf’s law (Figure 3a). Has the effect of this long-tailed distribution on the model’s ICL ability been evaluated? Does DemoDiff maintain strong performance on low-frequency property tasks? Since the number of examples per assay is imbalanced, could this lead to training bias toward high-frequency tasks?\n5. The authors use consistency score and related metrics. Could evaluation be extended to include metrics more closely tied to molecular properties, such as QED or SA score?\n\nReference:\n\n[1] PRODIGY: Enabling In-context Learning Over Graphs", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. DemoDiff implicitly embeds task information into the diffusion denoising process. The proposed Node Pair Encoding (NPE) is a substantive contribution to molecular graph tokenization, eliminating the need for handcrafted reaction rules and enabling automatic motif discovery.\n2. Extensive evaluation on 33 tasks demonstrates that DemoDiff significantly outperforms both domain-specific baselines and ICL-based LLMs. Despite having only 0.7B parameters, the model matches or even surpasses general-purpose LLMs.\n3. The paper provides detailed descriptions of the pretraining task design, motif tokenizer, context consistency score, and diffusion inversion inference. The experiments offer comprehensive analyses of parameter scale, context length, positive/negative sample ratios, and consistency filtering.", "weaknesses": "1. Lack of theoretical analysis. Although the paper mentions an implicit Bayesian interpretation, it lacks formal proofs or theoretical analysis on how the diffusion trajectories reflect “posterior inference of task concepts.” The comparison with ICL in language models remains largely analogical, without quantitative or interpretable mechanism-level parallels.\n2. Potential bias in data and task construction.\n- The pretraining tasks mainly rely on ChEMBL and polymer datasets, whose property distributions are highly skewed (Zipf-like). While the authors claim this facilitates ICL ability, no systematic study is provided on the relationship between task frequency and generalization performance.\n- The oracle evaluation for generation tasks depends on human scoring and predictor models, which may introduce noise.\n3. Insufficient validation of NPE’s chemical semantic consistency. The paper lacks statistical verification of motif interpretability or chemical validity. Although DemoDiff outperforms Graph-DiT, no strictly fair comparison (e.g., controlled parameter size or identical token count) is presented.\n4. High pretraining cost. The work lacks small-scale, reproducible experiments or an ablation-only release.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wRi6dyHHkx", "reviewer": "Reviewer_t5zC", "summary": "Through this paper, the authors propose DemoDiff, a demonstration-conditioned diffusion model that has a Graph DiT as the backbone, to construct a molecular generative model under in-context learning. In addition, the authors also propose a molecular tokenizer trained with Node Pair Encoding (NPE) for motif-level representation to support efficient pretraining of DemoDiff.", "questions": "Please see the *Weaknesses* section for my main concerns.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The authors provided the codebase.\n- The introduction of a motif-level tokenizer using Node Pair Encoding, which reduced the node count by 5.5×, is a reasonable approach in terms of both efficiency and performance.\n- The proposed DemoDiff was evaluated across 33 design tasks spanning 6 categories, achieving SOTA performance in most of them.", "weaknesses": "Weaknesses\nI will combine the *Weaknesses* section and the *Questions* section. My concerns are as follows:\n- There is little discussion on how sensitive the demonstration/context selection is to performance and how well the derived context applies to diverse new tasks.\n- SOTA molecular optimization baselines such as GenMol [1] and Genetic GFN [2] are missing. Comparisons with these baselines are necessary for the results to be considered meaningful.\n- No computational or memory efficiency was reported. This would be an effective method to demonstrate that the proposed DemoDiff is more efficient compared to larger generalist LLMs.\n\n---\n\n**References:**\n\n[1] Lee et al., GenMol: A Drug Discovery Generalist with Discrete Diffusion, ICML 2025.\n\n[2] Kim et al., Genetic-guided GFlowNets for Sample Efficient Molecular Optimization, NeurIPS 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fl7nTnB7PC", "reviewer": "Reviewer_WQjW", "summary": "This paper introduces a new paradigm for molecular design, named DemoDiff, which combines in-context learning with graph diffusion models to tackle property-guided molecular generation. The authors' core idea is to use molecule-property score pairs (demonstrations) to define a task, guiding a Graph Diffusion Transformer during the generation process. To support this framework, the paper also designs Node Pair Encoding, which compresses molecules from the atom level to the motif level, significantly increasing the context length the model can handle. This work is novel in its conception and experimentally thorough, demonstrating its potential as a foundation model for molecular design.", "questions": "1. Regarding model scaling (Table 2), the data shows that for the \"Structure Constrained\" task, the small (78M) and medium (311M) models outperform the large (739M) model (0.59/0.63 vs. 0.56). This contradicts the paper's conclusion that \"the benefits of parameter scaling become more evident at the large scale\". How do you explain this performance degradation? \n2. The core advantage of ICL is its ability to generalize to new tasks with few shots. The paper states that the 33 evaluation tasks are distinct from the pre-training data. However, given the scale of the pre-training data, I am concerned about the degree of this distinctness. To what extent are the chemical spaces, property targets, or molecular scaffolds in the evaluation tasks truly unseen during pre-training? \n3. In the baseline comparison, the paper includes general-purpose LLMs like GPT-4O but does not compare against LLMs pre-trained on chemical texts or molecular sequences (e.g., SMILES), which possess domain-specific knowledge. Why were these models not included in the comparison?\n4. The introduction states that directly applying the autoregressive framework of LLMs is \"infeasible\", thus justifying the choice of GraphDiT as the backbone. However, molecules can be represented as sequences like SMILES or SELFIES, and some baselines are indeed based on them. Is there any evidence to support that GraphDiT is superior to a powerful Transformer decoder trained on SMILES sequences within this ICL framework?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- A Novel and Inspired Framework: The idea of using \"demonstrations\" as a condition to guide a diffusion model is highly inspiring. DemoDiff dynamically defines the task context through molecule-score pairs, which is not only more flexible and scalable but also enables the model to learn and generalize to new, data-scarce tasks. \n- An Efficient and Innovative Graph Tokenizer: NPE is a purely data-driven motif learning algorithm that adaptively discovers high-frequency patterns from the pre-training data. Experiments show that this method achieves an average compression ratio of 5.5x , which directly supports the conclusion that the model performs better with longer contexts.\n- Comprehensive and In-depth Experimental Evaluation: The paper conducts a comprehensive test on 33 tasks across 6 major categories, including drug and materials design. It compares DemoDiff against 13 baselines, including traditional optimization algorithms, conditional generation models, and even general-purpose large language models with 100-1000x more parameters.", "weaknesses": "- Regarding Consistency Score: According to the appendix, the core performance metrics (Table 1) are not derived directly from DemoDiff's output but are obtained by first generating 1000 candidates and then using the consistency score to filter for the top 100 for evaluation. This introduces a significant confounding variable, making it impossible to determine the model's inherent generative capability. Also, Figure 5 shows this score has a very low correlation with the true oracle score for key tasks like Drug MPO and Materials Design (correlation coefficient close to 0), which questions its validity as an effective filtering metric.\n- NPE Tokenizer's Robustness and Limitations: While the paper showcases the excellent compression efficiency of the NPE tokenizer, one of its core contributions, it fails to discuss its performance when handling rare chemical motifs or molecules that deviate significantly from the pre-training data distribution. As a component intended to be the core of a foundation model, its reliability in out-of-distribution (OOD) scenarios is crucial. Furthermore, the claim of \"lossless reconstruction\" remains a qualitative description, lacking quantitative reconstruction accuracy metrics on a standard dataset, which makes it difficult to assess its reliability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)