3D Structure Prediction of Atomic Systems with Flow-based Direct Preference Optimization
Authors: Rui Jiao
Xiangzhe Kong
Wenbing Huang
Yang Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Flow Matching
Direct Preference Optimization
Geometric Graph Neural Networks
Structure Prediction
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: We propose FlowDPO, a novel framework to enhance Flow Matching models via Direct Preference Optimization on 3D structure prediction tasks.

Abstract: Predicting high-fidelity 3D structures of atomic systems is a fundamental yet challenging problem in scientific domains. While recent work demonstrates the advantage of generative models in this realm, the exploration of different probability paths are still insufficient, and hallucinations during sampling are persistently occurring. To address these pitfalls, we introduce FlowDPO, a novel framework that explores various probability paths with flow matching models and further suppresses hallucinations using Direct Preference Optimization (DPO) for structure generation. Our approach begins with a pre-trained flow matching model to generate multiple candidate structures for each training sample. These structures are then evaluated and ranked based on their distance to the ground truth, resulting in an automatic preference dataset. Using this dataset, we apply DPO to optimize the original model, improving its performance in generating structures closely aligned with the desired reference distribution. As confirmed by our theoretical analysis, such paradigm and objective function are compatible with arbitrary Gaussian paths, exhibiting favorable universality. Extensive experimental results on antibodies and crystals demonstrate substantial benefits of our FlowDPO, highlighting its potential to advance the field of 3D structure prediction with generative models.

3D Structure Prediction of Atomic Systems with Flow-Based Direct Preference Optimization
Rui Ji Xiangzhe Kong Wenbing Huang Yang Liu
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University
Institute for AIR, Tsinghua University
Gaoling School of Artificial Intelligence, Renmin University of China 4 Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China
Abstract
Predicting high-fidelity 3D structures of atomic systems is fundamental yet challenging problem in scientific domains. While recent work demonstrates the advantage of generative models in this realm, the exploration of different probability paths are still insufficient, and hallucinations during sampling are persistently occurring. To address these pitfalls, introduce FlowDPO, novel framework that explores various probability paths with flow matching models and further suppresses hallucinations using Direct Preference Optimization (DPO) for structure generation. Our approach begins with pre-trained flow matching model to generate multiple candidate structures for each training sample. These structures are then evaluated and ranked based on their distance to the ground truth, resulting in automatic preference dataset. Using this dataset, We apply DPO to optimize the original model, improving its performance in generating structures closely aligned with the desired reference distribution. As confirmed by our theoretical analysis, such paradigm and objective function are compatible with arbitrary Gaussian paths, exhibiting favorable universality. Extensive experimental results on antibodies and crystals demonstrate substantial benefits of our FlowDPO, highlighting its potential advance the field of 3D structure prediction with generative models.
Introduction
Predicting 3D structures atomic systems indispensable various scientific domains, ranging from pharmaceutical drug design [1] [7 to materials science [7]. Accurate 3D modeling is not only crucial for understanding the physical and chemical properties of substances at the atomic level [2] 18 but also for simulating and predicting their behavior in various environments [3] 26]. Nevertheless, it remains challenging due to the intricate nature of atomic interactions, the vastness of the conformational space, well limited resources of structure data.
Conventional methods typically employ physics- based algorithms to derive structures at local energy optimum 24 (1 (32. Recent advancements leverage deep generative model to learn the distribution of stable structures from available data, showcasing remarkable success across various domains. For example, DiffAb 20 designs diffusion-basee method for antigen-specific antibody design, which further available for antibody structure prediction, and DiffCSP [44 proposes a joint diffusion framework for crystal structure prediction. Despite these advancements, generative models for structure prediction are confronted with two primary challenges.
First, existing structure prediction methods predominantly utilize diffusion iuffusion-bbsed generative models. While effective, this foeus narrows the scope of exploration into other probability paths that could
Wenbing Huang and Yang Liu corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

D pair
Lve
Oref
Drel
Oref
Lyp
Pper
d(xioRag)
Opt
Init.
8
*
LoT
R1js
8112
reef
LnPo
(a) Training Flow Models
(b) Preference Dataset Construction
(c) Direct Preference Optimization
Figure Overview of the proposed FlowDPO pipeline. described in Section 3.1 the process begins by training flow matching model, denoted as lier. using arbitrary pre-defined Gaussian path. Next, as outlined in Section 3.2 we construct preference dataset, Paair by evaluating the distances between generated samples Tij and the ground structure r; under given context condition C c--such as an antibody sequence or crystal composition. These samples are derived from the reference training set Diet- This dataset is then used to fine-tune the model opt through the DPO training objective Cppo, detailed Section 3.3
potentially offer substantial benefits. A notable example is the Optimal Transport (OT) path, which has recently been demonstrated to be particularly effective in the field of molecular generation [28). Second, current training paradigm frequently leads to hallucinated distribution peaks [. Most generative models are trained through maximizing the likelihood or lower bound on the ground- truth structures, which are easily haunted by hallucinations due to the lack of negative samples during training. the field of natural language processing or computer vision, Direct Preference Optimization (DPO) [05]30] is proposed to align the model with human preferences, which effectively reduces hallucinations. For 3D structure prediction, such preferences can naturally extended to similarity with the reference structure (e.g. RMSD). However, it remains unclear whether the DPO method is compatible with arbitrary probability paths.
To address the above pitfalls, we introduce FlowDPO, a novel framework that explores flexible selection of Gaussian paths and enhances the quality of generated structures by alignment with the reference distribution. Specifically, we approach the structure prediction task via flow matching models regarding various paths. Given pre-trained flow matching model, we sample multiple structures for each entry in the training set, evaluate these candidates against known ground truths to compute similarity, and construct an automatic preference dataset. Notably, we theoretically derive the unified objective of DPO for arbitrary Gaussian paths, and leverage the preference enhance performance of the original generative model. Intuitively, such a paradigm not only augment data with self-distilled samples, but also endow the model with the ability to distinguish between high-fidelity and hallucinated samples.
In summary, our contributions are threefold:
We explore multiple accessible probability paths for the 3D structure prediction task, and to the best of our knowledge, with arbitrary Gaussian paths by deriving a unified objective.
the first theoretically prove the compatibility of DPO
Based on the theoretical results, we develop novel framework to encourage better alignment of flow matching models with desired reference distribution in 3D structure prediction, which effectively suppresses the probability of hallucinations.
Our approach yields promising results on antibody and crystal structure prediction tasks, showcasing the versatility and efficacy of our FlowDPO.

---

2 Related Work
Structure Prediction for Atomic Systems. 3D Structure prediction, including predicting con- formations from molecular topological graphs [3). determining unit cell structures from crystal compositions [[4], or inferring structures based on protein sequences []] crucial in computational chemistry and material science. Traditionally, these predictions have relied on physics-inspired scoring functions [I or density functional theory (DFT)-basee energy calculations [TO] to define the search space, with subsequent application of search algorithms to identify optimal structures. Recently, deep generative methods, particularly diffusion models [12] 27], have proven to be highly effective in this field. These models have been successfully applied across multiple specific do- mains, including small molecules [4, crystals [14), antibodies (20). complexes [6], and general biomolecules [0. The emergence of flow matching models [19). which generalize diffusion paths to more flexible probability flows, has further enhanced the generative capabilities for geometric graphs (28). The goal of our work is to explore structure prediction from the perspective of flow matching, and align these models towards more accurate predictions.
Aligning Generative Models. In the domain of generative model alignment, recent work has focused on refining models to better meet human preferences. Direct Preference Optimization (DPO), introduced by [25 offers a significant advancement over traditional Reinforcement Learning from Human Feedback (RLHF, (23) methods by directly optimizing a policy based on human preference data. This approach has proven effective aligning large language models (LLMs) with user expectations. Extending this concept. 30 propose Diffusion-DPO, a novel method that adapts DPO for text-to-image diffusion models. By reformulating the preference optimization for diffusion model likelihoods, Diffusion-DPO achieves state-of-the-an performance in generating images that are not only visually appealing but also closely aligned with textual prompts. Recently, (36) introduces ABDPO, DPO-base method tailored for antibody design. Unlike ABDPO, which concentrates on guiding diffusion models to generate antibody candidates with lower energy, our approach emphasizes aligning flow models for precise structure predictions.
3 FlowDPO
3.1 Flow Matching for Geometric Graphs
Flow Matching (FM, [9] is a general paradigm for generative tasks by learning a vector field to connect the pre-defined prior distribution with the targeted data distribution. Let q denote the data distribution, Zg is a data point acquired from Po = q. and x1 is a random sample from the prior distribution p1- A time-dependent flow th is then defined to shift samples from the prior distribution to the dimendependent distribution pt via the vector field ue, that is
1/2 (x)) = I1, (())(x)) dt = vx(()(x)).
The vector field can be further parameterized by a time-dependent model (2 l), leading to the continuous normalizing flows (CNFs, (5). To avoid numerical ODE simulations to train vo, FM simplifies the training target by aligning the model with a pre-defined vector field u, to yield pt. i.e.
LPM = (x)))l t) ue (xx)))1)
(2)
However, as Pt is still unknown, we are still unable to sample x, and apply the above objective. To address this gap. [19] leverages the more accessible conditional vector field (x(2z(z0) and its corresponding probability path (r (au), resulting in the following Conditional Flow Matching (CFM) objective, which is equivalent to LFM in terms of gradients and accessible for sampling:
LCFM = Ex (x.) [|l (x t) - uz [ro)
Different vector fields lead to different probability paths. For the ussd Gaussian distribu- tion defined as
(z =
Hr (x) ( (xo)),
(4)
3

---

the corresponding vector field [19 is calculated as
ux(o)/ (40)) (100) (rr per (2o)): (x00
where ,i oj are derivatives of Mt, Or W.E. t. We consider three lines of Gaussian paths in this paper, namely the Variance Exploding (VE), Variance Preserving (VP) and Optimal Transport (OT) paths, which are listed in T Table I: Parameters of different Gaussian paths. VE, VP and OT represent Variable Exploding. Variable Preserving and Optimal Transport, respectively.
Probability Path
Mean
Standard Deviation Conditional Vector Field
VE path
(e(2)) = To
(x(2)) Ot
u1(224 of2 (Zt Fo)
VP path
(e(za) = a,
(x(00)) = /1 of
(e( = a, T-o (att''s x0)
OT path
(x0) = (1 1):00
(x =
Up (2 = (o+ xo)
Based on these paths, we are capable of designing proper flow models to maintain symmetries for specific structure prediction tasks. In this paper, we mainly focus on the two typical tasks on atomic systems: antibody structure prediction and crystal structure prediction. Note that symmtries are crucial in 3D atomic systems, and we provide more discussions in Appendix B
Example 1: Antibody Structure Prediction. Antibodies are Y-shaped proteins generated by the immune system to identify and bind to specific antigens, with the structure depicted in Figure 2 Researchers mainly center on the variable domains of antibodies, which comprise heavy chain and a light chain. Each chain includes three Complementarity Derrii---DDeeiiinnggg Regions (CDRs) and four framework regions an alternating sequence. The six CDRs are volatile and crucial in defining the binding specificity and affinity, while the framework regions remain conserved. Among them, CDR-H3, which is the third CDR on the heavy chain, is the most diverse region and the primary focus of antibody design. Therefore, it is fundamental yet challenging problem to accurately predict the structure of the CDRs upon binding.
Antigen
FR1
H1
FR2
H2
FR31
H3
FR4
Heavy
FR1
L1
FR2
L2
FR3
FR4
Light Chain
Figure 2: Graphical depiction of antibody variable domains, which consist of heavy chain and light chain. Each chain is equipped with 4 Framework Regions (FRs) and 3 aampllemennartt--eeeemiin Regions (CDRs). The CDRs, especially CDR-H3, are volatile and thus the key focus.
Task Defimition: Let A = {aa, a2 aN} } denote the sequence of the targeted CDR region with the length of N, where a, {0, 1 is the one-hot type of the amino acid, and X { Ø2, NN the corresponding 3D structures with x; E [g3x4 as the backbone coordinates including N, C.. C, and O. Similarly, the sequence and structure of the context (i.e. framework regions and the antigen) defined as A Xc. The goal is to predict the structure of the CDR region given the context:
X ~ Po A AC)
Probability Paths and Training Objectives: DiffAb 20 has designed the VP path for the coordinates of the CDR region as
1 af of (ar (a,X, X. - Xo):
U. VP X No. A, X A9)
where a, is scheduled as = e 1o ((s)ds After sampling X = ē ~ N/ I), we have
X, a, /I aff. With proper reparameterizzation, the training objective is defined as
Lvp = E., 1 X, A, x A -
(8)

---

which only requires model & to predict the denoising term given the current state.
Moreover, it is also practicable to linearly connect the data point Xo and the noisy prior é via the OT path as X, (1 ()Xo LE. The vector field is then defined as
d Or X Xo. A, C A - (X, Xo) = Xo-
-
(9)
The training objective directly align the model with the simple vector field:
Lor E.,e El X A A) (E x1
(10)
Example 2: Crystal Structure Prediction. Crystal Structure Prediction (CSP), fundamental aspect of material science, requires to predict the stable 3D structure of a compound solely from its composition. Unlike molecules or proteins, which have finite number of atoms, the uniqueness of crystals lies their periodic repetition in infinite 3D space. The infinite crystal structure typically simplified by its repeating unit, which is called unit cell. The key point of CSP is the representation and generation of the unit cell.
Task Desinition: A unit cell is usually characterized by triplet M = (A, F), where A [aj, 02 .... aNl E AxxN represents the one-hot encoded atom types, L = [1 12, [3] E 33x3 denotes the lattice matrix with three basis vectors describing the crystal's periodicity, and F [x 322 .... NN] E R contains the fractional coordinates of the atoms, specifying their positions relative the lattice matrix. The goal CSP is to predict the lattice matrix and the atomic coordinates
0.11)
based on the given crystal composition
L F) ~ F|A).
(11)
Probability Paths and Training Objectives: As the lattice design
matrix L also lies in the Euclidean space, similar VP and OT paths as Eq. 10). Given eL ~
(0, I). with L, at to / af E., the loss function of the VP path is defined
L = Er,ez. O L, F, A) ez 12
(12)
Besides, with L, = (1-t) Lo tCL, the training objective of the OT path is similarly defined as
12
Figure 3: A crystal is the infinite peri- odic arrangement of atoms, and the re- (13) peating unit named as a unit cell.
L.. Er,er. (L, F,, A) (EL Lo)1l
The fractional coordinates lie the torus space of R (o.1)) inherently reflect the periodicity of the crystal. Previous works [4] project the VE path this manifold, and the Gaussian distribution changed into the Wrapped Normal (WN) distribution as (F, Fo) No.. F;; Fo, oII.
pr Fo) = No (F;
(14)
where (1; = Di=-pp (x + i; :.). An accessible way to learn this path is to match the score, iee. the negative logarithmic gradient, of Pt. and the loss function is defined
LF.VE = F, (eelEEe F,, VF. log (F)(F))
(15)
1 (o. where A =- Na
is the pre-computed weight. If o2 in Eq. (14) is sufficiently large. Pi would finally approach the uniform distribution, which can be selected as the prior dis- tribution. Apart from the VE path, it is also applicable to directly connect the data point and the prior sample via the shortest path on the manifold. Specfically, given Fo ~ Po: F. ~ Pt, where Pr is defined as the uniform distribution, the shortest path s F. Fi) can be determined by the logarithmic map from Fo to F1 s (Fo, Fj) = log,, the destination of s(Fo. F1) via the exponential map from Fg as expr, ((F,, Fi) = F(Fo + sFo, Fi)). To eliminate the effect of the overall translation introduced
wr( F; F. 0.5) - 0.5. Alternatively,
Fi can also be considered
by the prior, further normalize F F expF, 8(Fo, Fi) expr, (s F1) (.F.FF)))
5

---

where 3 averages the paths of all atoms. With the path of F; defined as F. = expr (tš(Fo. Fi)). the training objective for the OT path is
L Ei,Fs [L Fr, A) 8(Fo, Fi
(16)
Generalized Notations. Overall, the structure prediction tasks aims generating the targeted structure I given some condition c, i.e. to learn Po(xle). And the flow matching objective min- imizes the Mean Square Error (MSE) of the predicted and pre-defined vector fields with proper reparameterization or simplification, which can be generalized as
Et rup MSE (20, x1; 0)]
(17)
Hereinafter, we use these generalized notations for simplicity.
3.2 Preference Dataset Construction
Building on the flow paths introduced in $ 3.1 we now delve into the details of constructing preference dataset, which is pivotal for the application of DPO, as detailed in
Algorithm 1 Candidate Generation
Algorithm 2 Preference Pairs Construction
Input: N, M, Dres ((x,, Oxd.
1: Input: Dref. DD Dgens Dposs check, N, K, d(.,.), r
d(.,.),
2: Output: Dgen. Dpos. check
2: Output: D pair
3: Initialize: D Dgen. Dpos check +- 0.0.0
Initialize: DD Ppair <-
4: for
N do
4: for i to N do
5: "Den pooe + 0.1
D Dpair [1] +
6: match f- False, Jpos
6: for k 1 K do
7:
for = M do
7:
if k S <K or check]i] = False then
8:
Generate T.y ~ p(rlee;0d Poenli.j] fij
8:
Fik t I, Tak D
9:
9:
gen,
10:
if d((21, (b:s) S then
10:
else
11:
Dpos (i, [pos] + Kij
11:
2 -pooe I ~ D gen,d Swap if d( d(Ii, xxh) > d(zi, x/he) 13: end if
12:
match < True, Jpos +- Joos
12:
13:
end if
end for
14: D Dpair [i, +- (x 2 (2 2yy ci)
15:
check [i] + malch
15: end for
end for
16:
end for
Candidate Generation As shown in Algorithm 1 the construction of the preference dataset begins with the generation of multiple candidate structures for each sample in our reference dataset, Dres- Leveraging the pre-trained flow-based generative model 0 generate M candidate structures (ftz j=1 for each sample (æa,e, c;) via p(æ]cc c;; Qetr), ensuring that each generated structure is contextually relevant and adheres to the geometric constraints discussed previously.
As each candidate is generated, we compute the distance between ßij and the original structure a;; using a predefined metric d(..-). this distance is less than or equal to threshold 8, the candidate is considered close match and is added to Dpos+ subset of promising candidates. This step is crucial for efficiently filtering the generated data to retain only the most relevant candidates for DPO.
Preference Pairs Construction Subsequently, we construct K preference pairs (x% z(h) for each sample i by Algorithm 2 where zitk is preferred over 2/A-- This preference is determined based on their proximity the onginal structure T; Apart from sampling pairs from generated structures. we also use a ratio r to select the ground truth as the preferred sample. Moreover, if all generated structures for sample are far from the original, other pairs are formed by selecting ii from the promising subset Dpos and n/f from the broader set D Dgen- This process ensures that the pairs reflect a clear preference based on the closeness the original structure, facilitating effeettve training through DPO, which is explored in the next section.
original structure æ; always preferred. The

---

3.3 Direct Preference Optimization
To align large language models with human preference. DPO 25 is proposed to replace the RLHF [23] training objective with directly maximizing the likelihood of the preference. [0] extends DPO to text-to-image generation task, adapting the DPO target to diffusion models. Given preference pair ,x),, DPO [25] designs the training objective as
Popa(u")) Pres(
Popt (1e(( Pref(xt)
DPO -Egw log or log
B log
(18)
where Popt: Prer are probabilities yielded by the fine-tuned model Oope and the pre-trained flow model, and [ is hyperparameter to control the KL divergence of these two distributions.
nontrivial to efficiently acquire p(x) via iterative generative models. Inspired by 30
uniformly discretize the time interval into T steps, where step is located at = i/T.. By formulating the probability from the path To:T: Eq. (8 can be rewritten as
[log Poptl (20 Popt Pret (2
Pope (1p(44b:1)
DPO Rag we log (
log Prer (
(19)
To avoid costly sampling through the entire path, Jensen's inequality as
30 applied bound Eq. (19)
Pops (x (xx")) Popt (o) (x Peol (x.
Poo S -Ryy , log BB] log
log
(20)
Pref (x 20 (.:)7)
where B BT servers hyperparameter. As directly sampling 1;-1, Ty from arbitrary intermediate step is still unfeasible, we can estimate them via the accessible Gaussian paths p in Tas
Popl 1x) log Prer Prer 1/ (x,
Poptl (app (22))
LDo R at log a BB BBppar (op)) (st, [log
Piot (rot (2) (x1)
(21)
-E s log a (x," ip, prott)) - T [xx i Popt) - T (alip, Pret) + J J(ai Popt)]
(22)
where [(x:"; p.. po) denotes DEL (p(x1" (o%% lo lx:")) and the same for (z); P.Po). As and po are Gaussian distributions with the same noise scheduler, the KI. divergence can formulated
as
I [(E.P.p))
[u(0--1/z no (o(
(23)
20
According to DDIM 27 if time-dependent Gaussian path follows the form v; N N(n; ky I). we can further design p(z;-1 (2,0) ((1 12..), Given ob the mean can be formulated as
hi of o? 11: + (k:-1 M(Z0-1011)) σ,
T₁ - Vo o? ro
(24)
Fortunately, all paths defined in Table follows this form. And (1;--11/05)) can be parameterized
similarly Eq. (24), with estimating 2g via predicted vector field or denoising terms. Hence,
approximate J [xxpp. Pe) by MSE, 11; 0). With sufficiently large T, Eq. (22) can be changed
into an applicable form follows, which our final training objective.
COPO Exg, . log or [MSE 2. : Brer) (SE 2x:; 0ope)
MSE, at: + MSE, (1/ 0fx)))))))
(25)
7

---

Table 2: Ca and bb indicates RMSD calculated on C. atoms and backbone atoms, repectively. C.-w and bb-w averages the RMSDs of the worst generated conformations of each complex.
L
1.2
1,3
Model
C.-w
C..
bb-w
bb
C.-w
C,
bb-w
bb
C.-w
C.
bb-w
bb
VP Path [20]
2.71
2.00
2.56
2.06
1.11
0.95
1.08
0.96
1.32
0.99
1.39
1.08
Path
2.25
1.77
2.24
1.83
1.13
0.96
1.10
0.96
1.49
1.45
1.13
Path + DPO
2.47
1.91
2,31
1.95
1.09
0.94
1.07
0.94
1.22
0.94
1.30
1.01
Path + DPO
2.22
1.74
2.19
1.78
1.09
0.93
1.05
0.93
1.28
0.95
1.34
1.05
HI
H2
H3
Model
C.-w
C.
bb-w
bb
C.-W
Ca
bb-w
bb
C.-w
C,
bb-w
bb
VP Path [20
1.18
0.83
1.14
0.89
1.41
0,92
1.45
1.00
5.01
3.77
4.95
3.78
Path
1.31
0.89
1.26
0.94
1.69
1.06
1.60
1.13
4.81
3.66
3.70
VP Path + DPO
1.13
0.80
1.13
0.86
1.35
0.87
1.37
0.95
4.42
3.44
4.38
3.45
OT Path + DPO
1.23
0.83
1.19
0.89
1.46
0.95
1.41
1.02
4.28
3.32
4.23
3.32
4
Experiments
We validate our method on two distinct domains: antibody structure prediction ($ 4.1 and crystal
structure prediction ($ 4.2
4.1 Antibody Structure Prediction
Dataset Following previous literature 20) we extract antibody structures from the SAbDab database [8] for training and utilize the manually curated test set from DiffAb [20], which con- tains 19 antibody-antigen complexes. We first derive all structures deposited before April 11th, 2024, and remove those with resolution above 4.0A non-protein targets, resulting in 12,428 antibodies. Subsequently, we use mmseqs2 [29] to cluster the antibodies based on 50% sequence identity for each CDR, and exclude those in the same clusters the test antibodies. The dataset then split into training and validation sets at 9:1 ratio based on the clusters.
Metries We employ the following metrics for evaluation. RMSD measures the Root Mean Square Deviation of the generated alpha carbon coordinates with respect to the reference. RMSD is the RMSD calculated the four backbone atoms including C, Ca. N, O. To better profile the generated distribution, for each antibody, we generate 20 structures and use two strategies to aggregate the results across different antibodies. Strategy worst select the worst generated structure per antibody according to RMSD and then averagse across different antibodies, while strategy mean averages the RMSD of 20 candidates first, and then across antibodies. Strategy worst measures the furthest deviation of the generated distribution compared to the reference, while strategy mean is commonly adopted in previous works (20) [6]. Results aggregated with worst are denoted as C.-w and bb-w, while those with mean are denoted as C, and bb.
Results We evaluate VP path (DiffAb) 20 and OT path 19 with the proposed FlowDPO CDR structure prediction. Results Table 2 illustrate that either using VP path or OT path, further training with DPO consistently enhances performance across different CDRs. Notably, on the most challenging part (i.e. CDR-H3), the DPO phase yields the most significant improvement. Metrics aggregated with strategy worst demonstrate noticeable gains, indicating effective supperssion of low-quality samples by the DPO phase, which we attribute to the objective of DPO in distinguishing the prefered samples. Such characteristies are favorable practical applications where it requires blind selection of generated structures without prior knowledge of which structures might be more correct. We also depict the distributions of RMSD and examples of generated CDR-H3 structures in Figure 4 It shows that the blue curves, yielded by the original flow models, often exhibit a bimodal distribution. While the first peak lower RMSD indicating higher quality generations, the second peak at higher RMSD suggests the models experience hallucination, confidently generating conformations that significantly deviate from the ground truth. DPO effectively suppresses this erroneous second peak, leading to an overall improvement in the quality of generated samples. On closer inspection, this correction also addresses physical invalidities, such as the twisted backbone seen in Figure 4
8

---

Reference (PDB:
VP path
path
RM 2.09 BMSD 2.33A
RMSO 1.69A
RM5D - 1.67A
THÓDICA)
Reforence (PDB: 7D66)
or path
OT path + DPC
0.7
of
RMSD, 1.30A RMSD, 1.83A
RMSD, 1.14A RMSD,
AMSSACA)
Figure 4: Examples of generated CDR-H3 structures and the distribution of RMSD for different antigen-antibody complexes and different probability paths. The visualized samples are the ones with the lowest RMSD of all the generated counterparts for the corresponding complexes. In addition to driving the distribution towards lower RMSD, it is also observed that the DPO phase tends to rectify the physical invalidity (e.g. twisted backbone in the above examples) in the generated samples.
4.2 Crystal Structure Prediction
Dataset We conduct the crystal structure prediction task on three datasets in line with previous works 33 14). Perov-5 [4] includes 18.928 perovskite crystals, each characterized by similar structures but varying compositions, and exactly 5 atoms per unit cell. MP-20 [13] comprises 45,231 materials from the Materials Project, featuring a wide range of compositions and structures, with each material containing no more than 20 atoms per unit cell. These materials predominantly represent crystals that have been synthesized experimentally. MPTS-52 is an advanced version of MP-20, containing 40.476 structures with unit cells that include up to 52 atoms, presenting a more complex challenge. For Perov-5 and MP-20, we maintain the conventional 60-20-20 split for training, validation, and testing. For the MPTS-52 dataset, we use a chronological split, assigning 27,380 crystals for training, 5,000 for validation, and 8,096 for testing.
Metries For inference, we generate one structure given each composition. The predicted sample is then matched with the ground truth via the StructureMatcher class in pymatgen (2] with thresholds stol=0.5, [ogle_to==10 Itol=0.3 as applied in previous works [33] 14]. We use Match Rate (MR) as the proportion of matched structures among the testing set, and the RMSD averaged over the matched pairs, and normalized by VV/N where is the volume of the unit cell.
Results We compare the results with two generative baselines P-eG-SchNet (9] and CDVAE 33
The results are shown in Table where we explore three combinations of paths for jointly generating the lattice and the fractional coordinates: VP+VE, OT+OT, and OT+VE. Notably, the VP+VE path previously developed by DiffCSP 14. We find that the OT path is more effective for lattice generation, while the VE path provides more accurate predictions of atomic coordinates within the cell. Overall, the OT+VE combination generally delivers the best performance. Furthermore, DPO consistently enhances the performance of the model trained on each combination, demonstrating its capability to refine the predictions to a more precise alignment with experimental structures. We additionally visualize the RMSD distribution of predicted structures from different Gaussian paths. Results in Figure[3]evve a similar pattern Figure demonstrating that DPO reduces the probability of low-quality generations.
Conclusion
In this work, we propose FlowDPO, novel framework for 3D structure prediction that integrates flow- based generative models with Direct Preference Optimization. We achieve 3D structure prediction via flow matching models with various probability paths, and generalize the DPO training objective to arbitrary Gaussian paths. refine the model via DPO, we generate multiple candidate structures and construct the preference datasct by aligning with ground truth. The results demonstrate substantial

---

Table 3: Results on crystal structure prediction task. MR stands for Match Rate.
Perov-5
MP-20
MPTS-52
MR (%)
RMSE
MR (%)
RMSE
MR (%)
RMSE
P-eG-SchNel 9
48.22
0,4179
15.39
0.3762
3.67
0,4115
CDVAE
R3
45.31
0.1138
33.90
0.1045
5.34
0.2106
+ + VE Path 14
52.02
0.0760
51.49
0.0631
12.19
0.1786
OT + OT Path
53.95
0.1508
57.40
0.1185
17.40
0.2405
OT + VE Path
52.29
0.0782
58.94
0.0621
18.91
0.1435
VP + VE DPO
53.47
0.0762
59.98
0.0622
14.75
0.1780
OT + OT DPO
55.56
0.1376
59.62
0.0898
22.36
0.1678
OT + VE DPO
53.94
0.0765
62.47
0.0606
20.27
0.1419
Typical Example,
Li4Ge2F12
Example, RMSD-0.422
AIS51M63
Typical Example, RMS0=0 4998
Figure 5: Visualizations on crystal structure prediction results. The left column depicts the RMSD distribution of the models before (blue) and after (red) DPO. The middle column shows the ground truth structures, and the right column shows typical high RMSD generations to be suppressed.
improvements in prediction accuracy for both antibody and crystal structures, highlighting the effectiveness and versatility of FlowDPO the field of 3D structure prediction.
Acknowledgments
This work is jointly supported by the National Science and Technology Major Project under Grant A0107300 the National Natural Science Foundation of China (No. 61925601, No. 62376276. No. 62236011) 1), and Beijing Nova Program (20230484278).
References
[1] J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, pages 1-3, 2024.
[2] Batatia, P. Kovaes, G. Simm, C. Ortner, and Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural
10

---

Information Processing Systems, 35:11423: 12446213116,,,,0 2022.
[3] Batzner, A. Musaelian, L. Sun, M. Geiger, J. Mailoa, M. Kombluth, N. Molinari, T. Smidt, and B. Kozinsky. E (3)-eqvvvari graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
[4] I. E. Castelli, D. D. Landis, K. S. Thygesen, S. Dahl, I. Chorkendorff, T. F. Jaramillo, and K. W. Jacobsen. New cubic perovskites for one-and two-photon water splitting using the computational materials repository. Energy & Environmental Science, 5(10):9034 2012.
[5] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[6] G. Corso, B. Jing, R. Barzilay, T. Jaakkola, et al. Diffdock: Diffusion steps, twists, and turns for molecular docking. In International Conference Learning Representations (ICLR 2023), 2023.
[7] G. R. Desiraju. Cryptic crystallography, Nature materials, 1(2):77-79, 2002.
[8] J. Dunbar, K. Krawezyk, J. Leem, T. Baker, A. Fuchs, G. Georges, J. Shi, and C. M. Deane. Sabdab: the structural antibody database. Nucleic acids research, +2(DI):D1100DDD
2014.
[9] N. W. Gebauer, M. Gastegger, S. S. Hessmann, K.-R. Müller, and K. T. Schütt. Inverse design of 3d molecular structures with conditional generative neural networks. Nature communications, 13(1):1-11, 2022.
[10] J. Hafner. Ab-initio simulations of materials using vasp: Density-functional theory and beyond. Journal of computational chemistry, 293))204044-2 2078, 2008.
[11] T. A. Halgren, R. B. Murphy, R. A. Friesner, H. S. Beard, L. L. Frye, W. T. Pollard, and J. L. Banks. Glide: new approach for rapid, accurate docking and scoring. 2. enrichment factors in database screening. Journal of medicinal chemistry, 47(7):1750-11
2004.
[12] Ho, A. Jain, and Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[13] A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, Dacek, Cholia, D. Gunter, D. Skinner, G. Ceder, et al. Commentary: The materials project: A materials genome approach accelerating materials innovation. APL materials, 1(1):01 1002, 2013.
[14] R. Jiao, W. Huang, P. Lin, J. Han, P. Chen, Y. Lu, and Y. Liu. Crystal structure prediction by joint equivariant diffusion. Advances in Neural Information Processing Systems, 36, 2024.
[15] B. Jing, G. Corso, J. Chang, R. Barzilay, and T. Jaakkola. Torsional diffusion for molecular conformer generation. Advances in Neural Information Processing Systems, 35:24240-2422 24253,, 2022.
[16] X. Kong, W. Huang, and Liu. Conditional antibody design as 3d equivariant graph translation. In The Eleventh International Conference Learning Representations, 2022.
[17] J. Kreitz, M. J. Friedrich, A. Guru, B. Lash, M. Saito, R. K. Macrae, and F. Zhang. Programmable protein delivery with bacterial contractile injection system. Nature, 61 616(7956):357-344 2023.
[18] Y.-L. Liao and T. Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference on Learning Representations, 2022.
[19] Y. Lipman, R. T. Chen, H. Ben- Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2022.
[20] S. Luo, Y. Su, X. Peng, S. Wang, J. Peng, and J. Ma. Antigen-specific antibody design and optimization with diffusion-based generative models for protein structures. Advances Neural Information Processing Systems, 35:9754-9767. 2022.

---

[21] G. M. Morris, R. Huey, W. Lindstrom, M. F. Sanner, R. K. Belew, D. S. Goodsell, and A. J. Olson. Autodock4 and autodocktools4; Automated docking with selective receptor flexibility. Journal of computational chemistry, 30(16):2785-2791, 2009.
[22] S. P. Ong, W. D. Richards, A. Jain, G. Hautier, M. Kocher, S. Cholia, D. Gunter, V. L. Chevrier, K. A. Persson, and G. Ceder. Python materials genomics (pymatgen): A robust, open-source python library for materials analysis. Computational Materials Science, 68:314-315 2013.
[23] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[24] C. Pickard and Needs. Ab initio random structure searching. Journal of Physics: Condensed Matter, 23(5):053201, 2011.
[25] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
[26] M. Schreiner, O. Winther, and S. Olsson. Implicit transfer operator learning: Multiple time- resolution surrogates for molecular dynamics. arXiv preprint S.kiv:2335..804 2023.
[27] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020.
[28] Y. Song, J. Gong, M. Xu, Z. Cao, Lan, Ermon, H. Zhou, and W.- Y. Ma. Equivariant flow matching with hybrid probability transport for 3d molecule generation. Advances in Neural Information Processing Systems, 36, 2024.
[29] M. Steinegger and Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(11):1026-008, 2017.
[30] B. Wallace, M. Dang. R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik. Diffusion model alignment using direct preference optimization. arXiv preprint arXiv:2311.12900 2023.
[31] Y. Wang, J. Lv, I.. Zhu, and Y. Ma. Crystal structure prediction via particle-swarm optimization. Physical Review B, 82(9):0941 16, 2010.
[32] B. D. Weitzner, J. R. Jeliazkov, S. Lyskov, N. Marze, D. Kuroda, R. Frick, J. Adolf Bryfole, N. Biswas, R. Dunbrack, and J. Gray. Modeling and docking of antibody structures with rosetta. Nature protocols, 12(2):401-416, 2017.
[33] T. Xie, X. Fu, O.-E. Ganea, R. Barzilay, and T. S. Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. In International Conference on Learning Repre- sentations. 2021.
[34] M. Xu, L. Yu, Y. Song, C. Shi, S. Ermon, and J. Tang. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations, 2021.
[35] C. Zeni, R. Pinsler, D. Zügner, A. Fowler, M. Horton, X. Fu, S. Shysheya, J. Crabbé, Sun, J. Smith, et al. Mattergen: generative model for inorganic materials design. arXiv preprint WxXi:2312.00668 2023.
[36] X. Zhou, Xue, R. Chen, Z. Zheng, L. Wang, and Q. Gu. Antigen-specifie antibody design via direct energy-based preference optimization. arXiv preprint arXiv:2403.16576, 2024.
12

---

A From RLHF to DPO
Given the preference pair (x"", x²) with condition c, the Bradley-Terry (BT) model considers latent reward model r(r)c) behind them and formulates the preference as
cxp(r(="(c))
p("" 2'|c)- exp(r(a") (c)) exp(r(x")e))
(26)
optimizes the generative model by explicitly training reward model rg. and maximizing the reward with a KL. regularization term to control the model by the original reference Peet
RLHF 23
max E Popta))))) (an) [x)(2)] PPoo (1) [(a)
(27)
Poge
We omit the condition e for simplicity. Pre((r)e" (x)/8/ D)/B/, zZ, where Z is the normalization term, we can rewrite the optimal reward model as
As Eq. 27 exists a close-form solution P5(x)
r" (x) log Popt (I) + 8z. Pre
(28)
After introducing Eq. (28) into Eq. simplifies the training objective as
26 and directly maximizing the log likelihood, DPO
25
(ope(x"0)
B log Pope(al)) Prer(x!) ))
DPO Rag
log or
( (3log
(29)
Prer(x
B Required Symmetries of Atomic Systems
The design of flow paths constrained by the symmetry requirements of specific atomic systems, which are detailed as follows.
Antibody Structure Prediction
The designed vector field should maintain equivariance any rotation Q E (0(3 and be invariant any translation i E R3:
+ Ax +t, (X, A, AC).
(30)
consider the task defined in Eq. [1 as joint generation task on L and F. For the generative process, the vector field of the lattice should be cquivariant to an arbitrary rotation Q SO(3), and that of the coordinates required to ensure the periodic translation invariance for any translation vector t. Specfically, we have
Crystal Structure Prediction Previous works
UL Fo, A) Qu( L, Lo, Fo, A). uF (F, + Lo. w/ Fg + t), A) u(L, Lo, Fo. A),
(31)
(32)
where the operation w w(F) = F
E [0, 1)3x returns the coordinates back to the unit cell.
After maintaining the required symmetries, the proposed flow model is capable of generating equiva- lent structures under different E(3) transformations. An example of the OT-OT path for the crystal
shown in Figure
C Implementation Details
C.1 Antibody Structure Prediction
We use the framework of DiffAb 20 to train the flow models. The original denoising network in DiffAb requires orientation matrices as input, yet the OT path of the SO(3) matrices not naive to derive, which is out of our main scope. Therefore, we replace the denoising network with the multi-channel EGNN proposed MEAN [16] to avoid this problem. All experiments can be run on one GeForee RTX 3090 GPU. Detailed hyperparameters for FlowDPO presented in Table 4
13

---

Ground
Generated Samples
Figure 6: Visualizations of multiple generated crystals via OT-OT path on MP-20. As the designed path maintain the symmetries, the model is able generate structures equivalent the ground truth after proper rotations and (periodic) translations.
Table 4: Hyperparameters for the antibody structure prediction task.
Flow
Preference Dataset
DPO
CDR
d
L
Lr
Epoch
M
r
Lr
Epoch
B
L1
128
6
le-4
500
5
1
1.0
0.1
5e-6
50
200
L2
128
6
le-4
400
5
1
1.0
0.1
le-6
20
200
L3
128
6
le-4
500
5
1
1.0
0.1
5e-6
200
VP
128
6
le-4
500
1
1.0
3e-6
10
200
H2
128
6
le-4
500
5
1
1.0
0.1
5e-6
10
200
H3
128
6
le-4
500
5
1
1.0
0.1
5e-6
5
200
128
6
le-4
500
5
5e-6
50
200
L2
128
6
lc-4
500
5
1
1.0
0.05
5e-6
20
200
L3
128
6
le-4
900
5
1
1.0
0.1
5e-6
200
OT
128
6
le-4
900
5
1
1.0
0.0
5e-6
200
H2
128
6
le-4
700
5
1
1.0
0.1
5e-6
50
200
H3
128
6
1c-4
500
5
1
1.0
0.1
5c-6
5
200
C.2 Crystal Structure Prediction
We the denoising network designed in DifCCSP 14 the backbone model train the flow models. To predict the vector field via the denoising output, for the OT path designed on lattice, we apply the reparameterization
0,t=1, EL EL, F,, L. O 1-t
UL, Fc, A)
(33)
And for the OT path the fractional coordinates, directly VF.0 Lt, F,, A)
eF (L,, Fr, A). We select RMSD defined by StructureMatcher class in pymatgen [22] with thresh- olds stol=0.5, angle_tol= tol=10, Itol=0.3 the distance metric to construct the preference dataset. Specially, the RMSD of the unmatched structure is set as +oo, and such candidates will never be selected as the preferred sample.
The detailed hyperparameters for the FlowDPO pipeline on each crystal dataset are provided in
Table Each experiment is run GeForee RTX 3090 GPU.
D Comparison with Regressive Methods
To further investigate the advantages of the generative paradigm, employ the same backbone model (MEAN) for direct regression task as an additional bascline. The results are presented in Table 6, Our findings indicate that generative models outperform the regression model in 4 of the CDRs, particularly in the highly variable and functionally critical regions, CDR-H3 and CDR-L3.
Each time the pair randomly sampled from the 5 candidates plus the ground truth.
14

---

Table 5: Hyperparameters for the crystal structure prediction task.
Flow
Preference Dataset
DPO
d
Lr
Epoch
M
K
Lr
Epoch
VP+VE
Perov-5
256
le-3
3000
5
12
0.3
1/6
le-3
100
2000
MP-20
512
le-3
1000
5
12
0.3
1/6
le-3
300
2000
MPTS-52
512
le-3
1000
5
12
0.3
1/6
le-3
200
2000
OT+OT
Perov-5
256
le-3
3000
5
12
0.3
1/6
lc-4
70
2000
MP-20
512
1e-3
3000
5
12
0.3
1/6
le-4
30
2000
MPTS-52
512
le-3
2000
5
12
0.3
le-4
100
2000
OT+VE
Perov-5
256
1e-3
3000
5
12
0.3
1/6
5e-5
2000
MP-20
512
le-3
3000
5
12
0.3
1/6
le-3
300
2000
MPTS-52
512
1e-3
2000
5
12
0.3
1e-3
200
2000
Additionally, we report both the mean and minimum RMSD values across 20 generations for each generative model. The significantly lower minimum RMSD values demonstrate that generative models not only yield predictions closer to the observed reference structures but also possess the capability to generate multiple feasible conformations around the stable state.
Table 6: Results compare to regressive baselines antibody structure prediction tasks.
Regression
FlowDPO (VP, mean)
FPo (OT, meun)
FlowDPO (VP. min)
FlowDPO (OT, min)
RMSDp4
RMSDsa
RMSD
RMSDμa
RMSD
RMSDss
RMSD
RMSDss
RMSD
RMSDsa
LI
1.03
1.05
1.91
1.95
1.74
1.78
1.44
1.60
1.22
1.33
1.2
0.96
0.92
0.94
0.94
0.93
0.93
0.80
0.85
0.83
0.87
1.17
1.18
0.94
0.95
1.05
0.69
0.82
0.69
0.84
H1
1.68
1.67
0.80
0.86
0.83
0.89
0.53
0.67
0.52
0.67
H2
0.72
0.78
0.65
0.57
0.71
H3
3,46
3.48
3.44
3.45
3.32
3.32
2.60
2.64
2,55
2.61
E
Limitations
As detailed in $ $3.3] the derivation of the rationality of DPO for flow models primarily focuses on Gaussian paths. However, flow models have the potential to learn mappings from an arbitrary prior to the data distribution if the probability path is appropriately defined. Therefore, more general derivation that does not rely on Gaussian assumptions could be explored in future research. Additionally, our current evaluation based predominantly on computational metries. Conducting wet-lab experiments would provide a more robust validation of the model's effectiveness in practical applications.
F Broader Impacts
The introduction of FlowDPO marks pivotal advancement in scientific domains such as drug devel- opment, materials research, and molecular informatics. Recent developments, such as AlphaFold3, have demonstrated remarkable accuracy in predicting structures across various domains (1] However, issues such as hallucinations, like erroneous structural order inherently disordered regions, remain challenge. It is intriguing to explore whether alignment strategies based on DPO mitigate these hallucinations and enhance overall prediction accuracy.
G Codes
Our codes available htttpp///gthub.ccc///jat/////ppp
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: abstract and introduction, we summarize our contribution as enabling DPO for flow-based structure prediction models, constructing preference dataset to align model predictions with reference, and achieving promising results for antibody and crystal predic- sections.
tion tasks. These claims are detailed and verified by the method ($3) and experiment ($4)
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
The limitations of this paper are discussed in Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The derivation are provided in
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The used datasets and evaluation setups are provided in $ and we provide
more details in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
17

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Our codes provided in Appendix G
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. ec/
sublic/guidess heaii//ggiiies/oeeeubbmiisse
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips cc/ wttppppllllllppie http:///oooooooossssooooooooeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The hyperparameters provided in Appendix C
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The dataset used in the experiments are large and the results are relatively stable. Rerunning the pipeline for multiple times is costly. Instead, we further compare the prediction results for different models from the perspective of distribution in Figure 4
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources are provided in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: This paper definitely follows the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The broader impacts of this paper are discussed in Appendix ??
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
19

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets used in this paper have been properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcone com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
20

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "BolxMuJtM7", "reviewer": "Reviewer_SYsR", "summary": "This paper introduces FlowDPO, a framework that predicts 3D structures of atomic systems using diffusion flow matching models. To suppress hallucinations and improve sample quality, Direct Preference Optimization (DPO) is adopted to finetune a pretrained model using a preference dataset consisting of winning and losing pairs, which are selected based on their distance to the ground truth. Details on the flow matching model and DPO training paradigm has been discussed. Experiments have been conducted on antibody and crystal structure prediction tasks, demonstrating better accuracy than the two baselines (P-cG-SchNet and CDVAE).", "questions": "As mentioned in the weaknesses, I do not understand the motivation for symmetry preservation and transform invariance, and the claim about the theoretical proof.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- This work introduces the application of diffusion models with DPO into the field of atomic structure predictions. Unlike the original diffusion DPO paper [30], where samples have to be ranked by humans, this task benefits from known ground truths from a training dataset, allowing the winning and losing pairs to be automatically constructed by comparing the distances to the ground truth.\n- Experimental results demonstrate the advantage of the proposed method over other generative models, and ablation studies have revealed the contribution of the DPO fine-tuning.", "weaknesses": "- The motivation for symmetry preservation and transform invariance of the flow trajectory is unclear and seems not quite relevant to the main idea of using DPO to improve sample accuracy. The theoretical justification for the trajectory needing to be transform invariant is not well explained, apart from the empirical results.\n- In L57, the authors claim that they are \"the first to theoretically prove the compatibility of DPO with arbitrary Gaussian paths by deriving a unified objective.\" However, I can hardly find anything relevant to this claim, and the paper seems more focused on application rather than theory.\n- In general, the contribution is limited, considering that diffusion DPO is a known method in [30], which is theoretically universally applicable to all diffusion flow matching models. While this work touches on the new field of atomic system prediction, using a universally applicable method in this field does not seem very novel to me.\n- The writing of this paper has some issues, making the presentation a bit unclear:\n  - The overall structure is not focused. While the main idea of this work is the application of diffusion DPO rather than proposing a new theory, the paper spends a lot of space on preliminaries with very specific examples. The main body, especially the experiments section, is short and lacking in detail.\n  -  The subscript 'i' is a bit confusing as it sometimes denotes the index of a data sample and sometimes the index of a timestep.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive comments! We provide more explanations to address your concerns as follows.\n\n> **W1: The motivation for symmetry preservation and transform invariance of the flow trajectory is unclear and seems not quite relevant to the main idea of using DPO to improve sample accuracy. The theoretical justification for the trajectory needing to be transform invariant is not well explained, apart from the empirical results.**\n\nThanks for your feedback. In many atomic systems, symmetry indeed plays a fundamental role, as the physical laws governing these systems are invariant under certain transformations. More specifically, for antibody structure prediction, the E(3) equivariance of the trajectory in Eq. (7) ensures that the predicted CDRs are equivariant to the given context [20]. Similarly for crystal structure prediction, the periodic E(3) equivariance described in Eq. (13-14) ensures that the predictions retain the inherent symmetries in crystal structures [14]. As we also explore various new flow paths beyond those in previous studies [14,20], it is imperative to ensure that these newly proposed paths still adhere to the necessary symmetries, thereby validating their correctness. We will expand on these points in the revised paper to more clearly articulate the role and necessity of maintaining these symmetries.\n\n> **W2: In L57, the authors claim that they are \"the first to theoretically prove the compatibility of DPO with arbitrary Gaussian paths by deriving a unified objective.\" However, I can hardly find anything relevant to this claim, and the paper seems more focused on application rather than theory.**\n\nThank you for your observation. We apologize for any ambiguity in our presentation. In Section 3.3, we generalize the feasibility of DPO from diffusion-based VP paths to arbitrary Gaussian paths. Specifically, as detailed in Eq. (27), we demonstrate that Gaussian paths can be universally discretized using conditional mean values parameterized by $x_i$ and $x_0$. As $x_i$ is given, the KL divergence term in Eq. (26) can be reparameterized by the MSE of the ground truth $x_0$ and $x_{0,\\theta}$ predicted by the flow model. This formulation allows us to derive a unified DPO training objective applicable to all Gaussian paths, as outlined in Eq. (28). We acknowledge the need for clearer exposition on this theoretical contribution and will enhance the relevant sections in our paper to better articulate these derivations and their implications.\n\n> **W3: In general, the contribution is limited, considering that diffusion DPO is a known method in [30], which is theoretically universally applicable to all diffusion flow matching models. While this work touches on the new field of atomic system prediction, using a universally applicable method in this field does not seem very novel to me.**\n\nThanks for pointing this out. As mentioned above, while it is true that Diffusion-DPO [30] establishes the framework of DPO within the context of diffusion models like DDPM, which predominantly utilize the Variance Preserving (VP) path, our contribution extends this framework to arbitrary Gaussian paths and adapts to a broader class of flow models, and demonstrates the practical applicability on 3D structure prediction tasks.\n\n> **W4: The overall structure is not focused. While the main idea of this work is the application of diffusion DPO rather than proposing a new theory, the paper spends a lot of space on preliminaries with very specific examples. The main body, especially the experiments section, is short and lacking in detail.**\n\nThanks for your feedback. We acknowledge that the current structure of the paper may not optimally highlight the core contributions of our work. Our primary aim is to demonstrate the efficacy of DPO across a range of flow models characterized by arbitrary Gaussian paths, specifically within the context of 3D structure prediction tasks.\n\nTo establish this, we introduced multiple flow paths that extend beyond the typical diffusion-based paths previously explored in literature [14,20]. These paths are crucial for demonstrating the broad applicability of DPO in handling diverse flow models. Additionally, maintaining the necessary symmetries for each specific prediction task is essential for the design of each proposed path, which we underscore in Section 3.1. We derives the generality of DPO in Section 3.3, and demonstrate its effectiveness for multiple tasks and multiple flow paths in the experiments. We will carefully reorganize the content to ensure a more logical flow that aligns with the main contributions of our work.\n\n> **W5: The subscript 'i' is a bit confusing as it sometimes denotes the index of a data sample and sometimes the index of a timestep.**\n\nThanks for your kind advice. We would like to change the notation of timesteps into 's' to avoid confusion.\n\nThank you again for your valuable suggestions. We believe we have addressed your concerns and kindly hope for your feedback and reconsideration of the scores.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "miuqUMtQSo", "reviewer": "Reviewer_WEst", "summary": "The paper introduces a framework called FlowDPO subsuming different Gaussian probability paths for flow matching models predicting the 3D structure of atomic systems. Moreover, the authors propose a method for the automatic creation of a preference pair dataset used for finetuning a pre-trained flow matching model with direct preference optimization (DPO). As examples of the structure prediction task for atomic systems, the paper explores antibody and crystal structure prediction, each with specific symmetry requirements. The choice of probability path is evaluated empirically including a validation of DPO in terms of consistent performance boosts.", "questions": "* Is there uncertainty in two given tasks, i.e., can there be multiple different valid solutions to predicting the 3D structure given the respective prior knowledge as conditioning?\n  - If no, why would you use a generative approach?\n  - If yes:\n    - Why do you construct preferences based on distances to a single ground truth without accounting for the possibility of multiple solutions given the conditioning?\n    - Why does the evaluation consider distance to a single ground truth?\n* What do you understand as hallucinated samples?\n* Are there non-generative (regression) baselines and if so, how is the performance compared to them?", "limitations": "The authors have addressed the limitations and broader impacts without any potential negative societal impacts in the appendix.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper is very well structured:\n  - The presented formal framework abstracts the choice of different probability paths for flow matching models.\n  - The general task of structure prediction of atomic systems is well motivated and explained by the two instances in form of antibody and crystal structure prediction.\n  - The structure of the method section follows the training procedure: Flow Matching training, preference dataset construction, direct preference optimization.\n* The paper is original: The authors apply general flow matching and direct preference optimization to the new domain of atomic system structure prediction.\n* The contributions of DPO for flow matching and automatic preference dataset construction are general and therefore also applicable to other domains.\n* The experimental results validate the proposed framework:\n  - Exploration of different probability paths can be beneficial, as there is not always one performing the best.\n  - Direct preference optimization on the proposed automatically constructed datasets consistently improves performance.", "weaknesses": "* Missing motivation for a generative approach to 3D structure prediction of atomic systems.\n* Unclear/imprecise use of the term \"hallucination\". The paper claims the model can distinguish between high-fidelity and hallucinated samples (cf. 52 f) without defining what hallucinated samples really are. Recombination / generalization is usually desired for generative models.\n* Self-contradictory method:\n  - The framework is about flow matching models, i.e., a generative method.\n  - The preference definition is solely based on distances to the ground truth training examples. This way of creating a preference dataset does not account for the possibility of multiple possible solutions given the conditioning. Optimization w.r.t. the RMSD metric is not even significantly different from the flow matching objective, begging the question for why this improves performance at all?\n  - Originally, DPO is used in combination with crowd-sourced human preferences for human alignment. This type of metric is significantly different from the pixel-wise Gaussian noise and MSE regression during training of text-to-image models, for example.\n* Contradictory method and evaluation:\n  - The paper uses generative flow matching models.\n  - The evaluation is done in terms of distances to a single ground truth, which does not account for the possibility of multiple valid solutions. However, this might be a general problem of generative approaches for regression tasks.\n* The comparison with baselines is limited to generative methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments, and answer the reviewer’s questions as follows.\n\n> **W1: Missing motivation for a generative approach to 3D structure prediction of atomic systems.**\n> **Q1.1: why would you use a generative approach?**\n\n\nThank you for your insightful question. The motivation for using generative model first arises from the scarcity of experimental data, which typically provides only a single near-stable structure. Generative models can learn and represent a distribution of potential stable structures, by maximizing the likelihood of the observed samples within the modeled distribution, filling gaps in the data. Their success in works like DiffAb, DiffCSP, DiffDock, and AlphaFold3 further supports this direction.\n\n\n> **W2: Unclear/imprecise use of the term \"hallucination\".**\n> **Q2: What do you understand as hallucinated samples?**\n\nThank you for your feedback. We recognize that the term \"hallucination\" was used without a clear definition, leading to potential confusion. In the context of our work, \"hallucination\" refers to instances where the generative model produces conformations that are not only incorrect but also highly improbable from a physical view.\n\nAs depicted in Figure 4, the generative model's output distributions are represented by two curves on where the x-axis measures the RMSD from the ground truth and the y-axis indicates the probability. The blue curve, which illustrates the model's performance prior to the application of DPO, often displays a bimodal distribution. The first peak, with lower RMSD, indicates higher quality predictions, while the second peak, with higher RMSD, corresponds to conformations that deviate significantly from the ground truth.\n\nUpon closer examination, we found that this second peak often includes physically implausible conformations. In scenarios without DPO, the probability associated with this erroneous peak can surpass that of the more accurate peak. We have termed this phenomenon as \"hallucination,\" indicating the unexpected high probability of low-quality, implausible generations, which DPO effectively helps to suppress.\n\nIf the term \"hallucination\" leads to ambiguity, we are open to describing it more directly as incorrect or low-quality generations.\n\n> **W3: Self-contradictory method.**\n> **Q1.2: Why do you construct preferences based on distances to a single ground truth without accounting for the possibility of multiple solutions given the conditioning?**\n\n\nThank you for your insightful comments. As mentioned above, high RMSD values often indicate low-quality conformations. Our DPO approach treats low RMSD structures as preferred, guiding the model to reduce low-quality generations. This adapts DPO to a quantitative metric relevant to 3D atomic structures, ensuring objectivity and scalability without relying on costly expert evaluations.\n\n\n> **W4: Contradictory method and evaluation.**\n> **Q1.3: Why does the evaluation consider distance to a single ground truth?**\n\nThank you for your observations. We employ RMSD as an evaluation metric, recognizing its widespread acceptance and proven utility in our domain. Experimentally measured structures, which serve as benchmarks in our evaluations, are generally near a stable state. Consequently, structures generated close to these experimental conformations are typically more reliable and indicative of stability. Moreover, this metric has been widely adopted in the field, including in seminal works such as AlphaFold3, and is well-recognized within our community. \n\n> **W5: The comparison with baselines is limited to generative methods.**\n> **Q3: Are there non-generative (regression) baselines and if so, how is the performance compared to them?**\n\nThanks for your advice. We utilize the same backbone model (MEAN) directly for a regression task as an additional baseline. The results are shown in Table S1 in the general response. We observe that the generative models surpass the regressive model on 4 of the 6 CDRs, notably in the most variable and critical regions, CDR-H3 and CDR-L3. Additionally, we report not only the mean RMSD across 20 generations for each generative model but also the minimum RMSD. It can be seen that the minimum RMSDs are significantly lower, showcasing that the generative models not only provide predictions that are closer to the observed reference structure but also demonstrate the ability to generate multiple viable structures around the stable state.\n\n\n> **Q1: Is there uncertainty in two given tasks, i.e., can there be multiple different valid solutions to predicting the 3D structure given the respective prior knowledge as conditioning?**\n\n\nCertainly, there is inherent uncertainty in predicting the 3D structures of molecules due to the dynamic nature of molecular conformations. These conformations are not static or fixed; instead, there are many possible structures clustering around the energy minima[a]. Therefore, a generative modeling approach is employed to capture the probability distribution of these stable structures, aiming to maximize the likelihood of observing conformations close to these energy minima. Regarding the use of RMSD as a metric, it serves to assess how closely the generated structures align with a known stable conformation. Since the plausible structures only deviate slightly from each other, structures distinct from the observed stable conformations are still less reliable. Therefore, although this approach does not account for the multiplicity of valid solutions, it provides a practical measure of deviation from a recognized ground truth, facilitating the evaluation of model performance in generating physically plausible structures.\n\n[a] Fernández-Quintero, Monica L., et al. \"CDR-H3 loop ensemble in solution–conformational selection upon antibody binding.\" MAbs. Vol. 11. No. 6. Taylor & Francis, 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PRx6CY9BCI", "reviewer": "Reviewer_jaPH", "summary": "The paper proposes to apply the Direct Preference Optimization (DPO) procedure for the 3D Structure Prediction of atomic structure predictions. The approach is applied to crystal and antibody structures, considering different kinds of Gaussian Path for flow matching models, showing a general improvement in the state of the art.", "questions": "Adding to the points above:\n1) How efficient is the proposed approach w.r.t. the competitors [9, 14, 33]?\n2) From my intuition, symmetries in the atomic structures may be a relevant aspect to consider in constructing the reference dataset since it could lead to ambiguities but also provide a source of augmentation. I would like to know what is the authors' thoughts on this aspect.", "limitations": "The paper phrases the limitations mainly as future works but does not discuss the failure modes of the proposed approach. I suggest the authors comment on recurrent confusion cases for the observed generations.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The method seems to consistently outperform previous methods on the two different tasks\n- The implementation of the code approach can foster the application of DPO protocol to other problems in the chemical and molecular domains. \n- Experimentations with different kinds of Gaussian paths seem to be an interesting analysis that could help future works to build on top of them.\n\nI may have missed some important aspects since I am not an expert in this domain.", "weaknesses": "- As a non-expert, I have a hard time understanding some of the merits. In particular, the approach seems mainly to be a straightforward application of DPO to the specific case of atomic systems, and I cannot find a technical contribution specifically designed for the specific case. Given that I do not know how relevant the problems considered are and what their impact could be, I am unsure about the significance of the work. In case I am missing something, I am looking forward to authors' clarification.\n- I feel the general presentation could be improved. Section 3.3 does not offer particular insight and does not blend with the previous applicative part. The paper does not offer many visualizations of the results or quantitative comparisons. These could help the reader to better understand the impact of DPO in this domain. For example, recent works suggest that DPO may be prone to overfitting [A]. I would like to know what the impact of the procedure is in the atomic structure prediction and if they observed similar problems. Similarly, although the authors offer some possible future directions, they don't really discuss failure cases of the method, and the presented limitations seem a bit generic.\n\n[A]: A general theoretical paradigm to understand learning from human preferences, Azar et al., AISTATS 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments, and answer the questions as follows.\n\n> **W1: The significance of the work.**\n\nThank you for your kindly comment. Our paper, while applying DPO to atomic systems, introduces significant innovations tailored for this domain:\n\n- **Extension of Gaussian Paths**: We expand the typical diffusion paths from previous works[14,20] to a broader set of Gaussian paths, and ensure these paths still meet the symmetry requirements essential for accurate structural predictions, enhancing exploration of conformational space.\n\n- **Unified DPO Objective**: Our theoretical analysis ensures the universality of the DPO objective that is applicable across all Gaussian paths, not limited to the traditional diffusion paths [30].\n\n- **Automated Preference Dataset Generation**: We introduced a method to automatically generate preference datasets from the training data, reducing the need for manual annotations and lowering resource demands.\n\n- **Empirical Validation**: Our method has been validated on challenging antibody and crystal datasets, demonstrating its effectiveness and practical utility in real-world 3D structure prediction.\n\nThese contributions address specific challenges in 3D structure prediction for atomic systems, offering substantial advancements over existing methods.\n\n> **W2: The general presentation could be improved.**\n\nThank you for your constructive feedback. \n\nWe acknowledge your concerns regarding the general presentation and the intergration of Section 3.3. The primary intention of our paper is to demonstrate the effectiveness of DPO across various flow models and Gaussian paths for atomic structure prediction tasks. In Section 3.3, we aimed to establish the generality of DPO by ensuring the training objective is universal for arbitrary Gaussian paths, as introduced in Section 3.1. This theoretical foundation is crucial for supporting the feasibility of the experiments detailed in Section 4, bridging the gap between theoretical derivations and practical applications.\n\nMoreover, the impact of DPO has been visualized in Figure 4 in our paper. Specifically, Figure 4 illustrates the distribution of a set of samples generated by our model, where the x-axis represents the RMSD to the ground truth and the y-axis represents the probability density. The blue curve represents the model's performance before DPO optimization, and the red curve represents performance after DPO optimization. Notably, the blue curve often exhibits a bimodal distribution, with the first peak at a lower RMSD indicating higher quality conformations, and the second peak at a higher RMSD representing conformations that deviate more from the ground truth. These higher RMSD samples often include physically implausible conformations. After applying DPO, we observe a suppression of the erroneous peak, indicating an overall improvement in the quality of the generated structures. This visualization clearly demonstrates DPO's role in enhancing model performance by effectively reducing the generation of less accurate structures. We provide additional visualizations in Figure S1 of general response, which also support the above observations.\n\nFor the overfitting problem and the bad cases, one worth-noticed phenomenan is that the learning rate for DPO training affects the model performance. For instance, a high learning rate can cause the model to deviate from the initial state, and the original distribution is forgetten. We provide an example in Table S2 of the general response for the validation performance on MPTS-52 crystal structure prediction with OT-OT path as follows, where high learning rate hinders the model performance.\n\n\n\n> **Q1: How efficient is the proposed approach w.r.t. the competitors?**\n\nIn assessing the efficiency of our proposed approach relative to competitors, we provide the inference time (minutes) for each method when applied to MP-20 in Table S3 of the general response. Note that DPO does not change the inference process of the flow models. Consequently, models optimized with DPO maintain comparable generation efficiency to their original versions.\n\n\n> **Q2: The role of symmetries.**\n\nNice Question! Symmetries indeed play a crucial role in the field of atomic systems. In our study, we have carefully addressed this issue by ensuring that all proposed flow paths rigorously maintain the symmetries specified in Eq. (6) and Eq. (12). As a result, our model inherently respects these symmetries. Specifically, the predicted antibody CDRs exhibit equivariance relative to the provided context, and the modeled crystal distributions are invariant under E(3) transformations. Given this inherent compliance with symmetry requirements, we do not need to employ additional data augmentation strategies to artificially introduce symmetries.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers and ACs for their time and efforts on reviewing the paper. We are glad that the reviewers recognized the contributions of our paper, and appreciate the reviewers for their insightful comments. We provide additional visualizations and experment results in the supplementary PDF file for more details. We summarize the extra contents as follows.

- **Table S1** combines the results with regressive baselines, highlighting the superiority of generative models in predicting flexible CDRs.
- **Table S2** explores the impact of overfitting by comparing the effects of different learning rates.
- **Table S3** evaluates the efficiency of various models.
- **Figure S1** offers additional visualizations of crystals, illustrating the impact of DPO by reducing low-quality generations.

Meta-review: Initially, this paper got borderline scores also due to slightly out-of-domain reviewers that are familiar with diffusion and flow matching but not with the task of atomic system structure prediction, which is tackled here. Initially, reviewers were questioning the use of generative models in the given setting, the significance of contribution, i.e. if the generalized DPO actually has relevant impact on the results of the given task, and the need for the theoretical contributions.

The discussion phase of this paper was very valuable, helped to improve the paper and clarified the intuition and details behind the presented approach. After the discussion phase, the reviewers unanimously converged to accept scores, being convinced by the detailed answers of the authors. Without being an in-domain expert either, I agree with the reviewers. I think the paper is interesting and of high quality, showcasing successful application of diffusion flow matching in 3D structure prediction tasks. I follow with an accept recommendation.
Decision: Accept (poster)