GFlowNet Assisted Biological Sequence Editing
Authors: Pouya M. Ghari
Alex M Tseng
Gökcen Eraslan
Romain Lopez
Tommaso Biancalani
Gabriele Scalia
Ehsan Hajiramezanali
Venue: NeurIPS 2024 poster (2024)
Keywords: GFlowNet
Sequence Editing
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Editing biological sequences has extensive applications in synthetic biology and medicine, such as designing regulatory elements for nucleic-acid therapeutics and treating genetic disorders. The primary objective in biological-sequence editing is to determine the optimal modifications to a sequence which augment certain biological properties while adhering to a minimal number of alterations to ensure predictability and potentially support safety. In this paper, we propose GFNSeqEditor, a novel biological-sequence editing algorithm which builds on the recently proposed area of generative flow networks (GFlowNets). Our proposed GFNSeqEditor identifies elements within a starting seed sequence that may compromise a desired biological property. Then, using a learned stochastic policy, the algorithm makes edits at these identified locations, offering diverse modifications for each sequence to enhance the desired property. The number of edits can be regulated through specific hyperparameters. We conducted extensive experiments on a range of real-world datasets and biological applications, and our results underscore the superior performance of our proposed algorithm compared to existing state-of-the-art sequence editing methods.

GFlowNet Assisted Biological Sequence Editing
Pouya M. Ghari' University California Irvine
Alex M. Tseng Genentech
Gökcen Eraslan
Romain Lopez Genentech, Stanford University
Tommaso Biancalani Genentech
Genentech
Gabriele Scalia
Ehsan Hajiramezanali Genentech
Genentech
Abstract
Editing biological sequences has extensive applications in synthetic biology and medicine, such designing regulatory elements for nucleic-acid therapeutics and treating genetic disorders. The primary objective in biological-sequence editing is determine the optimal modifications to a sequence which augment certain biological properties while adhering to minimal number of alterations to ensure predictability and potentially support safety. In this paper, we propose GFNSeqEd- itor, a novel hiological-sequent proposed area of generative flow networks (GFlowNets). Our proposed GFNSe- qEditor identifies elements within starting seed sequence that may compromise a desired biological property. Then, using a learned stochastic policy, the algorithm makes edits these identified locations, offering diverse modifications for each sequence to enhance the desired property. The number of edits can be regulated through specific hyperparameters. We conducted extensive experiments on range real-world datasets and biological applications, and our results underscore the su- perior performance of our proposed algorithm compared to existing state-of-the sequence editing methods.
editing algorithm which builds on the recently
Introduction
Editing biological sequences has multitude of applications biology, medicine, and biotechnology. For instance, gene editing serves as a tool to elucidate the role of individual gene products in diseases [2 and offers the potential rectify genetic mutations in afflicted tissues and cells for therapeutic interventions [9]. The primary objective in biological-sequee sequencc editing is to enhance specific biological attributes
starting seed sequence, while minimizing the number of edits. This reduction in the number of alterations not only has the potential to improve safety but also facilitates the predictability and precision of modification outcomes.
Existing methodologies that leverage generative modeling in the context of biological sequences have predominantly concentrated on de novo generation of sequences with desired properties [52 61 3
A common feature of these approaches is generating entirely new sequences from scratch. As result, there an inherent risk of deviating significantly from naturally occurring sequences, compromising safety (e.g., the risk of designing sequences that might trigger an immune response) and predictability (e.g., obtaining misleading predictions from models that are trained on genomic sequences due to
"Work has been done while interning at Genentech CCoresponding author: haj hajiramezanali. www.iieeessaneeeeeeeee com.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

out-of out-of-distribution effects). Despite the paramount importance of editing biological sequences, there has been a noticeable scarcity of research leveraging generative modeling to address this aspect specifically.
Generative flow networks (GtlowNets) 5 6]. generative approach recognized for their ability to sequentially generate new objects, have shown remarkable performance in generating novel biological sequences from scratch [99 30J. Drawing inspiration from the emerging field of GFlowNets, this paper introduces a novel biological-sequence editing algorithm: GFNSeqEditor. GFNSeqEditor assesses the potential for significant property enhancement within given seed sequence by iteratively identifying and subsequently editing specific positions in the input sequence, More precisely, using the trained flow function, GFNSeqEditor first identifies positions in the seed sequence that require editing. Then, it constructs a stochastic policy using the flow function to select substitution from the available options for the identified positions. Our stochastic approach empowers GFNSeqEditor to generate a diverse set of edited sequences for each input sequence, which, due to the diverse nature of biological targets, is an important consideration in biological sequence design 34
In summary, this paper makes the following contributions:
We introduce GFNSeqEditor, a novel sequence-editing method which identifies and edits positions within given sequence. GFNSegEditor generates diverse edits for each input sequence based on a stochastic policy.
We theoretically analyze the properties of the sequences edited through EFNter,ditor, deriving lower and upper bounds on the property of edited sequences. Additionally, we demonstrate that the lower and upper bounds for the number of edits performed by GFNSe- qEditor can be controlled through the adjustment of hyperparameters (Subsection 4.3)
We conduct experiments across various DNA and protein sequence editing tasks, showcasing GFNSeqEditor's 's remarkable efficiency in enhancing properties with a reduced number of edits when compared to existing tfte--ff-the-ar art methods. (Subsection 5.1).
We highlight the versatility of GFNSeqEdiior, which can be employed not only for sequence generation models to produce novel sequences
editing but also alongside with improved properties and increased diversity (Subsection) (.2)p.
We demonstrate the usage of GFNSeqEditor for sequence length reduction, allowing the creation of new, relatively shorter sequences by combining pairs of long and short sequences (Subsection 5.3
2
Related Works
De Novo Sequence Design. The generation of biological sequences has been tackled using a diverse range of methods. including reinforcement learning ([]. Bayesian optimization [51), deep generative models for search and sampling
(18), generative adversarial networks [61]. diffusion
models B] model-based optimization approaches [5) adaptive evolutionary strategies [16 49),
likelihood-free inference [55], and surrogate- based black-box optimization [10, and GelowNet
[19] It is important to note that all these sequence- -eveerrttnnntttooonn methods generate sequences from scratch. However, ab initio generation carries the risk of deviating too significantly from naturally occurring sequences, which can compromise safety and predictability. In contrast, our proposed method enhances a target property while maintaining the similarity to seed sequences (e.g., naturally occurring sequences), thus improving predictability and potentially enhancing safety.
Sequence Editing. Traditional approaches commonly employed for biological sequence editing are evolution-based methods, where-over many iterations- a starting "seed" sequence is randomly mutated, retaining only the best sequences (i.c., highest desired property) for the next round 2 46 These approaches have several important limitations. First, they require the evaluation of numerous candidate sequences at every iteration. This computational demand can become prohibitively expensive, particularly for lengthy sequences. Additionally, evolution- heavily rely on evaluations provided by proxy model capable of assessing the properties unseen sequences; the efficacy of these methods is thus limited by the reliability of the underlying proxy. Moreover, these methods may require repeated rounds of interactions with the lab [1I, which can be costly and time-consuming.
50
41
methods

---

Sequence Editor &
A
G
DNA sequence æ with property y
DNA sequence @ with property
An example of editing the DNA sequence 'ATGTCCGC', The goal is to make limited number of edits to maximize the property ÿ. Each token in the sequence in this example called a base and can be any of ['A', 'C', 'T', 'G']. The editor function & accepts the initial sequence as an input and determines that the second and seventh bases require editing (highlighted in red). Then, modifies the bases at these identified locations to improve the property value.
Figure 1:
Beyond evolution- based methods, a handful of aptimization-base methods have been proposed by 42 48 21]. By treating sequence editing as optimization task, Ledidi [42) learns to perturb specific positions within a given sequence. Utilizing Bayesian optimization, LaMBO 48] gener- ates new sequences by optimizing batch of starting seed sequences. Building upon the LaMBO framework, MOGFN-AL 21 leverages GFlowNets to generate candidates in each round of Bayesian optimization loop, improving computational efficiency compared to LaMBO. Akin evolution- based models, these optimization based methods require the evaluation unseen sequences. Consequently, their effectiveness is contingent on the quality of the proxy model, which can compromise their performance the proxy model lacks sufficient generalizability for unseen sequences. Furthermore, both evolution-based and aptimization-basee methods perform local searches given either a single seed sequence or batch of seed sequences. Thus, these methods face issues related to low sam- ple efficiency. In contrast, GFNSeqEditor relies on pre-trained flow function that amortizes the search cost over the learning process, allocating probability mass across the entire space to facilitate exploration and diversity. Furthermore, GFNterEditor can employed for editing without neces- sitating the evaluation of unseen sequence properties. Theoretical analysis presented in this paper establishes that the bounds of edited sequence rewards, property improvement, and the number of edits can be effectively regulated through GFNSeqEditor hyperparameters, Therefore, GFNSeqEditor offers increased reliability and operational suitability in comparison to counterparts lacking a robust theoretical analysis.
We provide an extensive overview of the related literature, with additional discussion available in the Appendix
3 Preliminaries and Problem Statement
Let æ be a biological sequence with property For example, æ may be a DNA sequence, and may be the likelihood it binds a particular protein of interest. The present paper considers the problem of searching for edits in æ to improve y. To this end, the goal is to learn an editor function E-)) which accepts a sequence x and outputs the edited sequence {(x) â with property ÿ. The editor function E() should maximize j. while at the same time minimizing the number of edits between and ø. To achieve this goal, we propose GFNSeqEditor, GFNSeqEditor first identifies positions in given biological sequence such that editing those positions leads to considerable improvement in the property of the sequence. Then, the learned editor function E edits these identified locations (Figure D. GFNSeqEditor uses trained GFlowNet [5] 6 to identify positions that require editing and subsequently generate edits for those positions. The following Subsections present preliminaries on GFlowNets.
3.1 Generative Flow Networks
Generative Flow Networks (GFlowNets) [5] learn a stochastic policy πT-) to sequentially construct discrete object æ. Let X' be the space of discrete objects æ. assumed that the space X' is compositional, meaning that an object can be constructed using a sequence of actions taken from an action set A. At each step f, given partially constructed object st, GFlowNet samples an action a++1 from the set A using the stochastic policy T-))s;). Then, GFlowNet appends a,+1 to to obtain 8 1. this context, 8, can be viewed as the state at step I. The above procedure continues until reaching terminating state, which yields the fully constructed object a. To construct an object ar, the GFlowNet starts from an initial empty state 8p, and applying actions sequentially, all fully constructed objects must end in special final state sy. Therefore, the trajectory of states to construct
3

---

an object can be written as Ta = (so - +... 8y). Let T be the set of all possible trajectories. Furthermore, let R(-) X - R* be non-negative reward function defined X. The goal of GelowNet is to learn stochastic policy sT.) such that x (2x) R(æ:). This means that the GFlowNet learns a stochastic policy x(.) to generate an object a with probability proportional to its reward.
As described later, to obtain the policy æ(-), the GelowNet uses trajectory flow F T R+ The trajectory flow F(T) assigns a probability mass to the trajectory т. Then, the edge flow from state state s' is defined as F(s + s') Exr:s-aeer (T)). Moreover, the state flow defined F(s) EvT:Ser (T)). The trajectory flow F(.) induces probability measure Pr(.) over f(r)
completed trajectories that can be expressed as Pş(r) =
2 where Z = =EVET F(T) represents
the total flow. The probability of visiting state can be written as Pr(s) EvreTgee F(T) Then, the
forward transition probability from state to state can be obtained as PF(8'(8) F(a-ta') The F(a)
trajectory flow F(-) is called consistent flow if for any state it satisfies E' F(s' Exat """"" F(s + s"), which constitutes that the in-flow and out-flow of state 8 are equal. [5] shows that if (()) is a consistent flow such that the terminal flow is set reward (i.e. F(x 85) R(x)),
the policy π--) defined as T(s' s) (s 8) satisfies π (x) = R(m) which means that the policy
(.) samples an object proportional to its reward.
3.2 Training GFlowNet Models
In order to learn the policy TT-). a GFlowNet model approximates trajectory flow with a flow function (-(.) where 0 includes learnable parameters of the flow function. To learn the flow function that can formulates flow-matching loss function as follows:
provide consistency condition, 5
(FM 0) -log log Exd's' Fols' +s)
Ever s0" Fo(s + 8")
Moreover, alternative objective function, 31 introduces trajectory balance as:
LTB(3;0) 0) = =(108 Zo II PFo (s'(s) R(xx)
(2)
where Ze is a learnable parameter. The trajectory-balant ballnce objective function in can accelerate
training GFlowNets and provide robustness to long trajectories. Given training dataset, optimization techniques such as stochastic gradient descent can be applied to objective functions in ( and (2)
to train the GFlowNet model. We use trajectory balance in this paper due its well-documented performance. Furthermore, it is worth noting that generating sequences in an autoregressive fashion using GFlowNet involves only one path generate a particular sequence. such cases, generating biological sequences with GFlowNet can be viewed as Soft-Q- consistency learning (PCL) 35 problem.
[15
33 and path
4 Sequence Editing with GFlowNet
To edit given sequence we propose identifying sub-optimal positions of such that editing them can lead to considerable improvement in the sequence property. Assume that the flow function (-) is trained available offline training data. GFNSeqEEitor uses the trained GFlowNet's flow function (-(.) to identify sub-optimal positions of æ. and subsequently replace the sub-optimal parts with newly sampled edits based on the stochastic policy TT-).
4.1 Sub-Optimal-Posiine Identification
This Subsection provides intuition on how GFNSeqEditor uses pre-trained flow function (- (-) to identify sub-optimal positions in sequence æ to edit. Let z; and æ., denote the t-th element and the first elements in the sequence æ. respectively. For example, 'AT², GFNSeqEditor constructs edited sequences token by token, and
DNA sequence ær = 'ATGTCCGC"
we have 22 = 'T' and for each position + 1, examines whether 3 should be edited or not. Using the flow function Fo F-).,, given o.s, GFlowNet evaluates the average reward obtained by appending any possible token

---

to Z.e. In this context, each token can be viewed as an action. Let T. + denotes the expanded a by appending token a. Let represent the available action set. For each a E A. using the state flow (a.: + a), the value of action a given T. can be evaluated. As discussed in Section 3 the state flow (x. + a) is proportional to the total reward of all possible sequences that have Dr + a as their prefix. Therefore, Fe (I., + a1) > (a + ag) indicates that taking action 41 instead of action a2 can lead to obtaining better candidates for the final sequence. We can leverage this property of the flow function (-( to examine if Ic+11 is sub-optimal or not. If the reward resulting from having Tt-1 in the seed sequence is evaluated by (-(()) to be relatively small compared to other possible actions, then T++1 is considered sub- sub-optimal. In partieular, F++1 is identified as sub-optimal if:
(z + x++1) < 6 max Fe (2,, + a),
(3)
aEA
where 0 I 8 < 1 is a hyperparameter. A larger increases the likelihood that the algorithm identifies 1 as sub-optimal. From (3), it can be inferred that r++1 is identified as sub-optimal if its associated out-flow is considerably smaller than the out-flow associated with the best possible action in A. This means that the flow function (o(.) suggests that replacing Iz-1 with other actions can lead to remarkable improvement in the sequence property.
4.2 Sequence Editing with GFNSeqEditor
Using the flow function Fo(.), GFNSerEditor iteratively identifies and edits positions in a seed sequence. J. should be edited to improve the target property value (3) Based on this intuition, we now modify (3) to formally define the sub-optimal-posittee identification function D(-) used by GFNSeqEditor,
simple function for determining if a position T++1 in sequence
Let áे. denote the first elements of the edited sequence. Assume that 3y E A, Vi, meaning that xy is always in the available actions. At each step of the algorithm, D-)) accepts - and evaluates whether appending 2y (from the seed sequence) to the edited partial sequence - is detrimental to the performance. In particular, modifying (3), the sub-optimal identifier function D--) checks the following condition:
Fo + x..)
Fo + a) aEk Eares Fo + a')
< 6 max
+ v,
Ev'es Fo + a²)
where ~ o2) is a Gaussian random variable with variance of 0². The variance x² is hyperparameter. The relation between a and the algorithm performance will be analyzed in Section 4.3 The inclusion of additive noise on the right-hand side of (4) introduces degree of randomness into the process of identifying sub-optimal positions. This, in turn, fosters exploration in the editing function (()) determines if x, is sub-optimal as follows:
process. The paapppimal-ppsttionideeeeee
{ If 4 is met
D(x,, t.
(5)
Otherwise
If D(1, L. d.-1;d, a)- = 0, at step t the algorithm appends 2, from the original sequence æ to - Otherwise, if D(x,, 1; σ) 1, the algorithm samples an action a according to the following policy:
Fol
T (al = (1- A) 1 Enrer +o(
+ Alammo
(6)
where 0 < A < 1 is a regularization coefficient and 1 la=z, denotes the indicator function and is 1 if = Fz. The regularization parameter À allows tuning the sampling process to favor the original sequence (a larger À leads to a smaller number of edits). The policy in (6) constitutes trade-off between increasing the target property and decreasing the distance between the edited sequence à and the original sequence æ. Let 7, be the action sampled by the policy x in (6). In summary, the 1-th element in the edited sequence can be written as:
f.- D(x,, - 0)); + (1 D(z,, 2,,11
Therefore, at each step I, the edited sequence is updated as क., = f. + Gr. The process continues until step T, where T = |x|| denotes the length of the original sequence a. Note that d. is an empty sequence. Algorithm summarizes GFNSegEditor.
5

---

Algorithm GFiteqEditor: Sequence Editor using GFlowNet
Input: Sequence æ with length T, flow function F-(-) and parameters and a.
2: Initialize Z.o an empty sequence.
3: for t 1, ................................ T do
4: Check if z is sub-optimal by obtaining D(z,, 12:11 6, σ) according to (5).
5: if D( 6, or) then
6: Sample 4t according to policy क. (-1(i:r-1) in (6).
7: else
Assign क, Zt- 9: end if
8:
10: end for
11: Output: Edited sequence a.
4.3
Analysis
This Subsection analyzes the reward and properties of the edited sequence as well as the number of edits performed by GFNSeqEditor. Specifically, the bounds for the reward of edited sequences, property improvement and the number edits are determined by the algorithm's 's hyperparameters o, 6, and A. The following theorem specifies the lower bound for the reward of edited sequences.
Theorem 4.1. Let T be the length of the original sequence æ. The expected reward of the sequence edited by GFNSeqEditor & given is bounded from below
[[ 2 (1- 1-
(1 (1-
RERTT
(8)
where (--) denotes the cumulative distribution funictition (CDF) the normal distribution and RET represents the expected reward of a sequence with length T generated using the flow function Fo
Proof of Theorem 4.1 is deferred Appendix A The following Theorem obtains the expected property improvement upper bound of the proposed GFNSeqEditor. The property improvement of sequence æ is defined PI = y where denotes the edited sequence property.
Theorem 4.2. Let S be the set of all sequences with length T which have larger properties than that of x (i.e., 4). Assume that S a non-empty set. The expected property improvement by applying GFNSegEditor on is bounded from above as
B[P S
Σ
(1 1
20 2 11-0(-2)
(pw - y)
WESt
where Pur denote the property of the sequence w.
The proof of Theorem 4.2 be found in Appendix B Theorems 4.1 and 4.2 demonstrate that an increase results in an increase in both the lower bound of reward and the upper bound of property improvement. While higher value of corresponds to larger lower bounds for the reward, an increase in a diminishes the upper bound of the property improvement. The following theorem obtains the upper bound on the number of edits performed by the proposed GFNSeqEditor.
Theorem 4.3. The expected distance between the edited sequence & by GFNSeqEditor and the original sequence x is bounded from above as:
Bleev x:) < (1
(1--)(1-((-)))
(10)
where lev(..)) is the Levenshtein distance between two sequences.
The proof for Theorem 4.3 is available in Appendix C The following Theorem specifies the lower bound for the number edits.
Theorem 4.4. Let there exists such that the flow function Fo(.) satisfies:
Fol (a) Evees Fo + a²) $ - e, Vl,
max GEA
(11)

---

meaning that the probability of choosing of each action is always less than 1 - The expected distance between the edited sequence E by GFNtegEditor and the original sequence is bounded from below as:
(
c(1 - 21 Φ( - ())1((1)-0)
E Ellev(2 R)]
T.
(12)
Proof of Theorem 4.4 can be found in Appendix D Theorems 4.3 and 4.4 show that 5 increases, both the lower and upper bounds of distance increase. In contrast, an increase in À leads to a decrease in both the lower and upper bounds of distance. Furthermore, Theorem 4.1 demonstrates that reduction in À results in a larger lower bound for the reward. Therefore, Theorems 4.1 and 4.3 reveal trade-off between the expected number of edits and the lower bound for the expected reward. While it is preferable to select hyperparameters ó and À that reduce the expected number of edits, an increase in the number of edits corresponds to a larger lower bound for the reward.
5 Experiments
We conducted extensive experiments to assess the performance of GFNSeqEditor in comparison to several state-of-the-art baselines across diverse DNA- and protein-sequence editing tasks. We evaluate on T'Fbinding, AMP, and CRE datasets. TFbinding and CRE datasets consist DNA sequences with lengths of 8 and 200, respectively. The task in both datasets is to edit sequences to increase their binding activities. The vocabulary for both TFbinding and CRE is the four DNA bases, [A, C, G. т}. AMP dataset comprises positive samples, representing anti-microbial peptides (AMPs), and negative samples, which are non-AMPs. The vocabulary consists of 20 amino acids. The primary objective is to edit the Aon-AMP samples in such a way that the edited versions attain the characteristics exhibited by AMP samples. Additional information about the datasets can be found in Appendix E.I.I
To evaluate the performance of sequence editing methods, we compute the following metries:
Property Improvement (PI): The PI for given sequence e with label y is calculated as the average enhancement in property across edits, expressed as where n. is the number of edited sequences associated with the original sequence x and ý; denote the property of the i-th edited sequence ă, To evaluate the performance of editing methods, for each dataset we leverage an oracle to obtain is given Φ; More details about oracles can be found in Appendix E
The Σn (ū: - y),
Edit Percentage (EP): The average Levenshtein distance between æ and edited sequences normalized by the length of expressed as 1
ne. Σi= lev( d;;).
Diversity: For each sequence æ. the diversity among edited sequences can be obtained as 2
m E Ej=1+1 lev(0;,.t))
GMDPI: The geometric mean of diversity and PI is measured. This metric highlights algorithms that exhibit strong performance in both aspects simultaneously.
We compared GFN to several baselines, including Directed Evolution (DE) 46 Ledidi [4]. LaMBO 48 MOGFN-AL 21]. GFlowNet-AL [1). and Seq2Seq. To perform Directed Evolution for sequence editing, we select set of positions uniformly at random within given sequence and then apply the directed-evolution algorithm to edit these positions. The implementation of the directed-evolution algorithm is the same as that of the Adaleea framework in (46) Inspired by graph-to-graph translation for molecular optimization in (23], we implemented another editing baseline, which is called Seq2Seq. For the Seq2Seq baseline, we initially partition the dataset into two subsets: i) sequences with lower target-property values, and ii) sequences with relatively higher target- property values. Subsequently, we create pairs of data samples such that each low-property sequence is paired with its closest counterpart from the high seehh-poperty sequence set, based on Levenshtein distance. A transformer is then trained to map each low- now-property sequence to its high-propene pair. Essentially, the Seq2Seq baseline maps an input sequence to similar sequence with a higher property value. Furthermore, we adapted GelowNet-AL for sequence editing, and named it Gel in what follows. In this baseline, the initial segment of the sequence serves as the input, allowing the model to generate the subsequent portion of the sequence. For Ti-binding, AMP, and CRE datascts, GFlowNet-E takes in the initial 70,, 65%, and 60% of elements, respectively, from the
7

---

Table 1: Performance of GFttrr compared to the baselines in terms of property improvement (PI), edit percentage (EP), diversity, and geometric mean of property improvement and diversity (GMDPI) on TFbinding, AMP, and CRE datasets. EP is selected to be approximately the same for all algorithms (if possible). Higher PI, diversity and GMDPI are preferable.
TFbinding
AMP Diversity
CRE
Methods
F.
Diversity
GMDPI
PT
EP(%)
GMDPI
PT
(P
Diversity
GMDPI
DE
0.12
25.00
3.01
0.60
0.11
33.82
13.67
1.23
0.63
22.93
62.07
6.25
Ledidi
0.06
27.80
1.25
0.27
0.18
34.79
11.65
1.45
1.36
22.13
50.49
8.29
LaMBO
0.05
25.00
3.14
0.40
0.12
34.33
15.61
1.36
0.79
23.35
62.95
7.05
MOGFN-AL
0.09
25.00
2.66
0.49
0.10
35.26
7.59
0.87
2.45
22.99
10.96
5.18
GFlowNett-E
0.11
28.35
2.10
0.48
0.28
35.68
3.42
0.98
4.24
22.73
37.06
12.53
Seq2Seq
0.03
41.98
0.21
78.05
Fditor
0.14
24.27
3.84
0.73
0.33
34.49
14.34
2.17
9.90 21.90
40.41
20.00
input sequence æ, and generates the remaining elements using the pre-trained flow function. More details on the baselines can be found in Appendix E.
To train both the baselines and the proposed GFNSer we divide each dataset into training, validation, and test sets with proportions of 72%, 18% and 10,, respectively. The test set serves the purpose of evaluating the performance of the methods in sequence editing tasks. The flow function F-(-) utilized by GFNSeqEditor and the GFlowNet-E baseline is an MLP consisting of two hidden layers, each with a dimension of 2048, and |A] outputs corresponding to actions. Throughout our experiments, we employ the trajectory balance objective to train the flow function. Additional details regarding the training of the flow function can be found in Appendix
5.1 Sequence Editing
Hopprry woorreeeeoonnnn vreeeemeet
AMP
10
CAE
E
Table presents the performance of GFNSeqEd- itor and other baselines on TFbinding, AMP, and CRE d 3l We set GFNSeqEditor and all the
2.5
baselines except for Seq2Seq to create 10 edited sequences for each input sequence. The Seq2Seq implementation closely resembles deterministic machine translator and is limited to producing just one edited sequence per imput, resulting in a diversity score of zero. Additionally, Figure 2 shows the property improvement achieved by GFNSeqEditor, DE, Ledidi, LaMBO. and MOGFN-AN AL across a range of edit percentages. As evident from Table [] and Figure 22 GFNSeqEditor outperforms all baselines, achieving substantial property improvements with controlled number of edits. This superior performance is attributed to GFNSeqEditor's utilization of a pre-trained flow function from GFlowNet, enabling it achieve significantly higher property improvements than DE, Ledidi,
Percentagg of Gdtt (1))
Perzartage of Edits (40
GPNSegtditor +
DE
Lendel
-XC
MOGPNAL
-
LaMOO
Figure 2: Property improvement of AMP (left) and CRE (right) with respect to edit percentage.
LaMBO, and MOGFN-AL, which rely on local search techniques by optimizing either a given single sequence or a batch of sequences. Specifically, the flow function (- is trained to sample sequences with probability proportional to their reward and, as result, employing the policy in (6) for editing enables GFNSeqEditor to leverage global information contained in (- about the entire space of sequences. Furthermore, GFNSegEditor achieves larger property improvement than GFlowNet-B. The GFNSeqEditor identifies and edits sub-optimal positions within a seed sequence using 4 while GFlowNet- only edits the tail of the input seed sequence. This indicates the effectiveness of the sub-optimal position identifier function of GFNSeqEditor.
Ablation study. We further study the property improvement achieved by GFNSeqEditod along with edit percentage across various choices of hyperparameters ō and X. Figure illustrates that an increase in 8 generally corresponds to an increase in both property improvement and edit percentage, whereas, in most cases, an increase in results in decrease in property improvement and edit percentage. Furthermore, in Figure 7 in Appendix E. we illustrate the impact of changing a on property improvement and edit diversity for GFNSeqEditor, This figure highlights that increasing results in decreased property improvement and enhanced diversity. These results corroborate the theoretical analyses outlined in Theorems 4.1 and i.3 Section 4.3
*Seq2Seq relies on identifying pairs of similar sequences for training. However, we were unable to identify similar pairs for CRE, possibly because of the limited number of training samples relative to the lengthy nature of the sequences (i.e., sequences with length of 200).
8

---

A
Proporty impravement
0.3
Expression on Improvement
0.05
0.35
19.66
45.05
41.4
16.0s
-9.2
$4.50
0.30
10
20:36
20.03
10-1
10-3
0.01
0.1
0.15
0.1
0.2
Figure 3: Studying the effect of heyperparameters 8 and À on the performance of GFNSeqEditor over AMP (left) and CRE (right) datasets. The marker values are edit percentages.
5.2 Assisting Sequence Generation
In addition to editing sequences, we investigate the ability of GFNtoqEditor to be used alongside se- quence generative model to enhance the generation of novel sequences. This highlights the versatility of the proposed GFNSeqEditor. In this Subsection, we utilize a pre-trained diffusion model (DM) for sequence generation, with further details available in Appendix E.2 The sequences generated by the
Table 2: Performance of DM, GelowNet and combination of DM with GFNSeqEditor for generating novel sequences.
AMP
CRE
Algorithms
Property
Diversity Property
Diversity
DM
0.06
23.56
1.75
107.38
Get DM*GFNSeglatior
0.74
17.86
28.20
83.88
0.73
23.78
26.42
103.10
DM passed to GFNtegEditor to improve their target property. Given that GFNSeqEditor utilizes a trained GFlowNet model, this combination of a DM and GeNSeqEditor can be regarded as an ensemble approach, effectively leveraging both the DM and the GFlowNet for sequence genera- tion. presentss the property and diversity metrics for sequences generated by the DM, the GelowNet, and the combined GMGGFNNeEEddto across AMP and CRE datasets, with each method generating 1, 0000 sequences. As observed from Table 2 with higher property values compared the DM, while the DM exhibits greater sequence diversity than the GelowNet. Sequences generated by DM+GFNSeqEditor maintain similar property levels the GelowNet on its own, while their diversity is in line with that of the DM. This highlights the effectiveness of DM+GFNSeqEditor in harnessing the benefits of both the GFlowNet and the DM.
GFlowNet excels at producing sequences
Moreover, show the CDF of the property for sequences generated by the DM, the GFlowNet, and DM+GFNSeqEdito in Figure 4 As shown, the CDF of DM+GFNSeqEditor aligns with both DM and GelowNet. Specifically, for AMP dataset, DM+GFNSeqEditor generates more se- quences with higher properties than 0.78 com- pared to GFlowNet, while reducing the number of low-property generated sequences compared to DM alone. In the case of CRE dataset, the results in Figure Figure indicaae that increases, the CDF of DM+GFNSe becomes more akin to that of GelowNet. This is expected, as an increase in 6 leads to a greater number of edits.
Figure 4: CDF of generated sequence properties for AMP (left) and CRE (right). A right-shifted curve indicates that the model is generating more sequences that are high in the target property.
5.3 Sequence Combination
GFNSeqEditor possesses the capability to com- bine multiple sequences, yielding novel se- quence that closely resembles its parent se- quences. This capability proves invaluable in several applications. For example, when it is important to shorten relatively lengthy quences while retaining desired properties (see, e.g., 59).. GFNSeqEditor accomplishes this by combining a longer sequence with a shorter one. The resultant sequence maintains high similar- ity with the longer one retain its desired properties, while also resembling realistic, relatively shorter sequence to ensure safety and predictability. Algorithm in Appendix(E.5) describbs using GFNSeqEditor combine two sequences with the goal of shortening the longer one.
Andinicctaall
Lence
Figure 5: GFNSeq effectively reduces the length of AMP sequence inputs (right) while keep- ing their properties intact (left).
We evaluate EddSerdditor's performance in combining pairs of long and short sequences using the AMP dataset as test case. In this context, a long sequence is defined as one with length exceeding

---

30, while short sequence has length shorter than 20. Each initial pair consists of a long AMP sequence and its closest short sequence with an AMP property exceeding 0.7. Table Figure in Appendix F.5 present the results of sequence combination for sequence length reduction. As indicated Table 5 GFNSeqEditor not only enhances the properties of the initial long sequences, but also significantly shortens them by more than 63%. Additionally, the sequences generated by Edttor resemble both the initial long and short sequences, with an average Levenshtein similarity of approximately 65% to long sequences and 55% to short sequences.
Conclusions
This paper introduces GFNSeqEditor, a generative model for sequence editing built upon GFlowNet. Given an input seed sequence, GFNSeqEditor identifies and edits positions within the input sequence to enhance its property. This paper also offers theoretical analysis of the properties of edited sequences and the amount of edits performed by GFNtegEditor. Experimental evaluations using real-world DNA and protein datasets demonstrate that GFNSeqEditor outperforms state-of-the-an baselines in terms of property enhancement while maintaining a similar amount of edits. Nevertheless, akin to many machine learning algorithms, GFNSegEditor does have its limitations. It relies well-trained GFlowNet model, necessitating the availability high-quality trained GelowNet for optimal performance.
References
[1] Angermueller, C., Dohan, D., Belanger, D., Deshpande, R., Murphy, K., and Colwell, L. Model- based reinforcement learning for biological sequence design. In International conference on learning representations, 2019.
[2] Amoold, F. H. Design by directed evolution. Accounts of chemical research, 31(3):125-131, 1998.
[3] Avdeyev, P. Shi, C., Tan, Y., Dudnyk, K., and Zhou, J. Dirichlet diffusion score model for biological sequence generation. arXiv preprint arXiv:2305. 10699, 2023.
[4] Barrera, L. A., Vedenko, A., Kurland, J. V., Rogers, J. M., Gisselbrecht, S. S., Rossin, E. J., Woodard, J., Mariani, L., Kock, K. H., Inukai, S., al. Survey of variation in human transcrip- tion factors reveals prevalent dna binding changes. Science, 351 351(6280); 1450-1454, 2016.
[5] Bengio, E., Jain, M., Korablyov, M., Precup, D., and Bengio, Y. Flow network based generative models for non-iterative diverse candidate generation. In Advances in Neurul Information Processing Systems, volume 34, pp. 27381-27394, 2021.
[6] Bengio. Y., Lahlou, S., Deleu, T., Hu, E. J., Tiwari, M., and Bengio, E. Gflownet foundations. Journal of Machine Leurning Research, 24(210):1-55, 2023.
[7] Chen, C. S., Zhang, Y., Liu, X., and Coates, M. Bidirectional learning for offline model- based biological sequence design. Proceedings of the International Conference on Machine Learning, 2023.
[8] Chen, Y. and Mauch, L. Order-preserving GFlownets. In International Conference on Learning Representations, 2024.
[9] Cox, D. T., Platt, R. J., and Zhang, F. Therapeutic genome editing: prospects and challenges. Nature medicine, 21 21 2015.
[10] Dadkhahi, H., Rios, J., Shanmugam, K., and Das, P. Fourier representations for black-box optimization over categorical variables. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 10156-10165, 2022.
[11] Deleu, T., Góis, A., Emezue, C.,, Rankawat, M., Lacoste-Julien, S., Bauer, S., and Bengio, Bayesian structure learning with generative flow networks. In Uncertainty Artificial Intelligence, pp. 518-528, PMLR, 2022.
10

---

[12] Gosai, S. J., Castro, R. I., Fuentes, N., Butts, J. C.. Kales, S., Noche, R. R., Mouri, K., Sabeti, C., Reilly, S. K., and Tewhey, R. Machine Machine-guided design of synthetic cell type-specific cis-regulatory elements. bioRxiv, pp. 2023-08, 2023.
[13] Grau-Moya, J., Leibfried, F., and Vranex, P. Soft q-learning with mutual innioaaaiiooootoionn regular- ization. International Conference on Learning Representations, 2019.
[14] Guo, S., Chu, Zhu, L., and Li, Dynamic backtracking in gilownet: Enhancing decision steps with reward reward-dependent adjustment mechanisms. arXiv preprint arXiv:2404.05577 2024.
[15] Haarnoja, T., Tang, H., Abbeel, P., and Levine, S. Reinforcement learning with deep energy- based policies. In Proceedings of the International Conference Machine Learning, pp- 1352-1361, 2017.
[16] Hansen, N. The ema evolution strategy: a comparing review. Towards a new evolutionary computation: Advances in the estimation of distribution algorithms, pp. 75-102, 2006.
[17] Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020. URL /tttps://giii com/ hoooothhanho/ diiffssionffffoo
[18] Hoffman, S. C., Chenthamarakshan, V., Wadhawan, K., Chen, P.-Y., and Das, P. Optimizing molecules using efficient queries from property evaluations. Nature Machine Intelligence, 4(1): 21-31. 2022.
[19] Jain, M., Bengio, E., Hernandez-Garcia, A., Rector-Brooks, J., Dossou, B. F. P., Ekbote. C. A., Fu, J., Zhang. T., Kilgour, M., Zhang, D., Simine, L.,, Das, P., and Bengio, Y. Biological sequence design with GFlowNets. In Proceedings of the 39th International Conference Machine Learning, volume 162. pp. 9786-9801, Jul 2022.
[20] Jain, M., Deleu, T., Hartford, J., Liu, C.-H., Garcan, A., and Bengio, Gflownets for ai-driven scientific discovery. Digital Discovery, 2(3):557-577 2023.
[21] Jain, M., Raparthy, S. C., Germandez-GGarciaa A., Bector-Brooks, J., Bengio, Y., Miret, S., and Bengio, E. Multi-objective gflownets. In Proceedings of the International Conference on Machine Learning, 2023.
[22] Jang, H., Kim, M., and Ahn, S. Learning energy decompositions for partial inference in GFlownets. In International Conference on Learning Representations. 2024.
[23] Jin, W., Yang, K., Barzilay, R., and Jaakkola, T. Learning multimodal graph-to-graph translation for molecule optimization. In International Conference on Learning Representations, 2019.
[24] Kim, H., Kim, M., Choi, S., and Park, J. Genetic-guided gflownets: Advancing in practical molecular optimization benchmark. arXiv preprint 1xXv::2402.05961, 2024.
[25] Kim, M., Ko, J., Yun, T., Zhang, D., Pan, L., Kim, W. C., Park, J., Bengio, E., and Bengio, GFlowNets. In Proceedings of the
Y. Learning to scale logits for temperature-conittin International Conference Machine Learning, volume 235, pp.
Jul 2024.
[26] Kim, M., Yun, T., Bengio, E.., Zhang, D., Bengio, Y., Ahn, S., and Park, J. Local search GFlownets. In International Conference on Learning Representations, 2024.
[27] Koziarski, M., Abukalam, M., Shah, V., Vaillancourt, L., Schuetz, D. A., Jain, M., van der Sloot, A. M., Bourgey, M., Marinier, A., and Bengio, Y. Towards DNA-encoded library generation with GFlownets. In ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design, 2024.
[28] Li. H., Yang, Y., Hong. W., Huang, M., Wu, M., and Zhao, X. Applications of genome editing technology in the targeted therapy of human discases: mechanisms, advances and prospects. Signal transduction and targeted therapy, 5(1):1, 2020.
[29] Li, W., Li, Y., Li, Z., HAO, J., and Pang, Y. DAG matters! GFlownets enhanced explainer for graph neural networks. In The Eleventh International Conference on Learning Representations, 2023. URL htttps://ppeerriiieee net feett/forummiddaae jww.daM--jjuMMM-------
11

---

[30] Madan, K., Brooos, J.. Korablyov, M., Bengio, E., Jain, M.. Nica, A. C., Bosc, T., Bengio. Y., and Malkin, N. Learning gflownets from partial episodes for improved convergence and stability. In International Conference on Machine Learning, pp. 23467-23443 PMIR, 2023.
[31] Malkin, N., Jain, M., Bengio, E., Sun, C., and Bengio, Y. Trajectory balance: Improved credit assignment in GFlownets. In Advances in Neural Information Processing Systems, 2022.
[32] Malkin, N., Lahlou, S., Deleu, T., Ji, X., Hu, E. J., Everett, K. E.., Zhang, D., and Bengio, Y. GFlownets and variational inference. In The Eleventh International Conference on Learning Representations, 2023.
[33] Mohammadpour, S., Bengio, E., Frejinger, E.,, and Bacon, P.-L. Maximum entropy GFlowNets with soft Q-learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics, volume 238, pp. 2593-2601, May 2024.
[34] Mullis, M. M., Rambo, I. M., Baker, B. J., and Reese, B. K. Diversity, ecology, and prevalence of antimicrobials in nature. Frontiers in microbiology, pp. 2518, 2019.
[35] Nachum, O., Norouzi, M., Xu, K., and Schuurmans, D. Bridging the gap between value and policy based reinforcement learning. In Proceedings of the International Conference on Neural Information Processing Systems, pp. 2772-27833 2017.
[36] Vishikawa-Toomeyy M., Deleu, T., Subramanian, J., Bengio, Y., and Charlin, L. Bayesian learning of causal structure and mechanisms with gflownets and variational bayes. arXiv preprint N.iv:2211.027663 2022.
[37] Niu, P., Wu, S., Fan, M., and Qian, X. GFlowNet training by policy gradients. In Proceedings of the International Conference on Machine Learning, volume 235. pp. 38344-38380, Jul 2024.
[38] Pan, L., Zhang, D., Courville, A., Huang, L., and Bengio, Y. Generative augmented flow networks. arXiv preprint arXiv:22/0.03308, 2022.
[39] Pan, I.., Zhang, D., Jain, M., Huang, I.., and Bengio, Y. Stochastic generative flow networks. arXiv preprint Sidi:2302.094666 2023.
[40] Pirtskhalava, M., Amstrong, A. A., Grigolava, M., Chubinidze, M., Alimbarashvili, E., Vish- nepolsky, B., Gabrielian, A., Rosenthal, A., Hurt, D. E., and Tartakovsky, M. DBAASP v3: database of antimicrooiial/cytttooie activity and structure of peptides as resource for development of new therapeutics. Nucleic Acids Research, 49(D1)):D888-D297, 11 2020.
[41] Ren, Z., Li, J., Ding, F., Zhou, Y., Ma, J., and Peng, Proximal exploration for model-guided protein sequence design. In Proceedings of the International Conference on Machine Learning, volume 162, pp. 18520-18536, Jul 2022.
[42] Schreiber, J., Lu, Y. Y., and Noble, W. S. Ledidi: Designing genomic edits that induce functional activity. bioRxiv, 2020. doi: 10. seneeett/early/2220055555 202//0022002..2 05. 21, 109686
URL https::/www. biorxiv...vvvv.. org/
[43] Shen, M. W., Bengio, E., Hajiramezanali, E., Loukas, A., Cho, K., and Biancalani, T. Towards understanding and improving gflownet training. arXiv preprint arX7v:2305.07170, 2023.
[44] Sidorezuk, K., Gagat, P., Pietluch, F., Kala, J., Rafacz, D., Bąkata, I.,, Stowik, J., Kolenda, R., Rödiger, S., Fingerhut, L. C. H. W., Cooke, I. R., Mackiewicz, P., and Burdukiewicz, M. Benchmarks in antimicrobial peptide prediction are biased due to the selection of negative data. Briefings in Bioinformatics, bbbc)bb Sep 2022.
[45] Silva, Т., Carvalho, L. M., Souza, A. H., Kaski, S., and Mesquita, D. Embarrassingly parallel GFlowNets. In Proceedings of the International Conference on Machine Learning, volume 235, pp. 444401 Jul 2024.
[46] Sinai, S., Wang, R., Whatley, A., Slocum, S., Locane, E., and Kelsic, E. D. Adalead: A simple and robust adaptive greedy search algorithm for sequence design. arXiv preprint 2xXiv20010.02141, 2020.
12

---

[47] Song, Y., Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011 13456, 2021.
J., Brain, G., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B.
[48] Stanton, S., Maddox, W., Gruver, N., Maffettone, P., Delaney, E., Greenside, P., and Wilson, A. G. Accelerating Bayesian optimization for biological sequence design with denoising autoencoders. In Proceedings of the International Conference Machine Learning, volume 162. pp. 20459-20478, Jul 2022.
[49] Swersky, K., Rubanova, Y., Dohan, D., and Murphy, K. Amortized bayesian optimization over discrete spaces. In Conference on Uncertainty in Artificial Intelligence, pp. 769-778. PMLR, 2020.
[50] Taskiran, I., Spanier, K. I., Christiaens, V., Mauduit, D., and Aerts, S. Cell type directed design of synthetic enhancers. bioRxiv, pp. 2022 07, 2022.
[51] Terayama, K., Sumita, M., Tamura, R., and Tsuda, K. Black-box optimization for automated discovery. Accounts of Chemical Research, 54(6):1334-1344 2021.
[52] Trabucco, B., Kumar, A., Geng, X., and Levine, S. Conservative objective models for effective offline model-based optimization. In International Conference on Machine Learning, pp. 10358-10368. PMLR, 2021.
[53] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.,, Gomez, A. N., Kaiser, L. u., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017.
[54] Xu, X., Chemparathy, A., Zeng. L., Kempton, H. R., Shang. S., Nakamura, M., and Qi, L. S. Engineered miniature crispr-cas system for mammalian genome regulation and editing. Molecular Cell, ((20):4333-4345.e,, 4345.ee, 2021.
[55] Zhang, D., Fu, J., Bengio, Y., and Courville, A. Unifying likelihood-free inference with black-bor box optimization and beyond. arXiv preprint arXhv:2110.03372,, 2021.
[56] Zhang, D.,, Chen, R. T., Malkin, N., and Bengio, Y. Unifying generative models with gflownets. arXiv preprint arXiv: 02000,,,,2 2022.
[57] Zhang, D., Malkin, N., Liu, Z., Volokhova, A., Courville, A., and Bengio, Y. Generative flow networks for discrete probabilistic modeling. In International Conference on Machine Learning, pp. 26422-4222222 26412-26428, PMLR, 2022.
[58] Zhang, D., Pan, I.., Chen, R. T., Courville, A., and Bengio, Y. Distributional gflownets with quantile flows. arXiv preprint arXiv:2302.00599 2023.
[59] Zhao, F., Zhang, T., Sun, X., Zhang, X., Chen, L., Wang, H., Li, J., Fan, P., Lai, L., Sui, T., et al. A strategy for cas13 miniaturization based on the structure and alphafold. Nature Communications, 14(1):5545, 2023.
[60] Zimmermann, H., Lindsten, F,, van de Meent, J.-W., and Naesseth, C. A. A variational perspective on generative flow networks. arXiv preprint arXiv:2210.07992 2022.
[61] Zrimec, J., Fu, X., Muhammad, A. S., Skrekas, C., Jauniskis, V., Speicher, N. K., Börlin, C. S., Verendel, V., Chehreghani, M. H., Dubhashi, D., et al. Controlling gene expression with deep generative design of regulatory dna. Nature communications, 13(1)5099, 2022.
13

---

A Proof of Theorem 4.1
Let denotes sequence with length T generated from scratch using the policy Ty (+)
Fo(z, + a)
(r(a)za) Ew'ex (z. +2)
(13)
The expected reward of be obtained
RF
[|
Σ -riz= [))(w) w
Σ wET, II TF TF (wt.::/=-)))(000000
(14)
where Tr denotes the set of sequences with length 7 that can be generated by Fo(-). The probability that the GFNSegEditor outputs an arbitrary sequence w Tr given æ can be expressed
Pr[# w//x|
II Pr Pr(a, wy 1,
(15)
The probability Pr(a; w/ 1- 2] can be obtained as
Pr(f: we ar) Pr(D (2.
||][[w w.-11 Pr(D(0a-1;000)) 00 =
2 Pr[D( B.. 6, o) = []]r(w; w..-1)
(16)
where (-) defined 66. According to (6) it can be written that
F.
aww,
En EA Fo ( d() (1-
(17)
Furthermore, according to and we have D;
F ree)
Fol#:1-1 + a)
max ae
EeEa Fo + S.V
(18)
EN'E Fo + a')
In addition, it can be inferred that
F + 124) Ev'c Fo + a!)
Fo + a)
5 max ae
Enee Fo + a') 1
(19)
Therefore, it can concluded that if v/ > 1 - 6, it is guaranteed that 1 0) = 1. Since follows a Gaussian distribution with a variance o² we have 1 with probability ((==)). Hence, it can be written that
Pr(p(a...:ddo)- = 1] 21 2 -
(20)
Combining 200 and 177 with (T)), we get
Pr[a: = w/ w m| (1 x)= (1-0(--
(21)
Moreover, combining (1 with 15 obtain
II (1 () (1 (1-01--))
Pr(a w|x) ac
(22)
σ
Using 22 for the expected reward of å given we can write
E[R(t)z) = Σ
Pr(a w(x)
TETT
T [1 () (1 2 (1-() 10)--)) R(w).
(1-
Σ TET [=1
(23)
Combining 23 with we get
[[ 2 (1 - X) (1 RET
(24)
which proves g Moreover, the upper bound of property improvement by the proposed GFNSeqEdi- tor is analyzed in Appendix B
14

---

B Proof of Theorem 4.2
The expected property improvement of GFNoegdditor can be obtained as
[|
Σ
Pr(#
w|a](Pw y).
(25)
wEIr
Since Tr can be split into two sets S and Tr S the expected property improvement of GFNSe- qEditor can be obtained as
E P
Σ Pr(a (p v) wE3
Pr(a w w|x) y)-
(26)
W 182
If wo E Tr - then Pw S y. Therefore, the expected property improvement of GFNSeqEditor can be bounded from above as
Pr(a
Pr(a un/z y)-
(27)
WES
The probability that the EdNter outputs w S can expressed as
Pr[ø w|/]|
Pr(a. = w/ -0,
(28)
1=1
The probability Pr(f, we 22 x can be obtained as
Pr[ee = we w æ| = Pr[D )
[):(ww/w..r)) + +r 0]
(29)
24 # Wt, according (37) and considering the fact that ( I 1, the probability in (29) can be bounded from above as
Pr(a, we 11 x]|
(30)
Otherwise 2, Wa, can be written that Pr[c, different from æ in at least one position, combining (28 with 30
W/ - x S Since any w S should be
we can conclude that
Pr(s = w æ:
1
+
(31)
Combining
27 with 31 proves the Theorem.
C Proof of Theorem 4.3
We obtain the upper bound for the expected distance between edited sequence & and the original sequence æ. Since both æ and & have the same length T. the distance lev( .) be interpreted as the number of elements different in these two sequences. Therefore, in order to obtain lev( .)., sufficient to find the number of times that x; # ft, Vt 1 < T. If D(2; o) then f Fz. Furthermore, if 1( = 1. then according to (6), we have &; 2, with probability
Fo Fol + It.)
A) E Ca'ER F f + a')
+ X.
(32)
Therefore, the probability Pr[a, # xæ] can be obtained as
Pr[h; + x1] = Pr(D( = 1](1 - X) 1)(1-1 EvEE Fo a')
Fo +
(33)
Since Fo 21) 2 0, the probability Pr(a; 2x] can be bounded
Pr|a: + xz] <
= 1](1 A).
(34)
15

---

Moreover, if we have
v Fo : + x4))
( + a)
8 max aE
(35)
La'eA F +a)
Ex'Er Fe ( + a²)
then (( 0. Furthermore, the right hand side of (35) can be bounded from below as
Fo + x24) ò max En'eh Fo 0 + a') 2 5.
F +
(36)
En' Fo( a²)
Therefore, if v S -6, is ensured that D( 0, 0. The probability that S -6 is $ -))
Hence, we can conclude that
8
= - =
- 16 $(--)
(37)
Combining 37 with 34). we arrive at
Pr[fe + xzl S (1- (
(1 1 A).
(38)
Moreover, since both and & have the same length T, the expected Levenshtein distance between and è can obtained as
T Σ
(x a)]
Pr[as + mz).
(39)
Thus, combining 33 with
38 we can write that
(1 1 (1=0(-2)) (1 (1-
E lev(2 x)] <
(40)
which proves the Theorem.
D Proof of Theorem 4.4
According to (33 and the assumption in 11 it can be written that
Pr[& * x Pr Pr( 1](1 A)e.
(41)
Combining 41 with we get
Pr[s: + m. 2 (1- - A) (1)
(42)
Summing
42
over all elements in the sequence proves the theorem.
E
Supplementary Experimental Results and Details
This appendix provides comprehensive overview of the experimental setup in Section 5 and presents additional supplementary experimental results.
E.1
Implementation Details
All training and inferences, including GFNSeqEditor, have been conducted using a single Nvidia Quadro P6000.
E.1.1 Datasets
Detailed information about the datasets can be found below:
16

---

The dataset is taken from 4 and contains all possible DNA sequences with length 8. The vocabulary is the four DNA bases, (A, C. G, T). The goal is to edit a given DNA sequence to increase its binding activity with certain DNA-binding proteins called transcription factors. Higher binding activity preferable. For train, test and validation purposes 50% of the dataset is set aside. The task entails editing test dataset consisting of 10% of samples while the remaining data is utilized for training and validation.
TFbinding:
AMP: The dataset, acquired from DBAASP [40, is curated following the approach outlined by [19]. Peptides (i.e. short proteins) within sequence-length range of 12 to 60 amino acids are specifically chosen. The dataset comprises total of 6, 438 positive samples, representing anti-microbial peptides (AMPs), and 9,522 negative samples, which are non- AMPs. The vocabulary consists of 20 amino acids. The primary objective is to edit the non-AMP samples in such a way that the edited versions attain the characteristics exhibited by AMP samples. The task primarily centers on editing subset comprising 10% of the non-AMP samples, designated for use as test samples, with the remaining samples allocated for training and validation purposes.
CRE: The dataset contains putative human cis regulatory elements (CRE) which are reg- ulatory DNA sequences modulating gene expression. CREs were profiled via massively parallel reporter assays (MPRAs) [2] where the activity is measured as the expression of the reporter gene. For our analysis, we randomly extract 10, 000 DNA sequences, each with a length of 200 base pairs, utilizing a vocabulary of the four bases. The overarching objective is to edit the DNA sequences to increase the reporter gene's 's expression specifically within the K562 cell line, which represents erythroid precursors in leukemia. The task involves editing subset of 1, 000 test samples, while the rest are allocated for training and validation purposes.
E.1.2 Oracles
To evaluate the performance of each sequence editing method in terms of property improvement, it is required to obtain the properties of edited sequences. To this end, we employ an oracle for each dataset. The TFbinding dataset contains all possible 65, 792 DNA sequences with length of 8. Therefore, by looking into the dataset the true label of each edited sequence can be found. Following 19). the AMP dataset is split into two parts: D2 and D2. The oracle for the AMP dataset is set of trained models on partition D2 as a simulation of wet-lab experiments. We employed oracles trained by [19 for AMP dataset. It is worth noting that the performance of predictive models on AMP datasets can be influenced by negative sampling in the dataset [44]. Furthermore, for CRE dataset we leverage the Malinois model [12] which is deep convolutional neural network (CNN) for cell type-informed CRE activity prediction of any arbitrary sequence.
E.1.3 Baselines Implementation
DE and Ledidi. In order to implement DE and Ledidi baselines, there should be a proxy model to enable baselines to evaluate their candidate edits each iteration of these algorithms. For each dataset, we train a proxy model on the training split of each dataset. For the TFBinding dataset, we configure a three-layer MLP with hidden dimensions of 64. the case of AMP, we opt for a four-layer MIP,, also with hidden dimensions of 64. Finally, for CRE, we utilize four-layer MLP with hidden dimensions set to 2048. Across all models, the learning rate is consistently set to 10-4, ReLU serves as the activation function, and we set the number of epochs as 2, 000. To implement the DE baseline, we randomly select edit locations based on the desired edit percentages. At each selected location, we apply an edit by choosing the action that maximizes the proxy model's property prediction.
LaMBO. We utilize the official implementation of L Test sequences serve as the candidate pool, and all candidate samples in the pool are weighted similarly to maintain consistency with other sequence editing baselines. To ensure a fair comparison, we employ the same proxy GFNSeqEditor to calculate the property scores. Across all three datasets, we use mlm as the encoder objective, ei as the acquisition function, and DKL SVGP regression as the surrogate. To generate 10 edits per sample, we configure num gens=10, and window-siae is adjusted for each dataset to ensure the edited
For the AMP dataset, we have removed the samples with a length lower than window-size.
17

---

percentages closely match the desired values. Hence, we set it to 2. 14, and 23 for TFBinding, AMP, and CRE, respectively. Additionally, for all datasets, we set pref-conde cond-oood-a,s, pref alpha=1. and beta-sched=1.
MOGFN-AL. We utilize the official implementation of MOGEN-AI where sequence tasks based on the LaMBO implementation. Similar to LaMBO, test sequences serve as the candidate pool, weighted equally. The same proxy as GFtto and LaMBO employed calculate property scores. Like LaMBO, we utilize mlm as the encoder objective. Training and validation batch sizes are set to 16 and 64, respectively, with hyperparameters for the conditional transformer as follows: num-hic wow--iid--211 128, num-layers 3, and num- num-head-8 For each dataset, we adjust the max-len parameter to closely match the desired edited percentage. Thus, for TFBinding. AMP, and CRE, we set them as 6, 30, and 50, respectively. The reward function is modified to match that of GFNSeqEditor. To ensure fair comparisons and adapt LaMBO and MOGFN-AL for sequence editing, we do not use active learning settings for either baseline.
Seq2Seq. In order implement Seq2Seq baseline we use standard transformer [53] as the translator to map an input sequence to an output sequence with superior property. We paired samples in each dataset such that each pair contains a sequence with lower property and a similar sequence with higher property which is the most similar to the sequence with lower property in the dataset. The transformer trained map the low property sequence to the high property sequence in each pair. The transformer is trained using the standard configurations in Pytorch transformer module tutorial. Both the embedding dimension of the transformer and the dimension of the 2 layer feedforward network model in the transformer encoder are set to 200. The number of heads in multihead attention layer 2 and the drop-out rate is 0.2. We employ the CrossEntropyLLos function in conjunction with the stochastic gradient descent optimizer. The initial learning rate is set at 5.0 and follows a StepLR schedule.
E.1.4 GFlowNet Training
Both the baseline GFlowNet-E and the proposed GFNSeqEditor use the same trained GelowNet model. We trained an active learning based GFlowNet model following the setting in [19]. In active learning setting, at each round of active learning sampled and then top K samples based on scores given by proxy are chosen to be added to the offline dataset. Here offline dataset refers to an initial labeled dataset. To train the GFlowNet, we employed the same proxy models as those used by other baseline methods. For all datasets, we set the number of active learning rounds to I, with equal flow using MLP comprising two hidden layers, each with a dimension of 2048, and |A] outputs corresponding to individual actions. Throughout our experiments, we employ the trajectory balance objective for training. Adam optimizer with (3u, 31) = (0.9, 0.999) is utilized during the training process. The learning rate for log in trajectory balance loss is set to 10 -3 for all the experiments. The number of training steps for TFbinding. AMP and CRE are 5000, 10 and 104, respectively. The remaining hyperparameters were configured in accordance with the settings established in [9]
K candidates generated by GFlowNet are
and K equal to 100. We parameterize the
E.2 Diffusion Model Training
We trained our diffusion models on the full sequence datasets of AMP sequences or CRE sequences. The sequences were one-hot encoded, yielding 20- 2e-vers for protein sequences and 4-vectors for DNA sequences.
We employed the "variance- preserving stochastic differential equation" (VP-SDE) 47
We used
variance schedule of B(t) = 0.9t + 0.1. We set time horizon T (i.e. t E [0, 1)). This amounts to adding Gaussian noise over continuous time.
For our discrete-time diffusion model, we defined discrete-time Gaussian noising process, following 17]. We defined 3, = (1 x 10-4) + (1 x 10 10-5).. We set our time horizon T 1000 (i.e.
t [0, 1000]).
Our denoising network was based on a transformer architecture. The time embedding was computed as (sin f x), c2s(2 2)], where is 30-vector of Gaussian-distribute parameters that are not trainable. The time embeddings were passed through two dense layers with a sigmoid in between,
18

---

nanalp Irpuc AMP
Figure 6: GFNSeqEditor shifts the distribution of non-AMP inputs to the known AMPs.
22.5
Property improvement
Expressioo inpovmmerr
Diversity
20.0
0.35
Diversity
80
0.34
60
12.5
Figure Studying the effect of hyperparameter a on the diversity and performance GFNteq over AMP (leftt) and CRE (right) datasets.
mapping to 256-vector of time representations. For any input in batch, was concatenated with the time embedding and sinusoidal positional embedding (defined in (53) of dimension 64. This concatenation was passed to a linear layer to map it to 128 dimensions. This was then passed to standard transformer encoder of 3 layers, attention heads, and with hidden dimension of 128 and an MLP dimension of 64. The result was then passed to a linear layer which projected back to the input dimension.
We trained our diffusion models with a learning rate of 0.001, for 100 epochs. We noted that the loss had converged for all models that point. We also employed empirical loss weighting, where the loss of each input in a batch is divided by the L.2 norm of the true Stein score.
We trained our diffusion models on single Nvidia Quadro P6000.
When generating samples from continuous-tin diffusion model, we used the predictor-corrector algorithm defined in [47), using 1000 time steps from T to O. We then rounded all outputs to the nearest integer recover the one-hot encoded sample.
E.3
Supplementary Results for Sequence Editing
In Figure we illustrate the distribution of input non-AMP sequences, the sequences edited by GFNSegEditor, and the AMP samples from the AMP dataset. It is evident from Figure 6 that GFNSeqEditor shifts the property distribution of input non-AMP sequences towards that of AMP sequences. Moreover, diversity for GFNSeqEditor. As can be seen increasing a results decreased property improvement and enhanced diversity.
the impact of changing on property improvement and edit
worth noting that GFNSeqEditor capable of performing edits even when certain portions of the input sequence are masked and cannot be modified. Table showcases the performance of GFNSeqEditor compared to Ledidi on the CRE dataset, with the first 100 elements of the input sequences masked. As depicted in Table 3 GFNSeqEditor achieves significantly greater property improvement than Ledidi while utilizing a lower edit percentage.
E.4 Supplementary Results for Sequence Generation
This Subsection compares the performance of GFNSeqEditor in sequence generation task with that of GelowNet and diffusion model (DM) on CRE dataset. We relax the hyperparameters to allow a higher amount of edits and we set 5 = 0.4, X = 0.1 and 0.001 for GFNSeqEditor. The results are presented in Table GFlowNet and DM generate 1000 sequences. GFNSeqEditor also generates 1000 sequences by editing each of 1000 samples in the test dataset. As can be seen, GFNSegEditor
19

---

Table 3: Performance of GFttorEditor and Ledidi with 100 elements of each sequence masked for editing for CRE dataset.
Algorithms
PI
EP(%)
Diversity
PI
EP(%)
Diversity
Ledidi
0.52
18.69
38.34
0.26
14.39
37.45
GFNSegEditor
4.79
17.89
32.30
4.05
14.19
25.52
Table Performance of Edttor GFlowNet and DM on generating new sequences for CRE dataset.
Algorithms
Property
Diversity
Distance( (%)
DM
1.75
107.38
63.59
GFlowNet
28.20
83.88
54.41
GFNSeqEditor
29.25
87.32
47.34
achieves higher property than both GFlowNet and DM. It useful to note that the experimental study by [19] have shown that GFlowNet outperforms state-of-the-ar sequence design methods. For each sequence generated by GFlowNet and DM, the distance to the test set is measured as the distance between the generated sequence and its closest counterpart in the test set. On average, the distance between sequences generated by GelowNet and the test set 54.34%, while for DM, is 63.59%. GFNSeqEditor achieves superior performance by editing, average, 47.3 47.34% of sequence in the test dataset. The distance between test set and generated sequences by GFlowNet and DM cannot be controlled. As it is studied in Figures 3 and 7 the amounts of edits performed by GFNSeqEditor can be controlled by hyperparameters
and a.
E,5 Supplementary Discussion and Results for Sequence Combination
Algorithm 2 presents the GFNSeqEditor for combining two sequences in order to obtain new sequence whose length is the length of shorter sequence. In Figure we depict the distributions of input sequence lengths and properties, alongside the lengths and properties of the outputs generated by GFNSeqEditor. This scenario pertains the combination of long AMP sequence with short AMP sequence, as detailed in Subsection 5.3 As depicted in Figure the edited sequences produced by GFNSeqEditor exhibit property distributions akin to those of the lengthy input AMP sequences. Simultancously, these edited sequences are considerably shorter than the original long input sequences. This highlights GFNSeqEditor" 's effectiveness in shortening lengthy AMP sequences while preserving their inherent properties.
Furthermore, provides results for combining pairs of AMP sequences as well as pairs con- sisting of an AMP sequence and non-AMP sequence. In both cases, GFNSeqEditor generates sequence with a length matching that of the longer sequence. When combining two AMP sequences, GFNSeqEditor produces new sequences with higher properties than their parent sequences, maintain- ing an average resemblance of over 60% to each parent. Additionally, GFNSeqEditor can be applied to combine AMP sequence with an AMP sequence, offering the advantage of rendering the edited non-AMP sequence more akin to a genuine AMP sequence. The results in mebmonsjdemotttrte that EFNttr substantially enhances the properties of non-AMP sequences, surpassing the properties of their AMP parents. Furthermore, on average, 35% of the edited sequences bear a resemblance their AMP parents.
F
Supplementary Related Works
GFlowNets, initially proposed by [5 were introduced as a reinforcement-leamiin (RL) algorithm designed to expand upon maximum-entron RL, effectively handling scenarios with multiple paths leading to a common state. However, recent studies have redefined and generalized its scope, describing
general framework for amortized inference with neural networks [32 20 60 56).
There been recent surge of interest in employing GFlowNets across various domains. Noteworthy examples include its utilization in molecule discovery [5], Bayesian structure learning [|] 36), and
20

---

Algorithm 2 GFtor for combining two sequences to shorten the length of longer sequence.
Input: æ and æ2 with lengths Ii and 72, flow function (-(() and parameters and o.
2: Initialize 2., an empty sequence and Tiin = min T2).
3: for 1,...... Tuin do
4: Assign = arg (2.. ( + x1,1), Fo ( + x2,1)
5: Check if is sub-optimal by obtaining (2,, 1 8, σ) according (
6: if Daz,, 6; o) = then
7:
Sample & according policy n (--(a:e-1) as follows:
8:
i > T2 then
9:
x(a(#;--1 = -
Fo(d=-1ta) c'c Po Ala=z
else
11:
T(al = (11-A - Ld'en Pol Al Ala=z2
Fa(
end if
13: else
Assign it Z4.
15:
end if
end for
17:
Output: Edited sequence a.
Table 5: Performance of GFNSeqEditor for sequence reduction on AMP dataset in terms of variation in property, edit percentage of long sequences (EPLS), edit percentage of short sequences (EPSS), and percentage of length reduction in the long sequences.
Input Property Output Property EPLS(%) EPSS(%) Sequence Reduction
0.65
0.67
35.96
44.65
63.23
graph explainability [29]. Recognizing its significance, several studies have emerged to enhance the learning efficiency of GFlowNets 66
30 43) since the introduction of the flow matching learning objective by [5]. Moreover, GelowNets have demonstrated adaptability in being jointly trained with energy and reward functions [57]
32
38 introduce intrinsic exploration rewards into GFlowNets, addressing exploration challenges within sparse reward tasks. A couple of recent studies try to extend GFlowNets to stochastic environments, accommodating stochasticity in transition dynamics (39) and rewards [58] Several novel GFlowNet training methodologies have been recently proposed in accessible is explored in g Distributed training of GFlowNets is discussed in [45]. Accelerating GFlow
22
26
37
14] The application of GFlowNets when predefined reward function
training investigated
(25). Moreover, (27) employs GFlowNets for designing DNA-encoded
libraries. To reduce the need for expensive reward evaluations, 24 proposes new GFlowNet-basee method for molecular optimization.
The aforementioned works have primarily focused on theoretical developments of GFlowNet and its application in molecular generation, without directly addressing the challenges associated with sequence design or editing. In a departure from this trend, and inspired by Bayesian Optimization, active learning algorithm based on GFlow/Nets, i.e. GFlowWNetALL design novel biological sequences. GFlowNer-AL. 19 utilizes the epistemic uncertainty of the surrogate model within its reward function, guiding the GelowNet towards the optimization of promising yet less-cxplored regions within the state space. This approach fosters the generation of a diverse set of de novo sequences from scratch and bo-keen- Unlike GFNSegEditor, it lacks the capability to edit input seed sequences and combine multiple sequences. This distinction underscores the unique contribution of GFNSeqEditor addressing the sequence editing problem, positioning it valuable addition to the existing literature on GeLowNet.
19 proposed
G Societal Impact
Biological sequence optimization and design hold transformative potential for biotechnology and health, offering enhanced therapeutic solutions and vast range of applications. Techniques that
21

---

Table 6: Performance of GFNteqEditor for sequence combination over AMP dataset in terms of property improvements of first (PI-S1) and second (PI-S2) sequences, edit percentages of first (EP-S1) and second (EP-S2) sequences, and diversity.
Seqi AMP
Seq₂
PI-S|
PI-S2
EP-S (%)
EP)S2(%)
Diversity
AMP
0.05
0.06
36.10
39.47
7.29
non-AMP
AMP
0.41
0.04
41.41
65.39
12.77
enable refining sequences can lead to advancements like elucidating the role of individual gene products, rectifying genetic mutations in afflicted tissues, and optimizing properties of peptides, antibodies, and nucleic-acid therapeutics. However, the dual-edged nature of such breakthroughs must be acknowledged, as the same research might be misappropriated for unintended purposes. Our method can be instrumental in refining diagnostic procedures and uncovering the genetic basis of diseases, which promises a deeper grasp of genetic factors in diseases. Yet, we must approach with caution, as these advancements may unintentionally amplify health disparities for marginalized communities. As researchers, we emphasize the significance of weighing the potential societal benefits against unintended consequences while remaining optimistic about our work's 's predominant inclination towards beneficial outcomes.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction explain that the paper study biological sequence editing using GelowNet. The abstract and introduction summarize contributions and main results of the paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
In conclusion Section 6 and Appendix G the paper discusses its limitations.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [Yes]
Justification: The paper provides the full set assumptions and complete, correct proofs for all theoretical results. Each theorem is proven, with the proofs detailed in the appendices. The main text appropriately references these sections, ensuring that readers can easily locate and verify the proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results. the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Section 5 and Appendix E. provide details about implementation and training of baselines and the proposed algorithm.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
24

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The data is already public, stated in the experiments. We will release the code as well upon company approval.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ http:/.cc/pubbbcguuuceeee Codeeubmii ssi crrionPPoiccy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides detailed information about all the training and test details in Section 5 Appendix E.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper presents Figures and 6 which show the distribution of re- sults generated by the proposed algorithm, illustrating the statistical significance of the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eπτor rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
we specify that all training and inferences, including GFNSe- qEditor, have been conducted using single Nvidia Quadro P6000.
Justification: In Appendix E.I
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We believe that the research conducted in this paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., - if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The societal impact discussion is provided in Appendix G
Guidelines:
The answer NA means that there is no societal impact of the work performed.
26

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We believe that the paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
properly cited the original papers that produced the code packages or datasets we used.
Justification: In Section 5 and Appendix E.1
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
27

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
28

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "mFd2l0Xuf2", "reviewer": "Reviewer_6Uc5", "summary": "They propose a new sequence editing method using GFlowNets as priors and suggest additional hyperparameters to tune suboptimal gaps, randomness, and penalization. They theoretically analyze how these new hyperparameters can effectively control the lower and upper bounds of the number of edits. The performance results demonstrate some effectiveness of the proposed method.", "questions": "1. There are many hyperparameters, and biological tasks often involve expensive oracle functions. How can we tune these hyperparameters in real-world applications?\n\n2. Is there any tendency for the proposed method to work better on large sequences? Please provide insights on this.", "limitations": "They address their limitations in the conclusion.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "This approach introduces a new sequence editing method by leveraging the pretrained generative model, GFlowNets, as a prior. I appreciate their motivation for DNA editing and the clear narrative they present. Their theoretical analysis is robust and effectively explains the new hyperparameters.", "weaknesses": "**The baselines used in the study are too weak:** I don't think this method offers significant advantages over conditional GFlowNets, which can translate input sequences to output sequences while maintaining constraints on sequence distance. There are many other methods capable of achieving this, including Seq2Seq, which the authors used as a baseline. The Seq2Seq implementation seems overly simplistic; there are potentially better techniques for DNA editing. For example, hierarchical variational inference in latent space, combined with a well-designed reward model, could optimize the process more effectively. Any thoughts on this?\n\n**This is not a GFlowNet with multiple backward paths:** It appears that GFlowNets are used for sequence generation in an autoregressive manner (one-way generation) where \\( P_B = 1 \\). This approach is equivalent to Soft-Q-learning and path consistency learning (PCL) [1], so relevant soft RL literature should be clearly explained. The paper's title is somewhat misleading (though it need not be changed) as it essentially describes using a PCL-trained agent to assist in sequence editing. Reviewers suggest that the authors acknowledge this literature.\n\n**The literature review for GFlowNets should include works published in 2024:** Reviewers also expect the authors to include recent GFlowNets literature targeting biological or chemical applications, such as those published in ICLR and ICML 2024. Additionally, there are preprints discussing improvements and evolutionary methods (e.g., genetic algorithms) using GFlowNets; engaging with this existing literature would be beneficial. The current literature review seems not very up-to-date in 2024.\n\n[1] Nachum, Ofir, et al. \"Bridging the gap between value and policy based reinforcement learning.\" Advances in neural information processing systems 30 (2017).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our gratitude for taking the time to review our paper and letting us know your thoughtful comments. Please find below our responses to your comments.\n\n## Conditional GFlowNets\nConditional GFlowNets can be used for sequence editing by training the flow function with a sequence distance constraint. However, implementing this approach for sequence editing is not scalable and can be computationally intractable. The distance constraint depends on the seed sequence, requiring a separate flow function for each seed sequence to obtain diverse edits. This approach encounters several challenges:\n- Training GFlowNets for each seed sequence is infeasible for large datasets.\n- This approach requires evaluating the properties of unseen sequences using a proxy model, which can lead to inaccurate predictions for out-of-distribution sequences, resulting in poorly trained flow functions.\n- It may reduce the generalizability of the flow function, with the trained flow functions potentially containing only local information.\n\nMoreover, it is uncertain whether such an approach can provide theoretical guarantees on the amount of edits, similar to those provided by the proposed GFNSeqEditor.\n\nIn contrast, using GFNSeqEditor, one can train one flow function on offline data. Then for sequence editing, GFNSeqEditor only makes inferences with the flow function. This makes GFNSeqEditor computationally efficient. Moreover, the paper theoretically proves that employing GFNSeqEditor, the amount of edits can be controlled using three hyperparameters.\n\n## Variational Inference\nIn order to compare GFNSeqEditor with variational inference based methods, we include LaMBO [32] as a baseline. LaMBO maps sequences into a latent space and uses Bayesian optimization in this space to generate sequences with relatively higher properties. We find controlling the amount of edits challenging using LaMBO. To sum up, we believe that performing sequence editing with variational inference based methods can be an interesting future research direction as it requires careful and intricate algorithm design.\n\n## Soft-Q-Learning and PCL Literature Review\nWe will acknowledge the relevant literature on Soft-Q-Learning and path consistency learning (PCL). We will add that “Generating sequences in an autoregressive fashion using GFlowNet involves only one path to generate a particular sequence. In such cases, generating biological sequences with GFlowNet can be viewed as a Soft-Q-Learning [R1, R2, R3] and path consistency learning (PCL) [R4] problem.”\n\n## GFlowNets Literature Review\nWe will expand our literature review to include more recent studies on GFlowNets. We will add notes on references [R5]--[R13]. Several novel GFlowNet training methodologies are proposed in [R5, R7, R10, R12]. The application of GFlowNets when a predefined reward function is not accessible is explored in [R6]. Distributed training of GFlowNets is discussed in [R8]. Accelerating GFlowNet training is investigated in [R9]. Moreover, [R11] employs GFlowNets for designing DNA-encoded libraries. To reduce the need for expensive reward evaluations, [R13] proposes a new GFlowNet-based method for molecular optimization.\n\n## Responses to Questions\n**Response to Question 1**: The theoretical bounds obtained in Section 4.3 can be used to determine a range for hyperparameters. Subsequently, a few experiments can be conducted within this range to identify the optimal hyperparameter values.\n\n**Response to Question 2**: The intuition behind GFNSeqEditor’s superior performance with larger sequences is its utilization of a flow function which is trained to capture global information about the sequence space. In contrast, the performance of local search-based baselines decrease as sequence length increases. This is because, as the search space expands, it becomes more challenging for local search methods to find sequences with optimal performance.\n\n## References\n[R1] T. Haarnoja, H. Tang, P. Abbeel and S. Levine, “Reinforcement Learning with Deep Energy-Based Policies,” ICML 2017.\n\n[R2] J. Grau-Moya, F. Leibfried and P. Vrancx, “Soft Q-Learning with Mutual-Information Regularization,” ICLR 2019.\n\n[R3] S. Mohammadpour, E. Bengio, E. Frejinger and P. Bacon “Maximum entropy GFlowNets with soft Q-learning,” AISTATS 2024.\n\n[R4] O. Nachum, M. Norouzi, K. Xu and D. Schuurmans “Bridging the Gap Between Value and Policy Based Reinforcement Learning,” NeurIPS 2017.\n\n[R5] H. Jang, M. Kim, S. Ahn, “Learning Energy Decompositions for Partial Inference in GFlowNets,” ICLR 2024.\n\n[R6] Y. Chen and L. Mauch, “Order-Preserving GFlowNets,” ICLR, 2024.\n\n[R7] M. Kim, T. Yun, E. Bengio, D. Zhang, Y. Bengio, S. Ahn and J. Park, “Local Search GFlowNets,” ICLR, 2024.\n\n[R8] T. Silva, L. M. Carvalho, A. H. Souza, S. Kaski and D. Mesquita, “Embarrassingly Parallel GFlowNets,” ICML, 2024.\n\n[R9] M. Kim, J. Ko, T. Yun, D.i Zhang, L. Pan, W. C. Kim, J. Park, E. Bengio and Y. Bengio “Learning to Scale Logits for Temperature-Conditional GFlowNets,” ICML, 2024.\n\n[R10] P. Niu, S. Wu, M. Fan and X. Qian, “GFlowNet Training by Policy Gradients,” ICML, 2024.\n\n[R11] M. Koziarski, M. Abukalam, V. Shah, L. Vaillancourt, D. A. Schuetz, M. Jain, A. van der Sloot, M. Bourgey, A. Marinier and Y. Bengio, “Towards DNA-Encoded Library Generation with GFlowNets,” ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design, 2024.\n\n[R12] S. Guo, J. Chu, L. Zhu, Z. Li and T. Li, “Dynamic Backtracking in GFlowNets: Enhancing Decision Steps with Reward-Dependent Adjustment Mechanisms,” Arxiv, 2024.\n\n[R13] H. Kim, M. Kim, S. Choi and J. Park, “Genetic-guided GFlowNets for Sample Efficient Molecular Optimization,” Arxiv, 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pyaEMsxwe5", "reviewer": "Reviewer_a1PN", "summary": "This paper introduced a new algorithm for biological sequence editing, GFNSeqEditor. This algorithm is designed based on pre-trained Generative Flow Networks (GFNs), and improves target properties by identifying and editing sub-optimal sites of input sequences. Through theoretical analysis and experiments on three datasets, this new algorithm shows that it can improve biological properties with diversified edit while minimize the number of edits to ensure safety and predictability. By comparing with a few baseline models, the GFNSeqEditor outperforms the state-of-the-art methods on property improvements, diversity, and edit percentage.", "questions": "1. I would suggest refining the statement on fewer edits leads to safe and predictable sequence editing. Follow up analysis on how sequence editing influence structure and function change would be great to have, but not necessary. \n2. Please consider add more details on how the dataset select non-AMP samples, and briefly mention about the negative sample problem for clarify. \n3. It would be more clear to include statistical analysis or dig into possible reasons and future improvements would help the audience to understand the significance of this work and looking for future improvements.", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper includes comprehensive theoretical analysis of the algorithm, such as bounds on expected reward, property improvements, and the number of edits. Together with extensive experiments, this paper provides sufficient information on the influence of hyperparameter choices on algorithm performance, which offers valuable insights for downstream studies. \nThis paper shows the versatility of GFNSeqEditor through examples of sequence generation and sequence length reduction, indicating that this algorithm can not only be used on sequence editing but also in a broader potential applications in synthetic biology.", "weaknesses": "1. the algorithm was built on the assumption that fewer edits lead to safer and more predictable modification. I would argue this claim is valid under the condition that, compared with generating a new sequence, limited editing is possibly a safer choice. Many studies on single nucleotide variance (SNV) show that a single nucleotide change can lead to functional change or property loss. To avoid unexpected confusion, it would be better to either include followup analysis on structural changes caused by sequence editing, which I believe will be too much work to do, or improve the statement on the safety claim. \n2. Lack of details about the selection of non-AMP samples. Previous study [1] have shown that inappropriate selection of negative samples can introduce bias. Clarification on the criteria of selecting negative samples would help the audience better evaluate the validity of the results.  \n3. While GFNSeqEditor shows improvements compared with the baselines, the performance improvements are not substantial, especially on the aspect of editing percentage. Any possible reasons or statistical analysis on the significance?\n\n[1]. Sidorczuk K, Gagat P, Pietluch F, et al. Benchmarks in antimicrobial peptide prediction are biased due to the selection of negative data[J]. Briefings in Bioinformatics, 2022, 23(5): bbac343.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for taking time to review our paper and let us know your valuable comments. Please find below our responses to your comments and questions.\n\n## Safety Assumption\nIt is generally expected that fewer modifications in biological sequences are less likely to result in significant functional changes or property loss. However, we acknowledge that even limited modifications can lead to functional changes in the edited biological sequence. We agree that the safety assumption has its uncertainties and that investigating structural changes due to sequence editing is beyond the scope of this paper. Therefore, we will revise the safety statement in the paper to reflect this uncertainty.\n\n## Selection of non-AMP Samples\nFor the AMP dataset, we utilized predictive models and data splits from published works [1,14, 27]. To address the reviewer's concern, we will provide additional details about the data sampling process for the AMP dataset in Appendices E.1.1 and E.1.2, using the explanations provided in [1,14, 27]. Furthermore, we will highlight the importance of negative data selection in the performance of predictive models, as discussed in the study by Sidorczuk et al. [R1], suggested by the respected reviewer.\n\n## Statistical Analysis of Results\nIn our experiments, we observed that the property improvements provided by GFNSeqEditor and other baselines depend on the edit percentage. To ensure fair comparisons, we fixed the edit percentage across all algorithms to a similar level whenever possible. This is reflected in Table 1, where the edit percentages for virtually all algorithms are quite similar. According to Table 1, the property improvements provided by GFNSeqEditor are significant for the AMP and CRE datasets compared to other baselines. Furthermore, Figure 2 demonstrates the property improvements of GFNSeqEditor and other baselines as the edit percentage changes. To provide a statistical analysis of the results, Figure 6 in Appendix E.3 illustrates the distribution of properties of edited sequences. We are open to conducting additional statistical analyses based on your suggestions.\n\n## Reference\n[R1] Sidorczuk K, Gagat P, Pietluch F, et al. Benchmarks in antimicrobial peptide prediction are biased due to the selection of negative data[J]. Briefings in Bioinformatics, 2022, 23(5).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LRJSXYgktw", "reviewer": "Reviewer_Ns1N", "summary": "The author proposed GFNSeqEditor, a novel sequence editing and generation model built on GFlowNet, which provides different modifications for each sequence to enhance desired features. Several experiments have demonstrated the performance of the proposed algorithm.", "questions": "1. Existing biological sequence design methods, such as evolutionary approaches, can still perform sequence editing in specified regions. Examples include AdaLead[1], PEX[2], Coms[3], and BiB[4]. The author must compare their method with these state-of-the-art algorithms to demonstrate its advantages. \n\n\n2. The author used AMP and CRE datasets and trained Oracles independently. Evaluating with self-trained models can lead to inconsistent standards, as the author can modify their trained Oracles at any time to demonstrate the algorithm's superiority. Unlike Coms[3] or BiB[4], which use numerous sequence evaluation problems, this leads to unfair experimental evaluations. It is recommended that the author evaluate under existing various evaluation standards to demonstrate the model's advantages better.\n\n\n[1] Sinai, Sam, et al. \"AdaLead: A simple and robust adaptive greedy search algorithm for sequence design.\" arXiv preprint arXiv:2010.02141 (2020). \n\n[2] Ren, Zhizhou, et al. \"Proximal exploration for model-guided protein sequence design.\" International Conference on Machine Learning. PMLR, 2022.\n\n[3]  Trabucco, Brandon, et al. \"Conservative objective models for effective offline model-based optimization.\" International Conference on Machine Learning. PMLR, 2021.\n\n[4] Chen, Can, et al. \"Bidirectional learning for offline model-based biological sequence design.\" International Conference on Machine Learning. PMLR, 2023.", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "A new biological sequence editing method based on GFlowNet has been proposed, capable of identifying and editing positions in a given sequence. It has been demonstrated that the lower and upper bounds on the number of edits performed by GFNSeqEditor can be controlled by adjusting hyperparameters.", "weaknesses": "There are shortcomings in the model comparison, as it does not compare with existing biological sequence design methods. In terms of sequence evaluation, mainstream evaluation experiments were not used, leading to doubts about the model's effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your review and letting us know your valuable comments. Please find below our responses to your comments.\n\n## Evolutionary-Based Methods\nWe would like to clarify that we have already included the evolutionary method in AdaLead (reference [30] in the paper) among our baselines. As indicated on line 265 of page 7, the DE baseline refers to the evolutionary method presented in [30]. However, instead of naming this baseline as AdaLead, we referred to it as DE, which stands for Directed Evolution. To avoid any confusion, we can revise the name of the DE baseline to AdaLead.\n\nPEX is an evolutionary method that operates through multiple rounds of interactions with the lab. However, these wet lab evaluations can be costly and time-consuming. The focus of this paper is to propose edits without the need for any wet lab interactions. Therefore, we compared the performance of GFNSeqEditor with other baselines that do not have such need. Indeed, if we run PEX only in one iteration without any additional wet lab experiments, the PEX result would be similar to the DE reported in the paper. To address the reviewer's concern, we will add some notes about PEX in our literature review.\n\n## Model-Based Optimization Methods\nBoth Coms and BiB are model-based optimization (MBO) methods. To generate sequences, they perform several rounds of optimization on a set of sequences. However, in biological sequence editing, we often aim to generate sequences similar to a pre-specified seed sequence. Using MBO-based methods for this purpose requires performing optimization on each seed sequence, which makes computations infeasible, especially when it comes to editing thousands of sequences. Moreover, adapting Coms and BiB for sequence editing purposes may require some changes in these algorithms. The advantages of GFNSeqEditor over MBO-based methods are summarized as follows:\n1. To edit a sequence, GFNSeqEditor employs a pre-trained flow function. GFNSeqEditor only performs inference using the flow function and employing GFNSeqEditor does not involve any model training. This makes the GFNSeqEditor computationally efficient.\n2. Our paper theoretically proves that using GFNSeqEditor the amount of edits can be controlled while MBO based methods cannot provide such guarantee.\n3. MBO-based approaches require evaluating the properties of unseen sequences using a proxy model, whereas GFlowNet-based approaches can operate without this. Proxy models may provide misleading predictions for out-of-distribution sequences.\n\nFurthermore, we have already compared GFNSeqEditor with Ledidi, an optimization-based baseline specifically designed for biological sequence editing. To address your concern, we can include background information on BiB in our literature review.\n\n## Evaluations\n**We did not train oracles for AMP and CRE datasets and the oracles are not self-trained**. For AMP, we employed the oracles used by [14] which can be downloaded from Github repo https://github.com/MJ10/BioSeq-GFN-AL. For CRE, we used the Malinois model [10] which can be obtained from GitHub repo https://github.com/sjgosai/boda2. Therefore, we believe that our comparisons are fair. We clarify this in Appendix E.1.2 on page 16, where we provide detailed information about the oracles used in this paper. Furthermore, we performed experiments on both DNA and protein sequence datasets. Finally, following previous works (see e.g., [10], [14], [23]) and addressing the specific needs of this study, we evaluated the performance of the algorithms based on several important metrics in the biological sequence domain, including property improvement, diversity, and edit percentage.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a novel biological sequence editing algorithm, GFNSeqEditor, based on pretrained Generative Flow Networks (GFNs) with theoretical analysis on expected reward, property improvements, and the number of edits. It identifies and edits sub-optimal sites to enhance the desired property.

Overall, the reviews highlight its strengths of GFNSeqEditor along with theoretical anlyses to understand how it works as well as solid benchmarking evaluations. While the reviewers also raised several potential concerns, most were effectively addressed in the detailed author feedback.

Based on these evaluation, I would like to recommend accepting this submission. We believe this interesting work will attract the interest of the NeurIPS community.
Decision: Accept (poster)