GFlowNet Assisted Biological Sequence Editing

Authors: pouya m ghari
alex m tseng
genentech  gökcen
eraslan  genentech
tommaso biancalani genentech
gabriele  scalia
genentech  ehsan
hajiramezanali † genentech
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 15:10:27

Abstract: Editing biological sequences has extensive applications in synthetic biology and medicine, such as designing regulatory elements for nucleic-acid therapeutics and treating genetic disorders. The primary objective in biological-sequence editing is to determine the optimal modifications to a sequence which augment certain biological properties while adhering to a minimal number of alterations to ensure predictability and potentially support safety. In this paper, we propose GFNSeqEditor, a novel biological-sequence editing algorithm which builds on the recently proposed area of generative flow networks (GFlowNets). Our proposed GFNSeqEditor identifies elements within a starting seed sequence that may compromise a desired biological property. Then, using a learned stochastic policy, the algorithm makes edits at these identified locations, offering diverse modifications for each sequence to enhance the desired property. The number of edits can be regulated through specific hyperparameters. We conducted extensive experiments on a range of real-world datasets and biological applications, and our results underscore the superior performance of our proposed algorithm compared to existing state-of-the-art sequence editing methods.

Main paper: 
 introduction: 
 Editing biological sequences has a multitude of applications in biology, medicine, and biotechnology. For instance, gene editing serves as a tool to elucidate the role of individual gene products in diseases [28] and offers the potential to rectify genetic mutations in afflicted tissues and cells for therapeutic interventions [9] . The primary objective in biological-sequence editing is to enhance specific biological attributes of a starting seed sequence, while minimizing the number of edits. This reduction in the number of alterations not only has the potential to improve safety but also facilitates the predictability and precision of modification outcomes.Existing methodologies that leverage generative modeling in the context of biological sequences have predominantly concentrated on de novo generation of sequences with desired properties [52, 61, 3] . A common feature of these approaches is generating entirely new sequences from scratch. As a result, there is an inherent risk of deviating significantly from naturally occurring sequences, compromising safety (e.g., the risk of designing sequences that might trigger an immune response) and predictability (e.g., obtaining misleading predictions from models that are trained on genomic sequences due to out-of-distribution effects). Despite the paramount importance of editing biological sequences, there has been a noticeable scarcity of research leveraging generative modeling to address this aspect specifically.Generative flow networks (GFlowNets) [5, 6] , a generative approach recognized for their ability to sequentially generate new objects, have shown remarkable performance in generating novel biological sequences from scratch [19, 30] . Drawing inspiration from the emerging field of GFlowNets, this paper introduces a novel biological-sequence editing algorithm: GFNSeqEditor. GFNSeqEditor assesses the potential for significant property enhancement within a given seed sequence by iteratively identifying and subsequently editing specific positions in the input sequence. More precisely, using the trained flow function, GFNSeqEditor first identifies positions in the seed sequence that require editing. Then, it constructs a stochastic policy using the flow function to select a substitution from the available options for the identified positions. Our stochastic approach empowers GFNSeqEditor to generate a diverse set of edited sequences for each input sequence, which, due to the diverse nature of biological targets, is an important consideration in biological sequence design [34, 19] .In summary, this paper makes the following contributions:• We introduce GFNSeqEditor, a novel sequence-editing method which identifies and edits positions within a given sequence. GFNSeqEditor generates diverse edits for each input sequence based on a stochastic policy.• We theoretically analyze the properties of the sequences edited through GFNSeqEditor, deriving lower and upper bounds on the property of edited sequences. Additionally, we demonstrate that the lower and upper bounds for the number of edits performed by GFNSe-qEditor can be controlled through the adjustment of hyperparameters (Subsection 4.3).• We conduct experiments across various DNA and protein sequence editing tasks, showcasing GFNSeqEditor's remarkable efficiency in enhancing properties with a reduced number of edits when compared to existing state-of-the-art methods. (Subsection 5.1).• We highlight the versatility of GFNSeqEditor, which can be employed not only for sequence editing but also alongside biological-sequence generation models to produce novel sequences with improved properties and increased diversity (Subsection 5.2).• We demonstrate the usage of GFNSeqEditor for sequence length reduction, allowing the creation of new, relatively shorter sequences by combining pairs of long and short sequences (Subsection 5.3).Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The abstract and introduction explain that the paper study biological sequence editing using GFlowNet. The abstract and introduction summarize contributions and main results of the paper.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related works: 
 De Novo Sequence Design. The generation of biological sequences has been tackled using a diverse range of methods, including reinforcement learning [1] , Bayesian optimization [51] , deep generative models for search and sampling [18] , generative adversarial networks [61] , diffusion models [3] , model-based optimization approaches [52, 7] , adaptive evolutionary strategies [16, 49] , likelihood-free inference [55] , and surrogate-based black-box optimization [10] , and GFlowNet [19] . It is important to note that all these sequence-generation methods generate sequences from scratch. However, ab initio generation carries the risk of deviating too significantly from naturally occurring sequences, which can compromise safety and predictability. In contrast, our proposed method enhances a target property while maintaining the similarity to seed sequences (e.g., naturally occurring sequences), thus improving predictability and potentially enhancing safety.Sequence Editing. Traditional approaches commonly employed for biological sequence editing are evolution-based methods, where-over many iterations-a starting "seed" sequence is randomly mutated, retaining only the best sequences (i.e., highest desired property) for the next round [2, 46, 50, 41] . These approaches have several important limitations. First, they require the evaluation of numerous candidate sequences at every iteration. This computational demand can become prohibitively expensive, particularly for lengthy sequences. Additionally, evolution-based methods heavily rely on evaluations provided by a proxy model capable of assessing the properties of unseen sequences; the efficacy of these methods is thus limited by the reliability of the underlying proxy. Moreover, these methods may require repeated rounds of interactions with the lab [41] , which can be costly and time-consuming.DNA sequence x with property yDNA sequence x with property ŷ Figure 1 : An example of editing the DNA sequence 'ATGTCCGC'. The goal is to make a limited number of edits to maximize the property ŷ. Each token in the sequence in this example is called a base and can be any of ['A', 'C', 'T', 'G']. The editor function E accepts the initial sequence as an input and determines that the second and seventh bases require editing (highlighted in red). Then, E modifies the bases at these identified locations to improve the property value.Beyond evolution-based methods, a handful of optimization-based methods have been proposed by [42, 48, 21] . By treating sequence editing as an optimization task, Ledidi [42] learns to perturb specific positions within a given sequence. Utilizing Bayesian optimization, LaMBO [48] generates new sequences by optimizing a batch of starting seed sequences. Building upon the LaMBO framework, MOGFN-AL [21] leverages GFlowNets to generate candidates in each round of Bayesian optimization loop, improving computational efficiency compared to LaMBO. Akin to evolution-based models, these optimization-based methods require the evaluation of unseen sequences. Consequently, their effectiveness is contingent on the quality of the proxy model, which can compromise their performance if the proxy model lacks sufficient generalizability for unseen sequences. Furthermore, both evolution-based and optimization-based methods perform local searches given either a single seed sequence or a batch of seed sequences. Thus, these methods face issues related to low sample efficiency. In contrast, GFNSeqEditor relies on a pre-trained flow function that amortizes the search cost over the learning process, allocating probability mass across the entire space to facilitate exploration and diversity. Furthermore, GFNSeqEditor can be employed for editing without necessitating the evaluation of unseen sequence properties. Theoretical analysis presented in this paper establishes that the bounds of edited sequence rewards, property improvement, and the number of edits can be effectively regulated through GFNSeqEditor hyperparameters. Therefore, GFNSeqEditor offers increased reliability and operational suitability in comparison to counterparts lacking a robust theoretical analysis.We provide an extensive overview of the related literature, with additional discussion available in the Appendix F.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: In conclusion Section 6 and Appendix G, the paper discusses its limitations.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries and problem statement: 
 Let x be a biological sequence with property y. For example, x may be a DNA sequence, and y may be the likelihood it binds to a particular protein of interest. The present paper considers the problem of searching for edits in x to improve y. To this end, the goal is to learn an editor function E(•) which accepts a sequence x and outputs the edited sequence E(x) = x with property ŷ. The editor function E(•) should maximize ŷ, while at the same time minimizing the number of edits between x and x. To achieve this goal, we propose GFNSeqEditor. GFNSeqEditor first identifies positions in a given biological sequence such that editing those positions leads to considerable improvement in the property of the sequence. Then, the learned editor function E edits these identified locations (Figure 1 ). GFNSeqEditor uses a trained GFlowNet [5, 6] to identify positions that require editing and subsequently generate edits for those positions. The following Subsections present preliminaries on GFlowNets.Generative Flow Networks (GFlowNets) [5, 6] learn a stochastic policy π(•) to sequentially construct a discrete object x. Let X be the space of discrete objects x. It is assumed that the space X is compositional, meaning that an object x can be constructed using a sequence of actions taken from an action set A. At each step t, given a partially constructed object s t , GFlowNet samples an action a t+1 from the set A using the stochastic policy π(•|s t ). Then, GFlowNet appends a t+1 to s t to obtain s t+1 . In this context, s t can be viewed as the state at step t. The above procedure continues until reaching a terminating state, which yields the fully constructed object x. To construct an object x, the GFlowNet starts from an initial empty state s 0 , and applying actions sequentially, all fully constructed objects must end in a special final state s f . Therefore, the trajectory of states to construct an object x can be written asτ x = (s 0 → s 1 → • • • → x → s f ).Let T be the set of all possible trajectories. Furthermore, let R(•) : X → R + be a non-negative reward function defined on X . The goal of GFlowNet is to learn a stochastic policy π(•) such that π(x) ∝ R(x). This means that the GFlowNet learns a stochastic policy π(•) to generate an object x with a probability proportional to its reward.As described later, to obtain the policy π(•), the GFlowNet uses trajectory flow F : T → R + . The trajectory flow F (τ ) assigns a probability mass to the trajectory τ . Then, the edge flow from state s to state s ′ is defined as F (s → s ′ ) = ∀τ :s→s ′ ∈τ F (τ ). Moreover, the state flow is defined as F (s) = ∀τ :s∈τ F (τ ). The trajectory flow F (•) induces a probability measure P F (•) over completed trajectories that can be expressed as P F (τ ) = F (τ )Z where Z = ∀τ ∈T F (τ ) represents the total flow. The probability of visiting state s can be written as P F (s) = ∀τ ∈T:s∈τ F (τ ). Then, the forward transition probability from state s to state s ′ can be obtained asP F (s ′ |s) = F (s→s ′ )F (s) . The trajectory flow F (•) is called a consistent flow if for any state s it satisfies ∀s ′ :s ′ →s F (s ′ → s) = ∀s ′′ :s→s ′′ F (s → s ′′ ), which constitutes that the in-flow and out-flow of state s are equal. [5] shows that if F (•) is a consistent flow such that the terminal flow is set as reward (i.e.F (x → s f ) = R(x)), the policy π(•) defined as π(s ′ |s) = P F (s ′ |s) satisfies π(x) = R(x)Z which means that the policy π(•) samples an object x proportional to its reward.In order to learn the policy π(•), a GFlowNet model approximates trajectory flow with a flow function F θ (•) where θ includes learnable parameters of the flow function. To learn the flow function that can provide consistency condition, [5] formulates flow-matching loss function as follows:EQUATIONMoreover, as an alternative objective function, [31] introduces trajectory balance as:EQUATIONwhere Z θ is a learnable parameter. The trajectory-balance objective function in (2) can accelerate training GFlowNets and provide robustness to long trajectories. Given a training dataset, optimization techniques such as stochastic gradient descent can be applied to objective functions in (1) and (2) to train the GFlowNet model. We use trajectory balance in this paper due to its well-documented performance. Furthermore, it is worth noting that generating sequences in an autoregressive fashion using GFlowNet involves only one path to generate a particular sequence. In such cases, generating biological sequences with GFlowNet can be viewed as a Soft-Q-Learning [15, 13, 33] and path consistency learning (PCL) [35] problem.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: The paper provides the full set of assumptions and complete, correct proofs for all theoretical results. Each theorem is proven, with the proofs detailed in the appendices.The main text appropriately references these sections, ensuring that readers can easily locate and verify the proofs. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
sequence editing with gflownet: 
 To edit a given sequence x, we propose identifying sub-optimal positions of x such that editing them can lead to considerable improvement in the sequence property. Assume that the flow function F θ (•) is trained on available offline training data. GFNSeqEditor uses the trained GFlowNet's flow function F θ (•) to identify sub-optimal positions of x, and subsequently replace the sub-optimal parts with newly sampled edits based on the stochastic policy π(•).This Subsection provides intuition on how GFNSeqEditor uses a pre-trained flow function F θ (•) to identify sub-optimal positions in a sequence x to edit. Let x t and x :t denote the t-th element and the first t elements in the sequence x, respectively. For example, in the DNA sequence x = 'ATGTCCGC', we have x 2 = 'T' and x :2 = 'AT'. GFNSeqEditor constructs edited sequences token by token, and for each position t + 1, it examines whether x t+1 should be edited or not. Using the flow function F θ (•), given x :t , GFlowNet evaluates the average reward obtained by appending any possible token to x :t . In this context, each token can be viewed as an action. Let x :t + a denotes the expanded x :t by appending token a. Let A represent the available action set. For each a ∈ A, using the state flow F θ (x :t + a), the value of action a given x :t can be evaluated. As discussed in Section 3, the state flow F θ (x :t + a) is proportional to the total reward of all possible sequences that have x :t + a as their prefix. Therefore, F θ (x :t + a 1 ) > F θ (x :t + a 2 ) indicates that taking action a 1 instead of action a 2 can lead to obtaining better candidates for the final sequence. We can leverage this property of the flow function F θ (•) to examine if x t+1 is sub-optimal or not. If the reward resulting from having x t+1 in the seed sequence is evaluated by F θ (•) to be relatively small compared to other possible actions, then x t+1 is considered sub-optimal. In particular, x t+1 is identified as sub-optimal if:EQUATIONwhere 0 ≤ δ ≤ 1 is a hyperparameter. A larger δ increases the likelihood that the algorithm identifies x t+1 as sub-optimal. From (3), it can be inferred that x t+1 is identified as sub-optimal if its associated out-flow is considerably smaller than the out-flow associated with the best possible action in A. This means that the flow function F θ (•) suggests that replacing x t+1 with other actions can lead to remarkable improvement in the sequence property.Using the flow function F θ (•), GFNSeqEditor iteratively identifies and edits positions in a seed sequence. Subsection 4.1 presented a simple function for determining if a position x t+1 in a sequence should be edited to improve the target property value ((3)). Based on this intuition, we now modify (3) to formally define the sub-optimal-position identification function D(•) used by GFNSeqEditor.Let x:t denote the first t elements of the edited sequence. Assume that x t ∈ A, ∀t, meaning that x t is always in the available actions. At each step t of the algorithm, D(•) accepts x:t-1 and evaluates whether appending x t (from the seed sequence) to the edited partial sequence x:t-1 is detrimental to the performance. In particular, modifying (3), the sub-optimal identifier function D(•) checks the following condition:EQUATIONwhere ν ∼ N (0, σ 2 ) is a Gaussian random variable with variance of σ 2 . The variance σ 2 is a hyperparameter. The relation between σ and the algorithm performance will be analyzed in Section 4.3. The inclusion of additive noise ν on the right-hand side of (4) introduces a degree of randomness into the process of identifying sub-optimal positions. This, in turn, fosters exploration in the editing process. The sub-optimal-position-identifier function D(•) determines if x t is sub-optimal as follows:EQUATIONIf D(x t , x:t-1 ; δ, σ) = 0, at step t the algorithm appends x t from the original sequence x to x:t-1 . Otherwise, if D(x t , x:t-1 ; δ, σ) = 1, the algorithm samples an action a according to the following policy:EQUATIONwhere 0 ≤ λ < 1 is a regularization coefficient and 1 a=xt denotes the indicator function and is 1 if a = x t . The regularization parameter λ allows tuning the sampling process to favor the original sequence (a larger λ leads to a smaller number of edits). The policy in (6) constitutes a trade-off between increasing the target property and decreasing the distance between the edited sequence x and the original sequence x. Let xt be the action sampled by the policy π in (6) . In summary, the t-th element in the edited sequence can be written as:EQUATIONTherefore, at each step t, the edited sequence is updated as x:t = x:t-1 + xt . The process continues until step T , where T = |x| denotes the length of the original sequence x. Note that x:0 is an empty sequence. Algorithm 1 summarizes GFNSeqEditor.Algorithm 1 GFNSeqEditor: Sequence Editor using GFlowNet Check if x t is sub-optimal by obtaining D(x t , x:t-1 ; δ, σ) according to (5).if D( x:t-1 ; δ, σ) = 1 then 6:Sample xt according to policy π(•| x:t-1 ) in (6). Assign xt = x t . 9:end if 10: end for 11: Output: Edited sequence x.This Subsection analyzes the reward and properties of the edited sequence as well as the number of edits performed by GFNSeqEditor. Specifically, the bounds for the reward of edited sequences, property improvement and the number of edits are determined by the algorithm's hyperparameters σ, δ, and λ. The following theorem specifies the lower bound for the reward of edited sequences. Theorem 4.1. Let T be the length of the original sequence x. The expected reward of the sequence edited by GFNSeqEditor x given x is bounded from below as:EQUATIONwhere Φ(•) denotes the cumulative distribution function (CDF) for the normal distribution and R F,T represents the expected reward of a sequence with length T generated using the flow function F θ (•).Proof of Theorem 4.1 is deferred to Appendix A. The following Theorem obtains the expected property improvement upper bound of the proposed GFNSeqEditor. The property improvement of a sequence x is defined as PI = ŷ -y where ŷ denotes the edited sequence property. Theorem 4.2. Let S * x be the set of all sequences with length T which have larger properties than that of x (i.e., y). Assume that S *x is a non-empty set. The expected property improvement by applying GFNSeqEditor on x is bounded from above asEQUATIONwhere p w denote the property of the sequence w.The proof of Theorem 4.2 can be found in Appendix B. Theorems 4.1 and 4.2 demonstrate that an increase in δ results in an increase in both the lower bound of reward and the upper bound of property improvement. While a higher value of σ corresponds to larger lower bounds for the reward, an increase in σ diminishes the upper bound of the property improvement. The following theorem obtains the upper bound on the number of edits performed by the proposed GFNSeqEditor. Theorem 4.3. The expected distance between the edited sequence x by GFNSeqEditor and the original sequence x is bounded from above as:EQUATIONwhere lev(•, •) is the Levenshtein distance between two sequences.The proof for Theorem 4.3 is available in Appendix C. The following Theorem specifies the lower bound for the number of edits. Theorem 4.4. Let there exists ϵ > 0 such that the flow function F θ (•) satisfies:EQUATIONmeaning that the probability of choosing of each action is always less than 1 -ϵ. The expected distance between the edited sequence x by GFNSeqEditor and the original sequence x is bounded from below as:EQUATIONProof of Theorem 4.4 can be found in Appendix D. Theorems 4.3 and 4.4 show that as δ increases, both the lower and upper bounds of distance increase. In contrast, an increase in λ leads to a decrease in both the lower and upper bounds of distance. Furthermore, Theorem 4.1 demonstrates that a reduction in λ results in a larger lower bound for the reward. Therefore, Theorems 4.1 and 4.3 reveal a trade-off between the expected number of edits and the lower bound for the expected reward. While it is preferable to select hyperparameters δ and λ that reduce the expected number of edits, an increase in the number of edits corresponds to a larger lower bound for the reward.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: In Section 5 and Appendix E.1, we provide details about implementation and training of baselines and the proposed algorithm. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
experiments: 
 We conducted extensive experiments to assess the performance of GFNSeqEditor in comparison to several state-of-the-art baselines across diverse DNA-and protein-sequence editing tasks. We evaluate on TFbinding, AMP, and CRE datasets. TFbinding and CRE datasets consist DNA sequences with lengths of 8 and 200, respectively. The task in both datasets is to edit sequences to increase their binding activities. The vocabulary for both TFbinding and CRE is the four DNA bases, {A, C, G, T}. AMP dataset comprises positive samples, representing anti-microbial peptides (AMPs), and negative samples, which are non-AMPs. The vocabulary consists of 20 amino acids. The primary objective is to edit the non-AMP samples in such a way that the edited versions attain the characteristics exhibited by AMP samples. Additional information about the datasets can be found in Appendix E.1.1.To evaluate the performance of sequence editing methods, we compute the following metrics:• Property Improvement (PI): The PI for a given sequence x with label y is calculated as the average enhancement in property across edits, expressed as PI = 1 ne ne i=1 (ŷ i -y), where n e is the number of edited sequences associated with the original sequence x and ŷi denote the property of the i-th edited sequence xi . To evaluate the performance of editing methods, for each dataset we leverage an oracle to obtain ŷi given xi . More details about oracles can be found in Appendix E.• Edit Percentage (EP): The average Levenshtein distance between x and edited sequences normalized by the length of x expressed as 1 neT ne i=1 lev(x, xi ). • Diversity: For each sequence x, the diversity among edited sequences can be obtained as2 ne(ne-1) ne-1 i=1 ne j=i+1 lev( xi , xj ).• GMDPI: The geometric mean of diversity and PI is measured. This metric highlights algorithms that exhibit strong performance in both aspects simultaneously.We compared GFNSeqEditor to several baselines, including Directed Evolution (DE) [46] , Ledidi [42] , LaMBO [48] , MOGFN-AL [21] , GFlowNet-AL [19] , and Seq2Seq. To perform Directed Evolution for sequence editing, we select a set of positions uniformly at random within a given sequence and then apply the directed-evolution algorithm to edit these positions. The implementation of the directed-evolution algorithm is the same as that of the AdaLead framework in [46] . Inspired by graph-to-graph translation for molecular optimization in [23] , we implemented another editing baseline, which is called Seq2Seq. For the Seq2Seq baseline, we initially partition the dataset into two subsets: i) sequences with lower target-property values, and ii) sequences with relatively higher targetproperty values. Subsequently, we create pairs of data samples such that each low-property sequence is paired with its closest counterpart from the high-property sequence set, based on Levenshtein distance. A transformer is then trained to map each low-property sequence to its high-property pair.Essentially, the Seq2Seq baseline maps an input sequence to a similar sequence with a higher property value. Furthermore, we adapted GFlowNet-AL for sequence editing, and named it GFlowNet-E in what follows. In this baseline, the initial segment of the sequence serves as the input, allowing the model to generate the subsequent portion of the sequence. For TF-binding, AMP, and CRE datasets, GFlowNet-E takes in the initial 70%, 65%, and 60% of elements, respectively, from the Table 1 presents the performance of GFNSeqEditor and other baselines on TFbinding, AMP, and CRE datasets 3 . We set GFNSeqEditor and all the baselines except for Seq2Seq to create 10 edited sequences for each input sequence. The Seq2Seq implementation closely resembles a deterministic machine translator and is limited to producing just one edited sequence per input, resulting in a diversity score of zero. Additionally, Figure 2 shows the property improvement achieved by GFNSeqEditor, DE, Ledidi, LaMBO, and MOGFN-AL across a range of edit percentages. As evident from Table 1 and Figure 2 , GFNSeqEditor outperforms all baselines, achieving substantial property improvements with a controlled number of edits. This superior performance is attributed to GFNSeqEditor's utilization of a pre-trained flow function from GFlowNet, enabling it to achieve significantly higher property improvements than DE, Ledidi, LaMBO, and MOGFN-AL, which rely on local search techniques by optimizing either a given single sequence or a batch of sequences. Specifically, the flow function F θ (•) is trained to sample sequences with probability proportional to their reward and, as a result, employing the policy in (6) for editing enables GFNSeqEditor to leverage global information contained in F θ (•) about the entire space of sequences. Furthermore, GFNSeqEditor achieves larger property improvement than GFlowNet-E. The GFNSeqEditor identifies and edits sub-optimal positions within a seed sequence using (4), while GFlowNet-E only edits the tail of the input seed sequence. This indicates the effectiveness of the sub-optimal position identifier function of GFNSeqEditor.Ablation study. We further study the property improvement achieved by GFNSeqEditor along with edit percentage across various choices of hyperparameters δ and λ. Figure 3 illustrates that an increase in δ generally corresponds to an increase in both property improvement and edit percentage, whereas, in most cases, an increase in λ results in a decrease in property improvement and edit percentage. Furthermore, in Figure 7 in Appendix E. In addition to editing sequences, we investigate the ability of GFNSeqEditor to be used alongside a sequence generative model to enhance the generation of novel sequences. This highlights the versatility of the proposed GFNSeqEditor. In this Subsection, we utilize a pre-trained diffusion model (DM) for sequence generation, with further details available in Appendix E.2. The sequences generated by the DM are passed to GFNSeqEditor to improve their target property. Given that GFNSeqEditor utilizes a trained GFlowNet model, this combination of a DM and GFNSeqEditor can be regarded as an ensemble approach, effectively leveraging both the DM and the GFlowNet for sequence generation. Table 2 presents the property and diversity metrics for sequences generated by the DM, the GFlowNet, and the combined DM+GFNSeqEditor across AMP and CRE datasets, with each method generating 1, 000 sequences. As observed from Table 2 , GFlowNet excels at producing sequences with higher property values compared to the DM, while the DM exhibits greater sequence diversity than the GFlowNet. Sequences generated by DM+GFNSeqEditor maintain similar property levels to the GFlowNet on its own, while their diversity is in line with that of the DM. This highlights the effectiveness of DM+GFNSeqEditor in harnessing the benefits of both the GFlowNet and the DM. Moreover, we show the CDF of the property for sequences generated by the DM, the GFlowNet, and DM+GFNSeqEditor in Figure 4 . As shown, the CDF of DM+GFNSeqEditor aligns with both DM and GFlowNet. Specifically, for AMP dataset, DM+GFNSeqEditor generates more sequences with higher properties than 0.78 compared to GFlowNet, while reducing the number of low-property generated sequences compared to DM alone. In the case of CRE dataset, the results in Figure 4 indicate that as δ increases, the CDF of DM+GFNSeqEditor becomes more akin to that of GFlowNet. This is expected, as an increase in δ leads to a greater number of edits. GFNSeqEditor possesses the capability to combine multiple sequences, yielding a novel sequence that closely resembles its parent sequences. This capability proves invaluable in several applications. For example, when it is important to shorten relatively lengthy sequences while retaining desired properties (see, e.g., [54, 59] ). GFNSeqEditor accomplishes this by combining a longer sequence with a shorter one. The resultant sequence maintains high similarity with the longer one to retain its desired properties, while also resembling a realistic, relatively shorter sequence to ensure safety and predictability. Algorithm 2 in Appendix E.5 describes using GFNSeqEditor to combine two sequences with the goal of shortening the longer one.We evaluate GFNSeqEditor's performance in combining pairs of long and short sequences using the AMP dataset as a test case. In this context, a long sequence is defined as one with a length exceeding 30, while a short sequence has a length shorter than 20. Each initial pair consists of a long AMP sequence and its closest short sequence with an AMP property exceeding 0.7. Table 5 and Figure 5 in Appendix E.5 present the results of sequence combination for sequence length reduction. As indicated in Table 5 , GFNSeqEditor not only enhances the properties of the initial long sequences, but also significantly shortens them by more than 63%. Additionally, the sequences generated by GFNSeqEditor resemble both the initial long and short sequences, with an average Levenshtein similarity of approximately 65% to long sequences and 55% to short sequences. 
conclusions: 
 This paper introduces GFNSeqEditor, a generative model for sequence editing built upon GFlowNet.Given an input seed sequence, GFNSeqEditor identifies and edits positions within the input sequence to enhance its property. This paper also offers a theoretical analysis of the properties of edited sequences and the amount of edits performed by GFNSeqEditor. Experimental evaluations using real-world DNA and protein datasets demonstrate that GFNSeqEditor outperforms state-of-the-art baselines in terms of property enhancement while maintaining a similar amount of edits. Nevertheless, akin to many machine learning algorithms, GFNSeqEditor does have its limitations. It relies on a well-trained GFlowNet model, necessitating the availability of a high-quality trained GFlowNet for optimal performance.A Proof of Theorem 4.1Let z denotes a sequence with length T generated from scratch using the policy π F (•) asEQUATION)The expected reward of z can be obtained asEQUATIONwhere T T denotes the set of sequences with length T that can be generated by F θ (•). The probability that the GFNSeqEditor outputs an arbitrary sequence w ∈ T T given x can be expressed asEQUATIONThe probability Pr[x t = w t | x:t-1 , x] can be obtained asEQUATION) where π(•) defined in (6) . According to (6) , it can be written thatEQUATIONFurthermore, according to (4) and ( 5), we have D(EQUATIONIn addition, it can be inferred thatEQUATIONTherefore, it can be concluded that if ν > 1 -δ, it is guaranteed that D( x:t-1 ; δ, σ) = 1. Since ν follows a Gaussian distribution with a variance of σ 2 we have ν > 1 -δ with probability 1 -Φ( 1-δ σ ). Hence, it can be written thatEQUATIONCombining ( 20) and ( 17) with ( 16), we getEQUATIONMoreover, combining (21) with (15) , we obtainEQUATIONUsing (22) for the expected reward of x given x we can writeEQUATIONCombining ( 23) with ( 14), we getEQUATIONwhich proves (8) . Moreover, the upper bound of property improvement by the proposed GFNSeqEditor is analyzed in Appendix B. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: We believe that the paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: In Section 5 and Appendix E.1, we properly cited the original papers that produced the code packages or datasets we used.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The expected property improvement of GFNSeqEditor can be obtained asEQUATIONSince T T can be split into two sets S * x and T T \ S * x , the expected property improvement of GFNSe-qEditor can be obtained asEQUATIONIf w ∈ T T \ S * x , then p w ≤ y. Therefore, the expected property improvement of GFNSeqEditor can be bounded from above asEQUATION)The probability that the GFNSeqEditor outputs w ∈ S * x can be expressed asEQUATIONThe probability Pr[x t = w t | x:t-1 , x] can be obtained asEQUATIONIf x t ̸ = w t , according to (37) and considering the fact that π(w t |w :t-1 ) ≤ 1, the probability in ( 29) can be bounded from above asEQUATIONOtherwise if x t = w t , it can be written that Pr[x t = w t | x:t-1 , x] ≤ 1. Since any w ∈ S * x should be different from x in at least one position, combining (28) with (30) we can conclude thatEQUATIONCombining ( 27) with (31) proves the Theorem.We obtain the upper bound for the expected distance between edited sequence x and the original sequence x. Since both x and x have the same length T , the distance lev(x, x) can be interpreted as the number of elements different in these two sequences. Therefore, in order to obtain lev(x, x), it is sufficient to find the number of times thatx t ̸ = xt , ∀t : 1 ≤ t ≤ T . If D( x:t-1 ; δ, σ) = 0, then xt = x t . Furthermore, if D( x:t-1 ; δ, σ) = 1, then according to (6), we have xt = x t with probabilityEQUATIONTherefore, the probability Pr[x t ̸ = x t ] can be obtained asEQUATION)Since F θ ( x:t-1 + x t ) ≥ 0, the probability Pr[x t ̸ = x t ] can be bounded asEQUATIONMoreover, if we haveEQUATIONthen D( x:t-1 ; δ, σ) = 0. Furthermore, the right hand side of ( 35) can be bounded from below asEQUATIONTherefore, if ν ≤ -δ, it is ensured that D( x:t-1 ; δ, σ) = 0. The probability that ν ≤ -δ is Φ(-δ σ ). Hence, we can conclude thatEQUATIONCombining (37) with (34) , we arrive atEQUATIONMoreover, since both x and x have the same length T , the expected Levenshtein distance between x and x can be obtained asEQUATIONThus, combining (39) with (38) , we can write thatEQUATIONwhich proves the Theorem.According to (33) and the assumption in (11) , it can be written thatEQUATIONCombining (41) with (20) , we getEQUATIONSumming ( 42) over all elements in the sequence proves the theorem.This appendix provides a comprehensive overview of the experimental setup in Section 5 and presents additional supplementary experimental results.All training and inferences, including GFNSeqEditor, have been conducted using a single Nvidia Quadro P6000.Detailed information about the datasets can be found below:• TFbinding: The dataset is taken from [4] and contains all possible DNA sequences with length 8. The vocabulary is the four DNA bases, {A, C, G, T}. The goal is to edit a given DNA sequence to increase its binding activity with certain DNA-binding proteins called transcription factors. Higher binding activity is preferable. For train, test and validation purposes 50% of the dataset is set aside. The task entails editing a test dataset consisting of 10% of samples while the remaining data is utilized for training and validation.• AMP: The dataset, acquired from DBAASP [40] , is curated following the approach outlined by [19] . Peptides (i.e. short proteins) within a sequence-length range of 12 to 60 amino acids are specifically chosen. The dataset comprises a total of 6, 438 positive samples, representing anti-microbial peptides (AMPs), and 9,522 negative samples, which are non-AMPs. The vocabulary consists of 20 amino acids. The primary objective is to edit the non-AMP samples in such a way that the edited versions attain the characteristics exhibited by AMP samples. The task primarily centers on editing a subset comprising 10% of the non-AMP samples, designated for use as test samples, with the remaining samples allocated for training and validation purposes. • CRE: The dataset contains putative human cis-regulatory elements (CRE) which are regulatory DNA sequences modulating gene expression. CREs were profiled via massively parallel reporter assays (MPRAs) [12] where the activity is measured as the expression of the reporter gene. For our analysis, we randomly extract 10, 000 DNA sequences, each with a length of 200 base pairs, utilizing a vocabulary of the four bases. The overarching objective is to edit the DNA sequences to increase the reporter gene's expression specifically within the K562 cell line, which represents erythroid precursors in leukemia. The task involves editing a subset of 1, 000 test samples, while the rest are allocated for training and validation purposes.To evaluate the performance of each sequence editing method in terms of property improvement, it is required to obtain the properties of edited sequences. To this end, we employ an oracle for each dataset. The TFbinding dataset contains all possible 65, 792 DNA sequences with length of 8. Therefore, by looking into the dataset the true label of each edited sequence can be found. Following [1, 19] , the AMP dataset is split into two parts: D 1 and D 2 . The oracle for the AMP dataset is a set of trained models on partition D 2 as a simulation of wet-lab experiments. We employed oracles trained by [19] for AMP dataset. It is worth noting that the performance of predictive models on AMP datasets can be influenced by negative sampling in the dataset [44] . Furthermore, for CRE dataset we leverage the Malinois model [12] which is a deep convolutional neural network (CNN) for cell type-informed CRE activity prediction of any arbitrary sequence.DE and Ledidi. In order to implement DE and Ledidi baselines, there should be a proxy model to enable baselines to evaluate their candidate edits at each iteration of these algorithms. For each dataset, we train a proxy model on the training split of each dataset. For the TFBinding dataset, we configure a three-layer MLP with hidden dimensions of 64. In the case of AMP, we opt for a four-layer MLP, also with hidden dimensions of 64. Finally, for CRE, we utilize a four-layer MLP with hidden dimensions set to 2048. Across all models, the learning rate is consistently set to 10 -4 , ReLU serves as the activation function, and we set the number of epochs as 2, 000. To implement the DE baseline, we randomly select edit locations based on the desired edit percentages. At each selected location, we apply an edit by choosing the action that maximizes the proxy model's property prediction.LaMBO. We utilize the official implementation of LaMBO 4 . Test sequences 5 serve as the candidate pool, and all candidate samples in the pool are weighted similarly to maintain consistency with other sequence editing baselines. To ensure a fair comparison, we employ the same proxy as GFNSeqEditor to calculate the property scores. Across all three datasets, we use mlm as the encoder objective, ei as the acquisition function, and DKL SVGP regression as the surrogate. To generate 10 edits per sample, we configure num-gens=10, and window-size is adjusted for each dataset to ensure the edited percentages closely match the desired values. Hence, we set it to 2, 14, and 23 for TFBinding, AMP, and CRE, respectively. Additionally, for all datasets, we set pref-cond=False, pref-alpha=1, and beta-sched=1.We utilize the official implementation of MOGFN-AL 6 , where sequence tasks are based on the LaMBO implementation. Similar to LaMBO, test sequences serve as the candidate pool, weighted equally. The same proxy as GFNSeqEditor and LaMBO is employed to calculate property scores. Like LaMBO, we utilize mlm as the encoder objective. Training and validation batch sizes are set to 16 and 64, respectively, with hyperparameters for the conditional transformer as follows: num-hid=128, num-layers=3, and num-head=8. For each dataset, we adjust the max-len parameter to closely match the desired edited percentage. Thus, for TFBinding, AMP, and CRE, we set them as 6, 30, and 50, respectively. The reward function is modified to match that of GFNSeqEditor. To ensure fair comparisons and adapt LaMBO and MOGFN-AL for sequence editing, we do not use active learning settings for either baseline.Seq2Seq. In order to implement Seq2Seq baseline we use a standard transformer [53] as the translator to map an input sequence to an output sequence with superior property. We paired samples in each dataset such that each pair contains a sequence with lower property and a similar sequence with higher property which is the most similar to the sequence with lower property in the dataset. The transformer is trained to map the low property sequence to the high property sequence in each pair. The transformer is trained using the standard configurations in Pytorch transformer module tutorial.Both the embedding dimension of the transformer and the dimension of the 2 layer feedforward network model in the transformer encoder are set to 200. The number of heads in multihead attention layer is 2 and the drop-out rate is 0.2. We employ the CrossEntropyLoss function in conjunction with the stochastic gradient descent optimizer. The initial learning rate is set at 5.0 and follows a StepLR schedule.Both the baseline GFlowNet-E and the proposed GFNSeqEditor use the same trained GFlowNet model. We trained an active learning based GFlowNet model following the setting in [19] . In active learning setting, at each round of active learning t × K candidates generated by GFlowNet are sampled and then top K samples based on scores given by a proxy are chosen to be added to the offline dataset. Here offline dataset refers to an initial labeled dataset. To train the GFlowNet, we employed the same proxy models as those used by other baseline methods. For all datasets, we set the number of active learning rounds to 1, with t equal to 5 and K equal to 100. We parameterize the flow using a MLP comprising two hidden layers, each with a dimension of 2048, and |A| outputs corresponding to individual actions. Throughout our experiments, we employ the trajectory balance objective for training. Adam optimizer with (β 0 , β 1 ) = (0.9, 0.999) is utilized during the training process. The learning rate for logZ in trajectory balance loss is set to 10 -3 for all the experiments. The number of training steps for TFbinding, AMP and CRE are 5000, 10 6 and 10 4 , respectively. The remaining hyperparameters were configured in accordance with the settings established in [19] .We trained our diffusion models on the full sequence datasets of AMP sequences or CRE sequences. The sequences were one-hot encoded, yielding 20-vectors for protein sequences and 4-vectors for DNA sequences.We employed the "variance-preserving stochastic differential equation" (VP-SDE) [47] . We used a variance schedule of β(t) = 0.9t + 0.1. We set our time horizon T = 1 (i.e. t ∈ [0, 1)). This amounts to adding Gaussian noise over continuous time.For our discrete-time diffusion model, we defined a discrete-time Gaussian noising process, following [17] . We defined β t = (1 × 10 -4 ) + (1 × 10 -5 )t. We set our time horizon T = 1000 (i.e. t ∈ [0, 1000]).Our denoising network was based on a transformer architecture. The time embedding was computed as [sin(2π t T z), cos(2π t T z)], where z is a 30-vector of Gaussian-distributed parameters that are not trainable. The time embeddings were passed through two dense layers with a sigmoid in between, mapping to a 256-vector of time representations. For any input in a batch, it was concatenated with the time embedding and a sinusoidal positional embedding (defined in [53] ) of dimension 64. This concatenation was passed to a linear layer to map it to 128 dimensions. This was then passed to a standard transformer encoder of 3 layers, 8 attention heads, and with a hidden dimension of 128 and an MLP dimension of 64. The result was then passed to a linear layer which projected back to the input dimension.We trained our diffusion models with a learning rate of 0.001, for 100 epochs. We noted that the loss had converged for all models at that point. We also employed empirical loss weighting, where the loss of each input in a batch is divided by the L2 norm of the true Stein score.We trained our diffusion models on a single Nvidia Quadro P6000.When generating samples from a continuous-time diffusion model, we used the predictor-corrector algorithm defined in [47] , using 1000 time steps from T to 0. We then rounded all outputs to the nearest integer to recover the one-hot encoded sample.In Figure 6 , we illustrate the distribution of input non-AMP sequences, the sequences edited by GFNSeqEditor, and the AMP samples from the AMP dataset. It is evident from Figure 6 that GFNSeqEditor shifts the property distribution of input non-AMP sequences towards that of AMP sequences. Moreover, Figure 7 illustrates the impact of changing σ on property improvement and edit diversity for GFNSeqEditor. As can be seen increasing σ results in decreased property improvement and enhanced diversity.It is worth noting that GFNSeqEditor is capable of performing edits even when certain portions of the input sequence are masked and cannot be modified. Table 3 showcases the performance of GFNSeqEditor compared to Ledidi on the CRE dataset, with the first 100 elements of the input sequences masked. As depicted in Table 3 , GFNSeqEditor achieves significantly greater property improvement than Ledidi while utilizing a lower edit percentage.This Subsection compares the performance of GFNSeqEditor in sequence generation task with that of GFlowNet and diffusion model (DM) on CRE dataset. We relax the hyperparameters to allow a higher amount of edits and we set δ = 0.4, λ = 0.1 and σ = 0.001 for GFNSeqEditor. The results are presented in Table 4 . GFlowNet and DM generate 1000 sequences. GFNSeqEditor also generates 1000 sequences by editing each of 1000 samples in the test dataset. As can be seen, GFNSeqEditor 3 and 7 , the amounts of edits performed by GFNSeqEditor can be controlled by hyperparameters δ, λ and σ.Algorithm 2 presents the GFNSeqEditor for combining two sequences in order to obtain a new sequence whose length is the length of shorter sequence. In Figure 5 , we depict the distributions of input sequence lengths and properties, alongside the lengths and properties of the outputs generated by GFNSeqEditor. This scenario pertains to the combination of a long AMP sequence with a short AMP sequence, as detailed in Subsection 5.3. As depicted in Figure 5 , the edited sequences produced by GFNSeqEditor exhibit property distributions akin to those of the lengthy input AMP sequences. Simultaneously, these edited sequences are considerably shorter than the original long input sequences. This highlights GFNSeqEditor's effectiveness in shortening lengthy AMP sequences while preserving their inherent properties.Furthermore, Table 6 provides results for combining pairs of AMP sequences as well as pairs consisting of an AMP sequence and a non-AMP sequence. In both cases, GFNSeqEditor generates a sequence with a length matching that of the longer sequence. When combining two AMP sequences, GFNSeqEditor produces new sequences with higher properties than their parent sequences, maintaining an average resemblance of over 60% to each parent. Additionally, GFNSeqEditor can be applied to combine a non-AMP sequence with an AMP sequence, offering the advantage of rendering the edited non-AMP sequence more akin to a genuine AMP sequence. The results in Table 6 demonstrate that GFNSeqEditor substantially enhances the properties of non-AMP sequences, surpassing the properties of their AMP parents. Furthermore, on average, 35% of the edited sequences bear a resemblance to their AMP parents.GFlowNets, initially proposed by [5] , were introduced as a reinforcement-learning (RL) algorithm designed to expand upon maximum-entropy RL, effectively handling scenarios with multiple paths leading to a common state. However, recent studies have redefined and generalized its scope, describing it as a general framework for amortized inference with neural networks [32, 20, 60, 56] .There has been a recent surge of interest in employing GFlowNets across various domains. Noteworthy examples include its utilization in molecule discovery [5] , Bayesian structure learning [11, 36] , and Algorithm 2 GFNSeqEditor for combining two sequences to shorten the length of longer sequence.1: Input: x 1 and x 2 with lengths T 1 and T 2 , flow function F θ (•) and parameters δ, λ and σ.2: Initialize x:0 as an empty sequence and T min = min{T 1 , T 2 }.3: for t = 1, . . . , T min do 4:Assign x t = arg max {x1,t,x2,t} {F θ ( x:t-1 + x 1,t ), F θ ( x:t-1 + x 2,t )}.Check if x t is sub-optimal by obtaining D(x t , x:t-1 ; δ, σ) according to (5).if D(x t , x:t-1 ; δ, σ) = 1 then 7:Sample xt according to policy π(•| x:t-1 ) as follows:8:if T 1 > T 2 then 9:π(a| x:t-1 ) = (1 -λ)EQUATIONπ(a| x:t-1 ) = (1 -λ)F θ ( x:t-1+a) a ′ ∈A F θ ( x:t-1+a ′ ) + λ1 a=x2,t .12:end if graph explainability [29] . Recognizing its significance, several studies have emerged to enhance the learning efficiency of GFlowNets [6, 32, 30, 43] since the introduction of the flow matching learning objective by [5] . Moreover, GFlowNets have demonstrated adaptability in being jointly trained with energy and reward functions [57] . [38] introduce intrinsic exploration rewards into GFlowNets, addressing exploration challenges within sparse reward tasks. A couple of recent studies try to extend GFlowNets to stochastic environments, accommodating stochasticity in transition dynamics [39] and rewards [58] . Several novel GFlowNet training methodologies have been recently proposed in [22, 26, 37, 14] . The application of GFlowNets when a predefined reward function is not accessible is explored in [8] . Distributed training of GFlowNets is discussed in [45] . Accelerating GFlowNet training is investigated in [25] . Moreover, [27] employs GFlowNets for designing DNA-encoded libraries. To reduce the need for expensive reward evaluations, [24] proposes a new GFlowNet-based method for molecular optimization.The aforementioned works have primarily focused on theoretical developments of GFlowNet and its application in molecular generation, without directly addressing the challenges associated with sequence design or editing. In a departure from this trend, and inspired by Bayesian Optimization, [19] proposed a new active learning algorithm based on GFlowNets, i.e. GFlowNet-AL to design novel biological sequences. GFlowNet-AL [19] utilizes the epistemic uncertainty of the surrogate model within its reward function, guiding the GFlowNet towards the optimization of promising yet less-explored regions within the state space. This approach fosters the generation of a diverse set of de novo sequences from scratch and token-by-token. Unlike GFNSeqEditor, it lacks the capability to edit input seed sequences and combine multiple sequences. This distinction underscores the unique contribution of GFNSeqEditor in addressing the sequence editing problem, positioning it as a valuable addition to the existing literature on GFLowNet.Biological sequence optimization and design hold transformative potential for biotechnology and health, offering enhanced therapeutic solutions and a vast range of applications. Techniques that enable refining sequences can lead to advancements like elucidating the role of individual gene products, rectifying genetic mutations in afflicted tissues, and optimizing properties of peptides, antibodies, and nucleic-acid therapeutics. However, the dual-edged nature of such breakthroughs must be acknowledged, as the same research might be misappropriated for unintended purposes.Our method can be instrumental in refining diagnostic procedures and uncovering the genetic basis of diseases, which promises a deeper grasp of genetic factors in diseases. Yet, we must approach with caution, as these advancements may unintentionally amplify health disparities for marginalized communities. As researchers, we emphasize the significance of weighing the potential societal benefits against unintended consequences while remaining optimistic about our work's predominant inclination towards beneficial outcomes.NeurIPS Paper ChecklistSeq2Seq relies on identifying pairs of similar sequences for training. However, we were unable to identify similar pairs for CRE, possibly because of the limited number of training samples relative to the lengthy nature of the sequences (i.e., sequences with a length of 200).https://github.com/samuelstanton/lambo.For the AMP dataset, we have removed the samples with a length lower than window-size.https://github.com/MJ10/mogfn-al. 


Decision: This paper presents a novel biological sequence editing algorithm, GFNSeqEditor, based on pretrained Generative Flow Networks (GFNs) with theoretical analysis on expected reward, property improvements, and the number of edits. It identifies and edits sub-optimal sites to enhance the desired property.

Overall, the reviews highlight its strengths of GFNSeqEditor along with theoretical anlyses to understand how it works as well as solid benchmarking evaluations. While the reviewers also raised several potential concerns, most were effectively addressed in the detailed author feedback.

Based on these evaluation, I would like to recommend accepting this submission. We believe this interesting work will attract the interest of the NeurIPS community.

Reviews: review: paper_summary: paper_summary: They propose a new sequence editing method using GFlowNets as priors and suggest additional hyperparameters to tune suboptimal gaps, randomness, and penalization. They theoretically analyze how these new hyperparameters can effectively control the lower and upper bounds of the number of edits. The performance results demonstrate some effectiveness of the proposed method.
strength_weakness: strength_weakness: **The baselines used in the study are too weak:** I don't think this method offers significant advantages over conditional GFlowNets, which can translate input sequences to output sequences while maintaining constraints on sequence distance. There are many other methods capable of achieving this, including Seq2Seq, which the authors used as a baseline. The Seq2Seq implementation seems overly simplistic; there are potentially better techniques for DNA editing. For example, hierarchical variational inference in latent space, combined with a well-designed reward model, could optimize the process more effectively. Any thoughts on this?

**This is not a GFlowNet with multiple backward paths:** It appears that GFlowNets are used for sequence generation in an autoregressive manner (one-way generation) where \( P_B = 1 \). This approach is equivalent to Soft-Q-learning and path consistency learning (PCL) [1], so relevant soft RL literature should be clearly explained. The paper's title is somewhat misleading (though it need not be changed) as it essentially describes using a PCL-trained agent to assist in sequence editing. Reviewers suggest that the authors acknowledge this literature.

**The literature review for GFlowNets should include works published in 2024:** Reviewers also expect the authors to include recent GFlowNets literature targeting biological or chemical applications, such as those published in ICLR and ICML 2024. Additionally, there are preprints discussing improvements and evolutionary methods (e.g., genetic algorithms) using GFlowNets; engaging with this existing literature would be beneficial. The current literature review seems not very up-to-date in 2024.

[1] Nachum, Ofir, et al. "Bridging the gap between value and policy based reinforcement learning." Advances in neural information processing systems 30 (2017).
questions: questions: 1. There are many hyperparameters, and biological tasks often involve expensive oracle functions. How can we tune these hyperparameters in real-world applications?

2. Is there any tendency for the proposed method to work better on large sequences? Please provide insights on this.
limitations: limitations: They address their limitations in the conclusion.
paper_summary: paper_summary: This paper introduced a new algorithm for biological sequence editing, GFNSeqEditor. This algorithm is designed based on pre-trained Generative Flow Networks (GFNs), and improves target properties by identifying and editing sub-optimal sites of input sequences. Through theoretical analysis and experiments on three datasets, this new algorithm shows that it can improve biological properties with diversified edit while minimize the number of edits to ensure safety and predictability. By comparing with a few baseline models, the GFNSeqEditor outperforms the state-of-the-art methods on property improvements, diversity, and edit percentage.
strength_weakness: strength_weakness: 1. the algorithm was built on the assumption that fewer edits lead to safer and more predictable modification. I would argue this claim is valid under the condition that, compared with generating a new sequence, limited editing is possibly a safer choice. Many studies on single nucleotide variance (SNV) show that a single nucleotide change can lead to functional change or property loss. To avoid unexpected confusion, it would be better to either include followup analysis on structural changes caused by sequence editing, which I believe will be too much work to do, or improve the statement on the safety claim. 
2. Lack of details about the selection of non-AMP samples. Previous study [1] have shown that inappropriate selection of negative samples can introduce bias. Clarification on the criteria of selecting negative samples would help the audience better evaluate the validity of the results.  
3. While GFNSeqEditor shows improvements compared with the baselines, the performance improvements are not substantial, especially on the aspect of editing percentage. Any possible reasons or statistical analysis on the significance?

[1]. Sidorczuk K, Gagat P, Pietluch F, et al. Benchmarks in antimicrobial peptide prediction are biased due to the selection of negative data[J]. Briefings in Bioinformatics, 2022, 23(5): bbac343.
questions: questions: 1. I would suggest refining the statement on fewer edits leads to safe and predictable sequence editing. Follow up analysis on how sequence editing influence structure and function change would be great to have, but not necessary. 
2. Please consider add more details on how the dataset select non-AMP samples, and briefly mention about the negative sample problem for clarify. 
3. It would be more clear to include statistical analysis or dig into possible reasons and future improvements would help the audience to understand the significance of this work and looking for future improvements.
limitations: limitations: Yes.
paper_summary: paper_summary: The author proposed GFNSeqEditor, a novel sequence editing and generation model built on GFlowNet, which provides different modifications for each sequence to enhance desired features. Several experiments have demonstrated the performance of the proposed algorithm.
strength_weakness: strength_weakness: There are shortcomings in the model comparison, as it does not compare with existing biological sequence design methods. In terms of sequence evaluation, mainstream evaluation experiments were not used, leading to doubts about the model's effectiveness.
questions: questions: 1. Existing biological sequence design methods, such as evolutionary approaches, can still perform sequence editing in specified regions. Examples include AdaLead[1], PEX[2], Coms[3], and BiB[4]. The author must compare their method with these state-of-the-art algorithms to demonstrate its advantages. 


2. The author used AMP and CRE datasets and trained Oracles independently. Evaluating with self-trained models can lead to inconsistent standards, as the author can modify their trained Oracles at any time to demonstrate the algorithm's superiority. Unlike Coms[3] or BiB[4], which use numerous sequence evaluation problems, this leads to unfair experimental evaluations. It is recommended that the author evaluate under existing various evaluation standards to demonstrate the model's advantages better.


[1] Sinai, Sam, et al. "AdaLead: A simple and robust adaptive greedy search algorithm for sequence design." arXiv preprint arXiv:2010.02141 (2020). 

[2] Ren, Zhizhou, et al. "Proximal exploration for model-guided protein sequence design." International Conference on Machine Learning. PMLR, 2022.

[3]  Trabucco, Brandon, et al. "Conservative objective models for effective offline model-based optimization." International Conference on Machine Learning. PMLR, 2021.

[4] Chen, Can, et al. "Bidirectional learning for offline model-based biological sequence design." International Conference on Machine Learning. PMLR, 2023.
limitations: limitations: yes

Comments: title: Edit Official Comment by Reviewer Ns1N
comment: Thank you for addressing my concerns and providing detailed explanations in your rebuttal. As a result, I have raised my score.
Most of my concerns were addressed, so I increased the score accordingly.