Original: B IDIRECTIONAL L EARNING FOR O FFLINE
Revised: B IDIRECTIONAL L EARNING FOR O FFLINE

Original: M ODEL
Revised: M ODEL

Original: - BASED B IOLOGICAL S EQUENCE D ESIGN
Revised: - BASED B IOLOGICAL S EQUENCE D ESIGN

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Offline model-based optimization aims to maximize a black-box objective function with a static dataset of designs and their scores.
Revised: Offline model-based optimization aims to maximize a black-box objective function with a static dataset of designs and their scores.

Original: In this paper, we focus on biological sequence design to maximize some sequence score.
Revised: In this paper, we focus on biological sequence design to maximize some sequence score.

Original: A recent approach employs bidirectional learning, combining a forward mapping for exploitation and a backward mapping for constraint, and it relies on the neural tangent kernel (NTK) of an infinitely wide network to build a proxy model.
Revised: A recent approach employs bidirectional learning, combining a forward mapping for exploitation and a backward mapping for constraint, and it relies on the neural tangent kernel (NTK) of an infinitely wide network to build a proxy model.

Original: Though effective, the
Revised: Though effective, the

Original: NTK cannot learn features because of its parametrization, and its use prevents the incorporation of powerful pre-trained Language Models (LMs) that can capture the rich biophysical information in millions of biological sequences.
Revised: NTK cannot learn features because of its parametrization, and its use prevents the incorporation of powerful pre-trained Language Models (LMs) that can capture the rich biophysical information in millions of biological sequences.

Original: We adopt an alternative proxy model, adding a linear head to a pre-trained LM, and propose a linearization scheme.
Revised: We adopt an alternative proxy model, adding a linear head to a pre-trained LM, and propose a linearization scheme.

Original: This yields a closed-form loss and also takes into account the biophysical information in the pre-trained LM.
Revised: This yields a closed-form loss and also takes into account the biophysical information in the pre-trained LM.

Original: In addition, the forward mapping and the backward mapping play different roles and thus deserve different weights during sequence optimization.
Revised: In addition, the forward mapping and the backward mapping play different roles and thus deserve different weights during sequence optimization.

Original: To achieve this, we train an auxiliary model and leverage its weak supervision signal via a bi-level optimization framework to effectively learn how to balance the two mappings.
Revised: To achieve this, we train an auxiliary model and leverage its weak supervision signal via a bi-level optimization framework to effectively learn how to balance the two mappings.

Original: Further, by extending the framework, we develop the first learning rate adaptation module Adaptive - η , which is compatible with all gradient-based algorithms for offline model-based optimization.
Revised: Further, by extending the framework, we develop the first learning rate adaptation module Adaptive - η , which is compatible with all gradient-based algorithms for offline model-based optimization.

Original: Experimental results on DNA/protein sequence design tasks verify the effectiveness of our algorithm.
Revised: Experimental results on DNA/protein sequence design tasks verify the effectiveness of our algorithm.

Original: Our code is available here.
Revised: Our code is available here.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Offline model-based optimization aims to maximize a black-box objective function with a static dataset of designs and their scores.
Revised: Offline model-based optimization aims to maximize a black-box objective function with a static dataset of designs and their scores.

Original: This offline setting is realistic since in many real-world scenarios we do not have interactive access to the ground-truth evaluation.
Revised: This offline setting is realistic since in many real-world scenarios we do not have interactive access to the ground-truth evaluation.

Original: The design tasks of interest include material, aircraft, and biological sequence (Trabucco et al., 2021).
Revised: The design tasks of interest include material, aircraft, and biological sequence (Trabucco et al., 2021).

Original: In this paper, we focus on biological sequence design, including DNA sequence and protein sequence, with the goal of maximizing some specified property of these sequences.
Revised: In this paper, we focus on biological sequence design, including DNA sequence and protein sequence, with the goal of maximizing some specified property of these sequences.

Original: A wide variety of methods have been proposed for biological sequence design, including evolutionary algorithms (Sinai et al., 2020;
Revised: A wide variety of methods have been proposed for biological sequence design, including evolutionary algorithms (Sinai et al., 2020;

Original: Ren et al., 2022), reinforcement learning methods (Angermueller et al., 2019), Bayesian optimization (Terayama et al., 2021), search/sampling using generative models (Brookes et al., 2019; Chan et al., 2021), and GFlowNets (Jain et al., 2022).
Revised: Ren et al., 2022), reinforcement learning methods (Angermueller et al., 2019), Bayesian optimization (Terayama et al., 2021), search/sampling using generative models (Brookes et al., 2019; Chan et al., 2021), and GFlowNets (Jain et al., 2022).

Original: Recently, gradient-based techniques have emerged as an effective alternative (Trabucco et al., 2021).
Revised: Recently, gradient-based techniques have emerged as an effective alternative (Trabucco et al., 2021).

Original: These approaches first train a deep neural network (DNN) on the static dataset as a proxy and then obtain the new designs by directly performing gradient ascent steps on the existing designs.
Revised: These approaches first train a deep neural network (DNN) on the static dataset as a proxy and then obtain the new designs by directly performing gradient ascent steps on the existing designs.

Original: Such methods have been widely used in biological sequence design (Norn et al., 2021;
Revised: Such methods have been widely used in biological sequence design (Norn et al., 2021;

Original: Tischer et al., 2020; Linder & Seelig, 2020).
Revised: Tischer et al., 2020; Linder & Seelig, 2020).

Original: One obstacle is the out-of-distribution issue, where the trained proxy model is inaccurate for the newly generated sequences.
Revised: One obstacle is the out-of-distribution issue, where the trained proxy model is inaccurate for the newly generated sequences.

Original: To mitigate the out-of-distribution issue, recent work proposes regularization of the model (Trabucco et al., 2021;
Revised: To mitigate the out-of-distribution issue, recent work proposes regularization of the model (Trabucco et al., 2021;

Original: Yu et al., 2021; Fu & Levine, 2021) or the design itself (Chen et al., 2022).
Revised: Yu et al., 2021; Fu & Levine, 2021) or the design itself (Chen et al., 2022).

Original: The first category focuses on training a better proxy by introducing inductive biases such as robustness (Yu et al., 2021).
Revised: The first category focuses on training a better proxy by introducing inductive biases such as robustness (Yu et al., 2021).

Original: The second category introduces bidirectional learning (Chen et al., 2022), which consists of a forward mapping and a backward mapping, to optimize the design directly.
Revised: The second category introduces bidirectional learning (Chen et al., 2022), which consists of a forward mapping and a backward mapping, to optimize the design directly.

Original: Specifically, the backward mapping leverages the high-scoring design to predict the static dataset and vice versa for the forward mapping, which distills the information of the static dataset into the high-scoring design.
Revised: Specifically, the backward mapping leverages the high-scoring design to predict the static dataset and vice versa for the forward mapping, which distills the information of the static dataset into the high-scoring design.

Original: This approach achieves state-of-the-art performances on a variety of tasks.
Revised: This approach achieves state-of-the-art performances on a variety of tasks.

Original: Though effective, the proposed bidirectional learning relies on the neural tangent kernel (NTK) of an infinite-width model to yield a closed-form loss, which is a key component of its successful operation.
Revised: Though effective, the proposed bidirectional learning relies on the neural tangent kernel (NTK) of an infinite-width model to yield a closed-form loss, which is a key component of its successful operation.

Original: The NTK cannot learn features due to its parameterization (Yang & Hu, 2021) and thus the bidirectional learning cannot incorporate the wealth of biophysical information from Language Models (LMs) pre-trained over a vast corpus of unlabelled sequences (Elnaggar et al., 2021; Ji et al., 2021).
Revised: The NTK cannot learn features due to its parameterization (Yang & Hu, 2021) and thus the bidirectional learning cannot incorporate the wealth of biophysical information from Language Models (LMs) pre-trained over a vast corpus of unlabelled sequences (Elnaggar et al., 2021; Ji et al., 2021).

Original: To solve this issue, we construct a proxy model by combining a finite-width pre-trained LM with an additional layer.
Revised: To solve this issue, we construct a proxy model by combining a finite-width pre-trained LM with an additional layer.

Original: We then linearize the resultant proxy model, inspired by the recent progress in deep linearization (Achille et al., 2021; Dukler et al., 2022).
Revised: We then linearize the resultant proxy model, inspired by the recent progress in deep linearization (Achille et al., 2021; Dukler et al., 2022).

Original: This scheme not only yields a closed-form loss but also exploits the rich biophysical information that has been distilled in the pre-trained LM.
Revised: This scheme not only yields a closed-form loss but also exploits the rich biophysical information that has been distilled in the pre-trained LM.

Original: In addition, the forward mapping encourages exploitation in the sequence space and the backward mapping serves as a constraint to mitigate the out-of-distribution issue.
Revised: In addition, the forward mapping encourages exploitation in the sequence space and the backward mapping serves as a constraint to mitigate the out-of-distribution issue.

Original: It is important to maintain an appropriate balance between exploitation and constraint, and this can vary across design tasks as well as during the optimization process.
Revised: It is important to maintain an appropriate balance between exploitation and constraint, and this can vary across design tasks as well as during the optimization process.

Original: We introduce a hyperparameter γ to control the balance, and develop a bi-level optimization framework Adaptive - γ .
Revised: We introduce a hyperparameter γ to control the balance, and develop a bi-level optimization framework Adaptive - γ .

Original: In this framework, we train an auxiliary model and leverage its weak supervision signal to effectively update γ .
Revised: In this framework, we train an auxiliary model and leverage its weak supervision signal to effectively update γ .

Original: To sum up, we propose BI directional learning for model-based B iological sequence design ( BIB ).
Revised: To sum up, we propose BI directional learning for model-based B iological sequence design ( BIB ).

Original: Last but not least, since the offline nature prohibits standard cross-validation strategies for hyperparameter tuning, all gradient-based offline model-based algorithms preset the learning rate η .
Revised: Last but not least, since the offline nature prohibits standard cross-validation strategies for hyperparameter tuning, all gradient-based offline model-based algorithms preset the learning rate η .

Original: There is a danger of a poor selection, and to address this, we propose to extend Adaptive - γ to Adaptive - η , which effectively adapts the learning rate η via the weak supervision signal from the trained auxiliary model.
Revised: There is a danger of a poor selection, and to address this, we propose to extend Adaptive - γ to Adaptive - η , which effectively adapts the learning rate η via the weak supervision signal from the trained auxiliary model.

Original: To the best of our knowledge, Adaptive - η is the first learning rate adaptation module for gradient-based algorithms on offline model-based optimization.
Revised: To the best of our knowledge, Adaptive - η is the first learning rate adaptation module for gradient-based algorithms on offline model-based optimization.

Original: Experiments on DNA and protein sequence design tasks verify the effectiveness of BIB and Adaptive - η .
Revised: Experiments on DNA and protein sequence design tasks verify the effectiveness of BIB and Adaptive - η .

Original: To summarize, our contributions are three-fold:
Revised: To summarize, our contributions are three-fold:

Original: • Instead of adopting the NTK, we propose to construct a proxy model by combining a pre-trained biological LM with an additional trainable layer.
Revised: • Instead of adopting the NTK, we propose to construct a proxy model by combining a pre-trained biological LM with an additional trainable layer.

Original: We then linearize the proxy model, leveraging the recent progress on deep linearization.
Revised: We then linearize the proxy model, leveraging the recent progress on deep linearization.

Original: This yields a closed-form loss computation in bidirectional learning and allows us to exploit the rich biophysical information distilled into the LM via pretraining over millions of biological sequences.
Revised: This yields a closed-form loss computation in bidirectional learning and allows us to exploit the rich biophysical information distilled into the LM via pretraining over millions of biological sequences.

Original: • We propose a bi-level optimization framework Adaptive - γ where we leverage weak signals from an auxiliary model to achieve a satisfactory trade-off between exploitation and constraint.
Revised: • We propose a bi-level optimization framework Adaptive - γ where we leverage weak signals from an auxiliary model to achieve a satisfactory trade-off between exploitation and constraint.

Original: • We further extend this bi-level optimization framework to Adaptive - η .
Revised: • We further extend this bi-level optimization framework to Adaptive - η .

Original: As the first learning rate tuning scheme in offline model-based optimization, Adaptive - η allows learning rate adaptation for any gradient-based algorithm.
Revised: As the first learning rate tuning scheme in offline model-based optimization, Adaptive - η allows learning rate adaptation for any gradient-based algorithm.

Original:  P RELIMINARIES 2. O FFLINE MODEL - BASED OPTIMIZATION
Revised:  P RELIMINARIES 2. O FFLINE MODEL - BASED OPTIMIZATION

Original: Offline model-based optimization aims to find a design X to maximize some unknown objective f
Revised: Offline model-based optimization aims to find a design X to maximize some unknown objective f

Original: ( X ) .
Revised: ( X ) .

Original: This can be formally written as,
Revised: This can be formally written as,

Original: [Equation]
Revised: [Equation]

Original: where we have access to a size- N dataset D
Revised: where we have access to a size- N dataset D

Original: = { ( X 1 , y 1 ) } , · · · , { ( X N , y N ) } with X i representing a certain design and y i denoting the design score.
Revised: = { ( X 1 , y 1 ) } , · · · , { ( X N , y N ) } with X i representing a certain design and y i denoting the design score.

Original: In this paper, X i represents a biological sequence design, including DNA and protein sequences, and y i represents a property of the biological sequence such as the fluorescence level of the green fluorescent protein (Sarkisyan et al., 2016).
Revised: In this paper, X i represents a biological sequence design, including DNA and protein sequences, and y i represents a property of the biological sequence such as the fluorescence level of the green fluorescent protein (Sarkisyan et al., 2016).

Original: 2. B IOLOGICAL SEQUENCE R EPRESENTATION
Revised: 2. B IOLOGICAL SEQUENCE R EPRESENTATION

Original: Following (Norn et al., 2021; Killoran et al., 2017; Linder & Seelig, 2021), we adopt the positionspecific scoring matrix to represent a length- L protein sequence as X ∈ R L × 20 , where 20 represents 20 different kinds of amino acids.
Revised: Following (Norn et al., 2021; Killoran et al., 2017; Linder & Seelig, 2021), we adopt the positionspecific scoring matrix to represent a length- L protein sequence as X ∈ R L × 20 , where 20 represents 20 different kinds of amino acids.

Original: For a real-world protein sequence, X [ l, :] ( 0 ≤ l ≤ L − 1 ) is a one-hot vector denoting one kind of amino acid.
Revised: For a real-world protein sequence, X [ l, :] ( 0 ≤ l ≤ L − 1 ) is a one-hot vector denoting one kind of amino acid.

Original: During optimization, X [ l, :] is a continuous vector and softmax ( X [ l, :] ) represents the probability distribution of all 20 amino acids in the position l .
Revised: During optimization, X [ l, :] is a continuous vector and softmax ( X [ l, :] ) represents the probability distribution of all 20 amino acids in the position l .

Original: Similarly, for a DNA sequence, we have X ∈ R L × 4 where 4 represents 4 different DNA bases.
Revised: Similarly, for a DNA sequence, we have X ∈ R L × 4 where 4 represents 4 different DNA bases.

Original: The protein sequence X is fed into the embedding layer of the LM, which produces the embedding,
Revised: The protein sequence X is fed into the embedding layer of the LM, which produces the embedding,

Original: [Equation]
Revised: [Equation]

Original: The main block of the LM takes e as input and outputs biophysical features.
Revised: The main block of the LM takes e as input and outputs biophysical features.

Original: The DNA LM, which adopts the k -mer representation, is a little different from protein LMs.
Revised: The DNA LM, which adopts the k -mer representation, is a little different from protein LMs.

Original: See Appendix A.1 for details.
Revised: See Appendix A.1 for details.

Original: 2. G RADIENT ASCENT ON SEQUENCE
Revised: 2. G RADIENT ASCENT ON SEQUENCE

Original: A common approach to the posed offline model-based optimization problem is to train a proxy f θ ( X ) on the offline dataset,
Revised: A common approach to the posed offline model-based optimization problem is to train a proxy f θ ( X ) on the offline dataset,

Original: [Equation]
Revised: [Equation]

Original: Then we can obtain the high-scoring design X h by T gradient ascent steps:
Revised: Then we can obtain the high-scoring design X h by T gradient ascent steps:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the high-scoring design X h can be obtained as X T .
Revised: where the high-scoring design X h can be obtained as X T .

Original: Considering the discrete nature of biological sequences, the input of f θ ( · ) should be discrete one-hot vectors.
Revised: Considering the discrete nature of biological sequences, the input of f θ ( · ) should be discrete one-hot vectors.

Original: Following (Norn et al., 2021), we can perform the conversion and predict the score via:
Revised: Following (Norn et al., 2021), we can perform the conversion and predict the score via:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Then the gradient regarding X i can be approximated as,
Revised: Then the gradient regarding X i can be approximated as,

Original: [Equation]
Revised: [Equation]

Original: where we unroll the matrices X i , ˆ X i and Z i as vectors x i , ˆ x i and z i for notational convenience.
Revised: where we unroll the matrices X i , ˆ X i and Z i as vectors x i , ˆ x i and z i for notational convenience.

Original: This approximation allows us to use backpropagation directly from the proxy to the sequence design
Revised: This approximation allows us to use backpropagation directly from the proxy to the sequence design

Original: X i .
Revised: X i .

Original: For brevity, we will still use f θ ( X i ) to represent the proxy.
Revised: For brevity, we will still use f θ ( X i ) to represent the proxy.

Revised: 2. B IDIRECTIONAL LEARNING

Revised: As shown in Figure 1, bidirectional learning (Chen et al., 2022), consists of two mappings: the forward mapping leverages the static dataset ( X l , y l ) to predict the score y h of the high-scoring design X h , and the backward mapping leverages the high-scoring design data ( X h , y h ) to predict the static dataset ( X l , y l ).

Revised: The forward mapping loss can be written as

Original: M ETHOD
Revised: [Equation] where θ ∗ is given by [Equation] where β > 0 is a regularization parameter.

Revised: The backward mapping loss can be written as

Revised: [Equation]

Revised: ) where θ ∗ ( X h ) is given by

Revised: [Equation]

Revised: The high-scoring design X h can be optimized by minimizing the bidirectional learning loss L ( X h ) =

Revised: L l 2 h ( X h ) + L h 2 l ( X h ) .

Revised:  M ETHOD

Original: In this section, we first illustrate how to leverage deep linearization to compute the bidirectional learning loss in a closed form.
Revised: In this section, we first illustrate how to leverage deep linearization to compute the bidirectional learning loss in a closed form.

Original: Subsequently, we introduce a hyperparameter γ to control the balance between the forward mapping and the backward mapping.
Revised: Subsequently, we introduce a hyperparameter γ to control the balance between the forward mapping and the backward mapping.

Original: We then develop a novel bi-level optimization framework Adaptive - γ , which leverages a weak supervision signal from an auxiliary model to effectively update γ .
Revised: We then develop a novel bi-level optimization framework Adaptive - γ , which leverages a weak supervision signal from an auxiliary model to effectively update γ .

Original: Last but not least, we extend this framework to Adaptive - η , which enables us to adapt the learning rate η for all gradient-based offline model-based algorithms. 
Revised: Last but not least, we extend this framework to Adaptive - η , which enables us to adapt the learning rate η for all gradient-based offline model-based algorithms. We summarize our method in Algorithm 1.

Original: 3. DEEP LINEARIZATION FOR B IDIRECTIONAL L EARNING
Revised: 3. DEEP LINEARIZATION FOR B IDIRECTIONAL L EARNING

Original: Bidirectional learning, as presented in (Chen et al., 2022), consists of two mappings: the forward mapping leverages the static dataset ( X l , y l ) to predict the score y h of the high-scoring design X h , and the backward mapping leverages the high-scoring design data ( X h , y h ) to predict the static dataset ( X l , y l ).

Original: The forward mapping loss can be written as

Original: [Equation]

Original: (9) where θ ∗ is given by

Original: [Equation]

Original: where β > 0 is a regularization parameter.

Original: The backward mapping loss can be written as

Original: [Equation] where θ ∗ ( X h ) is given by [Equation] The backward mapping loss is intractable for a finite neural network, so Chen et al.
Revised: In bidirectional learning, the backward mapping loss is intractable for a finite neural network, so Chen et al.

Original: (2022) employ a neural network with infinite width, which yields a closed-form loss via the NTK.
Revised: (2022) employ a neural network with infinite width, which yields a closed-form loss via the NTK.

Original: This however makes it impossible to incorporate the rich biophysical information that has been distilled into a pre-trained LM (Yang & Hu, 2021).
Revised: This however makes it impossible to incorporate the rich biophysical information that has been distilled into a pre-trained LM (Yang & Hu, 2021).

Original: Considering this, we construct a proxy model by combining a finite-width pre-trained LM with an additional layer.
Revised: Considering this, we construct a proxy model by combining a finite-width pre-trained LM with an additional layer.

Original: We then linearize the resultant proxy model, inspired by the recent progress in deep linearization which has established that an overparameterized DNN model is close to its linearization (Achille et al., 2021; Dukler et al., 2022).
Revised: We then linearize the resultant proxy model, inspired by the recent progress in deep linearization which has established that an overparameterized DNN model is close to its linearization (Achille et al., 2021; Dukler et al., 2022).

Original: Denote by θ 0 = ( θ pt , θ lininit )  the proxy model parameters derived by combining the parameters of the pre-trained LM θ pt and a random initialization of the linear layer θ lininit .
Revised: Denote by θ 0 = ( θ pt , θ lininit ) ∈ R D × 1 the proxy model parameters derived by combining the parameters of the pre-trained LM θ pt and a random initialization of the linear layer θ lininit .

Original: In this paper, we adopt the pre-trained DNABERT (Ji et al., 2021) and Prot-BERT (Elnaggar et al., 2021) models, and compute the average of tokenhidden embeddings as the extracted feature, which is fed into the linear layer to build the proxy.
Revised: In this paper, we adopt the pre-trained DNABERT (Ji et al., 2021) and Prot-BERT (Elnaggar et al., 2021) models, and compute the average of token embeddings as the extracted feature, which is fed into the linear layer to build the proxy.

Original: Then we can construct a linear approximation for the proxy model: 
Revised: Then we can construct a linear approximation for the proxy model: [Equation]

Original: [Equation]
Revised: [Equation]

Revised: where f θ ( X ) , f θ 0 ( X ) ∈ R , ▽ θ f θ 0 ( X ) ∈ R 1 × D and ▽ θ f θ 0 ( X ) ∈

Revised: R D × 1 .

Original: Intuitively, if the fine-tuning does not significantly change θ 0 , then this linearization is a good approximation.
Revised: Intuitively, if the fine-tuning does not significantly change θ 0 , then this linearization is a good approximation.

Original: By leveraging this linearization, we can obtain a closed-form solution for Eq.(12) as:[Equation]
Revised: By leveraging this linearization, we can obtain a closed-form solution for Eq.(12) as:

Original: [Equation]
Revised: [Equation]

Original: Building on this result, we can compute the bidirectional learning loss as:
Revised: Building on this result, we can compute the bidirectional learning loss as:

Original: [Equation]
Revised: [Equation]

Original: where K ( X i , X j ) =
Revised: where K ( X i , X j ) =

Original: ▽ θ f θ 

Revised: ▽ θ f θ

Original: ( X i )
Revised: ( X i )

Original: ⊤ ▽ θ f θ 0 ( X j ) .
Revised: ⊤ ▽ θ f θ 0 ( X j ) .

Original: Following (Dukler et al., 2022), we can also only linearize the last layer of the network for simplicity, which defines the following kernel,
Revised: Following (Dukler et al., 2022), we can also only linearize the last layer of the network for simplicity, which defines the following kernel,

Original: [Equation]
Revised: [Equation]

Original: where BERT ( X ) denotes the feature of the sequence X extracted by BERT.
Revised: where BERT ( X ) denotes the feature of the sequence X extracted by BERT.

Original: Its kernel nature makes this approach suitable for small-data tasks (Arora et al., 2020), especially in drug discovery where the labeling cost of DNA/proteins is high. 
Revised: Its kernel nature makes this approach suitable for small-data tasks (Arora et al., 2020), especially in drug discovery where the labeling cost of DNA/proteins is high. Input pre trained LM Output

Original: 3. A DAPTIVE - γ
Revised: Leverage A DAPTIVE - γ

Original: The forward mapping and the backward mapping play different roles in the sequence optimization process: the forward mapping encourages the high-scoring sequence to search for a higher target score (exploitation) and the backward mapping serves as a constraint.
Revised: The forward mapping and the backward mapping play different roles in the sequence optimization process: the forward mapping encourages the high-scoring sequence to search for a higher target score (exploitation) and the backward mapping serves as a constraint.

Original: Since different sequences require different degrees of constraint, we introduce an extra hyperparameter γ ∈ [0 , 1] to control the balance between the corresponding terms in the loss function:
Revised: Since different sequences require different degrees of constraint, we introduce an extra hyperparameter γ ∈ [0 , 1] to control the balance between the corresponding terms in the loss function:

Original: [Equation]
Revised: [Equation]

Original: Thus γ = 1 . 0 corresponds to the forward mapping alone, γ = 0 results in backward mapping, and γ = 0 .
Revised: Thus γ = 1 . 0 corresponds to the forward mapping alone, γ = 0 results in backward mapping, and γ = 0 .

Original: 5 leads to the bidirectional loss of (Chen et al., 2022).
Revised: 5 leads to the bidirectional loss of (Chen et al., 2022).

Original: It is non-trivial to determine the most suitable value for γ since we do not know the ground-truth score for a new design.
Revised: It is non-trivial to determine the most suitable value for γ since we do not know the ground-truth score for a new design.

Original: One possible solution is to train an auxiliary f aux ( · ) to serve as a proxy evaluation.
Revised: One possible solution is to train an auxiliary f aux ( · ) to serve as a proxy evaluation.

Original: A reasonable auxiliary is a simple regression model fitted to the offline dataset.
Revised: A reasonable auxiliary is a simple regression model fitted to the offline dataset.

Original: Although this auxiliary model cannot yield ground-truth scores, it can provide weak supervision signals to update γ , since the auxiliary model and the bidirectional learning provide complementary information.
Revised: Although this auxiliary model cannot yield ground-truth scores, it can provide weak supervision signals to update γ , since the auxiliary model and the bidirectional learning provide complementary information.

Original: This is similar to co-teaching (Han et al., 2018) where two models leverage each other’s view.
Revised: This is similar to co-teaching (Han et al., 2018) where two models leverage each other’s view.

Original: Formally, we introduce the Adaptive - γ framework.
Revised: Formally, we introduce the Adaptive - γ framework.

Original: Given a good choice of γ , the produced X h is expected to have a high score f
Revised: Given a good choice of γ , the produced X h is expected to have a high score f

Original: aux
Revised: aux

Original: ( X h ) , based on which we can choose γ .
Revised: ( X h ) , based on which we can choose γ .

Original: To make the search for γ more efficient, we can formulate this process as a bi-level optimization problem:
Revised: To make the search for γ more efficient, we can formulate this process as a bi-level optimization problem:

Original: [Equation]
Revised: [Equation]

Original:  We can then use the hyper-gradient ∂f aux ( X ∗ h ( γ ))
Revised: [Equation] We can then use the hyper-gradient ∂f aux ( X ∗ h ( γ ))

Original: ∂γ to update γ .
Revised: ∂γ to update γ .

Original: Specifically, the inner level solution can be approximated via a gradient descent step with a learning rate η : 
Revised: Specifically, the inner level solution can be approximated via a gradient descent step with a learning rate η : [Equation]

Original: [Equation]
Revised: [Equation]

Original: For the outer level, we update γ by hyper-gradient ascent:
Revised: For the outer level, we update γ by hyper-gradient ascent:

Original: [Equation]
Revised: [Equation]

Original: where we unroll the matrix X h as a vector x h for better illustration.
Revised: where we unroll the matrix X h as a vector x h for better illustration.

Original: 3. A DAPTIVE - η
Revised: 3. A DAPTIVE - η

Original: We now extend the Adaptive - γ framework to Adaptive - η .
Revised: We now extend the Adaptive - γ framework to Adaptive - η .

Original: As the first learning rate adaptation module for offline model-based optimization, Adaptive - η is compatible with all gradient-based algorithms and can effectively finetune the learning rate η via the auxiliary model’s weak supervision signal.
Revised: As the first learning rate adaptation module for offline model-based optimization, Adaptive - η is compatible with all gradient-based algorithms and can effectively finetune the learning rate η via the auxiliary model’s weak supervision signal.

Original: All gradient-based methods that maximize L θ ( X ) with respect to X have the following general form:
Revised: All gradient-based methods that maximize L θ ( X ) with respect to X have the following general form:

Original: [Equation]
Revised: [Equation]

Original: where η represents the learning rate of the optimizer.
Revised: where η represents the learning rate of the optimizer.

Original: For methods such as simple gradient ascent (Grad), COMs (Trabucco et al., 2021), ROMA (Yu et al., 2021) and NEMO (Fu & Levine, 2021),
Revised: For methods such as simple gradient ascent (Grad), COMs (Trabucco et al., 2021), ROMA (Yu et al., 2021) and NEMO (Fu & Levine, 2021),

Original: L θ ( · ) is related to the proxy model f
Revised: L θ ( · ) is related to the proxy model f

Original: θ ( · ) ; for BDI Chen et al.
Revised: θ ( · ) ; for BDI Chen et al.

Original: and our proposed method, BIB,
Revised: and our proposed method, BIB,

Original: L θ ( · ) is the negative of the bidirectional learning loss, i.e., L θ = −L bi .
Revised: L θ ( · ) is the negative of the bidirectional learning loss, i.e., L θ = −L bi .

Original: Though the learning rate η can be adapted in some optimizers such as Adam (Kingma & Ba, 2015), these adaptations rely on only the past optimization history and do not consider the weak supervision signal from the auxiliary model.
Revised: Though the learning rate η can be adapted in some optimizers such as Adam (Kingma & Ba, 2015), these adaptations rely on only the past optimization history and do not consider the weak supervision signal from the auxiliary model.

Original: Our Adaptive - η optimizes η by solving:
Revised: Our Adaptive - η optimizes η by solving:

Original: [Equation]
Revised: [Equation]

Original: 23) where η can be updated via gradient ascent methods.
Revised: 23) where η can be updated via gradient ascent methods.

Original: Considering the sequence optimization procedure is highly sensitive to the learning rate η , we reset η to η 0 at each iteration and update η from η 0 ,
Revised: Considering the sequence optimization procedure is highly sensitive to the learning rate η , we reset η to η 0 at each iteration and update η from η 0 ,

Original: [Equation]
Revised: [Equation]

Original: In general, this serves to stabilize the optimization procedure.
Revised: In general, this serves to stabilize the optimization procedure.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We conduct extensive experiments on DNA and protein design tasks, and aim to answer three research questions: (1) How does BIB compare with state-of-the-art algorithms?
Revised: We conduct extensive experiments on DNA and protein design tasks, and aim to answer three research questions: (1) How does BIB compare with state-of-the-art algorithms?

Original: (2) Is every design component necessary in BIB?
Revised: (2) Is every design component necessary in BIB?

Original: (3) Does the Adaptive - η module improve gradient-based methods?
Revised: (3) Does the Adaptive - η module improve gradient-based methods?

Original: 4. B ENCHMARK
Revised: 4. B ENCHMARK

Original: We conduct experiments on two DNA tasks following (Chen et al., 2022) and three protein tasks in (Ren et al., 2022) which have the most data points.DNA Task 1 TFBind8(r).
Revised: We conduct experiments on two DNA tasks: TFBind8(r) and TFBind10(r), following (Chen et al., 2022) and three protein tasks: avGFP, AAV and E4B, in (Ren et al., 2022) which have the most data points.

Original: The goal is to find a length- 8 DNA sequence to maximize the binding activity score with a particular transcription factor, SIX6REFR1 (Barrera et al., 2016).

Original: We sample 5000 data points for the offline algorithms following (Chen et al., 2022).

Original: DNA Task 2 TFBind10(r).

Original: The task TFBind10(r) is the same as TFBind8(r) except that the goal is to find a length- 10 DNA sequence.

Original: Both DNA tasks have exact oracles for ground-truth evaluation. Protein Task 1 avGFP. This task aims to find a protein sequence with approximately 239 amino acids to maximize the fluorescence level of Green Fluorescent Proteins (Sarkisyan et al., 2016). The task oracle is constructed by using the full unobserved dataset (around 52,000 points) following (Ren et al., 2022).
Revised: See See Appendix A.2 for details.

Original: The oracle passes the average of the residue embeddings from the pre-trained Prot-T (Elnaggar et al., 2021) into a linear layer and then fits the dataset.

Original: The following two task oracles take the same form.

Original: The offline algorithms can only access the lowest-scoring 26,000 data points.

Original: Protein Task 2 AAV.

Original: The goal is to engineer a 28 -amino acid segment (positions 561 – 588 ) of the

Original: VP1 protein to remain viable for gene therapy (Bryant et al., 2021).

Original: We use the entire 284 , 000 data points to build the oracle and the lowest-scoring 142 , 000 points for the offline algorithms.

Original: Protein Task 3 E4B. This task aims to design a protein (around 102 amino acids) to maximize the ubiquitination rate to the target protein (Starita et al., 2013).

Original: The full dataset consisting of around 100 , 000 points is used to build the oracle and the bottom half is used for the offline algorithms.

Original: Following (Trabucco et al., 2021), we select the top N = 

Revised: Following (Trabucco et al., 2021), we select the top N =

Original: most promising sequences for each comparison method.
Revised: most promising sequences for each comparison method.

Original: Among these sequences, we report the 100 th percentile normalized ground truth score as the evaluation metric following (Ren et al., 2022).
Revised: Among these sequences, we report the maximum normalized ground truth score as the evaluation metric following (Ren et al., 2022).

Original: 4. C OMPARISON METHODS
Revised: 4. C OMPARISON METHODS

Original: We compare BIB with two groups of baselines: the gradient-based methods and the non-gradientbased methods.
Revised: We compare BIB with two groups of baselines: the gradient-based methods and the non-gradientbased methods.

Original: For a fair comparison, the pre-trained LM is used for all methods involving a proxy.
Revised: For a fair comparison, the pre-trained LM is used for all methods involving a proxy and we don’t finetune the LM.

Original: The gradient-based methods include: 1) Grad: gradient ascent on existing sequences to obtain new sequences; 2) COMs (Trabucco et al., 2021): lower bounds the DNN model by the ground-truth values and then applies gradient ascent; 3) ROMA (Yu et al., 2021): incorporates a smoothness prior into the DNN model before gradient ascent steps; 4) NEMO (Fu & Levine, 2021): leverages the normalized maximum-likelihood estimator to bound the distance between the DNN model and the ground-truth values; 5)
Revised: The gradient-based methods include: 1) Grad: gradient ascent on existing sequences to obtain new sequences; 2) COMs (Trabucco et al., 2021): lower bounds the DNN model by the ground-truth values and then applies gradient ascent; 3) ROMA (Yu et al., 2021): incorporates a smoothness prior into the DNN model before gradient ascent steps; 4) NEMO (Fu & Levine, 2021): leverages the normalized maximum-likelihood estimator to bound the distance between the DNN model and the ground-truth values; 5)

Original: BDI (Chen et al., 2022): adopts the infinitely wide neural network and its NTK to yield a closed-form bidirectional learning loss.
Revised: BDI (Chen et al., 2022): adopts the infinitely wide neural network and its NTK to yield a closed-form bidirectional learning loss.

Original: The non-gradient-based methods include: 1) BO-qEI (Wilson et al., 2017): builds an acquisition function for sequence exploration; 2) CMA-ES (Hansen, 2006): estimates the covariance matrix to adjust the sequence distribution towards the high-scoring region;
Revised: The non-gradient-based methods include: 1) BO-qEI (Wilson et al., 2017): builds an acquisition function for sequence exploration; 2) CMA-ES (Hansen, 2006): estimates the covariance matrix to adjust the sequence distribution towards the high-scoring region;

Original: 3) AdaLead (Sinai et al., 2020): performs a hill-climbing search on the proxy and then queries the sequences with high predictions; 4)
Revised: 3) AdaLead (Sinai et al., 2020): performs a hill-climbing search on the proxy and then queries the sequences with high predictions; 4)

Original: CbAS (Brookes et al., 2019): builds a generative model for sequences above a property threshold and gradually adapts the distribution by increasing the threshold; 5) PEX (Ren et al., 2022): prioritizes the evolutionary search for protein sequences with low mutation counts; 6) GENH (Chan et al., 2021): enhances the score through a learned latent space.
Revised: CbAS (Brookes et al., 2019): builds a generative model for sequences above a property threshold and gradually adapts the distribution by increasing the threshold; 5) PEX (Ren et al., 2022): prioritizes the evolutionary search for protein sequences with low mutation counts; 6) GENH (Chan et al., 2021): enhances the score through a learned latent space.

Original: 4. T RAINING D ETAILS
Revised: 4. T RAINING D ETAILS

Original: We follow the training setting in (Chen et al., 2022) if not specified.
Revised: We follow the training setting in (Chen et al., 2022) if not specified.

Original: We choose OPT as the Adam optimizer (Kingma & Ba, 2015) for all gradient-based methods.
Revised: We choose OPT as the Adam optimizer (Kingma & Ba, 2015) for all gradient-based methods.

Original: We implement the auxiliary model  as a linear layer with the feature from the pre-trained LM.
Revised: We implement the auxiliary model [Table] as a linear layer with the feature from the pre-trained LM.

Original: We set the number of iterations T as 25 for all experiments following (Norn et al., 2021) and η 0 as 0 .
Revised: We set the number of iterations T as 25 for all experiments following (Norn et al., 2021) and η 0 as 0 .

Original: 1 following (Chen et al., 2022).
Revised: 1 following (Chen et al., 2022).

Original: We run every setting over 16 trials and report the average score.
Revised: We run every setting over 16 trials and report the average score.

Original: See Appendix A.2 for other details.
Revised: See Appendix A.3 for other details [Table]

Original: 4. R ESULTS AND A NALYSIS
Revised: 4. R ESULTS AND A NALYSIS

Original: We report all experimental results in Table 1 and plot the ranking statistics in Figure 1.
Revised: We report all experimental results in Table 1 and plot the ranking statistics in Figure 2.

Original: We make the following observations.
Revised: We make the following observations.

Original: (1) As shown in Table 1, BIB consistently outperforms the Grad method on all tasks, which demonstrates that our BIB can effectively mitigate the out-of-distribution issue.
Revised: (1) As shown in Table 1, BIB consistently outperforms the Grad method on all tasks, which demonstrates that our BIB can effectively mitigate the out-of-distribution issue.

Original: Furthermore, BIB outperforms BDI on 4 out of 5 tasks, which demonstrates the effectiveness of the pre-trained biological LM over NTK.
Revised: Furthermore, BIB outperforms BDI on 4 out of 5 tasks, which demonstrates the effectiveness of the pre-trained biological LM over NTK.

Original: The reason why BDI outperforms BIB on TFBind10(r) may be that short sequences do not rely much on the rich sequential information from the pre-trained LM.
Revised: The reason why BDI outperforms BIB on TFBind10(r) may be that short sequences do not rely much on the rich sequential information from the pre-trained LM.

Original: (3) As shown in Figure 1, the gradient-based methods generally perform better than the non-gradient-based methods, as also observed by Trabucco et al.
Revised: (3) As shown in Figure 2, the gradient-based methods generally perform better than the non-gradient-based methods, as also observed by Trabucco et al.

Original: [Table]
Revised: [Figure]

Revised: (4) The gradient-based methods are inferior for the AAV task.

Revised: One possible reason is that the design space of AAV ( 20 28 ) is much smaller than those of avGFP ( 20 239 ) and E4B ( 20 102 ), which makes the generative modeling and evolutionary algorithms more suitable.

Revised: This conjecture is also supported by the experimental results on two DNA design tasks.

Revised: We compute the average ranking of gradient-based methods and non-grad-based methods on TFBind10(r) as 3 . 5 and 9 . 5 , respectively, and the average ranking of gradient-based methods and non-grad-based methods on TFBind8(r) as 5 . 8 and 6 . 8 , respectively.

Revised: The advantage of gradient-based methods are larger ( 9 . 5 − 3 .

Original:  BIB (ours) 0.
Revised: 5 = 6 . 0 ) in TFBind10(r) than that ( 6 . 8 − 5 . 8 = 1 . 0 ) in TFBind8(r). The generative modeling methods CbAS and GENH yield poor results on all tasks, probably because the high-dimensional data distribution is very hard to model. (7) Overall, BIB attains the best performance in 3 out of 5 tasks and achieves the best ranking results as shown in Table 1 and Figure 2.

Revised: We also visualize the trend of performance (the maximum normalized ground truth score) and tradeoff γ as a function of T on TFBind8(r) in Figure 3(a) and avGFP in Figure 3(b).

Revised: The performance generally increases with the time step T and then stabilizes, which demonstrates the effectiveness and robustness of BIB.

Revised: Furthermore, we find that the γ values of TFBind8(r) and avGFP generally increase at first.

Revised: This means that BIB reduces the impact of the constraint to encourage a more aggressive search for a high target value during the initial phase.

Revised: Then γ of TFBind8(r) continues to increase while the γ of avGFP decreases.

Revised: We conjecture that the difference is caused by the sequence length.

Revised: Small mutations of a biological sequence are enough to yield a good candidate (Ren et al., 2022).

Revised: For the length- 239 protein in avGFP, dramatic mutations 1) are not necessary and 2) can easily lead to out-of-distribution points.

Revised: The weak supervision signal from the auxiliary model therefore encourages a tighter constraint towards the static dataset.

Revised: By contrast, the DNA sequence is relatively short and a more widespread search of the sequence space can yield better results.

Revised: To investigate this conjecture, we further visualize the trend of E4B in Figure 3(c).

Revised: E4B also has long sequences ( 102 ) and we can observe its similar first-increase-then-decrease trend, although it is not as pronounced.

Original: 0 / 40 T 0.
Revised: 40 T 0.

Original: P e
Revised: P e

Original: r
Revised: r

Original: f
Revised: f

Original: o r m a n c e 0.
Revised: o r m a n c e 0.

Original: T r a d e - o ff
Revised: T r a d e - o ff

Original: PerformanceTrade-off (a) TFBind8(r).
Revised: PerformanceTrade-off (a) TFBind8(r).

Original: 40 T
Revised: 40 T

Original: P e
Revised: P e

Original: r
Revised: r

Original: f
Revised: f

Original: o r m a n c e 0.
Revised: o r m a n c e 0.

Original: T r a d e - o ff
Revised: T r a d e - o ff

Original: PerformanceTrade-off (b) avGFP.
Revised: PerformanceTrade-off (b) avGFP.

Original: 40 T 0.
Revised: 40 T 0.

Original: P e
Revised: P e

Original: r
Revised: r

Original: f
Revised: f

Original: o r m a n c e 0.
Revised: o r m a n c e 0.

Original: T r a d e - o ff
Revised: T r a d e - o ff

Original: PerformanceTrade-off (c) E4B.
Revised: PerformanceTrade-off (c) E4B.

Original: [Figure]
Revised: [Table]

Original: (4) The gradient-based methods are inferior for the AAV task.

Original: One possible reason is that the design space of AAV ( 20 28 ) is much smaller than those of avGFP ( 20 239 ) and E4B ( 20 102 ), which makes the generative modeling and evolutionary algorithms more suitable.

Original: This conjecture is also supported by the experimental results on two DNA design tasks: the advantage of gradient-based methods over non-gradient-based methods is greater for TFBind10(r) than TFBind8(r).

Original: The generative modeling methods CbAS and GENH yield poor results on all tasks, probably because the high-dimensional data distribution is very hard to model.

Original: (7) Overall, BIB attains the best performance in 3 out of 5 tasks and achieves the best ranking results as shown in Table 1 and Figure 1.

Original: We also visualize the trend of performance (the 100 th percentile normalized ground truth score) and trade-off γ as a function of T on TFBind8(r) in Figure 2(a) and avGFP in Figure 2(b).

Original: The performance generally increases with the time step T and then stabilizes, which demonstrates the effectiveness and robustness of BIB.

Original: Furthermore, we find that the γ values of TFBind8(r) and avGFP generally increase at first.

Original: This means that BIB reduces the impact of the constraint to encourage a more aggressive search for a high target value during the initial phase.

Original: Then γ of

Original: TFBind8(r) continues to increase while the γ of avGFP decreases. We conjecture that the difference is caused by the sequence length. Small mutations of a biological sequence are enough to yield a good candidate (Ren et al., 2022). For the length- 239 protein in avGFP, dramatic mutations 1) are not necessary and 2) can easily lead to out-of-distribution points.
Revised: 4. A BLATION STUDIES

Original: The weak supervision signal from the auxiliary model therefore encourages a tighter constraint towards the static dataset. By contrast, the DNA sequence is relatively short and a more widespread search of the sequence space can yield better results. To investigate this conjecture, we further visualize the trend of E4B in Figure 2(c). E4B also has long sequences ( 102 ) and we can observe its similar first-increase-then-decrease trend, although it is not as pronounced.
Revised: In this subsection

Original: [Table]
Revised: [Table]

Original: In this subsection, we conduct ablation studies to verify the effectiveness of the forward mapping, the backward mapping, and the Adaptive - γ module of BIB.
Revised: we conduct ablation studies to verify the effectiveness of the forward mapping, the backward mapping, and the Adaptive - γ module of BIB.

Original: We report the experimental results in Table 2.
Revised: We report the experimental results in Table 2.

Original: Forward mapping & Backward mapping.
Revised: Forward mapping & Backward mapping.

Original: We can observe that bidirectional learning ( γ = 0 .
Revised: We can observe that bidirectional learning ( γ = 0 .

Original: 5 ) performs better than both forward mapping ( γ = 1 . 0 ) and backward mapping ( γ = 0 . 0 ) alone in most tasks, which demonstrates the effectiveness of forward mapping and backward mapping.
Revised: 5 ) performs better than both forward mapping ( γ = 1 . 0 ) and backward mapping ( γ = 0 . 0 ) alone in most tasks, which demonstrates the effectiveness of forward mapping and backward mapping.

Original: The advantage of bidirectional mappings over the forward mapping is larger in the long-sequence tasks like avGFP (238) and E4B (102) compared with the short-sequence tasks.
Revised: The advantage of bidirectional mappings over the forward mapping is larger in the long-sequence tasks like avGFP (238) and E4B (102) compared with the short-sequence tasks.

Original: A possible explanation is that the constraint is more important for long sequence tasks than short sequence design since the search space is large and many mutations can easily go out of distribution.
Revised: A possible explanation is that the constraint is more important for long sequence tasks than short sequence design since the search space is large and many mutations can easily go out of distribution.

Original: Adaptive-
Revised: Adaptive-

Original: γ .
Revised: γ .

Original: BIB learns γ and this leads to improvements over bidirectional mappings ( γ = 0 .
Revised: BIB learns γ and this leads to improvements over bidirectional mappings ( γ = 0 .

Original: for all tasks, verifying the effectiveness of Adaptive- γ .
Revised: for all tasks, verifying the effectiveness of Adaptive- γ .

Original: We also consider the following variant,
Revised: We also consider the following variant,

Original: [Equation]
Revised: [Equation]

Original: (25) which jointly optimizes the bidirectional learning loss L bi ( X h , 0 . 5) and the auxiliary term f aux ( X h ) .
Revised: which jointly optimizes the bidirectional learning loss L bi ( X h , 0 . 5) and the auxiliary term f aux ( X h ) .

Original: We found this yields similar or even worse results than pure bidirectional learning.
Revised: We found this yields similar or even worse results than pure bidirectional learning.

Original: The reason may be that the weak supervision signal from f aux
Revised: The reason may be that the weak supervision signal from f aux

Original: ( X h ) can serve as a guide to update the scalar γ but not as a component of the main optimization objective that directly updates the sequence.
Revised: ( X h ) can serve as a guide to update the scalar γ but not as a component of the main optimization objective that directly updates the sequence.

Original: In the final column of Table 2, we examine the performance of the Adaptive- η module.
Revised: In the final column of Table 2, we examine the performance of the Adaptive- η module.

Original: Adding this module leads to improvements on all five tasks, which demonstrates its effectiveness.
Revised: Adding this module leads to improvements on all five tasks, which demonstrates its effectiveness.

Original: 4. A DAPTIVE - η
Revised: 4. A DAPTIVE - η

Original: In this subsection, we aim to further demonstrate the effectiveness of the Adaptive- η module on all six gradient-based methods.
Revised: In this subsection, we aim to further demonstrate the effectiveness of the Adaptive- η module on all six gradient-based methods.

Original: We conduct experiments on two tasks: TFBind8(r) and avGFP.
Revised: We conduct experiments on two tasks: TFBind8(r) and avGFP.

Original: Since the use of the infinitely wide neural network leads to poor performance for BDI, we modify its implementation via deep linearization so that it can make use of the pre-trained LM.
Revised: Since the use of the infinitely wide neural network leads to poor performance for BDI, we modify its implementation via deep linearization so that it can make use of the pre-trained LM.

Original: As shown in Table 3, Adaptive - η provides a consistent gain for all scenarios, which demonstrates the widespread applicability and effectiveness of the module.
Revised: As shown in Table 3, Adaptive - η provides a consistent gain for all scenarios, which demonstrates the widespread applicability and effectiveness of the module.

Original: Furthermore, Adaptive - η leads to a maximum improvement of 1 . 4% in TFBind8(r) and 12 .
Revised: Furthermore, Adaptive - η leads to a maximum improvement of 1 . 4% in TFBind8(r) and 12 .

Original: 5% in avGFP.
Revised: 5% in avGFP.

Original: ROMA is the algorithm that benefits the most.
Revised: ROMA is the algorithm that benefits the most.

Original: One possible explanation is that ROMA incorporates a local smoothness prior that leads to more stable gradients, with which Adaptive - η can be more effective.
Revised: One possible explanation is that ROMA incorporates a local smoothness prior that leads to more stable gradients, with which Adaptive - η can be more effective.

Original: Similar to Sec 4.5, we  consider the following variant,
Revised: Similar to Sec 4.5, we [Table] consider the following variant,

Original: [Equation]
Revised: [Equation]

Original: (26) which performs joint optimization instead of bi-level optimization on two objectives.
Revised: (26) which performs joint optimization instead of bi-level optimization on two objectives.

Original: As shown in Table 3, joint optimization generally deteriorates the performance.
Revised: As shown in Table 3, joint optimization generally deteriorates the performance.

Original: This again verifies that the auxiliary model can only serve as a guide instead of contributing to the main objective.
Revised: This again verifies that the auxiliary model can only serve as a guide instead of contributing to the main objective [Table]

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Biological sequence design.
Revised: Biological sequence design.

Original: There has been a wide range of algorithms for biological sequence design.
Revised: There has been a wide range of algorithms for biological sequence design.

Original: Evolutionary algorithms (Sinai et al., 2020; Ren et al., 2022) leverage the learned surrogate model to provide evolution guidance towards the high-scoring region.
Revised: Evolutionary algorithms (Sinai et al., 2020; Ren et al., 2022) leverage the learned surrogate model to provide evolution guidance towards the high-scoring region.

Original: Angermueller et al.
Revised: Angermueller et al.

Original: [Table] propose a flexible reinforcement learning framework where sequence design is a sequential decisionmaking problem.
Revised: (2019) propose a flexible reinforcement learning framework where sequence design is a sequential decisionmaking problem.

Original: Bayesian optimization methods propose candidate solutions via an acquisition function (Terayama et al., 2021).
Revised: Bayesian optimization methods propose candidate solutions via an acquisition function (Terayama et al., 2021).

Original: Deep generative model methods design sequences in the latent space (Chan et al., 2021) or gradually adapt the distribution towards the high-scoring region (Brookes et al., 2019).
Revised: Deep generative model methods design sequences in the latent space (Chan et al., 2021) or gradually adapt the distribution towards the high-scoring region (Brookes et al., 2019).

Original: GFlowNets (Jain et al., 2022) amortize the cost of search over learning and encourage diversity.
Revised: GFlowNets (Jain et al., 2022) amortize the cost of search over learning and encourage diversity.

Original: Gradient-based methods leverage a surrogate model and its gradient information to maximize the desired property (Chen et al., 2022; Norn et al., 2021; Tischer et al., 2020; Linder &
Revised: Gradient-based methods leverage a surrogate model and its gradient information to maximize the desired property (Chen et al., 2022; Norn et al., 2021; Tischer et al., 2020; Linder &

Original: Seelig, 2020).
Revised: Seelig, 2020).

Original: Our proposed BIB belongs to the last category and leverages the rich biophysical information (Ji et al., 2021; Elnaggar et al., 2021) to directly optimize the biological sequence.
Revised: Our proposed BIB belongs to the last category and leverages the rich biophysical information (Ji et al., 2021; Elnaggar et al., 2021) to directly optimize the biological sequence.

Original: Offline model-based optimization.
Revised: Offline model-based optimization.

Original: A majority of sequence design algorithms (Angermueller et al., 2019; Sinai et al., 2020; Ren et al., 2022) focus on the online setting where wet-lab experimental results in the current round are analyzed to propose candidates in the next round.
Revised: A majority of sequence design algorithms (Angermueller et al., 2019; Sinai et al., 2020; Ren et al., 2022) focus on the online setting where wet-lab experimental results in the current round are analyzed to propose candidates in the next round.

Original: The problem of this setting is that wet-lab experiments are often very expensive, and thus a pure data-driven, offline approach is attractive and has received substantial research attention recently (Trabucco et al., 2022;
Revised: The problem of this setting is that wet-lab experiments are often very expensive, and thus a pure data-driven, offline approach is attractive and has received substantial research attention recently (Trabucco et al., 2022;

Original: Kolli et al., 2022).
Revised: Kolli et al., 2022).

Original: Gradient-based methods have proven to be effective (Trabucco et al., 2021;
Revised: Gradient-based methods have proven to be effective (Trabucco et al., 2021;

Original: Yu et al., 2021; Fu & Levine, 2021;
Revised: Yu et al., 2021; Fu & Levine, 2021;

Original: Chen et al., 2022).
Revised: Chen et al., 2022).

Original: Among these algorithms, Chen et al.
Revised: Among these algorithms, Chen et al.

Original: (2022) propose bidirectional mappings to distill information from the static dataset into a high-scoring design, which achieves state-of-the-art performances on a variety of tasks.
Revised: (2022) propose bidirectional mappings to distill information from the static dataset into a high-scoring design, which achieves state-of-the-art performances on a variety of tasks.

Original: However, this bidirectional learning is designed for general tasks, like robot and material design, and the rich biophysical information in millions of biological sequences is ignored.
Revised: However, this bidirectional learning is designed for general tasks, like robot and material design, and the rich biophysical information in millions of biological sequences is ignored.

Original: In this paper, we leverage recent advances in deep linearization to incorporate the rich biophysical information into bidirectional learning.
Revised: In this paper, we leverage recent advances in deep linearization to incorporate the rich biophysical information into bidirectional learning.

Original: Controlled text generation.

Original: A related field of work is controlled text generation, which aims to generate a natural text sequence with a predefined attribute.

Original: PPLM (Dathathri et al., 2020) combines a pre-trained LM with attribute classifiers to guide sequence generation by changing the hidden states.

Original: Gedi (Krause et al., 2020) guides the generation of large LMs at each step by leveraging the supervision signals from the small LMs.

Original: As pointed out in (Chan et al., 2021), the above methods cannot be directly applied to our task setting since they require the attribute to be within the training distribution.

Original: GENH (Chan et al., 2021) utilizes a regularized encoder-decoder framework and manipulates the learned latent space to generate desired sequences.

Original: Bi-level optimization for hyperparameter tuning.

Original: Bi-level optimization has been widely used in hyperparameter tuning including learning rate (Franceschi et al., 2017), regularization (Franceschi et al., 2018), etc.

Original: In this paper, we study tuning hyperparameters in offline model-based optimization, which, to the best of our knowledge, has not been investigated in previous work.

Original: We leverage the weak supervision signal from an auxiliary model to update the hyperparameters via bi-level optimization.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose bidirectional learning for offline model-based biological sequence.
Revised: In this paper, we propose bidirectional learning for offline model-based biological sequence.

Original: Our work is built on the recently proposed bidirectional learning approach (Chen et al., 2022), which is designed for general inputs and relies on the NTK of an infinitely wide network to yield a closed-form loss computation.
Revised: Our work is built on the recently proposed bidirectional learning approach (Chen et al., 2022), which is designed for general inputs and relies on the NTK of an infinitely wide network to yield a closed-form loss computation.

Original: Though effective, the NTK cannot learn features.
Revised: Though effective, the NTK cannot learn features.

Original: We build a proxy model using the pre-trained LM model with a linear head and apply the deep linearization scheme to the proxy, which can yield a closed-form loss and incorporate the wealth of biophysical information at the same time.
Revised: We build a proxy model using the pre-trained LM model with a linear head and apply the deep linearization scheme to the proxy, which can yield a closed-form loss and incorporate the wealth of biophysical information at the same time.

Original: In addition, we propose Adaptive - γ to maintain a proper balance between the forward mapping and the backward mapping by leveraging the weak supervision signal from an auxiliary model.
Revised: In addition, we propose Adaptive - γ to maintain a proper balance between the forward mapping and the backward mapping by leveraging the weak supervision signal from an auxiliary model.

Original: Based on this framework, we further propose Adaptive - η , the first learning rate adaptation strategy compatible with all gradient-based offline model-based algorithms.
Revised: Based on this framework, we further propose Adaptive - η , the first learning rate adaptation strategy compatible with all gradient-based offline model-based algorithms.

Original: Experimental results on DNA and protein sequence design tasks verify the effectiveness of BIB and Adaptive - η .
Revised: Experimental results on DNA and protein sequence design tasks verify the effectiveness of BIB and Adaptive - η .

Original:  E THICS S TATEMENT
Revised:  E THICS S TATEMENT

Original: Protein sequence design aims to find a protein sequence with a particular biological function, which has a broad application scope.
Revised: Protein sequence design aims to find a protein sequence with a particular biological function, which has a broad application scope.

Original: This can lead to improved drugs that are highly beneficial to society.
Revised: This can lead to improved drugs that are highly beneficial to society.

Original: For instance, designing the antibody protein for SARS-COV-2 can potentially save millions of human lives (Kumar et al., 2021) and designing novel anti-microbial peptides (short protein sequences) is central to tackling the growing public health risks caused by anti-microbial resistance (Murray et al., 2022).
Revised: For instance, designing the antibody protein for SARS-COV-2 can potentially save millions of human lives (Kumar et al., 2021) and designing novel anti-microbial peptides (short protein sequences) is central to tackling the growing public health risks caused by anti-microbial resistance (Murray et al., 2022).

Original: Unfortunately, it is possible to direct the research results towards harmful purposes such as the design of biochemical weapons.
Revised: Unfortunately, it is possible to direct the research results towards harmful purposes such as the design of biochemical weapons.

Original: As researchers, we believe that we must be aware of the potential harm of any research outcomes, and carefully consider whether the possible benefits outweigh the risks of harmful consequences.
Revised: As researchers, we believe that we must be aware of the potential harm of any research outcomes, and carefully consider whether the possible benefits outweigh the risks of harmful consequences.

Original: We also must recognize that we cannot control how the research may be used.
Revised: We also must recognize that we cannot control how the research may be used.

Original: In the case of this paper, we are confident that there is a much greater chance that the research outcomes will have a beneficial effect.
Revised: In the case of this paper, we are confident that there is a much greater chance that the research outcomes will have a beneficial effect.

Original: We do not consider that there are any immediate ethical concerns with the research endeavour.
Revised: We do not consider that there are any immediate ethical concerns with the research endeavour.

Original:  R EPRODUCIBILITY S TATEMENT
Revised:  R EPRODUCIBILITY S TATEMENT

Original: We provide the code implementation of BIB and Adaptive - η here and we also attach the code in the supplementary material.
Revised: We provide the code implementation of BIB and Adaptive - η here and we also attach the code in the supplementary material.

Original: We describe the DNA/protein benchmarks in Sec.
Revised: We describe the DNA/protein benchmarks in Sec.

Original: 4.1 and the training details in Sec. 4.3.
Revised: 4.1 and the training details in Sec. 4.3.

Original: We also explain how to obtain the sequence embedding from the pre-trained LM and how to perform gradient ascent steps on the sequence in Sec. 2.
Revised: We also explain how to obtain the sequence embedding from the pre-trained LM and how to perform gradient ascent steps on the sequence in Sec. 2.

Original: A A PPENDIX A. DNA E MBEDDING
Revised: A A PPENDIX A. DNA E MBEDDING

Original: To incorporate richer contextual information, the DNA LM Ji et al.
Revised: To incorporate richer contextual information the DNA LM Ji et al.

Original: (2021) adopts the k -mer sequence representation, which is widely used in DNA sequence analysis.
Revised: (2021) adopts the k -mer sequence representation, which is widely used in DNA sequence analysis.

Original: For example, the sequence
Revised: For example, the sequence

Original: ATGGCT has its 3 -mer representation as { ATG, TGG, GGC, GCT } .
Revised: ATGGCT has its 3 -mer representation as { ATG, TGG, GGC, GCT } .

Original: In this paper, we adopt its 3 -mer representation and compute the probability of the 3 -mer token by multiplying the probabilities of the three individual bases.
Revised: In this paper, we adopt its 3 -mer representation and compute the probability of the 3 -mer token by multiplying the probabilities of the three individual bases.

Original: The 3 -mer representation is then sent to the pre-trained DNA LM.
Revised: The 3 -mer representation is then sent to the pre-trained DNA LM.

Original: A. T RAINING DETAILS
Revised: A. D ATASET DETAILS

Revised: We conduct experiments on two DNA tasks following (Chen et al., 2022) and three protein tasks in (Ren et al., 2022) which have the most data points.

Revised: We report the dataset details in Table 4.

Revised: DNA Task 1 TFBind8(r).

Revised: The goal is to find a length- 8 DNA sequence to maximize the binding activity score with a particular transcription factor, SIX6REFR1 (Barrera et al., 2016).

Revised: We sample 5000 data points for the offline algorithms following (Chen et al., 2022).

Revised: DNA Task 2 TFBind10(r).

Revised: The task TFBind10(r) is the same as TFBind8(r) except that the goal is to find a length- 10 DNA sequence.

Revised: Both DNA tasks measure the entire search space and we adopt these measurements as the approximate ground-truth evaluation.

Revised: Protein Task 1 avGFP.

Revised: This task aims to find a protein sequence with approximately 239 amino acids to maximize the fluorescence level of Green Fluorescent Proteins (Sarkisyan et al., 2016).

Revised: The task oracle is constructed by using the full unobserved dataset (around 52,000 points) following (Ren et al., 2022).

Revised: The oracle passes the average of the residue embeddings from the pre-trained Prot-T (Elnaggar et al., 2021) into a linear layer and then fits the dataset.

Revised: The following two task oracles take the same form.

Revised: The offline algorithms can only access the lowest-scoring 26,000 data points.

Revised: Protein Task 2 AAV.

Revised: The goal is to engineer a 28 -amino acid segment (positions 561 – 588 ) of the

Revised: VP1 protein to remain viable for gene therapy (Bryant et al., 2021).

Revised: We use the entire 284 , 000 data points to build the oracle and the lowest-scoring 142 , 000 points for the offline algorithms.

Revised: Protein Task 3 E4B. This task aims to design a protein (around 102 amino acids) to maximize the ubiquitination rate to the target protein (Starita et al., 2013).

Revised: The full dataset consisting of around 100 , 000 points is used to build the oracle and the bottom half is used for the offline algorithms.

Revised: The parameterization of the oracle is different from that of the regression model from two aspects:

Revised: 1) model architecture; 2) pre-trained information source.

Revised: First, the

Revised: oracle adopts the Prot-T5 model

Revised: [Table]

Revised: [Table]

Revised: which consists of an encoder and a decoder

Revised: [Table]

Revised: while the regression model adopts the Prot-BERT model which only has an encoder.

Revised: Second

Revised: [Table]

Revised: Prot-T5 is trained on the BFD and UniRef100 datasets and

Revised: ProtBert is trained on the UniRef50 dataset.

Revised: These two points demonstrate that the oracle and the regression model are different function classes.

Revised: We choose the Prot-T5 model as the oracle because this is the state-of-the-art protein LM to extract features and recent work Elnaggar et al.

Revised: (2021) has demonstrated its effectiveness.

Revised: In order to test how related the Prot-T5 (oracle)/Prot-BERT(proxy) models are, we trained them on a sampled training dataset and compared the test predictions of the testing set.

Revised: By evaluating the Pearson correlation coefficient (PCC) between the two prediction errors PCC(ProtT5 predictions - test labels, ProtBERT predictions -test labels), we obtain − 0 .on

Revised: avGFP, − 0 .

Revised: 0005 on AAV, and − 0 .

Revised: 0062 on E4B. These results suggest that the two models are not strongly related in terms of the predictions they form.

Original: {+Following (Trabucco et al., 2021), we select the top N = 
+}
Revised: {+Following (Trabucco et al., 2021), we select the top N = 
+}

Revised: most promising sequences for each comparison method.

Revised: Among these sequences, we report the maximum normalized ground truth score as the evaluation metric following (Ren et al., 2022).

Revised: A. T RAINING DETAILS

Original: We use Pytorch (Paszke et al., 2019) to run all experiments on one V100 GPU.
Revised: We use Pytorch (Paszke et al., 2019) to run all experiments on one V100 GPU.

Original: Following the setting in Norn et al.
Revised: Following the setting in Norn et al.

Original: (2021), we introduce a length- L protein sequence as a continuous random matrix
Revised: (2021), we introduce a length- L protein sequence as a continuous random matrix

Original: X h ∈ R L × 20 ( X h ∈ R L × 4 for DNA), initialized using a normal distribution with the mean 0 and the standard deviation of 0 . 01 .
Revised: X h ∈ R L × 20 ( X h ∈ R L × 4 for DNA), initialized using a normal distribution with the mean 0 and the standard deviation of 0 . 01 .

Original: To make this sequence correspond correctly to the candidate sequence, we exchange the largest value in X [ l, :] with the value in the amino acid index. 
Revised: To make this sequence correspond correctly to the candidate sequence, we exchange the largest value in X [ l, :] with the value in the amino acid index. A. D IFFERENT P RETRAINED LM S

Revised: As shown in Table 5, we have tested the ProtBERT, ProtAlbert, and ProtBert-BFD models and found that better-quality models generally work better.

Revised: The publicly available pre-trained DNA models are limited and thus we only perform experiments on the protein tasks.

Revised: Elnaggar et al.

Revised: (2021) demonstrate that the language model performances follow the ordering: ProtBert-BFD > ProtBert >

Revised: ProtAlbert.

Revised: We can see that the performance ranks over the three protein tasks avGFP, AAV, and E4B are the same.

Revised: A. D IFFERENT D ATASET S IZE

Revised: As shown in Table 6, we have tested the performance of BDI as a function of dataset size (N= 20 , 40 , 60 , 80 , 100 ) in TFBind8(r) and TFBind10(r) since they have exact oracle evaluations.

Revised: We see that performance is already good for N= 20 for TFBind8(r) and N= 40 for TFBind10(r).

Revised: A. R ANKING PERFORMANCE

Revised: As for prediction performances, the rank should be: a NN > linearized pre-trained LM > NTK.

Revised: We have conducted experiments to verify this.

Revised: We sample half of the data, train a model to predict another

Revised: [Table]

Revised: [Table]

Revised: half data, and report the mean squared loss here and in Appendix A.6 Table 7.

Revised: A small mean squared loss indicates a good prediction performance; thus, we have verified the above ranking order.