Unifying Likelihood-free Inference with Black-box Optimization and Beyond
Authors: Dinghuai Zhang
Jie Fu
Yoshua Bengio
Aaron Courville
Venue: ICLR 2022 Spotlight (2022)
Keywords: biological sequence design
black-box optimization
likelihood-free inference
Bayesian inference

Abstract: Black-box optimization formulations for biological sequence design have drawn recent attention due to their promising potential impact on the pharmaceutical industry. In this work, we propose to unify two seemingly distinct worlds: likelihood-free inference and black-box optimization, under one probabilistic framework. In tandem, we provide a recipe for constructing various sequence design methods based on this framework. We show how previous optimization approaches can be "reinvented" in our framework, and further propose new probabilistic black-box optimization algorithms. Extensive experiments on sequence design application illustrate the benefits of the proposed methodology.

Published as a conference paper at ICLR 2022
UNIFYING LIKELIHOOD-FREB INFERENCE WITH BLACK-BOX OPTIMIZATION AND BEYOND
Dinghuai Zhang-? Jie Fu Yoshua Bengio Aaron Courville Mila, University of Montreal, &CIFAR Fellow Montreal, Canada
{dinghuai. zhang, fujic] la. quebec
ABSTRACT
Black-box optimization formulations for biological sequence design have drawn recent attention due to their promising potential impact on the pharmaceutical industry. In this work, we propose to unify two seemingly distinct worlds: likelihood-free inference and black-box optimization, under one probabilistic framework. In tandem, design methods based on this framework. We show how previous optimization approaches can be "reinvented" in our framework, and further propose new prob- abilistic black-box optimization algorithms. Extensive experiments on sequence design application illustrate the benefits of the proposed methodology.
provide recipe for constructing various sequence
INTRODUCTION
Discovering new drugs to fulfill specific criteria, such as binding affinity towards a given molecular target, is fundamental problem in chemistry and the pharmaceutical industry (Hughes al., 201 I). In this work, we focus on an important subdomain: de novo biological sequence design. This task is challenging for two reasons: (1) the exploration space for sequences is combinatorially large; and (2) sequence usefulness is evaluated via a complicated process which usually involves time-consuming and expensive wet-lab experiments.
Despite the difficulty of this task, many approaches have been developed over the past few decades thanks to recent advances in biochemistry and machine learning. The Nobel Prize wining paradigm, directed evolution (Chen & Amold, 1991), which conducts local evolutionary search under human guidance, is one of the popular techniques. Unfortunately, and reliance on strong prior knowledge, e.g., about where to mutate (Ahn et al., 2020). Further- more, to compete with other machine learning methods (Gottipati et al., 2020), guided evolution (Yoshikawa et al., 2018; Jensen, 2019; Nigam et al., 2019) heavily relies on human intuition for designing domain -specff evolutionary operators, which may not always apply to tasks at hand.
limited by sample inefficiency
In this work, we deem sequence design to be black-box optimization problem, tasked with maxi- mizing an unknown oracle function. We assume that oracle queries are limited due to the constraint on resources, such the budgets for evaluating queries in wet-lab. Thus, sample efficiency is crucial. We develop probabilistic framework by reformulating the aforementioned black-box opti- mization target posterior modeling problem. With this framework, we draw surprising connec- tion between likelihood-free inference and sequence design, and thus linking two fields which are previously considered as unrelated. The key observation we leverage here for establishing this con- nection is that both settings share similar elements and targets which will be elaborated in Section 2.2. This connection facilitates our understanding of both fields and provides a recipe for developing sequence design algorithms. Going beyond, we also combine different probabilistic modeling in- sights and develop three novel composite probabilistic algorithms. We point out that our framework could actually be applied to any black-box optimization settings, but in this work we focus on its application to biological sequence design.
To demonstrate the empirical effectiveness of our methods, conduct systematical experiments to evaluate their performance on four in-silico sequence design benchmarks. Our proposed meth-
"Correpoondngg Author

---

Published as a conference paper at ICLR 2022
ods achieve at least comparable results to existing baselines, and the proposed composite methods behave consistently better than all other ones across various sequence design tasks.
We summarize contribution follows:
We develop probabilistic framework that unifies likelihood- free inference and black-box optimization.
Based on this framework, we provide recipe for designing algorithms for black- box prob- lems. We apply these ideas to propose a series of composite design algorithms.
We perform systematical evaluation on series of black-box sequence design benchmarks, and find that these algorithms achieves consistently comparable better results compared to previous ones, thus illustrating the benefit of the proposed unified framework.
A UNIFYING PROBABILISTIC FRAMEWORK
2.1 BACKGROUND
Likelihood-free inference (LFI). We use 0 θ and x E X' to separately denote the parameters and the data generated via the mechanism x p(x(0). In this scenario, LFI refers special kind of Bayesian inference setting where the likelihood function is not tractable but sampling (by sim- ulation) from the likelihood feasible. Consider the objective of modeling the Bayesian posterior when we cannot compute the likelihood ()):
p(0)x.) & p(0) p(x=10).
(1)
where X. is the observed data, p(0) is the (given) prior over the model parameters 0, p(x(0) is the intractable likelihood function and p(0) (f)x) is the desired posterior over O. While we do not have access to the exact likelihood, we can still simulate (sample) data from the model simulator: x p(x(0). Instead trying to obtain a numerical value of the generic posterior p(0 x) for arbitrary x, LFI only tries to obtain approximation of p(0 x.) for the given X.. During the inference process, we can take advantage the sampled data:
{(0,, x; ) where x, ~ p(x(0,) for selected
values of 0,
Biological black-box sequence design. We consider biological sequence design as a black-box optimization problem:
arg max (m),
mEM
where f()) is the oracle score function, and we would like discover values of for which ((m) is large. In real-world situations, a query of this oracle f could represent series of wet-lab ex- periments measure specific chemical properties or specificity for a given binding site target. In general, these experiments are time- and cost-consuming. As result, the total number of queries is limited.
In our setting, we use M = V to denote the search space for sequences with fixed length L, where is the vocabulary for each entry of the sequence: for DNA nucleotides |V] = 4, and for protein amino acids |V| = 20. For variable length setting, we have M = LL(Lmm (mmm V4, where 7 and Lmxx are the minimal and maximal length, respectively.
2.2 CONNECTING LFI AND BLACK- BOX OPTIMIZATION
In order to draw a connection to LFI, we require a probabilistic formulation of the black-box se- quence design problem. To this end, we relax the goal of searching for a single maximum of the oracle score function f to posterior modeling problem, i.e., finding representative sample of the configurations of m sampled with probability related to some target posterior. Think of C is the set of sequences with these desirable configurations, & is Boolean event about whether sequence m belongs to C, and our goal is to characterize the posterior distribution p(m] E from which we obtain the desired sequences. Below, consider two specific ways of doing this:
2

---

Published as a conference paper at ICLR 2022
Example A. We explicitly define C (and & accordingly) as all the sequences whose scores larger than given threshold s:
C= {m) 2 s}.
Here could be any fixed value, certain quantile of a particular score distribution. In this way, we have p(E mm = p(m E C\m) = 1 {m(mm 2 8} where 1(} is the indicator function.
Example B. softer version of and C, we can define its conditional probability of being true to follow Boltzmann distribution:
2(E m) = pmm E C)m) & exp(f(m)/T).
(3)
where T a temperature parameter. We introduce the exponential because f()) does not necessarily take positive values. Any monotone transformation of f(-) to non-negative reals could be used, so that sequences with larger oracle scores have greater probability of making & true.
With this posterior objective, our goal now becomes effectively modeling and sampling from the posterior ((m It is thus natural to resort to the tools of Bayesian inference for this task. In order examine this possibility, draw detailed comparison between the settings of black-box sequence design problem and likelihood-fii Bayesian inference in Table 1.
Likelihood--fee inference
Black-box optimization
Element
(0,x)
(m, s)
Target
p(0)x.)
p(m(E)
Constraint
limited simulation: x p(x intractable likelihood: p(x(0)
limited query: ((m) black-box oracle: f'm)
Table 1: Correspondence between likelihood inference and black-box optimization.
can be observed that both tasks share similar elements and targets. The two settings also share similar limitations the allowed queries, which are too time-consuming and cost-intensive. Notice that in sequence design, the oracle could be either exact or noisy, thus we use the more general s f (m) formulation rather than s fmm). We will further present several concrete examples as demonstrations of this correspondence in the following section.
Another way to understand this correspondence to consider the following mapping T:
T θ X - M R (0.x) - (m, s), s.t. = - xoll-
Here we can see the score value s as quantitative metric for how close the generated data (given 0) is to the target observed data x.. In addition. querying the oracle in the sequence design setting can also be thought of as follows: (1) sample x p(- and then (2) calculate s - ||x x, under some distance I|- In this manner, T could conceptually transform any LFI problem into black-box optimization task. this work, we only focus on the application of sequence design.
METHODOLOGY
We provide a recipe for designing new sequence design algorithms based on the correspondence in Section 2.2. The recipe induces different approaches by modeling different probabilistic component of the Bayesian inference problem. We begin with common algorithm restrictions under this setting-
Common constraint for algorithms. Due to the restriction of simulation / query in our setting, we constrain our algorithms to act in a sequential iterative way, gradually achieving the desired posterior round by round. Every algorithm starts with an empty dataset D Ø and an initial proposal P1 (.) = p(), where p(-) is the prior given by the task. In the r-th round this multi-round setting, the algorithm would use the proposal py (-) of this round to sample batch of data (0 / m) for simulation query, and augment the current dataset D with the newly obtained batch of data. We denote the batch size for each round's simulation query. Afterwards, the algorithm updates
3

---

Published as a conference paper at ICLR 2022
the proposal to P-++1-). The outcomes for the two settings we discuss may be slightly different: an algorithm for likelihood-free inference would return the posterior, while a sequence design method would return the dataset of all the sequences it has queried, which hopefully contains desired high scored sequences. On the other hand, a sequence design method could produce as an intermediate result generative model for sampling queries, which then completely fits with the LFI framework.
3.1 BACKWARD MODELING OF THE MECHANISM
Approximate Bayesian Computation (ABC) (Beaumont et al., 2002) is standard method for tack- ling LFI problems. In Algorithm 1, we display one of the most popular variants: Sequential Monte Carlo-Approximate Bayesian Computation (SMC-ABC) (Beaumont et al., 2009). In each round, pa- rameters 0 are sampled from the current proposal distribution (0) for simulation. A rejection step is then involved to remove the 0 whose simulation outcomes x; cannot reproduce the observed data x, with sufficient accuracy. The remaining accepted { are adopted to update the next round's s proposal Pr++() (.) towards the target posterior, i.e., by refitting do with the modified data. We defer more details of this approach to Section A. in Appendix.
Algorithm SMC-ABC
Algorithm FAE
(0) + p(0); for r in 1 to R do repeat
(m() <- --(m);; for to R do repeat
sample 0, ~ (0):
sample m, ~ pp():;
simulate x; ~ 0.);
query the oracle: s; + fm,))
until n samples are obtained
until n samples are obtained D t DU [m, s;)]:
D < D (0,, }}}}
sort D according to - -||x; - -xoll:
sort D according to s;
fit (0) with top {0.); in D;
fit g with top {m,}e in DD Pr- (m) < (m()
Pr-1 (0) + go (0):
end for
end for return {m (m. E D)
return p(0)x.) = (0)
would then be natural to construct an analogical sequence design algorithm using top scored entities {m to guide the update of a certain sequence distribution, see Algorithm 2. Interestingly, this is the proposed sequence design algorithm in Gupta & Zou (2019), where the authors name this kind of updating "feedback" because training of the parametrie generator ga (m) exploits feedback signals from the oracle. In this paper, we follow Brookes & Listgarten (2018) to crystallize go (m) to be variational autoencoder (Kingma & Welling, 2014), and use the term Feedback-Variationaa AutoEncoder (FB-VAE) to refer to Algorithm 2. We place Algorithm 1 & 2 side-by-side to highlight their correspondence. We also make the same arrangement for the following Algorithm 3 & 4, Algorithm & 6 and Algorithm 7 & 8.
Algorithm 3 Sequential Neural Posterior
Algorithm 4 Design by Adaptive Sampling
P1 (0) + p(0): for in 1 to R do repeat
Pi (1() < -p(m);
for in 1 to R do
repeat
sample 0 (f);
sample m, ~ (m);
simulate x; ~ 0:):
query the oracle: 8; + (m:); until samples are obtained
until n samples are obtained
D - DU [U, })) 96 - arg min, y Ex [KL (p(0)
D - DU [m,, 8t)
g6 + arg min, DKL (p(
(0) + 9o (0xx.);
Pr-1 (m) + (m);
end for
end for
return p(0 x.) = P
return {m (m, s) E D)
In comparison with SMC-ABC, the Sequential Neural Posterior (SNP) method (Papamakarios & Murray, 2016; Lueckmann et al., 2017; Greenberg et al., 2019) for likelihood-fres inference adopts more flexible approach, taking the power of conditional neural density estimator (e.g., Papamakar- ios et al. (2017)) to model the general posterior p( p(0]x), which takes arbitrary and x as two in- puts and outputs distribution. This neural estimator is trained via approximately minimizing the
4

---

Published as a conference paper at ICLR 2022
Betlbbacc-Libbbes (KL) divergence between 9a (0(x) and the true posterior p(0'x). We defer more training details to Section A. in Appendix. Under the connection viewpoint, one similar algorithm for sequence design is the Design by Adaptive Sampling (DbAS) proposed in Brookes & Listgarten (2018) which is characterized in Algorithm 4, fitting qo (m) through minimizing the KI. divergence with the posterior p(m E)) Based on the difference in specific implementations, both algorithms have more than one variant, whose details are deferred to Section A. in Appendix.
We refer to the above algorithms as "backward modeling" because the trained generative network qo (going from x/Ê to 0/m) is a sort of reverse model of the simulation mechanism (which goes from / m to x/s).
3.2 FORWARD MODELING OF THE MECHANISM
Whereas the above methods focus on directly modeling the target posterior with a generative model that learns a "reverse mechanism" of the simulation process, it also possible model the "forward mechanism", which is consistent with the simulation process. Papamakarios et al. (2019) claim that the forward modeling approach may be an easier task than its backward counterpart, as unbiased estimation of the likelihood does not depend on the choice of proposal. Consequently. in contrast to SNP, Papamakarios et al. (2019) chooses to train a neural density estimator to model the conditional likelihood distribution 4s (x|0) sequentially in each round. The training is achieved by maximizing the total log likelihood max, E, log qs (x:(0) with data samples from the dataset D at the current (r-th) round. The downside of this forward approach is additional computational Markov Chain Monte Carlo (MCMC) step is needed to sample from the r-th round posterior / proposal p-) The resulting approach, which is coined (Papamakarios al., 2019) the Sequential Neural Likelihood (SNL), is summarized in Algorithm 5.
In the spirit of directly modeling the forward mechanism of sequence design, we train a regressor (m) in a supervised manner fit the oracle scorer. In order to adapt this regressor into the update procedure of the proposal of the next round, we use ((m) to denote the unknown posterior p( pmm| E) with knowledge of fo (m) and prior p(m). The specific construction of 7(m) depends on the choice of E. For instance, if we choose Example B in Section 2.2 to be the definition of E, then q(m) is the
distribution with (unnormalized) probability p(m)- exp( (mfff)///) See Section A.2 in Appendix for more elaboration about this point. We then choose the update procedure of the proposal pr+1 ( (m) to be analogical to that of SNL. We name this proposed algorithm to be Iterative Scoring (IS) to avoid confusion with likelihood free inference algorithms. Furthermore, depending on different definition of E, we use the name "IS-A" and "IS-B" for them in the following sections.
Algorithm 6 Iterative Scoring
Algorithm 5 Sequential Neural Likelihood
Pi (m) < --(m); for in 1 to R do repeat
Pi(O) p(0):
for r in 1 to R do repeat
sample m, ~ (,):;
sample 0 ~ (0):
query the oracle: 8; < [m:))
simulate x; ~ (())
until samples are obtained
until n samples are obtained
D t D {m,, st))mm
D < D ((),,))) )) fit (x(0)) with D:
fit (m) with D:
construct ((m) with fo(+) and p(m);
(0) x p(0) 4o (xx 0)) end for
Pr-1 (m) + (m)):
end for
return ((0 xx) = PR-I (0)
return {m (m, s) D)
3.3 MODELING PROBABILITY RATIO
In this subsection, we discuss yet another approach, through the estimation of a probability ratio. Gutmann & Hyvärinen (2010) proposes noise contrastive estimation as a statistical inference ap- proach. This methodology turns hard probability modeling problem into binary classification, which is considered easier to leamn In contrast to the aforementioned likelihood-free inference methods which rely on form of density estimation to perform the task, Sequential Neural Ratio
5

---

Published as a conference paper at ICLR 2022
(SNR) (Hermans et al., 2019) takes similar approach noise contrastive estimation. SNR adopts classification approach to estimate the likelihood-to- to-evidence ratio r(0, x) p(0)x)/p(0)) SNR is summarized in Algorithm 7. Specifically, in each round, SNR fits binary classifier de (0, x) E [0,1] in the following manner:
arg min { log d(x;, 0:)1 + Eio log(11 - d(x;, d(xx,,8)]]
(-
(4)
We show that with the D and D' established in Algorithm 7, we have
p(0)x)
d" d0, p(0
d',0, = p(0) + p(Olx)
r0( = 1- (0,x) p(0)
where the "+" denotes the optimality. We have the following Proposition 1:
Proposition 1. Let (a) and pr (a) be distributions for d. du-ddimension random variable which takes value in the space of A = Rd,,, and d(a) : : > [0, 1] is a real-value function mapping any a positive real value number. Then the functional optimization problem
arg max (lg d(a)) + E. Ea-pi - d:A-((0.1]
will lead to the optimal solution d' da Po(a)+Pi
Po (n)
See the proof in Section A.3 in Appendix. After training d, SNR can obtain the posterior density value by p(0]x) = (0, (0). SNR mirrors SNL in that it samples the new proposal p+) without explicitly modeling the posterior.
On the other hand, we propose a sequence design algorithm analogous to SNR and named Iterative Ratio (IR), which estimates the probability ratio r(m) ((ml))/p(mm for the purpose of poste- rior sampling. IR first builds two datasets DD and D', corresponding to two different distributions p(m|E) and p(m). We take a similar binary classification approach, whose training objective is mina (Emepl- log dmm] + Emer log(1 d(m))]]- The desired ratio is then obtained by r(m) = d(mm/((
d(m)). Other components of IR follow the scheme of IS, and IR is schema- tized in Algorithm 8. We point out that IR does not have two variants IS does, as the definition for E in Example B is not usable because of the unknown normalizing constant for probability p(E p(E|m). mm).
See Section A.3 in Appendix for more explanation.
Algorithm 7 Sequential Neural Ratio
Algorithm 8 Iterative Ratio
Pi (0) + p(0): for in 1 to R do repeat
P1 < p(m);
for r in 1 to R do repeat
sample 0, 0 pr (r):
sample m, ~ (m);m);
simulate x; ~ p(x{0,);
query the oracle: 8; 4 fm:);
until n samples are obtained
until samples are obtained
D - DU 0, x+) H D' - D' ( x:) R=
t DU {(m; 8:)
construct D with m in 'D satisfying E;
train (0, x) classifying between and with the loss in Eq. 4;
construct D' from p(m);
train dg (m) classifying between D and D'; r& (m(m) d,(m)
x) (e.x)
x
[m)
pr-1(0) x ra( p00);
Pr-1(0) (0) x rm(m)) -p(m:
end for return p(0 x = PR-1 (0)
end for
return {m (m, s) E D)
Interestingly, a recent SNR-like work, EG-LF-MCMC (Begy & Schikuta, 2021), proposes to train the classifier on tuples of (0.c ||x xoll) instead of (0,x). This algorithm can be also seen more precise analogy of our Iterative Ratio in the LFI context, as we have pointed out in Section 2.2
that conceptual link could be drawn between and x,ll.

---

Published as a conference paper at ICLR 2022
3.4 COMPOSITE PROBABILISTIC METHODS
Building on the above analogies and framework, we move beyond the above algorithms in this subsection. The previous lines of approach the direct posterior modeling methods in Section 3.1 and the indirect methods in Section 3.2 and 3.3 - both have their own advantages and disadvantages. The former methods may fail to get accurate inference result due unmatched proposals, while the latter ones would need extra large amount of computation for the MCMC sampling process before obtaining accurate posterior samples, etc. Here we study composite algorithms that combine the aforementioned ingredients through the lens of our proposed unified framework. Our goal is to combine the strengths from both kinds of methods.
We first introduce Iterative Posterior Scoring (IPS) method illustrated in Algorithm 9. IPS also uses neural network fo (m) to model the forward mechanism as IS does, and again to denote the target posterior p( (() Instead of applying computational MCMC steps here, we train a second parametrized model qu model by minimizing the KL divergence between them. Notice that the optimization of qy is restricted within a neural network parameterization family. As result, in the next round, we can directly utilize go (m) serve as a flexible generative proposal of Pr+1 (m). Like IS, the IPS algorithm also has two different variants with regard to different choices of E, we name them to be IPS-A and IPS-B. Two choices differ in the detailed construction of distribution @m))
q here
In similar spirit, we propose the Iterative Posterior Ratio (IPR) algorithm (see Algorithm 10). IPR is close to the IR algorithm in many aspects, but also adopts second neural network model qu (m(m like IPS. IPR works similarly to IR, in that we also construct ((m). taking advantage of r. (m) and
the prior p(m) simply via (mm) + To (m(m)) p(m). Note that the usage of two models in IPR is not exactly the same as in IPS: go (m) also achieved via minimizing KL divergence with @(m). but
the training of model do (m) ) is closer to that in IR rather than the (m() ) in IS. Another similarity between IPR and IR is that IPR also only has one variant, since the Example B is not applicable for this ratio modeling approach (see Section A.4 in Appendix for more details).
Algorithm 10 Iterative Posterior Ratio
Pi (m) < p(m); for in 1 to R do repeat
Algorithm 9 Iterative Posterior Scoring
P1 (m) <- p(m);
for in 1 to R do repeat
sample m, ~ (m);m);
query the oracle: 8; 4 (m:);
sample m; (m)) query the oracle: s; f fmm);
until samples are obtained
t DU {m,, {:)):
until n samples are obtained D DeDu DD {(m;, s:) fit (m() with D;
construct D with m in 'D satisfying E;
construct D' from p(m);
train dg (m) classifying between D and D'; rs (mmm d,(m)
construct (m) with fa(-) and p(m); IN + arg min, DKL. (xz.((())lgg)
[m)
construct ((m) with r. (m) and p(m);
(m- + o (m):
9s <- arg min, Dku(g(m)lgg)
end for
return {m: (m, E D}
p (m) 90 (m);
end for
return {m (m, s) D)
4 EXPERIMENTS
4.1
SETUP
In this section, we systematically evaluate the proposed methods and baselines on four different in-silico biological sequence design benchmarks. In every round, we allow each algorithm to query the black-box oracle for a batch of n sequences m; to obtain their true scores 8,, with n = 100 for all experiments. The total number of rounds differs across different tasks.
We experiment with our six proposed methods: Iterative Scoring (-A/B) from Section 3.2, Iterative Ratio from Section 3.3, Iterative Posterior Scoring (-A/B) and Iterative Posterior Ratio from Sec-
7

---

Published as a conference paper at ICLR 2022
A
15:8
IPSA
IPS-B
IPR
Random
Evolution
DeAS
FB VAE
KLF11_R4020_F6 R1
P RET_RETRRR R2
CRX, C80A R1
0.44
0475
0.90
0.42
0.450
Top-10 scooe
195 score
0.40
0.38
0.400
Top- 10
0.40
0.38
0.75
0.24
0.300
0.32
Round
Round
Round
0.40
0.40
0.35
0.4
Top-100 9o080
Top-100 score
0.30
0.90
0.5
0.25
0.0
0.15
0.10
0.1
Round
Round
Round
Figure 1: Top score (y-axis) curves of different methods on TfBind problems (KLFI SKLFILR4022 R402Q_R1, PBX4_REF_R R2 and
R1) with regard the number of rounds.
IS-A
IS-B
IR
IPS-A
IPS-B
IPR
RANDOM
EVOLUTION
DBAS
TA-VAE
Top-10
8.43
6.93
1.79
6.21
8.36
7.00
1.14
4.36
5.07
2.71
Top-100
9.57
7.93
1.07
6.71
7.64
6.00
1.00
5.50
4.57
2.00
Table Mean rank of evaluated algorithms with regard to the area under the top score curve for TfBind problems. The rank ranges from 1 to 10. Higher rank is better.
tion 3.4. Apart from these proposed methods, we consider as baselines a battery of existing methods designed for batched black-box sequence design tasks: (1) Random, method that randomly se-
lect proposal sequences at every round; (2) FB-VAE (Gupta & Zou, 2019) depicted in Section 3.1; (3) Evolution based (Brindle, 1980; Real et al., 2019) sequence design algorithm; and (4) DbAS (Brookes & Listgarten, 2018), described Section 3.1.
We evaluate these sequence design algorithms by the average quences in the resulting dataset at each round. We plot the average score curves with regard to the number of rounds. We also use the compare the methods being evaluated. Specifically, since the area depends on the choice of x-axis, simply cumulate the scores of all rounds calculate the area. The result could be non-positive, since the score can take negative values.
of the top- 10 and top-100
under the curve
scalar metric for sample efficiency to
4.2 RESULTS
Transcription factor binding sites (TfBind). Protein sequences that bind with DNA sequences to adjust their activity are called transcription factors. In Barrera et al. (2016), the authors measure the binding properties between battery of transcription factors and all possible length-8 DNA sequences through biological experiments. Concretely, we choose 15 transcription factors serve as 15 different tasks. For each transcription factor, the algorithm needs search for sequences that maximize the corresponding binding activity score. The size of the search space is D/ 48 = 65536. The number of total rounds is fixed to 10. For validation, follow Angermüller et al. (2020b) and use one task (ZNF200 S265Y R1) for hyperparameter selection. Then we test the algorithms' performance the other 14 held-out tasks.
Figure displays a comparison for all ten methods on three of the chosen binding affinity tasks. We can observe that after 10 rounds, our proposed methods perform consistently better than the
8

---

Published as a conference paper at ICLR 2022
15-A
E-8
IPS
IP58
IPR
Pendom
Evoutin
D6AS
FB-VAE
UTR
AMP
Flue
1.38
-0.100
1.30
-0.125
2.4
Top-10 100-10 8opm
1.25
soore
00:60
2.2
1.20
-0.17
1.16
-0200
1.10
-0225
1.8
1.06
-0250
1.5
10
14
10
15
20
Round
Round
Round
-0.20
Top- 100 woore
-0.25
100
-0.35
-0.45
Round
Round
Round
Figure 2: Top score (y-axis) curves of different methods on 3 sequence design problems (left: UTR, middle: AMP, right: Fluo) with regard to the number of rounds.
IS-A
IPS-A
IPS-B
IPR
RANDOM
EVOLUTION
DBAS
FB-VAE
UTR
8.46
9.61
9.03
10.04
10.19
9.52
8.12
9.23
9.59
8.20
AMP
-5.67
-5.11
-5.37
-4.65
-4.39
-5.42
-5.9
-5.79
-5.39
-5.54
FLUO
32.76
33.33
32.95
44.51
43.13
42.42
31.64
35.44
41.40
32.52
Comparison of the area under 100 curves for UTR, AMP and Fluo benchmarks. Larger area means better sample efficiency.
Table 3:
backward modeling methods like DbAS and FB-VAE in terms of both Top- 10 and Top-100 scores. Among all baselines, the evolution method is the strongest one, and it beats IR on some of the tasks (see complete results in Table 4 and 5 in Appendix). We also find that IS-A and IS-B increase top scores slightly faster than other methods, especially on PBX4 REF R2 and CRX ES0A RI. This indicates that composite methods' way of using parameterized models to replace computational procedures is not the optimal solution for small-scale tasks.
5' untranslated regions (UTR). The translation efficiency is mainly determined by the sequence of 5' UTR (Alipanahi et al., 2015). In Sample et al. (2019), the authors create a library of gene sequences with ribosome loading level as labels. They further train a convolutional neural network with this library to predict the relationship between S'UTR sequence and the corresponding gene expression level. We use this neural network as an oracle for this benchmark. The length of the gene sequences is fixed to 50, and thus the size of the search space is 450 For this S'UTR benchmark, we also allow each algorithm to explore for 10 rounds. Figure (left) shows that our proposed composite methods significantly outperform other methods the UTR task. Different from the results on the TiBind task, forward modeling methods do not achieve the best performance.
Antimicrobial peptides (AMP). Protein modeling has recently become popular sub-area of ma- chine learning research. We are tasked to generate AMP sequences, which are short protein se- quences against multi-resistant pathogens. We train a binary classifier model classify whether short protein sequence belongs to AMP and defer the related details to Appendix. This is the only task we consider regarding sequence design with alterable lengths, where the length of sequences ranges from 12 to 60. Since each entry of protein sequence has | = 20 different choices on amino acids, the size of search space is EL=12 20". We set the number of total rounds be 15 for this task. Figure (middle) clearly shows that the performances of IPS-A and IPS-B B dominate the AMP generation task, which demonstrates the effectiveness of our composite strategy.

---

Published as a conference paper at ICLR 2022
Fluorescence proteins (Fluo). As another protein engineering task, consider the optimization over fluorescent proteins, which a commonly used test bed of modern molecular biology. This task is similar to the AMP task introduced above, but its ground-truth measurement relies on regressor. We use a pretrained model taken from Rao et al. (2019) to act as our task oracle, which regressor trained predict rigfluoreeceene intensity value over approximately 52,000 protein sequences of length 238 (Sarkisyan et al., 2016). More concretely, the regressor is fit on small neighborhood of parent green fluorescent protein, and is then evaluated on more distant protein. The training data is derived from the naturally occurring GFP in Aequorea victoria. The task Fluo's s search space is 20238 and we the number of rounds 20. We demonstrate the Fluo results in Figure 2 (right), where we can see both composite methods and DbAS achieve much better results than other approaches. This might signify that for long sequence design tasks, backward modeling is superior to other modeling methods, which is not consistent with L.FI (Papamakarios et al., 2019). We also summarize the sample efficiency results for the latter benchmarks in Table and 6, from which we can see that our proposed three composite methods perform remarkably promising results.
5 CONCLUSION
We propose a probabilistic framework that unifies likelihood-free inference and black-box optimiza- tion for designing biological sequences. This unified perspective enables us to design a variety of novel composite probabilistic sequence design methods combining the best of both worlds. Exten- sive experiments have demonstrated the benefits of the unified perspective. While the composite probabilistic methods usually outperform other baseline methods in most sequence design tasks consider in this work, the key contribution of our paper is not just about the superiority of those com- posite methods, different specific tasks might prefer different algorithmic configurations due to no free lunch theorem (Wolpert & Macready, 1997). Actually, we would like to attribute the strong performance to the unified probabilistic framework, which enables develop a richer algorithm poot, based on which we can design performant algorithms for particular sequence design tasks.
ACKNOWLEDGEMENT
The authors would like thank Christof Angermueller, Yanzhi Chen, Michael Gutmann, and anony- mous reviewers for helpful feedbacks. Jie Fu thanks Microsoft Research Montreal for funding his postdoctoral position at University of Montreal and Mila. Yoshua Bengio acknowledges the fund- ing from CIFAR, Samsung, IBM and Microsoft. Aaron Courville thanks the support of Samsung, Hitachi and CIFAR.
REFERENCES
Sungsoo Ahn, Junsu Kim, Hankook Lee, and Jinwoo Shin. Guiding deep molecular optimization with genetic exploration. ArXiv, 0as/2007.048977 2020.
Babak Alipanahi, Andrew Delong, Matthew T Weirauch, and Brendan J Frey. Predicting the se- quence specificities of dna-and ma-binding proteins by deep learning. Nature biotechnology, 33 (8):831-838, 2015.
Christof Angermüller, David Belanger, Andreea Gane, Zelda F. Mariet, David Dohan, Kevin Mur- phy, Lucy J. Colwell, and D. Sculley. Population-based black-box optimization for biological sequence design. In ICML., 2020a.
Christof Angermüller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy J. Colwell. Model-based reinforcement learning for biological sequence design. In ICLR, 2020b.
Luis A. Barrera, Anastasia Vedenko, Jesse V. Kurland, Julia M Rogers, Stephen S. Gisselbrecht, Elizabeth J Rossin, Jaie C. Woodard, Luca Mariani, Kian Hong Kock, Sachi Inukai, Trevor Sig- gers, Leila Shokri, Raluca Gordân, Nidhi Sahni, Chris Cotsapas, Tong Hao, S. Stephen Yi, Mano- lis Kellis, Mark J. Daly, Mare Vidal, David E. Hill, and Martha L. Bulyk. Survey of variation in human transcription factors reveals prevalent dna binding changes. Science, 351:1450 1454,
2016.
10

---

Published as a conference paper at ICLR 2022
M. Beaumont, Wenyang Zhang, and D. Balding. Approximate bayesian computation in population genetics. Genetics, 162 4:2025-35, 2002.
M. Beaumont, J. Cornuet. J. Marin, and C. Robert. Adaptive approximate bayesian computation. Biometrika, 96:983-990, 2009.
Volodimir Begy and Brich Schikuta. Error-guided likelihood-free meme. 2021 International Joint Conference on Newral Networks (IJCN)) pp. 1-7, 2021.
Michael G. B. Blum. Approximate bayesian computation: nonparametric perspective. Journal of the American Statistical Association, 105:1 178 1187,0009
Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. Mining gold from implicit models to improve likelihood-free inference. Proceedings of the National Academy of Sciences, 117:5242 -2249, 2020.
A. Brindle. Genetic algorithms for function optimization. 1980.
David Brookes and J. Listgarten. Design by adaptive sampling. ArXiv, abs/0/11,,3314, 2018.
David H. Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In ICML, 2019.
Jeffrey Chan, Valerio Perrone, Jeffrey Spence, Paul A. Jenkins, Sara Mathieson, and Yun S. Song. A likelihood-free inference framework for population genetic data using exchangeable neural networks. bioRxiv, 2018
K. Chen and F. Arnold. Enzyme engineering for nonaqueous solvents: Random mutagenesis to enhance activity of subtilisin in polar organic media. Bio/Technology 9:1073-1077, 1991.
Yanzhi Chen and Michael U. Gutmann. Adaptive gaussian copula abc. ArXiv, htss/992..10000 2019.
Yanzhi Chen, Dinghuai Zhang, M. Gutmann, Aaron C. Courville, and Zhanxing Zhu. Neural ap- proximate sufficient statistics for implicit models. ArXiv, abs/2010. 10079, 2021.
P. T. de Boer, Dirk P. Kroese, Shie Mannor, and Reuven Y. Rubinstein. A tutorial on the cross- entropy method. Annals of Operations Research, 134:19-67, 2005.
Christopher C. Drovandi, Clara Grazian, Kerrie L. Mengersen, and Christian P. Robert. Approxi- mating the likelihood in approximate bayesian computation. arXiv: Computation, 2018.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rihawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: towards crack- ing the language of life's code through sule-super deep learning and high performance com- puting, arXiv preprint arXiv:2007.0622 06620,, 2020.
Paul Fearnhead and Dennis Prangle. Constructing summary statisties for approximate bayesian computation: semi-automatic approximate bayesian computation (with discussion). 2012.
Rafael Gómez-Bombarelli, David Kristjanson Duvenaud, José Miguel Hemnáneez-Lobato, Jorge Aguilera-Iparrauuieee Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzze Automatic chemical design using data-driven continuous representation of molecules. ACS Central Sci- ence, 4:268 276, 2018.
Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, et al. Learning to navigate the synthetically accessible chemical space using reinforcement learminn In International Conference on Machine Learning, pp. 3668-3679, PMLR, 2020.
David S. Greenberg, M. Nonnenmacher, and Macke. Automatic posterior transformation for likelihood-frea inference. ArXiv, abs/1905.07488, 2019.

---

Published as a conference paper at ICLR 2022
Gabriel Lima Guimaraes, Benjamín Sánchez-Lengeling Pedro Luis Cunha Farias, and Alán Aspuru- Guzik. Objective-reinforeed generative adversarial networks (organ) for sequence generation models. ArXiv, abs/1705.1100 2017.
Anvita Gupta and J. Zou. Feedback gan for dna optimizes protein functions. Intelligence, 1:105-111, 2019.
Nature Machine
Michael U. Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: new estimation princi- ple for unnormalized statistical models. In AISTATS, 2010.
Michael U. Gutmann, Ritabrata Dutta, Samuel Kaski, and Jukka Corander. Likelihood-free infer- ence via classification. Statistics and Computing, 28:411 -22, 2018.
Tatsunori B. Hashimoto, Steve Yadlowsky, and John C. Duchi. Derivative free optimization via repeated classification. AISTATS, 2018.
J. Hermans, Volodimir Begy, and Gilles Louppe. Likelihood-free meme with amortized approximate likelihood ratios. arXiv: Machine Learning, 2019.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9:1735- 1780, 1997.
Jp Hughes, S. Rees, SB Kalindjian, and KL Philpott. Principles of early drug discovery. British Journal of Pharmacology, 162, 2011.
Rafael Izbicki, Ann B. Lee, and Chad M. Schafer. High-dimensional density ratio estimation with extensions to approximate likelihood computation. AISTATS, 2014.
Jan H Jensen. graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, (0(12):3567-3572.222 2019.
Diederik P. Kingma and M. Welling. Auto-encoding variational bayes. CoRR, abs/aa12..111111 2014.
Jian Ping Li, David J. Nott, Y. Fan, and Scott Anthony Sisson. Extending approximate bayesian computation methods to high dimensions via a gaussian copula model. Comput. Stat. Data Anal., 106:77-89, 2017.
Jarno Lintusaari, Michael U. Gutmann, Ritabrata Dutta, Samuel Kaski, and Jukka Corander. Funda- mentals and recent developments approximate bayesian computation. Systematic Biology, 66: e66 e82, 2017.
Ge Liu, Haoyang Zeng, Jonas Mueller, Brandon Carter, Ziheng Wang, Jonas Schilz, Geraldine Horny, Michael E. Birnbaum. Stefan Ewert, and David Kenneth Gifford. Antibody complemen- tarity determining region design using high-capacity machine learning. bioRxiv, 2020.
Jan-Matthis Lueckmann. Pedro Gonçalves, G. Bassetto, Kaan Öcal, M. Nonnenmacher, and
J. Macke. Flexible statistical inference for mechanistic models of neural dynamics. In NIPS, 2017.
Man-Matthis Lueckmann, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H. Macke. Likelihood-free inference with emulator networks. ArXiv, 0bss/180500929/ 2018.
Jean-Michel Marin, Pierre Pudlo, Christian P. Robert, and Robin J. Ryder. Approximate bayesian computational methods. Statistics and Computing, 22:1 167-110,,, 2012.
Kerrie I.. Mengersen, Pierre Pudlo, and Christian Robert. Approximate bayesian computation via empirical likelihood. 2012.
Daniel Neil, Marwin H. S. Segler, Laura Guasch, Mohamed Ahmed, Dean Plumbley, Matthew Sellwood, and Nathan Brown. Exploring deep recurrent models with reinforcement learning for molecule design. In ICLR, 2018.
AkshatKumar Nigam, Pascal Friederich, Mario Krenn, and Alán Aspuru-Guzik. Augmenting ge- netic algorithms with deep neural networks for exploring the chemical space. arXiv preprint arXiv: /909, 1655, 2019.
12

---

Published as a conference paper at ICLR 2022
George Papamakarios and lain Murray, Fast e-free inference of simulation models with bayesian conditional density estimation. In NIPS, 2016.
George Papamakarios, lain Murray, and Theo Pavlakou. Masked autoregressive flow for density estimation. ArXiv, abs/1 705.07057, 2017.
George Papamakarios, D. Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood- free inference with autoregressive flows. In AISTATS, 2019.
Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Xi Chen, John Canny, Pieter Abbeel, and Yun S Song. Evaluating protein transfer learning with tape. Advances in Neural Informa- tion Processing Systems, 2019.
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le. Regularized evolution for image classifier architecture search. In AAAI, 2019.
Reuven Y. Rubinstein and Dirk P. Kroese. The cross-entropy method. In Information Science and Statistics, 2004.
Tim Salimans, Jonathan Ho, Xi Chen, and llya Sutskever. Evolution strategies as a scalable alterna- tive reinforcement learning. ArXiv, abs/1703.0386, 2017.
Paul Sample, Ban Wang, David W. Reid, Vladimir Presnyak, Iain J Mcfadyen, David R. Morris, and Georg Seelig. Human 5 design and variant effect prediction from a massively parallel translation assay. Nature Biotechnology, 37:803-809, 2019.
Karen S Sarkisyan, Dmitry A Bolotin, Margarita V Meer, Dinara R Usmanova, Alexander S Mishin, George V Sharonov, Dmitry N Ivankov, Nina Bozhanova, Mikhail Baranov, Onuralp Soyle- mez, et al. Local fitness landscape of the green fluorescent protein. Nature, 533((6033:397--41,, 2016.
Bobak Shahriari, Kevin Swersky, Ziyun Wang, Ryan P. Adams, and Nando de Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104:148 175, 2016.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. ArXiv, abs/2001.099335 2020. 2020.
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15: 1929-1958, 2014.
Owen Thomas, Ritabrata Dutta, Jukka Corander, Samuel Kaski, and Michael U. Gutmann. Likelihood-free inference by ratio estimation. arXiv: Machine Learning. 2016.
Minh-Ngoc Tran, David J. Nott, and Robert Kohn. Variational bayes with intractable likelihood.
Journal of Computational and Graphical Statistics, 26:873 882,, 2015.
Laurens van der Maaten and Geoffrey E. Hinton. Visualizing data using t-sne. Journal of Machine Learming Research, 9:2579- 2008.
Daan Wierstra, Tom Schaul, Jan Peters, and Juergen Schmidhuber. Natural evolution strategies. 2008 IEEE Congress Evolutionary Computation (IEEE World Congress Computational Intelligence), pp. 3381-3387, 2008
Jacob Witten and Zack Witten. Deep learning regression model for antimicrobial peptide design. BioRxiv, pp. 692681, 2019.
David H. Wolpert and William G. Macready. No free lunch theorems for optimization. IEEE Trans. Evol. Comput., 1:67-82, 1997.
Simon N. Wood. Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466: 1102-1104, 2010.
13

---

Published as a conference paper at ICLR 2022
Zachary Wu, S B Jennifer Kan, Russell D Lewis, Bruce J. Wittmann, and Frances H. Amold. Ma- chine learning- assisted directed protein evolution with combinatorial libraries. Proceedings of the National Academy of Sciences, 116:8852 8858, 2019.
Kevin K. Yang, Zachary Wu, and Frances H. Amold. Machine-leaaming-guidee directed evolution for protein engineering. Nature Methods, pp. 1-8, 2019.
Naruki Yoshikawa, Kei Terayama, Masato Sumita, Teruki Homma, Kenta Oono, and Koji Tsuda. Population- based de novo molecule generation, using grammatical evolution. Chemistry Letters, 47(11):1431-1434, 2018.
Zhenpeng Zhou, Steven M. Kearnes, Li Li, Richard N. Zare, and Patrick F. Riley. Optimization of molecules via deep reinforcement learning. Scientific Reports, 9, 2019.
14

---

Published as a conference paper at ICLR 2022
A MORE ABOUT METHODOLOGY
A.1 BACKWARD MODELING THE MECHANISM
bbuu SMC-ABC. In Algorithm I, when we pick the top-mn O at the r-th round, the picked param- eters actually follow such a distribution
p)) & Σ (i())) p(11= <ol1<c(0)) = (x(ee) p(l1= xcll <c
(5)
Im
where the here is implicitly defined by how "top" the selection process is, namely the ratio As a result, we point out that in Algorithm 1, another more complicated form of the last step pr+ (0) <-
(0) is P (0) go p (())/ Σi (O), where an additional renormalizing term is involved. We ignore this term and used the simpler alternative in order to keep our main text clean. We refer interested readers to Beaumont al. (2009) for more details.
About Sequential Neural Posterior. Define p(x) J p(x and p(x) = (()) for any arbitrary proposal distribution p(0) which not necessary to be the prior p(0). What's more, we define p(0)x) x = p(xx{))p(0)/p(x) and p(0)x) p(x(0))(0)//ts> to be the true posterior and the proposal posterior.
Starting from the goal of approximating the true posterior,
f p(x)dx / p(0)x) log ((0)x)d0
arg min PPtx)|PP))((((00)))(((0000) arg max
/ p(0, log (01x))xxx
arg max
arg max 1x) q(0 x00' x)].
seems that we can directly train the parameterized neural density estimator ge in a data driven manner via max E, log Za (0:(x;) where i is the index for data sample. When the number of training samples as well as the parameterization family of Φ are large enough, the obtained ds would be close enough to the true posterior. This would require the data samples to follow (0,, x;) p(0, x) = p(0)p(x(0) (0). However, practically one uses a proposal p(0) to first generate some (0:): and then generate (x;), h by simulation. When the proposal distribution p(0) is not exactly the prior distribution p()), the resulting (-1-) would be:
p(0)p(x p(0)p(x)
<(0))) P(0)
p(0 p(0)x)
(6)
p())*
which is biased estimation and is not what we want.
Three variants of SNP take different approaches to try to fix this bias. SNP-A (Papamakarios & Mur- ray, 2016) first fits the biased proposal posterior 5(0 x) in the aforementioned way and utilize the relation in Eq. 6 to solve for an unbiased estimation. This approach is restricted to mixture of Gaus- sian distribution family and thus has limited expressiveness. SNP-B (Lueckmann al., 2017) uses
importance sampling address this issue via max p(x00)))(
p(0) F(O)
log (0 [ xx) One
downside of this approach is the high variance involved by the importance weights p(0)/5(0). SNP-C (Greenberg al., 2019) proposes use reparameterize the proposal posterior by setting p(O)
jo (0))) x) = qa 0 x) p()) Z where (x = f qs x) p())
d0 the corresponding normalizing p(o)
factor for SNP-C then maximizes (xx,0)=p(x ())()) [log d x)).
bbut Design by Adaptive Sampling. We aim to approximate the posterior via minimizing the KL divergence:
15

---

Published as a conference paper at ICLR 2022
arg max / p(ml() log ((m)
arg min DKL. (xt.(p(ml)))l((m)
arg max
log
p(m)
arg max E (q)))
p p((|m) (00mm)llgggg logu(m)
[(m)
where ((m) could be any distribution of m. Brookes et al. (2019) takes this formulation. Brookes & Listgarten (2018) only differs the place that ignores the denominator term. According to Angermüller et al. (2020b), we choose the latter variant one of our baselines because it is more stable in practice. We refer interested readers to Brookes Listgarten (2018) for more details.
Notice that we are not doing exactly the same things for LFI and black-box sequence design. Since LFI models flexible posterior p(0 x) which to model p(mls) for arbitrary Nevertheless, in the neural network modeling, conditioning by scalar value is not an effective approach as the effect of low dimensional scalar value conditioning may be covered by other high dimensional input. Therefore, we choose directly model the target posterior pl (m) with a single neural network.
distribution for arbitrary x, we can also choose
A.2 FORWARD MODELING OF THE MECHANISM
We still use p(0) to denote an arbitrary proposal distribution and p(0, == p((). Then we have
arg mig Ep(o [DKL (p(x(0)] = arg max p(0)do /
p(x 0) log (x
arg max (x(x [log
We point that with much enough data and large enough expressiveness of the neural density estimator parameterization family, no matter what proposal p(0) is used to provide training samples {OO,, x;)}; p(0, x), we have the resulting 9% (0)x) equals true likelihood p(x(0) in the support of the proposal. What SNL gives is unbiased estimation and thus does not have the same problem as SNP.
Now we elaborate the construction of ((m) in Iterative Scoring algorithm. Here we use the notation ((m) to denote our appreximation of the posterior p(m(E). Notice that we want g(m) x p(m) P(E m). If we choose Example A serve as the definition of event E, then the samples of (m)) can obtained this way: (1) sample m from prior p(m) and (2) accept this sample fa (m) larger than threshold s, otherwise reject it. Alternttvvly, if we choose Example B, we have (m)) x p(m)- (x)((f((m//)) Similar SNL, do MCMC sampling from this unnormalized probability function.
A.3 MODELING A PROBABILITY RATIO
About Sequential Neural Ratio. Dataset D is generated in the way that (1) first sample 0 ~ p(0) and (2) simulate p(x 0). Consequently, D follows the distribution px0)p(x(0) p00, x). On the other hand, the other dataset D' generates 0 and x in parallel and independent manner. Notice here p0) and follows the marginal distribution: x ~ pxx) f p(0 p()
Proof of Proposition! 1.
Proof. We define functional FF be the optimization objective:
F(d] (a) d(a)) + TF Ea (a) (lo((1 d(a)))
16

---

Published as a conference paper at ICLR 2022
We calculate its functional derivative. For arbitrary function and infinite small
Fld+ cu] - Fld] Exp [og((
+ < "
(2)) + Eop [og
+ 6 -u
1
= =
Po + 1- + +(())
-P1
d
Fld + Eu] - Fld
Po + -P1
/ 5F.
lim c+0
d
1 d
We set the functional derivative to zero:
d
8F = = pj 1
Po
3 d(a) = Po(a) + Pi (a)
This optimal function d' apparently takes value in [0, 1].
About Iterative Ratio. Notice that in Algorithm 8 construct two datasets: D and D', To generate D. we need be able pick some sequence samples m from D and make the selected ones follow the posterior p(m[E) (). This procedure will depend on our choice of event &. For Example A this is easy, since we just need to filter out the sequences whose oracle value is smaller than the threshold. However, for Example B, it is hard to do similar things, since given score value from 'D we only know the unnormalized value posterior probability, and cannot determine which sequence should be filtered out. The construction of D' which follows prior distribution p(m) is trivial.
A.4 COMPOSITE PROBABILISTIC METHODS
For IPS, the optimization with regard the second parameterized model qu is
/ @(m) log (mm)dm
= arg min (KL(((m))) (m(m))l9) = arg max
arg max / pp(m|E log (mm)mm= = arg max
/ p(()m))((m) log ((m)dm.
4
The exact value of p(Emm [)) depends different choices of configuration feature & in Section 2.2. On the other hand, IPR, like IR, also only has one variant, which is with Example A:
qV arg min DKL (Ā(())l4))) arg max / re (m)p(m) log g(m)dm,
which is tractable optimization problem. Both IPR and IR not fit for Example B since it cannot provide exact probability value and thus cannot be adopted to construct D.
B MORE ABOUT EXPERIMENTS
Random method uses no neural network model. FB-VAE uses a VAE model. The encoder of the VAE first linearly transform one-hot input into a hidden feature which is 64 dimension, and then sep- arately linearly transform to a 64-dimension mean output and 64-dimension variance output. The decoder contains 64 64 linear layer and linear layer that maps the hidden feature to categorical output. All other methods utilize bi-directional long short-term memory model (BiLSTM) (Hochre- iter & Schmidhuber, 1997) with linear embedding layer. Both the embedding dimension and the hidden size of LSTM set to 32. For composite methods that use two models, we use one-layer LST for each of them. For the other algorithms that only use LSTM,, set its number of
17

---

Published as a conference paper at ICLR 2022
KL-TI
Figure 3: Diversity visualization results for two TfBind tasks.
IS-A
IS-B
IR
IPS-A
IPS-B B
IPR
RANDOM Evo.
DBAS FB VAE
POU6F2
5
10
8
2
KLF1L_R422.RI
6
5
7
9
10
4
3
EGR2,R359W
5
8
9
7
6
1
2
10
3
HOXD13 S316C R1
10
3
4
7
9
5
2
HOXB7 K191R R1
8
6
4
9
3
5
2
PBX4 REF
4
3
GFIIB A204TTRI
7
3
4
9
10
6
2
FOXCI RI
10
8
3
5
7
9
4
2
ELFI_RE_RR
4
7
9
8
10
3
5
SIX6_REF.RY
10
5
4
ARX.L343Q.
4
7
9
5
6
2
CRX ES0A R1
10
7
5
8
2
6
4
ESXI_KI92S RI
6
3
5
10
8
4
2
VSXI VSXI.G160D R1
7
4
6
10
8
1
3
2
AGERAGE
8.43
6.93
4.79
6.21
8.36
7.00
1.14
4.36
5.07
2.71
Table 4: Top- 10 score ranking for the TfBind instances that we adopt. evolution algorithm.
"Evo." stands for
layers to be two. No Dropout (Srivastava et al., 2014) is used in LSTM models. In this way, the num- ber of parameters of the VAE slightly larger than that of the two layer BiLSTM, and all methods (except Random) share similar model parameter size.
All the experiments are repeated with fifty random seeds and report the mean value (and also stan- dard deviation in the figure plots). We set p(m) be uniform prior for all tasks for simplicity, which uniformly samples from the dictionary V for each entry of the sequence. For length alterable task, first uniformly sample the length between minimum length and maximum length and then sample each entry.
We explain details about evolution based method mentioned in the main text, which can be seen as substantial example of directed evolution (Chen & Arnold, 1991). Like other model based methods, Evolution also trains an LSTM regressor to predict the score of sequence, which further used to assist in the reproduce procedure. The Evolution algorithm maintains a generation list through the whole exploration process. In each round, the method mutates and reproduces the sequences to enlarge the generation list, and then utilizes the learned regressor to select top sequences for the next generation.
For validation, we follow (Angermüller et al., 2020b) and sweep each algorithm for fifty trials and pick the best configuration. We tune learning rate and whether to re-initialize the optimizer for each new round for all methods. We tune threshold for DbAS, FB-VAE and the methods that is with Example A. For the other choice of E, we tune the temperature. For evolution, we tune the number of offsprings for each sequence in generation list, the probability of substitution, insertion and deletion. For TfBind we use ZNF200 S265Y RI for validation. For UTR, AMP and Fluo, since only have oracle instance for each benchmark, do not hold-out validation method.
18

---

Published as a conference paper at ICLR 2022
IS-A
IS-B
IR
IPS-A
IPS-B
IPR
RANDOM EvO.
DBAS
FB-VAE
POU6F2 REF R1
10
8
3
5
9
6
7
4
2
KLF11 R402Q RI
10
7
4
8
9
6
5
2
EGR2.R359W
10
3
2
HOXD13_S316C.RI
10
9
3
4
7
6
5
2
HOXB7_K191R.N
10
9
3
6
8
4
5
2
PBX4_REF_R2
10
9
7
5
6
4
8
2
GFIIB N.BBA2244..RG RI
5
9
8
2
FOXCI REF
8
9
S
REFIREF_RR
6
5
9
7
10
4
2
SIX6_REFER
7
5
10
8
4
6
2
ARX L343Q
10
7
4
8
9
3
2
CRX E80A
2
ESXI_K193R
10
9
3
4
8
6
5
2
VSXI VSXI_GI _G160D_RI
10
9
3
5
8
7
4
2
AVERAGE
9.57
7.93
4.07
6.71
7.64
6.00
1.00
5.50
4.57
2.00
Table 5: Top- 100 score ranking for the TfBind instances that adopt.
"Evo." stands for the
evolution algorithm.
IS-A
IS-B
IR
IPS-A
IPS-B
IPR
RANDOM
EVOLUTION
DBAS
FB-VAE
UTR
10.87
11.71
11.43
12.06
12.15
11.94
10.60
11.20
11.89
10.65
AMP
-2.98
-2.67
-284
22.44
-2.6
2.74
-3.36
-3.09
-2.73
-2.0
FLUO
35.31
35.82
35.59
46.19
44.28
43.88
34.33
37.40
43.45
34.78
Comparison of the area under top-10 curves for UTR, AMP and Fluo benchmarks. Larger Table 6:
area means better sample efficiency.
the following transcription factor instances and treat them differ- ent black- box optimization tasks: ZNF200 S2655 RI, POU6F2 REF RI 8, KLF11 R402Q R1, EGR2 R359W R1, HOXD13 S316C RI, HOXB7 K191R RI, PBX4 REF R2, GFIIB A204T R1, FOXCI RI, KI3RRII and VSXI.GI6OD.RI We do not do post-processing such score nor- malization whitening for the data for simplicity. We first calculate the area under curve to summarize the performance in a sealar output, and put the ranking result for each algorithm in Table 4 and Table 5, which provide more details for Table 2. To further investigate the diversity of different algorithms, we choose two TfBind instances (POU6F2 REF and KLF1 SLF11_R402Q) R402Q_RI and visualize the resulting sequences with T-SNE (van der Maaten & Hinton, 2008) in Figure 3. We provide two visualization views for both task instances: (1) we uniformly sample 20 sequences from the whole n R sequences for each algorithm and visualize them; (2) for each algorithm, we visualize 20 sequences uniformly sampled from the last batch (i.e., at the last round). This is notated with "final sequences" in the figure. We do visualize all the sequences for simplicity. We use Hamming distance in the computation T-SNE. From Figure 3, we can see that there no obvious difference for the evaluated methods. This indicates that our proposed methods can achieve better performance while maintaining on-par diversity level with the baselines. This is not exactly consistent the findings of Angermüller et (2020a), which claims some algorithms such as DbAS achieve very limited diversity. We do not use the "optima fraction" metric in Angermüller al. (2020a;b), since this metric may not deal with multimode oracle landscape well and needs extra unstable computation such as clustering. Besides, this metric cannot generalize to other benchmarks.
For TfBind benchmark,
REFIRE
HIX6REEFR1.
ARX
CRX_ESOA_RI,
We elaborate the construction of our AMP oracle. We use the AMP dataset from (Witten & Witten, 2019) which contains 6,760 AMP sequences. A multilayer perceptron classifier trained to predict if protein sequence can prohibit the growth of a particular pathogen in that AMP dataset. This clas- sifier operates on the features extracted by ProtAlbert (Elnaggar et al., 2020) model. Following the setup in (Angermüller et al., 2020b), treat the predicted logits the ground- -truhthtrthhhho measurement. Moreover, we demonstrate the area under Top- 10 curves for UTR, AMP and Fluo benchmarks in Table 6, which is good complement for Table but is missing due to limited space in the main text.
19

---

Published as a conference paper at ICLR 2022
C RELATED WORKS AND DISCUSSION
Likelihood-free inference. We have already introduced the main classes of likelihood-free infer- ence algorithms in the main text: (1) Approximate Bayesian Computation (ABC) method (Beaumont et al.. 2009; Blum, 2009; Marin et al., 2012; Lintusaari et al., 2017) in Section 3.1; (2) Posterior modeling method that is also stated Section 3.1, including classical ones (Tran et al., 2015; Li et al., 2017: Chen & Gutmann, 2019) and modern SNP methods (Papamakarios & Murray, 2016; Lueckmann al., 2017: Greenberg et al., 2019): (3) Likelihood modeling method described in Sec- tion 3.2, also containing various classical algorithms (Wood, 2010; Mengersen et al., 2012; Drovandi et al., 2018) and modern SNL variants (Lueckmann et al., 2018; Papamakarios et al., 2019); and (4) Probability ratio modeling methods mentioned in Section 3.3 diverge in estimating likelihood ratio (Gutmann & Hyvärinen, 2010; Gutmann et al., 2018; Brehmer et al., 2020) or likelihood-to--w to-evidence ratio (Thomas al., 2016; Izbicki et al., 2014), where the latter paradigm is good fit for LFI prob- lem (Hermans et al., 2019). Besides, there are also works about how to construct wiwddimnsiooal summary statistics for LFI (Fearnhead & Prangle, 2012: Chan et al., 2018; Chen et al., 2021).
Machine learning based drug design. Generative modeling and discriminative modeling are two basic ways of thinking in machine learning. In literature for sequence design, generative modeling is also known as cross entropy method. This is famous kind of design method that is close to our "backward modeling of the mechanism" approach. Cross entropy methods seek to solve an expec- tation maximization problem (i.e., maxp pp(mm) [mm]) where the sequences follow a distribution p. This can also be related to simulated annealing, large family of black-box optimization algorithm the sequential neural posterior could be thought to maintain a distribution which is gradually be- coming sharper to a delta distribution at the optimal value. On the other hand, we think of this as a way for modeling the posterior p( (m EE) and develop corresponding analysis under the probabilistic framework, which is like a more accurate version of cross entropy method. Many related methods (including the ones stated in Section 3.1) train the distribution by likelihood maximization for quences with large scores, or use some sort of reweighting to achieve similar effects (Rubinstein & Kroese, 2004; de Boer al., 2005; Neil et al., 2018; Gupta & Zou, 2019; Brookes et al., 2019).
Discriminative modeling usually goes in a mmodel-based optimization" way (terminology from Angermüller et al. (2020a)), i.e,, use a discriminative model f(m to fit the real oracle ((m) and act as surrogate for it. The surrogate model can replace the true oracle f(m) which involves costly bi- ological experiments. This corresponds to our "forward modeling of the mechanism" in Section 3.2. Bayesian optimization (Shahriari et al., 2016) is a classical example, which utilizes f (typically a Gaussian process model) to define an acquisition function to guide the exploration and exploitation. Many modern biochemical methods also belong to this category
et al., 2018; Hashimoto et al., 2018; Yang et al., 2019; Wu et al., 2019; Sample et al., 2019; Liu et al., 2020).
There seems not much related literature about probability ratio estimation based method in this topic. On the other hand, Hashimoto et al. (2018) shares a classification based approach with IR but they differ how to the classifier. This algorithm utilizes the learned classifier to update the proposal with multiplicative weights algorithm, making the proposal to have large probability where the classifier logit is small. Other categories of drug design methods include evolution algorithms (Brindle, 1980; Wierstra et al., 2008; Salimans et al., 2017; Yoshikawa et al., 2018; 8; Jensen, 2019 1;; Real et al., 2019; Ahn et al., 2020) that search over the target space with genetic operators like insert, mutation, and crossover, and reinforcement learning (Guimaraes et al., 2017: Neil et al., 2018; Zhou al., 2019; Shi et al., 2020; Angermüller et al., 2020b) which see the formation of drug as a Markov decision process and train the policy to learn reghlyggewarding drugs. We do not find other work that is similar to our probability ratio modeling approach (Section 3.3) from the literature, which we take as a novel contribution.
20


Reviews:
[{"review_id": "jbo_2vhiic1", "reviewer": "Reviewer_LzqG", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The link described in this work is interesting and the novel algorithms proposed contain significant differences to existing sequence optimization techniques. Empirical results support claims that these novel algorithms are interesting and bear consideration for future design efforts. \n\nStrengths\n- Both Iterative Scoring and Iterative Ratio lead heavily on supervised learning (outputting low-dimensional predictions) compared to many existing design algorithms. Training regression models instead of likelihood models on protein sequence space could yield useful empirical advances, making this an interesting contribution.\n- The empirical evaluations are on well-known datasets and baselines appear to be used correctly. Results align with the conclusions of the paper.\n- Drawing a distinction between \"forward modeling\" and \"backward modeling\" could provide generally useful language for the community and enable communication of ideas.\n\nWeaknesses\nThe main weakness is in presentation of the link itself. In general the link seems to be \"correct\" in the sense that it is meaningful and consistently applied to link algorithms. However its exposition does give clear intuition for what is going on. I recognize this is not easy and try to provide some useful feedback below.\n- On the one hand, the quantities (theta, x) have clear distinctions as parameters and data. On the other hand, the quantities (E, m) are a set of sequence and a sequence and do not have such a clear distinction. It is quite hard to see how the set E pops out of the mapping T described beneath Table 1. \n- The notation p(E | m) = p(m \\in E | m) is very confusing. The function p(E | m) reads like a probability distribution over subsets of sequence space, not a distribution over sequence space restricted to the subset E. I recommend finding a better notation here. This comment is based on equation 3, which I could be misunderstanding.\n", "summary_of_the_paper": "The authors describe a mapping from likelihood free inference to black-box sequence optimization, then use this mapping to link common algorithms in both fields. They go on to describe novel black-box sequence design algorithms induced by known LFI algorithms. Empirical results show their methods are competitive on standard datasets.", "summary_of_the_review": "The contribution of algorithms which heavily rely on regression / classification to guide sequence design is interesting, especially since they are the product of a general mechanism for producing sequence optimization methods. The empirical results seem sound. The exposition of the model could use work to help readers have a crisper sense of how probabilistic modeling gets linked to a problem setting where the only randomness is in experimental noise.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FTUaQdJ-nhn", "reviewer": "Reviewer_JsdK", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Can you please comment on the relationship between your work and 'Derivative free optimization via repeated classification' https://arxiv.org/abs/1804.03761?\n\nThe paper builds up a variety of optimization methods, building on a line of work in the LFI literature. This leads to a lot of approaches to compare, and there is no clear indication as to what practitioners should use in practice. What do you actually suggest people should use?\n\nAlong these lines, the paper has far too few details about the actual optimization approaches. For example, IS-A and IS-B seem to be some of your strongest methods and these require MCMC to sample new proposed sequences. There is no discussion of how this MCMC is done. Further, there is no discussion of neural network architectures, optimization methods, etc. The paper would be stronger if it just focused on one method and provided sufficient details for practitioners to actually use it.\n\nI found this sentence in sec 4.2 very unsatisfying. Do you have any further insights about performance differences? \" This indicates that composite methods’ way of using parameterized models to replace computational procedures is not the optimal solution for small-scale tasks.\"\n\nI was surprised that the error bars were so small in all of your experiments, as I expect that the trajectory of an optimizer has high variance due, for example, to the initial set of sequences that are sampled. What do your error bars correspond to? Are they standard errors or standard deviations? Also, what sources of randomness are you accounting for when you generate multiple random trials?\n\nFor the methods that combine both a generative and discriminative models, there are two sources of approximation error. It would be very helpful if you isolated these by performing oracle experiments, where you assume that the forward model is exactly correct. This can be used to isolate the impact, for example, of using the amortized sampler q_\\phi in Alg 10. Just replace the forward model with the ground truth objective function. Can you run a quick experiment?\n", "summary_of_the_paper": "The paper draws on connections between likelihood-free inference and black-box optimization to propose new black-box optimization methods. In general, the goal here is to not find the exact optimum of the black-box objective, but to sample from a set of sequences with high-quality objective. This is akin to the problem of collecting posterior samples in a likelihood-free inference problem. \n\nThe paper provides a number of proposed methods and compares them on some benchmark sequence optimization problems that have appeared in recent literature.", "summary_of_the_review": "The paper has some interesting methods, and the connection to LFI is helpful. However, it does not have a clear empirical recommendation for what algorithm readers should use going forward and the paper does not provide adequate details to understand how to go about actually using these methods in practice.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "EXdydVliLk2", "reviewer": "Reviewer_1dZH", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Major comments\n1) The paper is missing a related works section with an overview of existing methods for sequence design and likelihood free inference, and how they relate to the methods that were introduced in this paper.\n\n2) Section 3.1: FB-VAE and DbAs are both Estimation of Distribution Algorithms (EDA) based on a generative model. EDA is closely related to expectation maximization (https://arxiv.org/pdf/1905.10474.pdf). Please describe more clearly the differences and similarities of SNP, EDA, and EM.\n\n3) Section 3.1: Please describe the differences between FB-VAE and DbAs more clearly. Both approaches update a VAE iteratively by fitting it on the top scoring sequences. What does 'top' mean? Are there differences in the 'fitting' of the VAE?\n\n4) Section 3.2: IS is a discriminative approach for sequence design (or blackbox optimization) similar to (Bayesian) model-based optimization (MBO). Please describe the similarities between IS and MBO and differences (if there are any).\n\n5) Section 3.2: Please describe what 'construct q(m) using f(m)' means. What is f(m) and q(m) in your experiments? How were they trained and which hyper-parameters were optimized?\n\n6) Section 3.2: What are the differences between IS-A and IS-B?\n\n7) Section 3.3: IR seems like a minor variation of IS that uses a classifier instead of a regressor as surrogate models, which is not new. Are there any other differences?\n\n8) Section 3.3: Please describe which models you used for IR in your experiments and how sequences were generated.\n\n9) Section 3.4: Please describe more clearly how IPS and IPR relate to (and differ from) existing design methods that combine generative and discriminative models, e.g. RL, GANs, or optimizing a surrogate model using DbAs/CbAs, for example.\n\n10) Section 3.4: Please describe the differences between IPS-A and IPS-B more clearly, including 'differ in the detailed construction of the distribution q(m)'.\n\n11) Experiments: Since the performance of algorithms can be sensitive to the batch size, I would like to see experiments with a different batch size than 100, e.g. small (1), medium (100) and large (500).\n\n12) Experiments: Please compare to Bayesian Optimization (and RL if possible) using the same surrogate model as used for IPR/IPS and tuning hyper-parameters in the same way.\n\n13) Experiments: Please describe which hyper-parameters you tuned and how they were tuned.\n\n14) Experiments: Please describe what the boolean feature \\mathcal{E} is in all or your experiments.\n\n15) Experiments: Please describe the 'Evolution' baseline more clearly. \n\n16) Experiments: Please motivate why you report the average reward of the top-10/100 sequences instead of just the maximum reward (top-1). The average can be maximized by reporting identical or very similar sequences. More important for practical applications is that the optimizer finds a diverse set of high-reward sequences as explained in Angermueller et al, who used additional diversity metrics to quantify this.\n\n17) Experiments: How did you initialize the optimization? I would like to also see experiments that are initialized with a small set of labeled sequence (e.g. one or few parent sequences/homologs), which often exist in practice.\n\n\n# Minor comments\nSection 1, 'de novo biological sequence design': Describe which kind of sequence (DNA, RNA, protein, molecules represented as strings, ...?).\n\nSection 2, 2nd paragraph: Please cite reviews (e.g. http://arxiv.org/abs/2106.05466, http://www.nature.com/articles/s41592-019-0496-6) of existing ML design methods instead of single papers (Ahn, Gottipati, ...).\n\nDenoting 'm' as sequence and 's = f(m)' as the oracle function value is confusing since 's' is the first letter of *s*equence. I strongly suggest to use 's' to denote a sequence and, for example, 'y = f(s)' to denote the function value.\n\nYour benchmark problems seem to be similar to the benchmark problems introduced in Angermueller et al (except for Flu). If this is the case, please describe that you reused the benchmark problems from Angermueller et al and describe possible differences. By referencing Angermueller et al instead of describing each benchmark problem in detail, you can also shorten the Experimental section.\n\nExperiments (Flu). You hypothesize that backward modeling techniques perform better since sequences are long. Although sequences are long, there may be only a few variable positions while most positions are conserved. Since generative models can easily fit this conservation better they may perform well in this case since the optimization problem becomes trivial as only few variable positions are mutated.", "summary_of_the_paper": "The paper relates likelihood free inference to methods for biological sequences design and proposes new sequence design methods based on this insight.", "summary_of_the_review": "* The outlined relation between likelihood free inference and sequence design (blackbox optimization) is interesting; I am not aware of any existing papers with this insight. However, I am not sure about the impact of this insight on how sequences are designed in practice.\n* It is not described clearly enough how the proposed methods differ from existing methods for sequence design such as Bayesian model-based optimization, GANs, or RL.\n* Important details about the proposed methods and performed experiments are missing, which makes it hard to understand and assess.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "3wL73HiLy5t", "reviewer": "Reviewer_5Uhg", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I found the paper to be extremely clear and well written, providing an excellent review of both the LFI and black-box sequence design literatures and drawing clear, clean parallels between the two fields.  The proposed algorithms are all sensible and seem to work well on the empirical tasks, and the connection between the two fields seems like a fruitful area for further exploration (especially using the presented framework).  I have few comments below:\n\nTypos:\n- On p. 4 it is stated that \"...to model the general posterior $p(\\theta|\\mathbf{x})$, which takes arbitrary $\\theta$ and $\\mathbf{x}$ as two inputs and outputs a distribution\", but these models only take $\\mathbf{x}$ as input and output the conditional distribution over $\\theta$ (i.e., the posterior).\n- There are a number of typos in the supplementary materials (appendix).  The meaning was always clear, but that document could use some thorough copy editing.", "summary_of_the_paper": "In this paper the authors draw direct parallels between likelihood-free inference (LFI) and black-box sequence design.  This allows that authors to draw parallels between existing methods from the LFI and black-box sequence design literatures.  In a few cases there is no direct analog in the black-box sequence design literature for a given LFI algorithm, and so the authors are able to immediately propose such an algorithm.  The authors also present a number of \"composite\" methods that combine ideas from a number of these approaches.", "summary_of_the_review": "The paper is clear and well-written and provides a very useful conceptual framework tying two subfields together, with sufficient empirical evidence to show real gains from this approach.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper investigates various approaches, and a unifying framework, for sequence design. There were a variety of opinions about the paper. It was felt, after discussion, that the paper would benefit from a sharper focus, and somewhat suffers from being overwhelmed by various approaches, lacking a clear narrative. But overall all reviewers had a positive sentiment, and the paper makes a nice contribution to the growing body of work on protein design.
Decision: Accept (Spotlight)