Multi-Fidelity Active Learning with GFlowNets
Authors: Alex Hernández-García
Nikita Saxena
Moksh Jain
Cheng-Hao Liu
Yoshua Bengio
Venue: Accepted by TMLR ()

Abstract: In the last decades, the capacity to generate large amounts of data in science and engineering applications has been growing steadily. Meanwhile, machine learning has progressed to become a suitable tool to process and utilise the available data. Nonetheless, many relevant scientific and engineering problems present challenges where current machine learning methods cannot yet efficiently leverage the available data and resources. For example, in scientific discovery, we are often faced with the problem of exploring very large, structured and high-dimensional spaces. Moreover, the high fidelity, black-box objective function is often very expensive to evaluate. Progress in machine learning methods that can efficiently tackle such challenges would help accelerate currently crucial areas such as drug and materials discovery. In this paper, we propose a multi-fidelity active learning algorithm with GFlowNets as a sampler, to efficiently discover diverse, high-scoring candidates where multiple approximations of the black-box function are available at lower fidelity and cost. Our evaluation on molecular discovery tasks shows that multi-fidelity active learning with GFlowNets can discover high-scoring candidates at a fraction of the budget of its single-fidelity counterpart while maintaining diversity, unlike RL-based alternatives. These results open new avenues for multi-fidelity active learning to accelerate scientific discovery and engineering design.

Published in Transactions on Machine Learning Research (07/2024)
Multi-Fidelity Active Learning with GFlowNets
Alex Hernandez-Garciat
alen.lernandees-grriieeniiiii Svmilla quebec
Mila, Université de Montréal
Nikita Saxena" i
Birla Institute of Technology and Science, Pilani
Moksh Jain Mila, Université Montréal
mokkh.jain Aimila. Guebla.uueee
Chong-Hao Liu
chenghao. http:/aw...ummmiimmmmccmmoc
Mila, McCill Universily
Yoshua Bengio
Benkuaabenggio@nnnn quebec
Mila, Université Montréal, CIFAR Fellow, IVADO
Reviewed on OpenReview: https: // oppnnreveew. net/ net/ forum? id- dLoazW9zuF
Abstract
In the last decades, the capacity to generate large amounts of data in science and engineer- ing applications has been growing steadily. Meanwhile, machine learning has progressed to become a suitable tool process and utilise the available data. Nonetheless, many rele- scientific and engineering problems present challenges where current machine learning methods cannot yet efficiently leverage the available data and resources. For example, in sci- entific discovery, are often faced with the problem of exploring very large, structured and gigh-dimensional spaces. Moreover, the high fidelity, black-box objective function is often very expensive to evaluate. Progress machine learning methods that can efficiently tackle such challenges would help accelerate currently crucial areas such as drug and materials dis- covery. In this paper, we propose multi-fidelity active learning algorithm with GFlowNets as a sampler, to efficiently discover diverse, high-scoring candidates where multiple approx- imations of the black-box function on molecular discovery lasks shows that multi-fidelity active learning with GFlowNels can discover high-scoring candidates at a fraction of the budget of its single-fidelity counterpart while maintaining diversity, unlike RI-based alternatives. These results open new avemues for multi- malti-fideiiyy active learning to accelerate scientific discovery and engineering design.
available at lower fidelity and cost. Our evaluation
Introduction
To lackle the most pressing challenges for humanity, such as the climate crisis and the threat of pandemics or antibiotic resistance, there is a growing need for new scientific discoveries. By way illustration, new matcrials can play an important role improving the efficiency of energy production and storage; and reducing the costs and duration of drug discovery cycles has the potential to effectively and rapidly mitigate
"Eqiivlent Contribution
Work done during internship Mila

---

Published in Transactions on Machine Learning Research (07 (07/2024)
D= (U (ai), mi)
Acquisition function a(2; m)
train
Reward
Data
f'M
fM-1
Multi-fd. surrogate
GFlowNet
Oracles see
Am
Aar-1
A2
A
p(fm (z,, m, D)
Ami cost
(a, m)
Top B
((s,m)))ssss)
Queries
Sample
(Algorithm D. Given a set of M oracles fi...., fas (center left) with varying fidelities and costs <... < xas respectively, we can construct a data set D (top left) with annotations from the oracles. With this data, we fit a multi-fidelity surrogate (center), modelling the posterior p(fm(x)) a. D). The surrogate is used to evaluate a multi-fidelity acquisition function-max-vane max-value entropy search in our experiments- which makes the reward to train Gel (right). The GFlowNet is trained to sample both object and the fidelity proportionally to this reward. Once the GelowNet is trained, sample N tuples (r, and select the top B according to the acquisition function (bottom left). Finally, we annotate each new candidate with the selected oracle, add them to the data set and repeat the process until the budget is exhausted.
Figure 1: Illustration of multi-fidelity active learning with GFlowNets (Aleenessssllooo
the consequences new diseases. recent years, researchers in materials science, biochemistry and other fields have increasingly adopted machine learning as a tool since holds the promise drastically accelerate
scientific discovery (Butler al. 2018 Zitnick et al. 2020 Bashir al. 2021 Das al. 2021
Although machine learning has alrcady made positive impact in scientific discovery applications (Stokes
et al. 2020 JJumper et. al. 2021), unleashing its full polential demands improving the current algorithms (Agrawal & Choudhary| 2016). For example, typical tasks in potentially impactful applications in materials and drug discovery require exploring combinatorially large, structured and high-dimensional spaces (Bohacck) et al. 1996 Polishchuk et al. 2013). where only small, noisy data sets are available. Furthermore, oblaining new annotations computationally or experimentally is often very expensive. Such scenarios present serious challenges even for the most advanced machine learning methods currently available.
In the search for useful discovery, we typically define quantitative proxy for usefulness, which we view as black-box function. One promising avenue for improvement is developing methods that more efficiently leverage the availability of multiple approximations of the target black-box function at lower fidelity but much lower cost than the highest fidelity oracle (Chen et al. 2021 Fare et al, 2022 For example, standard tool to characterise the properties of materials and molecules quant mechanics simulations such as Density Functional Theory (DFT) (Parr) 1980 [Sholl Stockel 2022). However, DFT is computationally too expensive for high-throughput exploration of large search spaces. Thus, large-scale exploration can only be achieved through cheaper but less accurate oracles. Nonetheless, solely relying low-fidelity approxinations is clearly suboptimal. Ideally, such "needle-in-a-haystaad" problems would be best tackled by methods that can efficiently and adaptively distribute the available computational budget between the multiple oracles depending on the already acquired information.
Another challenge that even the highest fidelity oracles are often undcrspecified with respect the actual, relevant, downstream applications. This inderpeecification problem can be mitigated by finding multiple candidate solutions (Jain et. al.] 2023a However, most current. machine learning methods used
2

---

Published in Transactions on Machine Learning Research (07/2024)
in scientific discovery problems, such as Bayesian optimisation (BO, Song et al. reinforcement learning (RI., Angermueller et al. 2020), are designed for global optimisation of the target function. Therefore, it is imperative to develop methods that not only find the global optimum, but also discover sets of diverse, high-scoring candidates.
2018 Garnett 2023 and
Recently, generative flow networks (GFlowNets or GFN, Bengio et al. 2021a) have demonstrated their ability to find diverse candidates through discrete probabilistic modelling, with particularly promising results when used in active learning loop (Jain et al., 2022). In this paper, we propose multi-fidelity active learning algorithm enhanced with these capabilities of GFlowNets. Our contributions can be summarized as follows:
We introduce a multi-fidelity active learning algorithm designed for combinatorially large, structured and high-dimensional spaces.
We propose an extension of GelowNets for this multi-fidelity setting, to sample both candidates and oracle indices, proportionally to a given acquisition function.
We conduct a comprehensive empirical evaluation across four scientific discovery tasks and demon- strate that multi-fidelity active learning with GFlowNets
discovers high-scoring samples with reduced computational costs compared to its single-fdellity counterpart, and
identifies multiple modes of the target function, unlike methods relying on reinforcement learning or standard Bayesian optimisation, thereby facilitating diverse sampling.
2 Related Work
Our work can be framed within the broad field of active learning (AL), class of machine learning methods whose goal is to learn an efficient data sampling scheme to accelerate training (Settles) 2009). For the bulk of the literature in AL, the goal is to train an accurate model h(z) of an unknown target function f(r), as in classical supervised learning. However, in certain scientific discovery problems, which motivate our work, a desirable goal is often instead to discover multiple, diverse candidates æ with high values of f(x), as discussed in Section
Our work is also closely connected to Bayesian optimisation (BO, Garnett 2023 Snoek et al., 2015 which aims at optimising a black-box objective function fxx that is expensive to evaluate. In contrast to the problems we address in this paper, slandard BO typically considers continuous domains and works best in relatively cow-dimeniional spaces (Frazier) 2018). Nonetheless, in recent years, approaches for BO with structured data (Deshwal & Doppal 2021 Papenmeier et al., 2023) and high-dimensional domains (Grosnit) le al. 2021 have been proposed in the literature. The main difference between standard BO and the problem we tackle in this paper that we are interested in finding multiple, diverse samples with high value of f and not only the optimum. Recent work by Maus et al. (2022) has proposed variant of traditional BO to find diverse solutions.
This goal, as well as the discrete nature of the search space, is shared with active search Garnett et al. 2012), a variant of active learning in which the task is to efficiently find multiple samples of a valuable (binary) class from a discrete domain X. This objective was already considered in the early 2000s by Warmuth et al. (200T) for drug discovery, and more formally analysed in later work (Jiang et al. 2017 2019). Another recent rescarch arca in stochastic optimisation that considers diversity is so-called Quality- Diversity (Chatzallggeroudis et al., 2021],, ; which typically uses evolutionary algorithms that search in a latent space. These and other problems such as multi-armed bandits (Robbins 1952) and the general framework of experimental design (Chaloner & Verdinelli 1995) all share the objective of optimising or exploring an expensive black-box function. Formal connections between some of these areas have been established in the
literature (Srinivas et al. 2010 Foster 2021 Jain et al. 2023a Fiore et al. 2023
Multi-fidelity mothods have been proposod in most of these arcas of rescarch. An carly survey on multi- fidelity methods for Bayesian optimisation was compiled by PPeherstorfer et al. (2018), and research on the subject has continued since with the proposal of specific acquisition functions (Takeno et. al.] 2020) and the
3

---

Published in Transactions on Machine Learning Research (07/2024)
Recently, works on multi-fidelity 2021), J, but interestingly the literature on multi-fidelity active learning QLi et al. 2022a) is scarcer. Finally, while multi-fidelity methods have started to be applicd in scientific discovery problems (Chen et al.) 2021 Fare et al. 2022) the literature is still limited probably because most approaches cannot. tackle the specifics of scientilic discovery, such as the need for diverse samples. Here, we aim at addressing this need with the use of GFlowNets (Bengio al. 2021a Jain et al. 2023b) for multi-fidelity active learning.
use of deep neural networks to improve the modelling (Li et al. 2020
active search have also appeared in the literature (Nguyen et al.
3
Method
In this section, we first briefly introduce the necessary background on GFlowNets and active learning. Then, we describe the proposed algorithm for multi multi-fideltty active learning with GFlowNets.
3.1 Background
Generative flow networks (GFN; Bengio et al. 2021a by are amortised samplers originally designed for sampling from discrete high-dimensional distributions. Given a space of compositional objects X' and non-negative reward function R(x)) GFlowNets are designed to learn stochastie policy =(x) that generates X with a probability proportional to the reward, that is T(x) R(x). This distinctive property induces sampling of diverse, high-reward objects, which is desirable property for scientific discovery, among other applications (Jain et al. 2023a).
GFlowNets
A key property of GetowNets is that objects x E X are constructed sequentially by sampling transitions 8 E A between partially constructed objects (states) E S, which includes a unique emply state 80- The stochastic forward policy is typically parameterised by a neural network PF 0), where 0 denotes the learnable paramcters, which models the distribution over transitions from the current state 8t to the next state 8141- The backward transitions are parameterised too and denoled Pn( (5((118441;0)) 0). Objects are generated by the sequential application of Pr, forming trajectories T (so - 81... - æ). To learn the parameters 0 such that (x) x R(x) we use the trajectory balance learning objective (Malkin et al. 2022
Ze II Pr( )
CTB(T; logg R(x) I1 Pn (n(8:(1441;0)
(1)
where Ze is a trainable approximation of the partition function Enex Mxx). The GFlowNet learning objective supports training from off- off-policy trajectories, so during training the trajectories are typically sampled from a mixture of the current policy with a uniform random policy. The reward can also be tempered to make the policy focus on the modes (sce Appendix C.3
Active Learning In its simplest formulation, the (single fidelity) active learning problem that we consider is as follows: start with initial data set D {(2,, (xx))} of samples X' and their evaluations by expensive, black-box objective function (oracle) f X - R, which we use to train a surrogate model M(r). A Net can then be trained to learn a generative policy Te(I) using h(x) as reward function, that is R(x) = h(x). After training, the policy To (z) can be used to generate a batch of samples to be evaluated by the oracle f, add them to the data set and repeat the process a number of active learning rounds.
As an alternative, instead of directly using the surrogate output as the reward, we can instead train a probabilistic surrogate D) and use reward the output of an acquisition function ((x,p(f[D)) that considers the epistemic uncertainty of the surrogate model, as typically done in Bayesian optimisation. This is the approach by Jain et al. (202)) with GFlowNet-AL. An important difference between traditional BO and active learning with GFloo Nets is that the latter samples from the acquisition function instead of optimising it (Jain et al.) 2023a). This difference accounts for the capability of active learning with GFlowNets to discover diverse candidates: if multiple, diverse candidates have high values of the acquisition function, GFlowNet has the potential generate them with high probability, whereas standard BO seeks find the optimum only.
4

---

Published in Transactions on Machine Learning Research (07/2024)
While much of the active learning literature (Settles 2009 has focused on so-called pool-based active learning, where the learner selects samples from pool of unlabelled data, we here consider the scenario of de nowo query synthesis, where samples are selected from the entire object space X. This scenario is particularly suited for scientific discovery {King et al., 2004 Xue et al. 2016 Yuan et al.) 2018 Kusne et al., 2020 The ultimate goal pursued in active learning applications is also heterogeneous. Often, the goal is the same as in classical supervised machine learning: to train an accurate (surrogate) model h(x) of the unknown target function (x)) In many scientific discovery problems, we are not interested in the surrogate's accuracy across the entire imput space X, but rather in discovering new, diverse objects with high values of f. We have reviewed the literature that is connected to our work in Section 2
3.2 Multi-Fidelity Active Learning
We now consider the following active learning problem with multiple oracles of different fidelities. Our ultimate goal is to generate batch of K samples {<i] E X' according to the following desiderata:
The samples obtain a high value when evaluated by the objective function f: x RRt
The samples are diverse, covering distinct, high-valued regions of f.
Furthermore, we are constrained by a computational budget A that limits our capacity to evaluate f. While f is extremely expensive to evaluate, we have access to a discrete set of approximate functions (oracles) (fm]ismsM X' + RR+ where m represents the fidelity index and each oracle has an associated cost Am and level of confidence fm (0, 1]. We assume, without loss of generality, that the larger m, the higher the fidelity or confidence, that À A2 <... < Am A and that ts 1. We also assume fM = f because, even though there may exist more accurate oracles, we do not have access to them. This scenario resembles many practically relevant problems in scientific discovery and motivates our approach: because the objective function SM is not a perfect proxy of the true usefulness of objects x, we seek diversily; and because JM may be expensive to evaluate, we make use of approximate models.
In multi-fidclity active learning-as as well as multi-fidelity Bayesian optimisation-the iterative sampling scheme consists of not only selecting the next object (or batch of objects) evaluate, but also the level of fidelity m, such that the procedure is cost-effective. Namely, after each active learning round we acquire triplets (zz, fm, (I:), mz).
Briefly, our algorithm, MF-GFN follows these iterative steps: at each iteration j. we use the currently available data Dj to train a probabilistic multi-fidelity surrogate model h(x,, m). We can use the surrogate to compute the worth of annotating a candidate x with the oracle fm via an acquisition function a(x, m). Next, we train a GFlowNet with the acquisition function as a reward. Once trained, we sample N tuples (z, m)) and select the top B, as per the acquisition function. Finally, we annotate each candidate x with the selected oracle m and start over with the extended data set. Figure [ contains visual illustration of MF-GFN and more detailed descriptions are provided in Algorithm [ and in Appendices A and B Below, we further describe the surrogate model and the acquisition function, and in Section 3.3 we introduce multi-fidelity GFlowNets.
Surrogate Model Given a dataset D, a candidate x and an oracle index m, we want to model the posterior distribution over the output of the oracle, (x))x, m, D). A natural modelling choice is Gaussian Processes (GP, Rasmussen & Williams 2005). commonly used in Bayesian optimisation. However, in order to better model structured, high- linear embedding of the inputs 2 (x) is learnt by deep neural network with parameters w. Then, we use the following multi-fidelity GP kernel:
data, we use deep kernel learning (Wilson et al., 2016): First, non-
K KMr(2x, m). (x', m²)) Kx (x), (x())) + KM (m, m') x KX,, (z).
(2)
where both Kx (2, 2²) and Kx (2,z') are Matérn kernels (with different lengthscale each) applied to the latent representations of the inputs from the network and
KM(m, (1 -1 - (mr)() + ememe)
5

---

Published in Transactions on Machine Learning Research (07/2024)
Algorithm 1: MF-GFN: Multi-fidelity active learning with GFlowNets. A graphical summary of this algorithm is shown in Fig. 1
Input: [[m, Amy m) M oracles and their corresponding cost and confidence;
Do = {(oa, fm, (2,), mi)): Initial data set;
h(x, m): Multi-fidelity Gaussian Process surrogate model;
alz, m): Multi-fidelity acquisition function;
R(a(z, m), reward function with temperature parameter / to train the GFlowNet:
N: Number of candidates sampled from the GFlowNet;
B: Acquisition batch size of oracles queries;
A: Maximum available budget;
K: Number of top-scoring candidates to be evaluated at termination;
Initialisation: D 4- Do, A + 0, f-
while A₃ < A do
Fit. surrogale h data set D;
Train GelowNet with reward R(a(I, m), to obtain policy TT(#);
Sample N > B tuples (z,,m,) ~ No:
Score each tuple using a(x, m) and select the top B tuples with the highest scores;
Evaluate each tuple with the corresponding oracle to form batch
+ {(11, (I1), mi)..... (æD, fmg (IB), ma)};
Update data set 'D DUB, B, budget A5 <- Aj + Ee Am, and index j j j+b 1;
end Result: Top- Top-K(D), set of K candidates with largest values of the highest-fidelity oracle sMs
is applied over the fidelity confidences, which we set equal to the normalised costs lm Am (0, 1] for Au
simplicity. The kernel KMF is known as lincar truncated kernel, is implemented in BoTorch (Balandat ct al. 2020 and has been used before in multi-fidelity Bayesian optimisation works, for example by Mikkola
et al. 2023). Additional details are provided in Appendix B Finally, we would like to note that our multi- fidelity active learning algorithm is independent of the specific choice of surrogate model.
Acqusition Function Multi-fidelity methods proposed in the Bayesian optimisation literature have adapted information theory-based acquisition functions (Li et al.. 2022a Wu et al., 2023 Li et al. 2022b). In our work, we use the multi-fidelity version (Takeno et al.) 2020 of max-value entropy search (MES, Wang & Jegelka 2017). MES captures the mutual information bet the value of candidate x and the maximum value attained by the objective function, f*. The multi-fidelity variant, MF-MES, is designed to select the candidate x and the fidelity m that maximise the mutual information between ft and the oracle at fidelity m, fm, weighted by the cost of the oracle Am-
1
a(x, m)
I(frr: fm()))))) Am
We chose MES the acquisition function for approach bocause it has been shown to be more efficient
than plain entropy search (Wagg & Jegelka 2017 However, the algorithm is independent of the choice of acquisition function. Additionally, for efficiency, we adopt the GIBBON approximation of MF-MES, which
has demonstrated good performance in the context of multi-fidelity optimisation (Moss et al. 2021) We provide further details about the acquisition function and the GIBBON approximation in Appendix B
3.3 Multi-Fidelity GFlowNets
A multi-fidelity acquisition function can be regarded as a cost-adjusted utility function. Therefore, in order to carry out a cost-aware search, we seek to sample diverse objects with high value of the acquisition function. sample the fidelity
To this purpose, we propose use a GFlowNet as a generative model by training m in addition the candidate itself. In other words, it learns a sampling policy Tt( m). Formally, given GelowNet with state and transition spaces S and A we augment. the state space with new dimension

---

Published in Transactions on Machine Learning Research (07/2024)
for the fidelity M' 20. (including m 0, which corresponds to unset fidelity), such that the augmented, multi-fidelity state space is Sar = S x M' The set of allowed transitions Aar is augmented such that a fidelity m > 0 of a trajectory must be selected once, and only once, from any intermediate state. Intuitively, allowing the selection of the fidelity at any step in the trajectory should give flcxibilityy for better generalisation. At the end, complete trajectories are the concatenation of an object and the fidelity m > O, that is (x. m) E Xys = M.
In summary, the proposed approach learns policy that jointly samples objects in possibly very large, structured and sigh-dimensional space, together with the level of fidelity. Since use multi-fidelity acquisition function as reward function to train the multi-fidelity GFlowNet, the learnt sampling policy will be approximately proportional to the acquisition function, hence providing diversity. As in the single-fidelity case, if multiple, diverse tuples (r.m) obtaín high values of the acquisition function a, the GFlowNet can potentially sample them with high probability. In practice, we rescale the acquisition function so as further increase the relative rewards of high values of a, as detailed in Appendix C.3
4 Empirical Evaluation
In this section, we present empirical ovaluations of multi-fidelity active learning with GFlowNets. Through our experiments, aim to answer the following questions:
Can our multi-fidelity active learning approach find high-scoring, diverse samples at lower cost than with a single-fieelity oracle?
Does MF-GFN, which samples objects and fidelities m), provide any advantage over sampling only x and selecting m randomly?
4.1 Metrics
As discussed Section 3.2 our goal is to sample diverse objects with high scores according to a reward function. Following Gao et al. (202)) and Jain al. (2022), we here consider a pair of metrics that capture both the scores and the diversity of the final batch of candidates. These metrics are computed from the set of best K candidates, obtained out of the full data set Dj at iteration j. We refer to this set top-K which consists of the K candidates with the largest values of the objective function (highest-fidelity oracle) fM. The two metrics we consider in this section are:
Mean top-K score: mean score, per the highest fidelity oracle SM, of the top-K samples.
Top-K diversity: mean pairwise distance within the top-K samples.
Additional details and formal definitions are provided in Appendix D We report the metrics averaged over 3 experiments with different random seeds, as well as the 95 % confidence intervals of the mean top- scores estimated via bootstrapping.
Since here we are interested in the cost effectiveness of the active learning process, we evaluate the above metrics function of the cost accumulated in querying the oracles. It is important to note that multi- fidelity approaches are not aimed at achieving better mean top- K scores than single-fidelity active learning counterpart, but rather the same mean top-K scores but with a smaller budget.
4.2 Baselines
In order to evaluate our approach, and to shed light on the questions stated above, we consider the following baselines:
GFlowNet with highest fidelity (SF-GFN) GFlowNet-bascc active learning (GFlowNettAT.) as in Jain et al. (2022) with the highest fidelity oracle, to establish benchmark for performance without considering the cost-accuracy trade-offs.
7

---

Published in Transactions on Machine Learning Research (07 (074)
Diversity
Diversity
0.20
0.25
0.30
0.35
0.40
0.45
0.50
0.3
0.4
0.5
0.6
0.7
0.8
Mean Top-100 energy
14
Meen Top: 100 energy
0.8
12
8
0.6
SF-GEN
MF-PPO
Random
MF-GFN
4
Rand. fid. GPN
0.5
10-3
10-2
10-1
10
Fraction of total SF-GEN budget
Fraction of total SF-GEN budget (log)
(a) DNA aptamers task
(b) Anti-microbial peptides (AMP) task
Figure 2: Results on the DNA aptamers and AMP tasks. The curves indicate the mean energy far within the top-100 samples computed at the end of each active learning round and plotted as a function of the budget used. The colour of the round markers indicates the diversity within the batch (darker colour indicating higher diversity), computed as the average pairwise sequence identity distance (see Appendix D Tn both the DNA and the AMP tasks, MF-GFN outperforms all baselines in termss of cost efficiency, while obtaining great diversity in the final batch of top-K candidates.
GFlowNet with random fidelities (Random fid. GFN) Variant of SF-GFN where the candidates are generated with the GelowNet but the multi-fidclity acquisition function is evaluated with random fidelities sampled from a uniform distribution. This allows us to investigate the contribution of learning to sample the fidelity for each object with GllowNets.
Random candidates and fidelities ranked by the acquisition function (Random) Quasi-random approach where both candidates and fidelities are randomly sampled from uniform distribution. We first sample N random (x, m) pairs, then select the top B according to the acquisition function. This baseline is typically competitive due to the value provided by the acquisition.
Multi-fidelity PPO (MF-PPO) Instantiation of multi-fidelity Bayesian optimisation where the acquisi- tion function is optimised using proximal policy optimisation (PPO, [Shhlmann al. (2077). Unlike with the other baselines, include initialisation of n/3 steps where is the maximum number of steps allowed. This is to help exploration and diversity, since without it PPO tends to collapse to generation of very similar candidates.
4.3 Benchmark Tasks
As proof of concept, we perform experiments two low-dimensional synthetic functions: Branin and Hartmann, widely used in the multi-fidelity Bayesian optimisation literature (Perdikaris al. 2017 Song
et al. 2018 Kandasamy et al. 2019 Li et al. 2020 Folch et al. 2023). These tasks show that MF-GFN is able to obtain results comparable to other multi-fidelity BO methods. We provide these results in Appendix C.4 Nonetheless, the motivation of our work is the challenges posed by large, structured and high-dinensional problems common in scientific discovery. Therefore, in order to assess the performance of ME-GEN on such sconarios, we evaluate on more complex tasks of practical scientific relevance. We present results on a variety of discovery domains: DNA aptamers (Section 4.3.1h, anti-microbial peptides (Section 4.3.2 and small molecules (Section 4.3.3
8

---

Published in Transactions on Machine Learning Research (07/ (204))
4.3.1 DNA Aptamers
DNA aptamers are single-stranded nucleotide sequences of nucleobases A, C, T and G, with multiple applica- tions in polymer design due to their specificity and affinity as sensors in crowded biochemical environments [Zhou et al, 2017] Corey et al.] 2022] Yesselman al., 2019 Kilgour et al.) 2021). The objective is to maximize the (negative) free energy of the sccondary structure of DNA sequences. This free energy can be seen as proxy of the stability of the sequences. We compute the diversity as one minus the mean pairwise sequence identity among a set of DNA sequences (see Appendix D for the details).
Setting In our experiments, we consider fixed-length sequences of 30 bases and design a GelowNet envi- ronment where the action space consists of the choice of base to append to the sequence, starting from an empty sequence. This yields a design space of size |X] = 430 (ignoring the selection of fidelity in MF-GFN).
Further details about the task are discussed in Appendix C.5.1
Oracles NUPACK (Zadeh et al. 2011), a nucleic acid structure analysis software, is used as the highest fidelity oracle, far. As a low fidelity oracle, trained transformer model on million randomly sampled sequences annotated with f's, and assigned it a cost 100x smaller than the highest-fidelity oracle. The cost difference is selected to simulate practical scenarios where wet lab experiments take hours for evaluation, while cheap online simulations take a few minutes.
Results As presented in Fig. 2a MF-GFN reaches the best mean top-K energy achieved by its single- fidelity counterpart with just about 25 % of the budget. is also more efficient than GFlowNet with random fidelities and MF-PPO. Crucially, we also see that MF-GFN maintains high level of diversity (0.32), even after converging to the top-K scores. On the contrary. MF-PPO (0.20) is not able to discover diverse samples, as is expected based on prior work (Jain et al. 2022
4.3.2 Antimicrobial Peptides
Antimicrobial peptides are short protein sequences which possess antimicrobial properties. As protcins, these are sequences of amino-acids vocabulary of size 20 along with a special stop token. The aim is to identify sequences with a high antimicrobial activity, measured by model trained on DBAASP (Pirtskhalava ct al. 2021). The diversity calculation is equivalent to that of DNA.
Setting We consider variable-length protein sequences with up to 50 residues. Analogous to DNA, if we ignore the fidelity, this yields a design space of size |X > 20
Oracles We construct a three-oracle setup by training deep learning models with different capacities on exclusive subsots of data points. We simulated setup wherein the two lower fidelity oracles are trained on specific subgroups of the peptides. Similar to the DNA experiment, the lower-fidelity oracles had both cost 100x less than the highest fidelity oracle. Additional details can be found in Appendix C.5.2
Results Fig. 2b indicates that in this task MF-GFN obtains even greater advantage over all other baselines in terms of cost-efficiency. It reaches the same maximum mean top- K score as the random baselines with 10x less budget and almost 100x less budget than SF-GFN. In this task, MF-PPO did not achieve comparable results. Crucially, the diversity of the final batch found by ME-GFN stayed high (0.87).
4.3.3 Small Molecules
Molecules are clouds of interacting atomss described by set of quantum mechanical properties. These properties dictate their chemical behaviours and applications. To demonstrate the capability of MF-GFN in the setting of quantuu chemistry, we consider two tasks in molecular electronic potentials: maximisation of the (negative) adiabatic ionisation potential (IP) and of the adiabatic electron affinity (EA). These electronic potentials dictate the molecular redox chemistry, and are crucial in organic semiconductors, photoredox catalysis and organometallic synthesis. In this task, the diversity measure is the average pairwise Tanimoto distance among the top-K K scoring molecules (Bajusz al.] 2015), detailed in Appendix D
9

---

Published in Transactions on Machine Learning Research (07 (024)
Diversity
0.70
0.75
0.80
Diversity
0.85
0.90
0.95
0.75
0.80
0.85
0.90
0.95
Mean Top-1 100 energy [eV]
Moan Top- energy
4
50
-7
SF-GFN
MFPPO MFGFN
N.
SF-GFN
MF-PPO
Random
Random
MF-GFN
Rand. fid. GFN
Rand. fid. GFN
0.8
0.6
Fraction of total SF-GEN budget
Fraction of total SF-GFN budget
(a) Molecules ionisation potential (IP) task
(b) Molecules electron affinity (EA) task
Figure 3: Results on the molecular discovery lasks: (a) ionisation potential (IP), (b) electron affinity (EA). These visualisations are analogous to those in Fig. 2 The diversity of molecules is computed the average pairwise Tanimoto distance (see Appendix D). Results generally show MF-GFN's faster convergence in discovering diverse molecules with desirable properties.
Setting We designed the GFlowNet state space by considering variable length sequences of SELFIES tokens (Krenn al. 2020) represent molecules, with vocabulary size of 26. The maximum length was 64, resulting in design space of [x 2664
Oracles Numerous approximations of these quantum mechanical properties have been developed with dif- ferent methods at different fidelities, with the famous example of Jacob's ladder in density functional theory (Perdew & Schmidt,, 2001). We employed three oracles that correlate with experimental results as approxi- mations the scoring function, by using various levels of geometry optimisation to obtain approximations the adiabatic geometrress The calculation of TP or EA was carried out with the semi-empirical quantum chemistry method XTB (Neugebauer et al., 2020). These three oracles had costs of 1, and (respectively), proportional to their computational running demands. See Appendix C.5.3 for further details.
Results The realistic configuration and practical relevance of these tasks allow us to draw stronger conclu- sions about the usefulness of multi-fidelity active learning with GFlowNets in scientific discovery applications. As in the other tasks evaluated, we here also found MF-GFN to achieve better cost efficiency at finding high- score top-K K molecules (Fig. especially for ionisation potentials (Fig. 3al By clustering the generated molecules, we find that MF-GFN captures as many modes as random generation, far exceoding that of MF- PPO. Indeed, while MF-PPO is able to quickly optimise the target function in the task of electron affinity (Fig, 3bj, all generated molecules were from few clusters (low diversity), which is of much less utility for chemists.
4.4 Ablation studies and additional results
Besides the main experiments presented above, we carried out additional experiments to gain further insights about MF-GFN and study the influence of its various components. We provide detailed results in Appendix and summarise the main conclusions here:
Analysing the results in terms of the top-K diverse samples confirms that the GFlowNeet-asook approaches are able jointly optimize scores and diversity, while RL approaches trade diversity for high scores (Appendix E.1
10

---

Published in Transactions on Machine Learning Research (07 (07/2024)
As is expected, the advantage of MF-GFN over its single-fidelity counterpart decreases as the cost of the lower fidelity oracles increases. Nonetheless, even with cost ratio of 1: in the DNA task, MF-GEN still outperforms all other methods (Appendix E.2
The same conclusions hold for various values of the final candidate set size, K (50, 100, 200) (Appendix E.3
Similar results to the ones presented above were obtained on proof-of-coneept experiments with the synthelic functions Branin and Harlmann, common in the multi-fidelity Bayesian optimisation literature (Appendix C.4
We expect MF-GFN to query cheap oracles to prune the input space and costly oracles for high- reward candidates. We validate this through visualisation using the two-dimensional Branin fune- lion (Appendix E,5
A slightly variation of the baseline Rand. fid. GFN where the fidelities are sampled from a distribu- tion proportional to the cost indicates that learning the fidelity with MF-GFN is still advantageous (Appendix E.6
5 Conclusions, Limitations and Future Work
In this paper, we have presented MF-GFN, a multi-fidelity active learning algorithm that leverages GFlowNets achieve exploration with diversity for scientific discovery applications. MF-GFN samples candidates as well as the fidelity at which the candidate is to be evaluated, when multiple oracles are avail- able with varying fidelities and costs. We evaluated MF-GFN on benchmark tasks of practical relevance, such as DNA aptamer generation, antimicrobial peptide and small molecule design. Through comparisons with previously proposed methods as well as with variants of our method designed to understand the con- tributions of different components, we conclude that multi-fidelity active learning with GFlowNets not only outperforms its single-fidelity active learning counterpart in terms of cost elfectiveness and diversily of sam- pled candidates, but it also offers an advantage over other multi-fidelity methods due to its ability to learn a stochastic policy to jointly sample objects and the fidelity of the oracle.
Limitations and Future Work Aside from the molecular modelling tasks, our empirical evaluations this paper involved simulated oracles with manually selected costs. Future work should evaluate MF-GFN with more practical oracles and costs that reflect their computational or financial demands. As a first approach to multi-fidelity active learning with GFlowNets, the proposed algorithm may be improved and adapted to specific applications in different ways. For example, instead of selecting the top-B candidates at the end of each active learning round, one could adopt stochastic batch acquisition strategies (Kirsch) et al. 2023 ). If the oracle costs reflect run-time, as is the case in many scientific discovery problems, then it would be interesting to consider strategics that would not block the exploration while waiting for the most expensive oracle. Furthermore, promising avenue that do not study in this paper is the application of MF-GFN in more complex, structured design spaces, such as hybrid (discrete and continuous) domains Lahhlou et al. 2023 A14Science et al. 2023). well as both multi-fidelity and multi-objective problems (Jain et al. 2023b).
Finally, given the complexity of multi-fidelity active learning algorithms, consisting of multiple components, we have barely optimised the many hyperparameters and design choices. In fact, we have opted for conser- vative, simple choices in several cases. For example, as oracle confidence we have simply used the inverse of the cost instead of more accurate estimate of the oracle fidelity. Therefore, we expect significant, room for improvement from simple hyperparameter optimisation.
Statement of Broader Impact
Our work is motivated by pressing challenges to sustainability and public health, and we envision applications of our approach to drug discovery and materials discovery. However, as with all work these topics, there
11

---

Published in Transactions on Machine Learning Research (07/2024)
is potential risk of dual use of the technology by nefarious actors (Urbina et al. 2022 The authors strongly oppose any uses or derivations of this work intended to cause harm to humans the environment and explicitly request the careful consideration of potential harms.
Reproducibility Statement
We have made an effort to include the most relevant details of our proposed algorithm in the main body of the paper. For example, a detailed procedure of the steps of the algorithm is presented in Algorithm [ Besides this, we have included additional details about the algorithm in Appendices A and B We have also provided the most relevant information about the experiments in Section | for instance including a description of the data reprosentation and the oracles for cach of the benchmark tasks. The rest of the details about the experiments are provided in Appendix C for the sake of better clarity, transparency and reprodueibility. Finally, the implementation of MF-GFN and the code to reproduce the experiments is publicly available in a GitHub repository:
https: http:///gggtttoooooooooaat/////t
References
RDKit: Open-source cheminformatics. https://www.rrkkt wwwit org. 2023. URL https://zenodo. .rg/
record/8254217
Ankit Agrawal and Alok Choudhary. Perspective: Materials informatics and big data: Realization of the "fourth paradigm" of science in materials science. APL Materials, 4(5):053208, 2016.
Mila AT4Science, Alex Marcin, Alexandre Duval, Alexandra Volokhova, Yoshua Bengio, Divya Sharma, Pierre Lue Carrier, Yasmine Benabed, Michal Koziarski, and Vietor Schmidt. Crystal-GFN: sampling crystals with desirable properties and constraints. arXiv preprint ar Xxxv:2310.04995,, 2023.
Christof Angermueller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Col- well. Model-based reinforcement learning for biological sequence design. In International Conference Learning Represendations (TCLR), 2020.
Dávid Bajusz, Anila Ráez, and Károly Héberger. Why lanimoto index an appropriate choice for fingerprint- based similarity calculations? Journal of Cheminformatics, 7, 05 2015.
Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Fytan Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. Tn Advances in Newral Information Processing Systems (NeurIPS), volume 33, 2020.
Christoph Bannwarth, Sebastian Ehlert, and Stefan Grimme. GFN2-XTB an accurate and broadly parametrized self-consistent tight-binding quantum chemical method with multipole electrostatics and denenty-de dispersion contributions. Journal of Chemical Theory and Computation, 15(3):1652- 1671, 2019.
Ali Bashir, Qin Yang, Jinpeng Wang, Stephan Hoyer, Wenchuan Chou, Cory McLean, Geoff Davis, Qiang Gong, Zan Armstrong, Junghoo Jang, et al. Machine learning guided aptamer refinement and discovery. Nature Communications, 12(1):2366, 2021.
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, 2021a.
Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J. Hu, Mo Tiwari, and Emmanuel Bengio. Gel foundations. arXiv preprint arXiv:2111.090: 2021b.
Regine S Bohacck, Colin McMartin, and Wayne C Guida. The art and practice of structuro-based drug design: molecular modeling perspective. Medicinal Research Revieus, (6(1)33 50, 1996.
12

---

Published in Transactions on Machine Learning Research (07/2024)
Keith T Butler, Daniel W Davies, Hugh Cartwright, Olexandr Isayev, and Aron Walsh. Machine learning for molecular and materials science. Noture, 559(7715):547-65))))) 2018.
Kathryn Chaloner and Isabella Verdinelli. Bayesian experimental design: review. Statistical Science, pp. 273 273-304, 1995.
Konstantinos Chatzilygeroudis, Antoine Cully, Vassilis Vassiliades, and Jean-Baptiste Mouret. Quality- divensity optimization: novel branch of stochastic optimization. In Black Box Optimization, Machine Learning, and No-Free Lunch Theorems, pp. 109-135. Springer, 2021.
Chi Chen, Yunxing Zuo, Weike Yc, Xiangguo Li, and Shyue Ping Ong. Learning propertics of ordered and disordered materials from multi-fidelity data. Nalure Compulational Science, 1(1):46 53, 2021.
Laming Chen, Guexin Zhang, and Hanning Zhou. Fast greedy map inference for determinantal point process to improve recommendation diversity. arXie preprint arXiv: 709.05 05135, 2018.
David Corey, Masad Damha, and Muthiah Manoharan. Challenges and opportunilies for nucleic acid therapeutics. Nucleic Acid Therapeutics, 32(1):8 13, 2022.
Payel Das, Tom Sercu, Kahini Wadhawan, Inkit Padhi, Sebastian Gehrmann, Flaviu Cipcigan, Vijil Chen- thamarakshan, Hendrik Strobelt, Cicero Dos Santos, Pin-Yu Chen, et al. Accelerated antimicrobial dis- covery via deep generative models and molecular dynamics simulations. Nature Biomedical Engineering, 5(6):613-6) 623: 2021.
Aryan Deshwal and Janardhan Rao Doppa. Combining latent space and structured kernels for Bayesian optimization over combinatorial spaces. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, 2021.
Clyde Fare, Peter Fenner, Matthew Benatan, Alessandro Varsi, and Edward O Pyzer-Knapp. A multi-fidelity machine learning approach to high thronghput materials screening, npj computational materials, 8(1):257, 2022.
Francesco Di Fiore, Michela Nardelli, and Laura Mainini. Active learning and Bayesian optimization: unified perspective to learn with goal. orXiv preprint arXin:2303.01560, 2023.
Jose Pablo Folch, Robert M Lee, Behrang Shafei, David Walz, Calvin Tsay, Mark van der Wilk, and Ruth Misener. Combining multi-fidelity modelling and asynchronous batch Bayesian optimization. Computers & Chemical Engincering, 172:108194, 2023.
Adam Evan Foster. Variational, Monte Carlo and pasicy-based approaches to Bayesion experimental design.
PhD thesis, University of Oxford, 2021.
Peter I. Frazicr. A tutorial on Bayesian optimization. arXiv preprint arXiv: 1807. 1807.02811,, 2018.
Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor W. Coley. Sample efficiency matters: A benchmark for practical molecular optimization. arXiv preprinl ar orx::22266.12411 2022.
Jacob R. Gardner, Gcoff Pleiss, David Bindel, Kilian Q. Weinberger, and Andrew Gordon Wilson. GPy- Torch: Blackbox matrix-matrix gaussian process inference with GPU acceleration. arXiv preprint arXiv:1809. 11001,,11,,,.11111111 2021.
Roman Garnett. Bayesian optimization. Cambridge University Press, 2023.
Roman Garnett, Yamuna Krishnamurthy, Xuehan Xiong, Jeff Schneider, and Richard Mann. Bayesian optimal active search and surveying. arXie preprint 1rXio:1206.6.4 2012.
Antoine Grosnit, Rasul Tutunov, Alexandre Max Maraval, Ryan-Rhys Griffiths, Alexander I. Cowen--ivers, Lin Yang, Lin Zhu, Wenlong Lyu, Zhitang Chen, Jun Wang, Jan Peters, and Haitham Bou-Ammar. High- dimensional Bayesian optimisation with variational autoencoders and deep metric learning. orXiv preprint arXiv:2106.0366 2021.
13

---

Published in Transactions on Machine Learning Research (07/2024)
Thomas A Halgren. Merck molecular force field. I. Basis, form, scope, paramcterization, and performance of MMFF94. Journal of Compulational Chemistry, 17(5-6):490-513 1996.
Moksh Jain, Emmanuel Bengio, Alex Hernandez-Garcia, Jarrid Rector-Brooks, Bonaventure FP Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, et al. Biological sequence design with GelowNets. In International Conference on Machine Learning (ICML), volume 162. PMLR, 2022.
Moksh Jain, Tristan Deleu, Jason Hartford, Cheng- Mao Liu, Alex Germandez-Garciaa and Yoshua Bengio. GFlowNets for AI-driven scientific discovery. Digital Discovery, 2023a.
Moksh Jain, Sharath Chandra Raparthy, Alex Garein,,z-GGaria, Jarrid Rector-Brooks, Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective GFlowNets. In International Conference on Machine Learning (ICML), 2023b.
Shali Jiang, Gustavo Malkomes, Geoff Converse, Alyssa Shofner, Benjamin Moseley, and Roman Garnett. Efficient nonmyopic active scarch. In International Conference on Machine Learning (ICML), volume 70. PMLR, 2017.
Shali Jiang, Roman Garnett, and Benjamin Moseley. Cost effective active search. In Advances in Newral Information Processing Systems (NeurIPS), volume 32, 2019.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):5833-56)) 2021.
Kirthevasan Kandasamy, Gaulam Dasarathy, Junier B. Oliva, Jeff Schneider, and Barnabas Poczos. Multi- fidelity gaussian process bandit optimisation. Journal of Artificial Intelligence Research (JAIR), 66:151- 196, 2019.
Michael Kilgour, Tao Liu, Brandon D Walker, Pengyu Ren, and Lena Simine. E2EDNA: Simulation protocol for DNA aptamers with ligands. Journal of Chemical Information and Modeling, 61 (00):411941444 2021.
Ross D King, Kenneth E Whelan, Ffoo M Jones, Philip GK Reiser, Christopher H Bryant, Stephen H Muggleton, Douglas B Kell, and Stephen G Oliver. Functional genomic hypothesis generation and exper- imentation by robot scientist. Nature, 42766991) 2004.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Confer- ence on Learning Representations (ICLR), 2015.
Andreas Kirsch, Sebastian Farquhar, Parmida Atighehchian, Andrew Jesson, Frédéric Branchaud-Charoon and Yarin Gal. Stochastic batch acquisition: A simple baseline for deep active learning. Transactions Machine Learning Research (TMLR), 2023. ISSN 2835-8856.
Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik. Self- referencing embedded strings (SELFIES): A 100% robust molecular string representation. Machine Learn- ing: Science and Technology, 1(4):043024, 2020.
Patrick Kunzmann and Kay Hamacher. Biotite: a unifying open source computational biology framework in Python. BMC Bioinformatics, 19:1-8, 2018.
A Gilad Kusne, Heshan Yu, Changming Wu, Huairuo Zhang, Jason Haltrick-Simpers, Brian DeCost, Suchis- mita Sarker, Corey Oses, Cormac Toher, Stefano Curtarolo, et al. On-the-fly fly closed-loop materials dis- covery via Bayesian active learning. Nature Communications, 11(1):5966, 2020.
Salem Lahlou, Tristan Deleu, Pablo Lemos, Dinghuai Zhang, Alexandra Volokhova, Alex Srnnnndz--Grrche Léna Néhale Ezzine, Yoshua Bengio, and Nikolay Malkin. A theory of continuous generative flow networks. Tn International Conference on Machine Learning (TCMT), 2023.
14

---

Published in Transactions on Machine Learning Research (07/2024)
Shibo Li, Wei Xing, Mike Kirby, and Shandian Zhe. Multi-fidelity Bayesian optimization via deep neural networks. In Advances in Neural Information Processing Systems (NewrIPS), volume 33, 2020.
Shibo Li, Robert M Kirby, and Shandian Zhe. Deep multi-fidelity active learning of high-dimensional outputs. In International Conference on Artificial Intelligence and Statistics (AISTATS), volume 151, pp. (694-1711. PMLR, 2022a.
Shibo Li, Jeff M. Phillips, Xin Yu, Robert M. Kirby, and Shandian Zhe. Batch multi-fidelity active learning with budget constraints. arXiv preprint 1rXi::221012770,, 2022b.
Wesley J. Maddox, Samuel Stanton, and Andrew Gordon Wilson. Conditioning sparse variational gaussian processes for online decision-making. CoRR, abs/2 als/2210.11112, 2021.
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Im- proved crodit assignment GFlowNets. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022.
Natalie Maus, Kaiwen Wu, David Eriksson, and Jacob Gardner. Discovering many diverse solutions with Bayesian optimization. arXiv preprint orXiv:2210.100 009993, 2022.
P. Mikkola, Julien Martinelli, Louis Filstroff, and Samuel Kaski. Multi-fidelity Bayesian optimization with unreliable information sources. In International Conference Artificial Intelligence and Statistics (AIS- TATS), 2023.
Henry B. Moss, David S. Leslie, Javier Gonzalez, and Paul Rayson. GIBBON: General-purpon information- based Bayesian optimisation. Journal of Machine Learning Research (JMLR), 22(235):1-49, 2021.
Hagen Neugebauer, Fabian Bohle, Markus Bursch, Andreas Hansen, and Stefan Grimme. Benchmark study of electrochemical redox potentials calculated with semiempirical and dft methods. The Journal of Physical Chemistry A, 7166377716 2020.
Quan Nguyen, Arghavan Modiri, and Roman Garnett. Nommyopic multifidelity active search. In Interna- tional Conference on Machine Learning (ICML), volume 139. PMLR, 2021.
Leonard Papenmeier, Luigi Nardi, and Matthias Poloczek. Bounce: Reliable high-dimensional Bayesian optimization for combinatorial and mixed spaces. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023.
Robert G Parr. Density functional theory of atoms and molecules. In Horizons of Quantum Chemistry: Proceedings of the Third International Congress of Quantum Chemistry Held at Kyoto, Japan, October November 3, 1979, pp. 5-15. Springer, 1980.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjic Bai, and Soumith Chintala. PyTorch: An imperative style, higl-performance deep learning library. In Advances in Newral Information Processing Systems (NeurIPS), volume 32, 2019.
Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. Survey of multifidelity methods in uncortainty propagation, inference, and optimization. SIAM Review, 60(3):550 591, 2018
John P Perdew and Karla Schmidt. Jacob's ladder of density functional approximations for the exchange- correlation energy. AIP Conference Proceedings, 577 2001.
Paris Perdikaris, M. Raissi, Andreas C. Damianou, ND Lawrence, and George Em Karniadakis. Nonlincar information fusion algorithms for data-cfficient multi-fidelity modelling. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 423(2198), 2017.
15

---

Published in Transactions on Machine Learning Research (07/2024)
Malak Pirtskhalava, Anthony A Amstrong, Maia Grigolava, Mindia Chubinidze, Evgenia Alimbarashvili, Boris Vishnepolsky, Andrei Gabrielian, Alex Rosenthal, Darrell E. Hurt, and Michael Tartakovsky. DBAASP v3: database of wattiirrobba/yyyttooooooo activity and structure of peptides as a resource for development of new therapeutics. Nucleic Acids Research, 49(D1))D288-D22 2021.
Pavel Polishchuk, Timur I Madzhidov, and Alexandre Varnek. Estimation of the size of drug-like chemical space based on GDB-17 data. Journal of Computer-Aided Molecular Design, 27:675-611 679, 2013.
Carl Edward Rasmussen and Christopher K. T. Williams. Goussian Processes for Machine Learning. The MIT Press, 11 2005,
Herbert E. Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathemalical Sociely, 58:527 535, 1952.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimiza- tion algorithms. arXiv preprint arxiv neee1777000344 2017.
Burr Settles. Active learning literature survey. Independent Technical Report, 2009.
David S Sholl and Janice A Steckel. Density functional theory: practical introduction. John Wiley & Sons, 2022.
Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable Bayesian optimization using deep neural networks. In International Conference on Machine Learning (ICML). PMLR, 2015.
András Sobester, Alexander Forrester, and Andy Keane. Engineering Design via Surrogate Modelling. Ap- pendiz: Ezample Problems, pp. 195 203. John Wiley & Sons, Ltd, 2008.
Jialin Song, Yuxin Chen, and Yisong Yue. A general framework for multi-fdeeitty Bayesian optimization with Gaussian processes. In International Conference Artificial Intelligence and Statistics, volume 89. PMLR, 2018.
Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In International Conference on Machine Learning (ICML), 2010.
Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating Bayesian optimization for biological sequence design with denoising autoencoders. In International Conference Machine Learning (ICML), volume 162. PMLR, 2022.
Jonathan M Stokes, Kevin Yang, Kyle Swanson, Wengong Jin, Andres Cubillos-Ruiz, Nina M Donghia, Craig R MacNair, Shawn French, Lindsey A Carfrae, Zohar Bloom-Ackermang et al. A deep learning approach to antibiotic discovery. Cell, (8)(4))888-70) 7028-7702-10 2020.
Shion Takeno, Hitoshi Fukuoka, Yuhki Tsukada, Toshiyuki Koyama, Motoki Shiga, Ichiro Takeuchi, and Masayuki Karasuyama. Multi-fidelity Bayesian optimization with max-value entropy search and its par- allclization. In International Conference on Machine Learning (ICML), volume 119. PMLR, 2020.
Fabio Urbina, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. Dual use of artiifiialiintellignne---ee drug discovery. Nature Machine Intelligence, 4(3):189 (83):189-199 191191919111199991 2022.
Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient Bayesian opt imization.. In International Conference on Machine Learning (ICML), volume 70. PMLR, 2017.
Manfred KK Warmuth, Gunnar Rätsch, Michael Mathicson, Jun Liao, and Christian Lemmen. Active learning the drug discovery process. In Advances in Neural Information Processing Systems (NeurIPS), volume 14, 2001.
16

---

Published in Transactions on Machine Learning Research (07/2024)
Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P Xing. Deep kernel learning. In International Conference on Artificial Intelligence and Stalistics (AISTATS), pp. 370-378, PMT.R, 2016.
Dongxia Wu, Ruijia Niu, Matteo Chinazzi, Yian Ma, and Rose Yu. Disentangled multi-fidelity deep Bayesian active learning. arxiv preprint axX::23355004992, 2023.
Dezhen Xue, Prasanna V Balachandran, John Hogden, James Theiler, Deqing Xue, and Turab Tookman. Accelerated search for materials with targeted properties by adaptive design. Nature Communications, 7 (1):1-9, 2016.
Joseph D Yesselman, Daniel Eiler, Erik D Carlson, Michael R Gotrik, Anne E d' Aquino, Alexandra N Ooms, Wipapal Kladwang, Paul D Carlson, Xuesong Shi, David A Costantino, et al. Computational design of three-dimensional RNA structure and function. Nature Nenotechnology, (4(9):866-5))) 873, 2019.
Ruihao Yuan, Zhen Liu, Prasanna V Balachandran, Deqing Xue, Yumei Zhou, Xiangdong Ding, Jun Sun, Dezhen Xue, and Turab Lookman. Accelerated discovery of large electrostrains in BaTiO3-baas piezo- electrics using active learning. Advanced Materials, 30(7):1702884, 2018.
Joseph N Zadch, Conrad D Steenberg, Justin S Bois, Brian R Wolfe, Marshall B Pierce, Asif R. Khan, Robert M Dirks, and Niles A Pierce. NUPACK: Analysis and design of nucleic acid systems. Journal of Computational Chemistry, 32(1):170-117 2011.
Wenhu Zhou, Runjhun Saran, and Juewen Liu. Metal sensing by DNA. Chemical Reviews, 117(12): 8325, 2017.
C Lawrence Zitnick, Lowik Chanussot, Abhishek Das, Siddharth Goyal, Javier Herass-omingg Caleb Ho, Weihua Hu, Thibaut Lavril, Aini Palizhati, Morgane Riviere, et al. An introduction to electrocatalyst design using machine learning for renewable energy storage. arXiv preprint Xrx::201100094 09435, 2020.
17

---

Published in Transactions on Machine Learning Research (07 (0742
A MF-GFN Algorithm
Our algorithm, MF-GFN, detailed in Algorithm T proceeds as follows: An active learning round j starts with a data set of annotated samples Dj = ((z,, fm, (2t), m)) If no initial data sot available, a set of data points could be sampled randomly. The data set is used to a probabilistic multi-fidelity surrogate model h(z, m) of the posterior p(fm (2)12, m, D). The output of the surrogate model is then used to compute the value of multi-fidclity acquisition function az m). In experiments, fidelity version (Takeno et. al. 2020) of max-value entropy search (MES, Wang & Jegelka 20170, which is cnprmatiinnthhoretti acquisition function widely used in Bayesian optimisation. Additional details about the surrogate and the acquisition function are provided Section and Appendix [
m().
the multi-
The acquisition function used to derive the reward to train a multi-fidelity GFlowNet (see details in Appendix C.3 The GFlowNet is trained to sample tuples (x, m) proportionally to the reward. An active learning round terminates by generating N tuples from the sampler (here the GFlowNet policy x) and forming batch with the best B objects, according to Note that N >> B, since sampling from GFlowNet is relatively inexpensive. The selected objects are annotated by the corresponding oracles and incorporated into the data set, such that Dj+1 = Dj U {(x1, fm, (I1), m.)...., (IB, mB)). This procodure is repeated, progressively acquiring new data, until the budget is exhausted.
B Surrogate Models and Acquisition Function
In this section, we discuss additional details about the surrogate model and acquisition function used in our experiments.
B.1 Gaussian Processes
Following the Bayesian optimisation literature, we assume Gaussian Process (GP) prior on the joint distribution over the function values at different fidelities. The posterior distribution over the oracles (=fm(=)) D) is a also a GP. Concretely, consider set of n points ((11mm)) (I2, mz)...., (I,, m,)) with observed values (Smy (21),... (xn)). Let D = {(11, Jm, (m))),))). )..,,, (I,, Jm. (x,), (m)), A key decision for modelling with GPs is the choice of the kernel. As discussed in Section 3.2 we use the linear truncated multi-fidelity kernel previously used by Mikkola et al. (2023). Assuming a GP prior with mean function μ, kernel KMF (Eq. (2) and an additive Gaussian noise observation model with variance o,, the posterior distribution conditioned on D also GP with the following mean and covariance:
Mn( m) = p(x1mn m1n) + KMF((Z, m), (Iim + (f - p M1 m1.)).
mis), (Xin) minn)
Ka((s; (a), m))) Kar ((ri, mi), (z),
KKMF(M mi), (Zim mim)) ((2mm, ((1m, (Iim) min))
+ - "MF mIn), (z,, m;)).
B.2 Deep Kernel Learning
For synthetic tasks, we exact GP for the surrogate. However, for larger, higher dimensional tasks, having a kernel which captures the structure in the space becomes critical as standard kernels such as RBF and Matérn have difficulty to accurately capture useful notion of similarity. Thus, for the larger benchmark tasks we use deep kernel learning (DKL; Wilson et al., 2016) to alleviate the challenges of defining a kernel. DKL involves using a deep neural network Iw, where w are the parameters, to learn a low-dimensional embedding& z for an input x, gu (x) and applying the kernel on these low-dimensional embeddings.
K Kx(x,, 2ty) + (x Iu (x;)),
18

---

Published in Transactions on Machine Learning Research (07/2024)
where Iw is transformer for our experiments. Additionally, to scale the GP to large datasets, implement the stochastic variational GP based on the greedy inducing point method (Chen et al. 2018 Training surrogate models with DKL can lead to some instabilities, so for our experiments we rely on recommendations from Stanton et al. 2022 to ensure stable training the surrogate model.
B.3 Acquisition Function
In our experiments, we use the max-value entropy search (MES, Wang & Jegelka, 2017 acquisition function. Specifically, we use the multi- fultii-idelity MF-MES variant proposed by Takeno et al. (2020), which for a given candidate (z. m) measures the mutual information between the value of fm( and the maximum value of the highest fidelity oracle fās.
a(x, m) = Iffi: fm (()m))))))))H(()))))))) ( Dz)
The acquisition function can also be interpreted as the expected information gain about f&r obtained by querying (x), where information gain is defined as the reduction in entropy over fMr induced by observing fm (x).
The mutual information can be expensive to compute and is typically approximated using Monte Carlo samples WWag & Jegelka, 2017). For our experiments, we use the GIBBON approximation (Moss et al. 2021). Moss et al. (2021) proposed approximation to the MES acquisition function that is tractable and efficient to compute. Concretely, it takes the following form:
a(x,)
1 1
Σ
ADprOO (fm()), with
Am LFl
2 log 1 11-2200(17 ((F) + *(>(F)) ((((E))
Approx log R
(()(F))
where & and & are the standard normal cunulative distribution and probability density functions (arising P-pa(am)
R is the correlation matrix with clements R. = E,,E,, p= (z), (m))), and F= {F.) F p(f(()))) m)) the predictive mean, on (2,, m. is the predictive standard deviation and Eg is the predictive covariance of the surrogate model for fidelities i, at x.
from the expression for the differential entropy of a truncated Gaussian), >(F) a. (2.m)
Exg
Note that our algorithm is not bound to particular acquisition function (or surrogate model) and therefore these components may be adapted to the specific needs of the application.
C Experimental Details
This section presents the details about the experiments discussed Section 4 First, we provide general details about all tasks and then present details specific each task separate sections.
C.1 Initial data set and budget
We define a budget (Ao) for the initial data set. Let Am the cost of evaluating x with oracle fm, and NSF, NMF be the number of initial training points in the single- and multi-fideli fideiiyiieeity experiments respectively. Also let Mm be the number of training points evaluated against fm in the multi-fidelity experiment such that ad
TMF >m= 7m. Then,
Ao N&F
AM
Σ
Tmm X Am-
The initial data split into train and validation in the ratio of 9:1 for all tasks. specffc information is summarized in Table
19

---

Published in Transactions on Machine Learning Research (07/2024)
For each task, assign total active learning budget A y AM (Table 2 Y selected based on the rate of convergence of the algorithms to the modes. Note that during active learning round, only the oracle evaluations of the sampled batch contribute to A. The cost of sampling from a trained GFlowNet is nearly negligible compared to the oracle evaluations. This is why we can afford to sample a large number of samples (N = 5 x B) to then select the best B, according to the acquisition function (Algorithm
C.2 DKL Implementation Details
Here, we describe our implementation of DKL, which is inspired by Stanton et al. (2022)
Neural Network Architecture For all experiments, the same base architecture was used, featuring transformer encoder layers with position masking for padding tokens. Standard pre-activation residual blocks were included, comprising two convolutional layers, layer normalisation, and swish activations. The encoder embods input sequenees with standard vocabulary and sinusoidal position embeddings. The encoder is trained with the Masked Language Modeling (MLM) objective which is calculated by randomly masking input tokens and subsequently computing the empirical cross- cross-entropy between the original sequence and the predictive distribution generated by the MLM head for the masked positions.
Optimiser Hyperparameters The running estimates of the first two moments in the Adam opti- miser Kingma & Ba 2015) were disabled by setting 0.0 and B2 0.01.
Kernel Hyperparameters In order to force the encoder to learn features appropriate for the initial lengthscale, we place a tight Gaussian prior with a = 0.01 around the intial lengthscale value. The reinitial- isation procodure for inducing point locations and variational parameters outlined by Maddox et al. (2021) was followed.
C.3 Reward Function and Policy Models Details
Neural Network Architecture For all tasks, the architecture of the GFlowNet forward policy (Pr) model is multi-layer perceptron with 2 hidden layers and 2048 units per layer. The backward policy (PB) model was set to share all but the last layer parameters with Pp. We use LeakyReLU as our activation function as in Bengio et al. 2021a). All models are trained with the Adam optimiser (Kingma & Ba 2015)
Reward Function As detailed in Algorithm D the GFlowNet is trained to generate samples with higher value of the MES acquisition function and its multi-fidelity variant in single- and multi-fidelity experiments, respectively. In order to increase the relative reward of higher values of the acquisition function, we scale the MES value ax m) by 1. with 0 < B < 1. On an additional note, MES exhibits increased sparsity as more samples are discovered. Hence, in order to facilitate optimisation, we introduce another scaling factor denoted by p²=1, which depends on the active learning round j. Altogether, our GFlowNet reward function is the following:
R(a(x, m).3.p.)) a(x, m) x pD-
Note that within active learning round, the GFlowNet samples (approxinately) from this fixed reward function and thus the policy need not be conditioned on j. The hyperparameter values for all tasks are detailed in Table Ø
Our models are implemented PyTorch Paszke et al. 2019 and rely on BoTorch Balandat et al. 2020 and Torcorch (Gardner et al. 2021
C.4 Synthetic Tasks
This section describes the experiments and results obtained on the synthetic tasks Branin and Hartmann, included for completeness, to allow a comparison with the traditional Bayesian optimisation literature.
20

---

Published in Transactions on Machine Learning Research (07/2024)
Table
Oracle costs (indexed by increasing level fidelity) and initial data set details.
Task
Oracle Costs
Initial Data Set
Ao
RSF
RMF
A
X2
AM
R1
T02
T2
Branin
0.01
0.1
20
20
Hartmann 6D
0.125
0.25
1
25
25
80
40
DNA
0.2
20
1600
80
3000
50
AMP
0.5
0.5
50
2500
50
2000
2000
10
Molecules
1
3
1050
150
700
68
16
Table Hyperparameters concerning the active learning setting and the policy reward function.
Task
Surrogate Model
Active Active-learning
Policy reward function
B
B
P
Branin Hartmann 6D
Exact GP
300
30
1
1
Exact GP
100
10
1e-2
1
DNA Antimicrobial Peptides
DKL
256
512
le-5
2
DKL
20
32
1e-5
1
Molecules
DKT.
180
128
1e-6
1.5
C.4.1 Branin
We consider an active learning problem in a two-dimensional space (x1, where the target function fM is the Branin function, as modified by (Sobester al. (2008) and implemented in BoTorch (Balandat al., 2020). Tn the standard domain [-5, 10) [0, 15), the Branin function has three modes and is evaluated using the following expression:
1.255x π²
5x
5
f(x) = (x2
6)² + (10
- cos(x1) + 10.
4
This corresponds to the modification introduced by Sobester et al. used the expressions from Perdikaris et al. (2017), which involve non-linean transformations of the true function as well as shifts and non-uniform scalings. The functions, indexed by increasing level of fidelity, are the following:
2008 As lower fidelity functions, we
f(x)= = f21.((x + 3xz + 1,
[e(x)= - 10/ 2(x1 0.5) 03302 - 1) -
This amounts to three levels of fidelity, including the true function. The lower-fidelity oracles, the costs of the oracles (0.01, 0.1, 1.0) as well as the number of points queried the initial training set were adopted from Li al. (2020).
In order to consider a discrete design space, we interpolate the domain into discrete 100 100 grid. We model this grid with GFlowNet in (Bengio et al. 2021a Malkin al. 2022): starting from the origin (01, the action space consists of the choice between the exit action the
(0, 0), for any state dimension to increment by 1, provided the next state is in the limits of the grid.
We use the Torch implementation an exact multi-fidelity Gaussian process as described in Appendix B.1 for regression. The active learning acquisition batch size B is 30 in the Branin task.
Fig. illustrates the results for this task. We observe that MF-GFN is able reach the minimum of the Branin function with smaller budget than the single-fidelity counterpart and the baselines.
21

---

Published in Transactions on Machine Learning Research (07 (024)
3.0
-10
-20
Mean Top-50
-30
2.0
SF-GFN
40
SF-GFN
Rand. fid. GFN
1.5
Rand. Dd. GFN
MF-PPO
-30
MF-GFN
MF-GFN
10-2
10-1
10
10
10-1
10P
Fraction of total SP-GEN budget (log)
Fraction of total SE-GEN budget (log)
(a) Branin task
(b) Hartmann task
Figure 4: Results the synthetic Lasks-Branin and Hartmann functions. The curves indicate the mcan score fa within the top-50 and top-10 samples (for Branin and Hartmann, respectively) computed at the end of each active learning round and plotted as function of the budget used. The random baseline is omitted from this plot to facilitate the visualisation since the results were significantly worse in these tasks. We observe that MF-GFN clearly outperforms the single-fidelity counterpart (SF-GEN) and slightly improves upon the GFlowNet baseline that samples random fidelities. On Hartmann, MF-PPO initially outperforms the other mothods.
C.4.2 Hartmann 6D
We consider the 6-dimensional Hartmann function as objective JM hyper-grid domain. It is typically evaluated on the hyper-cube Ia E [0, 1]6 and consists of six local maxima. The expression of the true Hartmann function is given by
3 Asya P,₂))),
f(z) Σ merpl
1=1
where 7 = 1,2, and A, P E 12x6
are the following fixed matrices:
10
3
17
3.5
1.7
8
0.05
10
17
0.1
8
1
A =
3
3.5
1.7
10
17
8
17
0.05
10
0.1
1
3689
1170
267
4699
4387
7470
P= 10
1091
8732
5547
381
5743
8828
To simulate the lower fidelities, we modify 7 nmm) where nj() = 7 + 8(M m) where 0001, 0.1] and M = 3. The domain is X = [0, 1j6. This implementation was adopted from Kandasamy al. (2019) As with Branin, consider three oracles, adopting the lower-fidelity oracles and the set of costs (0.125, 0.25, 1.0) from Song et al. (2018).
We discretise the domain into a six-dimensional hyper-grid of length 10, yielding 106 possible candidate points. For the surrogate, we use the same exact multi-fidelity GP implementation as of Branin. The active learning acquisition batch size is 10.
The results for the task are illustrated in Fig. which indicate that multi-fidelity active learning with GFlowNets (MF-GFN) offers advantage single-fidelity active learning (SF-GFN) well some of
22

---

Published in Transactions on Machine Learning Research (07/2024)
the other basclines in this higher-dimensionar synthetic problem as well. The better performance on MF- PPO can be attributed to the fact that while the GFN initiates its exploration from the origin point, the PPO commences from a random starting point within a bounded range, allowing at most three units of displacement (maximum possible displacement is 10 units) along each of the six axes. We hypothesise that this aids the PPO algorithm in expediting the discovery of modes within the optimisation process. While MF-PPO performs better in this task, as shown in the benchmark experiments, it tends to collapse to single modes of the function in complex high-dimensional scenarios.
C.5 Benchmark Tasks
C.5.1 DNA
We conduct experiments using a two-oracle setup (JM, fi) with costs Am = 20 and A = 0.2 for the high and low fidelity oracles, respectively. As fM, we use the free energy of the secondary structure of DNA sequences obtained via the software NUPACK (Zadeh et al. 2011), setting the temperature at 310 K. fi is transformer (with 8 encoder layers, 1024 hidden units per layer and 16 heads) trained on 1 million random soquences annotated by fM- To evaluate the performance of fi (with respect to fM), construct test set by sampling sequences from a uniform distribution of the free energy. On this test set, the explained variance of the fi is calculated to be 0.8. For the probabilistic surrogate model, we implement deep kernel learning, the hyperparameters of which are provided in Table ! The active learning acquisition batch size B 512.
C.5.2 Antimicrobial Peptides
We use data from DBAASP (Pirtskhalava et al. 2021). containing antimicrobial activity labels, which is originally split inlo three sets: Dj for training oracle, D2 as the initial data set in the active learning loop and D3 as the test set (Jain et al. 2022).
Table 3: Oracles for the antimicrobial peptides task.
Oracle
Training points
Model
Layers
Hidden units
Training Epochs
f1
3447
MLP
2
512
51
f2
3348
MLP
2
512
51
fM
6795
MLP
2
1024
101
We design a three-oracle setup (fM, f2, fi) where each oracle a different neural network model. The configurations of the oracle models are presented in Table 3 Biologically, each antimicrobial peptide can be classified into an antimicrobial group. fM is trained the entire dataset Dj. However, for f1 and f2, we divide Dj into two approximately equally-sized disjoint subsets. This simulated setup wherein each lower fidelity oracle specialised in different sub-regions of the entire sample space. We set costs Am 50 and A A2- = 0.5 as fi and f2 have similar configurations. The explained variance of fi and f2 (with respect to fM) on a uniform test set, Dg was 0.1435 and 0.099 respectively. For the surrogate, we implement deep kernel learning with the hyperparamcters detailed in Table The active learning acquisition batch size B is 32.
Table 4: Deep kernel hyperparameters for the DNA and antimierobial tasks.
Architecture hyperparameter
Value
Optimisation hyperparameter
Value
Number of layers
8
Batch size Learning rate
128
Number of heads
8
le-3
Latent dimension
64
Adam EMA parameters (81, 82)
(0.0, 1e-2)
GP likelihood variance init.
0.25
Maximum number of epochs
512
GP length scale prior Number of SGVP inducing points
(0.77, 0.01)
Early stopping patience
15
64
Early stopping holdout ratio
0.1
23

---

Published in Transactions on Machine Learning Research (07 (224))
C.5.3 Small Molecules
For the experiments with small molecules, we construct a three oracle setup (SM, f2, fi) with costs rep- resenting the actual compute time. We implement the oracles using RDKit 2023.03 (rdk 2023 and the semi-empirical quantum chemistry package xTB. We use GFN2-xTB (Bannwarth et al., 2019) method for
the single point calculation of ionisation potential (IP) and electron afiintyy (EA) with empirical correction terms.
In fi, consider one conformer obtained by RDKit with its geometry optimised via the force-field MMFF94 [Halgren 1996p. This geometry is used to calculate (vertical) IP IP/EA. In f2, we consider two conformers obtained by RDKit, and take the lowest energy conformer after optimisation by MMFF94, and further optimise via GFN2-xTB to obtain the ground state geometry; this remains a vertical IP/EA calculation. fM, we consider four conformers obtained by RDKit, and take the lowest energy conformer after optimisation by MMFF94, and further optimise it via GFN2-xTB; the corresponding ion is then optimised by GFN2- xTB, and the adiabatic energy difference is obtained via total electronic energy. The fidelities based the fact that vertical IP /EA approximates that of adiabatic ones (to varying degrees, depending the molecule). On uniform test set of 1400 molecules, the explained variance of fi and fa (with respect to fM) is 0.1359, 0.279 and 0.79, 0.86 for the EA and IP tasks respectively.
The surrogate model is a deep kernel with the hyperparameters are provided in Table 5 The active learning acquisition batch size B 128. In the environment for GFN, we consider a set of SELFIES vocabularies containing aliphatic and aromatic carbon, boron, nitrogen, oxygen, fluorine, sulfur, phosphorous, chlorine, and bromine, subject to standard valency rules.
Table 5: Deep Kernel hyperparameters for the molecular tasks
Architecture hyperparameter
Value
Optimisation hyperparamcter
Value
Number of layers
8
Batch size
128
Number of heads
8
Learning rate
1e-3
Latent dimension
32
Adam EMA parameters (31, 32)
(0.0, 1e-2)
GP likelihood variance init.
0.25
Maximum number of epochs
512
GP length scale prior Number of SGVP inducing points
(0.77, 0.01)
Early stopping patience
15
64
Early stopping holdout ratio
0.1
We note that this is proof-of-concept and hence neither do we conduct a full search of conformers, nor do we perform Density Functional Theory caleulations. Nonetheless, we observe that the highest fidelity oracle has a good correlation with experiments (Neugebauer et al.) 2020p. We do not consider syuthesisability in this study and we note it may negatively impact GFN as unphysical molecules could produce false results for the semi-cmpirical oracle.
D Metrics
In this section, we provide additional details about the metrics used for the evaluation of the proposed MF-GFN well as the baselines.
Since we are interested in discovering multiple objects with high scores, instead of only one, in order to compute our evaluation metrics, we consider the set of top-K candidates in the data set D. Formally, let DJT be the data set after active learning round j:
D, = (21 22.............................. p
such that fMI( 2 fM(z)) Then, we define the set of top-K candidates
'Rigorously, the data set consists of triplets (at, fn, (x;), m.)) as defined in Section 3 Here, better readability, we simplify the notation and define the data in terms of the candidates oy only.
24

---

Published in Transactions on Machine Learning Research (07 (024))
top-K(D,) = f2ila; E DI. (
Note that, for evaluation purposes, the top-K set is selected according to the highest-fidclity oracle (x4) and not the oracle selected by the algorithm for the candidate, Jm, (2:))
Mean top-K score We adapt this metrie from Bengio al. (001 which is meant to reflect the ability of the algorithm to discover multiple high-scoring samples. This metric is the mean value of the highest-fidelity oracle across the top-K set:
K Σ (ma)
(ean(top-K(D)))- = K
(4)
Top-K diversity This metric is meant to reflect the ability of the algorithm to discover diverse samples. In order to measure the diversity of the top-K set of candidates for each of the tasks, we use use one minus the similarity index, which is equal to one if all sequences/ saaooncces/mmmulllss molccules in a batch are identical, and zero if all sequences sequences/molcculee /ooleculss are maximally different. Specifically,
diversity(tpp-K(D))) =1
Σ
(k),
(5)
(K)
where (1;) is similarity measure between elements 24,, and Tk.
DNA aptamers: The similarity measure calculated by the mean pairwise sequence identity within set of DNA sequences. The higher the sequence identity, the more similar the sequences algorithm and standard nucleotide
are. We utilize global alignment with the Needleman- substitution matrix, calculated by the Biotite package (Kunzmann & Hamacher 2018 Here, Number of matching mucleotides sxx;, Total number of nucleotides the alignment"
Antimicrobial peptides: The similarity measure is calculated by the mean pairwise sequence
identity within set of peptide sequences. The higher the sequence identity, the more similar the sequences We utilize global alignment with the Needleman- Wunssh algorithm and BLOSUM62 substitution matrix, calculated by the Biotite package (Kunzmann & Hamacherr 2018 Here, Number of matching aminoncids s(x,, Total number of aminoacida the alignment"
The similarity measure calculated by the mean pairwise Tanimoto similarity within a set of molecules. The higher the Tanimoto similarity, the more similar the molecules are. Tanimoto metrics are calculated from Morgan Fingerprints (radius of two, size of 2048 bits) as implemented (F()))P()))
Molecules:
in the RDKit package rrk 2023). Here, s(2,, x4) [F(::)U((z))
Mean diverse top-K score This is an alternative version to the mean top-K score metric by which restrict the selection of the K candidales to dała points that are diverse between each other. We use similarity measures (vide infra) such that we sample the top-K K candidates where each candidate is at most similar to each other by a certain threshold. For antimicrobial poptides, the sequence identity threshold is 0.35; for DNA aptamers, the sequence identity threshold 0.60; for molecules, the Tanimoto similarity distance threshold 0.35. The results using this metric are reported in Appendix B
E Additional Results
This section includes additional results and ablation studies complement the analysis presented in the main body of the paper.
25

---

Published in Transactions on Machine Learning Research (07 2024)
E.1 Energy of Diverse Top-K
In this section, we complement the results presented in Section with the mean diverse top-K scores, as defined in Appendix D This metric combines that mean top-K score and the mcasure of diversity. Figure 3 shows the results on the DNA, AMP and the molecular tasks.
8-
SF-GFN
MF-PPO
Random
MF-GFN
0.8
Mean Top- energy
Rand. fid. GPN
Mean energy
6
4
0.4
0.2
0.0
10a
10-3
10
10-1
10
Fraction of total SF-GFN budget (log)
Fraction of total SF-GFN budget (log)
(a) DNA task
(b) Anti-microbial peptides (AMP) task
5
SP-FNN
MF-PPO
SF-GFN
Moan Top- 100 energy [eV]
Random
MF-GEN
Mean Top- 100 energy [eV]
Random
Rand. fid. GFN
4
Rand. fid, GFN
MF-GFN
3
-7
2
-8
-9
10-1
100
10-1
10a
Fraction of total SF-FN budget (log)
Fraction of total SF-GFN budget (log)
(c) Molecules ionisation potential (IP) task
(d) Molecules electron affinity (EA) task
Figure 5: Mean scores (energy) of diverse top-K candidates on the DNA (top left), AMP (top right) and molecular (bottom) tasks. The mean energy is computed across the top-K candidates at cach active learning round that also satisfy the criteria of diversity. Consistent with the diversity metrics observed in Fig. D here see that GFlowNet-based methods, and especially MF-GFN, obtain good results according to this metric, while MF-PPO achieves comparatively much lower mean energy.
The results with this metric allow to further confirm that multi-fidelity active learning with GFlowNets is able to discover sets of diverse candidates with high mean scores, is sought in many scientific discovery applications. In contrast, methods that do not encourage diversity such as RL-based algorithms (MF-PPO) obtain comparatively much lower results with this metric.
E.2 Impact of Oracle Costs
As discussed in Appendix B.3 multi-fidelity acquisition function like the one we usc-defined in Eq. (3 is cost-adjusted utility function. Consequently, the cost of each oracle plays a crucial role in the utility of acquiring each candidate. In our tasks with small molecules (Section 4.3.3), for instance, we used oracles with costs proportional to their computational demands and observed that multi-fidelity active learning largely outperforms single-fidelity active learning. However, depending on the costs of the oracles, the advantage of multi-fidelity methods significantly diminish.
26

---

Published in Transactions on Machine Learning Research (07/2024)
In order to analyse the impact of the oracle costs the performance of MF-GFN, the DNA task (Section 4.3.1 which consists of two oracles, with additional sets of oracle costs. In particular, besides the costs used in the experiments presented in Section 4.3.1 (0.2, 20) for the lowest and highest fidelity oracles, we run experiments with costs (1, 20) and (10, 20). Additionally, we perform similar experiments on the Harlmann task with sets of oracle costs.
several experiments
The results on the DNA task, presented in Fig, 6aa indeed confirm that the advantage of MF-GFN over SF- GFN decreases as the cost of the lowest-fidelity oracle becomes closer to the cost of the highest-fieelity oracle. However, it is remarkable that even with ratio of costs small as 1 : 2, MF-GFN still outperforms not only SF-GFN but also MF-PPO in terms of cost effectiveness, without diversity being negatively impacted. It is important to note that in practical scenarios of scientific discovery, the cost of lower fidelity oracles is typically orders of magnitude smaller than the cost of the most accurate oracles, since the latter may correspond to wet-lab experiments or expensive computer simulations. The results on the Hartmann task (Fig. 6bb further confirm the above conchusions.
Diversity
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
3.0
16
Moan Top-100 energy
Mean Top 10 score
14
12
10
2.0
ME-GFN orig. (0.125, 0.25, 1.0)
SE-GEN
MF-GFN (0.25, 0.5, 1)
MF-PPO
MF-GFN (0.5, 0.5,
1.5
MF-CPN (0.2, 20)
MF- (0.5, 0.75, 1)
MF-GFN (1, 20)
MF-CFN (0.8, 0.8.
MF-GFN (10,
1.0
SE-FN
0.2
0.4
0.6
0.8
1.0
0.2
0.6
0.8
1.0
Fraction of total SF-GFN budget
Fraction of total SF-GFN budget
(a) DNA
(b) Hartmann
Figure 6: Analysis of the impact the oracle costs on the performance of MF-GFN on the DNA task and the synthetic Hartmann task. On the DNA task, we observe that the advantage over SF-GFN and MF-PPO (0.2, 20) decreases the cost of the lower fidelity oracle becomes closer the cost of the highest fidelity oracle. Nonetheless, even with cost ratio of 1 2 MF-GFN displays remarkable performance with respect to other methods. Similar conclusions can be drawn from the analysis on the Hartmann task.
E.3 Impact of the Acquisition Batch Size
We evaluate the impact of the active learning acquisition batch size on the performance of MF-GFN and its comparison with the baselines for the small molecules IP task with different acquisition batch sizes. From the results in Fig. 7 notice that the reward curve becomes slightly steeper with larger batch sizes.
E.4 Impact of the Choice of the Final Candidate Set Size
For the set of results presented in the main paper, we computed the mean top-K energy and diversity on the final batch of candidates of size K 100. While the choice of K not arbitrary as it is related to the active learning acquisition size and in turn reasonable numbers in the domains of application, it is interesting to study whether our conclusions are robust to other choices of K. Tn Fig. g we provide the equivalent set of results for all the tasks with K = 50 and in Fig. g with K = 200, half and double the size, respectively.
In view of these results, we can conclude that the results are robust to the choice of this parameter, since we can derive the same conclusions for all values of K E (50, 100, 200): MF-GFN obtains the best trade-off between mean energies and diversity, all other GelowNet-baseed methods are able to discover diverse samples and using PPO as the sampler discovers high-scoring samples but strongly lacks diversity.
27

---

Published in Transactions on Machine Learning Research (07/2024)
Diversity 0.045
0.035
0.040
0.050
0.055
Mean Top- 100 energy [ev]
-4
MF-GFN
SF-GFN 64
55
MF-GFN 128 (arig.)
SE-GEN 128 (orig.)
-6
ME-GFN 256
-7
SF-GFN,256
-10
10
10
Fraction of total SF-GFN budget (log)
Figure 7: Impact the acquisition size on the small molecules (IP) task.
Diversity
Diversity
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.3
0.4
0.5
0.6
0.7
0.8
16
Mean Top-50 energy
Moan 50 energy
0.8
12
10
SF-GFN
MF-PPO
6
Random
MF-GFN
4
Rand. fid, GFN
0.5
0.2
044
0.6
10-3
10-2
10-1
Fraction of total SF-GFN budget
Fraction of total SF GFN budget (log)
(a) DNA aptamers lask
(b) AMP Lask
Diversity 0.25
Diversity
0.1
0.2
0.3
0.4
0.6
5.5
Mean Top-50 energy
Tep 50 energy
5.0
4.5
4.0
3.5
-7
SF-GPN
MFPPO
3.0
SF-GFN
MF-PPO
Random Rand. fid. GFN
MFGFN
Random
MF-GFN
Rand. fid. GFN
0.2
0.4
0.6
0.8
1.0
2.0
0.2
0.4
0.6
0.8
1,0
Praction SF.GFN budget
Fraction total SF-GFN budget
(c) Molecules IP task
(d) Molecules EA task
Figure 8: Results as in the original figures, but with K = 50, instead of K 100.
E.5 Visualisation of Sampled Candidates
Given that MF-GFN conducts a cost-aware scarch with the help of the multi-fidelity acquisition function, our expectation is that the algorithm will selectively query the less costly oracles for input space exploration and will query the more expensive oracles high-reward candidates. To substantiate this hypothesis,
28

---

Published in Transactions on Machine Learning Research (07/ (0244
Diversity
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
Diversity
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Mean Top-200 energy
14
Mean Top TTpp220000 200 energy
0.75
10
0.65
0.60
6
SF-GEN
MF-PPO
Random
MF-GFN
Rand. fid. GPN
0.50
2
10-3
10-2
10-1
100
Fraction of total SF-GFN budget
Fraction of total SF-GFN budget (log)
(a) DNA aptamers task
(b) AMP task
Diversity
Diversity
0.2
0.3
0.5
0.7
0.10
0.15
0.20
0.25
0.30
0.35 0.40
0.45
-5
Mean Top--20 energy (eV]
Meon Top-200 energy
-6
Go
7
-8
SP-GFN
MF-PPO
SF-GFN
MF-PPO MF-GFN
-9
Random
MF-CFN
Random
Rand. fid. GEN
Rand, fid, GFN
-10
0.2
0.4
0.6
0.8
1.0
Praction of total SF-GEN budget
Fraction total SF-GFN budget
(c) Molecules IP task
(d) Molecules EA task
Figure 9: Results as in the original figures, but with K = 200, instead of K 100.
provide two-dimensional visualisation (Fig. 10p of the sampled candidates after expending the allocated budget in the synthetic Branin task (Appendix C.4.1
Overall, we observe that the lower-fidelity oracles are queried extensively for exploration relatively large portion of the sample space, while the highest-fidelity oracle tends to be queried near the modes of the objective function.
E.6 GFlowNet with random fidelities proportional to inverse of cost
In Section 4.2 we describe the basclines used as comparisons against MF-GFN. The set of baselines includes slight. but relevant modification of MF-GFN, where the policy select the fidelity of each candidate is not learnt by the GFlowNet but sampled randomly from a uniform distribution (Random fid. GFN). This comparison has allowed sample tuples of candidate and fidelity, obtains better results.
conclude that the Gel variant that have introduced, which learns to
A reasonable alternative to sampling the fidelity from a uniform distribution is to sample the fidelity from distribution proportional to the inverse of the cost, that is p(m]ar) & - In order to verily whether this is a significantly stronger baseline than Random fid. GFN, the experiments in Fig. [ show a comparison on the Hartmann task. As a main conclusion, the two basclines with random fidelities perform very similarly and not better than MF-GFN. A limitation of this analysis that the simplicity of the Hartmann task does not allow MF-GFN to obtain large ad advantage with respect. to these baselines.
29

---

Published in Transactions on Machine Learning Research (07 (07/2022 2024)
15
x
Cost 0.01
Gash
Thairing Point
-2
-1
10
XAdss
Figure 10: We present a visualisation of the sampled candidates (zc m) in the synthetic Branin task (Ap- pendix C.4.1p. The domain of Branin is defined in [-5, 10] x [0, 15). Each round marker, identified by grid-specific coordinates, represents sampled candidate, The markers are colour-coded based the oracle the candidate is to be evaluated with, Our observation reveals that the lower fidelity oracles (with costs of 0.01 and 0.1) are primarily used for exploration across the input domain, while evaluations using the high-fidelity oracle (cost=1) are predominantly concentrated near the modes (denoted by the star marker). Note that the training points were intentionally chosen to exclude the modes.
A closer analysis of the results indicates that sampling the fidelities proportionally A strongly biased towards selecting the lowest fidelity. This may result in the stagnation of the improvement as the active learning algorithı progresses, since most often the lowest. fidelity is selected, preventing the algorithm to get closer to the modes of the objective function. Given the positive results of MF-GFN in the rest of the experiments and its nearly negligible overhead with respect to these basclines, we conclude that learning the fidelity rather than selecting it at random is a favourable choice.
3.20
Mean Top-10 score
3.15
3.10
3.05
3.00
MF-GPN
MF-CPN Cost-based fidelities
2.95
ME-GEN Uniform fidelities
0.2
0.4
0.6
0.8
1.0
Fraction of total SF-GFN budget
Figure 11: Evaluation of the performance of the MF-GEN baseline which samples the fidelity from distribution proportional to the inverse of the oracle cost - These experiments are on the Hartmann task. We observe that this alternative baseline performs similarly to the one included in the main results section.
1
30


Reviews:
[{"review_id": "nhueGQgQ2e", "reviewer": "Reviewer_as9b", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- The paper is well-written.\n- The presented method is evaluated on multiple challenging and somewhat realistic problems.\n- The presented method achieves very good results in the benchmarks. (I am avoiding the term \"state-of-the-art\" here because the benchmarks do not appear to be well-established.)\n- Multiple ablations are included.\n- Code is provided.\n\n**Weaknesses:**\n- I am not fully convinced about the way that the selection of the fidelity is handled, because while the acquisition function incorporates the cost of the fidelities, using a fixed fidelity kernel might not allow the model to learn the actual utility of the different fidelity levels. I would be interested in the following experiments: (a) Random fidelity GFN where the fidelity distribution is not uniform but proportional to $1/\\lambda_m$. This way, the same cost is spent on all fidelities in expectation. (b) MF-GFN vs. SF-GFN in a two-fidelity setting where the cheaper fidelity is not useful at all (e.g. constant or pure noise). Does MF-GFN learn to query this fidelity less over time? (If it turns out that MF-GFN cannot learn the utility of a fidelity, it might be possible to fix this by including $m$ as an input to the surrogate model instead of including it through a separate kernel, but I expect that this might require some non-trivial and problem-dependent architectural changes. Therefore, I do not expect the authors to actually do this.)\n- Algorithm 1 selects an unspecified number $N \\gg B$ of tuples and then picks the top $B$ out of them. As in pool-based active learning, making $N$ too large could lead to the top $B$ samples being near-identical. I did not find a discussion on how to choose $N$ or whether this is a sensitive hyperparameter in practice. It could be interesting to look at diversification strategies, such as a simple stochastic acquisition (https://arxiv.org/abs/2106.12059), although this is already what the policy $\\pi_\\theta$ should have learned to do. I leave the decision whether to do additional experiments on this to the authors, as I do not know how expensive they would be.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of black-box optimization in a setting where finding a diverse set of potential solutions is important. Unlike prior work, it considers the multi-fidelity setting, where multiple approximations of the target function at different costs are available for querying. The paper presents an approach to this problem through the use of GFlowNets, deep kernel learning with multi-fidelity kernels, multi-fidelity acquisition functions, and an iterative algorithm on top of these. The approach is evaluated on four biochemical tasks as well as two standard synthetic optimization problems, showing strong results with small budgets and high diversity of the obtained batch.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Overall, I like the paper, and I think it should be accepted with some minor revisions. The points mentioned in the weaknesses should, in my opinion, at least be discussed in the paper. Experiments would be better, especially for the first point, but it depends on the effort needed to include them.\n\n**Questions:**\n- Why do many curves stop before reaching the maximum budget? Is the computation too slow? Why do some have error bars and some don't? What do the error bars mean? (This should be discussed and/or improved in the paper.)\n- Section 3.2, last paragraph: \"MES [...] makes random sampling a strong baseline\": What is meant by that? How is random sampling used there? Like in successive halving? In particular, if an acquisition function can make random sampling better, it must mean that random sampling uses the acquisition function somehow? (Please provide a bit more detail in the paper.)\n\n**Minor Comments:** (should be easy to resolve, but they are not critical for acceptance)\n- Section 3.2, paragraph \"Surrogate Model\", last sentence: Can the authors briefly explain what is a \"linear downsampling kernel over the fidelity index\" (or refer to the Appendix)? (It could be mentioned that the used kernel is a product kernel of a deep kernel for the $x$ part and a fixed kernel for the $m$ part.)\n\n- Algorithm 1: In the second line, shouldn't it be $f_{m_i}$ instead of $f_m$?\n- Algorithm 1: What is $\\beta$?\n- Algorithm 1: At \"Result\": How is \"Diversity\" computed? (Maybe it should not be returned but computed outside of the algorithm to avoid confusion.) How is Top-K(D) computed? Based on the surrogate for the highest fidelity? Maybe the \"Result\" line should be moved to the end of the algorithm as a \"return\" statement?\n\n- Section B.1: In the first align/eqnarray, you used $x$ but the text above suggests that you wanted to use $z$ for these equations?\n- Section B.1: Equation for $K_n(x_1, x_2)$ has an extra ( and $x1:n$ instead of $x_{1:n}$.\n\n- Table 4: Should be $(\\beta_1, \\beta_2)$ instead of $(\\beta 1, \\beta 2)$.", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hwHIjWV9uy", "reviewer": "Reviewer_JQdA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is generally clear and well written. I usually find myself advocating for moving detail to the appendix when reviewing (particularly for TMLR where there is no strict page limit), but in this case I actually think too little detail is given in the main text so one is always having to go to the appendix. In particular I think the downsampling kernel and key equations for MES should be in the main text since these are so fundamental to the approach. I would also like more intuition explanation for both of these. \"downsampling kernel\" does not appear as a term in the cited Wu et al 2019 paper. Is this a choie of the authors? Why is this functional form chosen? For MES it is unclear where the IGApprox expression comes from. I think m is being reused to mean something other than the fidelity level which adds to the confusion. Fig 10 is also interesting enough that I would consider moving it to the main paper. \n\nThe work appears technically sound. It's a bit unclear to me how the method biases towards cheaper oracles - is this through the downsampling kernel or built into alpha (eqn 2 in the appendix). In either case, what is the motivation for these functional forms? \n\nOther minor comments/questions: \n- p3 there is a space after \"drug discovery\" \n- for eqn 1 (GFlowNet objective), is this summed over all trajectories x? \n- for algo 1 is N not also an input? \n- for the summation at the end of the algo the superscript should not include i=\n- is the downsampling kernel assuming similar m => similar accuracy? is this a good assumption? \n- how is m scaled for the downsampling kernel? presumably in [0,1] but this isn't specified\n- for appendix B.2 shouldn't the LHS be alpha(x) ? \n- next eqn D should be mathcal\n- next eqn: sample from f* means sample f then take argmax?\n- formula for correlation is missing sqrts in the denominator\n- C.5.1 \"explained variance\" = PERCENT variance? \n- fig 10: why are no Cost=1 points near the (3,2) mode?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors consider applying GFlowNets in the context of multi-fidelity active learning. This allows amortized, diverse sample generation in structured spaces. Nice results are presented on \"toy\" and real scientific applications including DNA aptamer design.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Clarifications and improved presentations. I think the results/technical contribution is sufficient for TMLR already.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1Hy91qDuSx", "reviewer": "Reviewer_teMv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper extends the GFlowNets to the multi-fidelity setting so that the network is able to utilize oracles of various costs/accuracies to get similar results with greatly reduced overall computational costs.\n\n2. The paper applies the proposed method to a variety of real-world scientific discovery tasks and achieves both good performance and diversity.\n\nWeaknesses:\n1. The paper does not have significant technical novelty as the major contribution is to extend the existing GFlowNets to the multi-fidelity scenario.\n2. Why GFlowNets can help generate diverse samples is not presented very clearly in the paper, especially after we introduce the multi-fidelity factor.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the problem of active learning with multi-fidelity oracles. Particularly, the paper proposes to utilize the GFlowNet trained on the samples and the acquisition functions to learn the policy to generate new samples. The paper shows that on both synthetic tasks and real-world tasks for scientific discoveries, the proposed method is able to generate samples with high scores and diversities.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The diversity metric is a bit vague. I suggest adding detailed equations for such metrics. For example, in the DNA task, it is mentioned that \"Diversity is computed as one minus the mean pairwise sequence identity among a set of DNA sequences.\" However, if I understand it correctly, such a metric is actually the negative of diversity.\n\n2. The notation for augmenting the state space of GFlowNets with the multi-fidelity is also confusing. It seems to me that if a new dimension of multi-fidelity is added, the space should be S cross M instead of the union.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I am pleased to inform you that the recommendation is for your paper to be accepted in TMLR. The work presents a valuable extension of GFlowNets to handle multi-fidelity optimization with promising results across several application domains. The reviewers appreciated the thorough experiments demonstrating the utility of your method. The additional edits during the discussion period have improved the overall presentation and readability --- I want to thank the authors and reviewer as9b, in particular, for making good use of the discussion period and review process.
Decision: Accept (poster)