Original: G ENERATIVE P RETRAINING FOR B LACK -B OX O PTIMIZATION
Revised: G ENERATIVE P RETRAINING FOR B LACK -B OX O PTIMIZATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Many problems in science and engineering involve optimizing an expensive blackbox function over a high-dimensional space.
Revised: Many problems in science and engineering involve optimizing an expensive blackbox function over a high-dimensional space.

Original: For such black-box optimization (BBO) problems, we typically assume a small budget for online function evaluations, but also often have access to a fixed, offline dataset for pretraining.
Revised: For such black-box optimization (BBO) problems, we typically assume a small budget for online function evaluations, but also often have access to a fixed, offline dataset for pretraining.

Original: Prior approaches seek to utilize the offline data to approximate the function or its inverse but are not sufficiently accurate far from the data distribution.
Revised: Prior approaches seek to utilize the offline data to approximate the function or its inverse but are not sufficiently accurate far from the data distribution.

Original: We propose
Revised: We propose

Original: BONET, a generative framework for pretraining a novel black-box optimizer using offline datasets.
Revised: BONET, a generative framework for pretraining a novel black-box optimizer using offline datasets.

Original: In BONET, we train an autoregressive model on fixed-length trajectories derived from an offline dataset.
Revised: In BONET, we train an autoregressive model on fixed-length trajectories derived from an offline dataset.

Original: We design a sampling strategy to synthesize trajectories from offline data using a simple heuristic of rolling out monotonic transitions from low-fidelity to high-fidelity samples.
Revised: We design a sampling strategy to synthesize trajectories from offline data using a simple heuristic of rolling out monotonic transitions from low-fidelity to high-fidelity samples.

Original: Empirically, we instantiate BONET using a causally masked Transformer (Radford et al., 2019) and evaluate it on Design-Bench (Trabucco et al., 2022), where we rank the best on average, outperforming state-of-the-art baselines.
Revised: Empirically, we instantiate BONET using a causally masked Transformer (Radford et al., 2019) and evaluate it on Design-Bench (Trabucco et al., 2022), where we rank the best on average, outperforming state-of-the-art baselines.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Many fundamental problems in science and engineering, ranging from the discovery of drugs and materials to the design and manufacturing of hardware technology, require optimizing an expensive black-box function in a large search space (Larson et al., 2019; Shahriari et al., 2016).
Revised: Many fundamental problems in science and engineering, ranging from the discovery of drugs and materials to the design and manufacturing of hardware technology, require optimizing an expensive black-box function in a large search space (Larson et al., 2019; Shahriari et al., 2016).

Original: The key challenge here is that evaluating and optimizing such a black-box function is typically expensive, as it often requires real-world experimentation and exploration of a high-dimensional search space.
Revised: The key challenge here is that evaluating and optimizing such a black-box function is typically expensive, as it often requires real-world experimentation and exploration of a high-dimensional search space.

Original: Fortunately, for many such black-box optimization (BBO) problems, we often have access to an offline dataset of function evaluations.
Revised: Fortunately, for many such black-box optimization (BBO) problems, we often have access to an offline dataset of function evaluations.

Original: Such an offline dataset can greatly reduce the budget for online function evaluation.
Revised: Such an offline dataset can greatly reduce the budget for online function evaluation.

Original: This introduces us to the setting of offline BBO.
Revised: This introduces us to the setting of offline BBO.

Original: A key difference exists between the offline BBO setting and its online counterpart; in offline BBO, we are not allowed to actively query the black-box function during optimization, unlike in online BBO where most approaches (Snoek et al., 2012;
Revised: A key difference exists between the offline BBO setting and its online counterpart; in offline BBO, we are not allowed to actively query the black-box function during optimization, unlike in online BBO where most approaches (Snoek et al., 2012;

Original: Shahriari et al., 2016) utilize iterative online solving.
Revised: Shahriari et al., 2016) utilize iterative online solving.

Original: One natural approach for offline BBO would be to train a surrogate (forward) model that approximates the blackbox function using the offline data.
Revised: One natural approach for offline BBO would be to train a surrogate (forward) model that approximates the blackbox function using the offline data.

Original: Once learned, we can perform gradient ascent on the input space to find the optimal point.
Revised: Once learned, we can perform gradient ascent on the input space to find the optimal point.

Original: Unfortunately, this method does not perform well in practice because the forward model can incorrectly give sub-optimal and out-of-domain points a high score (see Figure 1a).
Revised: Unfortunately, this method does not perform well in practice because the forward model can incorrectly give sub-optimal and out-of-domain points a high score (see Figure 1a).

Original: To mitigate this issue, COMs (Trabucco et al., 2021) learns a forward mapping that penalizes high scores on points outside the dataset, but this can have the opposite effect of not being able to explore high fidelity points that are far from the dataset.
Revised: To mitigate this issue, COMs (Trabucco et al., 2021) learns a forward mapping that penalizes high scores on points outside the dataset, but this can have the opposite effect of not being able to explore high fidelity points that are far from the dataset.

Original: Further, another class of recent approaches (Kumar & Levine, 2020; Brookes et al., 2019; Fannjiang & Listgarten, 2020) propose a conditional generative approach that learns an inverse mapping function values to the points.
Revised: Further, another class of recent approaches (Kumar & Levine, 2020; Brookes et al., 2019; Fannjiang & Listgarten, 2020) propose a conditional generative approach that learns an inverse mapping function values to the points.

Original: For effective generalization, such a mapping needs to be highly multimodal for high-dimensional functions, which in itself presents a challenge for current approaches.
Revised: For effective generalization, such a mapping needs to be highly multimodal for high-dimensional functions, which in itself presents a challenge for current approaches.

Original: We propose B lack-box O ptimization Net works (BONET), a new generative framework for pretraining black-box optimizers on offline datasets.
Revised: We propose B lack-box O ptimization Net works (BONET), a new generative framework for pretraining black-box optimizers on offline datasets.

Original: Instead of approximating the surrogate function (or its inverse), we seek to approximate the dynamics of online black-box optimizers using an autoregressive sequence model.
Revised: Instead of approximating the surrogate function (or its inverse), we seek to approximate the dynamics of online black-box optimizers using an autoregressive sequence model.

Original: Naively, this would require access to several trajectory runs of different blackbox optimizers, which is expensive or even impossible in many cases.
Revised: Naively, this would require access to several trajectory runs of different blackbox optimizers, which is expensive or even impossible in many cases.

Original: Our key observation is that we can synthesize synthetic trajectories comprised of offline points that mimic empirical characteristics
Revised: Our key observation is that we can synthesize synthetic trajectories comprised of offline points that mimic empirical characteristics

Original: [Figure]
Revised: [Figure]

Original: of online BBO algorithms, such as BayesOpt
Revised: of online BBO algorithms, such as BayesOpt

Original: While one could design many characteristic properties, we build off an empirical observation related to the function values of the proposed points.
Revised: While one could design many characteristic properties, we build off an empirical observation related to the function values of the proposed points.

Original: In particular, averaged over multiple runs, online black-box optimizers (e.g., BayesOpt) tend to show improvements in the function values of the proposed points (Bijl et al., 2016), as shown in Figure 1c.
Revised: In particular, averaged over multiple runs, online black-box optimizers (e.g., BayesOpt) tend to show improvements in the function values of the proposed points (Bijl et al., 2016), as shown in Figure 1c.

Original: While not exact, we build on this observation to develop a sorting heuristic that constructs synthetic trajectories consisting of offline points ordered monotonically based on their ascending function values.
Revised: While not exact, we build on this observation to develop a sorting heuristic that constructs synthetic trajectories consisting of offline points ordered monotonically based on their ascending function values.

Original: Even though such a heuristic does not apply uniformly for the trajectory runs of all combinations of black-box optimizers and functions, we show that it is simple, scalable, and quite effective in practice.
Revised: Even though such a heuristic does not apply uniformly for the trajectory runs of all combinations of black-box optimizers and functions, we show that it is simple, scalable, and quite effective in practice.

Original: Further, we augment every offline point in our trajectories with a regret budget , defined as the cumulative regret of the trajectory starting at the current point until the end of the trajectory.
Revised: Further, we augment every offline point in our trajectories with a regret budget , defined as the cumulative regret of the trajectory starting at the current point until the end of the trajectory.

Original: We train
Revised: We train

Original: BONET to generate trajectories conditioned on the regret budget of the first point of the trajectory.
Revised: BONET to generate trajectories conditioned on the regret budget of the first point of the trajectory.

Original: Thus, at test time, we can generate good candidate points by rolling out a trajectory with a low regret budget.
Revised: Thus, at test time, we can generate good candidate points by rolling out a trajectory with a low regret budget.

Original: Figure 1b shows an illustration.
Revised: Figure 1b shows an illustration.

Original: We evaluate our method on several real-world tasks in the Design-Bench (Trabucco et al., 2022) dataset.
Revised: We evaluate our method on several real-world tasks in the Design-Bench (Trabucco et al., 2022) dataset.

Original: These tasks are based on real-world problems such as robot morphology optimization,
Revised: These tasks are based on real-world problems such as robot morphology optimization,

Original: DNA sequence optimization, and optimizing superconducting temperature of materials, all of which requires searching over a high-dimensional search space.
Revised: DNA sequence optimization, and optimizing superconducting temperature of materials, all of which requires searching over a high-dimensional search space.

Original: We achieve a normalized mean score of 0 .
Revised: We achieve a normalized mean score of 0 .

Original: 772 and an average rank of 2 . 4 across all tasks, outperforming the next best baseline, which achieves a rank of 3 . 7 .
Revised: 772 and an average rank of 2 . 4 across all tasks, outperforming the next best baseline, which achieves a rank of 3 . 7 .

Original:  P RETRAINING B LACK -B OX O PTIMIZERS VIA BONET 2. P ROBLEM S TATEMENT
Revised:  P RETRAINING B LACK -B OX O PTIMIZERS VIA BONET 2. P ROBLEM S TATEMENT

Original: Let f : X → R be a black-box function, where X ⊆ R d is an arbitrary d -dimensional domain.
Revised: Let f : X → R be a black-box function, where X ⊆ R d is an arbitrary d -dimensional domain.

Original: In black-box optimization (BBO), we are interested in finding the point x ∗ that maximizes f :
Revised: In black-box optimization (BBO), we are interested in finding the point x ∗ that maximizes f :

Original: [Equation]
Revised: [Equation]

Original: Typically, f is expensive to evaluate and we do not assume direct access to it during training.
Revised: Typically, f is expensive to evaluate and we do not assume direct access to it during training.

Original: Instead, we have access to an offline dataset of N previous function evaluations D
Revised: Instead, we have access to an offline dataset of N previous function evaluations D

Original: = { ( x 1 , y 1 ) , · · · , ( x N , y N ) } ,
Revised: = { ( x 1 , y 1 ) , · · · , ( x N , y N ) } ,

Original: where y i =
Revised: where y i =

Original: f
Revised: f

Original: ( x i ) .
Revised: ( x i ) .

Original: For evaluating a black-box optimizer post-training, we allow it to query the black-box function f for a small budget of Q queries and output the point with the best function value obtained.
Revised: For evaluating a black-box optimizer post-training, we allow it to query the black-box function f for a small budget of Q queries and output the point with the best function value obtained.

Original: This protocol follows prior works in offline BBO
Revised: This protocol follows prior works in offline BBO

Original: (Trabucco et al., 2021; 2022; Kumar & Levine, 2020; Brookes et al., 2019; Fannjiang & Listgarten, 2020).
Revised: (Trabucco et al., 2021; 2022; Kumar & Levine, 2020; Brookes et al., 2019; Fannjiang & Listgarten, 2020).

Original: Overview of BONET We illustrate our proposed framework for offline BBO in Figure 2 and Algorithm 1 .
Revised: Overview of BONET We illustrate our proposed framework for offline BBO in Figure 2 and Algorithm 1 .

Original: BONET consists of 3 sequential phases: trajectory construction, autoregressive modelling,
Revised: BONET consists of 3 sequential phases: trajectory construction, autoregressive modelling,

Original: [Figure]
Revised: [Figure]

Original: Rx Rx
Revised: Rx Rx

Original: R T x T
Revised: R T x T

Original: ˆ xˆ xˆ x T
Revised: ˆ xˆ xˆ x T

Original: Autoregressive Transformer
Revised: Autoregressive Transformer

Original: Training( Phase 2 ) budget point
Revised: Training( Phase 2 ) budget point

Original: Rx
Revised: Rx

Original: R P x P ˆ R ˆ x P + ˆ R ˆ x T
Revised: R P x P ˆ R ˆ x P + ˆ R ˆ x T

Original: [Figure]
Revised: [Figure]

Original: predictions
Revised: predictions

Original: [Figure]
Revised: [Figure]

Original: 2. P HASE 1: C ONSTRUCTING T RAJECTORIES
Revised: 2. P HASE 1: C ONSTRUCTING T RAJECTORIES

Original: Our key motivation in BONET is to train a model to mimic the sequential behavior of online blackbox optimizers.
Revised: Our key motivation in BONET is to train a model to mimic the sequential behavior of online blackbox optimizers.

Original: However, the difficulty is that we do not have the ability to generate trajectories by actively querying the black-box function during training.
Revised: However, the difficulty is that we do not have the ability to generate trajectories by actively querying the black-box function during training.

Original: In BONET, we overcome this difficulty by synthesizing trajectories purely from an offline dataset based on two guiding desiderata.
Revised: In BONET, we overcome this difficulty by synthesizing trajectories purely from an offline dataset based on two guiding desiderata.

Original: Input Offline dataset D , Evaluation Regret Budget ˆ R , Prefix length P, Query budget Q , Trajectory length T , num trajs , Smoothing parameter K , Temperature τ , Number of bins N B
Revised: Input Offline dataset D , Evaluation Regret Budget ˆ R , Prefix length P, Query budget Q , Trajectory length T , num trajs , Smoothing parameter K , Temperature τ , Number of bins N B

Original: Output A set of proposed candidate points constraint | X | ≤ Q
Revised: Output A set of proposed candidate points constraint | X | ≤ Q

Original: First, the procedure for synthesizing trajectories should efficiently scale to high-dimensional data points and large offline datasets.
Revised: First, the procedure for synthesizing trajectories should efficiently scale to high-dimensional data points and large offline datasets.

Original: Second, each trajectory should mimic characteristic behaviors commonly seen in online black-box optimizers.
Revised: Second, each trajectory should mimic characteristic behaviors commonly seen in online black-box optimizers.

Original: We identify one such characteristic of interest.
Revised: We identify one such characteristic of interest.

Original: In particular, we note that the moving average of function values of points proposed by such black-box optimizers tends to improve over the course of their runs barring local perturbations (e.g., due to exploration).
Revised: In particular, we note that the moving average of function values of points proposed by such black-box optimizers tends to improve over the course of their runs barring local perturbations (e.g., due to exploration).

Original: While exceptions can and do exist, this phenomena is commonly observed in practice for optimizers such as BayesOpt (Bijl et al., 2016).
Revised: While exceptions can and do exist, this phenomena is commonly observed in practice for optimizers such as BayesOpt (Bijl et al., 2016).

Original: We also illustrate this behavior in Figure 1c for some commonly used test functions optimized via BayesOpt.
Revised: We also illustrate this behavior in Figure 1c for some commonly used test functions optimized via BayesOpt.

Original: Sorted Trajectories
Revised: Sorted Trajectories

Original: We propose to satisfy the above desiderata in BONET by following a sorting heuristic.
Revised: We propose to satisfy the above desiderata in BONET by following a sorting heuristic.

Original: Specifically, given a set of T offline points, we construct a trajectory of length T by simply sorting the points in ascending order from low to high function values.
Revised: Specifically, given a set of T offline points, we construct a trajectory of length T by simply sorting the points in ascending order from low to high function values.

Original: We note that sorting is just a heuristic we use for constructing synthetic trajectories from the offline dataset, and this behavior may not be followed by any general optimizer over any arbitrary functions.
Revised: We note that sorting is just a heuristic we use for constructing synthetic trajectories from the offline dataset, and this behavior may not be followed by any general optimizer over any arbitrary functions.

Original: We also perform ablations on different heuristics in Appendix C.1.
Revised: We also perform ablations on different heuristics in Appendix C.1.

Original: Further, we note that sorting does not provide any guidance on the rate or the relative spacing between the points i.e., how fast the function values increase.
Revised: Further, we note that sorting does not provide any guidance on the rate or the relative spacing between the points i.e., how fast the function values increase.

Original: This rate is important for controlling the sample budget for black-box optimization.
Revised: This rate is important for controlling the sample budget for black-box optimization.

Original: Next, we discuss a sampling strategy for explicitly controlling this rate.
Revised: Next, we discuss a sampling strategy for explicitly controlling this rate.

Original: [Figure]
Revised: [Figure]

Original: Sampling Strategies for Offline
Revised: Sampling Strategies for Offline

Original: Points
Revised: Points

Original: So far, we have proposed a simple heuristic for transitioning a set of offline points into a sorted trajectory.
Revised: So far, we have proposed a simple heuristic for transitioning a set of offline points into a sorted trajectory.

Original: To obtain these offline trajectory points from the offline dataset, one default strategy is to sample uniformly at random T points from D and sort them.
Revised: To obtain these offline trajectory points from the offline dataset, one default strategy is to sample uniformly at random T points from D and sort them.

Original: However, we found this strategy to not work well in practice.
Revised: However, we found this strategy to not work well in practice.

Original: Intuitively, we might expect a large volume of the search space to consist of points with low-function values.
Revised: Intuitively, we might expect a large volume of the search space to consist of points with low-function values.

Original: Thus, if our offline dataset is uniformly distributed across the domain, the probability of getting a high quality point will be very low with a uniform sampling strategy.
Revised: Thus, if our offline dataset is uniformly distributed across the domain, the probability of getting a high quality point will be very low with a uniform sampling strategy.

Original: To counter this challenge, we propose a 2 step sampling strategy based on binning followed by importance reweighting.
Revised: To counter this challenge, we propose a 2 step sampling strategy based on binning followed by importance reweighting.

Original: Our formulation is motivated by a similar strategy proposed by Kumar & Levine (2020) for loss reweighting.
Revised: Our formulation is motivated by a similar strategy proposed by Kumar & Levine (2020) for loss reweighting.

Original: First, we use the function values to partition the offline dataset D into N B bins of equal-width, i.e., each bin covers a range of equal length.
Revised: First, we use the function values to partition the offline dataset D into N B bins of equal-width, i.e., each bin covers a range of equal length.

Original: Next, for each bin, we assign a different sampling probability, such that (a) bins where the average function value is high are more likely to be sampled, and (b) bins with more points are sampled more often.
Revised: Next, for each bin, we assign a different sampling probability, such that (a) bins where the average function value is high are more likely to be sampled, and (b) bins with more points are sampled more often.

Original: The former helps minimize the budget, whereas the latter ensures diversity in trajectories.
Revised: The former helps minimize the budget, whereas the latter ensures diversity in trajectories.

Original: Based on these two criteria, the score s i for a bin B i is given as:
Revised: Based on these two criteria, the score s i for a bin B i is given as:

Original: [Equation]
Revised: [Equation]

Original: where ˆ y is the best function value in the dataset D ,
Revised: where ˆ y is the best function value in the dataset D ,

Original: | B i | refers to the number of points in the i th bin, and y b i is the midpoint of the interval corresponding to the bin B i .
Revised: | B i | refers to the number of points in the i th bin, and y b i is the midpoint of the interval corresponding to the bin B i .

Original: Here, the first term | B i | | B i | + K allows us to assign a higher weight to the larger bins with smoothing.
Revised: Here, the first term | B i | | B i | + K allows us to assign a higher weight to the larger bins with smoothing.

Original: The second term gives higher weight to the good bins using an exponential weighting scheme.
Revised: The second term gives higher weight to the good bins using an exponential weighting scheme.

Original: More details about K and τ can be found in Appendix B. Finally, we use these scores s i to proportionally sample n i points from bin B i where n i = T (cid:106) s i (cid:80)
Revised: More details about K and τ can be found in Appendix B. Finally, we use these scores s i to proportionally sample n i points from bin B i where n i = T (cid:106) s i (cid:80)

Original: j s j (cid:107) for i ∈ { 2 , · · · , N B } and n 1 = T − (cid:80) i> 1 n i , making the overall length of the trajectories equal to T .
Revised: j s j (cid:107) for i ∈ { 2 , · · · , N B } and n 1 = T − (cid:80) i> 1 n i , making the overall length of the trajectories equal to T .

Original: In Figure 3, we illustrate the shift in distribution of function values due to our sampling strategy.
Revised: In Figure 3, we illustrate the shift in distribution of function values due to our sampling strategy.

Original: We refer to the combined strategy of sampling and then sorting as SORT-SAMPLE in Figure 2 and Algorithm 1.
Revised: We refer to the combined strategy of sampling and then sorting as SORT-SAMPLE in Figure 2 and Algorithm 1.

Original: Augmenting Trajectories With Regret Budgets Our sorted trajectories heuristically reflect rollouts of implicit black-box optimizers.
Revised: Augmenting Trajectories With Regret Budgets Our sorted trajectories heuristically reflect rollouts of implicit black-box optimizers.

Original: However they do not provide us with information on the rate at which a trajectory approaches the optimal value.
Revised: However they do not provide us with information on the rate at which a trajectory approaches the optimal value.

Original: A natural choice for such a knob would be the cumulative regret.
Revised: A natural choice for such a knob would be the cumulative regret.

Original: Moreover, as we shall show later, cumulative regret provides BONET a simple and effective knob to generalize outside the offline dataset during the evaluation phase.
Revised: Moreover, as we shall show later, cumulative regret provides BONET a simple and effective knob to generalize outside the offline dataset during the evaluation phase.

Original: Hence, we propose to augment each data point x i in our trajectory with a Regret Budget (RB).
Revised: Hence, we propose to augment each data point x i in our trajectory with a Regret Budget (RB).

Original: The RB R i at timestep i defined as the cumulative regret of the trajectory, starting at point x i :
Revised: The RB R i at timestep i defined as the cumulative regret of the trajectory, starting at point x i :

Original: R i =
Revised: R i =

Original: (cid:80)
Revised: (cid:80)

Original: Tj = i ( f ( x ∗ ) − f ( x j )) .
Revised: Tj = i ( f ( x ∗ ) − f ( x j )) .

Original: Intuitively, a high (low) value for R
Revised: Intuitively, a high (low) value for R

Original: i is likely to result in a high (low) budget for the model to explore diverse points.
Revised: i is likely to result in a high (low) budget for the model to explore diverse points.

Original: Note, we are only assuming knowledge of an estimate for f ( x ∗ ) (and not x ∗ ).
Revised: Note, we are only assuming knowledge of an estimate for f ( x ∗ ) (and not x ∗ ).

Original: Thus, each trajectory in our desired set D traj can be represented as:
Revised: Thus, each trajectory in our desired set D traj can be represented as:

Original: [Equation]
Revised: [Equation]

Original: We will refer to R 1 as Initial Regret Budget (IRB) henceforth.
Revised: We will refer to R 1 as Initial Regret Budget (IRB) henceforth.

Original: This will be of significance for evaluating our model in Phase 3 (Section 2.4), as we can specify a low IRB to induce the model to output points close to the optima.
Revised: This will be of significance for evaluating our model in Phase 3 (Section 2.4), as we can specify a low IRB to induce the model to output points close to the optima.

Original: 2. P HASE 2: T RAINING AN A UTOREGRESSIVE G ENERATIVE M ODEL
Revised: 2. P HASE 2: T RAINING AN A UTOREGRESSIVE G ENERATIVE M ODEL

Original: Given our trajectory dataset, we design our BBO agent as a conditional autoregressive model and train it to maximize the likelihood of trajectories in D traj .
Revised: Given our trajectory dataset, we design our BBO agent as a conditional autoregressive model and train it to maximize the likelihood of trajectories in D traj .

Original: More formally, we denote our model parameterized by θ as g θ ( x t | x <t , R ≤ t ) , where by k <t we mean the set { k 1 , · · · , k t − 1 } .
Revised: More formally, we denote our model parameterized by θ as g θ ( x t | x <t , R ≤ t ) , where by k <t we mean the set { k 1 , · · · , k t − 1 } .

Original: Here, x i are the sequence of points in a trajectory, and R i refers to the regret budget at timestep i .
Revised: Here, x i are the sequence of points in a trajectory, and R i refers to the regret budget at timestep i .

Original: Building on recent advances in sequence modeling (Vaswani et al., 2017; Brown et al., 2020; Radford et al., 2019) , we instantiate our model with a causally masked transformer and train it to maximize the likelihood of our trajectory dataset D traj .
Revised: Building on recent advances in sequence modeling (Vaswani et al., 2017; Brown et al., 2020; Radford et al., 2019) , we instantiate our model with a causally masked transformer and train it to maximize the likelihood of our trajectory dataset D traj .

Original: [Equation]
Revised: [Equation]

Original: In practice, we translate this loss to the mean squared error loss for a continuous X (equivalent to a
Revised: In practice, we translate this loss to the mean squared error loss for a continuous X (equivalent to a

Original: Gaussian g θ with fixed variance), and cross entropy loss for a discrete X .
Revised: Gaussian g θ with fixed variance), and cross entropy loss for a discrete X .

Original: 2. P HASE 3: E VALUATION R OLLOUT OF F INAL C ANDIDATES
Revised: 2. P HASE 3: E VALUATION R OLLOUT OF F INAL C ANDIDATES

Original: Once trained, we can use our BBO agent to directly output new points as its candidate guesses for maximizing the black-box function.
Revised: Once trained, we can use our BBO agent to directly output new points as its candidate guesses for maximizing the black-box function.

Original: We do so by rolling out evaluation trajectories from our model.
Revised: We do so by rolling out evaluation trajectories from our model.

Original: Each trajectory will be subdivided into a prefix subsequence and a prediction subsequence .
Revised: Each trajectory will be subdivided into a prefix subsequence and a prediction subsequence .

Original: The prefix subsequence consists of P < T points sampled from our offline dataset as before.
Revised: The prefix subsequence consists of P < T points sampled from our offline dataset as before.

Original: These prefix points provide initial warm-up queries to the model.
Revised: These prefix points provide initial warm-up queries to the model.

Original: Thereafter, we rollout the prediction subsequence consisting of T − P points by sampling from our autoregressive generative model.
Revised: Thereafter, we rollout the prediction subsequence consisting of T − P points by sampling from our autoregressive generative model.

Original: Setting Regret Budgets One key question relates to setting the regret budget at the start of the suffix subsequence.
Revised: Setting Regret Budgets One key question relates to setting the regret budget at the start of the suffix subsequence.

Original: It is not preferable to set it to R P +1 of the sampled trajectory, as doing so will lead to a slow rate of reaching high-quality regions similar to the one observed in the training trajectories.
Revised: It is not preferable to set it to R P +1 of the sampled trajectory, as doing so will lead to a slow rate of reaching high-quality regions similar to the one observed in the training trajectories.

Original: This will not allow the suffix to generalize beyond the offline dataset.
Revised: This will not allow the suffix to generalize beyond the offline dataset.

Original: Alternatively, we initialize it to a low value in BONET to accelerate the trajectory towards good points following a prefix subsequence.
Revised: Alternatively, we initialize it to a low value in BONET to accelerate the trajectory towards good points following a prefix subsequence.

Original: We refer to this low value as Evaluation RB and denote it as ˆ
Revised: We refer to this low value as Evaluation RB and denote it as ˆ

Original: R in Figure 2 and in Section 3.
Revised: R in Figure 2 and in Section 3.

Original: Thereafter, we keep the RB for the suffix subsequence fixed ( ˆ R ), as the agent is expected to be already in a good region.
Revised: Thereafter, we keep the RB for the suffix subsequence fixed ( ˆ R ), as the agent is expected to be already in a good region.

Original: Moreover, updating the RB here would require sequential querying to the function f , which can be prohibitive.
Revised: Moreover, updating the RB here would require sequential querying to the function f , which can be prohibitive.

Original: Thus, our evaluation protocol can generate a set of candidate queries purely in an offline manner.
Revised: Thus, our evaluation protocol can generate a set of candidate queries purely in an offline manner.

Original: In practice, we also find it helpful to split the Q candidates among a few (and not 1) small ˆ R values, each with a different prefix.
Revised: In practice, we also find it helpful to split the Q candidates among a few (and not 1) small ˆ R values, each with a different prefix.

Original:  E XPERIMENTAL E VALUATION
Revised:  E XPERIMENTAL E VALUATION

Original: We first empirically evaluate BONET for optimizing a synthetic 2D function, Branin, in order to analyze its working and probe the various components.
Revised: We first empirically evaluate BONET for optimizing a synthetic 2D function, Branin, in order to analyze its working and probe the various components.

Original: Next, we perform large-scale benchmarking and experiments on Design-Bench (Trabucco et al., 2022), a suite of offline BBO tasks based on real-world problems.
Revised: Next, we perform large-scale benchmarking and experiments on Design-Bench (Trabucco et al., 2022), a suite of offline BBO tasks based on real-world problems.

Original: 3. B RANIN T ASK
Revised: 3. B RANIN T ASK

Original: Branin is a well-known benchmark function for evaluating optimization methods.
Revised: Branin is a well-known benchmark function for evaluating optimization methods.

Original: It is a 2D function evaluated on the ranges x 1 ∈
Revised: It is a 2D function evaluated on the ranges x 1 ∈

Original: − 5 , 10] and x 2 ∈ [0 , 15] :
Revised: − 5 , 10] and x 2 ∈ [0 , 15] :

Original: [Equation]
Revised: [Equation]

Original: where a = − 1 , b = 5 .
Revised: where a = − 1 , b = 5 .

Original: 1 4 π 2 , c = 5 π , r = 6 , s = − 10 , and t = 15 π .
Revised: 1 4 π 2 , c = 5 π , r = 6 , s = − 10 , and t = 15 π .

Original: In this square region, f br has three global maximas, ( − π, 12 . 275) , ( π, 2 . 275) , and (9 . 42478 , 2 . 475) ; with the maximum value of − 0 .
Revised: In this square region, f br has three global maximas, ( − π, 12 . 275) , ( π, 2 . 275) , and (9 . 42478 , 2 . 475) ; with the maximum value of − 0 .

Original: Figure 1b shows an illustration of the function contours.
Revised: Figure 1b shows an illustration of the function contours.

Original: For offline optimization we uniformly sample N = 5000 points in the domain, and remove the top 10 %-ile (according to the function value) from this set to remove points close to the optima to make the task more challenging.
Revised: For offline optimization we uniformly sample N = 5000 points in the domain, and remove the top 10 %-ile (according to the function value) from this set to remove points close to the optima to make the task more challenging.

Original: We then construct 400 trajectories of length 64 each according to the SORT-SAMPLE strategy.
Revised: We then construct 400 trajectories of length 64 each according to the SORT-SAMPLE strategy.

Original: [Table]
Revised: [Table]

Original: During the evaluation, we initialize four trajectories with a prefix length of 32 and unroll them for an additionalsteps, and output the best result, thus consuming a query budget of 128.
Revised: During the evaluation, we initialize four trajectories with a prefix length of 32 and unroll them for an additionalsteps, and output the best result, thus consuming a query budget of 128.

Original: As we see in Table 1, BONET successfully generalizes beyond the best point in our offline dataset.
Revised: As we see in Table 1, BONET successfully generalizes beyond the best point in our offline dataset.

Original: We also report numbers for a gradient ascent baseline, which uses the offline dataset to train a forward model (a 2 layer NN) mapping x to y and then performs gradient ascent on x to infer its optima.
Revised: We also report numbers for a gradient ascent baseline, which uses the offline dataset to train a forward model (a 2 layer NN) mapping x to y and then performs gradient ascent on x to infer its optima.

Original: Next, we perform ablations to understand the effect of the Evaluation
Revised: Next, we perform ablations to understand the effect of the Evaluation

Original: RB ˆ R and prefix length P on our rolled-out trajectories.
Revised: RB ˆ R and prefix length P on our rolled-out trajectories.

Original: Impact of ˆR Figures 4a and 4b shows rolled-out trajectories for our model for different ˆ R values, with prefix lengths 16 and 32 .
Revised: Impact of ˆR Figures 4a and 4b shows rolled-out trajectories for our model for different ˆ R values, with prefix lengths 16 and 32 .

Original: We see that low ˆ R rolls out higher quality points compared to high ˆ R .
Revised: We see that low ˆ R rolls out higher quality points compared to high ˆ R .

Original: To verify our semantics of regret budget as a knob for controlling the rate at which the model accelerates to high-quality points, in the Figure 4c, we also plot trajectories where we update the RB values in the suffix.
Revised: To verify our semantics of regret budget as a knob for controlling the rate at which the model accelerates to high-quality points, in the Figure 4c, we also plot trajectories where we update the RB values in the suffix.

Original: We stop the roll-out if RB becomes non-positive.
Revised: We stop the roll-out if RB becomes non-positive.

Original: It is evident that for smaller ˆ
Revised: It is evident that for smaller ˆ

Original: R , the agent quickly accelerates to high-quality regions, whereas for high ˆ R , it gradually shifts to high-quality points.
Revised: R , the agent quickly accelerates to high-quality regions, whereas for high ˆ R , it gradually shifts to high-quality points.

Original: This shows how ˆ R controls the rate of transition from low to high-quality points.
Revised: This shows how ˆ R controls the rate of transition from low to high-quality points.

Original: To further check whether our model has learned to generate a sequence having cumulative regret close to the initial RB R 1 , we plot R 1 vs the cumulative regret of a full rolled-out sequence in
Revised: To further check whether our model has learned to generate a sequence having cumulative regret close to the initial RB R 1 , we plot R 1 vs the cumulative regret of a full rolled-out sequence in

Original: Figure 5a.
Revised: Figure 5a.

Original: We observe that the curve is close to the desired ideal line y
Revised: We observe that the curve is close to the desired ideal line y

Original: = x . Notice that the range of R 1 values of D traj is quite narrow, but BONET is able to generalize well to a much wider range, allowing it to propose points even better than the dataset.
Revised: = x . Notice that the range of R 1 values of D traj is quite narrow, but BONET is able to generalize well to a much wider range, allowing it to propose points even better than the dataset.

Original: During training, the model has only seen low RB values towards the end of the trajectories.
Revised: During training, the model has only seen low RB values towards the end of the trajectories.

Original: However, the powerful stitching ability (Chen et al., 2021) of the model allows it to roll out a novel trajectory having low cumulative regret when conditioned on low unseen R 1 values.
Revised: However, the powerful stitching ability (Chen et al., 2021) of the model allows it to roll out a novel trajectory having low cumulative regret when conditioned on low unseen R 1 values.

Original: Finally, since in BBO our goal is to find the best point, we also plot the best rolled-out point across a trajectory versus ˆ R in Figure 5b while keeping the prefix sequence fixed.
Revised: Finally, since in BBO our goal is to find the best point, we also plot the best rolled-out point across a trajectory versus ˆ R in Figure 5b while keeping the prefix sequence fixed.

Original: As expected, we observe a decreasing trend, justifying our choice of small ˆ R values.
Revised: As expected, we observe a decreasing trend, justifying our choice of small ˆ R values.

Original: Impact of Prefix Length Figure 5c shows the obtained best function values for different prefix lengths, averaged over multiple ˆ R values, with same query budget Q = 32 .
Revised: Impact of Prefix Length Figure 5c shows the obtained best function values for different prefix lengths, averaged over multiple ˆ R values, with same query budget Q = 32 .

Original: As expected, we see an increasing trend in the best function value.
Revised: As expected, we see an increasing trend in the best function value.

Original: We also observe a decreasing variance, indicating that the trajectory roll-outs are more stable when augmented with history of points.
Revised: We also observe a decreasing variance, indicating that the trajectory roll-outs are more stable when augmented with history of points.

Original: Note that prefix lengths larger than 32 doesn’t perform very well in practice because they have fewer than 32 shots to propose a good point in a single trajectory.
Revised: Note that prefix lengths larger than 32 doesn’t perform very well in practice because they have fewer than 32 shots to propose a good point in a single trajectory.

Original: Empirically, we found prefix length equal to half of the trajectory length to perform well across the experiments.
Revised: Empirically, we found prefix length equal to half of the trajectory length to perform well across the experiments.

Original: We provide more details about the ablations, experimental setup, and model hyper-parameters in the Appendix B and C.
Revised: We provide more details about the ablations, experimental setup, and model hyper-parameters in the Appendix B and C.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 3. D ESIGN -B ENCH TASKS
Revised: 3. D ESIGN -B ENCH TASKS

Original: Next, we evaluate BONET on 7 complex real-world tasks of Design-Bench (Trabucco et al., 2022) 1 .
Revised: Next, we evaluate BONET on 7 complex real-world tasks of Design-Bench (Trabucco et al., 2022) 1 .

Original: TF-Bind-8 and TF-Bind-10 are discrete tasks where the goal is to optimize for a DNA sequence which has maximum affinity to bind with a particular transcription factor.
Revised: TF-Bind-8 and TF-Bind-10 are discrete tasks where the goal is to optimize for a DNA sequence which has maximum affinity to bind with a particular transcription factor.

Original: The sequences are of length 8 ( 10 ) for TF-Bind-8 (TF-Bind-10), where each element in the sequence is one of 4 bases.
Revised: The sequences are of length 8 ( 10 ) for TF-Bind-8 (TF-Bind-10), where each element in the sequence is one of 4 bases.

Original: ChEMBL is a discrete task where the aim is to design a drug with certain qualities.
Revised: ChEMBL is a discrete task where the aim is to design a drug with certain qualities.

Original: NAS is a discrete task where we want to optimize a NN for performance on CIFAR10 (Krizhevsky et al., 2010).
Revised: NAS is a discrete task where we want to optimize a NN for performance on CIFAR10 (Krizhevsky et al., 2010).

Original: In
Revised: In

Original: D’Kitty and Ant morphology tasks, we optimize the morphology of two robots: Ant from OpenAI gym
Revised: D’Kitty and Ant morphology tasks, we optimize the morphology of two robots: Ant from OpenAI gym

Original: (Brockman et al., 2016) and D’Kitty from ROBEL (Ahn et al., 2019).
Revised: (Brockman et al., 2016) and D’Kitty from ROBEL (Ahn et al., 2019).

Original: In Superconductor task, the aim is to find a chemical formula for a superconducting material with high critical temperature.
Revised: In Superconductor task, the aim is to find a chemical formula for a superconducting material with high critical temperature.

Original: D’Kitty, Ant and Superconductor are continuous tasks with dimensions 56 , 60 , and 86 respectively.
Revised: D’Kitty, Ant and Superconductor are continuous tasks with dimensions 56 , 60 , and 86 respectively.

Original: For the first four tasks, we have query access to the exact oracle function.
Revised: For the first four tasks, we have query access to the exact oracle function.

Original: For Superconductor, we only have an approximate oracle, which is a random forest regressor trained on a much larger hidden dataset.
Revised: For Superconductor, we only have an approximate oracle, which is a random forest regressor trained on a much larger hidden dataset.

Original: These tasks are considered challenging due to high dimensionality, low quality points in the offline dataset, approximate oracles in some cases, and highly sensitive landscapes with narrow optima regions (Trabucco et al., 2022).
Revised: These tasks are considered challenging due to high dimensionality, low quality points in the offline dataset, approximate oracles in some cases, and highly sensitive landscapes with narrow optima regions (Trabucco et al., 2022).

Original: Baselines We compare BONET with multiple canonical baselines like gradient ascent, REINFORCE (Sutton et al., 1999), BayesOpt (Snoek et al., 2012) and CMA-ES (Hansen, 2006).
Revised: Baselines We compare BONET with multiple canonical baselines like gradient ascent, REINFORCE (Sutton et al., 1999), BayesOpt (Snoek et al., 2012) and CMA-ES (Hansen, 2006).

Original: We also compare with more recent methods like MINs (Kumar & Levine, 2020), COMs (Trabucco et al., 2021) and CbAS (Brookes et al., 2019).
Revised: We also compare with more recent methods like MINs (Kumar & Levine, 2020), COMs (Trabucco et al., 2021) and CbAS (Brookes et al., 2019).

Original: For inherently active methods like BayesOpt, since we cannot query the oracle function f ( x ) during optimization (due to being in an offline setting), we follow the procedure used by Trabucco et al.
Revised: For inherently active methods like BayesOpt, since we cannot query the oracle function f ( x ) during optimization (due to being in an offline setting), we follow the procedure used by Trabucco et al.

Original: and perform BayesOpt on a surrogate model
Revised: and perform BayesOpt on a surrogate model

Original: ˆ f ( x ) (a feedforward NN) trained on the offline dataset.
Revised: ˆ f ( x ) (a feedforward NN) trained on the offline dataset.

Original: For the BayesOpt baseline, we use a Gaussian Process to quantify uncertainty and use the quasi-Expected Improvement (Wilson et al., 2017)
Revised: For the BayesOpt baseline, we use a Gaussian Process to quantify uncertainty and use the quasi-Expected Improvement (Wilson et al., 2017)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: acquisition function for optimization, similar to prior work (Trabucco et al., 2022; 2021).
Revised: acquisition function for optimization, similar to prior work (Trabucco et al., 2022; 2021).

Original: MINs (Kumar & Levine, 2020) also train a forward model to optimize over the conditioning parameters.
Revised: MINs (Kumar & Levine, 2020) also train a forward model to optimize over the conditioning parameters.

Original: Our method does not need a separate forward model and thus, is not dependent on the quality of the fit of the forward model.
Revised: Our method does not need a separate forward model and thus, is not dependent on the quality of the fit of the forward model.

Original: We provide other variants of BayesOpt baseline in Appendix D.2.
Revised: We provide other variants of BayesOpt baseline in Appendix D.2.

Original: Evaluation We allow a query budget of Q = 256 for all the baselines, except for NAS, where we use a reduced budget of Q = 

Revised: Evaluation We allow a query budget of Q = 256 for all the baselines, except for NAS, where we use a reduced budget of Q =

Original: due to compute restrictions.
Revised: due to compute restrictions.

Original: For BONET, across all tasks, we roll out 4 trajectories, each with a prefix and prediction subsequence length of 64 each.
Revised: For BONET, across all tasks, we roll out 4 trajectories, each with a prefix and prediction subsequence length of 64 each.

Original: The prediction subsequence is initialized with one of 4 candidate low
Revised: The prediction subsequence is initialized with one of 4 candidate low

Original: ˆ R values 0 , 0 . 01 , 0 . 05 , 0 .
Revised: ˆ R values 0 , 0 . 01 , 0 . 05 , 0 .

Original: For each ˆ R , we then roll out for 64 timesteps and choose the best point.
Revised: For each ˆ R , we then roll out for 64 timesteps and choose the best point.

Original: We report the mean and standard deviation over 5 trials for each of the models and tasks in Table 2.
Revised: We report the mean and standard deviation over 5 trials for each of the models and tasks in Table 2.

Original: Following the procedure used by (Trabucco et al., 2021; 2022; Yu et al., 2021), the results of Table 2 are linearly normalized between the minimum and maximum values of a large hidden offline dataset.
Revised: Following the procedure used by (Trabucco et al., 2021; 2022; Yu et al., 2021), the results of Table 2 are linearly normalized between the minimum and maximum values of a large hidden offline dataset.

Original: In Table 3, we also present the median function value of the the proposed output points for each method, averaged over 5 runs.
Revised: In Table 3, we also present the median function value of the the proposed output points for each method, averaged over 5 runs.

Original: Results Overall, we obtain a mean score of 0 .
Revised: Results Overall, we obtain a mean score of 0 .

Original: 772 and an average rank of 2 . 4 , which is the best among all the baselines.
Revised: 772 and an average rank of 2 . 4 , which is the best among all the baselines.

Original: We also achieve the best results on three tasks.
Revised: We also achieve the best results on three tasks.

Original: Additionally, we are among the top 2 for five out of seven tasks.
Revised: Additionally, we are among the top 2 for five out of seven tasks.

Original: We show significant improvements over generative methods such as MINs (Kumar & Levine, 2020) or CbAS (Brookes et al., 2019), and forward mapping methods such as COMs (Trabucco et al., 2021) on TF-Bind-8, TF-Bind-10, Ant and D’Kitty.
Revised: We show significant improvements over generative methods such as MINs (Kumar & Levine, 2020) or CbAS (Brookes et al., 2019), and forward mapping methods such as COMs (Trabucco et al., 2021) on TF-Bind-8, TF-Bind-10, Ant and D’Kitty.

Original: We also note that while BONET is placed second in Ant, it shows a much lower standard deviation ( 0 . 012 ) compared to the to the best performing method CMA-ES, which has a much larger standard deviation of 0 . 928 .
Revised: We also note that while BONET is placed second in Ant, it shows a much lower standard deviation ( 0 . 012 ) compared to the to the best performing method CMA-ES, which has a much larger standard deviation of 0 . 928 .

Original: We also have a lowest mean standard deviation across all tasks, suggesting that BONET is less sensitive to bad initializations compared to other methods.
Revised: We also have a lowest mean standard deviation across all tasks, suggesting that BONET is less sensitive to bad initializations compared to other methods.

Original: We report the unnormalized results, ablations and other experimental details for Design-Bench tasks in Appendix C.
Revised: We report the unnormalized results, ablations and other experimental details for Design-Bench tasks in Appendix C.

Original: BONET also performs best on 50 th percentile evaluation, showing that it has a better set of proposed points compared to other methods, and that the performance is not by randomly finding good points.
Revised: BONET also performs best on 50 th percentile evaluation, showing that it has a better set of proposed points compared to other methods, and that the performance is not by randomly finding good points.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Active BBO
Revised: Active BBO

Original: The majority of prior work in BBO have been in the active setting, where surrogate models are allowed to query the function during training.
Revised: The majority of prior work in BBO have been in the active setting, where surrogate models are allowed to query the function during training.

Original: This includes long bodies of work in Bayesian Optimization, e.g., (Snoek et al., 2012; Swersky et al., 2013; Srinivas et al., 2010;
Revised: This includes long bodies of work in Bayesian Optimization, e.g., (Snoek et al., 2012; Swersky et al., 2013; Srinivas et al., 2010;

Original: Nguyen & Osborne, 2020) and bandits, e.g., (Garivier et al., 2016; Riquelme et al., 2018; Joachims et al., 2018b;a; Swaminathan & Joachims, 2015; Jacq et al., 2019).
Revised: Nguyen & Osborne, 2020) and bandits, e.g., (Garivier et al., 2016; Riquelme et al., 2018; Joachims et al., 2018b;a; Swaminathan & Joachims, 2015; Jacq et al., 2019).

Original: Such methods usually employ surrogate models such as Gaussian Processes (Srinivas et al., 2010), Neural Processes (Garnelo et al., 2018a;b; Gordon et al., 2019; Anonymous, 2022; Singh et al., 2019) or Bayesian Neural
Revised: Such methods usually employ surrogate models such as Gaussian Processes (Srinivas et al., 2010), Neural Processes (Garnelo et al., 2018a;b; Gordon et al., 2019; Anonymous, 2022; Singh et al., 2019) or Bayesian Neural

Original: Networks (Chang, 2021; Goan & Fookes, 2020) to approximate the black-box function, and an uncertainty-aware acquisition strategy for querying new points.
Revised: Networks (Chang, 2021; Goan & Fookes, 2020) to approximate the black-box function, and an uncertainty-aware acquisition strategy for querying new points.

Original: Offline BBO
Revised: Offline BBO

Original: Recent works have made use of such datasets and shown promising results (Kumar & Levine, 2020; Trabucco et al., 2021;
Revised: Recent works have made use of such datasets and shown promising results (Kumar & Levine, 2020; Trabucco et al., 2021;

Original: Brookes et al., 2019; Fannjiang & Listgarten, 2020; Fu &
Revised: Brookes et al., 2019; Fannjiang & Listgarten, 2020; Fu &

Original: Levine, 2021; Yu et al., 2021).
Revised: Levine, 2021; Yu et al., 2021).

Original: Kumar & Levine (2020) train a stochastic inverse mapping from the outputs y to inputs x using a generative model similar to a conditional GAN (Goodfellow et al., 2014; Mirza & Osindero, 2014; Arjovsky et al., 2017; Nowozin et al., 2016).
Revised: Kumar & Levine (2020) train a stochastic inverse mapping from the outputs y to inputs x using a generative model similar to a conditional GAN (Goodfellow et al., 2014; Mirza & Osindero, 2014; Arjovsky et al., 2017; Nowozin et al., 2016).

Original: They then optimize over y to find a good design point.
Revised: They then optimize over y to find a good design point.

Original: Training GANs however can be difficult due to issues like mode collapse (Arjovsky et al., 2017).
Revised: Training GANs however can be difficult due to issues like mode collapse (Arjovsky et al., 2017).

Original: Other methods make use of gradient ascent to find an optimal solution.
Revised: Other methods make use of gradient ascent to find an optimal solution.

Original: Trabucco et al.
Revised: Trabucco et al.

Original: (2021) and Yu et al.
Revised: (2021) and Yu et al.

Original: (2021) train a model to be robust to outliers by regularizing the objective such that they assign low score to those points.
Revised: (2021) train a model to be robust to outliers by regularizing the objective such that they assign low score to those points.

Original: Fu & Levine (2021) train a normalized maximum likelihood estimate of the function.
Revised: Fu & Levine (2021) train a normalized maximum likelihood estimate of the function.

Original: We instead offer a fresh perspective based on generative sequence modelling and we show strong results in comparison to many of these prior works in Section 3.
Revised: We instead offer a fresh perspective based on generative sequence modelling and we show strong results in comparison to many of these prior works in Section 3.

Original: Offline reinforcement learning (RL) While both RL and BBO are sequential decision-making problems, the key difference is that RL is stateful while BBO is not.
Revised: Offline reinforcement learning (RL) While both RL and BBO are sequential decision-making problems, the key difference is that RL is stateful while BBO is not.

Original: In the offline setting (Schmidhuber, 2019; Jacq et al., 2019), both problems require models or policies that can generalize beyond the offline dataset to achieve good performance.
Revised: In the offline setting (Schmidhuber, 2019; Jacq et al., 2019), both problems require models or policies that can generalize beyond the offline dataset to achieve good performance.

Original: Related to our work, autoregressive transformers have been successfully applied on trajectory data obtained from offline RL (Chen et al., 2021; Janner et al., 2021).
Revised: Related to our work, autoregressive transformers have been successfully applied on trajectory data obtained from offline RL (Chen et al., 2021; Janner et al., 2021).

Original: However, there are important differences between their setting and offline BBO.
Revised: However, there are important differences between their setting and offline BBO.

Original: For example, the data in offline BBO is not sequential in nature, unlike in offline RL where the offline data is naturally in the form of demonstrations.
Revised: For example, the data in offline BBO is not sequential in nature, unlike in offline RL where the offline data is naturally in the form of demonstrations.

Original: One of our contributions is to design a notion of ‘high-to-low’ sequences to BBO for autoregressive modeling and test-time generalization.
Revised: One of our contributions is to design a notion of ‘high-to-low’ sequences to BBO for autoregressive modeling and test-time generalization.

Original: Learned optimizers Our work also bears resemblance to work on meta-learning optimizers like those of (Andrychowicz et al., 2016) and (Chen et al., 2017).
Revised: Learned optimizers Our work also bears resemblance to work on meta-learning optimizers like those of (Andrychowicz et al., 2016) and (Chen et al., 2017).

Original: However, a key difference between these learned optimizers and BONET is that they require access to gradients either during training time or during both training and evaluation time, whereas BONET has no such restriction, meaning it can work in situations where access to gradient information is not practical (for instance with nondifferentiable black-box functions).
Revised: However, a key difference between these learned optimizers and BONET is that they require access to gradients either during training time or during both training and evaluation time, whereas BONET has no such restriction, meaning it can work in situations where access to gradient information is not practical (for instance with nondifferentiable black-box functions).

Original: Furthermore, we concentrate on the offline setting, in contrast to learned optimizer work which usually looks at an active optimization setting.
Revised: Furthermore, we concentrate on the offline setting, in contrast to learned optimizer work which usually looks at an active optimization setting.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: We presented BONET, a novel generative framework for pretraining black-box optimizers using offline data.
Revised: We presented BONET, a novel generative framework for pretraining black-box optimizers using offline data.

Original: BONET consists of a 3 phased process.
Revised: BONET consists of a 3 phased process.

Original: In the first phase, we use a novel SORTSAMPLE strategy to generate trajectories from offline data that use the sorting heuristic to mimic the behavior of online BBO optimizers.
Revised: In the first phase, we use a novel SORTSAMPLE strategy to generate trajectories from offline data that use the sorting heuristic to mimic the behavior of online BBO optimizers.

Original: In phases 2 and 3, we train our model using an autoregressive transformer and use it to generate candidate points that maximize the black-box function.
Revised: In phases 2 and 3, we train our model using an autoregressive transformer and use it to generate candidate points that maximize the black-box function.

Original: Experimentally, we verify that BONET is capable of solving complex high-dimensional tasks like the ones in Design-Bench, achieving an average rank of 2.4 with a mean score of 0.772 .
Revised: Experimentally, we verify that BONET is capable of solving complex high-dimensional tasks like the ones in Design-Bench, achieving an average rank of 2.4 with a mean score of 0.772 .

Original: Limitations and Future Work BONET assumes knowledge of the approximate value of the optima f ( x ∗ ) to computes the regrets of points.
Revised: Limitations and Future Work BONET assumes knowledge of the approximate value of the optima f ( x ∗ ) to computes the regrets of points.

Original: Though we show in Appendix C.5 that different reasonable estimates of f ( x ∗ ) can give similar results, this is still something we would like to address in future work.
Revised: Though we show in Appendix C.5 that different reasonable estimates of f ( x ∗ ) can give similar results, this is still something we would like to address in future work.

Original: We are also interested in extending BONET to an active setting where our model can quantify uncertainty and actively query the black-box function after pretraining on an offline dataset.
Revised: We are also interested in extending BONET to an active setting where our model can quantify uncertainty and actively query the black-box function after pretraining on an offline dataset.

Original: On a practical side, we would also be interested in analyzing the properties of domains that dictate where BONET can strongly excel (e.g., D’Kitty) or struggle (e.g., Superconductor) relative to other approaches.
Revised: On a practical side, we would also be interested in analyzing the properties of domains that dictate where BONET can strongly excel (e.g., D’Kitty) or struggle (e.g., Superconductor) relative to other approaches.

Original: Finally, we aim to expand our scope to a meta-task setting, where instead of a single function, our offline data consists of past evaluations from multiple black-box functions.
Revised: Finally, we aim to expand our scope to a meta-task setting, where instead of a single function, our offline data consists of past evaluations from multiple black-box functions.

Original:  E THICAL AND S OCIAL R ESPONSIBILITY
Revised:  E THICAL AND S OCIAL R ESPONSIBILITY

Original: Offline black-box optimization can play a critical role in improving efficiency and safety in many real-world settings like nuclear reactors or the pharmaceutical industry, where active optimization may prove to be computationally inefficient (requiring too many queries) or even dangerous.
Revised: Offline black-box optimization can play a critical role in improving efficiency and safety in many real-world settings like nuclear reactors or the pharmaceutical industry, where active optimization may prove to be computationally inefficient (requiring too many queries) or even dangerous.

Original: However, while we do not anticipate anything inherently malicious about our work, it is possible to utilize our proposed method (and other optimizers in general) in malicious settings (e.g., optimizing for drugs that have harmful effects).
Revised: However, while we do not anticipate anything inherently malicious about our work, it is possible to utilize our proposed method (and other optimizers in general) in malicious settings (e.g., optimizing for drugs that have harmful effects).

Original: This is something to remember when deploying algorithms such as ours for high stake real-world applications.
Revised: This is something to remember when deploying algorithms such as ours for high stake real-world applications.

Original:  R EPRODUCIBILITY
Revised:  R EPRODUCIBILITY

Original: Throughout the paper, we maintained a high standard of rigor for reproducibility.
Revised: Throughout the paper, we maintained a high standard of rigor for reproducibility.

Original: We report the detailed pseudocode of our method in Algorithm 1, and we also provide the link to our code via an anonymized link here.
Revised: We report the detailed pseudocode of our method in Algorithm 1, and we also provide the link to our code via an anonymized link here.

Original: We provide more details on our training setup and choice of hyperparameters in Appendix B.
Revised: We provide more details on our training setup and choice of hyperparameters in Appendix B.

Original: We report results on multiple datasets in Design-Bench (Trabucco et al., 2022), each with different properties and benchmark our method over multiple baselines from different families of approaches.
Revised: We report results on multiple datasets in Design-Bench (Trabucco et al., 2022), each with different properties and benchmark our method over multiple baselines from different families of approaches.

Original: Our results are averaged over 5 seeds, and we also provide the standard deviations.
Revised: Our results are averaged over 5 seeds, and we also provide the standard deviations.

Original: We also conduct several ablations to evaluate sensitivity of BONET to different parameters.
Revised: We also conduct several ablations to evaluate sensitivity of BONET to different parameters.

Original: A N OTATIONS & T HEORETICAL A NALYSIS A. N OTATIONS
Revised: A N OTATIONS & T HEORETICAL A NALYSIS A. N OTATIONS

Original: For ease of reference
Revised: For ease of reference

Original: [Table]
Revised: [Table]

Original: we list the notations in Table 4.
Revised: we list the notations in Table 4.

Original: [Table]
Revised: [Table]

Original: A. T HEORETICAL A NALYSIS
Revised: A. T HEORETICAL A NALYSIS

Original: One crucial component in the SORT-SAMPLE is sorting the trajectories in the increasing order of the function values.
Revised: One crucial component in the SORT-SAMPLE is sorting the trajectories in the increasing order of the function values.

Original: Although our primary motivation for sorting is derived from the empirical observations from online black-box optimizers (Figure 1c), we note that for a certain class of functions that are non-trivial to solve from the perspective of optimization (maximization for our paper), lower (higher) function values occupy a larger (smaller) domain.
Revised: Although our primary motivation for sorting is derived from the empirical observations from online black-box optimizers (Figure 1c), we note that for a certain class of functions that are non-trivial to solve from the perspective of optimization (maximization for our paper), lower (higher) function values occupy a larger (smaller) domain.

Original: Thus, intuitively we can relate lower function values with exploration and higher function values with exploitation.
Revised: Thus, intuitively we can relate lower function values with exploration and higher function values with exploitation.

Original: With this perspective, sorting can be seen as moving from a high-diversity region to a low-diversity regiona behavior typically seen in online black-box optimizers (Bijl et al., 2016;
Revised: With this perspective, sorting can be seen as moving from a high-diversity region to a low-diversity regiona behavior typically seen in online black-box optimizers (Bijl et al., 2016;

Original: Garivier et al., 2016).
Revised: Garivier et al., 2016).

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we try to formally prove such properties for this constrained class of functions.
Revised: we try to formally prove such properties for this constrained class of functions.

Original: We consider the simplified case of differential 1D functions with certain assumptions for simplicity, and further extend this notion to a more general D -dimensional case.
Revised: We consider the simplified case of differential 1D functions with certain assumptions for simplicity, and further extend this notion to a more general D -dimensional case.

Original: First, we define a notion of ϵ -high points.
Revised: First, we define a notion of ϵ -high points.

Original: Definition 1.
Revised: Definition 1.

Original: ϵ -high values.
Revised: ϵ -high values.

Original: Let the range of f be denoted as [ y min , y max ] .
Revised: Let the range of f be denoted as [ y min , y max ] .

Original: Then, a function value y in this range is ϵ -high if y ≥ y min + ϵ ( y max − y min ) .
Revised: Then, a function value y in this range is ϵ -high if y ≥ y min + ϵ ( y max − y min ) .

Original: Intuitively, the above definition implies that y is ϵ -high if it is in the top 1 − ϵ fraction of the range of f .
Revised: Intuitively, the above definition implies that y is ϵ -high if it is in the top 1 − ϵ fraction of the range of f .

Original: The following result characterizes the relative diversity of regions consisting of ϵ -high points for 1 -D functions.
Revised: The following result characterizes the relative diversity of regions consisting of ϵ -high points for 1 -D functions.

Original: Proposition 1.
Revised: Proposition 1.

Original: Let f : [ a, b ] → R , a, b ∈ R , be a real-valued, continuous and differentiable function such that the f ( a ) and f ( b ) are not ϵ -high.
Revised: Let f : [ a, b ] → R , a, b ∈ R , be a real-valued, continuous and differentiable function such that the f ( a ) and f ( b ) are not ϵ -high.

Original: Let H ⊆
Revised: Let H ⊆

Original: [ a, b ] be a Lebesgue-measurable set of x values for which f
Revised: [ a, b ] be a Lebesgue-measurable set of x values for which f

Original: ( x ) is ϵ -high, with ϵ > 0 .
Revised: ( x ) is ϵ -high, with ϵ > 0 .

Original: Let H c
Revised: Let H c

Original: =
Revised: =

Original: [ a, b ] \ H .
Revised: [ a, b ] \ H .

Original: Without loss of generality, let’s assume that f ( a ) ≤ f ( b ) .
Revised: Without loss of generality, let’s assume that f ( a ) ≤ f ( b ) .

Original: If the Lipchitz constant L of f is upper bounded by
Revised: If the Lipchitz constant L of f is upper bounded by

Original: [Equation]
Revised: [Equation]

Original: then | H | < | H c | , where | · | denotes volume of a set w.r.t.
Revised: then | H | < | H c | , where | · | denotes volume of a set w.r.t.

Original: the Lebesgue Measure.
Revised: the Lebesgue Measure.

Original: Proof of Proposition 1.
Revised: Proof of Proposition 1.

Original: Note that if no point in the domain [ a, b ] achieves ϵ -high function value, then the statement holds trivially true.
Revised: Note that if no point in the domain [ a, b ] achieves ϵ -high function value, then the statement holds trivially true.

Original: So, we assume that there is atleast one point which has ϵ -high function value.
Revised: So, we assume that there is atleast one point which has ϵ -high function value.

Original: Let x 1 and x 2 be the smallest and largest such points in the domain.
Revised: Let x 1 and x 2 be the smallest and largest such points in the domain.

Original: Since the boundary points doesn’t have ϵ -high values, | H c | ≥ ( x 1 − a )
Revised: Since the boundary points doesn’t have ϵ -high values, | H c | ≥ ( x 1 − a )

Original: + ( b − x 2 ) and | H | ≤ ( x 2 − x 1 ) .
Revised: + ( b − x 2 ) and | H | ≤ ( x 2 − x 1 ) .

Original: Thus, if we prove that ( x 1 − a )+( b − x 2 ) ≥ ( x 2 − x 1 ) , then we are done.
Revised: Thus, if we prove that ( x 1 − a )+( b − x 2 ) ≥ ( x 2 − x 1 ) , then we are done.

Original: To prove this, we try to prove ( x 1 − a ) ≥ ( x 2 − x 1 ) .
Revised: To prove this, we try to prove ( x 1 − a ) ≥ ( x 2 − x 1 ) .

Original: Assume, on the contrary, that ( x 1 − a ) < ( x 2 − x 1 ) .
Revised: Assume, on the contrary, that ( x 1 − a ) < ( x 2 − x 1 ) .

Original: Rearranging, we get
Revised: Rearranging, we get

Original: [Equation]
Revised: [Equation]

Original: Now, by the definition of Lipchitz constant, we have:
Revised: Now, by the definition of Lipchitz constant, we have:

Original: [Equation]
Revised: [Equation]

Original: Last inequality holds because x 2 − a ≤ b − a .
Revised: Last inequality holds because x 2 − a ≤ b − a .

Original: This inequality contradicts the bound 6 on L , completing our proof for Proposition 1.
Revised: This inequality contradicts the bound 6 on L , completing our proof for Proposition 1.

Original: Now, we show an extension of this proposition for D -dimensional functions with hypercube domains.
Revised: Now, we show an extension of this proposition for D -dimensional functions with hypercube domains.

Original: Proposition 2.
Revised: Proposition 2.

Original: Let f : X → R be a D -dimensional, real-valued, continuous, and differentiable function with hypercube domain X
Revised: Let f : X → R be a D -dimensional, real-valued, continuous, and differentiable function with hypercube domain X

Original: =
Revised: =

Original: [ a 1 , b 1 ] × [ a 2 , b 2 ] , · · · , × [ a D , b D ] , such that none of the boundary points are ϵ -high, for some fixed ϵ .
Revised: [ a 1 , b 1 ] × [ a 2 , b 2 ] , · · · , × [ a D , b D ] , such that none of the boundary points are ϵ -high, for some fixed ϵ .

Original: Here by boundary points, we mean the points on the surface of the domain hypercube.
Revised: Here by boundary points, we mean the points on the surface of the domain hypercube.

Original: Let y max and y min be the maximum and the minimum values attained by f .
Revised: Let y max and y min be the maximum and the minimum values attained by f .

Original: Let H ⊆
Revised: Let H ⊆

Original: X be a Lebesgue-measurable set of points for which f
Revised: X be a Lebesgue-measurable set of points for which f

Original: ( x ) is ϵ -high.
Revised: ( x ) is ϵ -high.

Original: Let
Revised: Let

Original: H c = X \ H .
Revised: H c = X \ H .

Original: If the Lipchitz constant L of f is upper bounded by
Revised: If the Lipchitz constant L of f is upper bounded by

Original: [Equation]
Revised: [Equation]

Original: then | H | < | H c | , where | · | denotes volume of a set w.r.t.
Revised: then | H | < | H c | , where | · | denotes volume of a set w.r.t.

Original: the Lebesgue Measure.
Revised: the Lebesgue Measure.

Original: Proof.
Revised: Proof.

Original: We prove this proposition by induction on the number of dimensions D .
Revised: We prove this proposition by induction on the number of dimensions D .

Original: Notice that for
Revised: Notice that for

Original: D = 1 , the statement reduces to Proposition 1, which we have already proved.
Revised: D = 1 , the statement reduces to Proposition 1, which we have already proved.

Original: Next, we assume that the statement holds for ( D − 1) -dimensional functions and prove it for D dimensions, with
Revised: Next, we assume that the statement holds for ( D − 1) -dimensional functions and prove it for D dimensions, with

Original: D > 1 .
Revised: D > 1 .

Original: Let’s define H D,ϵ : F D → B D to be a functional that maps any D -dimensional function, say f , to a
Revised: Let’s define H D,ϵ : F D → B D to be a functional that maps any D -dimensional function, say f , to a

Original: Lebesgue-measurable subset of R D that corresponds to the set of points where f ( x ) is ϵ -high.
Revised: Lebesgue-measurable subset of R D that corresponds to the set of points where f ( x ) is ϵ -high.

Original: , Here,
Revised: , Here,

Original: F D and B D denote the set of all D -dimensional functions and the set of all Lebesgue-measurable subsets of R D respectively.
Revised: F D and B D denote the set of all D -dimensional functions and the set of all Lebesgue-measurable subsets of R D respectively.

Original: We similarly define the mapping H cD,ϵ to be a functional mapping a function f to the complement of its ϵ -high region.
Revised: We similarly define the mapping H cD,ϵ to be a functional mapping a function f to the complement of its ϵ -high region.

Original: Thus, H =
Revised: Thus, H =

Original: H D,ϵ ( f ) and H c = H cD,ϵ ( f ) .
Revised: H D,ϵ ( f ) and H c = H cD,ϵ ( f ) .

Original: Now, by definition,
Revised: Now, by definition,

Original: [Equation]
Revised: [Equation]

Original: And similarly,
Revised: And similarly,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Consequently, to prove
Revised: Consequently, to prove

Original: [Equation]
Revised: [Equation]

Original: · , • ) | , we prove |H
Revised: · , • ) | , we prove |H

Original: D f for every
Revised: D f for every

Original: D
Revised: D

Original: D ] .
Revised: D ] .

Original: To do this, we first fix the D th dimension to be c .
Revised: To do this, we first fix the D th dimension to be c .

Original: In other words, we are considering the ( D − 1) - dimensional slice of f ( x 1 , · · · , x D ) with x D = c .
Revised: In other words, we are considering the ( D − 1) - dimensional slice of f ( x 1 , · · · , x D ) with x D = c .

Original: Let g be such a slice with g ( x 1 , ·
Revised: Let g be such a slice with g ( x 1 , ·

Original: · · , x D − 1 ) =
Revised: · · , x D − 1 ) =

Original: f ( x 1 , · · · , x D − 1 , c ) .
Revised: f ( x 1 , · · · , x D − 1 , c ) .

Original: First we need ϵ g for which ϵ g -high value for g is ϵ -high for f :
Revised: First we need ϵ g for which ϵ g -high value for g is ϵ -high for f :

Original: [Equation]
Revised: [Equation]

Original: where y gmax and y gmin are the minimum and maximum values respectively achieved by g .
Revised: where y gmax and y gmin are the minimum and maximum values respectively achieved by g .

Original: By this choice of ϵ g , we are ensuring that a point ( x 1 , · · · , x D − 1 , c ) is ϵ -high w.r.t
Revised: By this choice of ϵ g , we are ensuring that a point ( x 1 , · · · , x D − 1 , c ) is ϵ -high w.r.t

Original: f
Revised: f

Original: if and only if ( x 1 , · · · , x D − 1 ) is ϵ g -high w.r.t g .
Revised: if and only if ( x 1 , · · · , x D − 1 ) is ϵ g -high w.r.t g .

Original: In other words,
Revised: In other words,

Original: [Equation]
Revised: [Equation]

Original: Let the Lipchitz constant of g be L g .
Revised: Let the Lipchitz constant of g be L g .

Original: First we show that L g ≤ L .
Revised: First we show that L g ≤ L .

Original: By definition of Lipchitz constant, for x = ( x 1 , · · · x D − 1 ) , z = ( z 1 , · · · , z D − 1 ) in the domain of g ,
Revised: By definition of Lipchitz constant, for x = ( x 1 , · · · x D − 1 ) , z = ( z 1 , · · · , z D − 1 ) in the domain of g ,

Original: [Equation]
Revised: [Equation]

Original: Where last inequality is by definition of L w.r.t f .
Revised: Where last inequality is by definition of L w.r.t f .

Original: Combining this with our bound on L in 9, we get
Revised: Combining this with our bound on L in 9, we get

Original: L g ≤
Revised: L g ≤

Original: [Equation]
Revised: [Equation]

Original: Thus, the Lipchitz bound assumption is followed by g with ϵ
Revised: Thus, the Lipchitz bound assumption is followed by g with ϵ

Original: = ϵ g .
Revised: = ϵ g .

Original: Also, the boundaries are not ϵ ghigh w.r.t g because of the choice of ϵ g .
Revised: Also, the boundaries are not ϵ ghigh w.r.t g because of the choice of ϵ g .

Original: This implies, by inductive assumption, that |H
Revised: This implies, by inductive assumption, that |H

Original: D − 1 ,ϵ g ( g ) | ≤ |H cD − 1 ,ϵ g ( g ) | .
Revised: D − 1 ,ϵ g ( g ) | ≤ |H cD − 1 ,ϵ g ( g ) | .

Original: This, combined with equality 13 proves that that |H D − 1 ,ϵ ( f ( • , · · · , • , c ))
Revised: This, combined with equality 13 proves that that |H D − 1 ,ϵ ( f ( • , · · · , • , c ))

Original: | ≤ |H cD − 1 ,ϵ ( f ( • , · · · , • , c )) | .
Revised: | ≤ |H cD − 1 ,ϵ ( f ( • , · · · , • , c )) | .

Original: Since this is true for all c ∈ [ a D , b D ] , by equations 10 and 11, our proof for |H D,ϵ
Revised: Since this is true for all c ∈ [ a D , b D ] , by equations 10 and 11, our proof for |H D,ϵ

Original: ( f ( • , · · · , • ) | ≤ |H cD,ϵ ( f ( • , · · · , • ) | is complete.
Revised: ( f ( • , · · · , • ) | ≤ |H cD,ϵ ( f ( • , · · · , • ) | is complete.

Original: B E XPERIMENTAL D ETAILS B. SORT-SAMPLE
Revised: B E XPERIMENTAL D ETAILS B. SORT-SAMPLE

Original: In SORT-SAMPLE, the score of each bin is calculated according to the formula
Revised: In SORT-SAMPLE, the score of each bin is calculated according to the formula

Original: [Equation]
Revised: [Equation]

Original: The two variables K and τ here act as smoothing parameters.
Revised: The two variables K and τ here act as smoothing parameters.

Original: K controls the relative priority given to the larger bins (bins with more points).
Revised: K controls the relative priority given to the larger bins (bins with more points).

Original: Higher value of K assigns higher relative weight to these large bins compared to smaller bins, whereas a low value of K the weight assigned to large and small bins would be similar.
Revised: Higher value of K assigns higher relative weight to these large bins compared to smaller bins, whereas a low value of K the weight assigned to large and small bins would be similar.

Original: In the extreme case where K = 0 , the weight due to | B i | | B i | + K will always be 1, regardless of bin size.
Revised: In the extreme case where K = 0 , the weight due to | B i | | B i | + K will always be 1, regardless of bin size.

Original: For very large value of K , the weight will be approximately linearly proportional to the bin size | B i | .
Revised: For very large value of K , the weight will be approximately linearly proportional to the bin size | B i | .

Original: The later case is not desirable because if there is a bin
Revised: The later case is not desirable because if there is a bin

Original: [Figure]
Revised: [Figure]

Original: As expected, we don’t see much sensitivity to the choice of K and τ .
Revised: As expected, we don’t see much sensitivity to the choice of K and τ .

Original: For N B , as expected, we a significant decrease for N B = 1 , but N B = 32 is comparable to N B =which has very large number of points (which might be a low quality bin), then most of the total weight will be given to that bin because of the linear proportionality.
Revised: For N B , as expected, we a significant decrease for N B = 1 , but N B = 32 is comparable to N B =which has very large number of points (which might be a low quality bin), then most of the total weight will be given to that bin because of the linear proportionality.

Original: Temperature τ controls how harshly the bad bins are penalized.
Revised: Temperature τ controls how harshly the bad bins are penalized.

Original: Lower the τ value, lower the relative score of the low quality bins (bins with high regret) and vice versa.
Revised: Lower the τ value, lower the relative score of the low quality bins (bins with high regret) and vice versa.

Original: In our experiments, we don’t tune the values of K , τ , and number of bins N B .
Revised: In our experiments, we don’t tune the values of K , τ , and number of bins N B .

Original: In all the tasks, we use K = 0 .
Revised: In all the tasks, we use K = 0 .

Original: 03 × N and τ = R 10 , where R 10 is the 10 th percentile regret value in D .
Revised: 03 × N and τ = R 10 , where R 10 is the 10 th percentile regret value in D .

Original: For Branin task, we use , N B = 32 and for all the Design-Bench tasks, we use N B = 64 .
Revised: For Branin task, we use , N B = 32 and for all the Design-Bench tasks, we use N B = 64 .

Original: Empirically, as we show in Figure 6a, we didn’t observe much effect on K in the range [0 . 01 , 0 . 1] , and for τ from the 50 th to the 10 th percentiles of R .
Revised: Empirically, as we show in Figure 6a, we didn’t observe much effect on K in the range [0 . 01 , 0 . 1] , and for τ from the 50 th to the 10 th percentiles of R .

Original: Figure 6b shows variation with N B , keeping all other parameters fixed.
Revised: Figure 6b shows variation with N B , keeping all other parameters fixed.

Original: As expected, N B = 1 doesn’t perform well, as having just one bin is equivalent to having no re-weighting.
Revised: As expected, N B = 1 doesn’t perform well, as having just one bin is equivalent to having no re-weighting.

Original: Beyond 32 , we don’t see much variation with the value of N B .
Revised: Beyond 32 , we don’t see much variation with the value of N B .

Original: B. M ODEL ARCHITECTURE & IMPLEMENTATION
Revised: B. M ODEL ARCHITECTURE & IMPLEMENTATION

Original: Architecture We use a GPT (Radford et al., 2019) like architecture, where each timestep refers to two tokens R t and x t .
Revised: Architecture We use a GPT (Radford et al., 2019) like architecture, where each timestep refers to two tokens R t and x t .

Original: Similar to Chen et al.
Revised: Similar to Chen et al.

Original: (2021), we add a new learned timestep embedding (in addition to the positional embedding already present in transformers).
Revised: (2021), we add a new learned timestep embedding (in addition to the positional embedding already present in transformers).

Original: Each token R t and x t that goes as input to our model is first projected into a 128 dimensional embedding space using a linear embedding layer.
Revised: Each token R t and x t that goes as input to our model is first projected into a 128 dimensional embedding space using a linear embedding layer.

Original: To this embedding, we also add the positional and timestep embeddings.
Revised: To this embedding, we also add the positional and timestep embeddings.

Original: This is passed through a causally masked transformer.
Revised: This is passed through a causally masked transformer.

Original: The prediction head for R t predicts ˆ x t , which is then used to compute the loss.
Revised: The prediction head for R t predicts ˆ x t , which is then used to compute the loss.

Original: The output of the prediction head for x t is discarded.
Revised: The output of the prediction head for x t is discarded.

Original: At each timestep, we feed in the last C timesteps to the model, where C here refers to the context length.
Revised: At each timestep, we feed in the last C timesteps to the model, where C here refers to the context length.

Original: For continuous tasks, the prediction head for R t outputs a d -dimensional prediction ˆ x t .
Revised: For continuous tasks, the prediction head for R t outputs a d -dimensional prediction ˆ x t .

Original: For discrete tasks, the prediction head outputs a V × d -dimensional prediction, where V refers to the number of classes in the discrete task.
Revised: For discrete tasks, the prediction head outputs a V × d -dimensional prediction, where V refers to the number of classes in the discrete task.

Original: Thus, each dimension in X corresponds to a V -dimensional logits vector.
Revised: Thus, each dimension in X corresponds to a V -dimensional logits vector.

Original: Code Our code (available at the anonymized link here) is built upon the code from minGPT 3 and
Revised: Code Our code (available at the anonymized link here) is built upon the code from minGPT 3 and

Original: Chen et al.
Revised: Chen et al.

Original: All code we use is under the MIT licence.
Revised: All code we use is under the MIT licence.

Original: Training The parameter details for all the tasks are summarized in the Table 5.
Revised: Training The parameter details for all the tasks are summarized in the Table 5.

Original: Note that almost all of the parameters are same across all the Design-Bench tasks.
Revised: Note that almost all of the parameters are same across all the Design-Bench tasks.

Original: Number of layers is higher for continuous tasks, as they are of higher dimensionalities.
Revised: Number of layers is higher for continuous tasks, as they are of higher dimensionalities.

Original: For all the tasks, we use a batch size ofand a fixed learning rate of 10 − 4 for 75 epochs.
Revised: For all the tasks, we use a batch size ofand a fixed learning rate of 10 − 4 for 75 epochs.

Original: All training is done using 10 Intel(R)
Revised: All training is done using 10 Intel(R)

Original: Xeon(R) CPU cores (E5-2698 v4443 @
Revised: Xeon(R) CPU cores (E5-2698 v4443 @

Original: 2.20GHz) and one NVIDIA Tesla V100 SXM2 GPU.
Revised: 2.20GHz) and one NVIDIA Tesla V100 SXM2 GPU.

Original: B. E VALUATION
Revised: B. E VALUATION

Original: For all the Design-Bench tasks (except NAS), we use a query budget Q = 256 .
Revised: For all the Design-Bench tasks (except NAS), we use a query budget Q = 256 .

Original: For NAS, we use a query budget of Q = 128 due to compute restrictions.
Revised: For NAS, we use a query budget of Q = 128 due to compute restrictions.

Original: Since we use a trajectory length of 128 and a prefix length of 64 , this means that we can roll-out four different trajectories.
Revised: Since we use a trajectory length of 128 and a prefix length of 64 , this means that we can roll-out four different trajectories.

Original: There are two variable parameters during the evaluation: Evaluation RB ( ˆ R ) and the prefix sub-sequence.
Revised: There are two variable parameters during the evaluation: Evaluation RB ( ˆ R ) and the prefix sub-sequence.

Original: We empirically observed that ˆ R has more impact on the variability of rolled-out points compared to prefix subsequence.
Revised: We empirically observed that ˆ R has more impact on the variability of rolled-out points compared to prefix subsequence.

Original: Hence, we roll-out trajectory for 4 different low
Revised: Hence, we roll-out trajectory for 4 different low

Original: ˆ R values (0 . 0 , 0 . 01 , 0 . 05 , 0 . 1) .
Revised: ˆ R values (0 . 0 , 0 . 01 , 0 . 05 , 0 . 1) .

Original: These values are kept fixed across all the tasks and are not tuned.
Revised: These values are kept fixed across all the tasks and are not tuned.

Original: They are chosen to probe the interval [0 . 0 , 0 . 1] , while giving slightly more importance to the low values by choosing 0 .
Revised: They are chosen to probe the interval [0 . 0 , 0 . 1] , while giving slightly more importance to the low values by choosing 0 .

Original: Evaluation strategies with lower query budget Q available are discussed in the section C.
Revised: Evaluation strategies with lower query budget Q available are discussed in the section C.

Original: [Table]
Revised: [Table]

Original: B.
Revised: B.

Original: [Table]
Revised: [Table]

Original: F IXED VS . U PDATED RB DURING EVALUATION
Revised: F IXED VS . U PDATED RB DURING EVALUATION

Original: During the evaluation of the prediction sequence, we proposed to keep the Regret Budget (RB) fixed.
Revised: During the evaluation of the prediction sequence, we proposed to keep the Regret Budget (RB) fixed.

Original: One alternative strategy is to sequentially update RB after every iteration, i.e. predict x t from R t , and compute R t +1 =
Revised: One alternative strategy is to sequentially update RB after every iteration, i.e. predict x t from R t , and compute R t +1 =

Original: R t − ( f ( x ∗ ) − f ( x t )) .
Revised: R t − ( f ( x ∗ ) − f ( x t )) .

Original: However, there are two issues with such an update rule:
Revised: However, there are two issues with such an update rule:

Original: Updating RB adds a sequential dependency on our model during evaluation, as we must query f ( x t ) to compute R t +1 .
Revised: Updating RB adds a sequential dependency on our model during evaluation, as we must query f ( x t ) to compute R t +1 .

Original: Thus, generating the Q candidate points is not purely offline.
Revised: Thus, generating the Q candidate points is not purely offline.

Original: While updating the regret budget R t , it is possible that at some timestep t , R t becomes negative.
Revised: While updating the regret budget R t , it is possible that at some timestep t , R t becomes negative.

Original: Since the model has never seen negative RB values during training, this is undesirable.
Revised: Since the model has never seen negative RB values during training, this is undesirable.

Original: Hence, we do not update RB during evaluation, and instead provide a fixed ˆ R value at every timestep after the prefix length.
Revised: Hence, we do not update RB during evaluation, and instead provide a fixed ˆ R value at every timestep after the prefix length.

Original: This way, point proposal is not dependent on sequential evaluations of f , making it much faster as the evaluations on f can then be parallelized.
Revised: This way, point proposal is not dependent on sequential evaluations of f , making it much faster as the evaluations on f can then be parallelized.

Original: Furthermore, by not updating ˆ
Revised: Furthermore, by not updating ˆ

Original: R we sidestep the issue of negative RBs.
Revised: R we sidestep the issue of negative RBs.

Original: Empirically, as we see in Figure 7, there is not much difference across different strategies, which justifies our choice of not updating RB, allowing our method to be purely offline.
Revised: Empirically, as we see in Figure 7, there is not much difference across different strategies, which justifies our choice of not updating RB, allowing our method to be purely offline.

Original: B. B ASELINES
Revised: B. B ASELINES

Original: For the gradient ascent baseline of Branin task, we train a 2 layer neural network (with hidden layer of size 128 ) as a forward model for 75 epochs with a fixed learning rate of 10 − 4 .
Revised: For the gradient ascent baseline of Branin task, we train a 2 layer neural network (with hidden layer of size 128 ) as a forward model for 75 epochs with a fixed learning rate of 10 − 4 .

Original: For gradient ascent on the learnt model during evaluation, we report results with a step size of 0 .
Revised: For gradient ascent on the learnt model during evaluation, we report results with a step size of 0 .

Original: 1 for 64 steps.
Revised: 1 for 64 steps.

Original: We average over 5 seeds, and for each seed we perform two random restarts.
Revised: We average over 5 seeds, and for each seed we perform two random restarts.

Original: For the baselines in the Design-Bench tasks, we run the baseline code 5 provided in Trabucco et al.
Revised: For the baselines in the Design-Bench tasks, we run the baseline code 5 provided in Trabucco et al.

Original: and report results with the default parameters for a query budget of 256.
Revised: and report results with the default parameters for a query budget of 256.

Original: B. D ESIGN -B ENCH T ASKS
Revised: B. D ESIGN -B ENCH T ASKS

Original: For the Design-Bench tasks, we pre-process the offline dataset to normalize the function values before constructing the trajectory dataset D traj .
Revised: For the Design-Bench tasks, we pre-process the offline dataset to normalize the function values before constructing the trajectory dataset D traj .

Original: Normalized y norm values are computed as y norm =
Revised: Normalized y norm values are computed as y norm =

Original: y − y min
Revised: y − y min

Original: y max − y min , where y min and y max are minimum and maximum function values of a much larger hidden dataset.
Revised: y max − y min , where y min and y max are minimum and maximum function values of a much larger hidden dataset.

Original: Note that we only use the knowledge of y min and y max from this hidden dataset, and not the corresponding x values.
Revised: Note that we only use the knowledge of y min and y max from this hidden dataset, and not the corresponding x values.

Original: With this normalization procedure, we use 1 . 0 as an estimate of f ( x ∗ ) while constructing trajectories in D traj .The
Revised: With this normalization procedure, we use 1 . 0 as an estimate of f ( x ∗ ) while constructing trajectories in D traj .The

Original: results we report in Table 2 are also normalized using the same procedure, similar to prior works (Trabucco et al., 2022; 2021).
Revised: results we report in Table 2 are also normalized using the same procedure, similar to prior works (Trabucco et al., 2022; 2021).

Original: We also report unnormalized results in Table
Revised: We also report unnormalized results in Table

Original: [Table] [Table]
Revised: [Table] [Table]

Original: B. H OPPER T ASK
Revised: B. H OPPER T ASK

Original: We didn’t include the Hopper task in our results in Table 2 because of the inconsistency between the offline dataset values and the oracle outputs.
Revised: We didn’t include the Hopper task in our results in Table 2 because of the inconsistency between the offline dataset values and the oracle outputs.

Original: Hopper data consist of 3200 points, each withdimensions.
Revised: Hopper data consist of 3200 points, each withdimensions.

Original: The lowest and highest function values are 87 .
Revised: The lowest and highest function values are 87 .

Original: 93 and 1361 .
Revised: 93 and 1361 .

Original: Figure 8 shows the distribution of the normalized function values.
Revised: Figure 8 shows the distribution of the normalized function values.

Original: This distribution is extremely skewed towards low function values.
Revised: This distribution is extremely skewed towards low function values.

Original: Only 6 points out of 3200 have a normalized function value greater than 0 .
Revised: Only 6 points out of 3200 have a normalized function value greater than 0 .

Original: We noticed that the oracle of Hopper is highly inaccurate for points with higher function values.
Revised: We noticed that the oracle of Hopper is highly inaccurate for points with higher function values.

Original: Figure 9 shows the function values in the dataset vs. the oracle output for the top 10 best points in the data, clearly showing the inconsistency between the two.
Revised: Figure 9 shows the function values in the dataset vs. the oracle output for the top 10 best points in the data, clearly showing the inconsistency between the two.

Original: In fact, the oracle minima and maxima for the dataset are just 56 .
Revised: In fact, the oracle minima and maxima for the dataset are just 56 .

Original: 26 and 786 . 79 , respectively, far from the actual dataset values.
Revised: 26 and 786 . 79 , respectively, far from the actual dataset values.

Original: Due to such discrepancies, we have decided not to include the Hopper task in our analysis.
Revised: Due to such discrepancies, we have decided not to include the Hopper task in our analysis.

Original: C A DDITIONAL E XPERIMENTAL R ESULTS C. A BLATIONS ON SORT-SAMPLE S TRATEGY
Revised: C A DDITIONAL E XPERIMENTAL R ESULTS C. A BLATIONS ON SORT-SAMPLE S TRATEGY

Original: SORT-SAMPLE algorithm has two main components:
Revised: SORT-SAMPLE algorithm has two main components:

Original: Sampling after re-weighting and sorting.
Revised: Sampling after re-weighting and sorting.

Original: Our sorting heuristic is primarily motivated by typical runs of online optimizers.
Revised: Our sorting heuristic is primarily motivated by typical runs of online optimizers.

Original: To show this, we run an online GP to optimize the three synthetic functions, namely the negative Branin, negative
Revised: To show this, we run an online GP to optimize the three synthetic functions, namely the negative Branin, negative

Original: Goldstein-Price and negative Six hump camel functions and plot the function values for the proposed points.
Revised: Goldstein-Price and negative Six hump camel functions and plot the function values for the proposed points.

Original: Figure 10 shows sample trajectories of the function values of the proposed maxima after each function evaluation.
Revised: Figure 10 shows sample trajectories of the function values of the proposed maxima after each function evaluation.

Original: We can see, on average, the function values tend to increase over time as the number of queries increases.
Revised: We can see, on average, the function values tend to increase over time as the number of queries increases.

Original: Such behavior has also been reported for other black-box functions and setups, see e.g. (Bijl et al., 2016).
Revised: Such behavior has also been reported for other black-box functions and setups, see e.g. (Bijl et al., 2016).

Original: However, in this section, we do perform ablations to see the effects of these components.
Revised: However, in this section, we do perform ablations to see the effects of these components.

Original: To this end, we construct trajectories using 4 strategies:
Revised: To this end, we construct trajectories using 4 strategies:

Original: Random: Uniformly randomly sample a trajectory from the offline dataset.
Revised: Random: Uniformly randomly sample a trajectory from the offline dataset.

Original: Random + Sorting: Uniformly randomly sample a trajectory from the offline dataset and sort it in ascending order of the function values.
Revised: Random + Sorting: Uniformly randomly sample a trajectory from the offline dataset and sort it in ascending order of the function values.

Original: Re-weighting + Partial Sorting: Perform re-weighting, uniformly randomly sample n i number of points from each bin, and concatenate them from lowest quality bin to the high-
Revised: Re-weighting + Partial Sorting: Perform re-weighting, uniformly randomly sample n i number of points from each bin, and concatenate them from lowest quality bin to the high-

Original: est quality bin.
Revised: est quality bin.

Original: This way, the trajectory will be partially sorted, i.e. the order of the bins themselves is sorted, but the points sampled from a bin will be randomly ordered.
Revised: This way, the trajectory will be partially sorted, i.e. the order of the bins themselves is sorted, but the points sampled from a bin will be randomly ordered.

Original: In this case, the trajectories are not entirely monotonic w.r.t.
Revised: In this case, the trajectories are not entirely monotonic w.r.t.

Original: the function values.
Revised: the function values.

Original: Intuitively, this intermingles exploration and exploitation phases within and across bins respectively.
Revised: Intuitively, this intermingles exploration and exploitation phases within and across bins respectively.

Original: Re-weighting + Sorting (default in BONET):
Revised: Re-weighting + Sorting (default in BONET):

Original: Sort the trajectory obtained in strategy 3.
Revised: Sort the trajectory obtained in strategy 3.

Original: This is the default setting we use in our experiments.
Revised: This is the default setting we use in our experiments.

Original: Figure 11 contains the results obtained by each of the four strategies.
Revised: Figure 11 contains the results obtained by each of the four strategies.

Original: Note that while going from strategy 1 to 2, we keep the points sampled in a trajectory the same, so the only difference between them is sorting.
Revised: Note that while going from strategy 1 to 2, we keep the points sampled in a trajectory the same, so the only difference between them is sorting.

Original: Figure 11 shows that strategy 1 clearly outperforms strategy 2.
Revised: Figure 11 shows that strategy 1 clearly outperforms strategy 2.

Original: This means that sorting has a significant impact on the results.
Revised: This means that sorting has a significant impact on the results.

Original: Next, note that strategy 2 and 4 differ only in their sampling strategy, and strategy 4 outperforms strategy 2, which shows the effectiveness of re-weighting.
Revised: Next, note that strategy 2 and 4 differ only in their sampling strategy, and strategy 4 outperforms strategy 2, which shows the effectiveness of re-weighting.

Original: This experiment justifies our choice for both re-weighting and sorting.
Revised: This experiment justifies our choice for both re-weighting and sorting.

Original: C. A NALYSIS ON QUERY BUDGET Q
Revised: C. A NALYSIS ON QUERY BUDGET Q

Original: So far, we have been discussing the results with query budget Q = 256 .
Revised: So far, we have been discussing the results with query budget Q = 256 .

Original: Here, we describe the evaluation strategy we use when lower query budget is available.
Revised: Here, we describe the evaluation strategy we use when lower query budget is available.

Original: Our strategy will be to give higher preference to lower ˆ R values when lower budget is available.
Revised: Our strategy will be to give higher preference to lower ˆ R values when lower budget is available.

Original: For example, when Q = 192 , we only roll-out and evaluate for ˆ R ∈ { 0 . 0 , 0 . 01 , 0 . 05 } .
Revised: For example, when Q = 192 , we only roll-out and evaluate for ˆ R ∈ { 0 . 0 , 0 . 01 , 0 . 05 } .

Original: For 192 < Q ≤ 256 , we will roll-out for ˆ R ∈ { 0 . 0 , 0 . 01 , 0 . 05 , 0 . 1 } , evaluate the entire predicted sub-sequences of lengths 64 for { 0 . 0 , 0 . 01 , 0 . 05 } , and evaluate the first Q − 192 points in the predicted sub-sequences for ˆ R = 0 . 1 .
Revised: For 192 < Q ≤ 256 , we will roll-out for ˆ R ∈ { 0 . 0 , 0 . 01 , 0 . 05 , 0 . 1 } , evaluate the entire predicted sub-sequences of lengths 64 for { 0 . 0 , 0 . 01 , 0 . 05 } , and evaluate the first Q − 192 points in the predicted sub-sequences for ˆ R = 0 . 1 .

Original: In the Figure 12, we present the results for different query budgets for our method compared to important baselines, for the D’Kitty task.
Revised: In the Figure 12, we present the results for different query budgets for our method compared to important baselines, for the D’Kitty task.

Original: We outperform the baselines for almost all the query budget values.
Revised: We outperform the baselines for almost all the query budget values.

Original: C. A DDITIONAL A BLATIONS
Revised: C. A DDITIONAL A BLATIONS

Original: Here we present ablations similar to Section 3 on the D’Kitty task, and observe similar trends to what we see in the Branin ablations.
Revised: Here we present ablations similar to Section 3 on the D’Kitty task, and observe similar trends to what we see in the Branin ablations.

Original: C. E FFECT OF P REFIX S EQUENCES
Revised: C. E FFECT OF P REFIX S EQUENCES

Original: During the evaluation, the unrolled trajectory depends on two factors affecting the unrolled output:
Revised: During the evaluation, the unrolled trajectory depends on two factors affecting the unrolled output:

Original: Evaluation Regret Budget ˆ R and the prefix sequence.
Revised: Evaluation Regret Budget ˆ R and the prefix sequence.

Original: Empirically, we found that ˆ R has a larger impact on the unrolled trajectory than the prefix sequence.
Revised: Empirically, we found that ˆ R has a larger impact on the unrolled trajectory than the prefix sequence.

Original: To show this, we first evaluate the Branin task for 10 different randomly sampled prefix sequences for a fixed ˆ R and then do the same with
Revised: To show this, we first evaluate the Branin task for 10 different randomly sampled prefix sequences for a fixed ˆ R and then do the same with

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 10 different ˆ R values sampled from the range (0 . 0 , 0 . 5) for the same prefix sequence.
Revised: 10 different ˆ R values sampled from the range (0 . 0 , 0 . 5) for the same prefix sequence.

Original: Figureshows the standard deviation of the minimum regret of the 10 different unrolled trajectories for
Revised: Figureshows the standard deviation of the minimum regret of the 10 different unrolled trajectories for

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: fixed ˆ R and prefix sequences, respectively.
Revised: fixed ˆ R and prefix sequences, respectively.

Original: The standard deviation for the variation in prefix length is consistently lower than that for the variation in ˆ R , explaining our choice of spending query budget on different ˆ R rather than different prefix sequences.
Revised: The standard deviation for the variation in prefix length is consistently lower than that for the variation in ˆ R , explaining our choice of spending query budget on different ˆ R rather than different prefix sequences.

Original: C. E FFECT OF ESTIMATING y MAX
Revised: C. E FFECT OF ESTIMATING y MAX

Original: A key assumption of our method is the knowledge of y max .
Revised: A key assumption of our method is the knowledge of y max .

Original: Though in many problems this is not an issue, there are many other problems where the value of y max may not be known.
Revised: Though in many problems this is not an issue, there are many other problems where the value of y max may not be known.

Original: A simple solution could be to just estimate y max .
Revised: A simple solution could be to just estimate y max .

Original: In Figure 16 we evaluate BONET on D’Kitty multiple varying values of y max starting from just beyond the dataset maxima.
Revised: In Figure 16 we evaluate BONET on D’Kitty multiple varying values of y max starting from just beyond the dataset maxima.

Original: We find that the value of y max initially affects performance alot, but beyond a point, it plateaus..
Revised: We find that the value of y max initially affects performance alot, but beyond a point, it plateaus..

Original: [Table] [Table]
Revised: [Table] [Table]

Original: C. A BLATION ON MODEL PARAMETERS
Revised: C. A BLATION ON MODEL PARAMETERS

Original: In this experiment we study the effect of changing the number parameters in BONET.
Revised: In this experiment we study the effect of changing the number parameters in BONET.

Original: We do this by altering the number of layers and heads in BONET on D’Kitty.
Revised: We do this by altering the number of layers and heads in BONET on D’Kitty.

Original: We find that increasing the number of parameters helps up to a point, beyond which the model over-fits.
Revised: We find that increasing the number of parameters helps up to a point, beyond which the model over-fits.

Original: It is important to note that we present this study only to understand the impact of model size on our performance.
Revised: It is important to note that we present this study only to understand the impact of model size on our performance.

Original: We don’t actually tune over these parameters in our experiments.
Revised: We don’t actually tune over these parameters in our experiments.

Original: They are kept fixed across all the discrete and continuous tasks (refer to Table 5).
Revised: They are kept fixed across all the discrete and continuous tasks (refer to Table 5).

Original: [Figure]
Revised: [Figure]

Original: C. A BLATIONS ON D ATASET S IZE
Revised: C. A BLATIONS ON D ATASET S IZE

Original: To test the limits of BONET we run experiments where we withhold offline training data from
Revised: To test the limits of BONET we run experiments where we withhold offline training data from

Original: BONET and evaluate the performance.
Revised: BONET and evaluate the performance.

Original: We have two settings, one where we withhold an x % size random subsection of data in Table 8, and another where we withhold the top x percentile of data during training and evaluation in Table 9.
Revised: We have two settings, one where we withhold an x % size random subsection of data in Table 8, and another where we withhold the top x percentile of data during training and evaluation in Table 9.

Original: We see that with just reducing the number of points, we don’t see as sharp of a drop off in performance as compared to when we withhold the good points in the dataset.
Revised: We see that with just reducing the number of points, we don’t see as sharp of a drop off in performance as compared to when we withhold the good points in the dataset.

Original: This leads us to believe that the dominating effect is not the size of the dataset exactly, but the quality of points in the dataset.
Revised: This leads us to believe that the dominating effect is not the size of the dataset exactly, but the quality of points in the dataset.

Original: Further, note that even here the points proposed are significantly larger than the maximum point in the dataset, which rules out the possibility of memorization for BONET.
Revised: Further, note that even here the points proposed are significantly larger than the maximum point in the dataset, which rules out the possibility of memorization for BONET.

Revised: C. N OISE A BLATION

Revised: We run an experiment where we add progressive larger amounts of noise to the y values in our offline dataset while training our model, to test how robust BONET is to noisy data.

Revised: We report the results in Table 10 for D’Kitty.

Revised: We find that, as expected, increasing noise reduces performance, and

Revised: BONET is in fact reasonably robust to noise, and the largest drop-off occurs when the magnitude of noise is equal to the magnitude of values.

Original: [Table]
Revised: [Table]

Original:  R ANDOM B ASELINE
Revised: C. R ANDOM B ASELINE

Original: One might argue that BONET just memorizes the best points in the offline dataset and outputs random points close to those best points during evaluation.
Revised: One might argue that BONET just memorizes the best points in the offline dataset and outputs random points close to those best points during evaluation.

Original: To rule out this possibility, we perform a simple experiment for the D’Kitty task.
Revised: To rule out this possibility, we perform a simple experiment for the D’Kitty task.

Original: We choose a small hypercube domain around the optimal point in the offline dataset and uniformly randomly sample 256 points in that domain.
Revised: We choose a small hypercube domain around the optimal point in the offline dataset and uniformly randomly sample 256 points in that domain.

Original: In Table 10, we show the results for different widths of this hypercube.
Revised: In Table 11, we show the results for different widths of this hypercube.

Original: 0 width means only the best point in the dataset.
Revised: 0 width means only the best point in the dataset.

Original: For smaller hypercubes around the best points in the offline dataset, we see that the best point found by 256 random searches is roughly 225 , which is significantly lower than what BONET finds ( 291 . 08 ).
Revised: For smaller hypercubes around the best points in the offline dataset, we see that the best point found by 256 random searches is roughly 225 , which is significantly lower than what BONET finds ( 291 . 08 ).

Original: For larger hypercubes, the points are highly diverse.
Revised: For larger hypercubes, the points are highly diverse.

Original: These observations suggest that this optimization problem cannot be solved by just randomly outputting points around the best point in the dataset.
Revised: These observations suggest that this optimization problem cannot be solved by just randomly outputting points around the best point in the dataset.

Original: If we look at the 256 points output by BONET, they are consistently good (mean is 220 ), with comparatively very low variance.
Revised: If we look at the 256 points output by BONET, they are consistently good (mean is 220 ), with comparatively very low variance.

Original: This suggests that BONET is not simply outputting random points around the best points in the dataset.
Revised: This suggests that BONET is not simply outputting random points around the best points in the dataset.

Original: D A DDITIONAL A NALYSIS 
Revised: D A DDITIONAL A NALYSIS D. V IZUALIZATION OF P REDICTED P OINTS

Original: [Figure] V IZUALIZATION OF P REDICTED P OINTS

Original: dataset.

Original: Three maxima regions don’t contain any dataset points because of the removal of the top 10 %-ile of uniformly sampled points, as described in the section 3.1.

Original: However, almost all the unrolled points fall into a maxima region, clearly showing the generalization capability of BONET.

Original: Here we try to visualize the predicted points of BONET compared to the points in the offline data to study the nature of the points proposed by the model.
Revised: Here we try to visualize the predicted points of BONET compared to the points in the offline data to study the nature of the points proposed by the model.

Original: As shown in Figure 18, BONET generalizes well on the unseen maxima regions of the function and produces low regret points.
Revised: As shown in Figure 18, BONET generalizes well on the unseen maxima regions of the function and produces low regret points.

Original: D. A CTIVE GP EXPERIMENT
Revised: D. A CTIVE GP EXPERIMENT

Original: We also run a experiment to compare BONET with an active BBO method.
Revised: We also run a experiment to compare BONET with an active BBO method.

Original: Namely, we compare
Revised: Namely, we compare

Original: BONET with active BayesOpt, using the same GP prior and acquisition function (quasi-Expected
Revised: BONET with active BayesOpt, using the same GP prior and acquisition function (quasi-Expected

Original: Improvement) as mentioned in Section 3.
Revised: Improvement) as mentioned in Section 3.

Original: The difference between the active method and the offline method we compare with in Table 2 is that while the active method directly optimizes the ground truth function, the offline method first trains a surrogate neural network on the data, and then performs bayesian optimization on the surrogate instead of the ground truth function.
Revised: The difference between the active method and the offline method we compare with in Table 2 is that while the active method directly optimizes the ground truth function, the offline method first trains a surrogate neural network on the data, and then performs bayesian optimization on the surrogate instead of the ground truth function.

Original: This is done to make the BayesOpt baseline fully offline, and is the same procedure followed by Trabucco et al.
Revised: This is done to make the BayesOpt baseline fully offline, and is the same procedure followed by Trabucco et al.

Original: Note that this would result in an unfair comparison since the method is both online and queries the oracle function resulting in it using more queries than our budget.
Revised: Note that this would result in an unfair comparison since the method is both online and queries the oracle function resulting in it using more queries than our budget.

Original: As shown in Table 11, We find that using oracle actually doesn’t necessarily improve performance across all tasks, and there are other tasks where the performance doesn’t change at all.
Revised: As shown in Table 12, We find that using oracle actually doesn’t necessarily improve performance across all tasks, and there are other tasks where the performance doesn’t change at all.

Original: And our model does outperform even the oracle GP-qEI method on several tasks.
Revised: And our model does outperform even the oracle GP-qEI method on several tasks.

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: D. T -SNE PLOTS ON D’K ITTY
Revised: D. T -SNE PLOTS ON D’K ITTY

Original: We show t-SNE plots on DKitty for the datasets of differing sizes, with the removal of randomly sampled x % of the data (setting one described in the previous section).
Revised: We show t-SNE plots on DKitty for the datasets of differing sizes, with the removal of randomly sampled x % of the data (setting one described in the previous section).

Original: The blue points represent points proposed by our model, and the red points represent points in the dataset.
Revised: The blue points represent points proposed by our model, and the red points represent points in the dataset.

Original: In general, blue points do not overlap the red points, indicating that the points proposed by BONET are from a different region.
Revised: In general, blue points do not overlap the red points, indicating that the points proposed by BONET are from a different region.

Original: [Figure]
Revised: [Figure]