Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques
Authors: Manh Cuong Dao
Phi Le Nguyen
Thao Nguyen Truong
Trong Nghia Hoang
Venue: NeurIPS 2024 poster (2024)
Keywords: Black-Box Optimization
Primary area: other
TL;DR: This paper incorporates a notion of model sharpness into the training loss of the surrogate as a sharpness regularizer to boost the performance of existing offline optimization methods..

Abstract: Offline optimization has recently emerged as an increasingly popular approach to mitigate the prohibitively expensive cost of online experimentation. The key idea is to learn a surrogate of the black-box function that underlines the target experiment using a static (offline) dataset of its previous input-output queries. Such an approach is, however, fraught with an out-of-distribution issue where the learned surrogate becomes inaccurate outside the offline data regimes. To mitigate this, existing offline optimizers have proposed numerous conditioning techniques to prevent the learned surrogate from being too erratic. Nonetheless, such conditioning strategies are often specific to particular surrogate or search models, which might not generalize to a different model choice. This motivates us to develop a model-agnostic approach instead, which incorporates a notion of model sharpness into the training loss of the surrogate as a regularizer. Our approach is supported by a new theoretical analysis demonstrating that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our analysis extends existing theories from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate on training data, providing a new perspective on sharpness regularization. Our extensive experimentation on a diverse range of optimization tasks also shows that reducing surrogate sharpness often leads to significant improvement, marking (up to) a noticeable 9.6% performance boost. Our code is publicly available at https://github.com/cuong-dm/IGNITE.

Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques
Manh Cuong Dao, Le Nguyen Hanoi University of Science and Technology
Cuong Boong.D22440 D124229992224 hust.. edu. vn, in,eenpsoict.. hustt.ccc.........oooooommme.. hust. edu. vn
Thao Nguyen Truong
National Institute of Advanced Industrial Science and Technology nguyen. bruong@aist.go..
Trong Nghia Hoang" Washington State University trongughia.. www.aa....annnu..coooooomommm edu
Abstract
Offline optimization has recently emerged as an increasingly popular approach mitigate the prohibitively expensive cost of online experimentation. The key idea is to learn a surrogate of the black-box function that underlines the target experiment using static (offline) dataset of its previous input-output queries, Such approach is, however, fraught with an out-of-distributie surrogate becomes inaccurate outside the offline data regimes. To mitigate this, existing offline optimizers have proposed numerous conditioning techniques to prevent the learned surrogate from being too erratic. Nonetheless, such conditioning strategies are often specific to particular surrogate search models, which might not generalize to a different model choice. This motivates us to develop model- agnostic approach instead, which incorporates notion of model sharpness into the training loss of the surrogate as a regularizer. Our approach is supported by a new theoretical analysis demonstrating that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our amalysis extends existing theories from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate training data, providing new perspective on sharpness regularization. Our extensive experimentation on a diverse range of optimization tasks also shows that reducing surrogate sharpness often leads significant improvement, marking (up to) a noticeable 9.6% performance boost. Our code is publicly available https: / //gi thub. com/ cuong- dm IINIIE.
issue where the learned
Introduction
A central task in numerous scientific disciplines is to optimize for some material configuration that maximizes certain utility metric. Previously, this would incur an expensive and repetitive experiment process that requires a huge amount of babon-labor. To bypass such inefficiencies, data-driven approach [1, 3, 4, 5, 7, 8, 9, 10] has recently been adopted. Instead of laboring on new set of expensive on-demand experimentation for each new optimization task, we can leverage past experimentation data build a parametric model that predicts the outcome of the experiments themselves. Its parameters are tuned to fit past experimental data and then fixed while optimizing for best input.
Cooresponding authors: Manh Cuong Dao, Trong Nghia Hoang.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

However, in most applications, offline data is rarely representative of the entire input space. As result, the surrogate model's prediction often not accurate outside the offline data regime, potentially overestimating the outputs at sub-optimal input candidates. To mitigate this, existing offline optimizers have introduced numerous regularizing strategies for either the surrogate or the search models. For example, [4, 11, 12, 13] and [3] regularize their surrogate models so that their predictions at inputs outside the offline data regime are pushed down or pushed towards a constant value. Alternatively, [1] and [2] restrict their search models to regions having certain domain-specific properties under which sampled inputs probably have high performance.
These strategies therefore depend on the specifics of either the surrogate- or the search procedures to characterize the out-of- of--isstibuution regions or the desirable domain- speciffc properties. Consequently, regions which are not sufficiently specified. For example, COMs [13] uses an ad-hoc specification that characterizes the out-o--distribution region in terms of inputs that are reached during the first few iterations of gradient ascent on an un-regularized surrogate. This only characterizes a local sub-region of broader out-of -iittrr-iitiitttoon data regime.
their regularization might not extend to
To mitigate such limitations in the regularizing behaviors of prior work, we instead aim to investigate a more generic approach that is independent of the specifics of the search or surrogate procedures. For instance, instead of regularizing the behavior of the model on certain input regions, we could impose constraints on the geometries of its loss landscape such as requiring it to be in a parameter regime that uniformly produces low loss values [14]. Such regularizing strategies can, therefore, be incorporated into existing offline optimizers as an additional regularizer to improve their performance. To substantiate this idea, we have made the following technical contributions:
1. We develop a model-agnostic regularizer (for surrogate training) based on a notion of model sharpness This is characterized in terms of the surrogate's s maximum output change under low- energy parameter perturbation (i.e., norm-bound perturbation) (Section 3.1). Intuitively, suppose a surrogate's 's prediction does not change substantially within a perturbation neighborhood (i.e., via adding perturbation to its parameters) that contains the oracle, its predictions are likely to be close to those of the oracle (see Fig. 1). As such, minimizing this sharpness measure can help suppress the erratic behavior of the surrogate model at
input.
2. We adopt practical approximation that interestingly reduces the above surrogate sharpness measurement to a function of the surrogate's gradient norm. The surrogate training can then be augmented into constrained optimization task whose constraint imposes user-specified threshold on the surrogate's 's gradient norm. We can then solve it to acquire the optimally regularized surrogate using existing constrained optimization solvers. The high-level pseudo-code of our proposed algorithms which incorporate surrogate gradient norms to improve existing offline optimization techniques (IGNITE) is detailed in Algorithm (Section 3.2).
3. We develop a detailed theoretical analysis to show that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our analysis extends existing theories [14] from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate on training data, providing new perspective on sharpness regularization (Section 4).
We demonstrate empirically that incorporating the proposed model-agnostic regularizer into existing offline optimizers as an additional conditioning component often results in significant improvement over existing offline optimizers in most cases. This sets the first step towards a new, synergistic research direction in offline optimization that can potentially support and complement both existing and future work (Section 5).
For interested readers, a concise review of existing literature is also provided in Section 2.
2 Problem Definition and Related Works
In this section, we will concisely review the preliminaries of offline optimization. Section 2.1 provides a mathematical formulation of offline optimization, and Section 2.2 summarizes prior works.
"Our sharpness notion is different from that of [14] which is applied to the training loss. Instead, our sharpness measure is accessed on the surrogate prediction. Empirically, we also notice that surrogate sharpness often appears to be more effective than loss sharpness in boosting the offline optimization performance (Section 5.3).

---

2.1 Problem Definition
Offline optimization a computational approach to variety of material engineering tasks that aim to find a material construction or design that maximizes certain desirable properties. Mathematically, we assume that there an oracle function g(x) that maps from material design x E X to an overall utility g(x) of its property measurements; and we need to find its maxima:
arg max g(x) xEX
However, the key challenge here is that g(x) is inaccessible. Instead, we only have access to an offline dataset of observations D = {(x, =;)]== where 2; = g(x)), which denote the past input-output queries extracted from g(x) in previous experiments. A direct approach to this problem is to learn surrogate g(x;w,) of gxx) via fitting its parameter w. to the offline dataset,
A arg min mig (D(w) arg min Σ Ee E((xxx;)) 2:),
where w denotes a parameter candidate of the surrogate and (g((;; w), 2) denotes the prediction loss of g(::w) on its oracle output is z. The (oracle) maxima of g(x) then approximated via,
X. A
arg max g(x:w.).
XEX
Suppose the surrogate g(x; W.)'s prediction is sufficiently accurate over the entire input space, solving Eq. (3) is all we need. However, in most cases, g(x; w.) often predicts erratically outside the offline data regime, which in turn misleads the optimization towards sub-optimal candidates. To mitigate this, numerous surrogate or search regularizers have been proposed, as summarized next.
2.2 Related Works
Most existing offline optimization methods have focused on regularizing either the search or the (surrogate) training procedures. The main focus is to either (1) avoid exploring input regions where the surrogate's prediction is not reliable or (2) regularize the prediction behavior of the surrogate at out-of-disse of--istribution input regimes. For example, to regularize the surrogate's 's prediction, [13] uses input candidates found during the first few iterations of gradient updates on un-regularized models to characterize the dittr---iitribttion regime. The surrogate can then be re-trained with an additional regularizer that penalizes high-value predictions at those sampled input candidates.
Alternatiell.. [11] maximizes the normalized data likelihood to reduce prediction uncertainty, which also helps suppress erratic prediction out-of-distribution regime. Existing techniques in model pre-training and adaptation [3] or transfer learning via co-teaching [15] can also be leveraged to enforce criteria of local smoothness that encodes a preference for conservative prediction to avoid overestimating the oracle output. Otherwise, to regularize the search procedure, [1] and [2] focus instead on learning a generative model of input candidate conditioned their oracle performance. Input candidates that likely achieve high performance can then be synthesized via conditioning the generative procedure on high-value oracle output. [1] characterizes such conditioned distribution via an adversarial zero-sum game. [12] learns direct inverse mapping from the performance output to the input design using conditional generative adversarial network [16].
Despite their reported successes, these approaches are still limited by their ad-hoc characterization of the -uttt--distribution regime. As discussed previously in Section I, the existing characterization of out-of -ff-o---iitriuttionn input is often based on the specifics of either the surrogate or the search procedures, which are not guaranteed to sufficiently characterize the entire out-o--distribution data regime. This motivates us to develop more generic out-of-distributin to both the search and surrogate models. Such an approach can be readily incorporated into most existing offline optimizers to boost their performance (Section 5).
characterization (Section 3) that is external
3 Surrogate Regularization with Sharpness Constraint
This section introduces an additional constraint that transforms the original surrogate optimization in Eq. (2) (Section 3.1) into constrained optimization problem (COP). The constraint imposes
3

---

user-specified pouer-bound on the sharpness of the surrogate. Our proposed formulation draws inspiration from prior work [14] that aims minimize the sharpness of the loss function to improve generalization. However, unlike the original work in [14], where the sharpness concept is applied to the loss function, we adapt for the surrogate's prediction. We find more suitable since offline optimization's search procedure operates on the surrogate landscape instead of the loss landscape. Moreover, while minimizing loss sharpness [14] can ensure low error for single predictions in the OOD regime, errors may accumulate over consecutive predictions in insight Section 3.1 (Fig. 1) suggests that such error accumulation can be mitigated by keeping the surrogate sharpness small during training. Furthermore, show that the sharpness can be practically approximated in terms of the surrogate's 's gradient norm, which is more tractable. This allows for direct adoption of existing constrained optimization algorithms [17] to effectively solve for the desired optimally regularized surrogate (Section 3.2).
meddient-based search. Our
perturbation neighborhood ony perturbation neighborhood
optimal solutions
CAs
perturbed model
Surogate
41
Function value
Fundiion value
(y at 8)) (xx
Ba 8.)
Surragate Sharpness
(w()) max [ever ex 5) Raex (g( ai]]
Surragate model
Oradle function
Surrogate model
Data
(a)
(b)
Figure I: (a) Illustration of surrogate sharpness; (b) Illustration surrogate sharpness- based offline optimization: Consider two surrogate parameters wi and W2 where w has smaller sharpness than W2. This means the predictions of the models in the perturbation neighborhood than those of the models in the perturbation neighborhood of w2. As such, if both neighborhoods contain the oracle, the prediction error d1 w1 is potentially smaller than the prediction error d2 of w2. Consequently, the optimal value of g(x: w1) is closer to the oracle optimal value than g(x; w2) )'s.
w1 will vary less
3.1 Surrogate Sharpness
Suppose the oracle lies within the parametric family of the surrogate, there must exist perturbation neighborhood of the surrogate's parameters that contains the oracle. That is, the oracle can be obtained by adding to the surrogate's parameters noise vector in this neighborhood. Now, suppose the predictions do not change substantially across the models (including both the oracle and surrogate) the perturbation neighborhood; the surrogate's predictions (and optimizers) must be close to those of the oracle (see Fig. 1). Motivated by this insight, we consider potential approach to mitigate the erratic prediction of the surrogate, which is to ensure that its worst-case prediction change across the perturbation neighborhood is sufficiently small. This is formalized below:
(w)
max ||
E
[(x: 6)]
XEX [
(4)
S
where || |/2 denotes the f2-norm, and p > bounds the maximum norm of the perturbation, which defines the perturbation neighborhood and be selected via prppe-paraeeter tuming (see Section 5. and Appendix G.4). To case the notation, we will omit the subseript and use ||.]] to consistently denote the E2 norm in the rest of this manuscript. We will also refer to Rx(w) in Eq. (4) as the generalized surrogate sharpness, which can used to regularize surrogate training.
arg min (D(w) s.t. Rx(w) S E
w
where e' is user-specified threshold. Solving Eq. (5) is non-trivial since Rx(w) is neither tractable nor differentiable. To sidestep this, we leverage a theoretical result that will established later in Section 4, which (informally) states that with high confidence, the generalized surrogate sharpness

---

Rx(w) can be approximated with its empirical estimate RD(w) in Eq. (16),
(v dim(w) log (n]lw]l?2)
Rx(w) & (DG(w) + 2 (Ro(w) + O
(6)
n
where G(w) and Xmax denote the norm of the expected (parameter) gradient the surrogate w, and the largest eigenvalue of the (parameter) Hessian of the surrogate' s expected prediction,
G(w)
-
E XEX
and
Amax
max Amax w
v E XEX
ooxx)w)
(7)
When n is sufficiently large, (dim(w) 3l) will be negligibly small. Then, suppose within the search region for w (see Assumption 2), G(w) is bounded by constant G, we can enforce (w) I e' (pG + (1/2)p² (maax) e via constraining instead RD(w) S.
w.
arg min LD(w) s.t. RD(w) S 6
(8)
To mitigate the non-differentiabiiiy (D(w), we further propose a practical approximation which reduces R.p (w) to function of the surrogate's gradient norm [Vwg(x; w|]l, which is differentiable, allowing the above COP to be solved effectively with existing constrained optimization algorithms. This is discussed in the next section.
Remark. Eq. (4) is similar in spirit the notion of loss sharpness [14] in which the same perturbation
model is used to characterize the sharpness or sensitivity of the loss function, |C (w + 6) - LD(W), small changes || S p in the model weights w. Our work, however, sets direct focus the sensitivity or sharpness of the surrogate's prediction - Eq. (4) which results regularizer, leading to better empirical performance (see Section 5.3). It also requires significant and non-trivial adaptation of the theories presented in [14], as detailed later in Section 4.
better surrogate
3.2 Practical Algorithms
Let hiw + 8) a FxxDlg(x;; w + 8)) and h(w) - (x; w)). The surrogate sharpness can be approximated via the first-order Taylor expansion of hiw + 8) at wt
Ro(w) = max
E XED
5)} - E [g(x;
- XED
[ (x; w)]
1811.5
max |
hiww + h(w)
max S
Vwdi(w)
(9)
Using the selected to make the equality happens,
inequality on the right-hand side of the above and noting that 5 can be
RD(w) ~
max م
V Vwh(w)
= max SSP
\w/(w>|] x ||o| x //
(10)
Using the above approximation, the COP Eq. (8) can be rewritten as
w. arg min (w(w) s.t. p- ||Whh(w)] S e.
(11)
which can be solved via optimizing the corresponding Lagrangian,
arg min L(w, A) where A(w,A) & Co(w) + X (p -c) (12)
with A denotes the Lagrange multiplier. This can be solved via our approach below.
IGNITE, We can optimize for both À and w using the basic differential multiplier method (BDMM) [17], which simultaneously gradient ascent for À and gradient descent for w, resulting in the following update rules:
Wt1 w nw (VeLo(w") + x..p- V / va/)
(13)
X+1
N + ax (p. Vahh (w)) -
(14)
5

---

Algorithm IGNITE
Input: offiine data D = {(x,, 2, h- initial surrogate g(x; () no, iterations T; batch size m; Lagrange multiplier X; perturbation radius and scalar r; step sizes and yxi threshold c.
I: Initialize ( < w and ( +
7: Compute = w(() + r. 92/ll9all
2: for t t 1: T do
8:
Compute Ia =
2im V
3: Sample B = {(x,, k:)
1= D
9:
Compute ge)) = 91 + 1 pr pr 92))
4: Compute & g(x;; (() for g [m]
10: Update w (1+1) -- w Rag
5: Compute 91 m
_= Vul(a;
11: Update ((++1) 1 + (ellezall -e)
6:
Compute 922 = m -1
Σ ==1 Vwii
m
12: end for
13: return learned surrogate (7+1)
where we represents the surrogate's parameter estimate iteration 6, X is the Lagrange multiplier estimate at iteration i, The is the step size for updating w, and 7Ix is the step size for updating X. We name this method IGNITE. We also conduct grid search to select the optimal value for p, as mentioned in Section 5.1.
As alternative approach, we can also treat À as nyper-parameter and optimize for using gradient descent; we call this method IGNITE-2. . The detailed algorithms, paramtr selection, and experimental results of IGNITE-2 are reported in Appendix G.2.
Remark.
Both of the above methods require differentiating [(w]] with respect to w, which involves computing the expensive Hessian of h(w). Fortunately, this expensive computation can be avoided by using the gradient approximation technique detailed in Appendix F. To summarize, we provide a complete pseudo-code of IGNITE in Algorithm I whose steps 3-8 implement the approximation in Eq. (84) and Eq. (85) of Appendix F.
4 Theoretical Analysis
In this section, we will provide detailed theoretical analysis to substantiate our earlier (informal)
statement in Eq. (6) that with high confidence, reducing the empirical sharpness RD (w) on the offline data will also reduce its generalized sharpness Rxx (w)) We will show that this is true (see Theorem 1)
under certain choices and mild assumptions of the surrogate model (see Assumptions and 2).
Assumption 1. The output of the above surrogate function g(x; ) is bounded within [0. 1].
Assumption 2.
Amin
V [lg((::))) 0 for all ||>||| < T for some 0.
We note that the specific bound within [0, 1] in Assumption is meant to ease the technical presen- tation of our theoretical analysis. Otherwise, it can be extended straightforwardly any bounded functions. Furthermore, we also show below that it is indeed possible to find non-trivial surrogate function that is bounded and satisfies Assumption
Theorem 1. There exists and w.,, and non-linear function r r(x; w)) such that,
g(x:w) A r(x;w4)+ + (w w+) T Vwr(x; + (ww w V2 vxr(x; w-) (15)
satisfies Assumption 2 and bounded on (w ||w||| S T). Detailed derivation of this theorem is deferred to Appendix A.
For such surrogate functions, their generalized sharpness can be upper-bound by a function of their empirical sharpness on the offline data. As detailed below, the bound depends on both the size of the surrogate dim(w) and the number n of offline data points.

---

Theorem 2. For any p > 0, m = dim(w) and 2/ (m)min) 2 o² > O with Amin being defined in Assumption 2, the following holds simultaneousty for all g( w) for which Assumption 2 met.
Rx(w)= S mmmmmin
2G(w)) Amaxp?
) (16)
1
|w01 mo?
log n +
-) + P(n, m, +(.
2
x RD(ww) +
m log 1
vn -
m
with probability at least a over the random choice of the offline dataset, and with P(n, m, mn,a) 2 log(n/a) + log(8n + 4m.. Detailed derivation of this theorem is deferred to Appendix E.
=
Proof Sketch, For clarity, we will provide below proof sketch of Theorem 2, which highlights the key steps in our derivation. Due to limited space, the specific of each step is deferred to the Appendix E. First, we note that
Rx(w) = max { Fx(w + 8) A Ex[g + 6)) - Ex(g(x;w)]]
(17)
Ro(w) = max |
FD(w + 5) A Eplg(( + 5) Exlg(x;
(18)
A relation between Rx(w) and (w can then be derived in three steps:
1. Upper-bound 188~N(0.0 (x 5)] with function of (B~Nro, (F 6)]. This can be achieved via a direct application of the PAC-Bayes bound [18] which views the perturbed model + 8 as random hypothesis sampled from the posterior N(w, o²1) see Appendix B.
Upper-bound ES [ + 8)] with a function of (w)) This can be achieved using a similar proving technique adopted from [14] see Appendix C.
3. Find { >
such that Rx (w) can be upper- upper-bounded bounded with E- E80.:00.0n|/xx(( 5)] where is small scaling factor. To achieve this, we will find the lower-bound for (E [Fx(w> 6)] using the Taylor remainder theorem to expand it around w, which can be lower-bounde using the minimum eigenvalue of its Hessian at w. Using the same approach, we can upper-bound Rx (w) with the maximum eigenvalue of the same Hessian see Appendix D.
Finally, we set & so that the upper- bound of Rx(wx) (w) is smaller than the multiplication of & with the lower-bound Fx (w + 5)]. To tighten the bound, we choose the smallest possible value of & such that the bound still holds. Lining up the results of the above steps then shows that Rx(w) can then be bounded with function of & (Dw See Appendix for complete derivation.
Remark. Note that Theorem 2 is more general than its informal statement in Eq. (6), which can be reproduced by choosing o² 2/ (mAmin) in Eq. (16) to make the bound tightest.
Experiments
This section evaluates the efficacy of our proposed method IGNITE in improving state-of-the-an offline optimizers. We describe our experiment settings in Section 5. and report detailed empirical results in Section 5.2. We also provide additional ablation studies of our method in Section 5.3.
5.1 Benchmarks, Baselines, and Evaluation
Benchmark Tasks. Our explorations focus on four real-world tasks from Design-Benchh [4], covering both discrete (TF-Bind-8 and TF-Bind-10) and continuous domains (Ant Morphology [19] and D'Kitty Morphology [20]).
Baselines. We meticulously curated diverse set of 11 widely acknowledged offline optimizers for comparative analysis. This ensemble comprises BO-qEI [4], CbAS [1], RoMA [3], ICT [15],
'W omit domains marked for their high inaccuracy and noise in oracle functions from prior works (ChEMBL, Hopper, and Superconductor), well those deemed excessively expensive evaluate (NAS).
7

---

Table 1: The percentage improvement in performance achieved by IGNITE across all tasks and baseline algorithms at the 100th percentile level is presented. Gain signifies the percentage gain over the baseline performance (Base).
Ant Morphology
D'Kitty Morphology
TF Bind 8
TF Bind 10
Algorithms
Performance
Gain
Performance
Gain
Performance
Gain
Performance
Gain
(best)
0.565
0.884
0.565
0.884
REINF-
Base
0.255 + 0.036
0.546 0.208
0.929 a 0.043
0.635 0.028
ORCE
IGNITE
0.282 = 0.021
+2.7%
0.642 = 0.160
+9.6%
0.944 = 0.030
+1.5%
0.670 = 0.060
+3.5%
GA
Base
0.303 = 0.027
0,881 + 0.016
0,980 + 0.016
0.651 + 0.033
IGNITE
0.320 # 0.044
+1.7%
0.886 0.017
+0.5%
0.985 a: 0.010
+0.5%
0.653 0.043
+0.2%
ENS-
Base
0.376 = 0.060
0.888 + 0.010
0,985 # 0.009
0.649 * 0.036
MEAN
IGNITE
0.435 # 0.058
+5.9%
0.896 + 0.013
+0.8%
0.987 # 0.007
+0.2%
0.662 # 0.091
+1.3%
ENS-
Base
0.385 0.067
0.889 0.014
0.980 + 0.012
0.681 0.095
MIN
IGNITE
0.468 0.062
+8.3%
0.897 0.010
+0.8%
0.986 = 0.010
+0.6%
0.705 0.118
+2.4%
Base
0.854 0.042
0.895 0.012
0,919 + 0.044
0.635 0.041
CbAS
IGNITE
0.859 0.039
+0.5%
0.900 0.015
+0.5%
0.921 + 0.042
+0.2%
0.652 0.055
+1.7%
Base
0.905 = 0.023
0,944 0.008
0.892 + 0.046
0.643 0.062
MINs
IGNITE
0,911 0.024
+0.6%
0.945 0.007
+0.1%
0.930 + 0.041
+3.8%
0.647 0.058
+0.4%
RoMA
Base
0.569 # 0.086
0.821 0.019
0.665 + 0.000
0.550 0.008
IGNITE
0.615 = 0.085
+4.6%
0.834 + 0.012
+1.3%
0.665 + 0.000
+0.0%
0.553 0.000
+0.3%
Base
0.897 = 0.031
0.931 + 0.013
0.955 + 0.030
0.645 # 0.038
COMs
IGNITE
0 * 0.030
+0.4%
0.934 + 0.010
+0.3%
0,952 + 0.043
-0.3%
0.638 0.053
-0.7%
CMA-ES
Base
1,955 = 1,484
0.724 + 0.002
0,928 + 0,040
0.668 0.035
IGNITE
1.957 # 1.910
+0.25
0.724 + 0.001
+0.0%
0.927 + 0.043
-0.1 1%
0.673 # 0.044
+0.5%
BO-qEI
Base
0.812 0.000
0.896 0.000
0.787 + 0.112
0.628 0.000
IGNITE
0.812 = 0.000
+0.0%
0.896 + 0.000
+0.0%
0.843 = 0.109
+5.6%
0.628 0.000
+0.0%
Rase
0.937 0.023
0.946 0.014
0.892 4 0.055
0.647 0.025
ICT
IGNITE
0.935 0.032
-0.2%
0.962 0.018 +1.6%
0.923 + 0.038
+3.1%
0.652 0.074 +0.5%
CMA-ES [21], COMs [13], MINs [12], REINFORCE [22], and three variations of gradient ascent (GA, ENS-MIN, ENS-MEAN), corresponding to vanilla gradient ascent, the min ensemble of gradient ascent, and the mean ensemble of gradient ascent, respectively.
To ensure a comprehensive assessment, follow the methodology in [4]. Each method generates 128 optimized design candidates evaluated by the oracle function. The candidates' performances are ranked, and the 50-th, 75-th, and 100-th percentile levels are recorded. All results are averaged over 16 independent runs to ensure reliability.
Evaluation Protocol.
Hyper-parameter Configuration. For each baseline algorithm, we carefully configure optimal hyper-parameters as outlined [4]. Our method IGNITE introduces five additional hyper-parameters: X, p, r. 'T,, and The hyper-parameter X, an initial value for the regularizer coefficient, is set to 0.01 through a grid search within {0.0001, 0.001, 0.01}. }. The hyper-parameters p and r are chosen from {0.01, 0.05, 0.1, 0.2). with P set to 0.05 and set to 0.05 for IGNITE. Additionally, IGNITE uses M le - 3 and 0.1, which are determined via the experiments in Section 5.3.
5.2 Results and Discussion
In this section, we presented the percentage improvement over baseline performance attained by IGNITE when it is applied to existing baselines. We have evaluated this at the 50-th, 80-th, and 100-th percentile levels. However, due to limited space, we only report results of the 100-th percentile level in the main text. The other results are instead deferred to Appendix G.3.
The first two columns of Table 1 show that out of 22 cases involving 11 baseline algorithms across 2 tasks, the IGNITE regularizer enhances baseline performance in 18 cases, with improvements reaching up to 9.6%. In only out of 22 instances IGNITE slightly decreases performance by 0.2%, which is negligible. Even in cases where performance is not improved, IGNITE reduces the variance results from 0.2% to 0.1% % for the CMA-ES baseline on the D D'Kitty Morphology task. Additionally, IGNITE helps establish new offte-of-the-ar (SOTA)
Results on Continuous Tasks:
8

---

Table 2: Percentage improvement over the baseline of IGNITE, SAM [14], and LI, L2 regularization across all tasks.
GALANT
Improvement (%)
TF10
Improvement (%)
GALTF10 COMS_ANT ANT
COMS, ANT
COMe TF10
COMP_TPP TF10
TF Bind
TF
-1
Algorithms
Ant
D'Kitty
Bind 10
IGNITE
2.7%
9.6%
1.5%
3.5%
REINE- ORCE
SAM
1.1%
7.9%
1.1%
0.2%
0.01 0.10
0.20
0.30
0.40
0.50
10-5
10-4
TTA.
10-3
16-2
L.-Reg
L2-Reg.
1.0%
1.0%
4.2%
5.2%
0.9%
1.0%
0.1%
0.3%
IGNITE
1.7%
0,5%
0.5%
0.2%
Figure 2: The percentage improvement in perfor- mance achieved by IGNITE across different algorithms (COMS and GA) and tasks (ANT and TF10) in the changes of (a) threshold e and (b) step size 7/a-
GA
SAM
0.7%%
-1.3%
0.2%
1.1%
L1-Reg.
1.0%
0.0%
0.1%
-0.8%
L.2 Reg.
1.1%
-0.7%
0.2%
-0.4%
performances in both tasks. For example, in the Ant Morphology task, it raises the SOTA baseline CMA-ES from 195.5% to 195.7%. the D'Kitty Morphology task, IGNITE achieves new SOTA of 96.2% with the ICT baseline.
Results Discrete Tasks: The last columns of Table 1 show the impact of the IGNITE reg- ularizer on the performance of baseline algorithms in two discrete domains (TF-BIND-8 and TF- BIND-10). Similar the continuous tasks, IGNITE significantly enhances baseline performance in most cases (17 out of 22), with improvements of up to 5.6%. There are only 3 instances where minor performance decrease of up 0.7%, which negligible. Ad-
integrating IGNITE results ditionally, in certain instances, IGNITE not only improves baseline performance but also establishes new state-of-thh -ace--of-hhhhaaeee (SOTA) results. For example, on TF-BIND-8 and TF-BIND-10, the original SOTA performances of 98.5% and 6% achieved by ENS-MEAN and ENS-MIN, respectively, are elevated to 98. 7% and 70.5% with the addition of IGNITE setting new SOTA records.
In summary, IGNITE consistently maintains a high probability of 91 % (40 out of 44) of not degrading baseline performance. There is a high likelihood (79.55% 35 performance, with an average improvement of approximately 1.91% and notable peak improvement of 9.6%. Conversely, IGNITE also exhibits relatively low probability (9.09% 4 of 44 cases) of decreasing performance, with an average degradation of approximately 0.3% and a minor peak degradation 0. 0.7%..ddiiionally. there minor probability (11.36% 5 out of 44 cases) of maintaining baseline performance.
44 cases) of improving baseline
5.3 Ablation Experiments
In this section, we conduct additional experiments to assess the sensitivity of two representative baselines, COMs and GA, when regularized with IGNITE, to variations in hyper-parametert c and 7x. Additionally, we perform experiments to compare the efficacy of IGNITE with other commonly used regularization methods.
Changing threshold We assess the performance enhancement of COMs and GA when regularized with IGNITE using various values of from the set (0.01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5). A high e value may result in surrogate that is overly sharp, potentially hampering the search process and hindering the discovery of optimal designs. Figure 2(a) demonstrates that excessively high e values lead to negative improvements. Conversely, excessively low values may cause the regularizer to dominate the original loss, resulting in a surrogate that is not well-fitted to the offline data. Negative improvements are observed with 0.01 and 0.05 in Figure 2(a). As result, we determine 0.1 as the optimal value based on these observations.
Changing step size 7/A- The step size 7/\ controls the rate at which X is updated during the optimization process. It's essential to choose an appropriate step size, avoiding being either too large or too small. Figure 2(b) demonstrates that 77x value of 1e -3 yields optimal results.

---

Table 3: Comparison of the surrogate sharp-
Table 4: The percentage improvement in performance achieved by IGNITE at the 100th percentile level for Superconductor and Chembl tasks.
ness - approximately | in Eq. 10 with and without IGNITE. These sur- rogate sharpness values were computed on unseen data, which are design candidates found by the GA and REINFORCE before and after being equipped with IGNITE
Algorithms
Performance
REINFORCE
0.471 = 0.011
Super- con-
REINFORCE + IGNITE
0.492 = 0.015 (+2.1%)
ductor
GA
0.514 + 0.021
Algorithms
Ant
TF Bind 10
GA + IGNITE
0.517 = 0.011 (+0.3%)
REINFORCE
0.18
0.24
REINFORCE
0.634 = 0.001
REINFORCE + IGNITE
0.09
0.14
REINFORCE IGNITE
0.636 + 0.008 (+0.2%)
Chembl
GA
1.88
1.07
GA
0.635 + 0.005
GA + IGNITE
1.69
0.63
GA + IGNITE
0.640 + 0.009 (+0.5%)
Comparing IGNITE with other regularization methods. We conduct a comparative experiment to assess the performance improvement achieved by using the IGNITE regularizer compared to other regularizers, including LI, L2, and SAM [14] (where SAM is considered as loss sharpness regularization with a coefficient equal to 1). Table 2 presents the results obtained on two baseline algorithms, REINFORCE and GA, across four tasks. Overall, IGNITE outperforms the other regularizers in most cases, with the largest difference compared to the second best being 3.2% with REINFORCE on the TF-BIND-10 task. Additionally. IGNITE achieves positive improvements in all cases. Conversely, while the simple regularizers L1 and I.2 help boost performance with REINFORCE, they lead to performance degradation with GA. SAM outperforms IGNITE with the GA baseline on the TF-BIND-10 task and shows notable improvement with REINFORCE on the D'Kitty task, though its integration with GA leads to a performance drop. Furthermore, we show that IGNITE also outperforms SAM when integrating with CbAS and BO-qEI in Appendix H.
Surrogate sharpness on unseen data before and after using IGNITE. We also conduct an experiment measuring the sharpness of the surrogate model, approximated by VllV.. as defined in Eq. (10), with and without IGNITE. This sharpness is computed on unseen data points which are, specifically, the design candidates generated by the GA and REINFORCE baselines. By testing on these unseen candidates, we simulate the out-of-distribution (OOD) conditions that are critical in assessing generalization in optimization tasks. Table 3 reports these surrogate sharpness measurements for some baselines with and without using the IGNITE regularizer. The results demonstrate consistently that the IGNITE regularizer helps reduce the surrogate sharpness on unseen data, as anticipated. This reduction indicates that IGNITE is effective in smoothing the surrogate model's 's landscape, leading to better and more stable generalized performance.
Results on tasks with noisy data. To demonstrate IGNITE's s robust performance in scenarios with noisy oracle, we conducted additional experiments using the GA and REINFORCE baselines on two benchmark tasks with particularly noisy oracles: Superconductor and Chembl. For each baseline, we compared its achieved performance with and without the regularizing effect of IGNITE, as shown in Table 4. Across both tasks, IGNITE helps improve the baseline performance substantially, achieving up to 2. % increase for REINFORCE on the Superconductor task, highlighting IGNITE's ability to enhance performance robustness even settings with noisy oracles.
Conclusion
This paper introduces the concept of generalized surrogate sharpness in offline optimization, resulting in the development a new regularization technique, IGNITE. Our theoretical analysis demonstrates that reducing surrogate sharpness unseen data. Empirically. IGNITE consistently maintains a high probability (91%) of not degrading baseline performance and 79.55% likelihood of improving it, with a peak improvement of 9.6%. Additionally, we believe that our novel technique can be adapted to related domains such as robust optimization (RO) and reinforcement learning (RL), suggesting potential future research directions.
offline dataset provably decreases its generalized sharpness on
A Acknowledgments and Disclosure of Funding
This work was funded by Vingroup Joint Stock Company (Vingroup JSC), Vingroup, and supported by Vingroup Innovation Foundation (VINIF) under project code VINIF.2021 D.000128...
10

---

References
[1] David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference machine learning, pages 773-782. PMLR, 2019.
[2] Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model-based design. Advances in Neural Information Processing Systems, 33: 13:12945-12956 2020.
[3] Sihyun Yu, Sungsoo Ahn, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for offline model-based optimization. Advances in Neural Information Processing Systems, 34:4619-4631. 2021.
[4] Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-bench: Bench- marks for data-driven offline model-based optimization. In International Conference on Machine Learning, pages 21658-21676 PMLR, 2022.
[5] Manh Cuong Dao, Phi Le Nguyen, Thao Nguyen Truong, and Trong Nghia Hoang. Boosting offline optimizers with surrogate sensitivity. In Forty-first International Conference on Machine Learning, 2024.
[6] Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for box optimization. arXiv preprint arXiv:2306.07/88, 2023.
[7] Siddarth Krishnamoorthyy Satvik Mehul Mashkaria, and Aditya Grover. Generative pretraining for black-box optimization. arXiv preprint arXiv:2206. 10786, 2022.
[8] Tung Nguyen, Sudhanshu Agrawal, and Aditya Grover. Expt: Synthetic pretraining for few-shot experimental design. Advances in Neural Information Processing Systems, 36:45856-45866 2023.
[9] Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, and Janardhan Rao Doppa. Offline model-based optimization via policy-guided gradient search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11239, 2024.
[10] Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, and Trong Nghia Hoang. Learning surrogates for offline black-box optimization via gradient matching. In Forty-first International Conference Machine Learning. 2024.
[11] Justin Fu and Sergey Levine. Offline model-based optimization via normalized maximum likelihood estimation. arXiv preprint arXiv:2102.07991 2021.
[12] Aviral Kumar and Sergey Levine. Model inversion networks for model-based optimization. Advances in Neural Information Processing Systems, 33:5126-5137, 2020.
[13] Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In International Conference on Machine Learning, pages 10358-10368. PMLR, 2021.
[14] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware mini- mization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020.
[15] Ye Yuan, Can Chen, Zixuan Liu, Willie Neiswanger, and Xue Liu. Importance-aware co- teaching for offline model-based optimization. arXiv preprint arXiv:2309 11600, 2023.
[16] Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint 141 2014.
[17] John Platt and Alan Barr. Constrained differential optimization. Neural Information Process- ing Systems, 1987.
[18] David McAllester. Pac-bayesian model averaging. In Proceedings of the twelfth annual conference on Computational learning theory, pages 164-170, 1999.

---

[19] Greg Brockman, Vicki Cheung, Ludwig Pettersson. Jonas Schneider, John Schulman, Jie Tang. and Wojeiech Zaremba. Openai gym. arXiv preprint arXi::1606.01540 2016.
[20] Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, and Vikash Kumar. Robel: Robotics benchmarks for learning with low-cost robots. In Confer- ence on robot learning, pages 1300-1313. PMLR, 2020.
[21] Nikolaus Hansen. The ema evolution strategy: a comparing review. Towards a new evolutionary computation: Advances in the estimation of distribution algorithms, pages 75-102.
[22] Ronald J Williams. Simple statistical gralient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8:229-256, 1992.
[23] D. Kingma and M. Welling. Auto-Encoding Variational Bayes. In Proc. ICLR, 2013.
[24] John Langford and Rich Caruana. (not) bounding the true enror. Advances in Neural Information Processing Systems, 14, 2001.
[25] Beatrice Laurent and Pascal Massart. Adaptive estimation of a quadratic functional by model selection. Annals of statistics, pages 1.02-138, 2000.
[26] Yang Zhao, Hao Zhang, and Xiuyuan Hu. Penalizing gradient norm for efficiently improving generalization in deep learning. In International Conference on Machine Learning, pages 26982-26992. PMLR, 2022.
[27] Barak A Pearlmutter. Fast exact multiplication by the hessian, Neural computation, 6(1):147- 160, 1994.
12

---

Proofs of Theoretical Results and Additional Experimental Results
A Proof of Theorem 1
Part I. Minimum Eigenvalue of Parameter Hessian is Positive.
First, we re-state the choice of our surrogate in Theorem I,
(): +(x;++)+ (w- w+) [xx(x;++) 1 7/(w - ww+ T Vär (w w+) (19)
Regardless of the choice of rxx;w) w), differentiating g(x; w) with respect yields,
(xx(x;w)
V (xx(x;++))
(20)
which implies the parameter Hessian of (x)() is always a constant matrix depending on the specific choice of r(x; w) and reference point w.. Hence, follows trivially that,
Amin
V (x)():))
Amin (x)(x(+)))
(21)
Therefore, if we can find r(x; w) such that its parameter Hessian is strictly positive definite at 4 the parameter Hessian of g(x; w) will always be strictly positive definite regardless of w. This means g(x; w) will be Amin -ttinnltrongly convex in w with Amin O. As result, its expectation E[g(x; w)] will also be Amin-Strongee convex with Amin
or equivalently, Assumption 2 holds.
This can be done by choosing r(x; w) to be any quantile prediction of a linear regressor with a (pre-trained) random feature map. For example,
Es [w [x)]xxx] + ? T V 2b(x)]
r(x:w)
(22)
2
where x(xx N ( diagivvx)))
(23)
where V[.] denote the variance function. We can interpret the random feature vb(x) is sampled from a pre-trained VAE [23] with two deep neural nets characterizing the mean and variance functions, m(x) and (x). These functions can be pre-trained and post-processed in whatever ways we need to control their value ranges. Their parameters are then frozen when we fit r(x; w) with respect to w.
To understand the behavior of r(x; we first note below the closed-form expression of Eq. (23),
r(x;w) w mm + 7 2 (w T Aw)
(24)
where A A diag[v(x))) Differentiating twice both sides of Eq. (24) w+, we have
Vir(;:w)) A
3 (w (w. 2
a A (wT[A..) I-
(25)
We can now choose wi (1/m) -i=1 e; where e1, ..,..... em are the eigenvectors of A. Since A diagonal, these are also the one-hot vectors and their corresponding eigenvalues are the entries on
the diagonal of A i.e., Aa [A]a. With this choice, closed-form expression of V2 Vār(x; w+) in terms of the entries of A can be derived as follow,
3
v3r(x;44)
?. 1
Σ
AA
AB
(26)
where is another diagonal matrix with entries [B] m "E'
[A]a). As such, is trivial to see that if we choose the activation unit (i.e., sigmoid or ReLU) for v(x) such that its output positive, then the entries of both A and B are positive.
[A],
Finally, note that Eq. (26) is a scaled matrix product of two diagonal matrices, A and B, which will result in diagonal matrix. Its entries will be positive if the entries of A and B are positive, as enforced above. This means the minimum eigenvalue of rxr(x; is positive as desired. C]]
Part II. Boundedness on {w =|] <)..
13

---

Rearranging terms in the expression of g(x; w) in Eq. (19),
g(x; w) -
(vr((x V (((x) + Vw + const. (27)
where const absorbs all constant vectors (i.e., not dependent on w). Hence, taking absolute values for both sides of Eq. (27), we obtain
[x(x:w)
3 (wwr(xx - Vig V3r(x50)))u)) + 2
1 T
Vy Vur(x;4)) const
(28)
I ||||| [wx((=+))
)w + 1 Var(x )um
+ ||const]
(29)
( Amaxx (V (x:
|const||
+
||o|| + |cconst||
(30)
|const||
Amax (+r(xx:w+))
72
ccontt|| const
(31)
where Eq. (30) follows from the fact that V r(x: w is positive definite by construction. Eq. (31)
follows because (1 (12)))mxxVV3(xx) rxr(( w }) is fixed value that does not change when changes.
Thus, g(x; w) is bounded as expected. L
B Upper-Bound EFEN(0.=1)|xx( + 6)] with 1FD + 8)]
Lemma 3. Let define
Fx(w 5)
E [g (x; + 5)] IxEd
E (xx
(32)
x
E xED
[g
x: w + 8)]
xxD
E
[g w)]l
(33)
xED
The following holds simultaneously for all w, 0²
> 0, E (0, 1) and m = dim(w).
E [px(w +5] I
E 2~
[F + +
(34)
0~1(0.022)
mm
[wll [1+
log a a + log(Sn + 4m)
ma
n2
1
with probability at least 1 - over the (random) choice of the offline dataset D.
random hypothesis sampled from hypothesis distribution (such as the zero-mean Gaussian N(0, o o²I)) and (w + 5) as its incurred loss. In this view, for any hypothesis distribution Q. BB- [ + 5)] and Es- [FD(w + 5)] denote the corresponding generalized and empirical Gibbs losses whose relationship is characterized via the following classical PAC-Bayesian [18] bound,
Proof. We view the random perturbation
(L( + +og())))))
Fx + +0] 8~
E
<
FD + 5)] 8~2
E
+
(35)
2(n 1)
which will hold simultaneously for all choices of w, P and Q. Historically, in probabilistic learning context, P is often referred to prior or reference distribution which, for example, encodes domain- specific information about certain properties of the solution distribution. Then, Q the posterior, which can be selected tighten the bound (hence, reducing the generalized loss).
In our specific context, we will simply use the choices of P and Q as technical vehicles to tighten the gap between E Fx + 5)] and [E[D + 5)] which will contribute to the gap between our generalized and empirical surrogate sharpness.
To derive the optimal choices P and Q, we will leverage part of the proof from [14], which starts with the following fact:
m log
+
mo
[
[|wl)]] ma²
+ 1/4 + 1/2 log log(8n + 4m)
(36)
4n
7
14

---

This means if |||||| > (exp((4n/m)
1), then
(1 4n
|| mo?
m log
(37)
which means the bound gap in Eq. (34) is larger than 1 and under Assumption that the surrogate's output - hence, Fx(w) (w)'s is bounded in [o, 1J, Eq. (34) holds trivially.
Hence, to choose P and 2. we make the assumption that ||w||| I p² (exp((An//)) - 1)). Now, let us choose both prior and posterior distribution to be Gaussians, P = N(Hp, I and Q = N(u 2Z Their KL divergence then be computed in closed-form:
mo + 1|pp Mell m mog
2
KL(O)IP)
op
(38)
o
02
Naively, we can minimize the above KL divergence via setting the derivative of KL with respect to Op zero, and solving for the optimal op, which gives op" = + m-
M ll..
However, for the PAC-Bayes bound hold, ap must be chosen independent of the rest, so the above approach does not work. Instead, we can define in advance prior set of ap and then choose the best one in this set, following the proof in [24]. We can choose this set as follow:
Given fixed a, > 0, let = fc- (1-p((1m)mm) E N} be predefined set for oip If the above bound holds for o then all above bounds hold simultaneously with probability at least - Ei=1 60m -2 = a.
exp((1 -)/m) with probability 1 ag with oş 60m 2 for any i N,
Now, let 0e = G. Hg w. Hp = O, we have
oe Up 1el12
2 /m o² + 1w/g/m & 0² exp(4n/m))
(39)
Choosing i [1 m log ( + 1|/ m) o² o2 (1+exp(4n/m)) and oz ceepp(( - (1-xp((11))mmm
/m))
-m log((o² + ||||2// m) /n) 5
S - 51-m +|||||1 |
(40)
(0² + (wll//m)/e/ m
exp((1 - i)/mm
S exp(1/m) x (o
/m/c
(41)
-² + || [wll//m
ex exp((11 i)/m)
s exp(l/m) x (o=
(42)
o² + 1wll/mm
of
I cxp(1/m) x (o
(43)
Plugging Eq. (43) and Eq. (39) into Eq. (38),
1
mog + 1| Mell3
op σe
2
KL(OIIP)
+ m mog
(44)
2
o
1
m (o² || ll//m)
m + m log
exp (1)
[
w
m
(45)
2
0² +
| 11 //m
σ²
exp (1) x (02 + ||
1
m mog
(46)
2
o²
1
(w113) mo?
1 + log
1
(47)
2
Thus, Eq, (35) holds for each prior (indexed by i) in the above set with probability ag Hence, Eq. (35) holds with all prior choices with probability - E, ag 1 - This allows to pick the prior that leads to the tightest bound gap, which is indexed with i = [1 - m log(( 2 + llwl18/mm/c)) For this choice, we have
KL + log "
E
[x + +6] g
<
E
[F + 8)]
+
(48)
8~
2(n 1)
15

---

Plugging Eq. (47) into Eq. (48),
[x 6~
8)] _
E
[Fw + 8)]
(49)
o0,(00,221)
m
1
[w
mo²
+ 1
+
1 2
log a
Y
Finally, to remove the dependence on i in the above bound, note that
n
n
log
log
log
(50)
04 0
n² m2 m logg((o² + 1-1lw///mm /
log
log
(51)
a
n m log (c/(0² + | (w13/m))
log "
+ log
(52)
3
2 m2 log 2 (c/o²)
log
log
(53)
a
3
"1
π T'm² log"
+
log
log
(54)
3
m2m² + 4n/m)²
log +log
(55)
a log n +2oog v3
3
a(2m 4n)
(56)
a
log a
log(8n 4m)
(57)
Plugging Eq. (57) into Eq. (49),
E
[F
5)]
s E 8~~(00,02)
[Fo(w + 5)]
(58)
2
mloo
(1+
[[lli]] mo?
+ +
log + log(8n + 4m)
72
which completes our proof. C
C Upper-Bound ES + 8)] with RD(w)
Lemma Following the same setup Lemma 3, the following holds simultaneously for all w.
x² > O, E (0, 1) and m = dim(w),
3) [Fx(w 8)] <
E
max [F + +)]
(59)
|
m log (1
||
+ 2 log
+ 4 log(8n 4m)
mo?
n2
with probability at least a over the (random) choice of the offiine dataset D.
Proof. To derive this result, need to upper- upper-bound bound [[Fx(w)) with Fo(w). First, we note that (0, so | follows a chi-square distribution. From Lemma in [25],
(1111 mo² 2 20 2mt + 210²)
exp(-I)
VI 00
(60)
Therefore, with probability 1 1/V2 we have:
2
< o2m 1 log(n) m
| < 0² 22log (Vn) m + 2 m log (Vn)
S p²_ (61)
16

---

This means with probability 1 1-1/yn, we have E B ( S |ax FD 6). Otherwise, with the remaining 1/ / chance, E[Fr(w 6)] s under Assumption 1. Putting this together,
(0, FD(w + +0]]
- (1 [
)1-
max [F + + [
1 Vn
(62)
Finally, plugging Eq. (62) into Eq. (34),
E
Fx(w + +6) s
Vnl
max FD(w + +0) ||01|2SP
1
n
tm log
1
[[wll3]
1 +
log(n)
2
+ 2 log + +log(8n + 4m)
1
max Fo(w + 6)
8 S
m log
1
[
+
Iog(n) 2
+ log + 4loog(8n + 4m)
mor
which completes our proof.
D Upper-Bound (w with a scaled value of (n) + 8)]
Lemma 5, Following the same setup in Lemma 3, the below holds for all o² E (0,2/(m)minn)
max Fx(w + 6) < |
1 1 1 02 Amin
(2G(W) + )mmas?) (0
Fx(w + +)]
(63)
with G(w) and Amax defined previously in Eq. (7), m dim(r) and Amin defined in Assumption 2.
Proof. Let h(w) A Blg(x; where the expectation is over x E X, have
max Fx 5) = max [01LSP
M + 8) - -h(w)l
(64)
| Sp
max |
Vwh(w) To + - 5 (())
(65)
max [wh(ww T8|+ 8 vo)()) |
(66)
1 ブ 1011 Amax
|wh(w)]] 1|611 + Sp
max
(67)
1 + 2 p >maxx = G(w)p + Ammaxx
\wh(w))l
(68)
with & w + c 8 with some constant e E [0, 1]. In the above, Eq. (65) follows from the Taylor's remainder theorem for multivariate function. Eq. (67) follows from the Cauchy-Schwartz inequality (for the first term) and the definition of Xmax as the upper-bound on the largest eigenvalue of the parameter Hessian of h(w). Note that Amax > Amin > under Assumption 2 which stipulates that the smallest eigenvalue of the parameter Hessian of h(w) is always positive.
17

---

On the other hand, we also:
Px(w+0]]
hlw + 8) - -h(w)
(69)
(0,
6~N(00,021)
B 6 (0,
V (w)(w)) To + 26 0 VV
(70)
2
E
T V [o)(6)
(71)
2 Amin
E 6-N(0,,021)
[[611]]]
(72)
Amin
Tr [
Amin)) o²
(73)
where dim(w). In the above, Eq. (70) follows from the Taylor's remainder theorem (similar to Eq. (65)) and Eq. (72) follows from the definition of Amin in Assumption 2 as the lower-bound on the smallest eigenvalue of the parameter Hessian. Since Amin >> O, the quadratic term inside the expectation is always positive which explains why we can remove the absolute operator. Eq. (73) follows from standard moment calculation of Gaussian random vector.
Finally, we note that:
1 G(w)p + p2 Amax
(2 (>xann) m o²
2G(w)pp + )maxp??
(74)
m o²
Amin
1
2Gi(w)p + )maxpp
Fx(w + +5] (75)
m
o²
Amin
6~N(0.001)
which means, intuitively,
2G(w)p - Xmaxx
(76)
m
Amin
the smallest scaling factor for the lower-bound of o5 [Fx(w + 5)] so that matches the upper-bound of R.x(w) = max 5 Isp (x - 8). as outlined the 3rd step of our proving plan for Theorem 2. As such, plugging Eq. (68) into the left-hand side of Eq. (75) completes
proof. O
Remark. the above. Eq. (73) suggests that Eg Fr + +6] 2 (1/2))min m o². But, under Assumption 1, we know that the surrogate's ' output and hence Fx(w + 8)'s is bounded within [0, 1]. which means (1 (1mn))min m d² or equivalently, d² < 2/(mmmiin) as stated in Lemma 5's statement. That is, under Assumption 1, Lemma 5 is only correct for o² E (0, 2/(mmmin)).
E
Proof of Theorem 2
We are now ready to prove our main result. First, we re-state the result of Lemma 4 below,
E [x + 8)] < max [F + +)]
|
m log (1
|| (2) mo
2 log
log(Sn 4m)
(77)
n 1
RD(w)
|
m log 11
+ 2 log + 4log(8n + 4m)
mo?
(78)
71
where the last step follows from the definition of RD. Finally, plugging Eq. (78) into Eq. (63) and replacing max|ø]iso Fx(w + 8) with (xww) (following its definition) completes proof.
18

---

F Effective Approximation of V.. V /
Explicitly, the gradient of the augmented loss in Eq. (12) is given below, [
VwCD(w) + X.p- p V [=]\>]](w>]]
(79)
where the computation bottleneck lies with the second gradient term which can be further expressed below using the chain rule,
V [w)(w)]] Van(w) (vwh(w)) /[V]\(w))))
(80)
It is evident that V h(w) in Eq. (80) represents Hessian matrix. Calculating the Hessian matrix for a deep neural network is impractical due to the extensive dimensions of the model's weights. Nevertheless, since Eq. (80) involves the multiplication of Hessian matrix with vector, specific techniques like Hessian-vector products can be employed to approximate this product. In particular, let the Hessian matrix M Vih(w), we have Taylor expansion for function Vi.)(w) follows:
Vwh(w + Aw) = Vwh(w) + MAw + (l
(81)
This approximation becomes precise the value of Aw approaches O. Following [26] and [27], we choose Aw- = rV where is a small scalar and v a vector, that transforms Eq. (81) into
1 -
[(vwh(w Aw) - Vah(w)) + O(r).
Mv
(82)
Voh(w) leads to Then, choosing v = TVWh(WI
V=/lVch(W\)] M Vul(w) 1 (vuh
Vuh(w) vwh(w)) [Vwh(w)))))
(83)
[[W)(w))]
is noticed that, as pointed out by [27], an inappropriate choice of can make Eq. (83) vulnerable to numeric and roundoff issues. The constant must be sufficiently small that the O(r) term becomes negligible. However, when r is too small, precision is compromised because the subtraction of the original gradient from the perturbed one, i.e., Vuh(w + Wlhh(w//lVVWhc will obtain a small difference between them. Based on Eq. (83). Eq. (79) would be Vwh(w)
Vwh(w),
VuC(w) VwLD(w) XP (val
[Vw)(w)])) Vwh(ww)
(84)
In practical applications. we typically employ an additional approximation to compute the second term in Eq. (84), thereby avoiding the need for Hessian computation induced by the chain rule.
Vw](w) [w
Vwh w +
Vwh(w)
(85)
Vuk(wl
TVck
G Hyperparameter Tuning and Additional Experimental Results
G.1
Computation Resource
All our experiments were conducted on a system with the following specifications: Ubuntu 18.04, NVIDIA RTX 3090 GPUs, and CUDA 11.8.
G.2 IGNITE-2
IGNITE-2. To solve the Lagrangian in Eq. (12), can treat À as a hyper-parameter and optimize for w using stochastic gradient descent (SGD).
? (VwLo(w")
A. Aop-valll (<pp)))
(86)
where w' is the estimation of the surrogate's parameter at the tth iteration, and nww is step size to update w. We name this method IGNITE-2 and its pseudo-code is Algorithm 2.
Hyper-parameter Configuration. Our method IGNITE-2 introduces three additional hyper- parameters: X, P. and The penalty coefficient À controls the gradient magnitude of our regularizer, set to 0.01 through a grid search within {0.0001, 0.001, 0.01). The hyper-parameter chosen from (0.01, 0.05, 0.1, 0.2, 0.5). with p set to 0.2 and set to 0.2 for IGNITE. These three hyper-parameters are determined through experiments in Section G.4. These hyper-parameters are consistently applied across all experiments, except for the ICT baseline (A = le 4) due to implementation differences.
and are
19

---

Table 5: The percentage improvement in performance achieved by IGNITE-2 and IGNITE across all tasks and baseline algorithms at the 100th percentile level is presented. Gain signifies the percentage gain over the baseline performance (Base).
Contimuous fasks
Discrete task
Ant Morphology
D'Kitty Morphology
TF Bind 8
TF Bind 10
Algorithms
Performance
Gain
Performance
Gain
Performance
Gain
Performance
Gain
(bbest)
0.565
0.884
0,565
0.884
REINE. ORCE
IGNITE-2
Base
0,260 0.037
0.255 &: 0.036
+0.5%
0.546 0.208
0.611 0.176
+6.5%
0,954 0.027
0.929 0.043
+2.5%%
0.646 0.028
0.635 a 0.028
+1.1%
IGNITE
0.282 a: 0.021
+2.7%
0.642 a 0.160
+9.6%
0,944 a: 0.030
+1.5%
0.670 a 0.060
+3.5%
Base
0.303 + 0.027
0.881 # 0.016
0,980 0.016
0.651 # 0.033
GA
IGNITE-2
0.312 2 0.038
+0.9%
0.885 0.022
+0.4%
0,985 0.007
+0.5%
0.663 0.090
+1.2%
IGNITE
0.320 4: 0.044
+1.7%
0.886 + 0.017
+0.5%
0,985 0.010
+0.55
0.653 a 0.043
+0.2%
ENS-
Base
0.376 + 0.060
0.888 0.010
0.985 0.009
0.649 0.016
IGNITE-2
0.437 0.068
+6.1%
0.890 0.010
+0.2%
0,988 0.005
+0.3%
0.665 0.091
+1.6%
MEAN
IGNITE
0.435 + 0.058
+5.9%
0.896 0.013
+0.8%
0,987 0.007
+0.2%
0.662 a 0.091
+1.3%
ENS-
Base
0.385 0.067
0.889 = 0.014
0,980 = 0.012
0.681 # 0.095
IGNITE-2
0.441 0.084
+5.6%
0.894 0.011
+0.5%
0,982 0.015
+0.2%
0.686 0.120
+0.5%
MIN
IGNITE
0.468 + 0.062
+8.3%
0.897 0.010
+0.8%
0,986 0.010
+0.6%
0.705 0.118
+2.4%
Base
0.854 a 0.042
0.895 = 0.012
0,919 = 0.044
0.635 + 0.041
CbAS
IGNITE-2
0.850 0.036
0,903 0.014
+0.8%
0,916 0.043
-0.3%
0.650 0.054
+1.5%
IGNITE
0.859 + 0.039
+0.5%
0.900 0.015
+0.5%
0,921 0.042
+0.25
0.652 0.055
+1.7%
Base
0.905 = 0.023
0,944 = 0.008
0,892 0.046
0.643 # 0.062
MINs
IGNITE-2
0.907 + 0.035
+0.2%
0.940 0.007
-0.4%
0,915 0.040
+2.3%
0.645 0.049
+0,2%
IGNITE
0.911 = 0.024
+0.6%
0.945 0.007
+0.1 %
0,930 0.041
+3.8%
0.647 0.058
+0.4%
Base
0.569 a: 0.086
0.821 +: 0.019
0.665
0.000
0.550 a 0.008
RoMA
IGNITE-2
0.063
+2.1%
0.833 0.028
+1.2%
0.665 0.000
+0.0%
0.553 0.000
+0.3%
IGNITE
0.615 a 0.085
+4.6%
0.834 0.012
+1.3%
0.665 0.000
+0.0%
0.553 0.000
+0.3%
Rase
0.897 + 0.031
0,931 0.013
0,955 0.030
0.645 + 0.038
COMs
IGNITE-2
0.911 + 0.030
+1.4%
0.940 = 0.014
+0.9%
0,948 0.025
-0.7%
0.637 0.033
-0.8%
IGNITE
0.901 = 0.030
+0.4%
0.934 0.010
+0.3%
0,952 0.043
-0.3%
0.638 0.053
-0.7%
Base
1.955 à 1.484
0.724 à 0.002
0,928 0.040
0.668 a 0.035
CMA-ES
IGNITE-2
1.970 1.971
+1.5%
0.725 0.006
+0.1%
0.938 0.031
+1.0%
0.670 0.033
+0.2%
IGNITE
1.957 a 1.910
+0.2%
0.724 0.001
+0.0%
0,927 0.043
-0.1%%
0.673 1 0.044
+0.5%
Base
0 + 0,000
0.896 0.000
0.787 0. 11112
0.628 0.000
BO-qEI
IGNITE-2
0,812 0,000
+0.0%
0.896 = 0.000
+0.0%
0,855 0.107
+6.8%
0.628 0,000
+0.0%
IGNITE
0.812 0.000
+0.0%
0.896 0.000
+0.0%
0.843 0.109
+5.6%%
0.628 0.000
+0.0%
Base
0.937 + 0.023
0,946 = 0.014
0.892 0.055
0.647 0.025
ICT
IGNITE-2
0.936 4: 0.017
-0.1%
0.947 = 0.019
+0.1%
0.920 0.035
+2.8%
0.656 0.029
+0.9%
IGNITE
0.935 à 0.032
-0.2%
0.962 0.018
+1.6%
0,923 0.038
+3.1%
0.652 0.074
+0.5%
Algorithm 2 IGNITE-2
Input: offline data D ((x,, =)) initial surrogate g(x; (0)): no. of iterations T; batch size m.; Lagrange multiplier X: perturbation radius p and sealar r; step sizes N/
I: Initialize (1)) <- (0)
7:
Compute e (1) + 9TT-22119991
2: for t 4- 1: T: do
8:
Compute O3 = m "E=" (wg(x;;
3: Sample B = {(xi, x)) ~ D
9: Compute g()) = 91 + Apr "(s - 92)
4: Compute & g(x;; wl))) for E [m]
10: Update w (t+1) +- (1) n(g
5: Compute 91 = m 11 >mm
i=1 Vul(E,, 21)
11: end for
6: Compute 92 m m Vwli
12: return learned surrogate w (T+1)
G.3 Performance Evaluation at 100-th, 80-th and 50-th Percentile Level of IGNITE and IGNITE-2
In this section, we presented the percentage improvement over baseline performance attained by IGNITE-2 and IGNITE when is applied to an existing baseline. We have evaluated this at the 100-th, 80-th, and 50-th percentile levels. The results reported in Table 5, 6, and 7, respectively.
20

---

Table 6: The percentage improvement in performance achieved by IGNITE-2 and IGNITE across
all tasks and baseline algorithms at the 80th percentile level is presented. Gain signifies the percentage gain over the baseline performance (Base).
Continuous tasks
Discrete task
Ant Morphology
D'Kitty Morphology
TF Bind 8
TF Bind 10
Algorithms
Performance
Gain
Performance
Gain
Performance
Gain
Performance
Gain
((best)
0,565
0.884
0.565
0.884
Base
0,185 = 0.035
0,508 = 0.200
0.613 = 0.029
0,523 + 0,008
REINE. ORCE
IGNITE-2
0.191 0.033
+0.6%
0,462 0.199
.4.6%
0.620 + 0.031
+0.7%
0.520 0.006
-0.3%
IGNITE
0.210 # 0.039
+2.5%
0.532 # 0.197
+2.4%
0.616 + 0.038
+0.3%
0.523 + 0.007
+0.0%
Base
0.195 1 0.011
0.784 à: 0.030
0.826 a: 0.032
0.517 a. 0,006
GA
IGNITE-2
0.189 + 0.019
.0.6%
0.794 + 0.039
+1.0%
0.832 0.032
+0.6%%
0.518 0.007
+0.1%
IGNITE
0,201 # 0.017
+0.6%
0.810 + 0.026
+2.6%
0.833 + 0.021
+0.7%
0,517 + 0.006
+0.0%
ENS-
Base
0,236 0.013
0.823 = 0.013
0.852 # 0.023
0.515 # 0,006
IGNITE-2
0.235 0.010
-0.1%
0.835 1 0.013
+1.2%
0.855 a 0.020
+0.3%
0.511 4: 0.006
-0.4%
MEAN
IGNITE
0.013
+0.8%
0.841 = 0.013
+1.8%
0.848 + 0.016
-0.4%
0.517 = 0.006
+0,2%
ENS-
Base
0,229 0.016
0.819 = 0.017
0.845 = 0.021
0,512 + 0,005
IGNITE-2
0.236 + 0.011
+0.7%
0.835 0.012
+1.6%
0.854 0.026
+0.9%
0.514 0.004
+0.2 %
MIN
IGNITE
0,249 0.015
+2.0%
0.845 = 0.010
+2.6%
0.834 # 0.015
-1.1%
0.516 + 0.006
+0.4%
Base
0,581 + 0.037
0.814 :: 0.011
0.576 a: 0.022
0.510 :: 0,009
ChAS
IGNITE-2
0.571 + 0.028
-1.0%
0.815 0.009
+0.1%
0.565 a: 0.025
-1.1%
0.512 a: 0.009
+0.2 % +0.2%
IGNITE
0.550 + 0.052
-3.1%
0.812 + 0.013
-0.25
0.565 + 0.023
-1.1%
0,512 + 0.009
Base
0,754 = 0.026
0.907 = 0.003
0.544 = 0.029
0.518 = 0,008
MINs
IGNITE-2
0.750 a 0.023
-0.4%
0.909 0.003
+0.2%
0.543 1 0.028
-0.1%
0.518 0.009
+0.0 %
IGNITE
0.753 + 0.024
-0.1%
0.909 + 0.004
+0.2%
0.551 + 0.030
+0.7%
0.515 + 0.009
-0.3%
Base
0,306 = 0.036
0.736 = 0.016
0.644 + 0.037
0.527 = 0,008
RoMA
IGNITE-2
0.305 = 0.031
-0.1%
0.746 = 0.022
+1.0%
0.646 = 0.049
+0.2%
0.526 0.003
-0.1 %
IGNITE
0,312 + 0.032
+0.6%
0.737 # 0.019
+0.1%
0.655 + 0.014
+1.1%
0.527 + 0.002
+0.0%
Base
0.629 : 0.028
0.887 = 0.005
0.747 +: 0.052
0.517 + 0,009
COMs
IGNITE-2
0.653 a 0.026
+2.4%
0.889 1 0.003
+0.2%
0.742 a 0.061
-0.5%
0.519 0.007
+0.2 %
IGNITE
0.629 + 0.025
+0.0%
0.891 + 0.005
+0.4%
0.751 + 0.063
+0.45
0.517 + 0.013
+0.0%
Base
0,013 0.017
0.718 = 0 0.001
0.653 = 0.020
0.546 = 0,014
CMA-ES
IGNITE-2
0.011 1 0.018
-0,2%
0.719 1 0.001
+0.1%
0.649 0.022
-0.4%
0.545 1: 0.010
-0.1 % +0.3%
IGNITE
0.013 + 0.024
+0.0%
0.718 e: 0.002
+0.0%
0.652 à: 0.021
-0.1%
0.549 :: 0.014
Base
0.629 = 0.000
0.884 + 0. 0.000
0.439 + 0 0.000
0.513 + 0.001
BO-qEI
IGNITE-2
0.629 = 0.000
+0.0%
0.884 = 0.000
+0.0%
0.439 + 0.000
+0.0%
0.513 : 0.001
+0.0 %
IGNITE
0.629 + 0.000
+0.0%
0.884 + 0.000
+0.0%
0.439 + 0.000
+0.0%
0,513 + 0.000
+0.0%
Base
0.710 0.022
0.896 0. 0.005
0.687 0.054
0.549 0.016
ICT
IGNITE-2
0.702 à 0.024
-0.8%
0.897 0.004
+0.0%
0.680 0.037
-0.7%
0.551 a 0.022
+0.2%
IGNITE
0.668 + 0.026
.4.2%
0.895 + 0.005
-0.1
0.651 + 0.039
-3.6%
0.503 + 0.043
...6%
G.3.1 Performance Evaluation at 100-th Percentile Level of IGNITE-2
According to the reported results in Table 5, IGNITE-2 consistently maintains a high probability of 86.36% (38 out of 44) of not degrading baseline performance. There is a high likelihood (72.27% = 34 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1.39% and notable peak improvement of 6.8%. Conversely, IGNITE-2 also exhibits a relatively low probability (13.64% = out of 44 cases) of decreasing performance, with an average degradation of approximately 0.45% and a minor peak degradation of ..d.iiiiionallly there is minor probability (9.09% = out of 44 cases) of maintaining baseline performance. Furthermore, while IGNITE-2 demonstrates significant efficiency, the results in Section 5.2 indicate that IGNITE even outperforms IGNITE-2.
G.3.2 Performance Evaluation at 80-th Percentile Level of IGNITE and IGNITE-2
As shown in Table 6, IGNITE-2 consistently maintains high probability high probability of not degrading baseline performance, with a likelihood of 61.36% (27 out of 44 cases). There a 47.73% chance (21 out of 44 cases) of improving baseline performance, with an average improvement of approximately 0.6% and a peak improvement of 2.4%. On the other hand, IGNITE-2 also indicates a low likelihood (38.64%- 17 out of 44 cases) of performance decrease, with an average decline of around O. 0.6%% and a maximum decline of 4.6%. Furthermore, there is a slight chance (13.64% = out of 44 cases) of maintaining the bascline performance.
21

---

Table 7: The percentage improvement in performance achieved by IGNITE-2 and IGNITE across all tasks and baseline algorithms at the 50th percentile level is presented. Gain signifies the percentage gain over the baseline performance (Base).
Continuous tasks
Discrete
Ant Morphology
D'Kitty Morphology
TF Bind 8
TF Bind 10
Algorithms
Performance
Gain
Performance
Gain
Performance
Gain
Performance
Gain
D(best)
0.565
0.884
0.565
0.884
Base
0.142 0.042
0,431 0.183
0.459 0.020
0.469 + 0.008
REINE- ORCE
IGNITE-2
0.157 0.031
+1.5%
0,432 + 0,187
+0.1%
0.459 + 0.020
+0.0%
0,470 + 0.005
+0.1%
IGNITE
0.167 = 0.043
+2.5%
0,453 + 0.188
+2.2%
0,450 + 0.018
-0.9%
0.472 # 0.005
+0.3%
Base
0.147 0.011
0.600 0.145
0.613 + 0.039
0,467 + 0.004
GA
IGNITE-2
0.142 1 0.017
-0.5%
0.623 + 0,181
+2,3%
0.612 + 0.023
+0.0%
0.467 1: 0.003
+0.0%
IGNITE
0.155 0.013
+0.8%
0.746 0.036
+14.6%
0.611 0.026
-0.2%
0.469 a 0.005
+0.2%
Base
0.189 0.011
0.752 0.026
0.643 0.027
0,468 0.002
ENS-
IGNITE-2
0.186 # 0.010
-0.3%
0,780 + 0.025
+2.8%
0.670 + 0.031
+2,7%
0,466 + 0.004
-0.2%
MEAN
IGNITE
0.192 0.008
+0.3%
0.796 0.022
+4.4%
0.644 0.037
+0.15%
0.467 1: 0.001
-0.1%
ENS-
Base
0.183 0.013
0,741 0.022
0.655 1 0.031
0.467 + 0.001
IGNITE-2
0.188 a 0.010
+0.5%
0.777 a: 0.025
+2.9%
0.658 1: 0.038
+0.3%
0,467 1. 0.002
+0.0%
MIN
IGNITE
0.197 0.015
+1.4%
0.805 0.016
+6.4%%
0.626 0.035
-2.9%
0.467 0.002
+0.0%
Base
0.382 0.028
0 + 0,015
0,433 + 0.015
0,458 + 0.006
CbAS
IGNITE-2
0,374 +: 0.020
-0.8%
0.744 :: 0,018
-0.7%
0.430 + 0.020
-0.3%
0,459 1: 0.008
+0.1%
IGNITE
0.374 0.026
0.8%
0.743 0.018
-0.8%
0.427 0.020
-0.6%
0.461 4: 0.007
+0.33
Base
0.637 0.035
0.888 +: 0,005
0.417 1: 0.019
0.465 A 0.007
MINS
IGNITE-2
0.628 0.025
0.9%
0,889 0.004
+0.1%
0.421 0.014
+0.4%
0.469 + 0.006
+0.45
IGNITE
0.628 0.027
-0.9%
0.888 0,005
+0.00%
0.422 + 0.019
+0.5%
0,466 + 0.008
+0. 1%
Base
0.233 4 0.020
0.612 # 0,146
0.484 + 0.045
0,513 # 0.006
RoMA
IGNITE-2
0.228 0.018
-0.5%
0.584 0.147
-2.8%
0.494 0.044
+1.0%
0,514 1 0.006
+0.1%
IGNITE
0.228 0.014
.0.5%
0.652 + 0,094
+4.0%
0.503 + 0.068
+1.9%
0,517 + 0.003
+0.45
Base
0.487 1 0.020
0.859 :: 0.007
0,588 &: 0.037
0.473 +: 0.013
COMS
IGNITE-2
0.503 0.025
+1.6%
0,862 0.004
+0.35
0.577 0.040
-1.1%
0.474 + 0.006
+0.1%
IGNITE
0.482 # 0.027
-0.5%
0.862 + 0.008
+0.3%
0.598 + 0.042
+1.0%
0.471 # 0.010
-0,2%
Base
-0.043 0.008
0.677 &: 0,014
0.538 a 0.013
0,492 1 0.013
CMA-ES
IGNITE-2
0.045 0.008
00.2%
0.685 0,01 1
+0.8%
0.532 0.020
.0.6%
0.493 4: 0.012
+0.1%
IGNITE
-0.049 + 0.009
-0.6%
0.681 + 0.014
+0.45%
0,542 + 0.020
+0.4%
0.496 + 0.013
+0.4%
Base
0.569 0.000
0.883 0.000
0.439 0.000
0.468 + 0.000
BO-qEI
IGNITE-2
0.569 0.000
40.0%
0.883 0,000
+0.0%
0.439 + 0.000
+0.0%
0.467 + 0.000
-0,19
IGNITE
0.569 : 0.000
+0.0%
0.883 + 0,000
+0.0%
0.439 : 0.000
+0.0%
0.467 : 0.000
+0.0%
Base
0.556 0.024
0.872 0.007
0.564 0.046
0.501 #: 0.018
ICT
IGNITE-2
0.559 0.025
+0.3%
0.873 + 0,007
+0. 1%
0.554 + 0.033
-1.0%
0.498 + 0.022
-0.3%
IGNITE
0.551 + 0.022
-0.5%
0.878 + 0.003
+0.6%
0.527 + 0.030
-3.7%
0.454 + 0.038
-4.7%
In addition, IGNITE also maintains a high probability of 72.73% (32 out of 44) of not degrading baseline performance. There is a 47.73% likelihood (21 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1.0% and a peak improvement of 2.6%. Besides, IGNITE also exhibits low probability (27. 27% = 12 out of 44 cases) of decreasing performance, with an average degradation of approximately 1.58% and peak degradation of 4.6%. Additionally, there is a small probability (25% = 11 out of 44 cases) of maintaining baseline performance.
G.3.3 Performance Evaluation at 50-th Percentile Level of IGNITE and IGNITE-2
Table 7 displays the results of IGNITE-2 and IGNITE at the 50th percentile level. The results indicate that both IGNITE-2 and IGNITE do not degrade performance compared to the baseline in 65.91% % of settings (29 out of 44) for each method. In which, IGNITE-2 outperforms the baseline in 50% likelihood, with an average improvement of approximately 0.85% and a peak improvement of 2.9%. For IGNITE those are 52.27% likelihood, 1.89% average improvement and a peak improvement of 14.6%. Conversely, there are only 34.09% cases in using our proposed method leads to the performance reduction. The average degradation of IGNITE-2 and IGNITE are 0.69% and 1.19%, respectively. Overall, we state that our proposed method helps to boost the performance of almost all algorithms and tasks.
22

---

1P-880--10
AMT
0:00 da www.teegea
0/90
0.00
Normaltzed objectiv
Blnding alfiviity
800
0016 (ON
0216:
Normaitzed abjective
0.00
Binding altiritt
COMS
GOMd D.07
GA (Bass)
COMs 1o-04
0A (ONITE)
GA 0.001
0.91
0.0
8.00
042
6.7
40
0.0
30
D197
20
Gradiont
Gradiont
Gradiont
stops
Figure 3: Performance vs. the no. gradient as- cent steps during optimization of IGNITE-2 and Baseline optimized algorithms, e.g, COMs and GA.
Figure 4: Performance variation of COMS
and GA (regularized by IGNITE-2) in the change of the regularization coefficient À E [0.0001, 0.001, 0.01].
Normatzed objoctive value
Normelieed abjective value
DOMs 0.01
COMs
Sluding altinity
DOMs 0.25
DOMe 0.1
0.00
Binding affintty
DOMB 3.1
OONS 3:66
COMN
COMS
OONS
0400.05
GA.005
0.1
G4
0.00
GA
0.00
GA
04.05
G405
Gradient ascent steps
Gradient ascent steps
Gradient ascent steps
Gradient ascent steps
Figure 5: Performance variation of
Figure 6: Performance variation of COMS and GA (regularized by IGNITE- 2) in the change of the hyper-parameter p e [0.01, 0.05, 0.2.5.
COMS and GA (regularized by IGNITE-
2) in the change of the hyper-parameter r E [0.01, 0.05, 0.2,0.
G.4 Hyper-parametery selection for IGNITE-2
This section provides the specific setup of the hyper-parameter selection experiments for IGNITE-2, which is also ran separately to determine the hyperparameters for IGNITE(in the main text).
G.4.1 IGNITE-2 enhances stability of COMs and gradient ascent (GA).
Figure provides step-by-step performance comparison between two baseline algorithms, COMs and GA, with and without our regularization method, IGNITE-2. IGNITE-2 consistently out- performs the baseline algorithms at every step in the ANT tasks. However, the trend differs for TF-BIND-10. Initially, the baseline algorithms perform better without IGNITE-2 However, as the number of gradient ascent steps increases, the performance of the baselines without IGNITE- 2 begins to decline. In contrast, the algorithms incorporating performance over time. This trend suggests that our method becomes increasingly beneficial in the later stages of the optimization process, ultimately enhancing the overall performance of the baseline algorithms.
method maintain or improve their
G.4.2 Choosing the regularization coefficient X.
Figure shows how the performance of baseline models regularized with IGNITE-2 is affected by different values of A. The results indicate that using excessively low high value for À will have a negative impact on performance. In all cases, the results suggest that a universal value 0.01 for tends generate consistent and effective performance across all tasks.
G.4.3 Choosing value for parameter and the perturbation radius
Figure 5 plot the performance of the GA and COMS baselines regularized with IGNITE-2 with respect to the change of hyper-parameter r. That is e (0.01, 0.05, 0. 0.2, 0.5). Figure visualizes how the performance of baselines regularized with IGNITE-2 is influenced by varying the hyper- parameter p. Based on those result, we choose to set and p = 0.2 in all of the experiments for IGNITE-2.
23

---

H Percentage improvement over other baselines of IGNITE, SAM across all tasks.
Table 8: Percentage improvement over other baselines of IGNITE, SAM across all tasks.
Algorithms
Ant
D'Kitty
Bind
Bind
REINFORCE
0.255 0.036
0,546 0.208
0.929 = 0.043
= 0.028
REINFORCE IGNITE
0.282 0.021 (+2.7%)
0.642 = 0.160 (+9.6%)
0,944 0.030 (+1.5%)
0.670 0.060 (+3.5%)
REINFORCE SAM
0.266 0.030 (+1.
0.625 + 0.182 (+7.9%)
0.940 = 0.035 (+1.1%)
0.637 0.037 (+0.2%)
GA
0.007
0.881 0.016
0.980 0.016
0.651 0.033
GA IGNITE
0.320 0.044 (+1.7%)
0,886 + 0.017 (+0.5%
0.985 = 0.010 (+0.5%)
0.653 = 0.043 (+0.2%)
GA + SAM
0,310 0.044 (+0.7%)
0.868 + 0.014 (-1.3%)
0.982 = 0.015 (+0.2%%
0.662 # 0.041 (+1.1%)
CbAS
0.854 a 0.042
0.895 + 0.012
00.919 = 0.044
0.635 #: 0.041
CbAS IGNITE
0.859 + 0.039 (+0.5%)
0.900 + 0.015 (+0.5%)
0.921 = 0.042 (+0.2%)
0.652 # 0.055 (+1.7%)
CbAS SAM
0.853 = 0.033 (-0.1%)
0.897 + 0.013 (+0.2%)
0.905 = 0.053 (-1.4%)
0.637 = 0.023 (+0.2%)
BO-qEI
0.812 0.000
0.896 + 0.000
0.787 = 0.112
0.628 # 0.000
BO- IGNITE
0,812 0.000 (+0.0%)
0.896 + 0.000 (+0.0%)
0.843 = 0.109 (+0.3%)
0.628 0.000 (+0.0%)
BO-qEI SAM
0.812 0.000 (+0.0%)
0.896 0.000 (+0.0%)
0.763 0.098 (-2.4%)
0.619 0.022 (-0.9%)
I Complexity Overhead of IGNITE.
To analyze the computational complexity of IGNITE, we break down the complexity of each step in Algorithm 1.
1. Initialization (Line 1): Initializing (1) (0) and ( x: O(1) each.
2. Main Loop (Line 2-12): The loop runs for T iterations. Thus, the complexity of the main loop will be multiplied by T.
3. Sampling (Line 3): Sampling a batch B ((x;, +)][= ~ D: O(m).
4. Computing 2, (Line 4): Evaluating the surrogate model ((x for each i [m]] Assuming the surrogate model evaluation has computational complexity of C. = (()) per sample where d is the number of surrogate parameters, the total complexity is O(m d).
5. Computing 9i and Ga (Line 5-6): Computing gradients V. 2;) and V have com- plexities OCC)) and O(C2) respectively per sample where Cj Cs = O(d). Therefore. the total complexities are O(m d).
6. Computing @ (Line 7): This involves simple vector operations with complexity O(d), where is the dimensionality of w.
7. Computing 93 (Line 8): Similar to lines 4 and 6, involving evaluating the surrogate and gradient computations, with complexity Omm C, + m C2) = O(m d).
8. Computing (*) (Line 9): Vector operations involving addition and scalar multiplication with complexity (d).
9. Updating (Line 10): Updating w involves simple subtraction operations with complexity (d).
10. Updating X (Line 11): Updating À is an O(1) operation
Overall Complexity: Considering the above steps, the most computationally expensive parts are the gradient computations in lines 5, 6, and 8. Thus, the overall complexity per iteration is:
(2m - Cg + m C2 + 2m C2)
Since this loop runs for iterations, the total complexity is:
O(T (2m- Cg + m Ct + 2m C2))
Furthermore, we have the total complexity of the original baseline is:
O(T (m Cg +m Ct))
24

---

GA ANT
GA DKITTY
REINFORCE ANT
REINFORCE DKITTY
Trahing les Burogua sharpress
Triniig ldss
Tainnng loss
Trih less Urrggia sharpenso
Sumopela sharpreos
Value
Epochs
Epocho
Epochs
Epochs
Figure 7: The convergence of the proposed optimization algorithm. Figure shows the training loss and the sharpness value (plotting [u](w)]] value) during the surrogate fitting process.
Table 9: The empirical time (seconds) of the participating baselines with and without IGNITE.
Algorithms
Ant
D'Kitty
TF Bind 8
TF Bind 10
REINFORCE
172.08
252.33
477.09
32.95
REINFORCE + IGNITE
194.02 (+12.75%)
275.15 (49.04%)
582.28 (+22.05%)
437.38 (+17.28%)
GA
69.99
168.81
149.63
364.16
GA + IGNITE
85.15 666%)
191 83s (+13.64%)
181.71 (+21.44%)
369.29 (+1.41 %))
Thereby, IGNITE will include an additional complexity:
((T (m Cg + 2m C+)) (TTmd)
where:
T' is the number of iterations.
m is the batch size.
C = (d) is the complexity of evaluating the surrogate model.
C₂ (d) is the complexity of computing the loss gradient.
C3 = (dd) is the complexity of computing the gradient of the surrogate output with respect
to its parameters.
d the no. of surrogate parameters.
The empirical training time of the participating baselines with and without IGNITEis reported in Table We observe that IGNITE solely consumes an additional negligible GPU memory, while the training time increases by 14.91% % on average.
J Convergence and effectiveness of IGNITE
According to our experiment, despite using a relatively simple BDMM to solve Eq. (12), we have achieved significant improvement in most cases. To demonstrate the convergence of the optimization algorithm, we have plotted the training loss and the sharpness value (plotting V [()))] value) during the surrogate fitting process. This is based baselines on Ant and Dkitty tasks. The results are illustrated in Figure 7. These results reveal that BDMM helps decrease both the training loss and sharpness value of the surrogate model during the training phase. This indicates that BDMM is effective and the optimization converges well in practice. Furthermore, our method, IGNITE, be seamlessly integrated with other, more robust optimization techniques to solve Eq. (12).
experiment using GA and REINFORCE
K Limitation
Our paper studies the offline optimization task, which has potential applications in material engi- neering. Similar to the existing literature, our method is extensively tested on universal benchmark set forward by the pioneering work of [4]. However, important to note that the benchmark consists mostly of small- to mid-scale optimization tasks. As such, method has not considered
25

---

the challenge of scalability in large-scale domain with extremely high-dimensional input spaces. In addition, as with all machine learning algorithms, while applications of our work to real data could result in ethical considerations, this is an indirect and unpredictable side-effect of our work. Our experimental work uses publicly available datasets to evaluate the performance of our algorithms. No ethical considerations are raised.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our contributions can be found in Section 3 and Section 4.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss this in Appendix K
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
27

---

All our assumptions are detailed in Section 4. All proofs are detailed in the Appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Yes. All information regarding our experiments are disclosed in Section 5 and in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
28

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Our experimental code is also submitted (as extra materials) along with our manuscript.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
Stwplc/guides/ooeSSuu sesiisssssoooooo oelioonoooolooooo olycyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cc/ mublic/guides SudeeS/mm Sunaiuuubssssss wwsssssollll.o..oo onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Such details can be founded in Section 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We do report error bars in our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
29

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The information regarding our compute resource is detailed in Appendix G.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http:///////us.eeeeeeeeee cttpeiipp.cpppbbbie seabblit/Ethissuiideee
Answer: [Yes]
Justification: We have read the NeurIPS Code of Ethics and do not find our work in violation of any aspects.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We did discuss this in Appendix K.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
30

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not create datasets. We only use existing, publicly available datasets. We also do not create any new pre-trained models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cite the source of all datasets used in our experiments.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
31

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasttss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our work does not release any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our work does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
32

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "136KFg9T4t", "reviewer": "Reviewer_xjU2", "summary": "This paper focuses on offline optimization, presenting a novel approach to enhance the surrogate landscape's sharpness, thereby improving generalization. The authors introduce a gradient norm as an approximation of surrogate sharpness through a first-order Taylor expansion, resulting in a Lagrangian formulation. They employ two widely adopted methods—BDMM and a fixed multiplier as solvers for the Lagrangian.\n\nTheoretically, the authors provide a generalization bound for the worst-case prediction change. Empirically, they validate their approach using the widely adopted offline optimization dataset, design-bench, demonstrating relatively good performance.", "questions": "Please answer my question mentioned above.", "limitations": "Please see the weaknesses part.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The idea of using a sharpness-aware method in offline optimization is well-motivated.\n\n2. The proposed method performs well on some datasets with specific baselines.\n\n3. The authors provide a generalization analysis.", "weaknesses": "Though I agree with the ideas of using a sharpness-aware method in offline optimization, I think more details should be justified:\n\n\n1. I am somewhat unsure whether the authors analyze the generalization bound on $\\mathcal{R}_{X}(\\omega)$ properly. \n\nDoes the generalization over $R_{X}(\\omega)$ help to improve the generalization of $L_{X}(\\omega)$, which is the objective we want to generalize. Why not analyze the generalization bound w,r,t $L_{X}(\\omega)$?\n    \n2. Is the bound tight? I understand that the original SAM paper [8] also provides a similar bound. But if it is not tight, we do not know whether it can guide the design of our algorithm.\n\n3. Moreover, the authors should add more details to their proofs to help readers better understand. For example, in line 416, the authors should present it as a lemma rather than directly stating the results of [8] as there are some assumptions in their proofs which might not be discussed in your proof. In line 458, the authors say that they use Taylor's remainder term, but it seems that the remainder term is directly eliminated. Could the authors discuss more details?\n\n4.  Another concern is that I am not very sure whether Assumption 2 is a strong assumption. If the assumption holds, then the Hessian is positive definite. Since the proof highly relies on it, I am afraid the generalization bound is not established in the general stationary point  (the Hessian is positive semi-definite rather than positive definite).\n\n5.  Though the proposed method shows relatively higher performance than some corresponding baselines on some datasets, the overall improvements are marginal and sometimes even decrease, as shown in table 1. Moreover, different percentages show vastly different performance. For example, reinforce+IGNITE shows a 6.5\\% gain over reinforce at the 100th percentile level, but a -4.6\\% at the 80th percentile level. How can this be explained? Does it mean that the proposed method is not robust?\n\n6. Is there any analysis on memory and time? The term includes the gradient of the gradient $\\nabla_{\\omega}|| \\nabla_{\\omega} h\\left(\\omega^t\\right) ||$, which I am concerned is both time and memory-consuming.\n\n\nIn all, I think the idea is interesting, and the empirical and theoretical results are promising. But I still think more details should be added before I can vote for acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed feedback. Your questions are addressed below.\n\n**Q1. Why not bounding $\\mathcal{L}_\\mathcal{X}(\\omega)$ directly?**\n\nBounding $\\mathcal{L}\\_\\mathcal{X}(\\omega)$ helps reduce the averaged prediction error in the OOD region but it does not guarantee such errors will not accumulate along consecutive predictions that guide the gradient-based optimization process. This could lead to substantial gaps between its recommended solutions and the true oracle optima. \n\nTo elaborate, $\\mathcal{L}\\_\\mathcal{X}(\\omega)$ can be bounded with a direct application of Sharpness-Aware Minimization (SAM) [1]. A surrogate that minimizes this bound might have on average a low (single) prediction error in OOD regions. However, offline optimization requires multiple predictions at consecutive inputs that occur during the gradient update process, where errors accumulate and mislead the gradient search toward suboptimal input candidates.\n\nTo mitigate this, our intuition in Fig. 1 (see lines 130-136) suggests that the optima of surrogates with low sharpness tend to be closer to the oracle optima. This means a low sharpness value of the surrogate landscape (rather than its loss landscape) will help the gradient search accumulate less error. \n\nThis is formulated in Eq. (5) which applies a low-sharpness constraint $\\mathcal{R}\\_\\mathcal{X}(\\omega) \\leq \\epsilon^{'}$ to the surrogate fitting loss. Solving Eq. (5) therefore requires upper-bounding $\\mathcal{R}\\_\\mathcal{X}(\\omega)$ with a tractable function — see Eq. (6). This insight is also supported by the empirical studies in Table 2. \n\n[1] Sharpness-aware minimization for efficiently improving generalization. ArXiv:2010.01412.\n\n**Q2. Does the generalization over $\\mathcal{R}\\_\\mathcal{X}(\\omega)$ help to improve the generalization of $\\mathcal{L}\\_\\mathcal{X}(\\omega)$?**\n\nFollowing the response to Q1, bounding $\\mathcal{L}\\_\\mathcal{X}(\\omega)$ with SAM does not help mitigate the error accumulation along the gradient search. Hence, our approach does not aim to do this.\nInstead, we generalize over $\\mathcal{R}\\_\\mathcal{X}(\\omega)$ to solve Eq. (5) which is essential to prevent such error accumulation. \nInvestigating whether generalizing over $\\mathcal{R}\\_\\mathcal{X}(\\omega)$ also improves generalization over $\\mathcal{L}\\_\\mathcal{X}(\\omega)$ is interesting but orthogonal to the scope of our work.\n\n**Q3. Is the bound tight?**\n\nYes. When the no. of offline data points is sufficiently large, $\\frac{\\log n}{n}$ tends to zero and Eq. (6) or Eq. (16) in Theorem 2 reduces to $\\frac{\\mathcal{R}\\_\\mathcal{X}}{\\mathcal{R}\\_\\mathcal{D}} \\leq constant$. This depends only on properties of the surrogate, such as the boundedness of its gradient and the largest eigenvalue of its Hessian. Both can be made small via a theoretic choice of a surrogate whose gradient has a low Lipschitz constant (i.e., smooth).\n\n**Q3. Adding more details to proofs and Taylor’s remainder term in line 458.**\n\nWe will present line 416 as a lemma in the revised paper. As for the question regarding the Taylor’s remainder in line 458: it is the second term in Eq. (65) which involves $\\hat{\\omega} = \\omega + c\\delta$, which is not eliminated. This is followed by the 1st-order Taylor expansion with an explicit characterization of the remainder term:\n\n$h(\\omega + \\delta) - h(\\omega) = \\nabla\\_\\omega h(\\omega)^T \\delta + R\\_1(\\omega, \\delta)$ where the remainder $R\\_1 (\\omega, \\delta) = 1/2 \\delta^T \\nabla^2\\_\\omega h(\\hat{\\omega}) \\delta$ with $\\hat{\\omega} = \\omega + c\\delta$.\n\nFor a quick reference, please refer to Eq. (4) in [1], and the 2nd equation in [2]. \n\n[1] https://sites.math.washington.edu/~folland/Math425/taylor2.pdf \n\n[2] https://people.math.sc.edu/josephcf/Teaching/142/Files/Worksheets/Estimation%20of%20the%20Taylor%20Remainder.pdf\n\n**Q4. Is Assumption 2 strong?**\n\nAssumption 2 can be satisfied with a class of surrogates established in Theorem 1 (see its proof in Appendix A). Since there exists an implementable surrogate formulation for which Assumption 2 holds, we believe it is not a strong assumption.\n\n**Q5. Is the proposed method (IGNITE) robust?**\n\nThe example that the reviewer pointed out – 6.5% gain over REINFORCE at the 100th percentile level, but a -4.6% at the 80th percentile level – does not pertain to our IGNITE method. Instead, it corresponds to a simpler version of our work (IGNITE-2) which is used to demonstrate the impact on performance if we use a manual choice for the Lagrangian parameter in Eq. (12). Our main method (IGNITE) uses BDMM to also optimize this Lagrangian parameter, leading to better and more stable performance than IGNITE-2. It can be observed that overall, IGNITE shows relatively high improvements over the baseline with very few cases of (very) slight performance decrease.\n\n**Q6. Complexity of IGNITE.** According to Algorithm 1 and Appendix F on the effective computation of $\\nabla\\_\\omega ||\\nabla\\_\\omega h(\\omega)||$, we provide a time complexity overhead for IGNITE below.\n\n**Complexity Analysis.** Following our detailed complexity analysis in response to Reviewer qc6B, the complexity overhead of IGNITE is  $O(Tmd)$ where:\n* $T$ is the number of iterations.\n* $m$ is the batch size.\n* $d$ is the no. of surrogate parameters.\n\n**Running Time.** We also report the running time of the participating baselines with and without IGNITE in Table 1 in the PDF attached to our summary response.  This is based on a NVIDIA RTX 3090 GPU with CUDA 11.8 system. It can be observed that with IGNITE, the training time increases by 14.91% on average, which is an acceptable overhead in exchange for the significant performance gain. In terms of memory, we also observe that IGNITE incurs a negligible GPU memory overhead.\n\nWe hope the reviewer will consider increasing the rating if our response above has addressed all questions sufficiently. We are happy to discuss further if the reviewer has any follow-up questions for us.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3YjbpCTw7u", "reviewer": "Reviewer_WGWE", "summary": "The paper proposed IGNITE, a promising method for solving out-of-distribution issue in offline training by introducing model sharpness into the training loss of the surrogate as a regularizer. The key innovation of IGNITE lies on incorperating sharpness-aware minimization into offline training and developing theoretical analysis to show that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Experiments on benchmark datasets show the promising performance.", "questions": "1. I recommend the authors to extent the contends on the related works of sharpness-aware minimization.\n2. Can the authors explain the potential reason on the performance drop shown in Table 4 and 5?\n3. Can the authors show the sharpness of objectives on unseen data before and after using your algorithm?", "limitations": "I don't think there are any negative societal impacts of this work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. Well motivated.  \n2. Easy to follow. I like the style of writing.\n3. Experimental results seem promissing (9.6% performance boost).\n4. Solid theoretical analysis are provided.", "weaknesses": "Although the paper has many strengths, there are still some weaknesses:\n1. Sharpness-aware minimization (SAM) is widely explored in many fields. I think this submission lacks sufficient introducing of SAM, especially on its development. Take some recent works as an example:  \n\nEnhancing sharpness-aware optimization through variance suppression.  (NeurIPS'23)  \nNormalization layers are all that sharpness-aware minimization needs.  (NeurIPS'23)  \nDomain-Inspired Sharpness-Aware Minimization Under Domain Shifts.  (ICLR'24)  \nLocally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization.  (ICML'24)  \n\n2. The gain on performance is not always stable, as shown in the Table 4 and 5.  \n3. Lacking sharpness visualization of objectives on unseen data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for recognizing the strengths of our work with an acceptance rating.\n\n**Q1. Improving the introduction of SAM.**\n \nWe appreciate the reviewer's insightful recommendation. We will cite the suggested works and discuss their impact on the development of SAM in our revision. We will also emphasize in the discussion that a key difference between such development and our work is that we provide a non-trivial adaptation of the proving technique in SAM to characterize and control the surrogate sharpness (rather than its loss sharpness). The intuition is that minimizing the loss sharpness guarantees that, on average, a single prediction at a randomly selected input in the OOD regime will have low error. However, such errors can accumulate along a gradient search process which results from multiple, consecutive predictions. Fortunately, our intuition in Figure 1 (as elaborated in lines 130-136) suggests that such error accumulation can be mitigated by controlling the surrogate sharpness. This is also verified via our empirical studies reported in Table 2 -- see also our response to Q1 of **Reviewer xjU2**.\n\n**Q2. Performance drop in Tables 4 and 5.**\n\nIt is expected that the results for the 80th and 50th percentiles are less impressive than those for the 100th percentile. This is because the baseline methods are inevitably less effective at lower percentiles. As seen in Tables 4 and 5, the solutions for the 80th and 50th percentiles from the base methods often show minimal improvement over the empirical best.\n\n**Q3. Sharpness of objectives on unseen data before and after using IGNITE.**\n\nWe conducted an experiment to compare the values of surrogate sharpness — $\\rho || \\nabla_\\omega h(\\omega) ||$ in Eq. (10) — with and without IGNITE. These surrogate sharpness values were computed on unseen data, which are design candidates found by the GA and REINFORCE baselines before and after being regularized with IGNITE. These are reported in the table below (Table 2 in the attached PDF in our summary response)\n\n| Baseline             | Ant  | Tf-bind-10 |\n|----------------------|------|------------|\n| GA w/o IGNITE        | 1.88 | 1.07       |\n| GA w/ IGNITE         | 1.69 | 0.63       |\n| REINFORCE w/o IGNITE | 0.18 | 0.24       |\n| REINFORCE w/ IGNITE  | 0.09 | 0.14       |\n\nThe reported results thus demonstrate that our method, IGNITE, helps decrease the sharpness of the surrogate model on unseen data (as expected).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tkWcBh2Voh", "reviewer": "Reviewer_tYoe", "summary": "This paper introduces a model-agnostic regularization method that reduces surrogate model sharpness to improve generalization in offline optimization. By incorporating a surrogate sharpness measure into the training loss, they provide theoretical proof and extensive experimental validation showing that this approach enhances performance on unseen data, achieving up to a 9.6% improvement in various optimization tasks. The proposed algorithm, IGNITE, demonstrates the effectiveness of this technique across different surrogate models and tasks.", "questions": "Please see the weaknesses part.", "limitations": "Please see the weaknesses part.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper introduces a novel concept of surrogate sharpness for offline optimization, providing a new robust optimization approach that differs from previous loss-based sharpness-aware minimization methods.\n2. Based on surrogate sharpness, the authors propose a new algorithm, IGNITE, and provide corresponding theoretical analysis, demonstrating its ability to bound the worst-case generalized surrogate sharpness.\n3. The proposed method can be combined with other offline optimization techniques, consistently enhancing generalization performance when integrated.", "weaknesses": "1. The Introduction section does not clearly emphasize the difference between surrogate sharpness and loss sharpness. Figure 1 could be positioned earlier in the paper. Additionally, the annotations and explanations in Figure 1a are somewhat confusing; the concepts of \\(\\sigma\\) and \\(\\sigma^*\\) are not adequately explained in the figure and caption, which could lead to misunderstandings.\n2. The Theoretical Analysis section (Section 4) should include more insightful discussions based on Theorems 1 and 2, ideally tying them to the results in the experimental section. The formatting of Equation 16 could also be improved.\n3. In the experimental section, the authors mention that the IGNITE algorithm introduces five hyperparameters, but the ablation study only analyzes the effects of a subset of these hyperparameters. Many experimental results are placed in the supplementary materials, but they are not referenced in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for recognizing our contribution with an acceptance rating.\n\n**Q1. Introduction Improvement.**\n\nWe would like to thank the reviewer for the suggestion. We will clearly emphasize the difference between surrogate sharpness and loss sharpness in the Introduction section. This will allow us to highlight an important intuition on the key difference between a direct application of SAM and its non-trivial adaptation to control the surrogate sharpness (rather than its loss sharpness) in offline optimization.\n\nThe intuition is that minimizing the loss sharpness guarantees that, on average, a single prediction at a randomly selected input in the OOD regime will have low error. However, such errors can accumulate along a gradient search process which results from multiple consecutive predictions. Fortunately, our intuition in Figure 1 (as elaborated in lines 130-136) suggests that such error accumulation can be mitigated by keeping the surrogate sharpness small during training. This is also verified via our empirical studies reported in Table 2. We will move Figure 1 to the Introduction to provide an earlier illustration of this insight.\n\n**Notation Clarification.** We will also provide a more detailed explanation of $\\delta$ and $\\delta_\\ast$ in the caption to avoid any confusion. Specifically, $\\delta_\\ast = \\mathrm{argmax}_{||\\delta||_2 \\leq \\rho} |\\mathbb{E}[g(x; \\omega + \\delta)] - \\mathbb{E}[g(x; \\omega)]| $\n\n**Q2. More insightful discussion.** \n\nWe will include a more insightful discussion in the revision. Concretely, we will emphasize the earlier intuition that instead of using a direct application of SAM to minimize the loss sharpness, we adapt SAM to select a surrogate (from those that fit the offline data equally well) with low sharpness.\n\nAccording to our intuition illustrated in Figure 1 and described in lines 130-136, the optima of surrogates with low sharpness tends to be closer to the oracle optima. This means a well-controlled sharpness of the surrogate landscape (rather than its loss landscape) will help the corresponding gradient search accumulate less error.\n\nThis is formalized in Eq. (5) which applies a low-sharpness constraint $\\mathcal{R}\\_\\mathcal{X} (\\omega) \\leq \\epsilon^{'}$ to the surrogate fitting loss. Solving Eq. (5) therefore requires upper-bounding $\\mathcal{R}_\\mathcal{X}(\\omega)$ with a tractable function — see Eq. (6) and Theorem 2.\n\nAs Theorem 2 depends on Assumptions 1 & 2, we put forward Theorem 1 to assert that both assumptions can be satisfied with implementable choices of the surrogate, indicating that those are not strong assumptions.\n\nWe will also improve the formatting of Eq. (16) for better readability.\n\n**Q3. Ablation studies.** \n\nDue to space constraints, we only presented ablation studies for a subset of the most important hyperparameters. We observed that the performance is less sensitive to changes in the others that were omitted from the ablation studies. In the revision, we will include a more comprehensive ablation study in the appendix and ensure that all previously unreferenced experimental results are appropriately referred to in the main text.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3Gzgimb59h", "reviewer": "Reviewer_qZmG", "summary": "The paper studies the problem of offline optimization for material design problems. The paper proposes a model-agnostic method that changes the parameters of a model by constraining the sharpness of the model's predictions. Since the model generates smoother predictions, the error between the predictions and the oracle tends to be smaller and therefore the performance of the offline model is potentially more accurate. The paper gives a theoretical analysis and shows that the empirical sharpness can bound the theoretical sharpness. The paper applies the method to various offline optimization method and shows improvements in most of the cases.", "questions": "Please address the weaknesses part.", "limitations": "Limitations are properly addressed. No obvious negative societal impact.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The paper starts with strong intuition, proposes a general optimization objective, and proposes a feasible algorithm to solve the objective efficiently. The paper also gives a theoretical analysis that connects the empirical objective to the theoretical sharpness.\n\n2. The proposed method is model agnostic and therefore the method can be applied to many existing offline optimization methods to get improved performance.\n\n3. The proposed method is shown to achieve good performance across a variety of tasks and is applied with various offline optimization methods.", "weaknesses": "1. After certain approximations, the objective is optimized using BDMM. It would be interesting to analyze the specific structures of the objective and see if more effective optimization techniques can be applied. This is also related to the model-agnostic feature of the proposed method. While having the model-agnostic property is convenient, it may also miss the specific structures of the problems/models so that the performance is not ideal. Also, there is no empirical evidence showing whether the optimization is effective or how well the optimization converges in practice.\n\n2. The comparisons with baseline methods are only compared for a limited number of tasks. To fully demonstrate the effectiveness, I think a thorough comparison as done in Table 1 would be necessary.\n\n3. The paper very briefly mentions the background of material design. I think a more comprehensive introduction could be beneficial and probably the paper should add detailed descriptions of the tasks and datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for recognizing our contribution with an acceptance rating.\n\n**Q1.** \n\n**A. More effective optimization techniques.** We agree with the reviewer that an in-depth investigation of the specific structure of the objective could reveal a more effective optimization technique. We will explore this direction in a follow-up of the current work.\n\n**B. Convergence and effectiveness of optimization.** According to our experiment, despite the use of a relatively simple BDMM to solve Eq. (12), we have achieved significant improvement in most cases. To demonstrate the convergence of the optimization algorithm, we have plotted the training loss and the sharpness value (plotting the $|| \\nabla_\\omega h(\\omega) ||$ value) during the surrogate fitting process. This is based on an experiment using GA and REINFORCE baselines on Ant and Dkitty tasks. The results are illustrated in Figure 1 in the attached PDF of our summary response.\n\nThese results reveal that BDMM helps decrease both the training loss and sharpness value of the surrogate model during the training phase. This indicates that BDMM is effective and the optimization converges well in practice. Furthermore, our method, IGNITE, can be seamlessly integrated with other, more robust optimization techniques to solve Eq. (12).\n\n**Q2. Comparison in Table 1.** \n\nAs explained in the paper, we excluded tasks known for their high inaccuracy and noise in oracle functions from prior works (ChEMBL, Hopper, and Superconductor), as well as those considered excessively expensive to evaluate (NAS). Nevertheless, for a more thorough comparison, we have conducted an experiment running the GA and REINFORCE baselines — with and without being regularized by IGNITE — in Superconductor and Chembl tasks. These results are shown in the table below (Table 3 in the attached PDF in our summary response)\n| Baseline             | Superconductor        | Chembl                |\n|----------------------|-----------------------|-----------------------|\n| GA w/o IGNITE        | 0.514 ± 0.021         | 0.635 ± 0.005         |\n| GA w/ IGNITE         | 0.517 ± 0.011 (+0.3%) | 0.640 ± 0.009 (+0.5%) |\n| REINFORCE w/o IGNITE | 0.471 ± 0.011         | 0.634 ± 0.001         |\n| REINFORCE w/ IGNITE  | 0.492 ± 0.015 (+2.1%) | 0.636 ± 0.008 (+0.2%) |\n\nThe table illustrates that IGNITE also boosts the performance of baselines in noisy tasks. With more time and computation resources, we will conduct and include a thorough comparison of these tasks in the appendix.\n\n**Q3. Background Discussion.** \n\nWe thank the reviewer for this suggestion. We will revise to include a more detailed description of the tasks and datasets. It will summarize the details of those datasets and tasks in Design-Bench [1]. We will include this information in the appendix.\n\n[1] Gao, Chen, et al. \"Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization.\" arXiv preprint arXiv:2202.08450 (2022).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pzcMZX4h0U", "reviewer": "Reviewer_CQYH", "summary": "This paper introduce a sharpness-aware optimization to improve out-of-distribution generalization. While inspired by SAM, the major difference is that this work considers the sharpness of predictor outputs rather than loss landscape. With the proposed notion of sharpness, practical algorithm, IGNITE, is developed through an empirical approximation (Eq. 6) and Taylor approximation of the sharpness constraint.", "questions": "- As illustrated in Figure 1, the intuition is that if the oracle is included in the perturbation neighborhood, a smoother predictor tends to have smaller error. However, Table 1 observes that IGNITE applied to some sub-optimal predictors, for example REINFORCE in Ant Morphology, it still achieves good performance. Since the suboptimal prediction performance may imply that $\\hat{\\omega}$ is not proximal to $\\omega^\\star$ hence the local approximation may break in practice. I am curious that why IGNITE should still work in this case, I appreciate if the authors could provide some high-level intuitions.", "limitations": "See above.\n\n\n---\npost-rebuttal: -> 5", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- This material is well written and easy to follow. \n\n- Generalization-aware optimizer is very relevant.\n\n- Technical development appears viable and clear.", "weaknesses": "- It is not immediately clear, on an intuition level, why this new notion of sharpness is better than loss sharpness in SAM [8].\n\n- Table 2 only presents SAM on two algorithms (REINFORCE and GA) with no error interval reported. A more throughout benchmarking could improve the empirical evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for recognizing the clarity and viability of our technical development. Your questions are addressed below.\n\n**Q1. Why the new notion of surrogate sharpness is better than loss sharpness?**\n\nTo understand this, note that a surrogate that minimizes its loss sharpness might, on average, have a low prediction error for a single random input sampled from the OOD regime. However, offline optimization requires multiple predictions at consecutive inputs that occur during the gradient search process, where errors can accumulate and lead to suboptimal input candidates.\n\nTo mitigate such error accumulation, an alternative approach is to select a surrogate (from those that fit the offline data equally well) with low sharpness. We refer to this as surrogate sharpness. According to our intuition illustrated in Figure 1 and described in lines 130-136, the optima of surrogates with low sharpness tends to be closer to the oracle optima.\n\nThis means a well-controlled sharpness of the surrogate landscape (rather than its loss landscape) will help the corresponding gradient search accumulate less error. This is formalized in Eq. (5) which applies a low-sharpness constraint $\\mathcal{R}\\_\\mathcal{X}(\\omega) \\leq \\epsilon^{'}$ to the surrogate fitting loss. Solving Eq. (5) therefore requires upper-bounding $\\mathcal{R}\\_\\mathcal{X}(\\omega)$ with a tractable function — see Eq. (6) and Theorem 2.\n\nSuch insight is also confirmed via an empirical comparison between IGNITE and a direct application of SAM to minimize the loss sharpness of the surrogate. The reported results in Table 2 show a clear improvement of IGNITE over SAM, which confirms that controlling surrogate sharpness is more beneficial than minimizing loss sharpness.\n\n\n**Q2. More thorough comparison with SAM in Table 2.** \n\nWe further run SAM with 2 other baselines BO-qEI and CbAS. In addition, we revise Table 2 by reporting the error interval as in the table below (see Table 4 in the PDF attached to our summary response):\n\n\n| Algorithms         | Ant                   | D'Kitty               | TF Bind 8             | TF Bind 10            |\n|--------------------|-----------------------|-----------------------|-----------------------|-----------------------|\n| REINFORCE          | 0.255 ± 0.036         | 0.546 ± 0.208         | 0.929 ± 0.043         | 0.635 ± 0.028         |\n| REINFORCE + IGNITE | 0.282 ± 0.021 (+2.7%) | 0.642 ± 0.160 (+9.6%) | 0.944 ± 0.030 (+1.5%) | 0.670 ± 0.060 (+3.5%) |\n| REINFORCE + SAM    | 0.266 ± 0.030 (+1.1%) | 0.625 ± 0.182 (+7.9%) | 0.940 ± 0.035 (+1.1%) | 0.637 ± 0.037 (+0.2%) |\n| GA                 | 0.303 ± 0.027         | 0.881 ± 0.016         | 0.980 ± 0.016         | 0.651 ± 0.033         |\n| GA + IGNITE        | 0.320 ± 0.044 (+1.7%) | 0.886 ± 0.017 (+0.5%) | 0.985 ± 0.010 (+0.5%) | 0.653 ± 0.043 (+0.2%) |\n| GA + SAM           | 0.310 ± 0.044 (+0.7%) | 0.868 ± 0.014 (-1.3%) | 0.982 ± 0.015 (+0.2%) | 0.662 ± 0.041 (+1.1%) |\n| CbAS               | 0.854 ± 0.042         | 0.895 ± 0.012         | 0.919 ± 0.044         | 0.635 ± 0.041         |\n| CbAS + IGNITE      | 0.859 ± 0.039 (+0.5%) | 0.900 ± 0.015 (+0.5%) | 0.921 ± 0.042 (+0.2%) | 0.652 ± 0.055 (+1.7%) |\n| CbAS + SAM         | 0.853 ± 0.033 (-0.1%) | 0.897 ± 0.013 (+0.2%) | 0.905 ± 0.053 (-1.4%) | 0.637 ± 0.023 (+0.2%) |\n| BO-qEI             | 0.812 ± 0.000         | 0.896 ± 0.000         | 0.787 ± 0.112         | 0.628 ± 0.000         |\n| BO-qEI + IGNITE    | 0.812 ± 0.000 (+0.0%) | 0.896 ± 0.000 (+0.0%) | 0.843 ± 0.109 (+0.3%) | 0.628 ± 0.000 (+0.0%) |\n| BO-qEI + SAM       | 0.812 ± 0.000 (+0.0%) | 0.896 ± 0.000 (+0.0%) | 0.763 ± 0.098 (-2.4%) | 0.619 ± 0.022 (-0.9%) |\n\n\nThe reported performance of the baselines in Table 2 shows that IGNITE surpasses SAM in terms of performance improvement over baselines across most tasks. It can also be observed that IGNITE in general has a smaller error interval than SAM.\n\n\n**Q3. Why does IGNITE still work on base optimizers with poor performance?**\n\n\nWe note that each offline optimizer often comprises a predictor component (surrogate) and a search component that navigates the input space using the surrogate gradient to recommend a design candidate. The poor performance of an optimizer therefore does not mean its surrogate is not sufficiently proximal to the oracle.\nInstead, it is possible that the surrogate fits well on the offline data and is reasonably proximal to the oracle (though not perfect) but the search does not recognize well areas where the surrogate prediction is not reliable and gets misled into exploring more in those regions, resulting in sub-optimal performance. In such cases, our insight holds and IGNITE can still improve the surrogate by minimizing its surrogate sharpness. The improved surrogate is more reliable and will lessen the impact of the ineffective search as observed throughout Table 1.\n\nWe hope the reviewer would consider increasing the rating if our response above has sufficiently addressed all questions. We will be happy to discuss further if the reviewer has any follow-up questions. Thank you for the detailed feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OfNlzQ9XJC", "reviewer": "Reviewer_qc6B", "summary": "This paper proposes a novel model-agnostic approach to enhance offline optimization methods by incorporating surrogate gradient norms.  The paper provides a thorough review of existing literature, a clear problem definition, and detailed descriptions of the proposed methods and their implementation. The experiments are well-designed, covering various benchmarks and baselines, and the results are robustly analyzed.", "questions": "See above.", "limitations": "Yes. In Appendix H", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- a new regularizer based on surrogate sharpness, characterized by the surrogate's maximum output change under low-energy parameter perturbation. This regularizer is designed to be model-agnostic, making it broadly applicable across different surrogate and search models.\n- a practical approximation that reduces the surrogate sharpness measurement to a function of the surrogate’s gradient norm. This allows the optimization task to be transformed into a constrained optimization problem, which can be solved using existing optimization solvers.\n- a theoretical analysis demonstrating that reducing surrogate sharpness on an offline dataset provably reduces its generalized sharpness on unseen data.\n- extensive experimentation on a diverse range of optimization tasks shows that reducing surrogate sharpness often leads to performance improvements.", "weaknesses": "- As the paper mentioned that they draw inspiration from the sharpness-aware minimization (SAM), the paper’s novelty could be better articulated in the context of existing work on surrogate model regularization and sharpness-aware optimization. Specifically, discuss how the proposed surrogate sharpness measure offers advantages over sharpness-aware minimization (SAM).\n- Assumption 2 (positive minimum eigenvalue of the parameter Hessian) might be too restrictive. It would be more helpful if the authors can provide more intuitive explanations or empirical evidence to justify these assumptions and discuss the implications if these assumptions are violated and how the method’s performance might be affected.\n- The proposed method involves computationally expensive operations, such as gradient norm calculations and constrained optimization. I'm wondering how expensive the proposed method is compared to existing base approaches in terms of the training time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for recognizing the strengths of our work with an acceptance rating.\n\n**Q1. How surrogate sharpness offers advantages over loss sharpness (SAM).**\n\nWe will highlight below an important intuition on the key difference between a direct application of SAM and its non-trivial adaptation to control the surrogate sharpness (rather than its loss sharpness) in offline optimization.\n\nThe intuition is that minimizing the loss sharpness guarantees that on average, a single prediction at a randomly selected input in the OOD regime will have low error. However, such errors can accumulate along a gradient search process which results from multiple predictions on sequentially dependent inputs. Fortunately, our intuition in Figure 1 (as elaborated in lines 130-136) suggests that such error accumulation can be mitigated by controlling the surrogate sharpness. The idea is that with a sufficiently large perturbation radius, the oracle will be within the perturbation neighborhood. As surrogates with small sharpness will, by definition, not have their predictions changed substantially within its perturbation neighborhood (including the oracle), their optima will be close to those of the oracle.\n\nAs such, keeping the surrogate sharpness small while fitting it to the offline data will lessen the impact of error accumulation in the search phase. This is also verified via our empirical studies reported in Table 2 which compares the impact of using surrogate and loss sharpness on surrogate conditioning for offline optimization.\n\n\n**Q2. Is Assumption 2 strong?**\n\nWe note that Assumption 2 can be satisfied for a class of surrogates established in Theorem 1 (see its proof in Appendix A). Since there exists an implementable surrogate formulation for which Assumption 2 holds, we believe it is not a strong assumption.\n\n**Q3. Complexity Overhead of IGNITE.**\n \nTo analyze the computational complexity of IGNITE, we break down the complexity of each step in Algorithm 1.\n\n1. **Initialization (Line 1):**\n \n    * Initializing $\\omega^{(1)} \\leftarrow \\omega^{(0)}$ and $\\lambda^{(1)} \\leftarrow \\lambda: O(1)$ each.\n\n2. **Main Loop (Line 2-12):**\n\n     * The loop runs for $T$ iterations. Thus, the complexity of the main loop will be multiplied by $T$.\n\n3. **Sampling (Line 3):**\n\n     * Sampling a batch $\\mathcal{B} = \\{(\\mathbf{x}\\_i, z\\_i)\\}\\_{i=1}^m \\sim \\mathcal{D}$: $O(m)$.\n\n4. **Computing $\\hat{z}_i$ (Line 4):**\n\n     * Evaluating the surrogate model $g(\\mathbf{x}\\_i; \\omega^{(t)})$ for each $i \\in [m]$. Assuming the surrogate model evaluation has a computational complexity of $C\\_g = O(d)$ per sample where $d$ is the number of surrogate parameters, the total complexity is $O(m \\cdot d)$.\n\n5. **Computing $g_1$ and $g_2$ (Line 5-6):**\n\n     * Computing gradients $\\nabla\\_\\omega \\ell (\\hat{z}\\_i, z\\_i)$ and $\\nabla\\_\\omega \\hat{z}\\_i$ have complexities $O(C\\_\\ell)$ and $O(C\\_{\\hat{z}})$ respectively per sample where $C\\_\\ell = C\\_{\\hat{z}} = O(d)$. Therefore, the total complexities are $O(m \\cdot d)$.\n\n6. **Computing $\\hat{\\omega}$ (Line 7):**\n\n     * This involves simple vector operations with complexity $O(d)$, where $d$ is the dimensionality of $\\omega$.\n\n7. **Computing $g_3$ (Line 8):**\n\n     * Similar to lines 4 and 6, involving evaluating the surrogate and gradient computations, with complexity $O(m \\cdot C\\_g + m \\cdot C\\_{\\hat{z}}) = O(m \\cdot d)$.\n\n8. **Computing $g^{(t)}$ (Line 9):**\n     * Vector operations involving addition and scalar multiplication with complexity $O(d)$.\n\n9. **Updating $\\omega$ (Line 10):**\n\n      * Updating $\\omega$ involves simple subtraction operations with complexity $O(d)$.\n\n10. **Updating $\\lambda$ (Line 11):**\n\n      * Updating $\\lambda$ is an $O(1)$ operation\n\n**Overall Complexity**\n\nConsidering the above steps, the most computationally expensive parts are the gradient computations in lines 5, 6, and 8. Thus, the overall complexity per iteration is:\n\n$ O(2m \\cdot C\\_g + m \\cdot C\\_\\ell + 2m \\cdot C\\_{\\hat{z}})$\n\nSince this loop runs for $T$ iterations, the total complexity is:\n\n$ O(T \\cdot (2m \\cdot C\\_g + m \\cdot C\\_\\ell + 2m \\cdot C\\_{\\hat{z}})) $\n\nFurthermore, we have the total complexity of the original baseline is: \n\n$O(T \\cdot (m \\cdot C\\_g + m \\cdot C\\_\\ell))$ \n\nThereby, IGNITE will include an additional complexity: \n\n$O(T \\cdot (m \\cdot C\\_g + 2m \\cdot C\\_{\\hat{z}} )) = O(Tmd)$\n\nwhere:\n\n* $T$ is the number of iterations.\n\n* $m$ is the batch size.\n\n* $C\\_g = O(d)$ is the complexity of evaluating the surrogate model.\n\n* $C\\_\\ell = O(d)$ is the complexity of computing the loss gradient.\n\n* $C\\_{\\hat{z}} = O(d)$ is the complexity of computing the gradient of the surrogate output with respect to its parameters.\n\n* $d$ is the no. of surrogate parameters.\n\n\nThe empirical training time of the participating baselines with and without IGNITE is reported in the table below (see Table 1 in the PDF attached to our summary response). This is based on a NVIDIA RTX 3090 GPU with CUDA 11.8 system.\n\n| Algorithms         | Ant               | D'Kitty           | TF Bind 8         | TF Bind 10        |\n|--------------------|-------------------|-------------------|-------------------|-------------------|\n| REINFORCE          | 172.08s           | 252.33s           | 477.09s           | 372.95            |\n| REINFORCE + IGNITE | 194.02s (+12.75%) | 275.15s (+9.04%)  | 582.28 (+22.05%)  | 437.38s (+17.28%) |\n| GA                 | 69.99s            | 168.81s           | 149.63s           | 364.16s           |\n| GA + IGNITE        | 85.15s (+21.66%)  | 191.83s (+13.64%) | 181.71s (+21.44%) | 369.29s (+1.41%)  |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SatbWvOm0l", "reviewer": "Reviewer_kVLN", "summary": "The article presents a method to improve offline optimization techniques by integrating the concept of model sharpness into the training.  A constraint is introduced that limits the model sharpness to not exceed a user-specified threshold.", "questions": "see weakness", "limitations": "NA", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The approach is model-agnostic, making it applicable across different types of models and optimization tasks without needing specific adjustments for each model or task.\n2. The method is backed by a solid theoretical analysis, providing a robust framework for understanding and predicting the behavior of the regularization effect on offline optimization.\n3. The method has been empirically tested on a variety of tasks, showing significant performance improvements, thus validating the theoretical predictions.", "weaknesses": "1. The effectiveness of the method heavily depends on the correct setting of hyperparameters like the sharpness threshold, which can be tricky to optimize without extensive experimentation.\n2. The computation complexity is increased. Can the author give the complexity analysis?\n3. How to ensure the proposed method converge to a stationary point of the original problem?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for recognizing the strengths of our work with an acceptance rating.\n\n**Q1. Hyper-parameter Tuning.**\n\nWe agree with the reviewer that hyperparameter tuning is important to achieve best performance. This is also true in the broader context of machine learning (ML). Most ML methods require some forms of hyperparameter tuning to achieve good performance. In our case, we adopt the hyperparameter tuning method from Design-Bench [1]. Specifically, we use the previously reported best hyperparameters for each baseline and only tune the additional hyperparameters introduced by the IGNITE regularizer. With a small set of hyperparameters, the tuning cost is not prohibitive.\n\n[1] Gao, Chen, et al. \"Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization.\" arXiv preprint arXiv:2202.08450 (2022).\n\n**Q2. Complexity Overhead of IGNITE.**\n\nTo analyze the computational complexity of IGNITE, we break down the complexity of each step in Algorithm 1.\n\n1. **Initialization (Line 1):**\n \n    * Initializing $\\omega^{(1)} \\leftarrow \\omega^{(0)}$ and $\\lambda^{(1)} \\leftarrow \\lambda: O(1)$ each.\n\n2. **Main Loop (Line 2-12):**\n\n     * The loop runs for $T$ iterations. Thus, the complexity of the main loop will be multiplied by $T$.\n\n3. **Sampling (Line 3):**\n\n     * Sampling a batch $\\mathcal{B} = \\{(\\mathbf{x}\\_i, z\\_i)\\}\\_{i=1}^m \\sim \\mathcal{D}$: $O(m)$.\n\n4. **Computing $\\hat{z}_i$ (Line 4):**\n\n     * Evaluating the surrogate model $g(\\mathbf{x}\\_i; \\omega^{(t)})$ for each $i \\in [m]$. Assuming the surrogate model evaluation has a computational complexity of $C\\_g = O(d)$ per sample where $d$ is the number of surrogate parameters, the total complexity is $O(m \\cdot d)$.\n\n5. **Computing $g_1$ and $g_2$ (Line 5-6):**\n\n     * Computing gradients $\\nabla\\_\\omega \\ell (\\hat{z}\\_i, z\\_i)$ and $\\nabla\\_\\omega \\hat{z}\\_i$ have complexities $O(C\\_\\ell)$ and $O(C\\_{\\hat{z}})$ respectively per sample where $C\\_\\ell = C\\_{\\hat{z}} = O(d)$. Therefore, the total complexities are $O(m \\cdot d)$.\n\n6. **Computing $\\hat{\\omega}$ (Line 7):**\n\n     * This involves simple vector operations with complexity $O(d)$, where $d$ is the dimensionality of $\\omega$.\n\n7. **Computing $g_3$ (Line 8):**\n\n     * Similar to lines 4 and 6, involving evaluating the surrogate and gradient computations, with complexity $O(m \\cdot C\\_g + m \\cdot C\\_{\\hat{z}}) = O(m \\cdot d)$.\n\n8. **Computing $g^{(t)}$ (Line 9):**\n     * Vector operations involving addition and scalar multiplication with complexity $O(d)$.\n\n9. **Updating $\\omega$ (Line 10):**\n\n      * Updating $\\omega$ involves simple subtraction operations with complexity $O(d)$.\n\n10. **Updating $\\lambda$ (Line 11):**\n\n      * Updating $\\lambda$ is an $O(1)$ operation\n\n**Overall Complexity**\n\nConsidering the above steps, the most computationally expensive parts are the gradient computations in lines 5, 6, and 8. Thus, the overall complexity per iteration is:\n\n$ O(2m \\cdot C\\_g + m \\cdot C\\_\\ell + 2m \\cdot C\\_{\\hat{z}})$\n\nSince this loop runs for $T$ iterations, the total complexity is:\n\n$ O(T \\cdot (2m \\cdot C\\_g + m \\cdot C\\_\\ell + 2m \\cdot C\\_{\\hat{z}})) $\n\nFurthermore, we have the total complexity of the original baseline is: \n\n$O(T \\cdot (m \\cdot C\\_g + m \\cdot C\\_\\ell))$ \n\nThereby, IGNITE will include an additional complexity: \n\n$O(T \\cdot (m \\cdot C\\_g + 2m \\cdot C\\_{\\hat{z}} )) = O(Tmd)$\n\nwhere:\n\n* $T$ is the number of iterations.\n\n* $m$ is the batch size.\n\n* $C\\_g = O(d)$ is the complexity of evaluating the surrogate model.\n\n* $C\\_\\ell = O(d)$ is the complexity of computing the loss gradient.\n\n* $C\\_{\\hat{z}} = O(d)$ is the complexity of computing the gradient of the surrogate output with respect to its parameters.\n\n* $d$ is the no. of surrogate parameters.\n\n\nThe empirical training time of the participating baselines with and without IGNITE is reported in the table below (see Table 1 in the PDF attached to our summary response). This is based on a NVIDIA RTX 3090 GPU with CUDA 11.8 system.\n\n| Algorithms         | Ant               | D'Kitty           | TF Bind 8         | TF Bind 10        |\n|--------------------|-------------------|-------------------|-------------------|-------------------|\n| REINFORCE          | 172.08s           | 252.33s           | 477.09s           | 372.95            |\n| REINFORCE + IGNITE | 194.02s (+12.75%) | 275.15s (+9.04%)  | 582.28 (+22.05%)  | 437.38s (+17.28%) |\n| GA                 | 69.99s            | 168.81s           | 149.63s           | 364.16s           |\n| GA + IGNITE        | 85.15s (+21.66%)  | 191.83s (+13.64%) | 181.71s (+21.44%) | 369.29s (+1.41%)  |\n\n**Q3. How to ensure convergence to an optima of the oracle?**\n\nWe would like to emphasize that finding the optima of the oracle is an ill-posed task in the offline context since there might exist infinitely many functions that fit the offline data perfectly but will have different behaviors on the unseen input regions.\n\nInstead, the ultimate goal of offline optimization is to focus the search on a safe region where the output prediction often does not change substantially across surrogate candidates. This can be achieved by finding surrogates with low sharpness. Their gradient will help shape a safe search region within which the surrogate optimum is close to the oracle’s local optimum (not necessarily a stationary point of the oracle). Our contribution here is to develop a rigorous framework to characterize and (provably) control the surrogate sharpness via a non-trivial adaptation of loss sharpness.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank the AC for securing seven reviews with high quality. 

We thank **Reviewers WGWE, tYoe, qZmG, qc6B, and kVLN** for their accepting scores.
 
We thank **Reviewers xjU2 and CQYH** for the detailed questions, which help us highlight better the key contribution of our work.  

Our responses to the reviewers’ main questions are summarized below.

1. We elaborated more on the advantage of surrogate sharpness over loss sharpness, which highlights the conceptual novelty of our work (**tYoe**,**qc6B**, **xjU2**, **CQYH**).
2. We showed that Assumption 2 is not a strong (**qc6B**, **xjU2**) and provided detailed complexity analysis of IGNITE (**qc6B**, **kVLN**, **xjU2**).
3. We showed the reduced surrogate sharpness on unseen data after conditioning with  IGNITE (**WGWE**).
4. We showed the convergence and effectiveness of the BDMM algorithm by plotting the training loss and the sharpness value during the surrogate fitting process (**qZmG**).
5. We articulated on the tightness of the generalization bound (**xjU2**).
6. We explained why IGNITE can help improve even base optimizers with poor performance (**CQYH**).

**Our additional experiment results (in response to some questions from the reviewers) are detailed in the attached PDF.**

We welcome any follow-up questions from the reviewers regarding our rebuttal. We hope that, based on our detailed responses, the reviewers will consider increasing their scores if their concerns have been sufficiently addressed.

Meta-review: This paper proposes a technique called IGNITE to regularize surrogate models used in black-box optimization. IGNITE introduces a sharpness constraint on the surrogate model, that computes the maximum distance between the expected outputs of the surrogate model using the current weights $\omega$ and perturbed weights $\omega + \delta$ for perturbations in a neighborhood $\\| \delta \\|_2 \leq \rho$. This ensures that the worst-case prediction change across the perturbation neighborhood is small. This sharpness objective is similar to the one used in sharpness-aware minimization (Foret et al., 2021), but applied to the surrogate model output instead of the loss function.

The authors perform experiments on four tasks from DesignBench (two continuous and two discrete), applying IGNITE to 11 existing optimization techniques (including CMA-ES, REINFORCE, and others). Overall, they find that IGNITE slightly improves over the baselines in most cases. They also provide theoretical analysis showing that reducing the empirical sharpness on offline data will also reduce its sharpness over the data distribution.

This paper received positive reviews, and the authors addressed most of the common points raised by the reviewers. The reviewers thought the paper was well-written and interesting, with good experimental results and theoretical analysis. They also found the proposed method to be well-motivated and model-agnostic.

Overall, this paper brings a useful new technique for regularizing surrogate models that is relevant to the NeurIPS community.
Decision: Accept (poster)