Title: Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques
Venue: NeurIPS 2024
Accepted: yes
Keywords: Black-Box Optimization
Abstract: Offline optimization has recently emerged as an increasingly popular approach to mitigate the prohibitively expensive cost of online experimentation. The key idea is to learn a surrogate of the black-box function that underlines the target experiment using a static (offline) dataset of its previous input-output queries. Such an approach is, however, fraught with an out-of-distribution issue where the learned surrogate becomes inaccurate outside the offline data regimes. To mitigate this, existing offline optimizers have proposed numerous conditioning techniques to prevent the learned surrogate from being too erratic. Nonetheless, such conditioning strategies are often specific to particular surrogate or search models, which might not generalize to a different model choice. This motivates us to develop a model-agnostic approach instead, which incorporates a notion of model sharpness into the training loss of the surrogate as a regularizer. Our approach is supported by a new theoretical analysis demonstrating that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our analysis extends existing theories from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate on training data, providing a new perspective on sharpness regularization. Our extensive experimentation on a diverse range of optimization tasks also shows that reducing surrogate sharpness often leads to significant improvement, marking (up to) a noticeable 9.6% performance boost. Our code is publicly available at https://github.com/cuong-dm/IGNITE.

Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques

Abstract

Offline optimization has recently emerged as an increasingly popular approach to mitigate the prohibitively expensive cost of online experimentation. The key idea is to learn a surrogate of the black-box function that underlines the target experiment using a static (offline) dataset of its previous input-output queries. Such an approach is, however, fraught with an out-of-distribution issue where the learned surrogate becomes inaccurate outside the offline data regimes. To mitigate this, existing offline optimizers have proposed numerous conditioning techniques to prevent the learned surrogate from being too erratic. Nonetheless, such conditioning strategies are often specific to particular surrogate or search models, which might not generalize to a different model choice. This motivates us to develop a modelagnostic approach instead, which incorporates a notion of model sharpness into the training loss of the surrogate as a regularizer. Our approach is supported by a new theoretical analysis demonstrating that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our analysis extends existing theories from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate on training data, providing a new perspective on sharpness regularization. Our extensive experimentation on a diverse range of optimization tasks also shows that reducing surrogate sharpness often leads to significant improvement, marking (up to) a noticeable 9 . 6% performance boost. Our code is publicly available at https://github.com/cuong-dm/IGNITE .

1 Introduction

A central task in numerous scientific disciplines is to optimize for some material configuration that maximizes a certain utility metric. Previously, this would incur an expensive and repetitive experiment process that requires a huge amount of human-labor. To bypass such inefficiencies, a data-driven approach [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] has recently been adopted. Instead of laboring on a new set of expensive on-demand experimentation for each new optimization task, we can leverage past experimentation data to build a parametric model that predicts the outcome of the experiments themselves. Its parameters are tuned to fit past experimental data and then fixed while optimizing for the best input.

However, in most applications, offline data is rarely representative of the entire input space. As a result, the surrogate model's prediction is often not accurate outside the offline data regime, potentially overestimating the outputs at sub-optimal input candidates. To mitigate this, existing offline optimizers have introduced numerous regularizing strategies for either the surrogate or the search models. For example, [4, 11, 12, 13] and [3] regularize their surrogate models so that their predictions at inputs outside the offline data regime are pushed down or pushed towards a constant value. Alternatively, [1] and [2] restrict their search models to regions having certain domain-specific properties under which sampled inputs probably have high performance.

These strategies therefore depend on the specifics of either the surrogate- or the search procedures to characterize the out-of-distribution regions or the desirable domain-specific properties. Consequently, their regularization might not extend to out-of-distribution regions which are not sufficiently specified. For example, COMs [13] uses an ad-hoc specification that characterizes the out-of-distribution region in terms of inputs that are reached during the first few iterations of gradient ascent on an un-regularized surrogate. This only characterizes a local sub-region of a broader out-of-distribution data regime.

To mitigate such limitations in the regularizing behaviors of prior work, we instead aim to investigate a more generic approach that is independent of the specifics of the search or surrogate procedures. For instance, instead of regularizing the behavior of the model on certain input regions, we could impose constraints on the geometries of its loss landscape such as requiring it to be in a parameter regime that uniformly produces low loss values [14]. Such regularizing strategies can, therefore, be incorporated into existing offline optimizers as an additional regularizer to improve their performance. To substantiate this idea, we have made the following technical contributions:

1. We develop a model-agnostic regularizer (for surrogate training) based on a notion of model sharpness 2 . This is characterized in terms of the surrogate's maximum output change under lowenergy parameter perturbation (i.e., norm-bound perturbation) (Section 3.1). Intuitively, suppose a surrogate's prediction does not change substantially within a perturbation neighborhood (i.e., via adding perturbation to its parameters) that contains the oracle, its predictions are likely to be close to those of the oracle (see Fig. 1). As such, minimizing this sharpness measure can help suppress the erratic behavior of the surrogate model at out-of-distribution input.

2. We adopt a practical approximation that interestingly reduces the above surrogate sharpness measurement to a function of the surrogate's gradient norm. The surrogate training can then be augmented into a constrained optimization task whose constraint imposes a user-specified threshold on the surrogate's gradient norm. We can then solve it to acquire the optimally regularized surrogate using existing constrained optimization solvers. The high-level pseudo-code of our proposed algorithms which incorporate surrogate gradient norms to improve existing offline optimization techniques ( IGNITE ) is detailed in Algorithm 1 (Section 3.2).

3. We develop a detailed theoretical analysis to show that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our analysis extends existing theories [14] from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate on training data, providing a new perspective on sharpness regularization (Section 4).

4. We demonstrate empirically that incorporating the proposed model-agnostic regularizer into existing offline optimizers as an additional conditioning component often results in significant improvement over existing offline optimizers in most cases. This sets the first step towards a new, synergistic research direction in offline optimization that can potentially support and complement both existing and future work (Section 5).

For interested readers, a concise review of existing literature is also provided in Section 2.

2 Problem Definition and Related Works

In this section, we will concisely review the preliminaries of offline optimization. Section 2.1 provides a mathematical formulation of offline optimization, and Section 2.2 summarizes prior works.

2.1 Problem Definition

Offline optimization is a computational approach to a variety of material engineering tasks that aim to find a material construction or design that maximizes certain desirable properties. Mathematically, we assume that there is an oracle function g ( x ) that maps from a material design x ∈ X to an overall utility z = g ( x ) of its property measurements; and we need to find its maxima:

However, the key challenge here is that g ( x ) is inaccessible. Instead, we only have access to an offline dataset of observations D = { ( x i , z i ) } n i =1 where z i = g ( x i ) , which denote the past input-output queries extracted from g ( x ) in previous experiments. A direct approach to this problem is to learn a surrogate g ( x ; ω ∗ ) of g ( x ) via fitting its parameter ω ∗ to the offline dataset,

where ω denotes a parameter candidate of the surrogate and ℓ ( g ( x ; ω ) , z ) denotes the prediction loss of g ( . ; ω ) on x if its oracle output is z . The (oracle) maxima of g ( x ) is then approximated via,

Suppose the surrogate g ( x ; ω ∗ ) 's prediction is sufficiently accurate over the entire input space, solving Eq. (3) is all we need. However, in most cases, g ( x ; ω ∗ ) often predicts erratically outside the offline data regime, which in turn misleads the optimization towards sub-optimal candidates. To mitigate this, numerous surrogate or search regularizers have been proposed, as summarized next.

2.2 Related Works

Most existing offline optimization methods have focused on regularizing either the search or the (surrogate) training procedures. The main focus is to either (1) avoid exploring input regions where the surrogate's prediction is not reliable or (2) regularize the prediction behavior of the surrogate at out-of-distribution input regimes. For example, to regularize the surrogate's prediction, [13] uses input candidates found during the first few iterations of gradient updates on un-regularized models to characterize the out-of-distribution regime. The surrogate can then be re-trained with an additional regularizer that penalizes high-value predictions at those sampled input candidates.

Alternatively, [11] maximizes the normalized data likelihood to reduce prediction uncertainty, which also helps suppress erratic prediction at out-of-distribution regime. Existing techniques in model pre-training and adaptation [3] or transfer learning via co-teaching [15] can also be leveraged to enforce criteria of local smoothness that encodes a preference for conservative prediction to avoid overestimating the oracle output. Otherwise, to regularize the search procedure, [1] and [2] focus instead on learning a generative model of input candidate conditioned their oracle performance. Input candidates that likely achieve high performance can then be synthesized via conditioning the generative procedure on high-value oracle output. [1] characterizes such conditioned distribution via an adversarial zero-sum game. [12] learns a direct inverse mapping from the performance output to the input design using conditional generative adversarial network [16].

Despite their reported successes, these approaches are still limited by their ad-hoc characterization of the out-of-distribution regime. As discussed previously in Section 1, the existing characterization of out-of-distribution input is often based on the specifics of either the surrogate or the search procedures, which are not guaranteed to sufficiently characterize the entire out-of-distribution data regime. This motivates us to develop a more generic out-of-distribution characterization (Section 3) that is external to both the search and surrogate models. Such an approach can be readily incorporated into most existing offline optimizers to boost their performance (Section 5).

3 Surrogate Regularization with Sharpness Constraint

This section introduces an additional constraint that transforms the original surrogate optimization in Eq. (2) (Section 3.1) into a constrained optimization problem (COP). The constraint imposes

a user-specified upper-bound on the sharpness of the surrogate. Our proposed formulation draws inspiration from a prior work [14] that aims to minimize the sharpness of the loss function to improve generalization. However, unlike the original work in [14], where the sharpness concept is applied to the loss function, we adapt it for the surrogate's prediction. We find it more suitable since offline optimization's search procedure operates on the surrogate landscape instead of the loss landscape. Moreover, while minimizing loss sharpness [14] can ensure low error for single predictions in the OOD regime, errors may accumulate over consecutive predictions in a gradient-based search. Our insight in Section 3.1 (Fig. 1) suggests that such error accumulation can be mitigated by keeping the surrogate sharpness small during training. Furthermore, we show that the sharpness can be practically approximated in terms of the surrogate's gradient norm, which is more tractable. This allows for direct adoption of existing constrained optimization algorithms [17] to effectively solve for the desired optimally regularized surrogate (Section 3.2).

(a)

(b)

W1'sperturbationneighborhood

g(x;∞+8):

g(x;∞):

★

optimalsolutions

w2'sperturbationneighborhood

aperturbedsurrogatemodel

Surrogatemodel

x

d

x2,*

d2

Function value

Functionvalue

Exex(g(x;∞ +8))

Exex(g(x; ∞))

Exex(g(x;w + 8*)

Surrogate Sharpness

Rx(∞) = max|Exex[g(x;∞ + 8)] -Exex[g(x; w)]1

Surrogatemodel

Oraclefunction

Surrogatemodel

8l2≤p

W

W1

Data

Data

3.1 Surrogate Sharpness

Suppose the oracle lies within the parametric family of the surrogate, there must exist a perturbation neighborhood of the surrogate's parameters that contains the oracle. That is, the oracle can be obtained by adding to the surrogate's parameters a noise vector in this neighborhood. Now, suppose the predictions do not change substantially across the models (including both the oracle and surrogate) in the perturbation neighborhood; the surrogate's predictions (and optimizers) must be close to those of the oracle (see Fig. 1). Motivated by this insight, we consider a potential approach to mitigate the erratic prediction of the surrogate, which is to ensure that its worst-case prediction change across the perturbation neighborhood is sufficiently small. This is formalized below:

where ∥ · ∥ 2 denotes the ℓ 2 -norm, and ρ > 0 bounds the maximum norm of the perturbation, which defines the perturbation neighborhood and can be selected via hyper-parameter tuning (see Section 5.1 and Appendix G.4). To ease the notation, we will omit the subscript and use ∥ . ∥ to consistently denote the ℓ 2 norm in the rest of this manuscript. We will also refer to R X ( ω ) in Eq. (4) as the generalized surrogate sharpness, which can be used to regularize surrogate training,

where ϵ ′ is a user-specified threshold. Solving Eq. (5) is non-trivial since R X ( ω ) is neither tractable nor differentiable. To sidestep this, we leverage a theoretical result that will be established later in Section 4, which (informally) states that with high confidence, the generalized surrogate sharpness

R X ( ω ) can be approximated with its empirical estimate R D ( ω ) in Eq. (16),

where G ( ω ) and λ max denote the norm of the expected (parameter) gradient of the surrogate at ω , and the largest eigenvalue of the (parameter) Hessian of the surrogate's expected prediction,

When n is sufficiently large, (dim( ω ) log( n ∥ ω ∥ 2 ) /n ) 1 2 will be negligibly small. Then, suppose within the search region for ω (see Assumption 2), G ( ω ) is bounded by a constant G , we can enforce R X ( ω ) ≤ ϵ ′ = ( ρG +(1 / 2) ρ 2 λ max ) ϵ via constraining instead R D ( ω ) ≤ ϵ ,

To mitigate the non-differentiability of R D ( ω ) , we further propose a practical approximation which reduces R D ( ω ) to a function of the surrogate's gradient norm ∥∇ ω g ( x ; ω ) ∥ , which is differentiable, allowing the above COP to be solved effectively with existing constrained optimization algorithms. This is discussed in the next section.

Remark. Eq. (4) is similar in spirit to the notion of loss sharpness [14] in which the same perturbation model is used to characterize the sharpness or sensitivity of the loss function, |L D ( ω + δ ) -L D ( ω ) | , to small changes ∥ δ ∥ ≤ ρ in the model weights ω . Our work, however, sets a direct focus on the sensitivity or sharpness of the surrogate's prediction - Eq. (4) - which results in a better surrogate regularizer, leading to better empirical performance (see Section 5.3). It also requires a significant and non-trivial adaptation of the theories presented in [14], as detailed later in Section 4.

3.2 Practical Algorithms

Let h ( ω + δ ) ≜ E x ∈D [ g ( x ; ω + δ )] and h ( ω ) ≜ E x ∈D [ g ( x ; ω )] . The surrogate sharpness can be approximated via the first-order Taylor expansion of h ( ω + δ ) at ω :

Using the Cauchy-Schwartz inequality on the right-hand side of the above and noting that δ can be selected to make the equality happens,

Using the above approximation, the COP in Eq. (8) can be rewritten as

which can be solved via optimizing the corresponding Lagrangian,

with λ > 0 denotes the Lagrange multiplier. This can be solved via our approach below.

IGNITE. We can optimize for both λ and ω using the basic differential multiplier method (BDMM) [17], which simultaneously gradient ascent for λ and gradient descent for ω , resulting in the following update rules:

Algorithm 1 IGNITE

Input: offline data D = { ( x i , z i ) } n i =1 ; initial surrogate g ( x ; ω (0) ) ; no. of iterations T ; batch size m ; Lagrange multiplier λ ; perturbation radius ρ and scalar r ; step sizes η ω and η λ ; threshold ϵ .

1: Initialize ω (1) ← ω (0) and λ (1) ← λ

2: for t ← 1 : T do

3: Sample B = { ( x i , z i ) } i m =1 ∼ D

4: Compute ˆ z i = g ( x i ; ω ( t ) ) for i ∈ [ m ]

6: Compute g 2 = m -1 ∑ m i =1 ∇ ω ˆ z i

5: Compute g 1 = m -1 ∑ m i =1 ∇ ω ℓ (ˆ z i , z i )

7: Compute ˆ ω = ω ( t ) + r · g 2 / ∥ g 2 ∥

8: Compute g 3 = m -1 ∑ m i =1 ∇ ω g ( x i ; ˆ ω )

9: Compute g ( t ) = g 1 + λ ( t ) ρr -1 ( g 3 -g 2 )

10: Update ω ( t +1) ← ω ( t ) -η ω g ( t )

11: Update λ ( t +1) ← λ ( t ) + η λ ( ρ ∥ g 2 ∥ -ϵ )

12: end for

13: return learned surrogate ω ( T +1)

where ω t represents the surrogate's parameter estimate at iteration t , λ t is the Lagrange multiplier estimate at iteration t , η ω is the step size for updating ω , and η λ is the step size for updating λ . We name this method IGNITE . We also conduct grid search to select the optimal value for ρ , as mentioned in Section 5.1.

Remark. As an alternative approach, we can also treat λ as a hyper-parameter and optimize for ω using gradient descent; we call this method IGNITE-2 . The detailed algorithms, hyper-parameter selection, and experimental results of IGNITE-2 are reported in Appendix G.2.

Both of the above methods require differentiating ∥∇ ω h ( ω ) ∥ with respect to ω , which involves computing the expensive Hessian of h ( ω ) . Fortunately, this expensive computation can be avoided by using the gradient approximation technique detailed in Appendix F. To summarize, we provide a complete pseudo-code of IGNITE in Algorithm 1 whose steps 3 -8 implement the approximation in Eq. (84) and Eq. (85) of Appendix F.

4 Theoretical Analysis

In this section, we will provide a detailed theoretical analysis to substantiate our earlier (informal) statement in Eq. (6) that with high confidence, reducing the empirical sharpness R D ( ω ) on the offline data will also reduce its generalized sharpness R X ( ω ) . We will show that this is true (see Theorem 1) under certain choices and mild assumptions of the surrogate model (see Assumptions 1 and 2).

Assumption 1. The output of the above surrogate function g ( x ; ω ) is bounded within [0 , 1] .

We note that the specific bound within [0 , 1] in Assumption 1 is meant to ease the technical presentation of our theoretical analysis. Otherwise, it can be extended straightforwardly to any bounded functions. Furthermore, we also show below that it is indeed possible to find a non-trivial surrogate function that is bounded and satisfies Assumption 2.

Theorem 1. There exists τ > 0 and ω + , and a non-linear function r ( x ; ω ) such that,

satisfies Assumption 2 and is bounded on { ω | ∥ ω ∥ ≤ τ } . Detailed derivation of this theorem is deferred to Appendix A.

For such surrogate functions, their generalized sharpness can be upper-bound by a function of their empirical sharpness on the offline data. As detailed below, the bound depends on both the size of the surrogate dim( ω ) and the number n of offline data points.

Theorem 2. For any ρ > 0 , m = dim( ω ) and 2 / ( mλ min ) ≥ σ 2 > 0 with λ min being defined in Assumption 2, the following holds simultaneously for all g ( . ; ω ) for which Assumption 2 is met,

with probability at least 1 -α over the random choice of the offline dataset, and with P ( n, m, α ) = 2 log( n/α ) + 4 log(8 n +4 m ) . Detailed derivation of this theorem is deferred to Appendix E.

Proof Sketch. For clarity, we will provide below a proof sketch of Theorem 2, which highlights the key steps in our derivation. Due to limited space, the specific of each step is deferred to the Appendix E. First, we note that

A relation between R X ( ω ) and R D ( ω ) can then be derived in three steps:

Upper-bound E δ ∼ N (0 ,σ 2 I ) [ F X ( ω + δ )] with a function of E δ ∼ N (0 ,σ 2 I ) [ F D ( ω + δ )] . This can be achieved via a direct application of the PAC-Bayes bound [18] which views the perturbed model ω + δ as a random hypothesis sampled from the posterior N ( ω , σ 2 I ) - see Appendix B.

Upper-bound E δ ∼ N (0 ,σ 2 I ) [ F D ( ω + δ )] with a function of R D ( ω ) . This can be achieved using a similar proving technique adopted from [14] - see Appendix C.

Find ξ > 0 such that R X ( ω ) can be upper-bounded with ξ · E δ ∼ N (0 ,σ 2 I ) [ F X ( ω + δ )] where ξ is a small scaling factor. To achieve this, we will find the lower-bound for E δ ∼ N (0 ,σ 2 I ) [ F X ( ω + δ )] using the Taylor remainder theorem to expand it around ω , which can be lower-bounded using the minimum eigenvalue of its Hessian at ω . Using the same approach, we can upper-bound R X ( ω ) with the maximum eigenvalue of the same Hessian - see Appendix D.

Finally, we set ξ so that the upper-bound of R X ( ω ) is smaller than the multiplication of ξ with the lower-bound of E [ F X ( ω + δ )] . To tighten the bound, we choose the smallest possible value of ξ such that the bound still holds. Lining up the results of the above steps then shows that R X ( ω ) can then be bounded with a function of ξ · R D ( ω ) . See Appendix E for a complete derivation.

Remark. Note that Theorem 2 is more general than its informal statement in Eq. (6), which can be reproduced by choosing σ 2 = 2 / ( mλ min ) in Eq. (16) to make the bound tightest.

5 Experiments

This section evaluates the efficacy of our proposed method IGNITE in improving state-of-the-art offline optimizers. We describe our experiment settings in Section 5.1 and report detailed empirical results in Section 5.2. We also provide additional ablation studies of our method in Section 5.3.

5.1 Benchmarks, Baselines, and Evaluation

Benchmark Tasks. Our explorations focus on four real-world tasks from Design-Bench 3 [4], covering both discrete ( TF-Bind-8 and TF-Bind-10 ) and continuous domains ( Ant Morphology [19] and D'Kitty Morphology [20]).

Baselines. We meticulously curated a diverse set of 11 widely acknowledged offline optimizers for comparative analysis. This ensemble comprises BO-qEI [4], CbAS [1], RoMA [3], ICT [15],

CMA-ES [21], COMs [13], MINs [12], REINFORCE [22], and three variations of gradient ascent ( GA , ENS-MIN , ENS-MEAN ), corresponding to vanilla gradient ascent, the min ensemble of gradient ascent, and the mean ensemble of gradient ascent, respectively.

Evaluation Protocol. To ensure a comprehensive assessment, we follow the methodology in [4]. Each method generates 128 optimized design candidates evaluated by the oracle function. The candidates' performances are ranked, and the 50 -th, 75 -th, and 100 -th percentile levels are recorded. All results are averaged over 16 independent runs to ensure reliability.

Hyper-parameter Configuration. For each baseline algorithm, we carefully configure optimal hyper-parameters as outlined in [4]. Our method IGNITE introduces five additional hyper-parameters: λ , ρ , r , η λ , and ϵ . The hyper-parameter λ , an initial value for the regularizer coefficient, is set to 0 . 01 through a grid search within { 0 . 0001 , 0 . 001 , 0 . 01 } . The hyper-parameters ρ and r are chosen from { 0 . 01 , 0 . 05 , 0 . 1 , 0 . 2 } , with ρ set to 0.05 and r set to 0.05 for IGNITE . Additionally, IGNITE uses η λ = 1 e -3 and ϵ = 0 . 1 , which are determined via the experiments in Section 5.3.

5.2 Results and Discussion

In this section, we presented the percentage improvement over baseline performance attained by IGNITE when it is applied to existing baselines. We have evaluated this at the 50 -th, 80 -th, and 100 -th percentile levels. However, due to limited space, we only report results of the 100 -th percentile level in the main text. The other results are instead deferred to Appendix G.3.

Results on Continuous Tasks: The first two columns of Table 1 show that out of 22 cases involving 11 baseline algorithms across 2 tasks, the IGNITE regularizer enhances baseline performance in 18 cases, with improvements reaching up to 9 . 6% . In only 1 out of 22 instances IGNITE slightly decreases performance by 0 . 2% , which is negligible. Even in cases where performance is not improved, IGNITE reduces the variance in results from 0 . 2% to 0 . 1% for the CMA-ES baseline on the D'Kitty Morphology task. Additionally, IGNITE helps establish new state-of-the-art (SOTA)

(a)

(q)

2

GA_ANT

3

GA_ANT

Improvement (%)

GA_TF10

Improvement (%)

2

GA_TF10

COMS_ANT

COMs_ANT

COMs_TF10

1

COMs TF10

0

0

-1

-1

-2

-2

0.010.10

0.20

0.30

0.40

0.5C

-3

1e-5

1e-4

1e-3

1e-2

3

nx

performances in both tasks. For example, in the Ant Morphology task, it raises the SOTA baseline CMA-ES from 195 . 5% to 195 . 7% . In the D'Kitty Morphology task, IGNITE achieves a new SOTA of 96 . 2% with the ICT baseline.

Results on Discrete Tasks: The last two columns of Table 1 show the impact of the IGNITE regularizer on the performance of baseline algorithms in two discrete domains ( TF-BIND-8 and TFBIND-10 ). Similar to the continuous tasks, IGNITE significantly enhances baseline performance in most cases ( 17 out of 22 ), with improvements of up to 5 . 6% . There are only 3 instances where integrating IGNITE results in a minor performance decrease of up to 0 . 7% , which is negligible. Additionally, in certain instances, IGNITE not only improves baseline performance but also establishes new state-of-the-art (SOTA) results. For example, on TF-BIND-8 and TF-BIND-10 , the original SOTA performances of 98 . 5% and 68 . 1% achieved by ENS-MEAN and ENS-MIN , respectively, are elevated to 98 . 7% and 70 . 5% with the addition of IGNITE , setting new SOTA records.

In summary, IGNITE consistently maintains a high probability of 91% ( 40 out of 44 ) of not degrading baseline performance. There is a high likelihood ( 79 . 55% = 35 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1 . 91% and a notable peak improvement of 9 . 6% . Conversely, IGNITE also exhibits a relatively low probability ( 9 . 09% = 4 out of 44 cases) of decreasing performance, with an average degradation of approximately 0 . 3% and a minor peak degradation of 0 . 7% .Additionally, there is a minor probability ( 11 . 36% = 5 out of 44 cases) of maintaining baseline performance.

5.3 Ablation Experiments

In this section, we conduct additional experiments to assess the sensitivity of two representative baselines, COMs and GA , when regularized with IGNITE , to variations in hyper-parameters ϵ and η λ . Additionally, we perform experiments to compare the efficacy of IGNITE with other commonly used regularization methods.

Changing threshold ϵ . We assess the performance enhancement of COMs and GA when regularized with IGNITE using various values of ϵ from the set { 0 . 01 , 0 . 05 , 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 } . A high ϵ value may result in a surrogate that is overly sharp, potentially hampering the search process and hindering the discovery of optimal designs. Figure 2(a) demonstrates that excessively high ϵ values lead to negative improvements. Conversely, excessively low ϵ values may cause the regularizer to dominate the original loss, resulting in a surrogate that is not well-fitted to the offline data. Negative improvements are observed with ϵ = 0 . 01 and 0 . 05 in Figure 2(a). As a result, we determine ϵ = 0 . 1 as the optimal value based on these observations.

Changing step size η λ . The step size η λ controls the rate at which λ is updated during the optimization process. It's essential to choose an appropriate step size, avoiding it being either too large or too small. Figure 2(b) demonstrates that an η λ value of 1 e -3 yields optimal results.

Comparing IGNITE with other regularization methods. We conduct a comparative experiment to assess the performance improvement achieved by using the IGNITE regularizer compared to other regularizers, including L1, L2, and SAM [14] (where SAM is considered as a loss sharpness regularization with a coefficient equal to 1 ). Table 2 presents the results obtained on two baseline algorithms, REINFORCE and GA , across four tasks. Overall, IGNITE outperforms the other regularizers in most cases, with the largest difference compared to the second best being 3 . 2% with REINFORCE on the TF-BIND-10 task. Additionally, IGNITE achieves positive improvements in all cases. Conversely, while the simple regularizers L1 and L2 help boost performance with REINFORCE , they lead to performance degradation with GA . SAM outperforms IGNITE with the GA baseline on the TF-BIND-10 task and shows notable improvement with REINFORCE on the D'Kitty task, though its integration with GA leads to a performance drop. Furthermore, we show that IGNITE also outperforms SAM when integrating with CbAS and BO-qEI in Appendix H.

Surrogate sharpness on unseen data before and after using IGNITE. We also conduct an experiment measuring the sharpness of the surrogate model, approximated by ρ ∥∇ ω h ( ω ) ∥ as defined in Eq. (10), with and without IGNITE . This sharpness is computed on unseen data points which are, specifically, the design candidates generated by the GA and REINFORCE baselines. By testing on these unseen candidates, we simulate the out-of-distribution (OOD) conditions that are critical in assessing generalization in optimization tasks. Table 3 reports these surrogate sharpness measurements for some baselines with and without using the IGNITE regularizer. The results demonstrate consistently that the IGNITE regularizer helps reduce the surrogate sharpness on unseen data, as anticipated. This reduction indicates that IGNITE is effective in smoothing the surrogate model's landscape, leading to better and more stable generalized performance.

Results on tasks with noisy data. To demonstrate IGNITE 's robust performance in scenarios with noisy oracle, we conducted additional experiments using the GA and REINFORCE baselines on two benchmark tasks with particularly noisy oracles: Superconductor and Chembl . For each baseline, we compared its achieved performance with and without the regularizing effect of IGNITE , as shown in Table 4. Across both tasks, IGNITE helps improve the baseline performance substantially, achieving up to a 2 . 1% increase for REINFORCE on the Superconductor task, highlighting IGNITE 's ability to enhance performance robustness even in settings with noisy oracles.

6 Conclusion

This paper introduces the concept of generalized surrogate sharpness in offline optimization, resulting in the development of a new regularization technique, IGNITE . Our theoretical analysis demonstrates that reducing surrogate sharpness on an offline dataset provably decreases its generalized sharpness on unseen data. Empirically, IGNITE consistently maintains a high probability ( 91% ) of not degrading baseline performance and a 79 . 55% likelihood of improving it, with a peak improvement of 9 . 6% . Additionally, we believe that our novel technique can be adapted to related domains such as robust optimization (RO) and reinforcement learning (RL), suggesting potential future research directions.

Acknowledgments and Disclosure of Funding

This work was funded by Vingroup Joint Stock Company (Vingroup JSC),Vingroup, and supported by Vingroup Innovation Foundation (VINIF) under project code VINIF.2021.DA00128.

References

David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference on machine learning , pages 773-782. PMLR, 2019.

Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model-based design. Advances in Neural Information Processing Systems , 33:12945-12956, 2020.

Sihyun Yu, Sungsoo Ahn, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for offline model-based optimization. Advances in Neural Information Processing Systems , 34:4619-4631, 2021.

Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-bench: Benchmarks for data-driven offline model-based optimization. In International Conference on Machine Learning , pages 21658-21676. PMLR, 2022.

Manh Cuong Dao, Phi Le Nguyen, Thao Nguyen Truong, and Trong Nghia Hoang. Boosting offline optimizers with surrogate sensitivity. In Forty-first International Conference on Machine Learning , 2024.

Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black-box optimization. arXiv preprint arXiv:2306.07180 , 2023.

Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Generative pretraining for black-box optimization. arXiv preprint arXiv:2206.10786 , 2022.

Tung Nguyen, Sudhanshu Agrawal, and Aditya Grover. Expt: Synthetic pretraining for few-shot experimental design. Advances in Neural Information Processing Systems , 36:45856-45869, 2023.

Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, and Janardhan Rao Doppa. Offline model-based optimization via policy-guided gradient search. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 11230-11239, 2024.

Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, and Trong Nghia Hoang. Learning surrogates for offline black-box optimization via gradient matching. In Forty-first International Conference on Machine Learning , 2024.

Justin Fu and Sergey Levine. Offline model-based optimization via normalized maximum likelihood estimation. arXiv preprint arXiv:2102.07970 , 2021.

Aviral Kumar and Sergey Levine. Model inversion networks for model-based optimization. Advances in Neural Information Processing Systems , 33:5126-5137, 2020.

Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In International Conference on Machine Learning , pages 10358-10368. PMLR, 2021.

Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412 , 2020.

Ye Yuan, Can Chen, Zixuan Liu, Willie Neiswanger, and Xue Liu. Importance-aware coteaching for offline model-based optimization. arXiv preprint arXiv:2309.11600 , 2023.

Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 , 2014.

John Platt and Alan Barr. Constrained differential optimization. In Neural Information Processing Systems , 1987.

David A McAllester. Pac-bayesian model averaging. In Proceedings of the twelfth annual conference on Computational learning theory , pages 164-170, 1999.

Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540 , 2016.

Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, and Vikash Kumar. Robel: Robotics benchmarks for learning with low-cost robots. In Conference on robot learning , pages 1300-1313. PMLR, 2020.

Nikolaus Hansen. The cma evolution strategy: a comparing review. Towards a new evolutionary computation: Advances in the estimation of distribution algorithms , pages 75-102.

Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning , 8:229-256, 1992.

D. Kingma and M. Welling. Auto-Encoding Variational Bayes. In Proc. ICLR , 2013.

John Langford and Rich Caruana. (not) bounding the true error. Advances in Neural Information Processing Systems , 14, 2001.

Beatrice Laurent and Pascal Massart. Adaptive estimation of a quadratic functional by model selection. Annals of statistics , pages 1302-1338, 2000.

Yang Zhao, Hao Zhang, and Xiuyuan Hu. Penalizing gradient norm for efficiently improving generalization in deep learning. In International Conference on Machine Learning , pages 26982-26992. PMLR, 2022.

Barak A Pearlmutter. Fast exact multiplication by the hessian. Neural computation , 6(1):147160, 1994.

Proofs of Theoretical Results and Additional Experimental Results

A Proof of Theorem 1

Part I. Minimum Eigenvalue of Parameter Hessian is Positive.

First, we re-state the choice of our surrogate in Theorem 1,

Regardless of the choice of r ( x ; ω ) , differentiating g ( x ; ω ) with respect to ω yields,

which implies the parameter Hessian of ∇ 2 ω g ( x ; ω ) is always a constant matrix depending on the specific choice of r ( x ; ω ) and a reference point ω + . Hence, it follows trivially that,

Therefore, if we can find r ( x ; ω ) such that its parameter Hessian is strictly positive definite at ω + , the parameter Hessian of g ( x ; ω ) will always be strictly positive definite regardless of ω . This means g ( x ; ω ) will be λ min -strongly convex in ω with λ min > 0 . As a result, its expectation E [ g ( x ; ω )] will also be λ min -strongly convex with λ min > 0 or equivalently, Assumption 2 holds.

This can be done by choosing r ( x ; ω ) to be any quantile prediction of a linear regressor with a (pre-trained) random feature map. For example,

where V [ . ] denote the variance function. We can interpret the random feature ψ ( x ) is sampled from a pre-trained VAE [23] with two deep neural nets characterizing the mean and variance functions, m ( x ) and v ( x ) . These functions can be pre-trained and post-processed in whatever ways we need to control their value ranges. Their parameters are then frozen when we fit r ( x ; ω ) with respect to ω .

To understand the behavior of r ( x ; ω ) , we first note below the closed-form expression of Eq. (23),

where A ≜ diag[ v ( x )] . Differentiating twice both sides of Eq. (24) at ω + , we have

We can now choose ω + = (1 /m ) ∑ m i =1 e i where e 1 , e 2 , . . . , e m are the eigenvectors of A . Since A is diagonal, these are also the one-hot vectors and their corresponding eigenvalues are the entries on the diagonal of A - i.e., λ i ( A ) = [ A ] ii . With this choice, a closed-form expression of ∇ 2 ω r ( x ; ω + ) in terms of the entries of A can be derived as follow,

where B is another diagonal matrix with entries [ B ] ii = m -2 ( ∑ m ι =1 [ A ] ιι -[ A ] ii ) . As such, it is trivial to see that if we choose the activation unit (i.e., sigmoid or ReLU) for v ( x ) such that its output is positive, then the entries of both A and B are positive.

Finally, note that Eq. (26) is a scaled matrix product of two diagonal matrices, A and B , which will result in a diagonal matrix. Its entries will be positive if the entries of A and B are positive, as enforced above. This means the minimum eigenvalue of ∇ 2 ω r ( x ; ω + ) is positive as desired.

Part II. Boundedness on { w : ∥ w ∥ 2 ≤ τ } .

Rearranging terms in the expression of g ( x ; ω ) in Eq. (19),

where const absorbs all constant vectors (i.e., not dependent on ω ). Hence, taking absolute values for both sides of Eq. (27), we obtain

where Eq. (30) follows from the fact that ∇ 2 ω r ( x ; ω + ) is positive definite by construction. Eq. (31) follows because (1 / 2) λ max ( ∇ 2 ω r ( x ; ω + )) is a fixed value that does not change when ω changes. Thus, g ( x ; ω ) is bounded as expected.

Lemma 3. Let us define

The following holds simultaneously for all ω , σ 2 > 0 , α ∈ (0 , 1) and m = dim( ω ) ,

with probability at least 1 -α over the (random) choice of the offline dataset D .

Proof. We can view the random perturbation δ as a random hypothesis sampled from a hypothesis distribution (such as the zero-mean Gaussian N (0 , σ 2 I ) ) and F X ( ω + δ ) as its incurred loss. In this view, for any hypothesis distribution Q , E δ ∼Q [ F X ( ω + δ )] and E δ ∼Q [ F D ( ω + δ )] denote the corresponding generalized and empirical Gibbs losses whose relationship is characterized via the following classical PAC-Bayesian [18] bound,

which will hold simultaneously for all choices of ω , P and Q . Historically, in probabilistic learning context, P is often referred to as prior or reference distribution which, for example, encodes domainspecific information about certain properties of the solution distribution. Then, Q is the posterior, which can be selected to tighten the bound (hence, reducing the generalized loss).

In our specific context, we will simply use the choices of P and Q as technical vehicles to tighten the gap between E [ F X ( ω + δ )] and E [ F D ( ω + δ )] which will contribute to the gap between our generalized and empirical surrogate sharpness.

To derive the optimal choices P and Q , we will leverage part of the proof from [14], which starts with the following fact:

This means if ∥ ω ∥ > ρ 2 (exp(4 n/m ) -1) , then

which means the bound gap in Eq. (34) is larger than 1 and under Assumption 1 that the surrogate's output - hence, F X ( ω ) 's - is bounded in [0 , 1] , Eq. (34) holds trivially.

Hence, to choose P and Q , we can make the assumption that ∥ ω ∥ ≤ ρ 2 (exp(4 n/m ) -1) . Now, let us choose both prior and posterior distribution to be Gaussians, P = N ( µ P , σ 2 P I ) and Q = N ( µ Q , σ 2 Q I ) . Their KL divergence can then be computed in closed-form:

Naively, we can minimize the above KL divergence via setting the derivative of KL with respect to σ P to zero, and solving for the optimal σ P , which gives σ P 2 = σ 2 Q + m -1 ∥ µ P -µ Q ∥ 2 2 .

However, for the PAC-Bayes bound to hold, σ P must be chosen independent of the rest, so the above approach does not work. Instead, we can define in advance a prior set of σ P and then choose the best one in this set, following the proof in [24]. We can choose this set as follow:

Given fixed a, b > 0 , let S = { c · exp((1 -i ) /m ) | i ∈ N } be predefined set for σ 2 P . If the above bound holds for σ 2 P = c × exp((1 -i ) /m ) with probability 1 -α i with α i = 6 απ -2 i -2 for any i ∈ N , then all above bounds hold simultaneously with probability at least 1 -∑ ∞ i =1 6 απ -2 i -2 = 1 -α .

Now, let σ Q = σ, µ Q = ω , µ P = 0 , we have

Choosing i = ⌊ 1 -m log(( σ 2 + ∥ ω ∥ 2 2 /m ) /c ) ⌋ , c = σ 2 (1+exp(4 n/m )) and σ 2 P = c · exp((1 -i ) /m ) ,

Plugging Eq. (43) and Eq. (39) into Eq. (38),

Thus, Eq. (35) holds for each prior (indexed by i ) in the above set with probability 1 -α i . Hence, Eq. (35) holds with all prior choices with probability 1 -∑ i α i = 1 -α . This allows us to pick the prior that leads to the tightest bound gap, which is indexed with i = ⌊ 1 -m log(( σ 2 + ∥ ω ∥ 2 2 /m ) /c ) ⌋ . For this choice, we have

Plugging Eq. (47) into Eq. (48),

Finally, to remove the dependence on i in the above bound, note that

Plugging Eq. (57) into Eq. (49),

which completes our proof.

C

Upper-Bound

E

δ

∈

N

(0

,σ

2

I

)

[

F

X

(

ω

+

δ

)]

with

R

D

(

ω

)

Lemma 4. Following the same setup in Lemma 3, the following holds simultaneously for all ω , σ 2 > 0 , α ∈ (0 , 1) and m = dim( ω ) ,

with probability at least 1 -α over the (random) choice of the offline dataset D .

Proof. To derive this result, we need to upper-bound E [ F X ( ω )] with F D ( ω ) . First, we note that δ ∼ N (0 , σ 2 I ) so ∥ δ ∥ 2 2 follows a chi-square distribution. From Lemma 1 in [25],

Therefore, with probability 1 -1 / √ n we have:

This means with probability 1 -1 / √ n , we have E [ F D ( ω + δ )] ≤ max ∥ δ ∥≤ ρ F D ( ω + δ ) . Otherwise, with the remaining 1 / √ n chance, E [ F D ( ω + δ )] ≤ 1 under Assumption 1. Putting this together,

Finally, plugging Eq. (62) into Eq. (34),

which completes our proof.

D Upper-Bound R X ( ω ) with a scaled value of E δ ∈ N (0 ,σ 2 I ) [ F X ( ω + δ )]

Lemma 5. Following the same setup in Lemma 3, the below holds for all σ 2 ∈ (0 , 2 / ( mλ min )) ,

with G ( ω ) and λ max defined previously in Eq. (7) , m = dim( γ ) and λ min defined in Assumption 2.

Proof. Let h ( ω ) ≜ E [ g ( x ; ω )] where the expectation is over x ∈ X , we have

with ˆ ω = ω + c · δ with some constant c ∈ [0 , 1] . In the above, Eq. (65) follows from the Taylor's remainder theorem for multivariate function. Eq. (67) follows from the Cauchy-Schwartz inequality (for the first term) and the definition of λ max as the upper-bound on the largest eigenvalue of the parameter Hessian of h ( ω ) . Note that λ max > λ min > 0 under Assumption 2 which stipulates that the smallest eigenvalue of the parameter Hessian of h ( ω ) is always positive.

On the other hand, we also:

where m = dim( ω ) . In the above, Eq. (70) follows from the Taylor's remainder theorem (similar to Eq. (65)) and Eq. (72) follows from the definition of λ min in Assumption 2 as the lower-bound on the smallest eigenvalue of the parameter Hessian. Since λ min > 0 , the quadratic term inside the expectation is always positive which explains why we can remove the absolute operator. Eq. (73) follows from standard moment calculation of Gaussian random vector.

Finally, we note that:

which means, intuitively,

is the smallest scaling factor for the lower-bound of E δ ∼ N (0 ,σ 2 I ) [ F X ( ω + δ )] so that it matches the upper-bound of R X ( ω ) = max ∥ δ ∥≤ ρ F X ( ω + δ ) , as outlined in the 3rd step of our proving plan for Theorem 2. As such, plugging Eq. (68) into the left-hand side of Eq. (75) completes our proof.

Remark. In the above, Eq. (73) suggests that E δ [ F X ( ω + δ )] ≥ (1 / 2) λ min · m · σ 2 . But, under Assumption 1, we know that the surrogate's output - and hence F X ( ω + δ ) 's - is bounded within [0 , 1] , which means 1 ≥ (1 / 2) λ min · m · σ 2 or equivalently, σ 2 ≤ 2 / ( mλ min ) as stated in Lemma 5's statement. That is, under Assumption 1, Lemma 5 is only correct for σ 2 ∈ (0 , 2 / ( mλ min )) .

E Proof of Theorem 2

We are now ready to prove our main result. First, we re-state the result of Lemma 4 below,

where the last step follows from the definition of R D ( ω ) . Finally, plugging Eq. (78) into Eq. (63) and replacing max ∥ δ ∥≤ ρ F X ( ω + δ ) with R X ( ω ) (following its definition) completes our proof.

F Effective Approximation of ∇ ω ∥∇ ω h ( ω ) ∥

Explicitly, the gradient of the augmented loss in Eq. (12) is given below,

where the computation bottleneck lies with the second gradient term which can be further expressed below using the chain rule,

It is evident that ∇ 2 ω h ( ω ) in Eq. (80) represents a Hessian matrix. Calculating the Hessian matrix for a deep neural network is impractical due to the extensive dimensions of the model's weights. Nevertheless, since Eq. (80) involves the multiplication of a Hessian matrix with a vector, specific techniques like Hessian-vector products can be employed to approximate this product. In particular, let the Hessian matrix M = ∇ 2 ω h ( ω ) , we have Taylor expansion for function ∇ ω h ( ω ) as follows:

This approximation becomes precise as the value of ∆ ω approaches 0 . Following [26] and [27], we choose ∆ ω = r v where r is a small scalar and v is a vector, that transforms Eq. (81) into

Then, choosing v = ∇ ω h ( ω ) ∥∇ ω h ( ω ) ∥ leads to

It is noticed that, as pointed out by [27], an inappropriate choice of r can make Eq. (83) vulnerable to numeric and roundoff issues. The constant r must be sufficiently small so that the O ( r ) term becomes negligible. However, when r is too small, precision is compromised because the subtraction of the original gradient from the perturbed one, i.e., ∇ ω h ( ω + r ∇ ω h ( ω ) / ∥∇ ω h ( ω ) ∥ ) -∇ ω h ( ω ) , will obtain a small difference between them. Based on Eq. (83), Eq. (79) would be

In practical applications, we typically employ an additional approximation to compute the second term in Eq. (84), thereby avoiding the need for Hessian computation induced by the chain rule.

G Hyperparameter Tuning and Additional Experimental Results

G.1 Computation Resource

All our experiments were conducted on a system with the following specifications: Ubuntu 18.04, NVIDIA RTX 3090 GPUs, and CUDA 11.8.

G.2 IGNITE-2

IGNITE-2. To solve the Lagrangian in Eq. (12), we can treat λ as a hyper-parameter and optimize for ω using stochastic gradient descent (SGD).

where ω t is the estimation of the surrogate's parameter at the t th iteration, and η ω is step size to update ω . We name this method IGNITE-2 and its pseudo-code is in Algorithm 2.

Hyper-parameter Configuration. Our method IGNITE-2 introduces three additional hyperparameters: λ , ρ , and r . The penalty coefficient λ controls the gradient magnitude of our regularizer, set to 0 . 01 through a grid search within { 0 . 0001 , 0 . 001 , 0 . 01 } . The hyper-parameters ρ and r are chosen from { 0 . 01 , 0 . 05 , 0 . 1 , 0 . 2 , 0 . 5 } , with ρ set to 0.2 and r set to 0.2 for IGNITE . These three hyper-parameters are determined through experiments in Section G.4. These hyper-parameters are consistently applied across all experiments, except for the ICT baseline ( λ = 1 e -4 ) due to implementation differences.

Algorithm 2 IGNITE-2

Input: offline data D = { ( x i , z i ) } n i =1 ; initial surrogate g ( x ; ω (0) ) ; no. of iterations T ; batch size m ; Lagrange multiplier λ ; perturbation radius ρ and scalar r ; step sizes η ω .

1: Initialize ω (1) ← ω (0)

2: for t ← 1 : T do

3: Sample B = { ( x i , z i ) } i m =1 ∼ D

4: Compute ˆ z i = g ( x i ; ω ( t ) ) for i ∈ [ m ]

6: Compute g 2 = m -1 ∑ m i =1 ∇ ω ˆ z i

7: Compute ˆ ω = ω ( t ) + r · g 2 / ∥ g 2 ∥

8: Compute g 3 = m -1 ∑ m i =1 ∇ ω g ( x i ; ˆ ω )

9: Compute g ( t ) = g 1 + λρr -1 ( g 3 -g 2 )

10: Update ω ( t +1) ← ω ( t ) -η ω g ( t )

11: end for

12: return learned surrogate ω ( T +1)

G.3 Performance Evaluation at 100-th, 80-th and 50-th Percentile Level of IGNITE and IGNITE-2

In this section, we presented the percentage improvement over baseline performance attained by IGNITE-2 and IGNITE when it is applied to an existing baseline. We have evaluated this at the 100 -th, 80 -th, and 50 -th percentile levels. The results are reported in Table 5, 6, and 7, respectively.

5: Compute g 1 = m -1 ∑ m i =1 ∇ ω ℓ (ˆ z i , z i )

G.3.1 Performance Evaluation at 100-th Percentile Level of IGNITE-2

According to the reported results in Table 5, IGNITE-2 consistently maintains a high probability of 86 . 36% ( 38 out of 44 ) of not degrading baseline performance. There is a high likelihood ( 72 . 27% = 34 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1 . 39% and a notable peak improvement of 6 . 8% . Conversely, IGNITE-2 also exhibits a relatively low probability ( 13 . 64% = 6 out of 44 cases) of decreasing performance, with an average degradation of approximately 0 . 45% and a minor peak degradation of 0 . 8% .Additionally, there is a minor probability ( 9 . 09% = 4 out of 44 cases) of maintaining baseline performance. Furthermore, while IGNITE-2 demonstrates significant efficiency, the results in Section 5.2 indicate that IGNITE even outperforms IGNITE-2 .

G.3.2 Performance Evaluation at 80-th Percentile Level of IGNITE and IGNITE-2

As shown in Table 6, IGNITE-2 consistently maintains a high probability high probability of not degrading baseline performance, with a likelihood of 61.36% (27 out of 44 cases). There is a 47 . 73% chance ( 21 out of 44 cases) of improving baseline performance, with an average improvement of approximately 0 . 6% and a peak improvement of 2 . 4% . On the other hand, IGNITE-2 also indicates a low likelihood (38.64% = 17 out of 44 cases) of performance decrease, with an average decline of around 0.68% and a maximum decline of 4.6%. Furthermore, there is a slight chance (13.64% = 6 out of 44 cases) of maintaining the baseline performance.

In addition, IGNITE also maintains a high probability of 72 . 73% ( 32 out of 44 ) of not degrading baseline performance. There is a 47 . 73% likelihood ( 21 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1 . 0% and a peak improvement of 2 . 6% . Besides, IGNITE also exhibits a low probability ( 27 . 27% = 12 out of 44 cases) of decreasing performance, with an average degradation of approximately 1 . 58% and a peak degradation of 4 . 6% . Additionally, there is a small probability ( 25% = 11 out of 44 cases) of maintaining baseline performance.

G.3.3 Performance Evaluation at 50-th Percentile Level of IGNITE and IGNITE-2

Table 7 displays the results of IGNITE-2 and IGNITE at the 50th percentile level. The results indicate that both IGNITE-2 and IGNITE do not degrade performance compared to the baseline in 65.91% of settings (29 out of 44) for each method. In which, IGNITE-2 outperforms the baseline in 50% likelihood, with an average improvement of approximately 0 . 85% and a peak improvement of 2 . 9% . For IGNITE , those are 52 . 27% likelihood, 1 . 89% average improvement and a peak improvement of 14 . 6% . Conversely, there are only 34 . 09% cases in using our proposed method leads to the performance reduction. The average degradation of IGNITE-2 and IGNITE are 0 . 69% and 1 . 19% , respectively. Overall, we state that our proposed method helps to boost the performance of almost all algorithms and tasks.

0.75

0.80

0.85

0.90

0.95

0

10

20

30

40

50

Gradient ascent steps

Normalized objective value

ANT

0.61

0.62

0.63

0.64

0.65

0.66

0.67

0

10

20

30

40

50

Gradient ascent steps

Binding affinity

TF-BIND-10

COMs (Base)

COMs (IGNITE)

GA (Base)

GA (IGNITE)

0.75

0.80

0.85

0.90

0.95

0

10

20

30

40

50

Gradient ascent steps

Normalized objective value

ANT

0.60

0.62

0.64

0.66

0

10

20

30

40

50

Gradient ascent steps

Binding affinity

TF-BIND-10

COMs 0.01

COMs 0.05

COMs 0.1

COMs 0.2

COMs 0.5

GA 0.01

GA 0.05

GA 0.1

GA 0.2

GA 0.5

G.4 Hyper-parameters selection for IGNITE-2

This section provides the specific setup of the hyper-parameter selection experiments for IGNITE-2 , which is also ran separately to determine the hyperparameters for IGNITE (in the main text).

G.4.1 IGNITE-2 enhances stability of COMs and gradient ascent (GA).

Figure 3 provides a step-by-step performance comparison between two baseline algorithms, COMs and GA , with and without our regularization method, IGNITE-2 . IGNITE-2 consistently outperforms the baseline algorithms at every step in the ANT tasks. However, the trend differs for TF-BIND-10 . Initially, the baseline algorithms perform better without IGNITE-2 . However, as the number of gradient ascent steps increases, the performance of the baselines without IGNITE2 begins to decline. In contrast, the algorithms incorporating our method maintain or improve their performance over time. This trend suggests that our method becomes increasingly beneficial in the later stages of the optimization process, ultimately enhancing the overall performance of the baseline algorithms.

G.4.2 Choosing the regularization coefficient λ .

Figure 4 shows how the performance of baseline models regularized with IGNITE-2 is affected by different values of λ . The results indicate that using an excessively low or high value for λ will have a negative impact on performance. In all cases, the results suggest that a universal value of 0.01 for λ tends to generate consistent and effective performance across all tasks.

G.4.3 Choosing value for parameter r and the perturbation radius ρ .

Figure 5 plot the performance of the GA and COMS baselines regularized with IGNITE-2 with respect to the change of hyper-parameter r . That is r ∈ { 0 . 01 , 0 . 05 , 0 . 1 , 0 . 2 , 0 . 5 } . Figure 6 visualizes how the performance of baselines regularized with IGNITE-2 is influenced by varying the hyperparameter ρ . Based on those result, we choose to set r = 0 . 2 and ρ = 0 . 2 in all of the experiments for IGNITE-2 .

0.75

0.80

0.85

0.90

0.95

0

10

20

30

40

50

Gradient ascent steps

Normalized objective value

ANT

0.60

0.62

0.64

0.66

0

10

20

30

40

50

Gradient ascent steps

Binding affinity

TF-BIND-10

COMs 0.001

COMs 0.01

COMs 1e-04

GA 0.001

GA 0.01

GA 1e-04

0.75

0.80

0.85

0.90

0.95

0

10

20

30

40

50

Gradient ascent steps

Normalized objective value

ANT

0.60

0.62

0.64

0.66

0

10

20

30

40

50

Gradient ascent steps

Binding affinity

TF-BIND-10

COMs 0.01

COMs 0.05

COMs 0.1

COMs 0.2

COMs 0.5

GA 0.01

GA 0.05

GA 0.1

GA 0.2

GA 0.5

H Percentage improvement over other baselines of IGNITE, SAM across all tasks.

I Complexity Overhead of IGNITE.

To analyze the computational complexity of IGNITE , we break down the complexity of each step in Algorithm 1.

Initialization (Line 1): Initializing ω (1) ← ω (0) and λ (1) ← λ : O (1) each.

Main Loop (Line 2-12): The loop runs for T iterations. Thus, the complexity of the main loop will be multiplied by T .

Sampling (Line 3): Sampling a batch B = { ( x i , z i ) } i m =1 ∼ D : O ( m ) .

Computing ˆ z i (Line 4): Evaluating the surrogate model g ( x i ; ω ( t ) ) for each i ∈ [ m ] . Assuming the surrogate model evaluation has a computational complexity of C g = O ( d ) per sample where d is the number of surrogate parameters, the total complexity is O ( m · d ) .

Computing g 1 and g 2 (Line 5-6): Computing gradients ∇ ω ℓ (ˆ z i , z i ) and ∇ ω ˆ z i have complexities O ( C ℓ ) and O ( C ˆ z ) respectively per sample where C ℓ = C ˆ z = O ( d ) . Therefore, the total complexities are O ( m · d ) .

Computing ˆ ω (Line 7): This involves simple vector operations with complexity O ( d ) , where d is the dimensionality of ω .

Computing g 3 (Line 8): Similar to lines 4 and 6, involving evaluating the surrogate and gradient computations, with complexity O ( m · C g + m · C ˆ z ) = O ( m · d ) .

Computing g ( t ) (Line 9): Vector operations involving addition and scalar multiplication with complexity O ( d ) .

Updating ω (Line 10): Updating ω involves simple subtraction operations with complexity O ( d ) .

Updating λ (Line 11): Updating λ is an O (1) operation

Overall Complexity: Considering the above steps, the most computationally expensive parts are the gradient computations in lines 5, 6, and 8. Thus, the overall complexity per iteration is:

Since this loop runs for T iterations, the total complexity is:

Furthermore, we have the total complexity of the original baseline is:

1

2

3

4

5

0

25

50

75

100

Epochs

Value

Training loss

Surrogate sharpness

GA ANT

1

2

3

4

5

0

25

50

75

100

Epochs

Value

Training loss

Surrogate sharpness

GA DKITTY

1

2

3

4

5

0

25

50

75

100

Epochs

Value

Training loss

Surrogate sharpness

REINFORCE ANT

1

2

3

0

25

50

75

100

Epochs

Value

Training loss

Surrogate sharpness

REINFORCE DKITTY

Thereby, IGNITE will include an additional complexity:

T is the number of iterations.

m is the batch size.

C g = O ( d ) is the complexity of evaluating the surrogate model.

C ℓ = O ( d ) is the complexity of computing the loss gradient.

C ˆ z = O ( d ) is the complexity of computing the gradient of the surrogate output with respect to its parameters.

d is the no. of surrogate parameters.

The empirical training time of the participating baselines with and without IGNITE is reported in Table 9. We observe that IGNITE solely consumes an additional negligible GPU memory, while the training time increases by 14.91% on average.

J Convergence and effectiveness of IGNITE

According to our experiment, despite using a relatively simple BDMM to solve Eq. (12), we have achieved significant improvement in most cases. To demonstrate the convergence of the optimization algorithm, we have plotted the training loss and the sharpness value (plotting ∥∇ ω h ( ω ) ∥ value) during the surrogate fitting process. This is based on an experiment using GA and REINFORCE baselines on Ant and Dkitty tasks. The results are illustrated in Figure 7. These results reveal that BDMM helps decrease both the training loss and sharpness value of the surrogate model during the training phase. This indicates that BDMM is effective and the optimization converges well in practice. Furthermore, our method, IGNITE , can be seamlessly integrated with other, more robust optimization techniques to solve Eq. (12).

K Limitation

Our paper studies the offline optimization task, which has potential applications in material engineering. Similar to the existing literature, our method is extensively tested on a universal benchmark set forward by the pioneering work of [4]. However, it is important to note that the benchmark consists mostly of small- to mid-scale optimization tasks. As such, our method has not considered where:

the challenge of scalability in large-scale domain with extremely high-dimensional input spaces. In addition, as with all machine learning algorithms, while applications of our work to real data could result in ethical considerations, this is an indirect and unpredictable side-effect of our work. Our experimental work uses publicly available datasets to evaluate the performance of our algorithms. No ethical considerations are raised.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our contributions can be found in Section 3 and Section 4.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss this in Appendix K

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All our assumptions are detailed in Section 4. All proofs are detailed in the Appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Yes. All information regarding our experiments are disclosed in Section 5 and in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Our experimental code is also submitted (as extra materials) along with our manuscript.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Such details can be founded in Section 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We do report error bars in our experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The information regarding our compute resource is detailed in Appendix G.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read the NeurIPS Code of Ethics and do not find our work in violation of any aspects.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We did discuss this in Appendix K.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work does not create new datasets. We only use existing, publicly available datasets. We also do not create any new pre-trained models.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite the source of all datasets used in our experiments.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Our work does not release any new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our work does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
The article presents a method to improve offline optimization techniques by integrating the concept of model sharpness into the training.  A constraint is introduced that limits the model sharpness to not exceed a user-specified threshold.

STRENGTHS:
1. The approach is model-agnostic, making it applicable across different types of models and optimization tasks without needing specific adjustments for each model or task.
2. The method is backed by a solid theoretical analysis, providing a robust framework for understanding and predicting the behavior of the regularization effect on offline optimization.
3. The method has been empirically tested on a variety of tasks, showing significant performance improvements, thus validating the theoretical predictions.

WEAKNESSES:
1. The effectiveness of the method heavily depends on the correct setting of hyperparameters like the sharpness threshold, which can be tricky to optimize without extensive experimentation.
2. The computation complexity is increased. Can the author give the complexity analysis?
3. How to ensure the proposed method converge to a stationary point of the original problem?

QUESTIONS:
see weakness

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel model-agnostic approach to enhance offline optimization methods by incorporating surrogate gradient norms.  The paper provides a thorough review of existing literature, a clear problem definition, and detailed descriptions of the proposed methods and their implementation. The experiments are well-designed, covering various benchmarks and baselines, and the results are robustly analyzed.\n\nSTRENGTHS:\n- a new regularizer based on surrogate sharpness, characterized by the surrogate's maximum output change under low-energy parameter perturbation. This regularizer is designed to be model-agnostic, making it broadly applicable across different surrogate and search models.\n- a practical approximation that reduces the surrogate sharpness measurement to a function of the surrogate’s gradient norm. This allows the optimization task to be transformed into a constrained optimization problem, which can be solved using existing optimization solvers.\n- a theoretical analysis demonstrating that reducing surrogate sharpness on an offline dataset provably reduces its generalized sharpness on unseen data.\n- extensive experimentation on a diverse range of optimization tasks shows that reducing surrogate sharpness often leads to performance improvements.\n\nWEAKNESSES:\n- As the paper mentioned that they draw inspiration from the sharpness-aware minimization (SAM), the paper’s novelty could be better articulated in the context of existing work on surrogate model regularization and sharpness-aware optimization. Specifically, discuss how the proposed surrogate sharpness measure offers advantages over sharpness-aware minimization (SAM).\n- Assumption 2 (positive minimum eigenvalue of the parameter Hessian) might be too restrictive. It would be more helpful if the authors can provide more intuitive explanations or empirical evidence to justify these assumptions and discuss the implications if these assumptions are violated and how the method’s performance might be affected.\n- The proposed method involves computationally expensive operations, such as gradient norm calculations and constrained optimization. I'm wondering how expensive the proposed method is compared to existing base approaches in terms of the training time.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nYes. In Appendix H", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

comments: SUMMARY:
This paper introduce a sharpness-aware optimization to improve out-of-distribution generalization. While inspired by SAM, the major difference is that this work considers the sharpness of predictor outputs rather than loss landscape. With the proposed notion of sharpness, practical algorithm, IGNITE, is developed through an empirical approximation (Eq. 6) and Taylor approximation of the sharpness constraint.

STRENGTHS:
- This material is well written and easy to follow. 

- Generalization-aware optimizer is very relevant.

- Technical development appears viable and clear.

WEAKNESSES:
- It is not immediately clear, on an intuition level, why this new notion of sharpness is better than loss sharpness in SAM [8].

- Table 2 only presents SAM on two algorithms (REINFORCE and GA) with no error interval reported. A more throughout benchmarking could improve the empirical evaluation.

QUESTIONS:
- As illustrated in Figure 1, the intuition is that if the oracle is included in the perturbation neighborhood, a smoother predictor tends to have smaller error. However, Table 1 observes that IGNITE applied to some sub-optimal predictors, for example REINFORCE in Ant Morphology, it still achieves good performance. Since the suboptimal prediction performance may imply that $\hat{\omega}$ is not proximal to $\omega^\star$ hence the local approximation may break in practice. I am curious that why IGNITE should still work in this case, I appreciate if the authors could provide some high-level intuitions.

LIMITATIONS:
See above.


---
post-rebuttal: -> 5
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of offline optimization for material design problems. The paper proposes a model-agnostic method that changes the parameters of a model by constraining the sharpness of the model's predictions. Since the model generates smoother predictions, the error between the predictions and the oracle tends to be smaller and therefore the performance of the offline model is potentially more accurate. The paper gives a theoretical analysis and shows that the empirical sharpness can bound the theoretical sharpness. The paper applies the method to various offline optimization method and shows improvements in most of the cases.\n\nSTRENGTHS:\n1. The paper starts with strong intuition, proposes a general optimization objective, and proposes a feasible algorithm to solve the objective efficiently. The paper also gives a theoretical analysis that connects the empirical objective to the theoretical sharpness.\n\n2. The proposed method is model agnostic and therefore the method can be applied to many existing offline optimization methods to get improved performance.\n\n3. The proposed method is shown to achieve good performance across a variety of tasks and is applied with various offline optimization methods.\n\nWEAKNESSES:\n1. After certain approximations, the objective is optimized using BDMM. It would be interesting to analyze the specific structures of the objective and see if more effective optimization techniques can be applied. This is also related to the model-agnostic feature of the proposed method. While having the model-agnostic property is convenient, it may also miss the specific structures of the problems/models so that the performance is not ideal. Also, there is no empirical evidence showing whether the optimization is effective or how well the optimization converges in practice.\n\n2. The comparisons with baseline methods are only compared for a limited number of tasks. To fully demonstrate the effectiveness, I think a thorough comparison as done in Table 1 would be necessary.\n\n3. The paper very briefly mentions the background of material design. I think a more comprehensive introduction could be beneficial and probably the paper should add detailed descriptions of the tasks and datasets.\n\nQUESTIONS:\nPlease address the weaknesses part.\n\nLIMITATIONS:\nLimitations are properly addressed. No obvious negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a model-agnostic regularization method that reduces surrogate model sharpness to improve generalization in offline optimization. By incorporating a surrogate sharpness measure into the training loss, they provide theoretical proof and extensive experimental validation showing that this approach enhances performance on unseen data, achieving up to a 9.6% improvement in various optimization tasks. The proposed algorithm, IGNITE, demonstrates the effectiveness of this technique across different surrogate models and tasks.\n\nSTRENGTHS:\n1. This paper introduces a novel concept of surrogate sharpness for offline optimization, providing a new robust optimization approach that differs from previous loss-based sharpness-aware minimization methods.\n2. Based on surrogate sharpness, the authors propose a new algorithm, IGNITE, and provide corresponding theoretical analysis, demonstrating its ability to bound the worst-case generalized surrogate sharpness.\n3. The proposed method can be combined with other offline optimization techniques, consistently enhancing generalization performance when integrated.\n\nWEAKNESSES:\n1. The Introduction section does not clearly emphasize the difference between surrogate sharpness and loss sharpness. Figure 1 could be positioned earlier in the paper. Additionally, the annotations and explanations in Figure 1a are somewhat confusing; the concepts of \\(\\sigma\\) and \\(\\sigma^*\\) are not adequately explained in the figure and caption, which could lead to misunderstandings.\n2. The Theoretical Analysis section (Section 4) should include more insightful discussions based on Theorems 1 and 2, ideally tying them to the results in the experimental section. The formatting of Equation 16 could also be improved.\n3. In the experimental section, the authors mention that the IGNITE algorithm introduces five hyperparameters, but the ablation study only analyzes the effects of a subset of these hyperparameters. Many experimental results are placed in the supplementary materials, but they are not referenced in the main text.\n\nQUESTIONS:\nPlease see the weaknesses part.\n\nLIMITATIONS:\nPlease see the weaknesses part.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposed IGNITE, a promising method for solving out-of-distribution issue in offline training by introducing model sharpness into the training loss of the surrogate as a regularizer. The key innovation of IGNITE lies on incorperating sharpness-aware minimization into offline training and developing theoretical analysis to show that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Experiments on benchmark datasets show the promising performance.\n\nSTRENGTHS:\n1. Well motivated.  \n2. Easy to follow. I like the style of writing.\n3. Experimental results seem promissing (9.6% performance boost).\n4. Solid theoretical analysis are provided.\n\nWEAKNESSES:\nAlthough the paper has many strengths, there are still some weaknesses:\n1. Sharpness-aware minimization (SAM) is widely explored in many fields. I think this submission lacks sufficient introducing of SAM, especially on its development. Take some recent works as an example:  \n\nEnhancing sharpness-aware optimization through variance suppression.  (NeurIPS'23)  \nNormalization layers are all that sharpness-aware minimization needs.  (NeurIPS'23)  \nDomain-Inspired Sharpness-Aware Minimization Under Domain Shifts.  (ICLR'24)  \nLocally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization.  (ICML'24)  \n\n2. The gain on performance is not always stable, as shown in the Table 4 and 5.  \n3. Lacking sharpness visualization of objectives on unseen data.\n\nQUESTIONS:\n1. I recommend the authors to extent the contends on the related works of sharpness-aware minimization.\n2. Can the authors explain the potential reason on the performance drop shown in Table 4 and 5?\n3. Can the authors show the sharpness of objectives on unseen data before and after using your algorithm?\n\nLIMITATIONS:\nI don't think there are any negative societal impacts of this work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on offline optimization, presenting a novel approach to enhance the surrogate landscape's sharpness, thereby improving generalization. The authors introduce a gradient norm as an approximation of surrogate sharpness through a first-order Taylor expansion, resulting in a Lagrangian formulation. They employ two widely adopted methods—BDMM and a fixed multiplier as solvers for the Lagrangian.\n\nTheoretically, the authors provide a generalization bound for the worst-case prediction change. Empirically, they validate their approach using the widely adopted offline optimization dataset, design-bench, demonstrating relatively good performance.\n\nSTRENGTHS:\n1. The idea of using a sharpness-aware method in offline optimization is well-motivated.\n\n2. The proposed method performs well on some datasets with specific baselines.\n\n3. The authors provide a generalization analysis.\n\nWEAKNESSES:\nThough I agree with the ideas of using a sharpness-aware method in offline optimization, I think more details should be justified:\n\n\n1. I am somewhat unsure whether the authors analyze the generalization bound on $\\mathcal{R}_{X}(\\omega)$ properly. \n\nDoes the generalization over $R_{X}(\\omega)$ help to improve the generalization of $L_{X}(\\omega)$, which is the objective we want to generalize. Why not analyze the generalization bound w,r,t $L_{X}(\\omega)$?\n    \n2. Is the bound tight? I understand that the original SAM paper [8] also provides a similar bound. But if it is not tight, we do not know whether it can guide the design of our algorithm.\n\n3. Moreover, the authors should add more details to their proofs to help readers better understand. For example, in line 416, the authors should present it as a lemma rather than directly stating the results of [8] as there are some assumptions in their proofs which might not be discussed in your proof. In line 458, the authors say that they use Taylor's remainder term, but it seems that the remainder term is directly eliminated. Could the authors discuss more details?\n\n4.  Another concern is that I am not very sure whether Assumption 2 is a strong assumption. If the assumption holds, then the Hessian is positive definite. Since the proof highly relies on it, I am afraid the generalization bound is not established in the general stationary point  (the Hessian is positive semi-definite rather than positive definite).\n\n5.  Though the proposed method shows relatively higher performance than some corresponding baselines on some datasets, the overall improvements are marginal and sometimes even decrease, as shown in table 1. Moreover, different percentages show vastly different performance. For example, reinforce+IGNITE shows a 6.5\\% gain over reinforce at the 100th percentile level, but a -4.6\\% at the 80th percentile level. How can this be explained? Does it mean that the proposed method is not robust?\n\n6. Is there any analysis on memory and time? The term includes the gradient of the gradient $\\nabla_{\\omega}|| \\nabla_{\\omega} h\\left(\\omega^t\\right) ||$, which I am concerned is both time and memory-consuming.\n\n\nIn all, I think the idea is interesting, and the empirical and theoretical results are promising. But I still think more details should be added before I can vote for acceptance.\n\nQUESTIONS:\nPlease answer my question mentioned above.\n\nLIMITATIONS:\nPlease see the weaknesses part.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-21"}