Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques

Authors: manh cuong dao
phi  le nguyen
thao  nguyen truong
nghia  trong
  hoang
trong  nghia
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 07:44:04

Abstract: Offline optimization has recently emerged as an increasingly popular approach to mitigate the prohibitively expensive cost of online experimentation. The key idea is to learn a surrogate of the black-box function that underlines the target experiment using a static (offline) dataset of its previous input-output queries. Such an approach is, however, fraught with an out-of-distribution issue where the learned surrogate becomes inaccurate outside the offline data regimes. To mitigate this, existing offline optimizers have proposed numerous conditioning techniques to prevent the learned surrogate from being too erratic. Nonetheless, such conditioning strategies are often specific to particular surrogate or search models, which might not generalize to a different model choice. This motivates us to develop a model-agnostic approach instead, which incorporates a notion of model sharpness into the training loss of the surrogate as a regularizer. Our approach is supported by a new theoretical analysis demonstrating that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our analysis extends existing theories from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate on training data, providing a new perspective on sharpness regularization. Our extensive experimentation on a diverse range of optimization tasks also shows that reducing surrogate sharpness often leads to significant improvement, marking (up to) a noticeable 9.6% performance boost. Our code is publicly available at https://github.com/cuong-dm/IGNITE.

Main paper: 
 introduction: 
 A central task in numerous scientific disciplines is to optimize for some material configuration that maximizes a certain utility metric. Previously, this would incur an expensive and repetitive experiment process that requires a huge amount of human-labor. To bypass such inefficiencies, a data-driven approach [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] has recently been adopted. Instead of laboring on a new set of expensive on-demand experimentation for each new optimization task, we can leverage past experimentation data to build a parametric model that predicts the outcome of the experiments themselves. Its parameters are tuned to fit past experimental data and then fixed while optimizing for the best input.However, in most applications, offline data is rarely representative of the entire input space. As a result, the surrogate model's prediction is often not accurate outside the offline data regime, potentially overestimating the outputs at sub-optimal input candidates. To mitigate this, existing offline optimizers have introduced numerous regularizing strategies for either the surrogate or the search models. For example, [4, 11, 12, 13] and [3] regularize their surrogate models so that their predictions at inputs outside the offline data regime are pushed down or pushed towards a constant value. Alternatively, [1] and [2] restrict their search models to regions having certain domain-specific properties under which sampled inputs probably have high performance.These strategies therefore depend on the specifics of either the surrogate-or the search procedures to characterize the out-of-distribution regions or the desirable domain-specific properties. Consequently, their regularization might not extend to out-of-distribution regions which are not sufficiently specified. For example, COMs [13] uses an ad-hoc specification that characterizes the out-of-distribution region in terms of inputs that are reached during the first few iterations of gradient ascent on an un-regularized surrogate. This only characterizes a local sub-region of a broader out-of-distribution data regime.To mitigate such limitations in the regularizing behaviors of prior work, we instead aim to investigate a more generic approach that is independent of the specifics of the search or surrogate procedures. For instance, instead of regularizing the behavior of the model on certain input regions, we could impose constraints on the geometries of its loss landscape such as requiring it to be in a parameter regime that uniformly produces low loss values [14] . Such regularizing strategies can, therefore, be incorporated into existing offline optimizers as an additional regularizer to improve their performance. To substantiate this idea, we have made the following technical contributions:1. We develop a model-agnostic regularizer (for surrogate training) based on a notion of model sharpness 2 . This is characterized in terms of the surrogate's maximum output change under lowenergy parameter perturbation (i.e., norm-bound perturbation) (Section 3.1). Intuitively, suppose a surrogate's prediction does not change substantially within a perturbation neighborhood (i.e., via adding perturbation to its parameters) that contains the oracle, its predictions are likely to be close to those of the oracle (see Fig. 1 ). As such, minimizing this sharpness measure can help suppress the erratic behavior of the surrogate model at out-of-distribution input.2. We adopt a practical approximation that interestingly reduces the above surrogate sharpness measurement to a function of the surrogate's gradient norm. The surrogate training can then be augmented into a constrained optimization task whose constraint imposes a user-specified threshold on the surrogate's gradient norm. We can then solve it to acquire the optimally regularized surrogate using existing constrained optimization solvers. The high-level pseudo-code of our proposed algorithms which incorporate surrogate gradient norms to improve existing offline optimization techniques (IGNITE) is detailed in Algorithm 1 (Section 3.2).3. We develop a detailed theoretical analysis to show that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Our analysis extends existing theories [14] from bounding generalized prediction loss (on unseen data) with loss sharpness to bounding the worst-case generalized surrogate sharpness with its empirical estimate on training data, providing a new perspective on sharpness regularization (Section 4).We demonstrate empirically that incorporating the proposed model-agnostic regularizer into existing offline optimizers as an additional conditioning component often results in significant improvement over existing offline optimizers in most cases. This sets the first step towards a new, synergistic research direction in offline optimization that can potentially support and complement both existing and future work (Section 5).For interested readers, a concise review of existing literature is also provided in Section 2.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: Our contributions can be found in Section 3 and Section 4. 
problem definition and related works: 
 In this section, we will concisely review the preliminaries of offline optimization. Section 2.1 provides a mathematical formulation of offline optimization, and Section 2.2 summarizes prior works.Offline optimization is a computational approach to a variety of material engineering tasks that aim to find a material construction or design that maximizes certain desirable properties. Mathematically, we assume that there is an oracle function g(x) that maps from a material design x ∈ X to an overall utility z = g(x) of its property measurements; and we need to find its maxima:x * ≜ arg max x∈X g(x) .(1)However, the key challenge here is that g(x) is inaccessible. Instead, we only have access to an offline dataset of observations D = {(x i , z i )} n i=1 where z i = g(x i ), which denote the past input-output queries extracted from g(x) in previous experiments. A direct approach to this problem is to learn a surrogate g(x; ω * ) of g(x) via fitting its parameter ω * to the offline dataset,EQUATIONwhere ω denotes a parameter candidate of the surrogate and ℓ(g(x; ω), z) denotes the prediction loss of g(.; ω) on x if its oracle output is z. The (oracle) maxima of g(x) is then approximated via,EQUATIONSuppose the surrogate g(x; ω * )'s prediction is sufficiently accurate over the entire input space, solving Eq. ( 3) is all we need. However, in most cases, g(x; ω * ) often predicts erratically outside the offline data regime, which in turn misleads the optimization towards sub-optimal candidates. To mitigate this, numerous surrogate or search regularizers have been proposed, as summarized next.Most existing offline optimization methods have focused on regularizing either the search or the (surrogate) training procedures. The main focus is to either (1) avoid exploring input regions where the surrogate's prediction is not reliable or (2) regularize the prediction behavior of the surrogate at out-of-distribution input regimes. For example, to regularize the surrogate's prediction, [13] uses input candidates found during the first few iterations of gradient updates on un-regularized models to characterize the out-of-distribution regime. The surrogate can then be re-trained with an additional regularizer that penalizes high-value predictions at those sampled input candidates.Alternatively, [11] maximizes the normalized data likelihood to reduce prediction uncertainty, which also helps suppress erratic prediction at out-of-distribution regime. Existing techniques in model pre-training and adaptation [3] or transfer learning via co-teaching [15] can also be leveraged to enforce criteria of local smoothness that encodes a preference for conservative prediction to avoid overestimating the oracle output. Otherwise, to regularize the search procedure, [1] and [2] focus instead on learning a generative model of input candidate conditioned their oracle performance. Input candidates that likely achieve high performance can then be synthesized via conditioning the generative procedure on high-value oracle output. [1] characterizes such conditioned distribution via an adversarial zero-sum game. [12] learns a direct inverse mapping from the performance output to the input design using conditional generative adversarial network [16] .Despite their reported successes, these approaches are still limited by their ad-hoc characterization of the out-of-distribution regime. As discussed previously in Section 1, the existing characterization of out-of-distribution input is often based on the specifics of either the surrogate or the search procedures, which are not guaranteed to sufficiently characterize the entire out-of-distribution data regime. This motivates us to develop a more generic out-of-distribution characterization (Section 3) that is external to both the search and surrogate models. Such an approach can be readily incorporated into most existing offline optimizers to boost their performance (Section 5).The loop runs for T iterations. Thus, the complexity of the main loop will be multiplied by T . Overall Complexity: Considering the above steps, the most computationally expensive parts are the gradient computations in lines 5, 6, and 8. Thus, the overall complexity per iteration is:O(2m • C g + m • C ℓ + 2m • C ẑ )Since this loop runs for T iterations, the total complexity is:O(T • (2m • C g + m • C ℓ + 2m • C ẑ ))Furthermore, we have the total complexity of the original baseline is: Thereby, IGNITE will include an additional complexity:O(T • (m • C g + m • C ℓ ))O(T • (m • C g + 2m • C ẑ )) = O(T md)where:• T is the number of iterations.• m is the batch size.• C g = O(d) is the complexity of evaluating the surrogate model.• C ℓ = O(d) is the complexity of computing the loss gradient.• C ẑ = O(d) is the complexity of computing the gradient of the surrogate output with respect to its parameters. • d is the no. of surrogate parameters.The empirical training time of the participating baselines with and without IGNITEis reported in Table 9 . We observe that IGNITE solely consumes an additional negligible GPU memory, while the training time increases by 14.91% on average.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss this in Appendix K Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
surrogate regularization with sharpness constraint: 
 This section introduces an additional constraint that transforms the original surrogate optimization in Eq. ( 2) (Section 3.1) into a constrained optimization problem (COP). The constraint imposes a user-specified upper-bound on the sharpness of the surrogate. Our proposed formulation draws inspiration from a prior work [14] that aims to minimize the sharpness of the loss function to improve generalization. However, unlike the original work in [14] , where the sharpness concept is applied to the loss function, we adapt it for the surrogate's prediction. We find it more suitable since offline optimization's search procedure operates on the surrogate landscape instead of the loss landscape. Moreover, while minimizing loss sharpness [14] can ensure low error for single predictions in the OOD regime, errors may accumulate over consecutive predictions in a gradient-based search. Our insight in Section 3.1 (Fig. 1 ) suggests that such error accumulation can be mitigated by keeping the surrogate sharpness small during training. Furthermore, we show that the sharpness can be practically approximated in terms of the surrogate's gradient norm, which is more tractable. This allows for direct adoption of existing constrained optimization algorithms [17] to effectively solve for the desired optimally regularized surrogate (Section 3.2). where ω 1 has a smaller sharpness than ω 2 . This means the predictions of the models in the perturbation neighborhood of ω 1 will vary less than those of the models in the perturbation neighborhood of ω 2 . As such, if both neighborhoods contain the oracle, the prediction error d 1 of ω 1 is potentially smaller than the prediction error d 2 of ω 2 . Consequently, the optimal value of g(x; ω 1 ) is closer to the oracle optimal value than g(x; ω 2 )'s.Suppose the oracle lies within the parametric family of the surrogate, there must exist a perturbation neighborhood of the surrogate's parameters that contains the oracle. That is, the oracle can be obtained by adding to the surrogate's parameters a noise vector in this neighborhood. Now, suppose the predictions do not change substantially across the models (including both the oracle and surrogate) in the perturbation neighborhood; the surrogate's predictions (and optimizers) must be close to those of the oracle (see Fig. 1 ). Motivated by this insight, we consider a potential approach to mitigate the erratic prediction of the surrogate, which is to ensure that its worst-case prediction change across the perturbation neighborhood is sufficiently small. This is formalized below:EQUATIONwhere ∥ • ∥ 2 denotes the ℓ 2 -norm, and ρ > 0 bounds the maximum norm of the perturbation, which defines the perturbation neighborhood and can be selected via hyper-parameter tuning (see Section 5.1 and Appendix G.4). To ease the notation, we will omit the subscript and use ∥.∥ to consistently denote the ℓ 2 norm in the rest of this manuscript. We will also refer to R X (ω) in Eq. ( 4) as the generalized surrogate sharpness, which can be used to regularize surrogate training,EQUATIONwhere ϵ ′ is a user-specified threshold. Solving Eq. ( 5) is non-trivial since R X (ω) is neither tractable nor differentiable. To sidestep this, we leverage a theoretical result that will be established later in Section 4, which (informally) states that with high confidence, the generalized surrogate sharpness R X (ω) can be approximated with its empirical estimate R D (ω) in Eq. ( 16),EQUATIONwhere G(ω) and λ max denote the norm of the expected (parameter) gradient of the surrogate at ω, and the largest eigenvalue of the (parameter) Hessian of the surrogate's expected prediction,EQUATION)When n is sufficiently large, (dim(ω) log(n∥ω∥ 2 )/n) 1 2 will be negligibly small. Then, suppose within the search region for ω (see Assumption 2), G(ω) is bounded by a constant G, we can enforceEQUATIONTo mitigate the non-differentiability of R D (ω), we further propose a practical approximation which reduces R D (ω) to a function of the surrogate's gradient norm ∥∇ ω g(x; ω)∥, which is differentiable, allowing the above COP to be solved effectively with existing constrained optimization algorithms. This is discussed in the next section.Remark. Eq. ( 4) is similar in spirit to the notion of loss sharpness [14] in which the same perturbation model is used to characterize the sharpness or sensitivity of the loss function,|L D (ω + δ) -L D (ω)|,to small changes ∥δ∥ ≤ ρ in the model weights ω. Our work, however, sets a direct focus on the sensitivity or sharpness of the surrogate's prediction -Eq. ( 4) -which results in a better surrogate regularizer, leading to better empirical performance (see Section 5.3). It also requires a significant and non-trivial adaptation of the theories presented in [14] , as detailed later in Section 4.Leth(ω + δ) ≜ E x∈D [g(x; ω + δ)] and h(ω) ≜ E x∈D [g(x; ω)].The surrogate sharpness can be approximated via the first-order Taylor expansion of h(ω + δ) at ω:EQUATIONUsing the Cauchy-Schwartz inequality on the right-hand side of the above and noting that δ can be selected to make the equality happens,EQUATIONUsing the above approximation, the COP in Eq. ( 8) can be rewritten asEQUATIONwhich can be solved via optimizing the corresponding Lagrangian,EQUATIONwith λ > 0 denotes the Lagrange multiplier. This can be solved via our approach below.IGNITE. We can optimize for both λ and ω using the basic differential multiplier method (BDMM) [17] , which simultaneously gradient ascent for λ and gradient descent for ω, resulting in the following update rules:EQUATIONEQUATIONInput: offline data D = {(xi, zi)} n i=1 ; initial surrogate g(x; ω (0) ); no. of iterations T ; batch size m; Lagrange multiplier λ; perturbation radius ρ and scalar r; step sizes ηω and η λ ; threshold ϵ. 1: Initialize ω (1) ← ω (0) and λ (1) ← λ 2: for t ← 1 : T do 3:Sample B = {(xi, zi)} m i=1 ∼ D 4: Compute ẑi = g(xi; ω (t) ) for i ∈ [m] 5: Compute g1 = m -1 m i=1 ∇ωℓ(ẑi, zi) 6: Compute g2 = m -1 m i=1 ∇ω ẑi 7: Compute ω = ω (t) + r • g2/∥g2∥ 8: Compute g3 = m -1 m i=1 ∇ωg(xi; ω) 9:Compute g (t) = g1 + λ (t) ρr -1 (g3 -g2) 10:Update ω (t+1) ← ω (t) -ηωg (t) 11:Update λ (t+1) ← λ (t) + η λ (ρ∥g2∥ -ϵ) 12: end for 13: return learned surrogate ω (T +1) where ω t represents the surrogate's parameter estimate at iteration t, λ t is the Lagrange multiplier estimate at iteration t, η ω is the step size for updating ω, and η λ is the step size for updating λ. We name this method IGNITE. We also conduct grid search to select the optimal value for ρ, as mentioned in Section 5.1.Remark. As an alternative approach, we can also treat λ as a hyper-parameter and optimize for ω using gradient descent; we call this method IGNITE-2 . The detailed algorithms, hyper-parameter selection, and experimental results of IGNITE-2 are reported in Appendix G.2. Both of the above methods require differentiating ∥∇ ω h(ω)∥ with respect to ω, which involves computing the expensive Hessian of h(ω). Fortunately, this expensive computation can be avoided by using the gradient approximation technique detailed in Appendix F. To summarize, we provide a complete pseudo-code of IGNITE in Algorithm 1 whose steps 3-8 implement the approximation in Eq. (84) and Eq. (85) of Appendix F.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: Our experimental code is also submitted (as extra materials) along with our manuscript.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
theoretical analysis: 
 In this section, we will provide a detailed theoretical analysis to substantiate our earlier (informal) statement in Eq. ( 6) that with high confidence, reducing the empirical sharpness R D (ω) on the offline data will also reduce its generalized sharpness R X (ω). We will show that this is true (see Theorem 1) under certain choices and mild assumptions of the surrogate model (see Assumptions 1 and 2). Assumption 1. The output of the above surrogate function g(x; ω) is bounded within [0, 1]. Assumption 2. λ min ∇ 2 ω E[g(x; ω)] > 0 for all ∥ω∥ ≤ τ for some τ > 0.We note that the specific bound within [0, 1] in Assumption 1 is meant to ease the technical presentation of our theoretical analysis. Otherwise, it can be extended straightforwardly to any bounded functions. Furthermore, we also show below that it is indeed possible to find a non-trivial surrogate function that is bounded and satisfies Assumption 2.Theorem 1. There exists τ > 0 and ω + , and a non-linear function r(x; ω) such that,g(x; ω) ≜ r(x; ω + ) + ω -ω + ⊤ ∇ ω r x; ω + + 1 2 ω -ω + ⊤ ∇ 2 ω r x; ω + ω -ω + (15)satisfies Assumption 2 and is bounded on {ω | ∥ω∥ ≤ τ }. Detailed derivation of this theorem is deferred to Appendix A.For such surrogate functions, their generalized sharpness can be upper-bound by a function of their empirical sharpness on the offline data. As detailed below, the bound depends on both the size of the surrogate dim(ω) and the number n of offline data points.Theorem 2. For any ρ > 0, m = dim(ω) and 2/(mλ min ) ≥ σ 2 > 0 with λ min being defined in Assumption 2, the following holds simultaneously for all g(.; ω) for which Assumption 2 is met,EQUATIONwith probability at least 1 -α over the random choice of the offline dataset, and with P (n, m, α) = 2 log(n/α) + 4 log(8n + 4m). Detailed derivation of this theorem is deferred to Appendix E.Proof Sketch. For clarity, we will provide below a proof sketch of Theorem 2, which highlights the key steps in our derivation. Due to limited space, the specific of each step is deferred to the Appendix E. First, we note thatEQUATIONEQUATIONA relation between R X (ω) and R D (ω) can then be derived in three steps:1. Upper-bound E δ∼N(0,σ 2 I) [F X (ω + δ)] with a function of E δ∼N(0,σ 2 I) [F D (ω + δ)].This can be achieved via a direct application of the PAC-Bayes bound [18] which views the perturbed model ω + δ as a random hypothesis sampled from the posterior N(ω, σ 2 I) -see Appendix B.2. Upper-bound E δ∼N(0,σ 2 I) [F D (ω + δ)] with a function of R D (ω). This can be achieved using a similar proving technique adopted from [14] -see Appendix C.3. Find ξ > 0 such that R X (ω) can be upper-bounded with ξ • E δ∼N(0,σ 2 I) [F X (ω + δ)]where ξ is a small scaling factor. To achieve this, we will find the lower-bound for E δ∼N(0,σ 2 I) [F X (ω + δ)] using the Taylor remainder theorem to expand it around ω, which can be lower-bounded using the minimum eigenvalue of its Hessian at ω. Using the same approach, we can upper-bound R X (ω) with the maximum eigenvalue of the same Hessian -see Appendix D.Finally, we set ξ so that the upper-bound of R X (ω) is smaller than the multiplication of ξ with the lower-bound of E[F X (ω + δ)]. To tighten the bound, we choose the smallest possible value of ξ such that the bound still holds. Lining up the results of the above steps then shows that R X (ω) can then be bounded with a function of ξ • R D (ω). See Appendix E for a complete derivation.Remark. Note that Theorem 2 is more general than its informal statement in Eq. ( 6), which can be reproduced by choosing σ 2 = 2/(mλ min ) in Eq. ( 16) to make the bound tightest. 
experiments: 
 This section evaluates the efficacy of our proposed method IGNITE in improving state-of-the-art offline optimizers. We describe our experiment settings in Section 5.1 and report detailed empirical results in Section 5.2. We also provide additional ablation studies of our method in Section 5.3.Benchmark Tasks. Our explorations focus on four real-world tasks from Design-Bench3 [4] , covering both discrete (TF-Bind-8 and TF-Bind-10) and continuous domains (Ant Morphology [19] and D'Kitty Morphology [20] ).Baselines. We meticulously curated a diverse set of 11 widely acknowledged offline optimizers for comparative analysis. This ensemble comprises BO-qEI [4] , CbAS [1] , RoMA [3] , ICT [15] , Base 0.812 ± 0.000 0.896 ± 0.000 0.787 ± 0.112 0.628 ± 0.000 IGNITE 0.812 ± 0.000 +0.0% 0.896 ± 0.000 +0.0% 0.843 ± 0.109 +5.6% 0.628 ± 0.000 +0.0%Base 0.937 ± 0.023 0.946 ± 0.014 0.892 ± 0.055 0.647 ± 0.025 IGNITE 0.935 ± 0.032 -0.2% 0.962 ± 0.018 +1.6% 0.923 ± 0.038 +3.1% 0.652 ± 0.074 +0.5% CMA-ES [21] , COMs [13] , MINs [12] , REINFORCE [22] , and three variations of gradient ascent (GA, ENS-MIN, ENS-MEAN), corresponding to vanilla gradient ascent, the min ensemble of gradient ascent, and the mean ensemble of gradient ascent, respectively. Evaluation Protocol. To ensure a comprehensive assessment, we follow the methodology in [4] . Each method generates 128 optimized design candidates evaluated by the oracle function. The candidates' performances are ranked, and the 50-th, 75-th, and 100-th percentile levels are recorded. All results are averaged over 16 independent runs to ensure reliability.Hyper-parameter Configuration. For each baseline algorithm, we carefully configure optimal hyper-parameters as outlined in [4] . Our method IGNITE introduces five additional hyper-parameters: λ, ρ, r, η λ , and ϵ. The hyper-parameter λ, an initial value for the regularizer coefficient, is set to 0.01 through a grid search within {0.0001, 0.001, 0.01}. The hyper-parameters ρ and r are chosen from {0.01, 0.05, 0.1, 0.2}, with ρ set to 0.05 and r set to 0.05 for IGNITE. Additionally, IGNITE uses η λ = 1e -3 and ϵ = 0.1, which are determined via the experiments in Section 5.3.In this section, we presented the percentage improvement over baseline performance attained by IGNITE when it is applied to existing baselines. We have evaluated this at the 50-th, 80-th, and 100-th percentile levels. However, due to limited space, we only report results of the 100-th percentile level in the main text. The other results are instead deferred to Appendix G.3.The first two columns of Table 1 show that out of 22 cases involving 11 baseline algorithms across 2 tasks, the IGNITE regularizer enhances baseline performance in 18 cases, with improvements reaching up to 9.6%. In only 1 out of 22 instances IGNITE slightly decreases performance by 0.2%, which is negligible. Even in cases where performance is not improved, IGNITE reduces the variance in results from 0.2% to 0.1% for the CMA-ES baseline on the D'Kitty Morphology task. Additionally, IGNITE helps establish new state-of-the-art (SOTA) Results on Discrete Tasks: The last two columns of Table 1 show the impact of the IGNITE regularizer on the performance of baseline algorithms in two discrete domains (TF-BIND-8 and TF-BIND-10). Similar to the continuous tasks, IGNITE significantly enhances baseline performance in most cases (17 out of 22), with improvements of up to 5.6%. There are only 3 instances where integrating IGNITE results in a minor performance decrease of up to 0.7%, which is negligible. Additionally, in certain instances, IGNITE not only improves baseline performance but also establishes new state-of-the-art (SOTA) results. For example, on TF-BIND-8 and TF-BIND-10, the original SOTA performances of 98.5% and 68.1% achieved by ENS-MEAN and ENS-MIN, respectively, are elevated to 98.7% and 70.5% with the addition of IGNITE , setting new SOTA records.In summary, IGNITE consistently maintains a high probability of 91% (40 out of 44) of not degrading baseline performance. There is a high likelihood (79.55% = 35 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1.91% and a notable peak improvement of 9.6%. Conversely, IGNITE also exhibits a relatively low probability (9.09% = 4 out of 44 cases) of decreasing performance, with an average degradation of approximately 0.3% and a minor peak degradation of 0.7%.Additionally, there is a minor probability (11.36% = 5 out of 44 cases) of maintaining baseline performance.In this section, we conduct additional experiments to assess the sensitivity of two representative baselines, COMs and GA, when regularized with IGNITE, to variations in hyper-parameters ϵ and η λ . Additionally, we perform experiments to compare the efficacy of IGNITE with other commonly used regularization methods.Changing threshold ϵ. We assess the performance enhancement of COMs and GA when regularized with IGNITE using various values of ϵ from the set {0.01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5}. A high ϵ value may result in a surrogate that is overly sharp, potentially hampering the search process and hindering the discovery of optimal designs. Figure 2 (a) demonstrates that excessively high ϵ values lead to negative improvements. Conversely, excessively low ϵ values may cause the regularizer to dominate the original loss, resulting in a surrogate that is not well-fitted to the offline data. Negative improvements are observed with ϵ = 0.01 and 0.05 in Figure 2(a) . As a result, we determine ϵ = 0.1 as the optimal value based on these observations. Changing step size η λ . The step size η λ controls the rate at which λ is updated during the optimization process. It's essential to choose an appropriate step size, avoiding it being either too large or too small. Figure 2 (b) demonstrates that an η λ value of 1e -3 yields optimal results. Comparing IGNITE with other regularization methods. We conduct a comparative experiment to assess the performance improvement achieved by using the IGNITE regularizer compared to other regularizers, including L1, L2, and SAM [14] (where SAM is considered as a loss sharpness regularization with a coefficient equal to 1). Surrogate sharpness on unseen data before and after using IGNITE. We also conduct an experiment measuring the sharpness of the surrogate model, approximated by ρ∥∇ ω h(ω)∥ as defined in Eq. ( 10), with and without IGNITE. This sharpness is computed on unseen data points which are, specifically, the design candidates generated by the GA and REINFORCE baselines. By testing on these unseen candidates, we simulate the out-of-distribution (OOD) conditions that are critical in assessing generalization in optimization tasks. Table 3 reports these surrogate sharpness measurements for some baselines with and without using the IGNITE regularizer. The results demonstrate consistently that the IGNITE regularizer helps reduce the surrogate sharpness on unseen data, as anticipated. This reduction indicates that IGNITE is effective in smoothing the surrogate model's landscape, leading to better and more stable generalized performance.Results on tasks with noisy data. To demonstrate IGNITE's robust performance in scenarios with noisy oracle, we conducted additional experiments using the GA and REINFORCE baselines on two benchmark tasks with particularly noisy oracles: Superconductor and Chembl. For each baseline, we compared its achieved performance with and without the regularizing effect of IGNITE, as shown in Table 4 . Across both tasks, IGNITE helps improve the baseline performance substantially, achieving up to a 2.1% increase for REINFORCE on the Superconductor task, highlighting IGNITE's ability to enhance performance robustness even in settings with noisy oracles. 
conclusion: 
 This paper introduces the concept of generalized surrogate sharpness in offline optimization, resulting in the development of a new regularization technique, IGNITE. Our theoretical analysis demonstrates that reducing surrogate sharpness on an offline dataset provably decreases its generalized sharpness on unseen data. Empirically, IGNITE consistently maintains a high probability (91%) of not degrading baseline performance and a 79.55% likelihood of improving it, with a peak improvement of 9.6%. Additionally, we believe that our novel technique can be adapted to related domains such as robust optimization (RO) and reinforcement learning (RL), suggesting potential future research directions.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: Such details can be founded in Section 5.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We do report error bars in our experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our work does not create new datasets. We only use existing, publicly available datasets. We also do not create any new pre-trained models.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the source of all datasets used in our experiments.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: Our work does not release any new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: Our work does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: Our work does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A Proof of Theorem 1Part I. Minimum Eigenvalue of Parameter Hessian is Positive.First, we re-state the choice of our surrogate in Theorem 1,g(x; ω) ≜ r(x; ω + ) + ω -ω + ⊤ ∇ ω r x; ω + + 1 2 ω -ω + ⊤ ∇ 2 ω r x; ω + ω -ω + (19)Regardless of the choice of r(x; ω), differentiating g(x; ω) with respect to ω yields,EQUATION)which implies the parameter Hessian of ∇ 2 ω g(x; ω) is always a constant matrix depending on the specific choice of r(x; ω) and a reference point ω + . Hence, it follows trivially that,EQUATIONTherefore, if we can find r(x; ω) such that its parameter Hessian is strictly positive definite at ω + , the parameter Hessian of g(x; ω) will always be strictly positive definite regardless of ω. This means g(x; ω) will be λ min -strongly convex in ω with λ min > 0. As a result, its expectation E[g(x; ω)] will also be λ min -strongly convex with λ min > 0 or equivalently, Assumption 2 holds.This can be done by choosing r(x; ω) to be any quantile prediction of a linear regressor with a (pre-trained) random feature map. For example,EQUATIONwhereEQUATIONwhere V[.] denote the variance function. We can interpret the random feature ψ(x) is sampled from a pre-trained VAE [23] with two deep neural nets characterizing the mean and variance functions, m(x) and v(x). These functions can be pre-trained and post-processed in whatever ways we need to control their value ranges. Their parameters are then frozen when we fit r(x; ω) with respect to ω.To understand the behavior of r(x; ω), we first note below the closed-form expression of Eq. ( 23),EQUATION)whereA ≜ diag[v(x)].Differentiating twice both sides of Eq. ( 24) at ω + , we haveEQUATIONWe can now choose ω + = (1/m) m i=1 e i where e 1 , e 2 , . . . , e m are the eigenvectors of A. Since A is diagonal, these are also the one-hot vectors and their corresponding eigenvalues are the entries on the diagonal of A -i.e., λ i (A) = [A] ii . With this choice, a closed-form expression of ∇ 2 ω r(x; ω + ) in terms of the entries of A can be derived as follow,EQUATION)where B is another diagonal matrix with entries[B] ii = m -2 ( m ι=1 [A] ιι -[A] ii ).As such, it is trivial to see that if we choose the activation unit (i.e., sigmoid or ReLU) for v(x) such that its output is positive, then the entries of both A and B are positive.Finally, note that Eq. ( 26) is a scaled matrix product of two diagonal matrices, A and B, which will result in a diagonal matrix. Its entries will be positive if the entries of A and B are positive, as enforced above. This means the minimum eigenvalue of ∇ 2 ω r(x; ω + ) is positive as desired. Part II. Boundedness on {w : ∥w∥ 2 ≤ τ }.Rearranging terms in the expression of g(x; ω) in Eq. ( 19),g(x; ω) ≜ ω ⊤ ∇ ω r(x; ω + ) -∇ 2 ω r(x; ω + )ω + + 1 2 ω ⊤ ∇ 2 ω r(x; ω + )ω + const (27)where const absorbs all constant vectors (i.e., not dependent on ω). Hence, taking absolute values for both sides of Eq. ( 27), we obtainEQUATIONwhere Eq. ( 30) follows from the fact that ∇ 2 ω r(x; ω + ) is positive definite by construction. Eq. (31) follows because (1/2)λ max (∇ 2 ω r(x; ω + )) is a fixed value that does not change when ω changes. Thus, g(x; ω) is bounded as expected.EQUATIONEQUATIONThe following holds simultaneously for all ω, σ 2 > 0, α ∈ (0, 1) and m = dim(ω),EQUATION+ 1 4 m log 1 + ∥ω∥ 2 2 ) mσ 2 + 1 4 + 1 2 log n α + log(8n + 4m) n -1with probability at least 1 -α over the (random) choice of the offline dataset D.Proof. We can view the random perturbation δ as a random hypothesis sampled from a hypothesis distribution (such as the zero-mean Gaussian N(0, σ 2 I)) and F X (ω + δ) as its incurred loss. In this view, for any hypothesis distribution Q, E δ∼Q [F X (ω + δ)] and E δ∼Q [F D (ω + δ)] denote the corresponding generalized and empirical Gibbs losses whose relationship is characterized via the following classical PAC-Bayesian [18] bound,EQUATIONwhich will hold simultaneously for all choices of ω, P and Q. Historically, in probabilistic learning context, P is often referred to as prior or reference distribution which, for example, encodes domainspecific information about certain properties of the solution distribution. Then, Q is the posterior, which can be selected to tighten the bound (hence, reducing the generalized loss).In our specific context, we will simply use the choices of P and Q as technical vehicles to tighten the gap between E[F X (ω + δ)] and E[F D (ω + δ)] which will contribute to the gap between our generalized and empirical surrogate sharpness.To derive the optimal choices P and Q, we will leverage part of the proof from [14] , which starts with the following fact:EQUATION)This means if ∥ω∥ > ρ 2 (exp(4n/m) -1), thenEQUATIONwhich means the bound gap in Eq. ( 34) is larger than 1 and under Assumption 1 that the surrogate's output -hence, F X (ω)'s -is bounded in [0, 1], Eq. (34) holds trivially.Hence, to choose P and Q, we can make the assumption that ∥ω∥ ≤ ρ 2 (exp(4n/m)-1). Now, let us choose both prior and posterior distribution to be Gaussians, P = N(µ P , σ 2 P I) and Q = N(µ Q , σ 2 Q I). Their KL divergence can then be computed in closed-form:EQUATION)Naively, we can minimize the above KL divergence via setting the derivative of KL with respect to σ P to zero, and solving for the optimal σ P , which givesσ P 2 = σ 2 Q + m -1 ∥µ P -µ Q ∥ 2 2 .However, for the PAC-Bayes bound to hold, σ P must be chosen independent of the rest, so the above approach does not work. Instead, we can define in advance a prior set of σ P and then choose the best one in this set, following the proof in [24] . We can choose this set as follow:Given fixed a, b > 0, let S = {c•exp((1-i)/m)|i ∈ N} be predefined set for σ 2 P . If the above bound holds for σ 2 P = c × exp((1 -i)/m) with probability 1 -α i with α i = 6απ -2 i -2 for any i ∈ N, then all above bounds hold simultaneously with probability at least 1 -EQUATIONEQUATIONPlugging Eq. (43) and Eq. (39) into Eq. (38),KL(Q||P) = 1 2 mσ 2 Q + ∥µ P -µ Q ∥ 2 2 σ 2 P -m + m log σ P σ Q 2 (44) ≤ 1 2   m σ 2 + ∥ω∥ 2 2 /m σ 2 + ∥ω∥ 2 2 /m -m + m log   exp 1 m × σ 2 + ∥ω∥ 2 2 m σ 2     (45) = 1 2   m log   exp 1 m × σ 2 + ∥ω∥ 2 2 m σ 2     (46) = 1 2 1 + m log 1 + ∥ω∥ 2 2 ) mσ 2 Q (47)Thus, Eq. (35) holds for each prior (indexed by i) in the above set with probability 1 -α i . Hence, Eq. ( 35) holds with all prior choices with probability 1i α i = 1 -α. This allows us to pick the prior that leads to the tightest bound gap, which is indexed with i = ⌊1 -m log((σ 2 + ∥ω∥ 2 2 /m)/c)⌋. For this choice, we haveEQUATIONPlugging Eq. (47) into Eq. ( 48),EQUATION+ 1 4 m log 1 + ∥ω∥ 2 2 ) mσ 2 + 1 4 + 1 2 log n αi n -1Finally, to remove the dependence on i in the above bound, note thatlog n α i = log n α + log π 2 i 2 6 (50) ≤ log n α + log π 2 (1 -m log((σ 2 + ∥ω∥ 2 2 /m)/c)) 2 6(51)EQUATIONEQUATIONEQUATIONEQUATIONPlugging Eq. (57) into Eq. ( 49),EQUATION+ 1 4 m log 1 + ∥ω∥ 2 2 ) mσ 2 + 1 4 + 1 2 log n α + log(8n + 4m) n -1 which completes our proof. C Upper-Bound E δ∈N(0,σ 2 I) [F X (ω + δ)] with R D (ω)Lemma 4. Following the same setup in Lemma 3, the following holds simultaneously for all ω, σ 2 > 0, α ∈ (0, 1) and m = dim(ω),EQUATION+ m log 1 + ∥ω∥ 2 2 ) mσ 2 + 2 log n α + 4 log(8n + 4m) n -1with probability at least 1 -α over the (random) choice of the offline dataset D.Proof. To derive this result, we need to upper-bound E[F X (ω)] with F D (ω). First, we note that δ ∼ N (0, σ 2 I) so ∥δ∥ 2 2 follows a chi-square distribution. From Lemma 1 in [25] ,EQUATIONTherefore, with probability 1 -1/ √ n we have: ∥δ∥ 2 2 ≤ σ 2 2 log √ n + m + 2 m log √ n ≤ σ 2 m 1 + log(n) m 2 ≤ ρ 2 . (61)EQUATIONFinally, plugging Eq. (62) into Eq. ( 34),E δ∼N (0,σ 2 I) F X (ω + δ) ≤ 1 - 1 √ n max ∥δ∥2≤ρ F D (ω + δ) + 1 √ n + 1 4 m log 1 + ∥ω∥ 2 2 ) mσ 2 1 + log(n) m 2 + 1 2 log n α + log(8n + 4m) n -1 ≤ max ∥δ∥2≤ρ F D (ω + δ) + m log 1 + ∥ω∥ 2 2 ) mσ 2 1 + log(n) m 2 + 2 log n α + 4 log(8n + 4m) n -1 which completes our proof. D Upper-Bound R X (ω) with a scaled value of E δ∈N(0,σ 2 I) [F X (ω + δ)]Lemma 5. Following the same setup in Lemma 3, the below holds for all σ 2 ∈ (0, 2/(mλ min )),EQUATIONwith G(ω) and λ max defined previously in Eq. ( 7), m = dim(γ) and λ min defined in Assumption 2.Proof. Let h(ω) ≜ E[g(x; ω)] where the expectation is over x ∈ X , we haveEQUATIONwith ω = ω + c • δ with some constant c ∈ [0, 1]. In the above, Eq. (65) follows from the Taylor's remainder theorem for multivariate function. Eq. ( 67) follows from the Cauchy-Schwartz inequality (for the first term) and the definition of λ max as the upper-bound on the largest eigenvalue of the parameter Hessian of h(ω). Note that λ max > λ min > 0 under Assumption 2 which stipulates that the smallest eigenvalue of the parameter Hessian of h(ω) is always positive.On the other hand, we also:EQUATIONwhere m = dim(ω). In the above, Eq. ( 70) follows from the Taylor's remainder theorem (similar to Eq. ( 65)) and Eq. ( 72) follows from the definition of λ min in Assumption 2 as the lower-bound on the smallest eigenvalue of the parameter Hessian. Since λ min > 0, the quadratic term inside the expectation is always positive which explains why we can remove the absolute operator. Eq. ( 73) follows from standard moment calculation of Gaussian random vector.Finally, we note that:G(ω)ρ + 1 2 ρ 2 λ max = 1 m • 1 σ 2 • 1 λ min • 2G(ω)ρ + λ max ρ 2 1 2 λ min • m • σ 2 (74) ≤ 1 m • 1 σ 2 • 1 λ min • 2G(ω)ρ + λ max ρ 2 • E δ∼N(0,σ 2 I) F X (ω + δ) (75)which means, intuitively,EQUATIONis the smallest scaling factor for the lower-bound of E δ∼N(0,σ 2 I) [F X (ω + δ)] so that it matches the upper-bound of R X (ω) = max ∥δ∥≤ρ F X (ω + δ), as outlined in the 3rd step of our proving plan for Theorem 2. As such, plugging Eq. ( 68) into the left-hand side of Eq. (75) completes our proof.Remark. In the above, Eq. (73) suggests that E δ [F X (ω + δ)] ≥ (1/2)λ min • m • σ 2 . But, underAssumption 1, we know that the surrogate's output -and hence F X (ω + δ)'s -is bounded within [0, 1], which means 1 ≥ (1/2)λ min • m • σ 2 or equivalently, σ 2 ≤ 2/(mλ min ) as stated in Lemma 5's statement. That is, under Assumption 1, Lemma 5 is only correct for σ 2 ∈ (0, 2/(mλ min )).We are now ready to prove our main result. First, we re-state the result of Lemma 4 below,EQUATIONwhere the last step follows from the definition of R D (ω). Finally, plugging Eq. (78) into Eq. ( 63) and replacing max ∥δ∥≤ρ F X (ω + δ) with R X (ω) (following its definition) completes our proof.Explicitly, the gradient of the augmented loss in Eq. ( 12) is given below,∇ ω L(ω, λ) = ∇ ω L D (ω) + λ • ρ • ∇ ω ∇ ω h(ω) .(79) where the computation bottleneck lies with the second gradient term which can be further expressed below using the chain rule,EQUATIONIt is evident that ∇ 2 ω h(ω) in Eq. ( 80) represents a Hessian matrix. Calculating the Hessian matrix for a deep neural network is impractical due to the extensive dimensions of the model's weights. Nevertheless, since Eq. (80) involves the multiplication of a Hessian matrix with a vector, specific techniques like Hessian-vector products can be employed to approximate this product. In particular, let the Hessian matrix M = ∇ 2 ω h(ω), we have Taylor expansion for function ∇ ω h(ω) as follows:∇ ω h(ω + ∆ω) = ∇ ω h(ω) + M∆ω + O ∥∆ω∥ 2 .(81) This approximation becomes precise as the value of ∆ω approaches 0. Following [26] and [27] , we choose ∆ω = rv where r is a small scalar and v is a vector, that transforms Eq. ( 81) intoEQUATIONThen, choosing v = ∇ωh(ω) ∥∇ωh(ω)∥ leads toEQUATIONIt is noticed that, as pointed out by [27] , an inappropriate choice of r can make Eq. ( 83) vulnerable to numeric and roundoff issues. The constant r must be sufficiently small so that the O(r) term becomes negligible. However, when r is too small, precision is compromised because the subtraction of the original gradient from the perturbed one, i.e., ∇ ω h(ω + r∇ ω h(ω)/∥∇ ω h(ω)∥) -∇ ω h(ω), will obtain a small difference between them. Based on Eq. ( 83), Eq. ( 79) would beEQUATIONIn practical applications, we typically employ an additional approximation to compute the second term in Eq. (84), thereby avoiding the need for Hessian computation induced by the chain rule.EQUATIONAll our experiments were conducted on a system with the following specifications: Ubuntu 18.04, NVIDIA RTX 3090 GPUs, and CUDA 11.8.IGNITE-2. To solve the Lagrangian in Eq. ( 12), we can treat λ as a hyper-parameter and optimize for ω using stochastic gradient descent (SGD).EQUATION)where ω t is the estimation of the surrogate's parameter at the t th iteration, and η ω is step size to update ω. We name this method IGNITE-2 and its pseudo-code is in Algorithm 2.Hyper-parameter Configuration. Our method IGNITE-2 introduces three additional hyperparameters: λ, ρ, and r. The penalty coefficient λ controls the gradient magnitude of our regularizer, set to 0.01 through a grid search within {0.0001, 0.001, 0.01}. The hyper-parameters ρ and r are chosen from {0.01, 0.05, 0.1, 0.2, 0.5}, with ρ set to 0.2 and r set to 0.2 for IGNITE. These three hyper-parameters are determined through experiments in Section G.4. These hyper-parameters are consistently applied across all experiments, except for the ICT baseline (λ = 1e -4) due to implementation differences. Input: offline data D = {(xi, zi)} n i=1 ; initial surrogate g(x; ω (0) ); no. of iterations T ; batch size m; Lagrange multiplier λ; perturbation radius ρ and scalar r; step sizes ηω.1: Initialize ω (1) ← ω (0) 2: for t ← 1 : T do 3: Sample B = {(xi, zi)} m i=1 ∼ D 4: Compute ẑi = g(xi; ω (t) ) for i ∈ [m] 5: Compute g1 = m -1 m i=1 ∇ωℓ(ẑi, zi) 6: Compute g2 = m -1 m i=1 ∇ω ẑi 7: Compute ω = ω (t) + r • g2/∥g2∥ 8: Compute g3 = m -1 m i=1 ∇ωg(xi; ω) 9:Compute g (t) = g1 + λρr -1 (g3 -g2) 10:Update ω (t+1) ← ω (t) -ηωg (t) 11: end for 12: return learned surrogate ω (T +1) G.3 Performance Evaluation at 100-th, 80-th and 50-th Percentile Level of IGNITE and IGNITE-2In this section, we presented the percentage improvement over baseline performance attained by IGNITE-2 and IGNITE when it is applied to an existing baseline. We have evaluated this at the 100-th, 80-th, and 50-th percentile levels. The results are reported in Table 5 , 6, and 7, respectively. Base 0.629 ± 0.000 0.884 ± 0.000 0.439 ± 0.000 0.513 ± 0.001 IGNITE-2 0.629 ± 0.000 +0.0% 0.884 ± 0.000 +0.0% 0.439 ± 0.000 +0.0% 0.513 ± 0.001 +0.0 % IGNITE 0.629 ± 0.000 +0.0% 0.884 ± 0.000 +0.0% 0.439 ± 0.000 +0.0% 0.513 ± 0.000 +0.0% ICT Base 0.710 ± 0.022 0.896 ± 0.005 0.687 ± 0.054 0.549 ± 0.016 IGNITE-2 0.702 ± 0.024 -0.8% 0.897 ± 0.004 +0.0% 0.680 ± 0.037 -0.7% 0.551 ± 0.022 +0.2% IGNITE 0.668 ± 0.026 -4.2% 0.895 ± 0.005 -0.1% 0.651 ± 0.039 -3.6% 0.503 ± 0.043 -4.6%According to the reported results in Table 5 , IGNITE-2 consistently maintains a high probability of 86.36% (38 out of 44) of not degrading baseline performance. There is a high likelihood (72.27% = 34 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1.39% and a notable peak improvement of 6.8%. Conversely, IGNITE-2 also exhibits a relatively low probability (13.64% = 6 out of 44 cases) of decreasing performance, with an average degradation of approximately 0.45% and a minor peak degradation of 0.8%.Additionally, there is a minor probability (9.09% = 4 out of 44 cases) of maintaining baseline performance. Furthermore, while IGNITE-2 demonstrates significant efficiency, the results in Section 5.2 indicate that IGNITE even outperforms IGNITE-2 .As shown in Table 6 , IGNITE-2 consistently maintains a high probability high probability of not degrading baseline performance, with a likelihood of 61.36% (27 out of 44 cases). There is a 47.73% chance (21 out of 44 cases) of improving baseline performance, with an average improvement of approximately 0.6% and a peak improvement of 2.4%. On the other hand, IGNITE-2 also indicates a low likelihood (38.64% = 17 out of 44 cases) of performance decrease, with an average decline of around 0.68% and a maximum decline of 4.6%. Furthermore, there is a slight chance (13.64% = 6 out of 44 cases) of maintaining the baseline performance. Base -0.043 ± 0.008 0.677 ± 0.014 0.538 ± 0.013 0.492 ± 0.013 IGNITE-2 -0.045 ± 0.008 -0.2% 0.685 ± 0.011 +0.8% 0.532 ± 0.020 -0.6% 0.493 ± 0.012 +0.1% IGNITE -0.049 ± 0.009 -0.6% 0.681 ± 0.014 +0.4% 0.542 ± 0.020 +0.4% 0.496 ± 0.013 +0.4%Base 0.569 ± 0.000 0.883 ± 0.000 0.439 ± 0.000 0.468 ± 0.000 IGNITE-2 0.569 ± 0.000 +0.0% 0.883 ± 0.000 +0.0% 0.439 ± 0.000 +0.0% 0.467 ± 0.000 -0.1% IGNITE 0.569 ± 0.000 +0.0% 0.883 ± 0.000 +0.0% 0.439 ± 0.000 +0.0% 0.467 ± 0.000 +0.0% ICT Base 0.556 ± 0.024 0.872 ± 0.007 0.564 ± 0.046 0.501 ± 0.018 IGNITE-2 0.559 ± 0.025 +0.3% 0.873 ± 0.007 +0.1% 0.554 ± 0.033 -1.0% 0.498 ± 0.022 -0.3% IGNITE 0.551 ± 0.022 -0.5% 0.878 ± 0.003 +0.6% 0.527 ± 0.030 -3.7% 0.454 ± 0.038 -4.7%In addition, IGNITE also maintains a high probability of 72.73% (32 out of 44) of not degrading baseline performance. There is a 47.73% likelihood (21 out of 44 cases) of improving baseline performance, with an average improvement of approximately 1.0% and a peak improvement of 2.6%. Besides, IGNITE also exhibits a low probability (27.27% = 12 out of 44 cases) of decreasing performance, with an average degradation of approximately 1.58% and a peak degradation of 4.6%. Additionally, there is a small probability (25% = 11 out of 44 cases) of maintaining baseline performance.Table 7 displays the results of IGNITE-2 and IGNITE at the 50th percentile level. The results indicate that both IGNITE-2 and IGNITE do not degrade performance compared to the baseline in 65.91% of settings (29 out of 44) for each method. In which, IGNITE-2 outperforms the baseline in 50% likelihood, with an average improvement of approximately 0.85% and a peak improvement of 2.9%. For IGNITE , those are 52.27% likelihood, 1.89% average improvement and a peak improvement of 14.6%. Conversely, there are only 34.09% cases in using our proposed method leads to the performance reduction. The average degradation of IGNITE-2 and IGNITE are 0.69% and 1.19%, respectively. Overall, we state that our proposed method helps to boost the performance of almost all algorithms and tasks. This section provides the specific setup of the hyper-parameter selection experiments for IGNITE-2, which is also ran separately to determine the hyperparameters for IGNITE(in the main text).Figure 3 provides a step-by-step performance comparison between two baseline algorithms, COMs and GA, with and without our regularization method, IGNITE-2. IGNITE-2 consistently outperforms the baseline algorithms at every step in the ANT tasks. However, the trend differs for TF-BIND-10. Initially, the baseline algorithms perform better without IGNITE-2 . However, as the number of gradient ascent steps increases, the performance of the baselines without IGNITE-2 begins to decline. In contrast, the algorithms incorporating our method maintain or improve their performance over time. This trend suggests that our method becomes increasingly beneficial in the later stages of the optimization process, ultimately enhancing the overall performance of the baseline algorithms.G.4.2 Choosing the regularization coefficient λ.Figure 4 shows how the performance of baseline models regularized with IGNITE-2 is affected by different values of λ. The results indicate that using an excessively low or high value for λ will have a negative impact on performance. In all cases, the results suggest that a universal value of 0.01 for λ tends to generate consistent and effective performance across all tasks. G.4.3 Choosing value for parameter r and the perturbation radius ρ. To analyze the computational complexity of IGNITE, we break down the complexity of each step in Algorithm 1.1. Initialization (Line 1): Initializing ω (1) ← ω (0) and λ (1) ← λ: O(1) each.According to our experiment, despite using a relatively simple BDMM to solve Eq. ( 12), we have achieved significant improvement in most cases. To demonstrate the convergence of the optimization algorithm, we have plotted the training loss and the sharpness value (plotting ∥∇ ω h(ω)∥ value) during the surrogate fitting process. This is based on an experiment using GA and REINFORCE baselines on Ant and Dkitty tasks. The results are illustrated in Figure 7 . These results reveal that BDMM helps decrease both the training loss and sharpness value of the surrogate model during the training phase. This indicates that BDMM is effective and the optimization converges well in practice. Furthermore, our method, IGNITE, can be seamlessly integrated with other, more robust optimization techniques to solve Eq. ( 12).Our paper studies the offline optimization task, which has potential applications in material engineering. Similar to the existing literature, our method is extensively tested on a universal benchmark set forward by the pioneering work of [4] . However, it is important to note that the benchmark consists mostly of small-to mid-scale optimization tasks. As such, our method has not considered the challenge of scalability in large-scale domain with extremely high-dimensional input spaces. In addition, as with all machine learning algorithms, while applications of our work to real data could result in ethical considerations, this is an indirect and unpredictable side-effect of our work. Our experimental work uses publicly available datasets to evaluate the performance of our algorithms. No ethical considerations are raised.• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Our sharpness notion is different from that of[14] which is applied to the training loss. Instead, our sharpness measure is accessed on the surrogate prediction. Empirically, we also notice that surrogate sharpness often appears to be more effective than loss sharpness in boosting the offline optimization performance (Section 5.3).We omit domains marked for their high inaccuracy and noise in oracle functions from prior works (ChEMBL, Hopper, and Superconductor), as well as those deemed excessively expensive to evaluate (NAS). 


Decision: This paper proposes a technique called IGNITE to regularize surrogate models used in black-box optimization. IGNITE introduces a sharpness constraint on the surrogate model, that computes the maximum distance between the expected outputs of the surrogate model using the current weights $\omega$ and perturbed weights $\omega + \delta$ for perturbations in a neighborhood $\\| \delta \\|_2 \leq \rho$. This ensures that the worst-case prediction change across the perturbation neighborhood is small. This sharpness objective is similar to the one used in sharpness-aware minimization (Foret et al., 2021), but applied to the surrogate model output instead of the loss function.

The authors perform experiments on four tasks from DesignBench (two continuous and two discrete), applying IGNITE to 11 existing optimization techniques (including CMA-ES, REINFORCE, and others). Overall, they find that IGNITE slightly improves over the baselines in most cases. They also provide theoretical analysis showing that reducing the empirical sharpness on offline data will also reduce its sharpness over the data distribution.

This paper received positive reviews, and the authors addressed most of the common points raised by the reviewers. The reviewers thought the paper was well-written and interesting, with good experimental results and theoretical analysis. They also found the proposed method to be well-motivated and model-agnostic.

Overall, this paper brings a useful new technique for regularizing surrogate models that is relevant to the NeurIPS community.

Reviews: review: paper_summary: paper_summary: This paper focuses on offline optimization, presenting a novel approach to enhance the surrogate landscape's sharpness, thereby improving generalization. The authors introduce a gradient norm as an approximation of surrogate sharpness through a first-order Taylor expansion, resulting in a Lagrangian formulation. They employ two widely adopted methods—BDMM and a fixed multiplier as solvers for the Lagrangian.

Theoretically, the authors provide a generalization bound for the worst-case prediction change. Empirically, they validate their approach using the widely adopted offline optimization dataset, design-bench, demonstrating relatively good performance.
strength_weakness: strength_weakness: Though I agree with the ideas of using a sharpness-aware method in offline optimization, I think more details should be justified:


1. I am somewhat unsure whether the authors analyze the generalization bound on $\mathcal{R}_{X}(\omega)$ properly. 

Does the generalization over $R_{X}(\omega)$ help to improve the generalization of $L_{X}(\omega)$, which is the objective we want to generalize. Why not analyze the generalization bound w,r,t $L_{X}(\omega)$?
    
2. Is the bound tight? I understand that the original SAM paper [8] also provides a similar bound. But if it is not tight, we do not know whether it can guide the design of our algorithm.

3. Moreover, the authors should add more details to their proofs to help readers better understand. For example, in line 416, the authors should present it as a lemma rather than directly stating the results of [8] as there are some assumptions in their proofs which might not be discussed in your proof. In line 458, the authors say that they use Taylor's remainder term, but it seems that the remainder term is directly eliminated. Could the authors discuss more details?

4.  Another concern is that I am not very sure whether Assumption 2 is a strong assumption. If the assumption holds, then the Hessian is positive definite. Since the proof highly relies on it, I am afraid the generalization bound is not established in the general stationary point  (the Hessian is positive semi-definite rather than positive definite).

5.  Though the proposed method shows relatively higher performance than some corresponding baselines on some datasets, the overall improvements are marginal and sometimes even decrease, as shown in table 1. Moreover, different percentages show vastly different performance. For example, reinforce+IGNITE shows a 6.5\% gain over reinforce at the 100th percentile level, but a -4.6\% at the 80th percentile level. How can this be explained? Does it mean that the proposed method is not robust?

6. Is there any analysis on memory and time? The term includes the gradient of the gradient $\nabla_{\omega}|| \nabla_{\omega} h\left(\omega^t\right) ||$, which I am concerned is both time and memory-consuming.


In all, I think the idea is interesting, and the empirical and theoretical results are promising. But I still think more details should be added before I can vote for acceptance.
questions: questions: Please answer my question mentioned above.
limitations: limitations: Please see the weaknesses part.
paper_summary: paper_summary: The paper proposed IGNITE, a promising method for solving out-of-distribution issue in offline training by introducing model sharpness into the training loss of the surrogate as a regularizer. The key innovation of IGNITE lies on incorperating sharpness-aware minimization into offline training and developing theoretical analysis to show that reducing surrogate sharpness on the offline dataset provably reduces its generalized sharpness on unseen data. Experiments on benchmark datasets show the promising performance.
strength_weakness: strength_weakness: Although the paper has many strengths, there are still some weaknesses:
1. Sharpness-aware minimization (SAM) is widely explored in many fields. I think this submission lacks sufficient introducing of SAM, especially on its development. Take some recent works as an example:  

Enhancing sharpness-aware optimization through variance suppression.  (NeurIPS'23)  
Normalization layers are all that sharpness-aware minimization needs.  (NeurIPS'23)  
Domain-Inspired Sharpness-Aware Minimization Under Domain Shifts.  (ICLR'24)  
Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization.  (ICML'24)  

2. The gain on performance is not always stable, as shown in the Table 4 and 5.  
3. Lacking sharpness visualization of objectives on unseen data.
questions: questions: 1. I recommend the authors to extent the contends on the related works of sharpness-aware minimization.
2. Can the authors explain the potential reason on the performance drop shown in Table 4 and 5?
3. Can the authors show the sharpness of objectives on unseen data before and after using your algorithm?
limitations: limitations: I don't think there are any negative societal impacts of this work.
paper_summary: paper_summary: This paper introduces a model-agnostic regularization method that reduces surrogate model sharpness to improve generalization in offline optimization. By incorporating a surrogate sharpness measure into the training loss, they provide theoretical proof and extensive experimental validation showing that this approach enhances performance on unseen data, achieving up to a 9.6% improvement in various optimization tasks. The proposed algorithm, IGNITE, demonstrates the effectiveness of this technique across different surrogate models and tasks.
strength_weakness: strength_weakness: 1. The Introduction section does not clearly emphasize the difference between surrogate sharpness and loss sharpness. Figure 1 could be positioned earlier in the paper. Additionally, the annotations and explanations in Figure 1a are somewhat confusing; the concepts of \(\sigma\) and \(\sigma^*\) are not adequately explained in the figure and caption, which could lead to misunderstandings.
2. The Theoretical Analysis section (Section 4) should include more insightful discussions based on Theorems 1 and 2, ideally tying them to the results in the experimental section. The formatting of Equation 16 could also be improved.
3. In the experimental section, the authors mention that the IGNITE algorithm introduces five hyperparameters, but the ablation study only analyzes the effects of a subset of these hyperparameters. Many experimental results are placed in the supplementary materials, but they are not referenced in the main text.
questions: questions: Please see the weaknesses part.
limitations: limitations: Please see the weaknesses part.
paper_summary: paper_summary: The paper studies the problem of offline optimization for material design problems. The paper proposes a model-agnostic method that changes the parameters of a model by constraining the sharpness of the model's predictions. Since the model generates smoother predictions, the error between the predictions and the oracle tends to be smaller and therefore the performance of the offline model is potentially more accurate. The paper gives a theoretical analysis and shows that the empirical sharpness can bound the theoretical sharpness. The paper applies the method to various offline optimization method and shows improvements in most of the cases.
strength_weakness: strength_weakness: 1. After certain approximations, the objective is optimized using BDMM. It would be interesting to analyze the specific structures of the objective and see if more effective optimization techniques can be applied. This is also related to the model-agnostic feature of the proposed method. While having the model-agnostic property is convenient, it may also miss the specific structures of the problems/models so that the performance is not ideal. Also, there is no empirical evidence showing whether the optimization is effective or how well the optimization converges in practice.

2. The comparisons with baseline methods are only compared for a limited number of tasks. To fully demonstrate the effectiveness, I think a thorough comparison as done in Table 1 would be necessary.

3. The paper very briefly mentions the background of material design. I think a more comprehensive introduction could be beneficial and probably the paper should add detailed descriptions of the tasks and datasets.
questions: questions: Please address the weaknesses part.
limitations: limitations: Limitations are properly addressed. No obvious negative societal impact.
paper_summary: paper_summary: This paper introduce a sharpness-aware optimization to improve out-of-distribution generalization. While inspired by SAM, the major difference is that this work considers the sharpness of predictor outputs rather than loss landscape. With the proposed notion of sharpness, practical algorithm, IGNITE, is developed through an empirical approximation (Eq. 6) and Taylor approximation of the sharpness constraint.
strength_weakness: strength_weakness: - It is not immediately clear, on an intuition level, why this new notion of sharpness is better than loss sharpness in SAM [8].

- Table 2 only presents SAM on two algorithms (REINFORCE and GA) with no error interval reported. A more throughout benchmarking could improve the empirical evaluation.
questions: questions: - As illustrated in Figure 1, the intuition is that if the oracle is included in the perturbation neighborhood, a smoother predictor tends to have smaller error. However, Table 1 observes that IGNITE applied to some sub-optimal predictors, for example REINFORCE in Ant Morphology, it still achieves good performance. Since the suboptimal prediction performance may imply that $\hat{\omega}$ is not proximal to $\omega^\star$ hence the local approximation may break in practice. I am curious that why IGNITE should still work in this case, I appreciate if the authors could provide some high-level intuitions.
limitations: limitations: See above.


---
post-rebuttal: -> 5
paper_summary: paper_summary: This paper proposes a novel model-agnostic approach to enhance offline optimization methods by incorporating surrogate gradient norms.  The paper provides a thorough review of existing literature, a clear problem definition, and detailed descriptions of the proposed methods and their implementation. The experiments are well-designed, covering various benchmarks and baselines, and the results are robustly analyzed.
strength_weakness: strength_weakness: - As the paper mentioned that they draw inspiration from the sharpness-aware minimization (SAM), the paper’s novelty could be better articulated in the context of existing work on surrogate model regularization and sharpness-aware optimization. Specifically, discuss how the proposed surrogate sharpness measure offers advantages over sharpness-aware minimization (SAM).
- Assumption 2 (positive minimum eigenvalue of the parameter Hessian) might be too restrictive. It would be more helpful if the authors can provide more intuitive explanations or empirical evidence to justify these assumptions and discuss the implications if these assumptions are violated and how the method’s performance might be affected.
- The proposed method involves computationally expensive operations, such as gradient norm calculations and constrained optimization. I'm wondering how expensive the proposed method is compared to existing base approaches in terms of the training time.
questions: questions: See above.
limitations: limitations: Yes. In Appendix H
paper_summary: paper_summary: The article presents a method to improve offline optimization techniques by integrating the concept of model sharpness into the training.  A constraint is introduced that limits the model sharpness to not exceed a user-specified threshold.
strength_weakness: strength_weakness: 1. The effectiveness of the method heavily depends on the correct setting of hyperparameters like the sharpness threshold, which can be tricky to optimize without extensive experimentation.
2. The computation complexity is increased. Can the author give the complexity analysis?
3. How to ensure the proposed method converge to a stationary point of the original problem?
questions: questions: see weakness
limitations: limitations: NA

Comments: title: Post-rebuttal summary
Thank you very much for increasing the score to 5
Thank you
Thanks for the rebuttal
Follow-up
Follow-up
Thank you
Thank you
Thank you for increasing the rating
Thank you
Thank you very much for increasing the score to 5
Response to author
Follow-up
comment: Dear AC and reviewers,

We would like to express our gratitude to the Area Chair for coordinating the review process of our paper. Following the rebuttal discussion, we are very happy that all reviewers have recognized our contribution and provided acceptance scores.

We sincerely thank **Reviewers WGWE, tYoe, qZmG, and kVLN** for maintaining their acceptance scores.

We appreciate **Reviewer qc6B** for raising the score to 7 in support of our paper's acceptance.

We are also thankful to **Reviewers xjU2 and CQYH** for increasing their scores toward acceptance.

Overall, based on the post-rebuttal feedback from **Reviewers WGWE, tYoe, kVLN, qc6B, xjU2, CQYH** and the sustained acceptance rating (7) from **Reviewer qZmG**, we believe our rebuttal has addressed all concerns raised by the reviewers. 

--

A summary of our responses to address these concerns is provided below:

1. We provided further elaboration on the advantage of surrogate sharpness over loss sharpness, emphasizing the conceptual novelty of our work (**tYoe**,**qc6B**, **xjU2**, **CQYH**).

2. We demonstrated that Assumption 2 is not overly restrictive (**qc6B**, **xjU2**) and provided detailed complexity analysis of IGNITE (**qc6B**, **kVLN**, **xjU2**).

3. We showed the reduction in surrogate sharpness on unseen data after conditioning with IGNITE (**WGWE**).

4. We illustrated the convergence and effectiveness of the BDMM algorithm by plotting the training loss and sharpness value during the surrogate fitting process (**qZmG**).

5. We provided a detailed explanation of the tightness of the generalization bound (**xjU2**).

6. We clarified how IGNITE can enhance even base optimizers with poor performance (**CQYH**).

Once again, we would like to thank the Area Chair and all reviewers for their time and effort in reviewing our paper.

Best regards,

Authors
Dear Reviewer CQYH,

We sincerely appreciate your prompt decision and the supportive comments regarding our work.

We are deeply grateful for the time and effort you invested in reviewing our research, and for the valuable feedback that has significantly enhanced our work.

Best regards,

Authors
Thank you for your responses, and the additional experiments! My questions are sufficiently addressed.
Dear Reviewer WGWE,

Thank you for the response.

We are glad our response has addressed your concerns.

Best regards,

Authors
Thanks for the rebuttal. I don't have further questions.
Dear Reviewer CQYH,

We hope this message finds you well. 

As the rebuttal discussion will end soon, we really hope to follow up with you on whether our response has addressed your questions sufficiently. Your timely feedback is very valuable for us.

Thank you very much for your feedback and consideration.

Best regards,

Authors
Dear Reviewer CQYH,

May we know if our response has addressed your questions?

Thank you very much for the interesting questions and suggestions. 

Best regards,

Authors
Dear Reviewer tYoe,

Thank you for the response. 

We are glad our response has addressed your questions.

Best regards,

Authors
Thanks for the authors' response. I have no additional questions concerning this paper.
Dear Reviewer kVLN,

Thank you very much for the prompt response.

We are glad that our response has addressed your concern.

Best regards,

Authors
Dear Reviewer qc6B,

Thank you very much for increasing the overall rating of our work.

We really appreciate your support!

Best regards,

Authors
Thank you for the rebuttal. all my concerns have been addressed by the authors. I've increased my score to support this paper to be accepted.
Dear Reviewer,

We sincerely appreciate your prompt decision and your support for our work.

We are very grateful for the time you dedicated to discussing our research and the valuable feedback that has greatly improved our work.

Best regards,

Authors
Thank you for your detailed response. Most of my concerns have been adequately addressed, so I have decided to increase my score.
Thank you for the detailed response from the author. The major concern has already been addressed.  I will keep my score.
Dear Reviewers,

Thank you again for the detailed reviews.

Please let us know if you have additional questions for us. We will be more than happy to provide further clarification and discussion.

Reviewers CQYH and xjU2:

May we know if our responses have addressed your concerns sufficiently?

We hope our responses have improved your overall assessment of our work.

Best regards,

Authors