Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes
Authors: Syrine Belakaria
Benjamin Letham
Jana Doppa
Barbara E Engelhardt
Stefano Ermon
Eytan Bakshy
Venue: NeurIPS 2024 poster (2024)
Keywords: Global Sensitivity Analysis
Gaussian Processes
Bayesian Active Learning
Bayesian optimization
Primary area: probabilistic_methods
TL;DR: We propose several uncertainty based and information gain acquisition functions for derivative based global sensitivity analysis

Abstract: We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Our aim is to efficiently learn the importance of different input variables, e.g., in vehicle safety experimentation, we study the impact of the thickness of various components on safety objectives. Since function evaluations are expensive, we use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-based global sensitivity measures (DGSMs) under Gaussian process surrogate models.
We showcase the first application of active learning directly to DGSMs, and develop tractable uncertainty reduction and information gain acquisition functions for these measures. Through comprehensive evaluation on synthetic and real-world problems, our study demonstrates how these active learning acquisition strategies substantially enhance the sample efficiency of DGSM estimation, particularly with limited evaluation budgets. Our work paves the way for more efficient and accurate sensitivity analysis in various scientific and engineering applications.

Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes
Syrine Belakaria Stanford University
Benjamin Letham Meta
Janardhan Rao Doppa Washington State University
Barbara Engelhardt Stanford University
Stefano Ermon Stanford University
Eytan Bakshy Meta
Abstract
We consider the problem of active learning for global sensitivity analysis of expen- sive black-box functions. Our aim is to efficiently learn the importance of different input variables, e.g., in vehicle safety experimentation, we study the impact of the thickness of various components on safety objectives. Since function evaluations are expensive, use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-basse global sensitivity measures (DGSMs) under Gaussian process surrogate models. We showcase the first ap- plication of active learning directly DGSMs, and develop tractable uncertainty reduction and information gain acquisition functions for these measures. Through comprehensive evaluation on synthetic and real-world problems, our study demon- strates how these active learning acquisition strategies substantially enhance the sample efficiency of DGSM estimation, particularly with limited evaluation bud- gets. Our work paves the way for more efficient and accurate sensitivity analysis in various scientific and enginecring applications.
1 Introduction
Sensitivity analysis is the study of how variation and changes the output of a function can be attributed to distinct sources of variability in the function inputs [[3]. More precisely, we seek to determine how changes each input variable impact the output. Sensitivity analysis can be used for several purposes, including identifying the input variables that are most influential for the function output and those that are least influential [T3]. and quantifying variable importance in order to ex- plore and interpret model's behavior [+O]. Sensitivity analysis is important tool in many fields of science and engineering to understand complex, often black-box box systems. It has proven particularly important for environmental modeling [L]. [M]] geosciences [22 A]. chemical engineering [35], bi- ology [[I5], engineering safety experimentation [Mg]] and other simulation-heay domains. these settings, function evaluations often involve cine-ccnsuming simulations costly lab experiments, so it is important to perform the sensitivity analysis with as few function evaluations as possible. For example, sensitivity analysis in environmental modeling can help understand how parameters such as CO2 emissions, temperature increases, and deforestation rates influence the output of climate models. Identifying the variables that most directly impact model outcomes allows researchers to better prioritize efforts in data collection, model refinement, and policy development [3I]]
Sensitivity analysis can be either local global. Local sensitivity analysis (LSA) [[X]] studies the ef-
fect of perturbations of single input variables at fixed, nominal point. Input sensitivity is measured only locally to that nominal point and does not take into account any interactions. Global sensitivity analysis (GSA) [33], on the other hand, evaluates sensitivity over an entire compact input space
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Postorior
Aequisition: Information gain f(a)
Posterior of of de
Acquisition: information of (of(
100
M
1.00
Posterior of [d[/dx]
Acquisition: information gain of 14f (s)/da)
1.00
Posterior /ds)
Aoquisition: information gain of (4) (d))
10000
5.0
2.5
Figure 1: (Left) Posteriors of f. df/ffdd /x. |df /dr), and (df dx are computed from a GP surrogate given six observations of f (black dots). Posteriors shown as posterior mean (line) and 95% cred- ible interval (shaded). (Right) Acquisition functions are computed from these posteriors, targeting f and derivative sensitivity measures. Dotted vertical lines show the maximizer. Acquisition functions that directly target DGSMs, not just generally, are required to learn the DGSMs efficiently.
and includes measures of interactions between the various input dimensions. Here, we focus on GSA. Approaches for GSA can be categorized into two main types: variance-based measures, also referred to as ANOVA decomposition or Sobol methods [28], and derivative-based global sensitivity measures (DGSMs) [ل Variance-basee measures quantify the importance of different input vari- ables based on their contribution to the global variability the function output. In contrast, DGSMs quantify importance based the global variability of the function's gradient. They defined as an integral over the gradients or a function of the gradients across the input space.
DGSMs can often be computed directly from the function. However, for expensive black-bor box func- tions, integrating function the gradients over the input space is infeasible due to the limited number of function evaluations and lack of gradient information. In this case, the function is modeled by a surrogate Gaussian process (GP) [Bi], which allows for tractable computation both the function surrogate and its gradient. Previous work IC TT. [X] used random and quasirandom sequences to select the data points for learning the GP; however, these space-filling approaches still require substantial number of evaluations to accurately estimate the DGSMs.
We show here that DGSMs can be targeted for active learning with information based acquisition functions that are tractable under GP surrogate models, which are standard for GSA. Applying active learning to DGSM quantities (e.g., gradient, absolute value of the gradient, squared value of the gradient) allows for sensitivity analysis to be performed in highly sample-efficient manner, suitable for engineering and science applications with small evaluation budgets. To the best of our knowledge, this is the first study proposing active learning acquisition functions directly targeting the DGSM measures.
We illustrate the utility of the proposed acquisition functions for active learning using the classic Forrester [ло] test function (Fig. The posteriors of f and df / /dr given observations of are computed from the GP (Sections 4 and 2.5).. Posteriors of Idf /dz| and (df /dx)2 derive from that of df dr (Section 4.4 the absolute and squared DGSMs are the integrals of these functions (Section 2.3). Acquisition functions give the value of evaluating any particular point under different targets: one quantifies the information gain the function generally (Section R), and the others quantify the information gain of the derivative quantities (Section [.4). The acquisition functions illustrate why active learning strategies that only target learning f are effective for learning DGSMs. To learn f. active learning selects points that far from existing observations, where f is most uncertain.

---

To learn the adjacent existing observations, as that adjacency is valuable for derivative estimation. The contributions of this paper are:
raw, absolute, qquare-active learning selects points that are
We introduce the first active learning methods to directly target the quantities used for derivative- based global sensitivity analysis, namely, the gradient, its absolute value, and its square. Our acquisition functions are based on uncertainty reduction and information gain, and we provide tractable approximations of information gain for DGSMs using GP models.
With a thorough evaluation on synthetic and real-world problems, we show that active learning substantially accelerates GSA in settings with limited evaluation budgets. The implementation for our methods, the baselines, and the synthetic and real-world problems is available in our code (https://github... wwt.:/owo.beeaaaaee httpp///ggthhbbb.ccm//aaaaar/// /LL AS-GSA-DSMM
2
Background
In this section, define the problem and provide review of DGSMs, GPs, and their derivatives. For a thorough review of GSA, see looss and Saltelli [[3].
2.1 Problem Setup
We wish to analyze the sensitivity of a black-box function defined over a compact d-dimensional input space X. We suppose that evaluating f at any particular input E X, that is, observing = f(x + with the observation noise, is expensive. The ground-truth sensitivity measure, which we denote
5(f,X), has d-dimensional output that provides sensitivity measure for each input dimension IG. i I,...,d. d. We estimate S by making / observations of f. for which we denote with X [x [[x xy] the set of observed inputs, Y ../....,, ........... v/l the function evaluations at those inputs, and D = {X, }} the full observed data. We leam surrogate model of f from D and then evaluate S on surrogate for f. Here, will denote the surrogate as f; in practice, will use the GP posterior mean fxx μ(x),, which we introduce in Section 2. Given the surrogate, we estimate $(f, S(f,
Our active learning problem is select the imput locations so that $ provides the best estimate of the true global measure S. We do so sequentially with a budget of T'' total evaluations. Generally, will better approximate S as the surrogate f better approximates f. However, when T is small, is important to consider the particular form of S design the most effective strategy, as opposed to simply trying to learn a good global surrogate. Here, we develop strategies tailored for S being DGSM. We introduce DGSMs in Section 2.3, and in Section g develop the acquisition strategies.
2.2 Bayesian Active Learning
The methods we develop here are cast as Bayesian active learning algorithms. Bayesian active learning is a flexible framework that combines Bayesian inference principles with active learning strategies, where data points are sequentially selected in a sample-efficient manner [[22]. The algo- rithm iterates through three steps. First, iteration evaluated data. Second, we apply an acquisition function to the GP posterior to score the utility of unevaluated inputs, and select the input with the highest acquisition value. Third, we evaluate the black-box box function on the selected input and augment D with the new observation. We summarize the process in Algorithm M In Section we develop acquisition functions, (<-), that are tailored to learning DGSMs.
build the GP surrogate model from the
2.3 Derivative- Based Global Sensitivity Measures
In this section, we provide the background and definitions for our target sensitivity measures, DGSMs. DGSMs are defined as the integral over the input space of function of the derivative of the black-box function. There are three widely-used gradient functions in DGSMs: the raw gra-
3

---

Algorithm Baaesian Active Learning
Input: X, f(x), surrogate model GP, utility function a(x, 9P), total budget T.
Output: Dr,gP.
1: Initialize data Do with To starting observations.
2: for each iteration E [To, T do
3: Fit the surrogate model (P(,) using D1-1.
4: Select the next input for evaluation by maximizing the acquisition function,
x" <- arg m axx, (P(D)-))
5: Evaluate the black-box function to observe y = f(x").
6: Update data D, Di-1 U {(x"(""' y²)))
7: end for
dient, the absolute value of the gradient, and the square of the gradient [18]]
Sn(Jxx); = [2| /
@f(x)
/x
0f(x)
dx, SA |.0 Oz;
dx.
dz;
(f((x) Ss,(fxX), = X Or,
2
dx.
In the remainder of the paper, we will refer to these quantities as the raw DGSM, absolute DGSM, and squared DGSM. These quantities may also be defined with non-uniform densities on X.
For the purpose of evaluating input sensitivity, the raw DGSM considered uninformative due to phenomenon known as the cancellation effect. In nonmonotonic functions, positive parts of the gra- dient cancel out negative parts when integrated over the entire input space, leading small value for the raw DGSM even for important dimensions. The most commonly used DGSMs in practice are the absolute and squared DGSMs, which avoid the cancellation effect. The squared DGSM is es- pecially popular because of its connection to the variance of the gradient [[77]] Computing DGSMs requires computing d-dimensional integral over X. This integration is usually done via Monte Carlo (MC) or quasi-Monte Carlo (QMC) sampling [[]]
2.4
GP Surrogates for Sensitivity Analysis
When function evaluations are expensive, the integrals over the input space required compute the DGSMs may not be evaluated tractably from f. Moreover, if f is black-box, we may not have access to its gradients. Both of these issues can avoided by using a surrogate function for f.
GSA of expensive, black-box functions is usually done using a GP surrogate model [[]] GPs are characterized by mean function m X R and kernel function covariance K. X' X + R. A GP prior for the function, f Gm(m,, (m, means that the function values at any finite set of inputs are jointly normally distributed. For any input x, E X, the function value that input has normally-distributt posterior f(x.) )) N (M,,, o2), whose predictive mean and variance are:
11a Kx., K mx) mx.; o² = = K K
where Kx..x. = K((x,, x.), Xx..x [K[x.,, x mx m(x), and KD = Kx.x + 1, with Xx.x = XCXXXX X) and n² the observation noise variance of [[O]. We introduce the short-hand notation μ4, == (xx.) and o² == oxx.)² for the posterior mean and variance functions.
GPs are differentiable when using any twice- ditredfiffeentibble kernel function K. and differentiable mean function m. The gradient of the GP provides tractable estimate of the gradient of the expen- sive black-box function / under commonly-user kernels such as the ARD RBF. DGSMs can then be computed in fast and scalable way on the posterior of f [[]]
2.5 Derivatives Gaussian Processes
GPs are closed under linear operations, therefore the derivative of a GP is itself GP [30]. Since defined over d-dimensional input space, the model's gradient has a d-dimensional output. Under GP prior, the joint distribution between (potentially noisy) observations of f and the gradient of

---

at new point x, is as follows [BI]:
vf(x.) N ([VIx
mx
KD Kx.,x
Vx, Exxx. V Nx.,xx.
V
Given the observed data D as before, the gradient at x, has a multivariate normal distribution: Vf(x.)) N(p,, Σ,), where
14 Vix. + Vx, X mx):
Σ, = V2 x Kxx,x. - Vx, Kx V Xxxx.
Here, Jl, = ,x(xx. and Σ = (x.) are shorthand for the posterior mean and covariance functions of the gradient. Note that the posterior for the derivative may be obtained from observations only of f, and does not require direct observations of the derivative. The greatest computational expense in computing the GP posterior is the matrix inversion in Kp', which has complexity ((t"). This same term is also the most expensive term in the posterior for the derivative. Consequently, once the posterior of the GP has been computed, the computation of the derivative does not increase the overall complexity. Given the posterior of the derivative, the DGSMs are estimated by substituting the gradient of the function with the predictive mean of the gradient from (II)
3 Related Work
The GP surrogate allows for computing DGSMs with a limited set of function evaluations. However, in budget-restricted experiments, the GP will only provide faithful representation of the sensitivity of f if the right set of inputs are evaluated. There has been limited work on efficiently selecting the inputs that lead accurate DGSM estimation, particularly with limited evaluation budget.
The most common approach for estimating DGSMs with a GP is to evaluate the function on either random set of inputs or with space- ppacc-filling design. For the latter, quasirandom sequences like scrambled Sobol sequences [[6]] and Latin hypercube sampling [24] are two common choices. Space-filling designs are effective for GSA with sufficiently large evaluation budget, however, as we will see below, they fail when the budget is limited.
Random and space-filling designs:
General uncertainty reduction methods: Several Bayesian active learning approaches have been developed for the purpose of reducing global uncertainty about f. Information- based strategies that select the point that produces the largest information gain about a function's 's outputs are popular and effective for global identification of [3G 16. 12]. Other global active learning approaches are based on variance reduction [34]] and expected improvement (EI) [I9]] These general-purpont active learning strategies have been applied the GSA problem. Pfingsten [27]] used global predictive
variance reduction as the active learning target for the purpose of GSA; Chauhan et al. [[]] applied the El criterion to GSA. These approaches are designed to generally minimize uncertainty of f. and do not specifically target improvement of any particular GSA measure. Acquisition functions that target f are not sufficient to learn the DGSMs efficiently on a budget (Figure M..
Active learning involving derivatives: Some work has incorporated derivatives into active learning for problems unrelated to GSA. Salem et al. [62]] and Spagnol et al. [Р7] use sensitivity measures to eliminate variables during Bayesian optimization. Erickson et al. [ and Marmin et al. [[X]] include a
derivative term in an acquisition function for learning non-stationary functions. Wycoff et al. [ do
active subspace identification with an acquisition function targeting the outer product of the gradient. See the Appendix for further discussion and an empirical comparison to these methods.
Active learning for GSA: There has been limited work on applying active learning to GSA mea- sures. Existing work considers only the Sobol index (variance- based measures). Le Gratiet et al. [20] applied variance reduction directly to the Sobol index. However, this could not be done in closed form and required expensive simulations within each active learning step. More recently, Chauhan et al. [[3] developed an analytic improvement criterion that targets the numerator of the Sobol index. In this work, propose the first active learning acquisition functions directly targeting DGSMs.
5

---

4 Bayesian Active Learning for Derivative-Based Sensitivity Analysis
In this section, we develop active learning acquisition functions that target the DGSM measures. We derive acquisition functions following three general strategies. The maximum variance acquisition functions select the point with the largest posterior variance in the quantity of interest, indicating the point with the most uncertainty. The variance reduction acquisition functions measure how much an observation a point will reduce the variance that point in expectation over the possible outcomes of the observation. Finally, the information gain acquisition functions quantify the expected reduc- tion in entropy of the posterior of the quantity of interest for each point. The latter two strategies require computing the look-ahead distribution for the derivative, which we introduce in Section A.I. Finally, we discuss global look-ahead acquisition functions that measure the impact of evaluating an input on the posterior across the entire input space.
4.1 The Derivative Look-Ahes Distribution
Effective active learning often relies computing look-ahead distributions that predict the impact that making a particular observation will have on the model. For our purposes, we wish to predict the impact that observing f at a candidate point x, will have on the model's 's derivatives at that location. Conditioned on the observations DD fxx)) and 0f(x.) have a bivariate normal joint distribution for each input dimension The well-known formula for bivariate normal conditioning provides the
ds.
look-ahead distribution [22]] mean and variance are
0f(x.) f(x.) = y.. D ~ N (o"",,"""" (o",))), where the look-ahead oz:
(y+ p+), (o",)²= = (o'.)) Ōx,e²
n", = 14,s +
(3)
o?
with o.,, = Cov(f(x.), ((xx)) [D] the posterior covariance between f and the derivative at X.. 82;
and (o',,)² = xxxx))² [2' )ai the posterior variance of the derivative. As before, we use the notational short-hand o²,, = of (x.) and p." = (x.). This result holds when y, is a noisy observation by replacing o² with +²= Remarkably, the look-ahead variance is independent of the actual observed y.. so acquisition functions that are based on the future variance of the derivative can be computed exactly in closed form. In the Appendix, we provide the look-ahead posterior distribution of the derivative of f at any point in the mput space after observing f at x,.
4.2 Gradient Acquisition Functions
Maximum variance. The posterior variance of each derivative is given in (L). The maximum derivative variance acquisition function uses the sum of the variances across dimensions to find points with high total uncertainty in the derivatives: apv (x) ا (x)2
Variance reduction. The derivative variance reduction acquisition computes the expected reduc- tion in variance of the derivatives produced by making an observation of at x:
d
CrDV, (x) Σ
Ex) Σ of -x)
where (x)2 is the look-ahead variance of the derivative, from (L) The expectation is dropped because the look-ahead variance is independent of the observed y at the candidate point.
Information gain. We express derivative information gain acquisition function the sum of information gains for each derivative. Let H((x) = h [f(x))
(D) be the differential entropy
Oz:
of each derivative posterior and H h [f(x)) D. [(x v) the look-ahead entropy. The Gaussian entropy well-known [S], and since is independent of the mean, the look-ahead entropy is independent of y. The information gain, in nats, is:
Olsia (x) = Σ H(((x) - E, [H,
Σ log (x((x)) - log (x)(
1=1

---

4.3 Absolute Gradient Acquisition Functions
Maximum variance. The absolute value of normal distribution is the folded normal distribution [[8], whose mean and variance, 14 (x) and o (x)², are analytical and can be computed from the
moments the corresponding normal distribution. Using those results, the posterior of 0f(x) Oz;
has
mean and variance:
(x) V 2 o -x(()) + (x((x) (1 (x)-):
Le (x)² px() + dx(x)² - - (x)²
where + is the standard normal CDF and we have denoted ri(x) (((x)
We define the maximum
variance acquisition for the absolute value of the derivative as: ('xav Σ o (x)².
Variance reduction. The look-ahead variance for the absolute value of the derivative, denoted o (x)?, can be computed by plugging the look-ahead moments from (E) into the formula for the folded normal variance. However, unlike for the raw derivative, this variance depends 5 (x) and thus a function of y, making the expectation in the variance reduction formula intractable. We follow the strategy of Lyu et al. [22]] and approximate Eylo sss (x)²) with plug-in estimator, fixing xxx). Plugging this estimator into (E)) gives estimate for the look-ahead derivative mean that is independent of denoted 0i (x)2, and it follows that
OLpASV, (x) = Σo oo (x)² Eylo (x) R Σ lo (x)² - (x)² Do'no
1=1
Information gain. The differential entropy of the folded normal distribution is not available in closed form. Tsagris et al. [[8] provide approximation using truncated Taylor series. However, we show in the Appendix that it is numerically poorly behaved this application. We introduce a novel approximation for the folded normal differential entropy, in nats:
Hx) = H'(x) (K/An (x)² (4)*)) T log(2)
(x)
The derivation of this approximation and an evaluation of accuracy alongside other approximations are in the Appendix. The look- ahead entropy H. is estimated using the same plug-in in strategy as for variance reduction, remove dependence on y and get
O'DAM (x) = Σ Hx*(x) - H.bbe(s)
1=1
4.4 Squared Gradient Acquisition Functions
dz, Maximum variance. Let Q (ff(x)
and Z
Q The posterior of Z has a non-
of(x)2.
This allows computing the posterior variance of the squared derivative using the known moments of the noncentral x² distribution: of,, (x)² (x) (x))2 + (x) As before, we construct the maximum variance acquisition function as apsey (x) = Ed= o, (x)
central x² distribution 7 xf2
[((x) [((
Variance reduction. As with the absolute value, the look-ahead variance for the square of the derivative depends on the observed value y via the term 14 (x), making the expectation in variance reduction intractable. We again approximate the variance reduction with a plug-in estimator in (8), substituting 14: (x) for n (x) estimate look-ahead variance ate independent of y. The variance reduction then be computed as:
Orosqy, (x) = Σ %f (x)² Eo (x)²]= R Σ
vig (x)2 b,g (x)²
1=1
1=1
7

---

Information gain. Using properties of entropy [5], the differential entropy of the squared deriva- tive follows from that of the noncentral x² distribution as:
(x) - (Q - (2 + (x)(
(5)
In the Appendix we develop two approximations for the entropy of the noncentral x². As shown there, the most effective approach derives from an earlier approximation of the quantile function for the noncentral x² by Abdel-Aty [M]]. Our novel entropy approximation relies recent analytical expressions for the expected logarithm by Moser [25]], as detailed in the Appendix. We plug the approximated noncentral x2 entropy into (S)) to obtain an entropy for the squared normal, and then the information gain acquisition, Onsqia (x), follows analogously to aAANN
d
aDsqia = Σ HH - H.
(6)
4.5 Global Variance Reduction and Information Gain
The acquisition functions described so far all evaluate the impact of an observation at only on the posterior x,, We also considered global look-ahead acquisitions, which evaluate the impact of an observation at x, on the posterior across the entire input space. These acquisition functions are expressed as an integral of the acquisition functions already described. We provide their full expres- sions, their evaluation, and discussion about their complexity and performance in the Appendix.
Experiments
5.1 Experimental Setup
We compared the proposed active learning acquisition functions for DGSMs space-filling and general uncertainty reduction approaches. We refer to quasirandom sequences as QR, variance maxi- mization of f as fVAR, and information gain about f [i.e.. BALD, D as fIG. Since our experiments use noiseless observations of f. variance reduction of f. fV,, is equal to fVAR. For the acquisition functions developed here, we use the following acronyms (Section [. max variance of the raw, abso- lute, and squared derivatives are DV, DABV, and DSqV; corresponding variance reduction acquisitions are DV,, DAbV, and DSqV,; and information gain acquisitions are DIG, DABIG, and DSqIG. For the ab- solute and squared derivative information gains, we evaluated two different entropy approximations, labeled as e.g. DSqIG, and DSqIG2, with the corresponding descriptions the Appendix.
We used synthetic and real-world problems for evaluation. For synthetic experiments we used fam- ily of functions designed specifically for evaluating sensitivity analysis measures [[S [T]: Ishigamil
(d = 3), Ishigami2 (d 3), Gsoboló (d = 6), a-function (d = 6). Gsobol10 (d 10). Gsobol15 (d 15) and Morris (d 20). Ground-truth DGSMs are available for these problems. We ad- ditionally used other general-purge purpose synthetic functions where sensitivity might be challenging to estimate [8]: Branin (d 2). Hartmann3 (d 3) and Hartmann4 (d 4). For these functions, we numerically estimated ground-truth DGSMs. The results for Hartmann3, Gsobol6, and a- a-function are given in the Appendix. We considered three real-world design problems. The Car Side Impact Weight problem simulates the impact of d = 7 design variables on the weight of a car, to study the impact of weight on accident scenarios. Design variables are the thickness of pillars, the floor, cross members, etc. We also used the Vehicle Safety problem, which has two functions: the weight and the acceleration of the vehicle. Both are functions of d = 5 design variables describing the thickness of frontal reinforcement materials. We study the two functions independent problems. Ground truth DGSMs for these problems were estimated numerically.
We study settings with limited evaluation budgets. Quasirandom sequences are known to perform well given enough data [R. B]] Here, we focus on the restrictive case where we initialize our exper- iments using five random inputs and run 30 iterations of active learning. Our results are averaged over replicates from different initial points, and we report the mean and two standard errors over replicates. Our primary evaluation metric is root mean squared error (RMSE) of the DGSM estimate versus ground truth. All acquisition functions were implemented in BoTorch [2] and were designed to be auto-differentiable and efficiently optimized with gradient optimization.

---

12328122 Rand
Costalod Dmn-10
Sourr CCN 2912
SquaR DOCK PAIzT
Square COCH FIACE
OCEN
Actis Losming migg teaters
Adtes leaming henatios
Activ Learning Enration
Activ Leartig tervices
ACW LeS 18:99995
Merbo Drewzo
Squrm DOCH RACCE
Squac SCORR pAnCO
DOCY
QR
NAR
NIG
DV
DV,
DIG
Figure 2: RMSE (mean over 50 runs. two standard errors shaded) for learning the DGSM, for 10 test problems. Results are shown for active learning methods targeting the raw derivative. Active learning targeting the derivative consistently outperformed space-filling designs and active learning targeting f. Derivative information gain was generally the best-performing acquisition function.
Dini2
Inigen 3, Dine
Hartmanni Dlni
200
Apn CC1 RAACE
Alc OCCAA SUICE
Adove
Adrve
(eticn Sabey Dirre
Vebitt safee Arceeraiin Stru
Cor 1ate Irggett mest Wite
Ali COCAA DAACZ
COOPH
Activ Leurring Banatons
Jcha Leaming Berstons
Adhve leurnng tarelarn
Learning berations
ACtr Learning Banations
QR
fIG
DABV
DABV,
DAbIG;
DAbiGa
Figure 3: RMSE results as Fig. here for the absolute derivative acquisition functions. These also outperformed the baselines, with absolute derivative information gain generally the best.
5.2 Results and Discussion
Experimental results for the new acquisition functions are separated by their targets for clarity: Fig. 2 for the derivative, Fig. El for the absolute derivative, and Fig. + for the squared derivative. Across this wide set of problems, the active learning targeting DGSM quantities developed here con- sistently outperformed quasirandom sequences (QR) and active learning methods that target learning about f (fVAR and fiG). The DGSM information gain acquisition functions (DIG, DABIG1, DSqIG,) performed best in the majority of experiments.
There were rare instances which RMSE increased with active learning iteration. The ex- plore/cxploit trade-off is fundamental to active learning. During exploration, adding data point in one part of the space may cause global adjustment in the model predictions that can cause errors in another area. With more exploration and data, the model self-corrects and RMSE will decrease.
In the nigh-dimensional problems (GSobol15 and Morris), we see substantial degradation of perfor- mance for the baseline methods (QR, fVAR, and fIG), with little reduction of RMSE across iterations. Active learning targeting the DGSM quantities, on other hand, continued to perform well high dimensions. On these problems the derivative max variance acquisition functions (DV, DABV, and DSqV) were competitive with or outperformed the derivative information gain acquisition functions. We hypothesize this is due to the myopic nature of the one-step look- look-ahead used for information gain.

---

12538192 Rand
Poreetiiy Ba-in
St DOCH paazz
Equari DICN FIACE
Actist Loaning mig teaters
Aihve leeriigg Reaslans
Adtve Learning Enrition
kON Leartigg tersices
ACNE Loaring Ewiters
Sustoii,, Unrwlp
Marb, Drewdo
Saldy Acceerstin Stmed
SQua DOCY PAICE
looning tinoggioooosss
Leaning
Learning
QR
NAR
fIG
DSqV
DSqV,
DSqiG,
DSqiG2
Figure 4: RMSE results as in Fig. A here for the squared derivative acquisition functions. As with the other derivative active learning approaches. these outperformed the baselines, and squared derivative information gain generally performed best.
Information gain maximized near existing points (Fig. M. and so in high dimensions, one-step look-ahead is not sufficiently exploratory to capture the whole function. Max variance is naturally more exploratory and thus can perform better in high-dimensional setrings. However, the derivative information gain acquisitions still outperformed the baselines these problems. In practice, deriva- tive information gain measures are likely to be the best choice, possibly ensembled with a derivative max variance acquisition in high dimensional problems [[9]]
5.3 Additional Results
In the Appendix, we also evaluate the ability of active learning identify the correct ordering of variable importance, by using normalized discounted cumulative gain (NDCG) [[]]] the evaluation metric. The results were generally consistent with what seen with RMSE. We further evaluated global versions of these same acquisition functions, and found that they did not substantially improve over the results shown here, while creating large computational burden. Finally, we evaluated a heuristic baseline that incorporates the insight of Fig. DI by using space-filling design plus small perturbations those same points. This did not significantly improve performance over QR.
Conclusions
In this work, we developed a collection active learning methods that directly target DGSMs. These strategies substantially enhanced the sample efficiency of DGSM estimation when compared to quasirandom search and even compared to active learning strategies targeting f. Information gain about the derivative (raw, absolute, squared) was generally the best approach.
Our work paves way for additional work on active learning for DGSMs in several directions. Although both variance reduction and information gain approaches perform well in high dimensions, information gain approaches might benefit from increased exploration by using a two-step or batch look-ahead to select pairs of points in acquisition function optimization. Our acquisition functions also all take the form for the gradient is another possible avenue. Active learning for DGSMs could also be developed for non non-Gaussen an-Gaussian models. Finally, DGSMs have been linked via several lower/upper bound inequalities to ANOVA- -oov--baeedda sensitivity indices [[[7]. Understanding the impact of active learning for DGSMs on ANOVA-based sensitivity indices another useful direction for future work.
sum over dimensions. Computing the entropy of multivariate posteriors
Acknowledgments. Belakaria is supported by Stanford Data Science Postdoctoral Fellowship. Doppa is supported part by USDA-NIFA funded AgAID Institute award #2021-67021- and NSF CAREER award IIS- 1SS-922..2.
10

---

References
[1] S. H. Abdel-Aty. Approximate formulae for the percentage points and the probability integral of the non-central x" distribution. Biometrika, 41:538-544 1954.
[2] M. Balandat, B. Karrer, D. R. Jiang. S. Daulton, B. Letham, A. G. Wilson, and E. Bakshy. BoTorch: a framework for efficient Monte-Carlo Bayesian optimization. Advances in Neural Information Processing Systems 33, NeurIPS, 2020.
[3] M. S. Chauhan, M. Ojeda-Tuz, R. A. Catarelli, K. R. Gurley, D. Tsapetis, and M. D. Shields. On active learning for Gaussian process-based global sensitivity analysis. Reliability Engineer- ing & System Safery, page 109945, 2024.
[4] V. Ciriello, I. Lauriola, and D. M. Tartakovsky. Distribution-based global sensitivity analysis in hydrology. Water Resources Research, 55(11))8708-8729 2019.
[5] T. M. Cover. Elements of Information Theory. John Wiley & Sons, 1999.
[6] M. De Lozzo and A. Marrel. Estimation of the derivative-based global sensitivity measures using a Gaussian process metamodel. STAM/ASA Journal on Uncertainty Quantification, 4(1): 708-738, 2016.
[7] J. Dick, F. Y. Kuo, and I. H. Sloan. High-dimensional integration: the quasi-Monte Carlo way. Acta Numerica, 123-2888 2013.
[8] L. C. W. Dixon. The global optimization problem: an introduction. Towards Global Optimia-
tion 2. pages 1-15, 1978.
[9] C. B. Erickson, B. E. Ankenman, M. Plumlee, and S. M. Sanchez. Gradient based criteria for sequential design. In 2018 Winter Simulation Conference (WSC), pages 467-478. IEEE, 2018.
[10] A. I. J. Forrester, A. Sóbester, and A. J. Keane. Engineering Design via Surrogate Modelling: a Practical Guide, Wiley, 2008
[11] Gustafson, Srinivasan, and L. Wasserman. Local sensitivity analysis. Bayesian Statistics, 5:197-210, 1996.
[12] N. Houlsby, F. Huszár, Z. Ghahramani, and M. Lengyel. Bayesian active learning for classifi- cation and preference learning. arXiv preprint
2011.
[13] B. Iooss and A. Saltelli. Introduction to sensitivity analysis. In Handbook of Uncertainty Quantification, pages 1103-1122. Springer, 2017.
[14] K. Järvelin and Kekäläinen. Cumulated gain- evaluation of IR techniques. ACM Trans- actions on Information Systems, 20(4))422-446, 2002.
[15] A. Kiparissides, S. S. Kucherenko, A. Mantalaris, and E. N. Pistikopoulos. Global sensitiv- ity analysis challenges in biological systems modeling. Industrial & Engineering Chemistry Research, (8811):716) 11111188778111 2009.
[16] A. Krause, A. Singh, and C. Guestrin. Near-optimal sensor placements in Gaussian processes: Theory, efficient algorithms and empirical studies. Journal of Machine Learning Research, (2):235-284, 2008
[17] S. Kucherenko and B. looss. Derivative based global sensitivity measures. arXiv preprint arXiv:1412.2619, 2014.
[18] S. Kucherenko, M. Rodrriueez-Ferandez.. C. Pantelides, and N. Shah. Monte Carlo evaluation of derivative-based global sensitivity measures. Reliability Engineering & System Safety, 94
(7):1135-1148, 2009.
[19] C. Q. Lam and W. I. Notz. Sequential adaptive designs in computer experiments for response surface model fit. Statistics and Applications, 6(1 & 2):207-233, 2008
[20] L. Le Gratiet, M. Couplet, B. looss, and L. Pronzato. Planification dexpériences séquentielle pour lanalyse de sensibilité. 46èmes Journées de Statistique, Rennes (France), 60, 2014.
[21] L. Le Gratiet, S. Marelli, and B. Sudret. Metamodeel-bsse sensitivity analysis: polynomial chaos expansions and Gaussian processes. In Handbook of Uncertainty Quantification, pages
1 1289-1325. Springer, 2017.
[22] X. Lyu, M. Binois, and M. Ludkovski. Evaluating Gaussian process metamodels and sequential designs for noisy level set estimation. Statistics and Computing, 31(4):43, 2021.
11

---

[23] S. Marmin, D. Ginsbourger, J. Baccou, and J. Liandrat. Warped Gaussian processes and derivative-based sequential designs for functions with heterogeneous variations. SIAM/ASA Journal on Uncertainty Quantification, 6(3):991-1018, 2018
[24] M. D. McKay, R. J. Beckman, and W. J. Conover. A comparison of three methods for selecting values of output variables in the analysis of output from a computer code. Technomelrics, 21
(2):239-245, 1979.
[25] S. M. Moser. Expected logarithm and negative integer moments a noncentral x² -dditrributee
random variable. Entropy, 22(9))1048, 2020.
[26] A. B. Owen. Scrambling Sobol' and Niederreiter-Xing points. Journal of Complexity, 14: 466-489, 1998.
[27] T. Pfingsten. Bayesian active learning for sensitivity analysis. In European Conference on Machine Learning, ECML, pages 353-364, 2006.
[28] C. Prieur and S. Tarantola. Variance-based sensitivity analysis: Theory and estimation algo- rithms. Handbook of uncertainty quantification, pages 1217-1239, 2017.
[29] G. Qian, M. Massenzio, D. Brizard, and M. Ichchou. Sensitivity analysis of complex en- gineering systems: Approaches study and their application to vehicle restraint system crash simulation. Reliability Engineering & System Safety, 187:1 10-118, 2019.
[30] C. E. Rasmussen and C. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
[31] S. Razavi and H. V. Gupta. What do we mean by sensitivity analysis? The need for comprehen- sive characterization of "global" sensitivity in earth and environmental systems models. Water Resources Research, (165)3307-3002, 2015.
[32] M. B. Salem, F. Bachoc, O. Roustant, F. Gamboa, and L. Tomaso. Gaussian dimension reduction for goal-oriented sequential design. STAM/ASA Journal on uncertainty quantification, 7(4):1369-139) 2019.
[33] A. Saltelli, M. Ratto, T. Andres, F. Campolongo, J. Cariboni, D. Gatelli, M. Saisana, and Tarantola. Global Sensitivity Analysis: the Primer. John Wiley & Sons, 2008
[34] A. I. Schein and L. H. Ungar. Active learning for logistic regression: evaluation. Machine Learning, 68:235-225 265, 2007.
[35] F. D. Sepúlveda, L. A. Cisternas, and E. D. Gálvez. The use of global sensitivity analysis for improving processes: applications to mineral processing. Computers & Chemical Engineering, 66:221-232, 2014.
[36] M. C. Shewry and H. P. Wynn. Maximum entropy sampling. Journal of Applied Statistics, 14 (2):165-170) 1705, 1987.
[37] A. Spagnol, R. L. Riche, and S. D. Veiga. Global sensitivity analysis for optimization with variable selection. SIAM/ASA Journal on uncertainty quantification, 7(2):417-443 2019.
[38] M. Tsagris, C. Bencki, and H. Hassani. On the folded normal distribution. Mathematics, 2(1): 12-28. 2014.
[39] R. Turner, D. Eriksson, M. McCourt, Kiili, E. Laaksonen, Z. Xu, and I. Guyon. Bayesian op- timization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. In NewrlPS 2020 Competition and Demonstra- tion Track, pages 3-26. PMLR, 2021.
[40] B. Van Stein, E. Raponi, Z. Sadeghi, N. Bouman, R. C. Van Ham, and T. Bäck. A compari- son of global sensitivity analysis methods for explainable Al with an application in genomic prediction. IEEE Access, 22110364--033388 2022.
[41] T. Wagener and F. Pianosi. What has global sensitivity analysis ever done for us? A sys- tematic review support scientific advancement and to inform molicy-making in earth system modelling. Earth-Science Reviews, 194:1-18, 2019.
[42] H. M. Wainwright, Finsterle, Y. Jung, Q. Zhou, and J. T. Birkholzer. Making sense of global sensitivity analyses. Computers & Geosciences, 65:84-94, 2014.
[43] N. Wycoff, M. Binois, and S. M. Wild. Sequential learning of active subspaces. Journal of Computational and Graphical Statisties, 30(4): 2021.
12

---

Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes (Supplementary Material)
A Additional Discussion of Related Work
Here we give further discussion of lines of work that incorporate derivatives into active learning, albeit not for the purpose of GSA.
Goal-Oriented Sequential Design. Salem et al. [32] proposed an optimization approach where at each step the dimensionality is reduced by identifying unimportant features. Unimportant variables are fixed while simultaneously optimizing the important ones with expected improvement. These variables identified through their lengthscales. As part of theoretical analysis, the paper proves an asymptotic relationship between lengthscale and the square DGSM: as lengthscale goes to infinity, DGSM for that parameter goes zero. This work does not propose an active learning approach for DGSMs but rather uses DGSMs as a metric to asses the quality of the dimensionality reduction ap- proach. Spagnol al. [[7]] used sensitivity analysis to eliminate variables during optimization. The sensitivity analysis is goal-oriented rather than global, by applying the Hilbert-Schmid indepen- dence criterion to portions of the function below a particular output quantile. Sensitivity measures are part of the algorithm and are assumed to be accurate, the paper does not study learning them.
Active learning in the presence of non-stationarity. Erickson et al. [ developed method for active learning non-stationary function that specifically targets areas of high gradient, although without trying to estimate the gradient globally. Variance reduction of the function serves as the acquisition, but weighted by the estimated derivative and its variance. The method thus focuses on minimizing the variance at points with large derivatives, but does not target learning the derivative itself. Marmin et al. [|3]] proposed an active learning approach for non-stationary functions using warping. The acquisition strategy uses both variance reduction and derivatives, following a similar idea as Erickson et al. [*] that areas of high gradient will be helpful for learning the function.
Active Subspaces Learning. Wycoff et al. [B3] proposed a set of three methods (Trace, Var1, and Var2) for active subspace identification. While the goal is different from our proposed methods, the acquisition functions developed in the paper do aim to learn a quantity of the derivatives, as in our work. Specifically, the acquisitions attempt to learn the expectation of the outer product of the gradient. The elements of the diagonal of the outer product of the gradient are the squared DGSMs, so learning the outer product will learn a function of the squared DGSMs, albeit without targeting them specifically.
Because these methods target learning function of the gradient, compared against them three representative problems. We used the reference implementation of the methods, from the R package activegp. The results shown in Fig. EL For clarity and due to the large number of methods, this figure includes only QR and DIG alongside Trace, Var1, and Var2; performance relative to other methods can be seen by comparing the results to Figs. 2 B. and gL The figure shows that active subspace learning methods do not perform as well as derivative information gain. While they do target related quantity, learning the active subspace is not the most effective approach to learning the squared DGSM.
B Entropy Approximations
Here we develop approximations for the differential entropies of the absolute and squared derivative posteriors.
B.1 Folded Normal Entropy
The posterior of the absolute value of the derivative in this setting is folded normal distribution. Specifically, we consider X ~N describe three approximations for the entropy of a folded normal.
V(,,,"") o²) and are then interested the entropy of =|X]. We
13

---

Hortmonn3, Dim-3
Harzmann4, Dim=4
Vehicle Safety Weight, Dim-5
Square DGSM RMSE
Square DGSM RMSE
Square DGSM RMSE
Active Learning iterations
Active Learning Iterations
Active Learning Iterations
Hartmann3, Dim=3
Hartmenn4, Dim=4
Vehicle Safety Weight, Dim=5
DGSM
ADS DGSM
ADS
Active Learning Iterations
Active Learning iterations
Active Learning Itérations
QR
DIG
Activegp: Trace
Activegp:Varl
Activegp:Var2
Figure 5: An empirical evaluation of active subspace methods on the task learning the square and absolute DGSM, with derivative information gain, derivative square information gain, and quasiran- dom as points comparison.
B.1.1 Folded Normal: Approximation 1
The folded normal distribution has two limiting forms. For f different than 0, the variance of the (pre-folded) normal o² goes to zero, the amount of probability density being folded goes to zero, and thus the distribution approaches normal distribution with that same variance. The limit of the entropy in this case is the normal distribution entropy (here in nats):
((X) = 2 (og(2
the other hand, as the variance becomes large relative to the mean, the proportion of density being folded goes to 1/2. and the limiting distribution is the half-normal distribution. For Z with half-normal distribution, the differential entropy analytic: h(Z) = h(X) log(2). Furthermore. the variance of Y is upper-hounded by the variance of the X, and nower-bounded by the variance of Z.
We approximate the folded normal entropy as convex combination these two limiting distribu- tions, using the variance bounds. Specifically, we take h(Y) h(X)= log(2), where w goes as the distribution approaches a normal, and w goes to as the distribution approaches half-normal.
This is achieved by taking:
Var(Y') Var(Z) ())
w
Var(
Var
1²2 + 0² - 2r - -02 2/TT) 0² 0²(1 - 2//x)
x (12 122)
202
where, of course, jy = E[.], which is known analytically. The final result for the entropy approxi- mation is thus, in nats,
(11 1 log (og(2
(Hξ 122) o² n log(2))
M(Y) ~ 2/
14

---

Folded normal distribution
Numerical
2,00
Approx. 1
Approx. 2
Differential entropy
Approx. 3
1.30
1.26
1.00
0.75
0.25
10
10a
Variance d²
Figure 6: Differential entropy of folded normal distribution as function of the underlying normal variance o², The figure compares high-precision numerical estimate with three approximation schemes, of which scheme provides high-fidelity analytic approximation.
B.1.2 Folded Normal:
Approximation 2
To our knowledge, the only analytic approximation already found in the literature is that of Tsagris al. [G8, Equation 40], which uses truncated Taylor series:
+ 4² ppy
h(Y) ~ log( V 2TT²) + 2
o2
(-1)at1
(nu
2mpz)2 112 ))
(_
2mp σ3
Φ ("
2nj
exp
20
==1
Tsagris et al. [38]
recommend truncating the infinite series after 3 terms, which is what we did here.
B.1.3
Folded Normal: Approximation 3
For given variance, differential entropy maximized by the normal distribution. A (first and second) moment-matcher normal distribution thus provides an upper-bound for differential entropy that can be used an approximation for any distribution. In the of the folded normal, this produces the approximation
1(Y) ~
log( (og(2TTo})
2 2
log (2T) + log (n² + 0² + 1²2)
B.1.4 Evaluation of Folded Normal Entropy Approximations
Fig. shows an evaluation of the accuracy of the three approximations described for the differential entropy of the folded normal distribution. They compared to high-precision numerical estima- tion of the differential entropy obtained via numerical integration. Entropy values are shown as a function of the variance o², for fixed μ 1.
The numerically estimated entropy in Fig. B clearly shows the limiting behavior described in Sec. B.F. for small o² it converges the normal upper bound, and for large o² it converges to one bit of entropy less than the normal upper bound (the half-normal lower bound). Approximation (convex combination of normal and half-normal entropies) provides close estimate of the true entropy. Approximation (truncated Taylor series) is numerically unstable for this range of variances and so provided poor approximation and was unsuitable for optimization in active learning.
We included both approximations and in the experiments the main text (DAbIG; and DABIG3 respectively). and they produced similar active learning performance.
15

---

B.2 Squared Normal Entropy
The posterior of the square of the derivative in this setting is the square a normal distribution. For X ~ (H, d²), we are now interested in the entropy of Y = xx.. As described in the main text, if we let Z = oz, then Z has noncentral x2 distribution Z ~ (A), with noncentrality parameter
gz. Because
h(Y = h(())(22)))) + log)
the problem of computing the entropy of the squared normal reduces to that of computing the entropy of the noncentral x distribution, with one degree of freedom. We describe approximations for this entropy.
B.2.1 Noncentral Chi-Squared: Approximation 1
Abdel-Aty [II] 'first approx."] showed that the following provides an accurate approximation of the quantiles of a noncentral x² with one degree of freedom:
Z 1+>
~((1-,,))
where C 2 1+2A Let gZ)
Because g is bijection,
9 [I+A]=
I+X
0g az
h(g(Z)) = h(Z) +
log
(8)
We know h(g(Z)) because g(Z) has a normal distribution. For the expectation term,
0g az
1 Z
log
=
log
3
1
{+
1 1+
=
llog(3)
log
log(1 + 2)
3
3
1
3
2
log(3)
log(1 +
Bllog(2)) (2))
3
In recent work, Moser [25, Equation 54] provide an analytic result for the log expectation of noncentral x" random variable:
Elog(()) = FF 1, 1,1,2,2, 2 2) - log(2),
where 7 R 0.577 Fuler's constant and 2 F2 a generalized hypergeometric function. Combining these results yields an approximation for the noncentral x² differential entropy:
1 + log(2) + 2 2
[AT(1 + 2A)
A 2,
M(Z)
log
F2 1,
A.
9(1 x|²
2 2
B.2.2 Noncentral Chi-Squared: Approximation 2
As in Sec. BT we can approximate the entropy of the noncentral x" using the moment- matched normal distribution upper bound to its entropy. The variance of a noncentral x² distribution with one degree freedom is simply 2(1 + 2A), and so the upper bound on the entropy is:
1
h M(Z)
log(4 4m(1 + 2A))-
B.2.3 Evaluation of Noncentral Chi-Squared Entropy Approximations
Figure X shows an evaluation of the accuracy of the two approximations described for the differential entropy of the noncentral x2 distribution with one degree of freedom. They are compared high-precision numerical estimation obtained via numerical integration. The left panel shows the
16

---

Noncentral x² distribution
Squared normal distribution
Numerical
Approx.
Approx.
3.3
Differential entropy
Differential entropy
2.5
2.5
2.0
1.5
1.0
10
10a
10
10a
Noncentrality parameter
Variance d²
Figure 7: Differential entropy of (left) a noncentral X2 distribution as a function of its parameter X, and (right) a squared normal distribution function of the underlying normal variance o², The figure compares a high-precision numerical estimate with two approximation schemes, of which scheme provides high-fidelity analytic approximation.
noncentral x" entropy as a function of the noncentrality parameter A. The right panel shows the entropy of the squared normal distribution computed via (0)) as a function of o² and with fixed J 1. Approximation (using the quantile function of Abdel-Aty) provides close estimate of the true entropy, for both the noncentral x² distribution and after transformation to the squared normal entropy. The normal upper bound in approximation 2 is accurate only for small variances, when the amount of reflection at the origin is small.
Both of these approximations and DSqIG2. They usually performed similarly, though DSqIG2 performed slightly better in some problems despite being worse approximation for the actual underlying entropy. Approximation 2 overestimates the entropy, and thus the information gain, at inputs with high predictive variance. This will generally make the acquisition more exploratory, which seems have been beneficial those problems.
also included in the experiments in the main text, DSqIG1
C Computational Complexity
All of our acquisition functions have closed-fomm expressions. Computing the GP posterior is O(t id)) with i the number of data and d the dimensionality of the function. Here the 1 comes from inverting kx, and the tdd comes from multiplying it with the gradient of kz,,x- Given that, the remaining computation will scale linearly with d, since just applying various formulae the posteriors for each dimension.
D Global Look-Ahead Acquisition Functions
Our look-ahead acquisition functions were all local, in that they evaluated the impact of observa- tion at x, only on the posterior at x, In this section, we consider the global look-ahead acquisitions where evaluate the impact the posterior across the entire input space. These acquisition functions are expressed as integral over the previously proposed local acquisition functions. Their complexity is Ol1 idM) with M defined the granularity of the integration. We provide the results and discussion Appendix E
D.1 Derivative Look-Ahead Distribution at Different Input Locations
We wish to predict the impact that observing f at a candidate location x, will have on the model's derivatives at a different location X4. This will allow to select point x, that most improves the
17

---

model of the derivatives in the overall search space, in expectation. Under a GP, this look-ahead distribution is tractable. Conditioned the observations D, f(x.) and 0f(+) have bivariate
0z:
normal joint distribution for each input dimension i. The formula for bivariate normal conditioning once again provides the look-ahead distribution:
(x(+))
f(x.) - yo. ~ N (uffin (o (o+1m.)))
dr;
where the look-ahead mean and variance are respectively
14 14,,3
T++d o²
(v- -.),,
(o (o=)).))²= (o+,))=
(9)
0s(x+] |D] the posterior covariance between f at x, and the derivative at with 0+..; Cov (x.),
de:
X4, and (04,,)² )x )² [E'(x))ls the posterior variance of the derivative. We use a similar notation short-hand o² "1 = of (x+) and p' = nut (x) as the look-ahead posterior mean and variance at the input x+ if (x., 1/-) were added to the data.
D.2 Global Look-Ahead Acquisition Functions
Global look-ahead acquisition functions are constructed by using the distribution in ( as the look- ahead distribution when constructing the acquisition functions as described in Section The acqui- sition then provides the value that making an observation at will have for the model predictions at X.. To make the acquisition function global, we integrate over the input space. For example, global derivative variance reduction is:
ONDY (x.)
/
1 |
[x)()
Ey, (x)
/ Σ dx(()) of (x)
For all of these global acquisition functions, the integral over the input space is estimated with QMC sampling, meaning the impact of the observation at x, is evaluated on global reference set of size M, over which the acquisition function is averaged.
E
Additional Results and Discussion
E.1
Additional Synthetic Problems
Figs. & A and I provide experimental results for the Hartmann3, Gsobol6, and a-function prob- lems. The results are largely consistent with those of the other problems in the main text, the main exception being the stronger than usual performance of QR on the Gsoboló problem. The deriva- tive max variance methods also perform better than usual on this problem, suggesting that higher degree of exploration is important.
The main text evaluated performance of the raw and squared derivative acquisition functions on the squared DGSM, and the absolute derivative acquisition functions on the absolute DGSM. Fig. I shows the performance of all acquisitions both DGSMs, for all 13 problems. On most problems, the squared and absolute derivative information gains performed similarly, without clear distinction between learning the absolute or squared DGSM.
E.2 Running Times
Table M gives the wall time running times for acquisition optimization. Generally information gain methods are more expensive, with the squared derivative information gain particularly expensive due the use of a hypergeometric function in its approximation. However, even the maximum time for that method (214 seconds) is fast relative to the time-consuming function evaluation setting that the focus of this paper. It is interesting to also note that running time generally increases with dimension but not always. This is because the optimization time will depend on the shape of the acquisition function surface and how hard the optimization is, which is separate from the dimensionality of the problem.
18

---

Hortmonn3, Dim-3
Gsobol6, Dim-6
Dim=6
Square DGSM RMSE
DGSM RMSE
DGSM RMSE
Squann
Active Learning lterations
Active Learning Iterations
Active Learning Iterations
QR
NAR
DV
DV,
DIG
Figure 8: Additional experiment problems for the results in Fig. h
Hartmann3, D
Gsobol6, Dim=6
function. Dim-6
DGSM RMSE
AbS DGSM RMSE
Abs DGSM
Active Learning Iterations
Active Leaming Iterations
Active Learning Iterations
QR
MAR
fIG
DABV
DAbV,
DAbIG1
DAbIG,
Figure 9: Additional experiment problems for the results in Fig.
E.3 Results and Discussion for Ranking Metries
In Figure 12 we show the performance evaluated with NDCG. NDCG accounts for the position of each dimension, giving higher importance dimensions at the top of the list, and normalized to a scale from to 1, where represents perfect recovery of the order. RMSE is generally a better metric since it shows the ability of the method to converge the is the most commonly used metric in the literature. As far as we are aware, NDCG has not been used before performance metrie. However, in some applications, the ranking of the variables is important for practitioners, for instance when doing parameter selection for downstream tasks. We thus included ranking metric provide another perspective into the results. When evaluated with this ranking metric, we find that active learning is able to efficiently discover the correct ordering of variables according to their importance, and that derivative information gain acquisitions continue to perform generally best.
values of DGSMs, and because
Fig. I shows the results of the ranking metric NDCG for two of the synthetic functions, a-function and Morris. For these problems, there clear instability in their ranking results. These two particular problems have several dimensions with the very similar DGSMss values that thus cannot be reliably sorted. Thus the ranking of the variables changes frequently over iterations leading to the instability in the curves. This is issue of NDCG metric, and shows that care should be taken when using it for this purpose.
E.4 Results and Discussion for Global Look-ahead Acquisition Functions
In this study, we compare local look-ahead acquisition functions proposed in the main text to the global look-ahead acquisition functions described in Appendix DI We denote the global variance reduction of the absolute and squared derivatives as IDAbV, and IDSqV, and global information gain of the raw and squared derivatives IDIG and IDSqIG.
19

---

Hortmonn3, Dim-3
Gsobol6, Dim-6
Dim=6
Square DGSM RMSE
Squan DGSM RMSE
Square DGSM RMSI
Active Learning lterations
Active Learning Iterations
Active Learning Iterations
QR
NAR
fIG
DSqV
DSqV,
DSqiG1
DSqIG2
Figure 10: Additional experiment problems for the results in Fig.
Method
Ishigamil
Gsobol6
Gsobol10
Gsobol15
Morris
IVAR
0.44
0.39::0.02
0.1910.01
0.21
0.2:50.02
fIG
0.43+0.03
0.37
0.29=:0.04
0.17=0.02
0.16#10.01
DV
0.6210.08
0.31
0.45+00007
0.3410.04
0.16#0.01
DV,
1.43+0.22
3,43:0.49
10.56
6.32+1.06
3.54
DIG
8.59+1.115
16.75+0.44
24.6++2.5
28.15+44.75
13.92=3.78
DABV
1.37+00113
4.26+t0.3
3.93+-0.9
5.31+0.63
5.87=0.7
DABY,
2.4+0.3
9.69+1.19
13.31
8.48+1.43
3.96+0042
DSqV
1.58:1.222
4.23+:0.2
12.1814.23
16.1
6.95=1.0
DSQV,
2.84
6.66
8.871..77
6.29-10.9
9.0-2.29
DSqIG
96.3219.55
214.39-30.84
184.77
122.27-+18.61
95.24
Table 1: Average acquisition function optimization times across the active learning loop for each method and for representative sample of problems.
As discussed above, the complexity of the global acquisition function depends on the granularity of the numerical integration. Given that the granularity of the integration M needs to be high for numerical accuracy, the computation time and memory requirement becomes prohibitive, especially in high dimensions. Figs. 14 and IS show that despite the extra computational cost, the global acquisition functions have comparable performance for information gain than their local versions. Given the high computational cost incurred by the integration, we found the local acquisition functions to be more practical.
variance reduction and lower performance
F.5
Adding Perturbations the Space-Filling Design
Fig. D showed that derivative active learning methods selected points adjacent to existing points, which aids in estimating the derivative. reasonable heuristic for adapting space-filling design to derivative estimation is to sample pairs of points, in which one point in each pair is small perturbation the other. We implemented that heuristic to it could improve performance of the baseline.
We generated quasi-random sequence and then interleaved it with points that small perturba- tions of the point before it. Specifically, for all n odd, we took x" from the quasi-random sequence, and then took x x" + c, where AO0 In). with That is, each dimension was given a small Gaussian perturbation on the value from the previous iteration. We denote the method as QR-P. Fig. IG shows the performance results alongside QR and some active learning methods. This approach did significantly improve performance on one of the problems, but generally the results were comparable to QR without the interleaved pairing. This approach does rely on the perturbation hyperparameter 17, and the "right" choice likely depends on the problem.
0.01 and box bounds sealed [o. 1].
20

---

Iargani.. Dinn
3. Dmne]
IMVgami2, 3mm>
Roazs, Dined
Scuam OOCON RANSE
Ala DOOR PAOE
Sqpare DC24
Square EICON
Achve Leuning Ereation
Adilen Learning betotions
Active Leurning Erration
Acthe Learing terasces
Actie Learing trratione
Hartnaans),
Hartmaakk
Ale Do0e SUIEE
ARA DOGH PAICE
DOORT PUODE
DOGON SANCE
Active learing Ereation
Adte learnnng terstion
Adtiv Leurnnng Enratione
Activ Learting terstces
Adtie leariigg barations
DOGe bage
APA FLOO RArU
Starm DOCH SANET
ALL OOCOH PACCE
Sparm COSO Saice
Veride Accaleation. Skmve
Gustal3 201213 231
Marra Dhrmg)
Vorra Der-28
No OCCB D30E
Sqare D0CN PAICE
AX4 DOOR NUYE
SOANC GOUK PPOC
AdOve Searnng fereter
Actvv
Adhre
Actvv
Casbolt, Bindd
Costall, 3tme6
Gastodio Dime12
aJandin Cired
DA0 ZOCA DAOO
COCN PM22
Squam DOGH
Adtve Louning firekrs
Adhve Leerving Iteuttoo
Actvv Learning Rarptans
Aci Learning terstooa
Adive Learn leuttat
Al COON DAGCE
Actsa
DV,
DIG
DABW
DABV,
DABIG
DANIG,
DSaV
DSqV.
DSglG,
DSqic
Figure 11: All acquisition functions evaluated on all benchmark problems for both the absolute and squared DGSMs. A superset the results in Figs. 21 and !!
21

---

I2agerd Dited
S2egart Dete3
129ger5 Ditted
s2apre2, Bried
CO2 NOC0
PAC THOW BE
SQUAR CISA
Aln DOGK
SQaR
Ictnv Loaning fooaioe
Adlve Loeri toutie
ACNe Loaring Reratons
4cTR Loariin Rerities
AJNC LON 1E-05991
Woores
Abe 00000 NDOG
Squar COON NOOKS
Aa DOON MONg MDCS
Squar COON COONO WIDDO
Actvv
mg
2ttm
Whbtk Suitz Dermt
Winsie Seldy Dres)
Xittll Seltuz Scutho Deres
PELR COC NOO
ALNE Learing fookors
Abrc Lanengg forttiis
Kile Loaring Rottooi
4c Loarien Rertiies
Adveian 10:00ano
Sahecl Selidy Aweehaiirr.
Sutdi 20ml
Duska 13, Derelh
Sestadh 2tmet
Getelt Dared
Al D020 NCOG
SCARM DOCO BOOO
Loaring
loomingtiooiss
ACEN
Ge Dmeij
BOOK ROCU MY
Adhc Learnng flcters
Acieo loaming Aeratins
PARR
10
DAVV
DANH/
DAAIN
DAMSS
050
05e
05g6
05y6:
Figure 12: Synthetic and world experiments evaluated using NDCG. The NDCG of the square DGSMs and absolute DGSMs is reported with the mean and two standard errors of 50 repeats.
function, Dm--3
funncioonn Dm-66
Morris, Dim-20
Morri, Dim-20
Supurre Dizun nOoGs
POS DOSSN NDCS
ADS DESMMNGNG
Learning tarations
Ceaming harations
Active Learning karations
Lsamiing itarations
DAVV
DANI/
DAVIG
DANG
DSef
DSeg
05g6
36p6:
Figure 13: Synthetic experiments with multiple variables holding the same ranking evaluated using NDCG. The NDCG of the square DGSMs and absolute DGSMs is reported with the mean and two standard errors of 50 repeats.
22

---

Nhigamil,
Ishggem22
IMSE
DOCOM IAAGE
5tp NEW DOOM RMM
Squarr COSM NOMSE
Active Leaming terations
Learning Ite oolonns
Active ning Iterations
Activa Learning harations
Als D330 RMSE
Abs DOUSRN HAASE
Ab5 DESS RASE
Ais DOUSON RASE
Active Leamning térations
Active Learning herations
Active Leamning térations
Advie Leamiing kerations
DV,
DABV,
DSqV,
IDV,
IDADV,
IDSqV,
Figure 14: Experiments comparing the global (integrated) variance reduction acquisition function their local (non-integrated) version. The RMSE of the square DGSMs and absolute DGSMs is reported with the mean and two standard errors of 50 repeats.
Hartmann3, 3ime3
Hertmannd, Dimed
Nhigane,, Dine3
Ishigamii Dm
Seguaree DIC04 more
Square DIOOM RMFO
Squurr DOSS RHOE
DCSM
Active Learning ttaratins
Actree Learning laeretoom
Active Leariin tterations
Active Learning Baralions
Hartmann3, Dima
Hartmarnd, Dittéé
Istigamii, Dim-3
Ishiammi, Dimw3
Ab: DOSK PASE
Abr DOSO VASSE
MBS DOSK RASE
Abe DOSH HASSE
Active Leaming ttrations
Actio Learning ftactions
Active Leaming iteration
Active Learning harations
DASIG,
DSelG,
DSelG,
IDAbIG,
IDSe/G,
IDSeIG,
Figure 15: The same results as Fig. 14 for the global (integrated) information gain acquisition functions and their local counterparts.
23

---

shigami7,
Hartmann3, Dim-3
Square DGsi RMSE
Squarr DGSS mAsE
Square C65M RMSE
Active Learning terations
Active Learring ations
Accive Lear Learning lerations
Active Leaminng Iterat
WeighT,
Groboll5, Dim=15
Mon Dim=20
b054 RMSE
Squan DISSH RASE
Square CSSSN RAISE
Active Learning iterations
A
Learning iterations
Active ning Iterations
Active Leaming hérations
QR
QR-P
DABV
DSqV
DIG
DSqiG1
Figure 16: The RMSE of the squared DGSMs is reported as the mean (and two standard errors shaded) over 50 runs of active learning. New results of QR-P baseline are added.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the paper supported by mathematical derivations in Section E. related work discussion in Section B and experiments Section B Please also refer to additional related work and experiments in the Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings. It is fine to include aspirational goals motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the empirical limitations of some of our proposed approaches in the experiment discussion Section & We explain the conditions under which these limi- tations emerge. We suggest, under these limiting conditions, using alternative approaches, that we also propose in our paper. We additionally suggest potential future work to over- come the limitations in Section i
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-tex, system might be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
25

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes].
Justification: We provide the mathematical steps and justification of each derivation in Section and in the Appendix. We do not provide a theoretical regret bound convergence study in our paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proof's provided appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes].
Justification: We provide detailed description of our experimental setup in Section B Our code has been released on github, with the link provided in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
26

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer:[Yes)
Justification:
Code has been released, with the link in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (httpa:)
mripss.cc/uulic/ggudddee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they chosen, type of optimizer, etc.) necessary understand the results?
Answer:[Yes)
Justification: We provide a description of and references to our benchmark problems. The code for running the experiments, which includes all train/test splitting and hyperparameter selection, is released with the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question:
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]..
Justification: We report the mean and 2 standard errors of the mean as error bars on all our experiments, over 50 runs. We state these details explicitly in the captions of the figures.
Guidelines:
27

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report 1-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of eπrоrs is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [No].
Justification: We our experiments on different types of machines for the sake of paral- lelization. However, we provided a detailed time complexity discussion in Appendix O and wall-clock running times in Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't it make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]..
Justification: Our proposed methods are widely applicable science and engineering ap- plications and do not have any conflict with the NeurIPS code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
28

---

Answer: [NA].
Justification:
The proposed methods can be applied to a wide variety of science and engi-
neering appreciation. There no clear path to negative societal impact from our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considcrations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e-g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not include data models with a high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards
allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]..
Justification:
All benchmarks used in the paper properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
29

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswithcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced the paper well documented, and is the documenta- tion provided alongside the assets?
Answer: [Yes].
Justification: Code has been submitted with this paper and includes clear documentation for running the experiments.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA].
Justification:
This paper does not include crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper deseribe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA].
Justification: This paper does not include research with human subjects.
Guidelines:
30

---

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva-
lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "NRHuVk658K", "reviewer": "Reviewer_uHUu", "summary": "The authors develop and compare sequential design criteria for Gaussian-process-based estimation of gradient-based sensitivity metrics to assess the importance of individual variables.", "questions": "I think it's great that the authors considered so many different approaches in their experiments, but it's disappointing that this does not result in practical takeaways for the reader in the conclusion section. If I'm an engineer and want to learn about the sensitivity of my simulator to its parameters, which of your acquisitions would you recommend I start with? Such a discussion would benefit the practioner-reader. Maybe just a sentence or two (beyond what is in 5.1) saying \"starting with XYZ is our general purpose recommendation\" or some such. \n\nIf there is any possibility of adding execution timings to this article I think it would make it more useful to practitioners, regardless of what those timings show.", "limitations": "The numerical experiments consider a variety of functions, but consist of very limited budgets. \nThere appears not to be any discussion of the overhead required to conduct the sequential design. \nThe authors mention that \n\"All methods were implemented to be auto-differentiable and, therefore, are efficiently optimized with gradient optimization\".\nIt is great that the methods were implemented in a framework allowing for automatic differentiation.\nBut this does not guarantee that the optimization will be efficient. \nIt's true that BO literature often glosses over the overhead required, and that for experiments of very high cost, it is negligible by comparison. \nBut it is still important to report the overhead execution time. \nAs they discuss in the checklist, there is some discussion of the complexity order in Appendix B but no reporting of actual execution timings.\nThis is important to give practitioners thinking about using this method when faced with an expensive-but-not-overwhelmingly-so simulator to decide whether the overhead associated with this method is tolerable relative to their particular application or not.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The problem of learning variable sensitivities is well-motivated and a matrix of criteria are proposed to address it, from considering different gradient quantities (the gradient itself, its absolute value, and its square) and different measures of sequentially learning them. \n\nThis paper clearly addresses an interesting problem and presents itself well. \n\nThe numerical experiments are sufficient in rigor and quantity to establish the performance of the proposed method.", "weaknesses": "As the authors point out on the checklist, theoretical guarantees on the performance of the acquisition strategies are not available but the numerical experiments are sufficient to establish them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback and for supporting our paper.  Below we provide answers to your concerns. \n\n**Questions**\n\n1. We agree that adding a general takeaway would be beneficial for practitioners. We will add the following paragraph to the discussion: “Our general recommendation is to use the information gain of the gradient (DIG) or the information gain of the squared gradient (DSqIG) for low dimensional problems (up to d=10). For high dimensional problems (d>10), we recommend using the variance of the absolute value of the gradient (DAbV) or the variance of the squared gradient (DSqV)”. This will complement the existing discussion in Sec. 5.1 on why the exploratory nature of variance-based approaches are important in high dimensions.\n\n2. See the table below for the acquisition optimization time for each method across different problems. Generally information gain methods are more expensive, with the squared derivative information gain particularly expensive due to the use of a hypergeometric function in its approximation. However, even the maximum time for that method (214 seconds) is fast relative to the time-consuming function evaluation setting that is the focus of this paper. It is interesting to also note that running time generally increases with dimension but not always. This is because the optimization time will depend on the shape of the acquisition function surface and how hard the optimization is, which is orthogonal to the dimensionality of the problem.\n\n\n| Method | Ishigami1 | Ishigami2 | Hartmann4 | Gsobol6 | Gsobol10 | Gsobol15 | Morris |\n|:--|--|--|--|--|--|--|--|\n| DAbV | 1.37±0.13 | 2.03±0.21 | 0.78±0.05 | 4.26±0.3 | 3.93±0.39 | 5.31±0.63 | 5.87±0.7 |\n| DAbV_r | 2.4±0.3 | 2.06±0.27 | 2.77±0.23 | 9.69±1.19 | 13.31±1.16 | 8.48±1.43 | 3.96±0.42 |\n| DSqV_r | 2.84±0.64 | 2.3±0.37 | 2.0±0.21 | 6.66±0.88 | 8.8±1.77 | 6.29±0.9 | 9.0±2.29 |\n| DV_r | 1.43±0.2 | 1.32±0.23 | 1.66±0.14 | 3.43±0.49 | 10.56±1.78 | 6.32±1.06 | 3.54±0.51 |\n| DSqV | 1.58±0.22 | 2.44±0.25 | 0.81±0.07 | 4.23±0.62 | 12.18±4.23 | 16.1±2.03 | 6.95±1.0 |\n| DIG | 8.59±1.15 | 9.45±0.99 | 8.09±1.09 | 16.75±0.46 | 24.6±2.5 | 28.15±4.75 | 13.92±3.78 |\n| fIG | 0.43±0.03 | 0.44±0.03 | 0.41±0.03 | 0.37±0.01 | 0.29±0.04 | 0.17±0.02 | 0.16±0.01 |\n| DV | 0.62±0.08 | 0.79±0.09 | 0.1±0.01 | 0.31±0.01 | 0.45±0.07 | 0.34±0.04 | 0.16±0.01 |\n| fVAR | 0.44±0.03 | 0.48±0.03 | 0.37±0.02 | 0.39±0.02 | 0.19±0.01 | 0.21±0.02 | 0.2±0.02 |\n| DSqIG | 96.3±19.52 | 100.95±21.65 | 151.56±37.84 | 214.39±30.84 | 180.5±24.79 | 122.27±18.61 | 95.24±28.67 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HH5RxH4xgk", "reviewer": "Reviewer_oJrN", "summary": "In this paper, the authors study how to select observation data to improve the efficiency of sensitivity analysis. The focus is on measuring sensitivity through a function’s gradient variability. The authors provide a derivation on gradient variance formulae based on Gaussian process surrogates. Various acquisition functions based on gradient, absolute gradient, and squared gradient are described. Empirical evaluations show that the information gain of derivatives and squared derivative perform best in the majority of experiments but in high-dimensional problems, max variance of derivative and squared derivative performs the best. Moreover, the proposed methods can effectively discover the correct ranking for different variables in the sensitivity analysis.", "questions": "1. What GP kernels did the authors use in their experiments?\n\n2. In Figure 2, RMSE sometimes went up with more active learning iterations. Why is that?\n\n3. Based on the discussion in section 5.1, would it be possible to combine the information gain-based acquisition function with the max variance-based acquisition function to produce a new acquisition function that works well in both low and high-dimensional problems?", "limitations": "The authors discussed limitations on some of their acquisition functions.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This work is novel in its systematic study of active learning in DGSM (derivative-based global sensitivity measure). The exposition is clear to me even though I am not an expert in sensitivity analysis. \n\n2. The authors provide comprehensive empirical evaluations of their proposed approaches including both synthetic functions and real-world applications. The results of RMSE estimations show significant improvements over baseline methods.", "weaknesses": "1. It seems that some categories are missing some methods. For example, in section 4.3, there is no acquisition function based on information gain for absolute gradient. In the baseline methods, there is no $f_{V_r}$, i.e., maximizing variance reduction on function values. These omissions make the empirical evaluations incomplete.\n\n2. The NDCG (normalized discounted cumulative gain) results are weak compared to the RMSE results. Can the authors elaborate on the practical implication of this? Is RMSE typically more important than NDCG?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback and for recognizing the strengths of our paper.  Below we provide answers to your concerns. \n\n**Weaknesses:**\n1. Thank you for the suggestion to fill out the matrix of acquisition functions. For the baseline f_{V_r}, since our experiments used noiseless function evaluations, the observation at x reduces the GP posterior variance at x to 0; thus f_{V_r} is equivalent to f_{VAR} (max variance of f). We will make sure this is clear in the paper. Information gain for the absolute value of the gradient was indeed missing. We derived and implemented information gain for the absolute derivative based on an approximation of the entropy of the folded normal distribution [1]. We evaluated this acquisition on the illustrative test problem in Figure 1 from the main text; the result is in Fig. 3 in the rebuttal results. While generally, the results look similar to other derivative information gain acquisitions in Fig. 1 of the main text, there is a noticeable discrepancy at x=0.6. This is because of numerical issues. The approximation from [1] uses a truncated Taylor expansion that includes an exponential term that can blow up and become a poor approximation depending on the posterior values. To be useful for acquisition optimization, a new, more accurate and stable approximation of the folded normal distribution entropy will be necessary. We will add this result and discussion of future work to the paper.\n\n2. RMSE is typically more important than NDCG since it shows the ability of the method to converge to the true values of DGSMs. It is also the most commonly used metric in the literature. As far as we are aware, NDCG has not been used before as a performance metric. However, our own experience has been that in some applications, the ranking of the variables is important for practitioners. For instance, parameter selection for downstream tasks. We thus included a ranking metric to provide another perspective into the results. However, ranking metrics sometimes fail to provide insights when several variables are equally important and have the similar DGSMs values. We discussed this case in the appendix and provided results in figure 6. Hence, RMSE generally provides more important insights into the efficacy of various methods.\n\n**Questions:**\n\n1. We used an Automatic Relevance Determination (ARD) RBF kernel.\n\n2. Active learning involves a trade-off between exploring new regions of the input space and exploiting known regions. While the model is still exploring, adding a data point in one part of the space may cause a global adjustment in the model predictions that can cause it to err in another part of the space. This is especially possible early on when the model is still learning the kernel lengthscales, and has the capacity to over- or under-fit the data. With more exploration and data, the model self-corrects and RMSE decreases.\n\n3. Thank you for the suggestion. This certainly seems possible, and in fact methods that ensemble acquisition functions have recently shown great promise for general-purpose black-box optimization tasks [2]. We will draw connections between those results and this work in our discussion of future work.\n\n[1] M. Tsagris, C. Beneki, H. Hassani (2014) \"On the folded normal distribution.\" Mathematics 2:12-18\n\n[2] R Turner et al. \"Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020.\" NeurIPS 2020 Competition and Demonstration Track. PMLR, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VlyTrn0Nlp", "reviewer": "Reviewer_nwDp", "summary": "The authors propose eight acquisition functions for active learning of functions of the gradient of a Gaussian Process, which is motivated by the use of the gradient for global sensitivity analyses. They provide experimental evidence assessing the relative performance of these acquisition functions.", "questions": "- On pg. 2, the authors state “To learn the squared DGSM, active learning selects points that are adjacent to existing observations, as that adjacency is valuable for the derivative estimate.” This suggests to me that a natural baseline comparison in the experiments would be a heuristic method that selects pairs of points some small distance apart. Do the authors have thoughts on how their proposed acquisition functions would compare to such a heuristic method?\n- On line 135, should $m_{\\boldsymbol{x}}$ be $m_X$?\n- Why is an information gain acquisition function not derived for the absolute gradient?", "limitations": "The authors discuss limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper’s contribution (a novel application of active learning algorithms and the derivation of several acquisition functions tailored to this application) seems strong. The experiments are thorough, and the paper is well-written and the method clearly presented.", "weaknesses": "- Although the application is novel, the proposed acquisition functions are straightforward applications of standard acquisition criteria in the active learning literature.\n- The authors provide experimental evidence on the performance of their proposed method(s), but do not provide explicit convergence guarantees or other theoretical results.\n- There appear to be many assumptions implicit in the authors’ method, such as that the sensitivity measures are well-approximated by the GP gradient. I would be interested in seeing more discussion of whether these assumptions are required for the effectiveness of their proposed method(s).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comments and helpful suggestions. Here are responses to the weaknesses and questions:\n\n**Weaknesses :**\n\n1. While the variance and IG criteria are known in the literature, formulating them in a computationally efficient and tractable fashion is not straightforward for DGSMs, which is our main technical contribution.\n\n3. It is correct that the assumption that the GP derivative approximates the gradient of the true function of interest is vital for the success of the methods we develop. This assumption is motivated both practically and theoretically. On the practical side, GPs are indeed the standard surrogate model for sensitivity analysis of blackbox expensive functions; see e.g. [5] from the main text, and citations therein. On the theoretical side, GPs are favored because they are universal approximators, meaning that, with a suitable kernel, they can indeed approximate any arbitrary continuous target function [1]. Thus, requiring the GP gradients to provide a good approximation of the gradients of f is not, in theory, a limiting assumption.\n\n**Questions:**\n1. Thank you for the suggestion, we implemented and evaluated the suggested idea where we evaluate pairs of points near each other. We generate a quasi-random sequence and then interleave it with points that are small perturbations of the point before it. Specifically, for all n odd, we took x_n from the quasi-random sequence, and then took x_{n+1} = x_n + eps, where eps ~ MVN(0, I*eta) and eta=0.01 (with box bounds scaled to [0, 1]). That is, each dimension is given a small Gaussian perturbation on the value from the previous iteration. Fig. 1 in the rebuttal results shows the performance of this method, which is there called QR-P. In one problem (Ishigami2) it performs very well, though generally the results are comparable to QR without the interleaved pairing. This approach does rely on the perturbation hyperparameter eta, and the “right” choice likely depends on the problem. We will add this ablation study to our paper.\n\n2. In line 135, we use x to refer to any possible value, so it can be m_X if we are evaluating the mean function on X or can be m_{x*} if we are evaluating it on a new input. \n\n3. Thank you for the suggestion. We derived and implemented the information gain for the absolute value of the gradient based on an approximation of the entropy of the folded distribution. We show the results in Fig. 3 in the rebuttal results, on the illustrative problem of Fig. 1 from the main text. Please see the response to reviewer oJrN for full details. In short, the approximation uses a truncated Taylor series with an exp() term that is numerically unstable for values of the posterior mean and variance that we run into in practice. Thus, this acquisition function is not a suitable optimization target, but we will add the result and discussion of it to the paper.\n\n[1] CA Micchelli, Y Xu, H Zhang (2006) “Universal Kernels”, Journal of Machine Learning Research 7:2651-2667.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ef2ZDheF47", "reviewer": "Reviewer_Uip4", "summary": "This paper considers active learning strategies for global sensitivity analysis of expensive black-box functions to efficiently learn the importance of different input variables. Novel active learning acquisition functions are proposed to target key quantities of derivative-based global sensitivity measures (DGSMs) under Gaussian process surrogate models. The study showcases the application of active learning directly to DGSMs and develops tractable uncertainty reduction and information gain acquisition functions for these measures. Through evaluation on synthetic and real-world problems, it shows that active learning acquisition strategies enhance the sample efficiency of DGSM estimation, especially with limited evaluation budgets.", "questions": "1. In the right panel of Figure 1, why are the curves in the middle and bottom graphs so similar to each other? \n2. In Figure 2, why do some curves rise again as the number of active learning iterations increases?\n3. Even if the global sensitivity estimation is very accurate, in practical applications, is it still necessary to use another model to accurately estimate f?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well structured and the main points are clearly outlined.\n2. Novel acquisition functions are derived to target key quantities of derivative-based global sensitivity measures (DGSMs) under Gaussian process surrogate models.\n3. The performance of the proposed active learning acquisition strategies are evaluated by numerical experimens.", "weaknesses": "1. The main goal of global sensitivity analysis is to determine the impact of input variables on the output of a model. The most effective performance metric for this purpose is yet to be identified. \n2. There is a need for a practical application that showcases the real-world value of using acquisition functions as presented in the article.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review of our paper and for your questions, which we answer below.\n\n**Weaknesses:**\n1. It is correct that we do use two performance metrics in our paper, RMSEs and NDCGs. As discussed also in the response to reviewer oJrN above, RMSE is the most effective performance metric for this purpose. RMSE is typically more important than NDCG since it shows the ability of the method to converge to the true values of DGSMs. It is also the most commonly used metric in the literature. As far as we are aware, NDCG has not been used before as a performance metric. However, our own experience has been that in some applications, the ranking of the variables is important for practitioners. For instance, parameter selection for downstream tasks. We thus included a ranking metric to provide another perspective into the results. However, ranking metrics sometimes fail to provide insights when several variables are equally important and have the similar DGSMs values. We discussed this case in the appendix and provided results in figure 6. Hence, RMSE generally provides more important insights into the efficacy of various methods.  We will clarify in the paper that RMSE (the typical performance metric in the literature) is the most effective.\n\n2. Estimating the importance of the variables of black box expensive functions is used in several real-world applications (systems safety [2], biomedicine[3], environmental models [4,5], hydrology [6], and more [1]) where the function evaluations are expensive and the functions are high dimensional, so sample efficiency from active learning is crucial. Prior work has discussed the budget efficiency issue repeatedly [e.g., 7,8,9]. Within our paper, we did apply our methods to 3 real-world problems: the Car Side Impact problem, and two Vehicle Safety problems. These problems use simulations of real safety applications. While the simulations are not costly, we would not be able to rigorously evaluate a large suite of methods as we do in the paper on problems with actual costly evaluations.\n\n**Questions:**\n\n1. The two figures represent the acquisition function values for the information gain over the gradient and the square of the gradient. The figures show that both acquisition functions score most inputs in the search space similarly, reflecting the fact that in this problem, the best points for learning df/dx also learn (df/dx)**2, and vice versa. This will, of course, not always be the case.\n\n2. This is a good question that was also asked by reviewer oJrN; see response there for more details. In short, as the method is exploring new regions, it might add points that are not immediately useful, thus temporarily increasing the RMSE by causing an over- or under-fitting of the model. This can temporarily lead to higher RMSE. However, with more data, the model self-corrects and RMSE decreases.It is also important to note that even when this happens, our proposed approaches still outperform the baselines. \n\n3. To answer your question, we ran experiments to see if our methods that learn the derivatives of f also learn f, or if a separate procedure would be necessary to also estimate f. Conceptually, learning the derivatives of f will also learn f (via integration, by the Fundamental Theorem of Calculus). This is especially true in our setting where we are learning the derivatives from observations of f itself, with a model that is explicitly modeling f alongside its derivatives. However, it is not necessarily the most sample-efficient approach for learning f. The results of the experiments are in Fig. 2 in the rebuttal results, where we evaluate methods on their ability to learn f, by using RMSE of f as the performance metric. Those results show that the derivative information gain (DIG) is actually better at learning f than the f_{VAR} acquisition function that selects points of maximum uncertainty in f. So, the direct answer to your question is no, it is not necessary to use a separate method to accurately estimate f, as estimating the derivatives will in practice also estimate f. The fact that DIG outperforms f_{VAR} can be attributed to DIG being less myopic than f_{VAR}.\n\n[1] Saltelli, Andrea, et al. Global sensitivity analysis: the primer. John Wiley & Sons, 2008.\n\n[2] Qian, Gengjian, et al. \"Sensitivity analysis of complex engineering systems: Approaches study and their application to vehicle restraint system crash simulation.\" Reliability Engineering & System Safety 187 (2019): 110-118.\n\n[3] Wirthl, Barbara, et al. \"Global sensitivity analysis based on Gaussian‐process metamodelling for complex biomechanical problems.\" International journal for numerical methods in biomedical engineering 39.3 (2023): e3675.\n\n[4] Pianosi, Francesca, et al. \"Sensitivity analysis of environmental models: A systematic review with practical workflow.\" Environmental Modelling & Software 79 (2016): 214-232.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all of the reviewers for their constructive reviews. We have included new results in the rebuttal to address the major questions from each review:

1. Running times (Reviewer uHUu): The table in that review response provides running times for all of the active learning methods, for 7 of the benchmark problems.

2. Information gain for the absolute DGSM (Reviewers oJrN and nwDp): We added an implementation of this method, and Fig. 3 in the rebuttal results shows how it works in the test problem of Fig. 1 from the main text. In the detailed response below, we explain why this approach has numerical issues that make it a poor optimization target.

3. A new baseline method using paired sampling (Reviewer nwDp): We implemented the new baseline method suggested by the reviewer (here called QR-P; details below), and give the results for 8 of the benchmark problems in Fig. 1 in the rebuttal results.

4. Does learning DGSMs also learn f (Reviewer Uip4): We evaluated the active learning methods with RMSE of f, instead of RMSE of the DGSMs, and found that derivative information gain does also learn f, better than the max-variance-of-f method (Fig. 2 in the rebuttal results). So subsequent evaluation to learn f in addition to the DGSMs is not necessary.

These additional analyses have strengthened the paper, for which we again thank the reviewers.

Meta-review: The paper introduces novel active learning acquisition functions for global sensitivity analysis of expensive black-box functions, focusing on derivative-based global sensitivity measures (DGSMs) using Gaussian process models. The proposed methods significantly improve sample efficiency and accuracy in sensitivity analysis, particularly with limited evaluation budgets, as demonstrated through comprehensive synthetic and real-world evaluations. The rebuttals have resolved other reviewer concerns, and the proposed changes are straightforward for the camera ready version. I concur with the reviewers' consensus on acceptance.
Decision: Accept (poster)