Incorporating Expert Priors into Bayesian Optimization via Dynamic Mean Decay
Authors: Chongqi Qu
Meiqin Liu
Jian Lan
Shanling Dong
Zhunga Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Bayesian optimization
Gaussian processes
Expert prior knowledge
Hyperparameter optimization
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We introduce DynMeanBO, a simple and general BO framework that dynamically incorporates expert priors, ensuring both faster convergence and robustness to inaccurate knowledge.

Abstract: Bayesian optimization (BO) is a powerful approach for black-box optimization, and in many real-world problems, domain experts possess valuable prior knowledge about promising regions of the search space. However, existing prior-informed BO methods are often overly complex, tied to specific acquisition functions, or highly sensitive to inaccurate priors. We propose DynMeanBO, a simple and general framework that incorporates expert priors into the Gaussian process mean function with a dynamic decay mechanism. This design allows BO to exploit expert knowledge in the early stages while gradually reverting to standard BO behavior, ensuring robustness against misleading priors while retaining the exploratory behavior of standard BO. DynMeanBO is broadly compatible with acquisition functions, introduces negligible computational cost, and comes with convergence guarantees under Expected Improvement and Upper Confidence Bound. Experiments on synthetic benchmarks and hyperparameter optimization tasks show that DynMeanBO accelerates convergence with informative priors and remains robust under biased ones.

Published as a conference paper at ICLR 2026
INCORPORATING EXPERT PRIORS INTO BAYESIAN OP- TIMIZATION VIA DYNAMIC MEAN DECAY
Chongqi Qu' Meiqin Liu 2,, Jian Lan", Shanling Dong", Zhunga Liu'
National Key Laboratory of
Hybrid Augmented Intelligence,
Xi' Xi'an Jiaotong University, Xi' an 710049, China "College of Electrical Engineering, Zhejiang University,
Hangzhou 3 310027, China
"School of Electronics and Information Engineering,
Xi' an Jiaotong University. an 710049, China
"School of Automation, Northwester Polytechnical University, Xi' an 710072, China
BaChongqi@stin xjtu. edu. en, {liumeigin, shan ihnnngggdgggggg] edu. janjianemail ..w....uuuucc..........mo liuzhunga@nxpu.edi edu. en
ABSTRACT
Bayesian optimization (BO) is powerful approach for black-box optimization, and in many real-world problems, domain experts possess valuable prior knowl- edge about promising regions of the search space. However, existing prior- informed BO methods are often overly complex, tied to specific acquisition func- tions, or highly sensitive inaccurate priors. We propose DynMeanBO, sim- ple and general framework that incorporates expert priors into the Gaussian pro- cess mean function with a dynamic decay mechanism. This design allows BO to exploit expert knowledge in the early stages while gradually reverting to stan- dard BO behavior, ensuring robustness against misleading priors while retaining the exploratory behavior of standard BO. DynMeanBO is broadly compatible with acquisition functions, introduces negligible computational cost, and comes with convergence guarantees under Expected Improvement and Upper Confidence Bound. Experiments on synthetic benchmarks and hyperparameter optimization tasks show that DynMean accelerates convergence with informative priors and remains robust under biased ones.
INTRODUCTION
Black-box optimization aims to optimize objective functions that are expensive to evaluate and lack analytical expressions or gradient information. Among various approaches, Bayesian optimization (BO) (Jones et al., 1998; Shahriari et al., 2015; Frazier, 2018; Garnett, 2023) has emerged as powerful and sample-efficient framework by constructing probabilistic surrogate model of the objective function and selecting promising candidates via acquisition function. BO has found widespread applications hyperparameter optimization (HPO) (Bergstra et al., 2011; Snoek al., 2012), automated materials discovery (Li et al., 2017; Zhang al., 2020), and robotics control optimization (Antonova et al., 2017; Calandra et al., 2016).
While BO has achieved remarkable success, fully automated optimization remains challenging in many real-world scenarios. In domains where evaluations are costly, practitioners often rely on prior knowledge guide the search. For example, in HPO, experts frequently heuristic rules or accumulated experience to identify promising hyperparameter regions rather than applying BO from scratch (Bouthillier& & Varoquaux, 2020; Smith et al., 2018: He et al., 2019; Li al., 2024a; Marek et al., 2025). Incorporating expert priors into BO can accelerate optimization by complementing BO's efficiency with expert insights. However, prior-informer BO methods (Ramachandran al.,
"Corresponding author.

---

Published as a conference paper at ICLR 2026
2020: Souza al., 2021; AV et al., 2022; Hvarfner al., 2022; Huang et al., 2023; Hvarfner al., 2024) are often complex or difficult to generalize across acquisition functions.
In this paper, we propose Dynamic Mean Bayesian Optimization (DynMe a framework that incorporates expert priors into the Gaussian process(GP) mean function and gradually decays their influence more data is collected. This design allows the optimizer to exploit prior knowledge in the early stages of exploration while mitigating potential bias from inaccurate priors in the long run.
Our main contributions
We introduce DynMeanBO, BO framework that integrates expert priors via a dynamically decaying mean function.
The method lightweight, broadly compatible with acquisition functions, and adds negli- gible computational overhead.
We provide convergence guarantees for DynMeanBO under Expected Improvement (ET) and Upper Confidence Bound (UCB).
Experiments synthetic functions and HPO tasks show that DynMeanBO accelerates convergence with accurate expert priors, remains robust under biased expert priors, and consistently outperforms other prior-informed approaches.
BACKGROUND
Notations. Scalars are denoted by lowercase letters (c.g. f), vectors by bold lowercase letters (e.g., x), and matrices by bold uppercase letters (e.g.. K). The search space is X C g and the observed dataset of n points is D,, yX.y]. We denote the GP posterior mean and variance as μt, (x) and s (x), respectively, and the observation noise variance as o². The global optimum location and its value are denoted by x" and f(x"), expert priors over the optimum location are denoted as π (xx), acquisition functions are denoted as ax (xx) and B[.) denotes expectation.
2.1 BAYESIAN OPTIMIZATION
BO is framework for optimizing expensive black-box functions by sequentially selecting evalua- tion points. Given an unknown objective function f. the goal is to find its global maximizer:
arg max f(x). xEX
At iteration n. a point x,, is evaluated with noisy observation yn = f(x,) + En. En ~ ((0
Conditioned on the observed dataset D., = 1 (xn, ya), probabilistic surrogate model defines the posterior p(f D..). We adopt GP as the surrogate model, which naturally provides posterior distribution for the objective function; alternatives such as random forests (Hutter et al., 2011) or Bayesian neural networks (Springenberg al., 2016; Li et al., 2024b) can also be used. The next evaluation point is selected by maximizing an acquisition function axx), which balances exploration and exploitation.
2.2 GAUSSIAN PROCESS
A GP (Williams & Rasmussen, 2006) places a distribution over functions, enabling Bayesian non- parametric regression with principled uncertainty estimation. is fully specified by a mean func- tion m(x) (m X + I)) and a covariance (kernel) function k(x, x') (k: X x X + R). The mean function m(x) can take any form, though is often set to zero for simplicity in stan- dard BO. The kernel function k(x,x') encodes correlations between any two inputs, with com- mon choices including the squared exponential (SE) and Matérn kernels (Frazier, 2018). The unknown objective f(x) is modeled as a GP prior, f(x) GP(m((), x(x, Given dataset Dn = (X,y), where X = [x1,..,,,,, x.] and = yy....,ya the posterior also a GP: p(f D.) (P(u)(x)) (x, x')), with
o'1j-1(y (y m), kn kx(xx)) x') = k(x,x' x') kn (x) [K, + o - (x'),
(x) = m(x) + k., (x) T Kn + o -1
2

---

Published as a conference paper at ICLR 2026
where K,, is the n kernel matrix with entries [Knly = xxx, fori, {l....,m) n. kx) [m(x1), m(x))] and I is the xx identity matrix. The posterior variance is (x) = kn (x,
[k(x,,x), ((x,; x)]
Kernel hyperparameters 0 are typically learned by maximizing the marginal likelihood. For GP with mean function m, the marginal likelihood is ply X, 0) = =W m, K,, + =o which leads to the log marginal likelihood
log p(y X, 0) = = (mmmmmm (K, to m) - 1/2 log + o log(2r). (3)
2.3 ACQUISITION FUNCTION
Acquisition functions (AFs) are utility functions that guide the selection of the next evaluation point in BO, trading off exploitation of high-value regions with exploration of uncertain regions (Wang et al., 2023). A wide variety of AFs have been proposed, each with distinct characteristics. improvement-basee methods include Probability of Improvement (PI) (Kushner, 1964), EI (Jones et al., 1998), and the Knowledge Gradient (KG) (Frazier et al., 2008), which aim to maximize the respective measures of improvement. Confidence-bound approaches include UCB (Srinivas et al., 2010); sampling-based methods include Thompson Sampling (TS) (Agrawal & Goyal, 2012); and information-theerrtiic strategies include Entropy Search (ES) (Hennig & Schuler, 2012), Predic- tive Entropy Search (PES) (Hernández-Looel et al., 2014), and Max-value Entropy Search (MES) (Wang& & Jegelka, 2017). Among these, EI and UCB are most widely used in practice.
El maximizes the expected improvement over the current best observation f
Oer (x) [max (0, (x) F5]
[Mn(x) fn 8n
Pnn [nn(x)
(4)
(xm(x) [7) Φb
+ sn (x)
5x(x
8n 8n(x)
where (-) and o6-) denote the cumulative distribution function and probability density function of the standard normal distribution, respectively.
UCB selects the next observation point based on the upper confidence bound of the predictive dis- tribution. It balances exploration and exploitation as:
QUCB(X) un-1 / (x))
(5)
where 8n, > 0 a parameter controlling the exploration trade-off.
2.4
EXPERT PRIOR
In many domains, experts often possess prior knowledge about the likely location of the optimum x before evaluating new task or model (Perrone et al., 2019). Such knowledge can be formalized as probability distribution over the optimum location:
(x) =
arg max f(x') X'EX'
(6)
which encodes the likelihood that different inputs correspond the global maximizer. In principle, the expert prior distribution (x) can take any form. In practice, the most commonly used distribu- tions are Gaussian distributions, representing single promising region, mixtures of Gaussians, which can capture multiple promising regions in the search space. Figure 6 in Appendix A il- lustrates several examples of expert priors in one dimension, and the concept naturally extends to higher-dimensional search spaces.
Preference-basel expert priors can also constructed using a similar approach: when experts pro- vide relative pairwise preferences over candidate inputs, these preferences can be converted into a probability distribution over the optimum location. A mixture of Gaussians often a convenient choice in this setting, it can flexibly represent multiple favored regions implied by the expert preferences. In fact, expert priors can be easily converted into a probabilistic form, and this form not limited the Gaussian Gaussian mixture examples used in the paper. Any probability distribution that can adequately express the expert's 's belief is valid. Further details can be found in Appendix A
3

---

Published as a conference paper at ICLR 2026
3 RELATED WORK
Incorporating expert prior knowledge into BO has significant practical and theoretical value. Al- though the related literature remains limited, several representative studies have explored this direc- tion. Nguyen & Osborne (2020) proposed Bayesian framework for the scenario where the opti- mal function value l known, but its optimal location remains unknown. This approach works well when experts can precisely provide the optimal value; however, in most real-world tasks, such knowledge is rarely available, limiting its applicability. Huang et al. (2023) introduced the Prefer- ence Bayesian Neural Network (PBNN), which leverages Siamese neural network to incorporate expert-proe expert-provided provided preference feedback, effectively accelerating the BO process. Similarly, AV et al. (2022) proposed human-in- in-an---ttt---oo-- BO framework, where human experts can directly intervene in the point selection process to improve search efficiency and performance.
The above methods incorporate expert priors in terms of the optimal value f(x"), preference infor- mation, or feedback mechanisms. However, a more common and practical setting involves expert priors the location of the optimum x'. BOPro (Souza et al., 2021) integrated expert priors into the BO-TPE framework (Bergstra et al., 2011), where experts design "good" and "bad" priors over x", which are then combined within the BO-TPE structure to guide the optimization. However, this method cannot be applied to other more general BO approaches. Ramachandran et al. (2020) proposed a novel framework that directly embeds the cumulative distribution of expert priors over x" into the kernel function. While interesting, this method is highly sensitive to inaccurate pri- ors since they affect the entire kernel, potentially degrading performance significantly. Li et al. (2020) constructed conditional posterior distribution incorporating expert priors over x", defined as p(x* Dn. π). x p(x" Dn,) (xx),, and determined the next evaluation point by repeatedly sampling from this distribution. However, this approach can only be used with specific sampling strategies and cannot integrate with general acquisition functions, which restricts its flexibility. BBO (Hvarfner al., 2022) integrates expert prior distributions into the acquisition function through weighting mechanism, where the influence of the prior gradually decreases as more evaluations are performed. Although BBO is simple and effective and provides convergence guarantees for the acquisition function, it does not explicitly model the prior in the surrogate model and remains essen- tially heuristic. More recently, ColaBO (Hvarfner et al., 2024) highly flexible framework that injects expert priors as additional priors over the surrogate model, orthogonal to the traditional priors on kernel hyperparameters. While compatible with Monte Carlo (MC)-based acquisition functions, aBO cannot be used with non-MC acquisition functions and incurs substantial computational costs.
Despite these advances, existing methods still face several limitations: (1) some approaches are overly complex and difficult to implement in practice; (2) many rely on specific acquisition func- tions or sampling strategies, limiting their general upplicability; and (3) several methods are highly sensitive to the quality of expert priors, which reduces robustness. Motivated by these challenges, propose DynMeanBo knowledge into the surrogate model by embedding it in the mean function of the GP. Unlike existing heuristic-based approaches, DynMeanBO achieves a principled integration of expert knowledge the model level, is compatible with arbitrary acquisition functions, and demonstrates strong empiri- cal performance across diverse benchmarks.
simple yet effective framework that directly incorporates expert prior
4 METHODOLOGY
We now present the proposed DynMeanBO framework. Unlike existing approaches that either em- bed priors heuristically or require specific acquisition functions, our method incorporates expert prior distributions on the location of the optimum by embedding them directly into the GP mean function. Although the design of DynMe in this paper is implemented using a GP surrogate model, the method itself is not tied to any specific surrogate. It can equally be combined with other types of models, such as random forests (Hutter et al., 2011) or Bayesian neural networks (Springen- berg et al., 2016; Li et al., 2024b). In Section 4.1, construct GP mean function based on expert priors. Section 4.2 details the overall algorithm framework, and Section 4.3 provides a theoretical analysis.
4

---

Published as a conference paper at ICLR 2026
4.1 EXPERT- PRIOR- BASED MEAN FUNCTION
In BO, we adopt a GP surrogate model, f(x) ~ (P(m(xx,k(x x')), where k(x, x') (x)( [(x)) denotes the covariance and m(x) [x](x)] encodes prior beliefs. In practice, m(x) is often set to zero when the functional form of ʃ is unknown. When partial knowledge is available, for example if f is approximately linear, a parametric mean of the form m(x) = ax + b can be employed. The hyperparameters a and b are estimated from Dn via maximum likelihood, in the same way as kernel hyperparameters.
In most woaldworld problems, even domain experts rarely know the explicit form of f(x). Instead, they may provide prior distribution π (x) over the likely location of the optimum x". We incorpo- rate this expert knowledge into the GP mean function as
Mprior (x) = A T(x + B,
(7)
where A > 0 is a parameter controlling the scaling of π (x), and B is another parameter introducing an additive shift to AT(x). Further details on the interpretation of A and B, as well as their sensitiv- ity analysis, can be found in Appendix B. Before any observations, Mprior (x) reflects the shape of the expert prior, providing a coarse estimate of the function landscape consistent with prior beliefs about x"
Figure/ illustrates an example where dimensionii Gaussian prior x(x) is incorporated into the GP mean function. Using the mean function mprrio (x), which encodes expert prior knowledge, has a pronounced effect on both the prior and posterior distributions of f. Samples drawn from the prior Pprior (f) and posterior Pprior Dn,) show clear peak in the region deemed "good" by the expert (green area in Figure 1), indicating that the optimal values are highly likely to fall within this region. This confirms that the expert prior knowledge has been effectively integrated into the BO framework. For an example with a one-dimensional mixture of Gaussians as the expert prior, see Appendix C.
Moments of prior
Moments of posterior
3.5
3.5
Posl erior maments Pel(G.)
25
coments Pu -6
2.5
Post moments Pptalf
1.5
Good under
Expert prionati)
4
0.5
-0.5
0.5
-15
-2.5%
0.0
Samples from prior
Samples from posterior
f- pul0
Observations
2.5
2.5
f- Py01.)
Oyor)))))
-05
-0.5
15-
1.5
0.0
0.2 0.4 6.6
0.8
1.0 0.0
0.2
0.4
0.6 0.
1.
Figure (Top left) Mean function and 95% confidence interval without any observations. denotes the GP prior with m(x) = O, i.e., p(f) GP k(x, x')), while Pprior (f) (P(mprior (x), 1i( 1((x, x')). (Top right) Mean function and 95% confidence interval conditioned on observed data D,,, p(f D,) uses m(x) O, whereas Pprior (f (D,) uses (x)ior (Bottom left) Samples drawn from the prior distributions p(f) and Pprior without observations. (Bottom right) Samples drawn from the posterior distributions p(f) (..) and Pprior (f (D,)) given D,,.
p(f)
4.2 BAYESIAN OPTIMIZATION WITH A DYNAMICALLY DECAYING MEAN FUNCTION
As more points are evaluated in BO, we gain increasing information about the objective function f. enabling more accurate model of Thus, the reliance the expert prior should progressively decline, which not only reflects the growing confidence in observed data but also preserves robust- ness in cases where the expert prior is far from the true optimum. Inspired by mBO (Hvarfner al., 2022), we incorporate a decay mechanism into the mean function, defining the mean function at iteration n after initialization as
mn m,(x) (x) 7/n Mprior (x) (17))) -o(x), In = =xxp( exp(-X(1xx No));
(8)
5

---

Published as a conference paper at ICLR 2026
where À 0 controls the decay rate and No is the number of initial evaluations. A sensitivity analysis of À is provided in Appendix J. The baseline mean function Ho(x) corresponds to the mean function used in the standard BO setting, which is typically chosen commonly the zero mean function. By gradually decaying the influence of the expert prior mean toward the baseline mean, the method remains robust even when the expert prior is substantially misaligned with the true optimum.
constant function- most
The complete procedure for incorporating expert prior knowledge into the BO framework is summa- rized in Algorithm 1. During initialization, we sample a portion of the initial points from the expert prior distribution π (xx), while the remaining points are drawn using Sobol sequences to ensure uni- form coverage of the search space X. This hybrid initialization strategy facilitates building a more accurate surrogate model and improves the subsequent optimization performance. Further analysis regarding the choice of the initialization ratio p for sampling from (xx) is presented in Appendix K.
Algorithm DynMeanBO: Bayesian Optimization with Dynamic Mean Decay
Require: Search space X, kernel function k(x, x'), expert prior distribution & (xx), decay coefficient X, initial design size No. initialization ratio p. max iterations N, the parameters A and B
Ensure: Optimized design x*
1: Initialization:
2: Draw (
pNNol samples from axx)
3: Draw Ny Ar(Sohol) = No (prior) samples via Sobol sequences
4: Set No (xitim
No
(etrioo))
{
No (Sbobl))
5: Observe Yi (x;; + cs. and set DN, = {(x; 3/) } No
6: Initialize GP prior mean Merror (x) A 17 (x)) +
7: Fit GP posterior (No) GP (x), kNo (x, xx) according Eq. (2)
8: for n = No + I,...,,, do
9:
X, arg maXxEx axx, D 1
10:
yn = -f(x)) + En
11:
D,, D U {(x,, vn))
12:
Update mean function: m. (x) according to Eq.(8)
13:
Update GP posterior: p(f(D.)) GP (x), Kn (x, x') according to Eq. (2)
14:
end for
return x arg w.) Vi
4.3 THEORETICAL ANALYSIS
We provide a theoretical analysis the convergence properties of DynMearBO, establishing guar- antees for the commonly used E. and UCB acquisition functions. Complete proofs are presented in Appendix D and Appendix The analysis can be straightforwardly extended to other standard acquisition functions, following similar arguments; we omit these extensions for brevity.
Convergence under EI. To analyze the convergence of DynMeanBO under EI, we adopt the as- sumptions of Bull (2011). Although our focus is on maximizing the objective function, the theoreti- cal framework of Bull (2011) assumes minimization. This distinction is immaterial, as maximization can be equivalently reformulated as minimization by considering fxx). Let H2 denote the repro- ducing kernel Hilbert space (RKHS) associated with a symmetric positive-definite kernel k. In our analysis, employ the Matérn kernel (Matérn, 1960), where the smoothness of functions in H4 is controlled by the parameter v. We assume that the unknown objective function f lies within a ball BR in H4, i.c. xx))) I R.
We define the loss as
(u, Dn. He)()), R)
A sup [y([(x))) min f. II Is SR
where u denotes the strategy, and x", is the best point selected after evaluations. We denote the EI strategy under anBO as DynMeanBO-EH and the standard BO with EI strategy as BO-EI. Based the detailed proof in Appendix D, obtain the following theoretical result.

---

Published as a conference paper at ICLR 2026
Theorem (Convergence of DynMeanBO under EI). Let X' R be compact, f E (x and let DynMeanBO use the dynamic prior mean (x)x) = nn. mprior(X) + (1 (1-7)) 7m) (o(x) with Th = exp(-X(n -N)),, X > O. Then, DynMeanBO under EI achieves the same asymptotic convergence rate as standard BO under EI, namely
(vAl)/ L. Ln(DynMeanBO-EI) Dn, (x))), R) (nn (log)))))))))
where B 2 O is a constant depending on the kernel and v.
Convergence under UCB. We establish the convergence of DynMeanBO under the UCB strategy by following the proof techniques of Srinivas et al. (2010). Our objective is to maximize the tar- get function f. and at each iteration the next query point is chosen according to the UCB rule:
X2 arg X (X) arg maxxex fn-1 (x) + gl/2 sn-1(x). The resulting convergence guarantee is summarized below, with the complete proof provided Appendix E.
Theorem 2 (Convergence of DynMeanBO under UCB). Let ā E 1). Assume that the true un- derlying function f lies in the RKHS Hy associated with the kernel k, with llf S B, and let 8n 2B + 300G,, log"(v (n/0). Assume further that the observational noise is o-sub-Gaussian Let m, (x) = "nn prior (x) + (1 "n)Po(x) ) be the dynamic prior meun of DynMean nBO at iteration n, where "n + O. When using the UCB acquisition function with parameters Bn, the cumulative regret of DDnMMeannB satisfies, with probability at least 1 8,
Pr PI(RN RN S SCINNBNGN + C2 E Ta 1N
whereRy == E (f(x*) - fxxx)) denotes the cumulative regret, GN is the maximum informa- tion gain up to N, and C1, C2 > 0 are constants independent of N. In particular, if E Inn < co, DynMeanBO-UC3 achieves the same asymptotic convergence rate as BO-UCB:
Ry = Ol NVNGGGN)
EXPERIMENTS
We systematically evaluate the performance of DynMeanBO on diverse tasks under both "good" (informative) and "bad" (misleading) expert priors. The experiments examine its compatibility with different acquisition functions and demonstrate its advantages over other prior-informed BO meth- ods. Section 5.1 describes the experimental setup. Section 5.2 presents the compatibility results, while Section 5.3 provides the comparative analysis. Our implementation is publicly available at https: /ttp//gtttb///t com/ hwtp//gggo/o/Daeee......
5.1 EXPERIMENTAL SETUP
Expert Priors. We adopt the expert prior settings used in TBC (Hvarfner et al., 2022) and ColaBO (Hvarfner et al., 2024), modeling both "good" and "bad" expert priors Gaussian dis- tributions. The mean of the "good" prior is located 10% away from the location of the global optimum, with variance set to 20% of the search space width. For the "bad" prior, the mean is shifted 70% away from the location of the global optimum; if this position lies outside the domain, is clipped to the boundary, with the same variance setting. Detailed configurations are provided in Appendix F. To further investigate how different expert prior configurations influence the perfor- mance of DynMeanBO, we additionally study the effect varying the prior variance under three prior-quality conditions: strong prior, weak prior, and wrong prior. The definitions and constructions of these prior types are provided in Appendix L, where we also present the corresponding detailed settings and analyses.
Tasks. We consider two types of tasks: synthetic functions and HPO benchmarks. The synthetic functions span 4D to 7D search spaces, including Hartmann (4D), Levy (SD), Hartmann (6D), Rosenbrock (6D), and Stybtang (7D) (Wang et al., 2020), all implemented BoTorch' (Balan- dat et al., 2020). For HPO, evaluate three 4D deep learning optimization problems from the
'tttps:////t www..h/.o/gttttce/////ee////

---

Published as a conference paper at ICLR 2026
PDI benchmark suite (WMT, CIFAR, and LMIB). Although their true optima unknown, leverage expert priors from MF-Prior-Benchee (Mallik et al., 2023). Moreover, we also examine higher-dimensional settings. In Appendix I, report additional results two 20-dimensional syntheric tasks - Levy (20D), Rosenbrock (20D).
Comparison Algorithms. To test the compatibility of DynMeanBO with different acquisition strategies, we evaluate it under seven widely used acquisition functions: PI, EI, LogEI, TS, UCB, KG, and MES. We compare the performance of standard BO and DynMeanBO under each acquisi- tion function. Additionally, we benchmark DynMeanBO in comparison with the prior-informed BO methods, including TBO and ColaBO.
Computational Platform. All experiments are conducted on a dual-socket Intel Xeon Platinum 8575C server (2x48 cores, 192 threads, 2.80 GHz base 4.00 GHz turbo, 4 NUMA nodes).
5.2 DYNMEANBO'S GENERALITY ACROSS ACQUISITION FUNCTIONS
To evaluate the generality of DynMeanBO, we test it with seven widely used acquisition strategies: PI, EI, LogEI, TS, UCB, KG, and MES. Since DynMeanBO incorporates expert priors directly into the GP mean function, can be seamlessly combined with any acquisition function without requiring algorithmic modifications. We compare DynMeanBO against standard BO under all seven acquisition functions, and additionally include random sampling baseline (Sampl ing) as a refer- ence. For this experiment, "good"" expert prior is used to construct the DynMean model.
F1
LC Dyt BG UCB
Legfi Uyreoar 80 Luge!
KG DyiMcureO- KG)
HES Uyeiccre0 Mes
Samsling
Dy: Mlun 80
SysPonn50 bJ
Dyt MoMarr
hartmann4
levy
hartmann6
rosenbrocké
Bast-so--ar value
D-
3030
10
2000
1000
Iteration
80
110
0
25
52
100
25
50
73
102
stybtang?
pd Im1b
pdl wmt
pdl cifar
150
200
0.6
0.5
0.7
222
250
300
Figure 2: Performance on synthetic functions and HPO tasks. When "good" expert prior is incor- porated, DynMeanBO consistently finds better solutions faster than the standard BO across different acquisition functions.
The results Figure show that across all acquisition functions, DynMeanBO consistently acceler- ates convergence relative to standard BO. These improvements demonstrate its strong compatibility and effectiveness across a diverse set of strategies. In particular, during the early stages of opti- mization, DynMeanBO can leverage the expert prior to quickly identify better solutions, far more efficiently than standard BO. In practice, the tasks optimized with BO are often very expensive and time- monnsminn,,ng, only very limited number of evaluations can typically be performed. By in- able to achieve substantial gains during these
corporating expert prior knowledge, DynMeanBi early stages of optimization, demonstrating even greater advantages in real-world scenarios.
We also compare the per-iteration evaluation time of DynMeanBO and standard BO, as shown in Figure 11 in Appendix G. The results indicate that the computational overhead of DynMeanBO is negligible. When provided with informative expert priors, DynMeanBO achieves faster convergence without sacrificing computational efficiency.
hhtpps://gi httpgttttt......oooooooooooo / http://tttt/mmm---rrr----be
8

---

Published as a conference paper at ICLR 2026
5.3 COMPARATIVE STUDY DYNMEANBO AND EXISTING PRIOR-INFORMED BO FRAMEWORKS
We now compare D DynMeanBO with other prior-informed BO methods. Under "good" expert prior, DynMeanBO achieves performance comparable to #BC and ColaBO, while requiring lower com- putational cost. Under "bad" expert prior, DynMeanBO exhibits strong robustness, maintaining stable performance even in the presence of misleading prior information.
"Good" expert prior. TBO and ColaBO are representative approaches that incorporate expert pri- ors into BO. While TBO employs EI, Co aBO utilizes LogEI and MES, denoted as MCpi LogEI and MCpi-MES in the figures. For a fair comparison, we evaluate DynMeanBO using the same acquisition functions- EI, LOgEI, and MES. As shown in Figure 3, DynMeanBO achieves per- formance on par with TBO and ColaBO, while consistently accelerating the optimization process across all benchmarks.
EI
LogEl
MES
MCpi-LogEI MCpi-MES
PIBO
-E
DynMeanBO-LoggE
DynMeanBO-MES
Sampling
hartmannd
levy5
hartmanné
rosenbrockd
Best-op-far valuee
5000
10
2000
60
Iteration in
40
65
30
100
0
25
35
15
100
23
50
73
105
stybtang7
Im1b
cifar
0.3
0.0
0.8
200
0.7
-100
0.4
0.2
50
205
100
Figure 3: Performance synthetic functions and HPO tasks under "good" expert prior. DynMeanBO, TBO, and ColaBO achieve comparable results.
As discussed in Section 5.2, DynMeanBO introduces negligible computational overhead com- pared to standard BO. Furthermore, when compared with other prior-informed BO frameworks, DynMeanBO achieves substantially better computational efficiency. Figure 4 reports per-iteration evaluation time under identical acquisition functions, where DynMeanBO is markedly faster than both BBC and ColaBO.
DynMeanBO-EI
PIBO
DynMeanBO-LogE
MCpi-LogEI
DymMeanBO MES
MCpi-MES
hartmann4
levyS
hartmann6
rosenbrocks
150 (TT
stybtang7
pdl Im1b
pdl wmt
pdl cifar
3
3.0
Figure 4: Per-iteration evaluation time (logıo scale) of DynMeanBO, xBO, and ColaBO on syn- thetic functions and HPO tasks under the "good" expert prior setting.
"Bad" expert prior. In practice, domain experts may provide priors that deviate substantially from the true optimum. The setup for the "bad" prior is detailed in Section 5.1. As shown in Fig. 5, DynMeanBO remains highly robust in this scenario, quickly approaching the performance of stan- dard BO even when guided by inaccurate prior. In terms of robustness, DynMeanBO clearly

---

Published as a conference paper at ICLR 2026
LogEr
MCpi-LogEI MCpi-MES
DynMeanBO-El
DynMean8O-LogEf
DynMeanBO-MES
Sampling
hartmanne
levy3
hartmanne
rosenbrockd
Best-sofar valuee
0-
6030
10
2030
Iteratiin
<tybtang7
pd Im1h
p41 wmt
pdl. cifar
100
0.6
-20
0.5
Figure 5: Performance on synthetic functions and HPO tasks under "bad" expert prior. DynMeanBO demonstrates strong robustness.
outperforms both Co and TBO, with particularly large margin over TBO. Interestingly, on the PDI (LMIB) task. all prior-informed methods (DynMeanBO, TBC, and ColaBO) converge faster than vanilla BO despite the misleading prior. This occurs because the "bad" prior, although far from the global optimum, still points a high-quality suboptimal region. A comparison of computational overhead under the "bad"" prior is provided in Appendix H.
CONCLUSION
We proposed DynMeanBO, BO framework that incorporates expert prior knowledge via dy- namically decaying mean function. Our approach is compatible with any acquisition function and introduces negligible computational overhead. Empirically, SynMeanBO accelerates convergence when expert priors are informative, while remaining robust when priors are inaccurate. These find- ings demonstrate the practical benefit of integrating expert knowledge into and show that dy- namically modulating the influence of the expert prior can effectively balance expert guidance with data-driven exploration.
For future work, plan to combine DynMeanBO with complementary techniques, such multi- fidelity optimization and parallel evaluation strategies. This combined approach aims to further accelerate the optimization process and enhance scalability, enabling applications to more complex models, diverse search spaces, and large-scale HPO tasks.
ACKNOWLEDGMENTS
This research was supported by the National Natural Science Foundation of China under Grants U23B2035 and U24B20178.
REFERENCES
Shipra Agrawal and Navin Goyal. Analysis of Thompson sampling for the multi-armed bandit problem. In 25th Annual Conference on Learning Theory (COLT), volume 23, pp. 39.1-39.26. JMLR: Workshop and Conference Proceedings, 2012.
Rika Antonova, Akshara Rai, and Christopher G Atkeson. Deep kernels for optimizing locomotion controllers. In Conference on Robot Learning, pp. 47-56. PMLR, 2017.
Arun Kumar AV, Santu Rana, Alistair Shilton, and Svetha Venkatesh. Human-Al collaborative Bayesian optimisation. Advances in Neural Information Processing Systems, 022166333224333 2022.
10

---

Published as a conference paper at ICLR 2026
Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wil- son, and Eytan Bakshy. Botorch: framework for efficient Monte- Monte-Carlo Bayesian optimization. Advances in Neural Information Processing Systems, 11221224-21535 2020.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for optimization. Advances in Neural Information Processing Systems, 24:2546-2554, 2011.
Xavier Bouthillier and Gaël Varoquaux. Survey of machine-learning experimental methods at NeurIPS2019 and ICLR2020. Technical report, Inria Saclay Ile de France, 2020.
Adam D Bull. Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research, 12(10), 2011.
Roberto Calandra, André Seyfarth, Jan Peters, and Mare Peter Deisenroth. Bayesian optimization for learning gaits under uncertainty: An experimental comparison on a dynamic bipedal walker. Annals of Mathematics and Artificial Intelligence, 76(1):5-23, 2016.
Peter Frazier. A tutorial on Bayesian optimization. arXiv preprint
2018.
Peter Frazier, Warren B Powell, and Savas Dayanik. A knowledge-gradient policy for sequential information collection. SIAM Journal on Control and Optimization, (7(5):2410-2439) 2008
Roman Garnett, Bayesian optimization. Cambridge University Press, 2023.
Fengxiang He, Tongliang Liu, and Dacheng Tao. Control batch size and learning rate to generalize well: Theoretical and empirical evidence. Advances in Neural Information Processing Systems, 32:1141-1150, 2019.
Philipp Hennig and Christian Schuler. Entropy search for informatiin-efficient global optimiza- tion. The Journal of Machine Learning Research, 13(1):1809-1837, 2012.
José M Liriindez--obbtoo Matthew W Hoffman, and Zoubin Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. Advances in Neural Information Processing Systems, 27:918 1--9268 2014.
Daolang Huang, Louis Filstroff, Petrus Mikkola, Runkai Zheng, Milica Todorovic, and Samuel Kaski. Augmenting Bayesian optimization with preference-based expert feedback. In ICML 2023 Workshop The Many Facets of Preference-Basee Learning, 2023.
Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In International Conference on Learning and Intelligent Optimization, pp. 507-523. Springer, 2011.
Carl Hvarfner, Danny Stoll, Artur Souza, Marius Lindauer, Frank Hutter, and Luigi Nardi. TBO: Ac- quisition functions with user beliefs for Bayesian optimization. In Tenth International Conference of Learning Representations, 2022.
Carl Hvarfner, Frank Hutter, and Luigi Nardi. A general framework for user-guided Bayesian opti- mization. In 12th International Conference on Learning Representations, 2024.
Donald R Jones, Matthias Schonlau, and William] Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455 492, 1998.
Harold J Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86(1):97-106, 1964.
Cheng Li, David Rubín de Celis Leal, Santu Rana, Sunil Gupta, Alessandra Sutti, Stewart Greenhill, Teo Slezak, Murray Height, and Svetha Venkatesh. Rapid Bayesian optimisation for synthesis of short polymer fiber materials. Scientific Reports, 7(1): 2017.
Cheng Li. Sunil Gupta, Santu Rana, Vu Nguyen, Antonio Robles-Kell Kelly, and Svetha Venkatesh. Incorporating expert prior knowledge into experimental design via posterior sampling. arXiv preprint arXiv:2002. //256 2020.
11

---

Published as a conference paper at ICLR 2026
Shuaipeng Li, Penghao Zhao, Hailin Zhang, Xingwu Sun, Hao Wu, Dian Jiao, Weiyan Wang, Chengjun Liu, Zheng Fang, Jinbao Xue, et al. Surge phenomenon in optimal learning rate and batch size scaling. Advances in Neural Information Processing Systems, 37:152722-132746, 2024a.
Yucen Lily Li, Tim GJ Rudner, and Andrew Gordon Wilson. A study of Bayesian neural network surrogates for Bayesian optimization. In 12th International Conference on Learning Representa- tions, 2024b.
Neeratyoy Mallik, Edward Bergman, Carl Hvarfner, Danny Stoll, Maciej Janowski, Marius Lin- dauer, Luigi Nardi, and Frank Hutter. Priorband: Practical hyperparameter optimization in the age of deep learning. Advances in Neural Information Processing Systems, 36.7377-7391, 2023.
Martin Marek, Sanae Lotfi, Aditya Somasundaram, Andrew Gordon Wilson, and Micah Goldblum. Small batch size training for language models: When vanilla SGD works, and why gradient accumulation is wasteful. arXiv preprint arXivv2507.07101, 2025.
Bertil Matérn. Spatial Variation, volume 49. Meddelanden från Statens Skogsffrsknniggsissituu, 1960.
Vu Nguyen and Michael A Osborn.. Knowing the what but not the where in Bayesian optimization. In International Conference on Machine Learning, pp. 7317-7326. PMLR, 2020.
Valerio Perrone, Huibin Shen, Matthias W Seeger, Cedric Archambeau, and Rodolphe Jenatton. Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning. Advances in Neural Information Processing Systems, 32:12751-127611 2019.
Anil Ramachandran, Sunil Gupta, Santu Rana, Cheng Li, and Svetha Venkatesh. Incorporating expert prior in Bayesian optimisation via space warping. Knowledge-Bal Based Systems, 195:105663,
2020.
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1): 148-175, 2015.
Samuel L Smith, Pieter-Jan Kindermans, Chris Ying, and Quoc Le. Don't decay the learning rate, increase the batch size. In International Conference on Learning Representations, 2018.
Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical Bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 15:2951-2959, 2012.
Artur Souza, Luigi Nardi, Leonardo B Oliveira, Kunle Olukotun, Marius Lindauer, and Frank Hutter. Bayesian optimization with a prior for the optimum. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 265-296 665-296.55 Springer, 2021.
Jost Tobias Springenberg, Aaron Klein, Stefan Falkner, and Frank Hutter. Bayesian optimization with robust Bayesian neural networks. Advances in Neural Information Processing Systems, 29: 4134 4142, 2016
Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process opti- mization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference Machine Learning (ICML), pp. 1015-1022, 2010.
Jialei Wang, Scott C Clark, Eric Liu, and Peter Frazier. Parallel Bayesian global optimization of 2020.
expensive functions. Operations Research, 68(6):
Xilu Wang, Yaochu Jin, Sebastian Schmitt, and Markus Olhofer. Recent advances in Bayesian optimization. ACM Computing Surveys, 55(13s):1 2023.
Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient Bayesian optimization. In International Conference on Machine Learning, pp. 3627-3635. PMLR, 2017.
Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning,
volume 2. MIT press Cambridge, MA, 2006.
Yichi Zhang, Daniel W Apley, and Wei Chen. Bayesian optimization for materials design with mixed quantitative and qualitative variables. Scientific Reports, 10(1):4924, 2020.
12

---

Published as a conference paper at ICLR 2026
A DETAILED EXPLANATION OF EXPERT PRIORS
To complement the discussion in Section 2.4, we provide visual illustrations of several represen- tative expert prior distributions. These examples highlight how different choices encode domain knowledge about the likely location of the global optimum.
As shown Figure 6, a unimodal Gaussian prior emphasizes a single promising region, while Gaussian mixture prior flexibly represents multiple candidate regions. Preference-bases priors can also be approximated using mixtures of Gaussians, which capture favored regions implied by expert preferences. Although display one-dimensional cases for clarity, the same idea naturally extends to higher-dimensione spaces.
Normal Distribution
Mixture of Gaussians
Arbirary Distribation
Figure 6: Illustration of different types of expert prior distributions.
Expert priors can naturally be expressed in the form of probability distributions. They are not limited Gaussian or Gaussian mixture distributions; any distribution that captures the expert's belief can be used. For example, consider arbitrary function l(x), whose optimum reflects the expert's belief about where the optimal solution of the current task lies. We can normalize this function to obtain expert prior distribution:
h(x) [(x dx
(x)
This allows us to incorporate expert knowledge into the optimization process through a probabilistic formulation.
B INTERPRETATION OF PARAMETERS A AND B
We define the prior mean function Mprrio (x) A T(x) + B, where Txx) denotes the expert prior distribution. This construction leverages expert knowledge to roughly shape the mean function, thereby highlighting the region where the function f is most likely to achieve its maximum.
Single Gaussian expert prior
Mixture Gaussian expert prior
3.0
nóc)
3.0
interval where may
-4
Figure 7: Illustration of the prior mean construction.
In this setup, serves as a scaling factor to align (xx) with the magnitude of f. while acts as shift that positions Ar appropriately (e.g., around the mid-level of f). As illustrated in Figure 7,
13

---

Published as a conference paper at ICLR 2026
the two subplots show how A and B influence the mean function when T(x) is chosen Gaussian distribution and as a Gaussian mixture distribution, respectively.
The selection of A and B is flexible. They be manually set, for example A 1 and B 0. Alternatvell,, they can be derived from the initial evaluations. Suppose the initial set of evaluated points, the maximum and minimum observed values Umax and Ymin- Then may set
Vmax
Vmin
Vmax + Ymni
A
B
(10)
maxxEx =xx)
B.1 SENSITIVITY ANALYSIS OF PARAMETERS A
In constructing our mean function, Mprior A (4)(()++ B, the parameters A and B are primarily used to better approximate the objective function. However, our main concern is the shape (x) of the mean function, as it is the key factor governing the balance between exploitation and exploration. In principle, the specific values of and B should not have major impact on the optimization process. To investigate the sensitivity of the algorithm to these parameters, we conduct a sensitivity analysis for both A and B.
In this section, we focus the sensitivity with respect to A. We consider A = 0.1, 0.5, 1,2, 10 while fixing O, and study how different values of affect the optimization process. We use CymMeanBO-Ei as the test case, i.e., DynMeamBO with the El acquisition function, to evaluate the sensitivity to A. The experimental results are shown in Figure 8
hartmannd
levy5
hartmann6
resenbracké
20
3010
15
Bast-aoffa
2.5
10:00
Iteration
stybtang77
pd], pdl_Imbb Im1b
pdl. wmt
pdl_citar
-100
3.0
0.5
110
0.0
0.0
0.7-
0.4
230
2.4
0.6
0.2
30
Figure 8: Ablation study of the parameter under the "good" expert prior setting
From the experimental results, we can see that the optimization process is largely insensitive to different values of A. In other words, the specific choice of not critical; what matters most the shape of the mean function.
B.2 SENSITIVITY ANALYSIS OF PARAMETERS B
In this section, we analyze the sensitivity of the parameter Fixing A 1, we examine how dif- ferent values of B = 0.0. 0.2, 0.4, 0.6, 0.8 affect the optimization process within DynMeanBCC BBO-E.. The results are presented in Figure 9. From the experimental observations, the optimization process is highly insensitive to the choice of B, further confirming that the most critical factor is the shape of the mean function rather than the specific values of its scaling or shifting parameters.
C MIXTURE OF GAUSSIAN AS EXPERT PRIOR
Figure 10 illustrates example where oie-dimensional mixture of Gaussians is used the expert prior (xx and incorporated into the GP mean function. The mean function Mprior (x) now reflects multiple regions that the expert considers promising.
Samples drawn from the prior Pprior (ff and the posterior Pprior (f D) show pronounced peaks in these regions (highlighted in green in Figure 10), indicating that the optimum is likely to lie within
14

---

Published as a conference paper at ICLR 2026
hartmannd
lwvs
hartmanne
resenbracké
3000
-1-
0
20
40
65
22
200
stybtanng?
pd]
_dl_cifar
3.0-
0.0
-150
0.0-
0.0
0.0
0.7
0,4-
250
0.4
0.2
50
300
30
20
30
45
30 20
30
30
25
30
40
Figure 9: Ablation study of the parameter B under the "good" expert prior setting
one or more these areas. This demonstrates that, even when the expert prior takes the form of a Gaussian mixture model,
still effectively encode multiple promising regions and significantly shape both the prior and posterior distributions of the objective function f within the BO framework.
Moments of posterior
Moments of prior
3.5
Guod region and
Piir omenns ol paló
Posterior momeets Px(1O.)
2.5
Expert privinnta
oments
Pare(1Da)
1.5
0.5
-0.5
-1.5
-1.5
2.5
0.0
62
1.0
0.0
0,2
0,4
0.6
10
Samples from prior
Samples from posterior
3.5
3.5
Obiervatios f 0y(19))
f Psiet
1.5
1.5
0.5
0.5
-2.
-2.5
Figure 10: (Top left) Mean function and 95% confidence interval without any observations. p(f) denotes the GP prior with m(x) O, i.c., p(f) GP (mm(xx) ((xxx')), while Pprior (f) (p(mprr (x), l(x, x))). (Top right) Mean function and 95% confidence interval conditioned on observed data D,, p(f D,,) uses m(x) O, whereas Pprior DD,) uses mprior(x), (Bottom left) Samples drawn from the prior distributions p(f) and Pprior (f) without observations. (Bottom right)
Samples drawn from the posterior distributions p(f (Dn) and Pprior(J (f (Dx) given Dn..
D PROOF OF THEOREM
Theorem (Convergence of DynMeanBO under EI). Let X C Id be compact, f E Hx(X), and let DynMeanBO use the dynamic prior mean mx = "n- mpriio (x) + (1
(1-7m) 7m) Mo(x) with Th = exp(->(n -N)), X > O. Then, DynMeanBO under EI achieves the same asymptotic convergence rate standard BO under EI, namely
Lu (DynMMennBOOEE. Dn, Hu(M), R) = Onn (vAd)///
where B 2 0 is constant depending the kernel and v.
Proof. The primary difference between DynMeanBO and standard BO lies in their mean functions. To analyze the convergence of DynMeanBO under EI, we first characterize the deviation of the mean function across iterations, which then allows us to study the convergence of the entire algo- rithm. To this end, we define the perturbation induced by the dynamic prior mean as
5,, (x) m, (x) xx(x) 7/ (Mprior (x). po(x));
15

---

Published as a conference paper at ICLR 2026
which captures the difference between the dynamic mean and the standard BO mean iteration n. Since both mpiior and Po are bounded, there exists M >> 0 such that supxex (x)) $ Mys
which vanishes +
The GP posterior mean is linear the prior according to Eq. 2 implying
sup |Mprior, - An (x) = sup xEd'
|m, (u(x)) + (x) [Kn + o'1)-1 - mn)l
xeX
sup (x) + kx) KK, + 0 (X)]
XEX
(sup 18 (x)))
xeX
=(mm),
where Mprior,n and jo denote the posterior means with dynamic prior m, (x) base prior pi (x), respectively. And μo = [xo((x)).... (x..)))) m, = [m,,(x)),... m., (x.))
According to the definition of EI in Equation 4. if we fix the posterior variance, then ( g(u(x)), where g(.) is a function determined by the probability density function the cumula- tive distribution function of the standard normal distribution. It follows that g differentiable and
Lipschitz continuous. Therefore, there exists constant >> 0 such that
sup (DDyMMeaBB--B (IBO-EI(X,)) n)] sup [gpuurir (x)) g(un (x)) xEX
xEx
< L sup [uprion, Un (x)l xeX
0(mx)
supxex (DDyaMeanB--E((x.) - Q-o-EI (x, n²] = O(%)). and let X,+1 arg maxxex OBO-BI x, n), x = arg maxxex ODynMeanBO-E n). Here, we make a reason- able assumption: there exist constants 0. 1, and > 0 such that, for the maximizer
We define cn
x" arg max CBO-E (x, n) xeX
and any point satisfying |z x"| S r, we have
OBO-EI(X ,n) - ORROFI(Z, n) 2 e c||x*= - -
Since Xn+1 x", we have
c| Xa+1 - prior || S (Bo.E((X. n) Cp n)
UBo-t (xn+1, n)
(X n) + + (0yMManBO-E((X441.1
aBB-Ei pire n1)
< (B0-E1(Xn+117) n.)
CDynMeaBBB-A((X44411 n) + CDDMMennB--E n))
ago--a (x:
< (IBO.EI(Xx130 ODyyMMemBO II (x + ODDymMeanBO. mx prior n)
-I (xpriee n)]
< 2en
So ||Xn+1 price
|| < (2mm)) 1/p under the RKHS radius constraint Rt all functions f are Lipschitz
c
continuous. Let Ls denote a uniform Lipschitz constant. Then, we have
[ f(x++1)] s Ly -1xx11111111 prior S Ly /
Taking the supremum both the expectation and the worst-case scenario, obtain the following bound on the loss difference:
2En
1/p
c, (DynMeenBB-FFI, D,, H4(x), R) C, (BO-EI, D., (x)).. R) S C.
16

---

Published as a conference paper at ICLR 2026
where the constant C depends L, and the RKHS norm R of the function.
Combining this with the known convergence rate of standard BO-EI (Bull, 2011):
Ln (BO-EI, Pa (x'),, R) (nn-(wa))/(( n))), where y is the smoothness parameter of the kernel & (Matér kernel,, and / is defined as
if v 1
0,
if
where is the logarithmic correction exponent determined by the covering number of the RKHS.
Since 7n = exp X(n - No)) decays exponentially, so
L. (DyynMccaBBO-EI) D,, Hx(x), C.. (BO-EI, D.. He R) + ( (2(m)
1/r
(2en)
nen
1/2
(C, D,,, Hz R)) + O(C
O(n-(v11) (logn))
Therefore, DynMeanBO achieves the same asymptotic rate of convergence standard BO.
E PROOF OF THEOREM 2
Theorem 2 (Convergence of DynMeanBO under UCB). Let ô E (0, Assume that the true un- derlying function f lies the RKHS H4 associated with the kernel k. with || S B, and let 3n = 2B + 300G, logo/o////0 (n/0). Assume further that the observational noise is o-sub-Gausiian. Let m, (x) = "n Vr.ror (x) + (1 (x)) be the dynamic prior mean of DynMeanBO iteration n, where + O. When using the UCB acquisition function with parameters Bn, the cumulative regret
of DynMean satisfies, with probability least 1 8,
N
Pr(RA BN RN S SCINBBGG + C2 E IN 2
Pr
where Ry E tion gain up to N, and C1, C2 0 constants independent of N. In particular, if E inn 00, DynMeanBO achieves the same asymptotic convergence rate BO-UCB:
denotes the cumulative regret, GN is the maximum informa-
RN O OVNBNGNN.
Proof. We follow the analysis of BO-UCB by Srinivas et al. (2010), adapting it to account for the dynamic prior mean. Similar the proof of Theorem 1, the key lies analyzing the deviation between the mean functions of DynMeanBO and standard BO.
Following the proof Theorem 1, the deviation between mean functions of DynMeanBO and standard BO is given by
A_ = sup |Aprion Mhm (xn(x)) -xx( -(mm) xEX
Therefore, there exists a constant Cmean > 0 such that A, < Cmean Tas which implies that the posterior mean perturbation vanishes as Yn + 0.
For baseline BO-UCB, Srinivas et al. (2010) show that, with probability at least 1 - 8,
If(x) (x)) /Buti (x)), Vxx X.
(See Lemma 5.1 in Srinivas et al. (2010) for details.)
Combining this with the bound on An yields a high biob-brobabilltyy confidence bound for DynMeanBO: [f(x Mprior, (x)l
[f(x) (x() + (x(x) Mprior, (x)) S [xx) - [x](x) + 1/ - Mprior, (x) v B (x)) + An. Vx X,
17

---

Published as a conference paper at ICLR 2026
Here, let ODDyMMeaBB-UUE denote the UCB acquisition function in DynMeanBO. According to Equation (5), we have aDynMeanBO-UCB (x, n) Mprior, (x) + v Bn+1 (xxx). Therefore, the func- tion value satisfies
f(x) BO n) + A..
Let X2 be the point chosen by DynMeanBO and x* = arg maxxeX fxx). Using the above confi- dence bound and the fact that x, maximizes 4DDyMeanBBO UCB (x, 1), the instantancous regret satisfies
n = -===(xx)--] (xx)
OnynMeennBO.UCC 1) An ((xn)
QDynNeanBO-UUC 1) + A 1 - f(xn)
Mprior,n 1 + v 3n 8n (xn) + An-1 8 (xx)
$ 1f(x)) - Mprior, (xn) + v Bn 3n- + n
2( Bn 8n (xn) + An-1)
< 2 3n $n-1 (x,) + 20meau in-1-
Summing over n to N gives the cumulative regret
N
N
N Σ
RN = Σ
2 n=1
Σ n=1
V snn) + 2Cmean
n=1
Next, we show that the cumulative regret is upper bounded. Let K == supxex ((x, x) (the kernel diagonal bound, finite since X' compact). For each term define th, := 2222 22 (x,): then 0 -
Un S Umax == k//² and GN == max (YYD:DD) DCX:DDI=N
log I I+o -
DCX:|D|=N/
1
log
KDxl
2
1
K Dx-1
KN-1
log
I+0
2
k
k(Xn, X,n
1
I
K KDx
KNN-
log
2
I o 2k(X,,, Xx)
1
log
I+ 0 K KDN
+ -2 (k((Xx, Xxn) (I+ -
"Ko )
2
1
log
I+ 2K (1+0 2333))-((xN))
2
1 N
log(1 (1 +o - -x22_(((x))
2 n=1
1 N
log(1 + un).
2 Fm
Here, I(.) denotes the mutual information, and Gx denotes the maximum information gain after N
steps; for definition and computation, we refer the reader to Srinivas et al. (2010). The covariance matrix KDx and the vector kx defined in Equation (2).
Consider the function (u) = log(1 + u)/u for u > 0; is decreasing, so for all u (0, Umax]s
log(1 + u)
log(1 ( + Umax)
Umax
u
It then follows that
U Umax
Un < log(1 + Umax)
log(1 + un),
18

---

Published as a conference paper at ICLR 2026
and therefore
o o²U max
- (Xx) uu I
log(1 + un)
log(1 + x/ log(1 + un).
log(1 Umax)
Summing over n and using
Σn=1 log(1 + un) = MiyV:NNiIN: S 2G yields
N
Σ 2
(xn) R=1
K log(1 + x/o²)
2G = Co GN,
where Ny fi:N) denotes the mutual information between the noisy observations yI- and the latent function values SuN. And we set
2x
Co = log(1 + K/O2)
By Cauchy-Schward and monotonicity of {3x).
N
N (xm)) & VNBN CoGN.
Bn 1 S
Σ
8m
Σ n=1
n
==
Hence, for constants C₁ = C0 and C2 2Cmean, obtain (with probability least. 1 5)
RN S Ci CIVNBBGN + C2 7m-1
In particular, if En The < co then the second term is bounded and we recover the asymptotic rate
Ry (VNBNGN)
which matches the BO-UCB rate.
F SUPPLEMENTARY EXPERIMENTAL SETUP
For simplicity, we set the prior mean to Mprioo (x) = A (x)) B with A = 1, 0. use initialization ratio of p 0.4, and fix the decay factor in 7n exp(-X(g -N)) to experiments employ the RBF kernel. For synthetic benchmarks, each experiment is repeated 10 times with different random seeds, whereas for HPO tasks, due to slower evaluation and higher computational cost, each experiment is repeated 5 times. The benchmarks and their respective settings are summarized Table All other parameters not explicitly specified are set to their default values in the BoTorch framework (Balandat al., 2020).
All
G COMPUTATIONAL OVERHEAD UNDER DYNMEANBO AND STANDARD BO
As described in the DYnMeanBO algorithm section, DynMeanBO integrates expert priors into the Gaussian process mean function in the form probabilistic distribution, effectively redesigning the BO mean function rather than introducing additional inference components. Therefore, this approach does not incur extra computational overhead. Theoretically, the computational complexity of DBB is nearly identical to that of standard BO, and experimental results confirm this.
H COMPUTATIONAL OVERHEAD UNDER THE "BAD" EXPERT PRIOR
As shown in Figure 12, when "bad" expert prior is used, the per-iteration evaluation time of DynMeanBO remains almost identical to that of standard BO under the same acquisition function, indicating that dynamic mean adjustment introduces negligible additional overhead. In sharp contrast, both TBO and ColaBO are considerably slower due to their reliance on Monte Carlo sam- pling for incorporating expert prior information into the acquisition function. The computational
19

---

Published as a conference paper at ICLR 2026
Benchmark
Search space
X
No
N
Hartmann (4D)
[0, 1]4
[0.19, 0.19, 0.56, 0.26
5
80
Levy (5D)
[-5, 5
[1]
6
100
Hartmann (6D)
[0. 1
[0.20, 0.15, 0.48, 0.28, 0.31, 0.66)
7
120
Rosenbrock (6D)
[- 2.048, 2.048)
[1]
7
120
Stybtang (7D)
[-4,4]
[-2.9]
8
PDI-WMT
0, 1]
[0.90, 0.69, 0.02, 0.97
5
40
PD1-CIFAR
0. 1]
[1. 0.80, 0.0, 0.0)
5
40
PDI-LMIB
[0, 1]4
[0.91, 0.67, 0.36, 0.85
5
40
Levy (20D)
[-5, 5)20
8 140
Rosenbrock (20D)
1-24048, 2.048
1 [
8 140
Table I: Experimental benchmarks used our study. For each benchmark, we specify search space, the location of the global optimum x", the number of initial points (No). and the total evalu- ation budget (N).
ICB
lngFl Dyn
T5
KG
MIS BySME
DysMeandiO-1
Dystesnttoess
PyRiteantO-GG
hartmann4
levy5
hartmann6
rasenbrackó
logul topulTmmg mrmm CD
stybtang7
pd1 Im16
pdl wmt
pól cifar
Figure 11: Per-iteration evaluation time (log scale) of DynMeanBO and standard BO on synthetic functions and HPO tasks.
burden is particularly pronounced for ColaBO, where employing MES necessitates nested Monte Carlo sampling to approximate the mutual information between candidate evaluations and the func- tion maximum, leading dramatic runtime increase.
These results highlight that under inaccurate expert priors, TBO not only suffers from poor robust- ness but also incurs substantially higher computational cost. ColaBO alleviates some of these is- sues, showing improved robustness over TBC, but it still falls short of DynMeanBO and comes with considerably higher overhead. In contrast, DynMeanBo delivers both superior robustness and efficiency.
DynMeanBO-EI
PIBO
DynMeanBO-LogE
MCpi-LogEI
DynMeanBO MEES
MC
hartmann4
levy5
hartmann6
rosenbrackó
SHAP
stybtang?
pd1 Im1b
pd1 wmt
pdl cifar
Figure 12: Per-iteration evaluation time (logı0 scale) of DynMe anBO, TBO, and ColaBO on syn- thetic functions and HPO tasks under the "bad" expert prior setting.
20

---

Published as a conference paper at ICLR 2026
I ADDITIONAL
EXPERIMENTS
In addition the eight commonly used benchmark tasks reported in the main text, also evalu- ated DynMeanBO on higher-dimension problems. Specifically, we included two 20-dimensional tasks Levy (20D) and Rosenbrock (20D). The corresponding search space settings, optimum lo- cations, number of initial points, total evaluation budget, and other relevant details are provided in Table
First, we evaluate the performance of DynMean8O on these two high-dimensional tasks under seven commonly used acquisition functions: PI, EI, LogEI, TS, UCB, KG, and MES. this experiment, use a "good" expert prior. The results are shown in Figure 13. Across all acquisition functions, DynMeanBO consistently accelerates convergence, demonstrating that not only seamlessly compatible with wide range of acquisition strategies but also highly effective in high-dimensional settings.
Light
Sampiig
levy20
rosenbrock20
60
15000
5200
Best Best-so-fat so-far value
40
10000
20
5000
50
100
0
Iteration
50
100
Figure 13: Performance on two high dimension tasks. When "good" expert prior is incorporated, DynMeanBO consistently finds better solutions faster than the standard BO across different acqui- sition functions.
"Good" expert prior. Under the "good" expert prior, we compare TBO, ColaBO, and DynMeanBO on these high-dimensional tasks. While BBO employs EI, Col aBO utilizes LogEI and MES, denoted as MCpi-LogEI and MCpi-MES in the figures. For fair comparison, we evaluate DynMeanBO using the same acquisition functions-ET, LogEI, and MES. The results are shown in Figure 14. Across both tasks, all three prior-based methods (DynMeanBO, TBO, and ColaBO) consistently accelerate convergence under the "good" expert prior, and their performance is compa- rable.
We also compare the average per-iteration evaluation time of TBO, ColaBO, and DynMeanBO, as shown in Figure 15. As discussed in Section 5.2, DynMeanBO introduces negligible computational overhead compared to standard BO. The results further show that under the "good" expert prior, although all three methods achieve similarly strong optimization performance, DynMeanBO incurs noticeably lower computational cost compared to both rBO and ColaBO.
expert prior is usually not perfectly accurate, but it rarely performs very poorly. In the few cases where the expert prior is indeed poor, we evaluate the robustness of BynMeanBO by comparing its performance with wBO and ColaBO under "bad" expert prior. The results on the two high- sigh-dimensiona tasks are shown in Figure 16, from which we can see that DynBeanBO exhibits strong robustness.
"Bad" expert prior. In practice,
We also compare the average per-iteration evaluation time of TBO, ColaBO, and DynMeanBO under the "bad" expert prior, shown in Figure 17.
21

---

Published as a conference paper at ICLR 2026
LogEr
MCpi-LogEI MCpi-MES
-ynMeanBO-EI
DynMean8O-LogEl
DynMeanBO-MES
Sampling
levy20
rosenbrock20
60
15000
5200
Best-so-far value
40
10000
5000
1000
50
100
Iteration
50
100
Figure 14: Performance two high dimension tasks under "good" expert prior. DynMeanBO, TBC, and ColaBO achieve comparable results.
DynMeanBO EI
PIBO
DOnMeanBB LagEl
MCpi-LogEI
DynMeanBO MES
MCpi MES
levy20
rosenbrock20
(Time [s])
4
x17.21
X30.94
log10 (Tgee
2
x14.65
Figure 15: Per-iteration evaluation time (log10 scale) of DynMeanBO, #BO, and CBO two high dimension tasks under the "good" expert prior setting.
El
LogEl
MES
MCpi-LogEI MCPl-MES
PIBO
DynMeanBO-EI
DynMeanBO-LogEl
DynMeanBO-MES
Sampling
levy20
rosenbrock20
Best Best-so-fan so-far Value
100
20000
15000
10000
5000
100
Iteration
50
100
Figure 16: Performance on two high dimension tasks under a "bad" expert prior. DynMeanBO demonstrates strong robustness.
22

---

Published as a conference paper at ICLR 2026
DynMean8O-Ei
DynMeanBO-LogE
MCpi-LogEI
Men
MC
levy20
rosenbrock20
log2o! (Time [s]]
4
x12.10
x32.67
2
x2.55
X5.77
x9.14
Figure 17: Per-iteration evaluation time (log1o scale) of DynMeanBO, =BO, and ColaBO on two high dimension tasks under the "bad" expert prior setting.
J SENSITIVITY ANALYSIS OF DECAY COEFFICIENT À
When constructing the mean function, we define it as mx,(x) Mnmmpror (x) + (1 7n (x),, In = exx(-A(rx) N))) ,weee Mprior (x) A ((x) + The hyperparameter decay rate of the expert prior. In principle, smaller À slows down the decay, allowing the expert prior to influence the optimization for longer period and thereby reducing exploration in other regions. Conversely, larger À accelerates the decay, diminishing the expert prior's influence earlier and encouraging broader exploration. To examine the effect of X, we conducted an ablation study with À 0.25, 0.5, 1.0, and 2.0 using El as the acquisition function across eight benchmark tasks. In these experiments, we used the "good" expert prior. The results are shown in Figure 18.
controls the
025
2,0
500
-2.4
0.5
-2.
-2.6
Aw 1.0
2.0
200
0.5
0.0
Reration
53
200
25
50
100
stybtong?
pd1 Jm1b
pdl wmt
pd1 citar
-200
0.5
0.40
0.35
-200
0.675
0.652
0.625
250
0.603
Figure 18: Ablation study of the parameter under the "good" expert prior setting
From the experimental results, observe pattern consistent with the above analysis. A smaller prolongs the influence of the expert prior, leading to less exploration in the early stages of opti- mization. If the expert prior is believed be highly reliable, smaller is therefore preferable. In contrast, a larger À shortens the duration of the prior's influence and results in more exploration early on. Thus, the expert prior considered less reliable, choosing larger À is more appropriate.
K SENSITIVITY ANALYSIS OF INITIALIZATION RATIO P
In our full algorithm (see Algorithm 1), we generate the initial points N₂ by combining samples from the expert prior distribution with Sobol sequences, which helps achieve better trade-off between exploitation and exploration. The initialization ratio p indicates the proportion of initial points drawn from the expert prior: a larger p corresponds more points sampled from the expert prior, while a smaller P corresponds to fewer such points. To investigate the sensitivity to p. we conducted an ablation study with p 0. 0.4, 0.8, and 1.0 using El as the acquisition function and the "good" expert prior across eight benchmark tasks. As a baseline, we used standard BO-EI. The experimental results shown in Figure 19.
23

---

Published as a conference paper at ICLR 2026
hartmannd
kwys
hartmanne
resenbracke
4000
20
3000
-1-
pe
pe
10-
2010
1000
0
20
40
60
22
200
102
stybtanng?
pd] Im]b
cifar
-150
0.0
3.0-
-150
0.5
0.0
0.0
-250
0.7
0,4-
-250
0.4
0.6
0.2
50
300
30
30
30 20
30
30
25
30
40
Figure 19: Ablation study of the parameter under the "good" expert prior setting
From the results of the ablation study, we observe that sampling entirely from the Sobol sequence (p 0) performs worse than sampling some points from the expert prior (p > 0). This because, in the initialization phase, if the expert prior is sufficiently reliable, points drawn from it tend to be closer to the optimum than those sampled from Sobol.
We also observe that sampling entirely from the expert prior (p = 1) performs worse than combi- nation of Sobol and expert prior sampling (0 < p 1).
Finally, even when all initial points are sampled from Sobol (p 0) (as can be seen in the Figure 19, where the blue and black lines start from the same position), DynMeanBO-BB EI still outperforms standard BO-EI, especially on the Hartmann4, Hartmann6, and Stybtang7 tasks. This is because, although both DynMea (aMMMManBOEEE) MBQ-ET and BO-EI initialize entirely from Sobol when p = 0, DynMeanBO incorporates the expert prior into its mean function, which guides subsequent iterations toward better regions and enables faster convergence. These results demonstrate the effectiveness of integrating expert prior knowledge into the mean funetion.
Therefore, for the choice of the initialization ratio p, we recommend < < 1. If the expert prior sufficiently reliable, larger p is preferable, whereas if the expert prior is less reliable, a smaller p is more appropriate.
L EXPERIMENTS UNDER DIFFERENT PRIOR STRENGTHS
In the main text, compared "good" expert prior (offset = 0.1 from the optimum, with a standard deviation equal to 0.2 times the length of the search-space interval) and a "bad" expert prior (offset 0.7 from the optimum, with the same standard deviation). To more comprehensively characterize different prior conditions. we further examine strong expert priors, weak expert priors, and wrong expert priors, well different uncertainty levels with standard deviations set to 0.2, 0.4, 0.6, and 0.8 times the search-space interval. The detailed configurations of the strong, weak, and wrong expert priors are provided in Table 2. Here, we still use the EI acquisition function; that is, this section presents an ablation study of different expert prior settings under DynMeanBO-BD EI. The experimental results are shown in Figure 20.
Table 2: Settings strong, weak, and wrong expert priors.
Expert Prior Type
Offset
Std (x search-space length)
Strong Expert Prior
0.05
0.2
Weak Expert Prior
0.20
0.2
Wrong Expert Prior
0.90
0.2
From the experimental results, we observe that the variance the expert prior has little effect on the optimization performance, whereas the quality of the expert prior has significant impact. When
24

---

Published as a conference paper at ICLR 2026
strong expert prior is used, the DynMeanBO algorithm converges to the optimum almost immedi- ately. With a weak expert prior, DynMeanBO performs slightly better than standard BO. In the case of wrong expert prior, the initialization phase is initially misled, resulting in worse convergence compared to standard BO. However, due to the robustness of DynMeanBO, it eventually converges performance comparable to that of the original BO.
hartmann4 (std=0.4)
hartmannd
hartmanne
0-
Weong
Itaration
60
60
43
60
levyS (sid=0.22 2)
levy5 (std=0.4)
levy5 (stdm0.0) 6)
levy5 Istd=0 ()
25
30
20
15
20
20
30
10
10
hartmann6 (std=0.2)
hartmanné (std=0.4)
hartmann6 (std==0
hartmanné (std=0 B)
50
15
100
30
305
30
73
100
30
79
200
stybtang7 (std=0.2)
styotang? (std (.44
stybtang7 (sta 0.66
styotang? (std=0.8)
-50
50
-150
-130
100
150
-150
150
-250
-200
-210
-250
-250
pd1 /m1b (std=0.2)
pd1 Jm1b (std-0.4)
pd1 Im1b (std=0.6)
pd1 Im1b (std=0.8)
0.0
0.7
0.6
0.5
wmt (stdm0.2)
pdl wmt (stdm0.4)
pdl wmt (stdm0.6)
pd] wmt (sId=0.0)
6.8
0.0
0.8
0.8
6.6
0.6-
0.6
20
20
30
10
20
30
10
20
20
10
22
pdl
3.0
0.5
0.5
0.8
3.5
5.4
0.4
0.2
0.2
40
25
30
20
10
25
30
40
Figure 20: A comparative analysis of strong, weak, and wrong expert priors under varying prior variances.
25

---

Published as a conference paper at ICLR 2026
M USE OF LARGE LANGUAGE MODELS
We employed the large language model GPT-4 to polish the manuscript. The process mainly in- volved: (1) checking for grammatical errors; (2) evaluating the appropriateness of voice usage; and (3) identifying and eliminating redundant expressions.
26


Reviews:
[{"review_id": "O7ZLW2gjB5", "reviewer": "Reviewer_ZGXS", "summary": "The paper introduces DynMeanBO, a Bayesian optimization framework that incorporates expert prior knowledge into a GP mean function and gradually reduces that influence over time using an exponential decay factor. This dynamic mean decay allows the optimizer to exploit useful expert beliefs early on but prevents long-term bias from incorrect priors, making the method computationally light, acquisition-function agnostic, and preserving standard convergence guarantees. Empirical results across synthetic functions and hyperparameter tuning tasks show that DynMeanBO matches or outperforms prior-informed BO methods while being more robust and nearly as fast as standard BO.", "questions": "1. How sensitive is DynMeanBO’s performance to the choice of decay rate $\\lambda$ and the number of initial prior-guided points $N_0$? The experiments fixed $\\lambda=1$ and used a 40% prior-based initialization without ablation. Would a slower or faster decay significantly change the outcomes?\n2. Beyond the specific “bad prior” tested (a single offset Gaussian), are there other failure modes to be aware of? For example, what if the expert prior is not just slightly off in location but completely adversarial (e.g. concentrated in a region of very low function value)? In such extreme cases, does the hybrid initialization (mixing Sobol points) and decay suffice to guarantee recovery, or could there be situations where DynMeanBO still performs worse than standard BO for a while? A related point is that how does observation noise affect the trust in the prior? If the objective evaluations are noisy, one might need more iterations to “override” a misleading prior.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.\tIntegrating prior distributions into the GP mean with a decaying coefficient is conceptually straightforward and compatible with arbitrary acquisition functions. It avoids the complexity of prior-specific acquisition designs.\n2.\tBy decaying the influence of the expert mean, the method can recover from misleading prior beliefs. Theoretical analysis establishes that DynMeanBO does not sacrifice long-term performance despite using potentially biased priors. The authors prove that under standard assumptions (e.g. the objective lies in the RKHS of the GP kernel), DynMeanBO with EI or UCB achieves the same asymptotic convergence rate and regret bounds as standard BO.\n3.\tThe approach is modular and compatible with a broad set of acquisition functions, which avoids the complexity of prior-specific acquisition designs.", "weaknesses": "1.\tMost experiments involve low  to medium dimensional tasks and a specific set of deep-learning hyperparameter benchmarks; it remains unclear how DynMeanBO performs in very high-dimensional or multi-fidelity scenarios, or on domains where priors are scarce.\n2.\tThe paper did not provide detailed analysis on how sensitive the method is to mismatched prior modes or variances, which would help practitioners understand risk when experts provide only coarse or biased priors.\n3.\tAlthough mixtures of Gaussians are considered, the framework implicitly assumes that expert priors can be parameterized easily and effectively, which may not hold in many real-world domains.\n4.\tDynMeanBO introduces new tuning parameters (for the prior mean and its decay) that are set heuristically and not thoroughly examined. The GP mean uses a scaled version of the expert’s prior distribution $m_{\\text{prior}}(x) = A,\\pi(x) + B$, but the paper fixes $A=1$ and $B=0$ “for simplicity” without discussing how these might affect results. Likewise, the decay schedule $\\gamma_n=\\exp(-\\lambda (n-N_0))$ uses a fixed rate $\\lambda=1$ in all experiments, and 40% of initial samples are drawn from the prior (a fixed ratio). There is no ablation study or guidance on choosing these hyperparameters. This raises questions: e.g. if $\\lambda$ is too small (slow decay), a bad prior might mislead longer, whereas if too large, a good prior’s benefit might vanish prematurely. The robustness and performance of DynMeanBO could be sensitive to these settings, but the paper provides only a single fixed configuration. Without exploring different decay rates or prior weightings, it’s unclear how one would tune DynMeanBO in practice when the reliability of the expert prior is unknown.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r6oRUAdwmX", "reviewer": "Reviewer_udxX", "summary": "The authors claim to contribute:\n- A lightweight framework to incorporate expert priors into the GP mean function and decay it toward standard BO behavior, so that the resulting method can take advantage of a good prior but be robust to a bad one\n- Convergence guarantees under EI and UCB acquisitions\n- DynMeanBO performs comparably to existing methods that use a prior in good/bad prior case, but is much more computationally efficient because it only uses the prior in the GP mean function.", "questions": "I’d be curious to see what happens when a really bad prior is given (e.g., strong and in a place pretty far from the optimum) — does DynMeanBO still recover, or does it perform much worse than using regular BO?\nWhat are the compute time factors for DynMeanBO vs. PiBO, ColaBO respectively? 10x faster? 2x faster? Specify for both good and bad prior experiments if they’re different. Also, some more discussion of the relative time for DynMeanBO vs. acquisition costs in real-world BO settings, compared to ColaBO/PiBO vs. acquisition costs would help elucidate the real-world relevance and applicability of this method.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "- Paper is well-organized and the main ideas and contributions concisely laid out. The background is thorough. The related works is thorough and motivates their method well. The method is simple and articulated clearly. \n- DynMeanBO is substantially faster than MCpi (~10x?), and a little bit faster than PiBO. It would be good to know exactly how much faster it is than each method (perhaps write in the text or caption).\n- DynMeanBO seems roughly comparable to PiBO and MCpi, though those methods seem to perform a little better in general, but is more computationally efficient (Fig. 4). It might be more informative to fix the total computation time to some budget, and then show regret for the more expensive methods, which may be able to take many fewer queries, with compute time as the x-axis. It seems generally better than PiBO and MCpi when a bad prior is given (Fig. 6). I’d also be curious to see what happens when a really bad prior is given (e.g., strong and in a place pretty far from the optimum) — does DynMeanBO still recover, or does it perform much worse than using regular BO?", "weaknesses": "- The prior is only incorporated into the GP mean function, though it seems desirable that it affect the covariance matrix too. It seems previous works have tried this and so far found it computationally expensive or unstable, though.\n- The method seems relatively simple, which is not necessarily a bad thing, but I’m unsure if it constitutes a substantive contribution to the BO community.\n- The regret results are not super impressive, though the method is much faster to run. It seems like this would be useful in certain BO cases, but it’s not totally clear to me how important this kind of work is, since BO typically deals with cases where querying the objective function is pretty expensive such that it’s often not a huge deal if it takes a while to choose the next point to evaluate. Some more discussion of the relative time for DynMeanBO vs. acquisition costs in real-world BO settings, compared to ColaBO/PiBO vs. acquisition costs would help elucidate the real-world relevance and applicability of this method.\n- Fig.2: It’s pretty difficult to read the graphs because they’re so small. Maybe a log scale should be used for the y-axis. There primarily seems to be a performance improvement at early iterations, which makes sense because the prior just starts in the good region. However, on none of these problems is this gain held in later iterations.\n- The method seems highly dependent on choosing a prior decay schedule, which should to some extent express how good the prior is expected to be anyway. It seems like it might be easier to just choose a weaker (more dispersed) prior instead. I can appreciate that they’ve chosen an exponential decay with only one hyperparam, $\\lambda$, which seems to work reasonably well in the cases tested.\n- I’m unsure how meaningful the theoretical results are in that they seem to rely on the fact that the mean will revert to having effectively no prior influence eventually, which seems like a trivial statement—if I do something for a finite number of iterations and then revert to a BO procedure that already has convergence guarantees, it’s unsurprising that this method also converges. That being said, I didn’t read the details super closely.\n\nMy recommendation would be weak accept--I think the paper and experiments are really nicely executed, but I'm unsure if the methodological contribution and results matter enough. I'm not super well-versed in the BO + priors subfield though.\n\n*Some general feedback:*\n\nPlease make figure legends larger so they’re easier to read (e.g., Fig. 1). Also typo in Fig.1 — “Samples *form* prior” —> from. Some of the figures are generally too small / too many plots; please print out your paper and scale stuff so it’s readable in print.\nBased on the algorithm description, I wouldn’t expect the runtime to be that much different from not using a prior, so I don’t think it’s that important to put this as Figure 3 — maybe a later figure, or supplemental? Figure 5 is more informative, maybe these two can be combined? It also might be helpful to make one big plot instead of many little plots, where the x-axis is the different problems, and each method is plotted as a line with error bars.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YXwJnGQmp8", "reviewer": "Reviewer_xfoY", "summary": "This paper introduces Dynamic Mean Bayesian Optimization (DynMeanBO) that incorporates expert priors into the Gaussian process mean function with a dynamic decay mechanism. Expert priors are modeled as probability distributions over the likely location of the global optimum. The dynamic decay mechanism is introduced to handle potential bias from bad priors. This approach is lightweight, compatible with arbitrary acquisition functions (AFs). Theoretical analysis provides convergence guarantees: DynMeanBO-EI and DynMeanBO-UCB. Experiments evaluate DynMeanBO on synthetic functions and hyperparameter optimization (HPO) tasks, using \"good\" and \"bad\" Gaussian priors. The proposed method accelerates convergence with informative priors, remains robust to misleading ones, and outperforms baselines like πBO and ColaBO.", "questions": "Please see the Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses a practical gap in prior-informed BO by integrating priors directly into the GP mean function in a principled way, rather than heuristically modifying AFs or kernels. The dynamic decay is a clever, simple mechanism that balances exploitation of priors early on with robustness later.\n2. The paper provides solid convergence proof for EI and UCB, extending standard results to the dynamic mean setting.\n3. The writing is clear. Figures are helpful, and appendices are referenced for details.", "weaknesses": "1. The proposed method relies exclusively on GPs, which may not extend easily to other surrogates like random forests or neural networks used in modern BO.\n2. Parameters like the decay rate, scaling/shift, and initial points could be sensitive, but tuning guidance is minimal. In bad prior cases, if decay is too slow, bias might persist. Thus, empirical ablation would help.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)