Event-Triggered Time-Varying Bayesian Optimization
Authors: Paul Brunzema
Alexander von Rohr
Friedrich Solowjow
Sebastian Trimpe
Venue: Accepted by TMLR ()

Abstract: We consider the problem of sequentially optimizing a time-varying objective function using time-varying Bayesian optimization (TVBO). Current approaches to TVBO require prior knowledge of a constant rate of change to cope with stale data arising from time variations. However, in practice, the rate of change is usually unknown. We propose an event-triggered algorithm, ET-GP-UCB, that treats the optimization problem as static until it detects changes in the objective function and then resets the dataset. This allows the algorithm to adapt online to realized temporal changes without the need for exact prior knowledge. The event trigger is based on probabilistic uniform error bounds used in Gaussian process regression. We derive regret bounds for adaptive resets without exact prior knowledge of the temporal changes and show in numerical experiments that ET-GP-UCB outperforms competing GP-UCB algorithms on both synthetic and real-world data. The results demonstrate that ET-GP-UCB is readily applicable without extensive hyperparameter tuning.

Published in Transactions on Machine Learning Research (01/2025)
Event-Triggere Time-Varying Bayesian Optimization
Paul Brunzema
Institute for Data Science in Mechanical Engineering RWTH Aochen University
Alexander Rohr
Institute for Data Science in Mechanical Engineering RWTH Aochen University
Friedrich Solowjow
Instätute for Data Science in Mechanical Engineering RWTH Aachen University
Scbastian Trimpe
trimpe Samnpe@@dsm.ruutbaaaaaeee
Institute for Data Science in Mechanical Engineering RWTH Aochen University
Reviewed OpenReview: https: // ottp://veeeereeee.eommme net/ forum? id= WEYMCLuBU7
Abstract
We consider the problem of sequentially optimizing time-varying objective function using time-varying Bayesian optimization (TVBO). Current approaches to TVBO require prior knowledge of a constant rate of change to cope with stale data arising from time variations. However, practice, the rate of change is usually unknown. We propose an event-Lriggered algorithm, ET-GP-UCB, that treats the optimization problem as static until it detects changes in the objective function and then resets the dataset. This allows the algorithm to adapt online to realized temporal changes without the need for exact prior knowledge. The event trigger is based on probabilistic uniform error bounds used in Gaussian process regression. We derive regret bounds for adaptive resets without exact prior knowledge of the temporal changes and show in numerical experiments that ET-GP-UCB outperforms com- peting GP-UCB algorithms on both synthetic and real-world data. The results demonstrate that ET-GP-UCB is readily applicable without extensive hyperparameter tuning.
1 Introduction
Over the last two decades, Bayesian optimization (BO) has emerged as powerful method for sequential decision-making and design of experiments under uncertainty (Garnelt et al., 2010; Snoek et al., 2012; Ca- landra et al., 2016; Frazier & Wang, 2016; Chen et al., 2018; Neumann-Brow Hernández-Lobato, 2020; Colliandre & Muller, 2023). These problems are typically formalized as optimiza- tion problems of a stotic unknown objective function. At the core of BO is the exploratioon-cxplitatii trade-off, where the decision maker weights the potential benefit of a new unexplored query against. the known reward of the currently assumed optimum. The performance of BO algorithms typically measured in terms of regret, i.e., the difference between the actual decision taken and the (unknown) optimal one. If one assumes static objective function, there exist algorithms that achieve desirable sub-linear regret (see Garnett (2023) for overview). mcaning they will efficiently converge to the globally optimal solution.
et al., 2019; Griffiths &
In the real world, however, objective functions are often lime-varying. Time-varying objective functions are widespread across various domains, including economics due to fluctuations in price or supply and de- mand (Primiceri, 2005; Dangl Halling, 2012), environmental science (Marchant & Ramos, 2012; Marchant et al., 2014), and control systems due to changing system dynamics (Aström & Wittenmark, 1995; Ellis & Christofides, 2014; Colombino et al., 2019; Simonetto et. al., 2020; König al., 2021; Brunzema et al., 2022).

---

Published in Transactions on Machine Learning Research (01/2025)
Data set
Acquisition function
Øt
Objective f(
Di
at = arg max a(x D2) xEX
fela)
next time step
(1 M1
* detected change
x: Event trigger bound GP model
D4+1
Event Trigger
Preset E {0, 1}
Figure Illustration of our proposed concept. After the standard BО steps of optimizing the acquisition function and obtaining an observation, an event trigger decides whether reset the data set (Treset 1) if a change detected (see red cross), or augment with the observed data point in the usual way (Treset = 0). This event trigger allows our algorithm ET-GP-UCE to be adaptive to changes in the objective function without relying on exact prior knowledge of the rate of change.
A timevngryng objective function alters the nature of the the optimal decision and the information decision maker gains from each query, and any collected data set becomes stale. Generally, in setting of ongoing changes and without further assumptions, we cannot expect to find the optimal query, and conversely, no algorithm can achieve sub-linear regret (Besbes et al., 2015, Proposition 1). Instead, our goal is to design algorithuss with an explicit relation between regret and the rate of change E, which quantifies how fast the objective function varies. In their seminal work, Bogunovic et al. (2016) proposed two algorithms for the time-varying Bayesian optimization (TVBO) problem; however, the proposod algorithms and their subsequent variants rely on prior knowledge of the rate of change as a hyperparameter. In typical BO applications, where the objective is unknown, knowledge of its rate of change is hard to obtain. Moreover, in these algorithms, is specified at the start and assumed to remain constant over time, which is rarcly the case in practice.
trade-off. Time affects
To address these limitations, propose the novel concept of event-triggered TVBO (see Figure 1). We model the objective function time-invariant until event trigger detects significant deviation of the observed data from the current model. The algorithm then reacts to this change by resetting the data set. This way, the time-varying nature of the problem is considered in the algorithm's exploration-exploitatii tradc-off only when necessary. The event trigger replaces previously required exact prior knowledge of a rale of change, ET-GP-UCB exhibits strong empirical performance for different time variations, including gradual, sudden, and no change, and desirable theoretical properties under common regularity assumptions and upper and lower bounds on the rate of change. In summary, contributions
follows:
(i) A conceptually new event-triggered BO algorithm for TVBO;
Bayesian regret bounds for event-triggered BO for an unknown but bounded rate of change;
(ili) Empirical evaluations of ET-GP-UCB on synthetic data and real-world benchmarks.
2 Problem Setting
We aim to sequentially optimize unknown time-varying objective function f: X RR compact set X C Rd over the discrete time steps E Ir. = (1....,T) up to the time horizon T
a arg max fx)).
(1)
#EX
Here and in the following, subscript t denotes time dependency. At each time step, algorithm chooses query D; E X and obtains noisy observations following the standard assumption in BO.
Assumption 1 Observations y, fi f( +w, are perturbed by independent and identically distributed (i.i.d.) Gaussian noise with noise variance oî wi ~ NO, o2))
Without assumptions on the temporal change, the problem is intractable. We impose the Markov chain model proposed by Bogunovie al. (2016) and used in various prior applications (see Sec. 3).
2

---

Published in Transactions on Machine Learning Research (01/2025)
Assumption 2 (Bogunovic et al. (2016)) Let GP (p. k(x, x')) de a Gaussian process (GP) with mean function X + R and kernel k: X x X + R. Given i.i.d. samples 91,92 .2.............................. with yi ~ GP (0, k(a, a²)) where k(x, x') is stationary kernel with k(æ, x') S 1 for all æ, x' E R, and rate of change E [0, 11, the time-varying objective function f((Ø) follows the Markov chain:
gt(2)) f((x) = (VI
t=
(2)
+ VEgi(z), 22 2.
Remark 1 Due to Assumption 2, any choice of E E [0, 1] yields that fa ~ gP (0, k) for all E In.-
We further use standard smoothness assumptions on the kernel choice introduced in Srinivas et al. (2010)
as well as Bogunovic et al. (2016).
Assumption 3 The kernel k(x, is such that, given f ~ GP (0,K L, 2 1, and au bu a1, the following holds:
{ sup [(x)) > 11 age (by//b0) and IP sup
ff(x) xxx drg
> s ,j -.=l..., I...,d
PP
(3)
axx
This assumption limits the choice of kernel to those that are at least four times differentiable (Ghosal & Roy, 2006, Theorem 5), such as the popular squared exponential (SE) kernel.
The performance of an algorithm optimizing (1) is measured in terms the cumulative regret Ry.
Definition 1 Let ot be the mazimizer of f((x)), and x, E X be the query at time step t. Then, the instan- tancous regret at t is T, = f((xt) - f((x)), and the regret after T time steps Ry = -i= Tt.
Sub-linear regret is defined as limy-+o0 =T O. In the time-invariant setting (c = 0), algorithms can achieve such sub-linear regret, e.g., GP-UCB (Srinivas al., 2010, Theorem 2) with regret of order Ö(VT)². For time-varying objective function following Assumption 2, Bogunovie et al. (2016, Theorem 4.1) shows that the lower bound on the regret of algorithm is linear P(Tss).
Problem Statement We consider the tare-ng optimization problem (1) under Assumptions 1 1-3. To develop a practical algorithm and in contrast to the state-of-the-art in TVBO we assume the exact rate of change & to be unknown. According to prior work (Bogunovic et al., 2016), it is reasonable to demand regret bounds that are no more than linear in and explicitly dependent on the true E.
3 Related Work and Background
We build on prior work in BO and TVBO. This section gives an overview of these two fields.
Bayesian Optimization Our event trigger design and algorithm extends prior work by Srinivas et al. (2010) on the BO algorithm GP-UCB and transfers some of their concepts to the time-varying setting. Srinivas et al. (2010) proved sub-linear regret bounds when using an upper confidence bound (UCB) acqui- sition function of the form μDD (z) (B,OD, (2), with standard posterior updates of the GP model, given a dataset D.- = ((o,,,8))) as
PDD. (x) = (x)) (K, + +1 y and of (x) = k(x - k,)))" (K, + 2.11) kx(
(4)
time-invariant posterior mean
time-invariant posterior variance
where K, [k(,,, x, Ma. is the Gram matrix, (x) [k< >) and noisy measurements y: (B,,..., 11- follow Assumption 1. Our algorithm uses these standard posterior updates only as long as all the measurements comply with the current posterior model; resets the dataset if this is no longer the case. Consequently, in contrast to GP-UCB, our algorithm can adapt to changes the objective function by discarding stale data. For a detailed overview of other standard BÔ algorithms as well as the standard BO framework, refer to Garnett (2023).
1 As in Bogunovic al. (2016) and Srinivas al. (2010), we denote asymptatics up to log factors Ö(.)
3

---

Published in Transactions on Machine Learning Research (01/2025)
Time- Varying Bayesian Optimization TVBO can be considered special case of contextual BO (Krause & Ong, 2011); with the key difference that time is treated as a strictly increasing variable. The inclusion of time as an input for spatial-temporal monitoring within the BO framework was explored by Marchant & Ramos (2012) and Marchant et al. (2014). Building on spatio-temporal GPs, Bogunovic et al. (2016) introduced two time-varying UCB algorithms tailored for TVBO, where they suggest that changes in the objective function can be effectively captured using the model outlined in Assumption 2. Embodding this assumption into the GP surrogate model yields the following posterior updates
iuD, (x)) - (xx) (K, + + -1 yr and T (x) = k(), kr(x)" (K, + onl) k,(2))
(5)
time-varying posterior mean
fine-varying posterior variance
k=== and k((x)) = (x) o ktime with ktime [(1 c)1(t1-=1//1[=1 R=1: and is the Hadamard product. Using updates following (5) ensures that the posterior distribution follows the same dynamics as ft. We will utilize this in the proof of regret bounds for algorithms without exact knowledge of & under the mentioned regularity assumptions.
where K, = K2 Ktime with Ktimee = [(1 - -)
Since the introduction of this setting by Bogunovic et al. (2016), their algorithm TV-GP-UCB with posterior updates in (5) has been used in various applications, such as controller learning (Su et al., 2018), safe adaptive control (König et al., 2021), and hyperparameter optimization for reinforcement learning (Parker- Holder et al., 2020; 2021). Similarly, Brunzema et al. (2022) use time-varying posterior updates in their algorithm UI-TVBO but with a different temporal kernel. Both approaches rely on a priori estimation of the rate of change and assume that it stays constant over time. In contrast, the event trigger in ET-GP-UCE does not need an estimate of the rate of change and enables our algorithm to adapt to changes the objective. We note that if the rate of change is known beforehand and the structural assumptions are satisfied, the TV-GP-UCB is efficient choice since these properties are explicitly embedded in the algorithm.
Bogunovic et al. (2016) further introduced R-GP-UCB, which resets the data set periodically to account for data becoming uninformative over time. In essence, ET-GP-UCB is an adaptive version of R-GP--CB and resets only if an obtained mcasurement is no longer consistent with the current datasct (cf. Sec. 5). As in R-GP-UCB, our algorithm uses time-invariant posterior updates within time intervals, but our time intervals do not have to be specified priori based on E; rather, they are determined online by the event trigger.
Under frequentist regularity assumption, Zhou & Shroff (2021) also propose an algorithm with a constant reset time, BR-GP-UCB3, as well as a sliding window algorithm. The crucial difference to our pettle is their assumption of a fired variational budget, i.c., Et= f1 fillx < Br, where H. is a Reproducing Kernel Hilbert Space (RKHS) and Br a scalar. Leveraging these same assumptions, Deng al. (2022) propose an algorithm, W-GP-UCB, which utilizes similar weighted GP model to TV-GP-UCB, In this setting of a fixed variational budget, sub-linear regret is possible (Besbes et al., 2015). In contrast, we are interested in the setting given by Assumption 2, which corresponds to a variational budget that increases linearly in time; hence, the best possible regret is also linear (Besbes et al., 2015, Proposition 1). Nevertheless, in Appendix A.5, we empirically compare ET-GP-UCB against the algorithms of Zhou & Shroff (2021) and Deng el al. (2022) and display improved performance on their benchmarks with less prior information.
Event-Triggers Learning The idea of resetting a dataset only an important change occurs as detected by a trigger event is similar to the concepts of event-triggered learning (Solowjow & Trimpe, 2020) and event- triggered online learning (Umlauft & Hirche, 2019). In the multi-armed bandit setting, Wei & Luo (2021) discuss the idea of including tests into optimization algorithms to detect and adapt to non-stationarity. Our algorithm ET-GP-UCB is the first to introduce event triggers to BO and specifically TVBO. Since then, this idea has been taken up for a different BO problem, namely safe BO, in recent work (Holzapfel et al., 2024). The trint-triggered concepts aim to be efficient with the available data, only updating model when necessary or only adding new data that is relevant. FT-GP--CB also aims for data efficiency; as long as we have good model of our objective function, we want to use already available data to minimize regret. If the event trigger detects a significant model mismalch, we reset the dataset and thereby delete stale data.
We label it BR-GP-UCB to avoid con fusion with R-GP-UCB of Bogunovic et al. (2016).
4

---

Published in Transactions on Machine Learning Research (01/2025)
Dynamic and Non-Stationary Multi-Armed Bandits Related problems to (1) have also been con- sidered in the non-stationary multi-armed bandit, (MAB) literature. The key distinction to our problem setting is that for MABs the number of arms is finite and they are not correlated. Similar to our setting with ongoing changes, Whittle (1988) introduce restless MABs; here, the state of an arm can change in each stop according to a known (yet arbitrary) stochastic transition function (Slivkins Upfal, 2008). Such restless MABs are known to be intractable (Papadimitriou & Tsitsiklis, 1994; Slivkins & Upfal, 2008). For restless bandits with a "time-variation parameter" greater zero linear regret as Q(T) is inevitable (Slivkins & Upfal, 2008, Theorem 1.3). The setting in Slivkins & Upfal (2008) is, in its effects, similar to the regularity assump- tion on the temporal changes in Assumption 2 with corresponding lower bound on the regret of (2(Ts) (Bogunovie et al., 2016, Theorem 4.1). Also related are MABs under a fixed number of abrupt distribution shifts ( (Auer et al., 2019; Abbasi-Yadkond et. al., 2023). For these problems, adaplive resetting algorithms have been proposed (Chen et al., 2019; Wei & Luo, 2021; Suk & Kpotufe, 2022). In this more structured problem setting, similar to the frequentist regularity assumptions discussod above, sub-linear regret possible.
4 Regret Bounds for TVBO with Adaptive Resets
The core goal of this work is to develop an algorithm for an unknown rate of change. To achieve this goal, we first generalize the results by Bogunovic et al. (2016) and derive regret bounds for a BO algorithm with variable resets. We analyze the general problem for all strategics that reset anytime between a lower (N) and upper (N) reset threshold. Our analysis reveals that for any resetting strategy, given reasonable N and N, linear regret with explicit dependency on the true is guaranteed, even if the exact e is unknown to the resetting algorithm. This is in contrast to previous work on TVBO, where the true does not only appear in the regret bounds but needs to be known to the algorithm or GP model. We first introduce theorem applicable to all algorithms with such a general resetting strategy, and then apply it to specific kernels to derive scaling laws. In See. 5, we will propose a specific event trigger as a resetting strategy.
Theorem 1 Let the domain X C [0, c Idd be convez and compact with d E N+ and let fo follow Assumption 2 with kernel k( x') such that Assumption 3 is satisfied. Pick 8 E (0,1) and set Br = 21n (27²12/(3)) + 2d (Da / /nn(da (335)) Pick a lower block lenglh N E N+ and a upper block length Ñ E N with N S Ñ < T. Let IN be the marimum information gain over Ñ points. Then, running any algorithm with block sizes between N and N satisfies
RT S CIT (N +1) IN + N)
(6)
with probability at least 1- 8, where Ci = 881 (1 + on 2) and we defined
Φr(e, 2 3r(30 2 + oo N) + 2( + on - N3 + v20n) V 1)
36
Sketch of proof: We extend the analysis of R-GP-UCB to setting in which rescts are not enforced periodically but can occur at any time step between N and N. Building on Assumptions 1 to 2, we use the upper threshold N to bound the difference between the model used in the algorithm (time-invariant model, see (4)) and the time-varying model in (5). We use the lower threshold Ñ to bound the maximum number of resets over the time horizon. The full proof is in Appendix F.
We next apply this theorem to specific kernels. We specify how N and Ñ scale with upper and lower bounds on the rate of change to obtain the desirable scaling of the regret.
Corollary 1 Let the assumplions in Theorem 1 hold. Given a lower and upper bound on the true s such that ē (0, 1) there erist A E (0, 1] and X2 () such that g = >1 E and X2 6. For the and N = O(min(T,e -1/4]). Then, with probability 1 5, we have
SE kernel, set N- (T,
R₁ = Ô 1:mmma (max { VT, 1//21//1 X 3/8 3//8/1/8TT XT -3/4 3//41/T))
(8)
5

---

Published in Transactions on Machine Learning Research (01/2025)
For the Matèrn kernel (v > 2), define & d(d+1)
2x+d(d+1] and set N
O(min (T, (TTTT-/// 1114-11)))
and N
0(min(T,==V/aa-51) Then, with probability 1 - ô, we have
21 2 कं g 244 Le A2
Le का 21 E
T, X1 x (
Ry = Ô max VT X1
T, x
(9)
From Corollary 1, can derive the following insights: (t) It is possible to design algorithms with regret explicitly depending on e, without knowing its exact value. (ti) A larger interval for E, i.e. decreasing λ or increasing X2, results in a worse scaling of the regret bound. It essentially is the price one has to pay for not knowing the exact value of e. (Mii For & = = E, hence A = ) 1, the scaling of R-GP-UCB is recovered. From Theorem 1, we can further observe that for & = 0 the bounds recover the sub-linear scaling of GP-UCB iff an appropriate lower bound on e is specified such that N T which follows intuition as no resets should should take place.
The above results demonstrate that it is possible to design algorithms that do not require the exact rate of change; as long lower and upper bounds on are known and resets occur within [N, NJ] the algorithm has bounded regret. Crucially, we did not actually specify when to reset, thus our analysis includes random and "worst case" resets. Next, we utilize the flexibility in choosing the reset timing within [N, N] to achieve better empirical regret, e.g., as compared to periodic resets.
5 An Event Trigger for Adaptive Resets in TVBO
We will now introduce our algorithm ET-GP-UCB. It will decide when to reset through its event trigger (see Figure 1). We build on established uniform error bounds for GP regression to decide if new data is consistent with the posterior. When new data deviates significantly, we reset. the dataset. This idea constitutes a new approach compared to previous work in TVBO, as the resulting algorithm does not rely on the true rate of change E as a hyperparameter. The core component of our algorithm is the event triggor:
Definition 2 (Event trigger in TVBO) Given lest function the and a threshold function Kt, both of which can depend on the current dataset Di and the latest query location and measurement pair (xt, Ve), the event trigger at time is
Treset
the (Dr, (x1, > (Di, y())
(10)
where Freese the binory indicator for whether to reset the dataset (Treset = 1) or not (Treset = 0).
We next design and develop 2/1 and K2 for the specific event trigger of ET-GP-UCB. We base these functions on Bayesian uniform error bounds for GP regression (Srinivas et al., 2010). Other design options would also be possible, c.g., likelihood-basei or performance-basce, event triggers. In the following, we denote the time step of a reset as 1,, 12,()).... .... and define 4, E (1, [411]] C N4, as the time in between resets. We have t, = if no reset has yet occurred.
Lemma 1 (Srinivas et al. (2010, Lemma 5.5)) Pick 8 E (0,1) and set Pt. = 21n _gg where Ex.2 -1 = 1, nt. > O. Let f ~ GP k(x, Then, (f(x)) - 'DD (DD (xx) & /Pt, aD. (xx) holds for Tte all t 2 with probability at least 1 1-66
Remark 2 For Σi.1 o. = to hold, one can choose n., = n 2,, as in Srininas el al. (2010).
The result gives a high probability bound for all time steps on the absolute deviation between the posterior mean and time-invariant objective function at the query location x; based on the posterior variance and parameter Pt, If a time-varying objective function satisfies this bound for all time steps, the time variations are negligible with high probability (i.e., & R 0). On the other hand, if the bound in Lemma 1 is violated at a chosen query location &z, the objective function has likely changed.
The algorithm needs to evaluate the trigger event at every time step. While it cannot directly evaluate (a,) - MD. (a;) in Lemma (cf. Assumption 1), we can evaluate 1/t H', (æ;)). Thus, we adapt Lemma 1 to design probabilistic bound that includes only yi instead of fxx)). For this, we first bound the noise in probability. All proofs of the following Lemmas are in Appendix G.

---

Published in Transactions on Machine Learning Research (01/2025)
Algorithm Event- Erent-triggered GP-UCB (ET-GP-UCB).
1: Define: GP(O, k), X C R", 6B E (0, 1), D1 = 0, lower reset bound N upper reset bound N, & =
2: for t 1,2....... do
3: Train GP model with the current data set D.
4: Choose 3e (e.g,, according to Theorem 1)
Select 2t = arg m IDD, (a) /B (x)) Sample next observation J/2 fxx + Wt
time-invariant posterior using (4)
Treeet + EVENT TRIGGER Di, (ye, xz), 5B)
evaluate (10) with (11)
8:
if (Treset and f, E [N, M)) or 4, = N then
> reset only if within reset window
9:
Reset dataset D1+1= (1r, }z and set 4 =
10: else
11:
Update dataset D1+1 = D. U ((V,, }z}} and set t, = ty +
Lemma 2 Pick 5 E (0,1) and set w² 20 In "g. where Ex.21 "y. 1, ne, 0. Then, the noise sequence in Assumption 1 obtained since the last reset satisfies wr,, S Wr, for all t, 2 with probability at least 1- 8.
With Lemma 2, can extend Lemma to build probabilistic error bound that can act as the test and corresponding threshold functions of our event Irigger.
Lemma 3 Let fx) follow Assumption 2 with & = O. Pick ÓB E (0. and set Pt. = 21n 2
on where
Σ1,21 Tt 1, Tt, > O, Also set wł,, = 20 In 2 Then, observations yt under Assumption 1 satisfy D1 PD, (2,)) S vPr, (x+) + Wi for all t,
Ag probability least 1 8p.
We can now formally define the event trigger for ET-GP-UCB following the framework outlined in Defini- tion 2. Leveraging the result from Lemma 3, choose as the test and threshold function
t/ (Dr; (zt, vb)) = ye MDD, (ax)) and (Dr, (2t, = V (ara) ) + Wx,..
(11)
= test function v, in Def. 2
= threshold function in Def. 2
respectively. With this, we have designed a principled event trigger to detect changes in the objective function. Note that our event trigger monitors for changes in the objective function only at the current query location and cannot detect changes anywhere elsc. Our problem sctting restricts an algorithm to querying the objective only once per time step. The resulting algorithu ET-GP-UCB then as follows: We perform standard GP-UCB as long as the event trigger evaluates to 'reset 0. When the event trigger is activated the objective function fe has likely changed, and the data has bocome stale. If t, E [N, N1, ET- GP-UCB resets the dataset and restarts the optimization. This is summarized in Algorithm 1. The single design parameter of our event. trigger is oB in Lemma 3, which has a well-understood meaning as defining the tightness of the probabilistic uniform error bound. Since resets are confined to the range between the lower and upper reset thresholds, ET-GP-UCB guarantecs a regret no worse than that specified in Corollary 1.
6 Empirical Evaluations
In the following, we will first investigate the empirical impact of N and N on the reset times of the algorithm as well on the regret. In Section 6.2, we will benchmark our algorithm against other baselines on various synthetic and real-world problems and end with practical extensions for our proposed algorithm.
6.1 Empirical Impact of N' and Ñ on Reset Times and Regret
Impact on the Reset Times To analyze the impact of N and Ñ the behavior and performance of ET-GP-UCB, we first define the window size in which the event trigger can reset the data set W = N - Further, note that the time of reset using (11) in the event trigger framework is a random variable as it depends on the realization of ft as well as the observation model. The ridgeline plot in Figure 2 (left) shows the empirical distribution of this reset time as (scaled) histograms: starting from the reset time of R-GP-UCB, the window size increases. For this, we ran ET-GP-UCH on 30 000 different objective functions following Assumption 2, so-called within-model comparisons (cf. Section 6.2.1), for each window size. Small window sizes cut off the probability mass in the tails of the histograms, while for larger window sizes, the histogram resembles an inverse gamma distribution.
7

---

Published in Transactions on Machine Learning Research (01/2025)
Constant reset time of R-GP-UCB
0.01
0.05
R-GP-UCB
E 0.03
Reset at N
N N
0
Window size W
27
Mean empirical Rr
200
55
110
150
137
165
193
100
0
50
100
150
200
50
100
150
200
Reset time (s 0.03)
Window size W
Figure 2: Lest: Influence of the window size on the empirical PDF of reset times. For small window sizes, the probability mass in the tails is cut off through N and N. As the window size W increases, hence the flexibility of our event trigger increases, the histogram resembles an inverse-gamuna-lke distribution. Right: Influence of the window size on the empirical regret for different E. As the window size increases, the regret of FF-GPP-UCB (markers) decreases always staying below R-GP-UCB (dash-dotted lines). Increasing the flexibility of our trigger empirically reduces regret. Always resetting at N (dashed line) shows increased empirical regret. This indicates that Corollary 1 is a relatively loose bound for our trigger.
Impact on the Regret Figure 2 (right) demonstrates how increasing the window size affects regret for different E. As the window size grows, ET-GP-UCBE performance (lines with markers) consistently remains below the regret of R-GP-UCB with known (dash-dotted lines). This suggests that even coarse bounds lead to robust performance, which is desirable from practical viewpoint. The improved performance is contrary to the regret bounds in Corollary 1, which gets larger the window size grows. However, Corollary gives regret bounds for all triggers, including the worst-case. Empirically, the proposed event trigger performs much better than the worst-case and even improves on R-GP-UCB despite less prior knowledge of the rate of chance. ET--P-UCB performs best for larger window sizes. This suggests that prior knowledge of the rate of change is less important than the regret bounds in Corollary 1 imply. The proposed event trigger uses the gained flexibility from the theoretical investigation to perform resets when necessory within the reset window. Other event triggers may not show the same behavior: resetting, c.g., always at the lower bound N (dashed lines in Figure 2) exhibits increased regret with growing window sizes highlighting the importance of choosing suitable resetting strategy for good empirical performance.
6.2 Empirical Evaluation on Synthetic and Real-World Data
Using both synthetic data and standard BO benchmarks, we compare ET-GP-UCB to state-of-the-ar, GP- UCB baseline for TVBO. Specifically, we compare against R-GP-UCB, which resets periodically, TV-GP- UCB (Bogunovic et al., 2016), which gradually forgets past data points, and UI-TVBO (Brunzema et al., 2022), which continuously injects uncertainly al past data points. We further compare these time-varying algorithms to the time-invariant GP-UCB (Srinivas et al., 2010). In our experiments, we find that:
(1) The adaptive resetting of ET-GP-UCB always outperforms periodic resetting:
(i) ET-GP-UCB outperforms all baselines on synthetic data if the rate of change is misspecified; (Ml) ET-GP-UCB outperforms all baselines on real-world data (where the rate of change is unknown).
We reiterate that ET-GP-UCB does not rely on knowledge about the true rate of change. To highlight the generalization capabilities of ET CT-GPP-CBB to different scenarios, use the same parameter for the event trigger (0g 0.1 in Lemma 3) in all experiments. A sensitivity analysis of the hyperparameter óg on the regret is in Appendix B and shows that ET-GP-UCB performs similarly for wide range of 6g.
As in (Bogunovic et al., 2016) and (Srinivas et al., 2010), we utilize a logarithmic scaling 3, as Br = (ddn())), where 8 e In(czt). This approximates B, in Theorem 1 and allows for a direct comparison to Bogunovic et al. (2016), as they suggest to set 61 0.8 and c2 = for a suitable experiments in this section conducted using Torch (Balandat et al., 2020) and GoyThrch (Gardner
tradc-off. All
8

---

Published in Transactions on Machine Learning Research (01/2025)
Table 1: futhin-model comparisons. Top: Algorithms with time-invariant updates following (4). Bottom: Algorithms with more compute intensive timevaryiig updates similar to (5). The best median performance for each section is highlighted in bold, and in green for the best performance across top and bottom.
E = 0.01
= 0.03
= 0.05
Miss. = 0.001
Miss. e 0.2
Algorithm
Ry /T (4)
RT/T (4)
RT/T (4)
RT (L)
Rr /T (1)
GP-UCB
0.748 0.904
0.630
1.051
1.243
0.924
1.276
1.377
0.088
1.276 1. 1.054
1.877
1.276 1 Luus
1.377
R-GP-UCB
0.622 0.681
0.831
0.770
0.894
0.985
1.045
0.902 829
0.585
1.054 0.996
1.119
ET-GP-UCBa
0.604 0.531
0.682
0.778
0.879
0,966
0.879 0.824
0.879 0.824
0.720
824
ET-CP-UCBa
0.507 0.448
0:581
0.688
0.654
782
0.866 0.927
0.866 0.806
0-927
0.866 0.806
0.927
ET-GP-UCBL6
0.483 0.407
6868
0. 639
0.849 0. 748
890
0.849 0. 748
0.849 0.748
TV-GP-UCB
0.299 0.865
50000
0.622 0.582
0.960 1-036
1.223 1.
0.447
UI-TVBO
0.351 D.311
0.376
0.647 0,600
0.678
0.868 0.833
0.504
0.953 D. 8.8
1. 1.072
1.381 1.340
1.417
* Our algorithms with formatting ET-GP-UCB
upper bound on
True rate of change is = 0.05.
lower bound on e
et al., 2018), and full details are in Appendix A. All results show the median and interquartile performance. In the following, we indicate the direction of better performance with upward (f) and downward (4) arrows.
6.2.1 Wodhin-Model Comparison
First, we consider the same two-dimensional setting for synthetic data as in Bogunovic et al. (2016). We choose the compact set to be X = [0, 1]² and generate 50 different objective functions according to Assump- tion 2 with a known E, a SE kernel with fixed lengthscales ( = 0.2, time horizon T = 400, and a noise variance of of = 0.02. The setting in which all the hyperparameters are known to the algorithm referred to as wodein-model comparison (Hennig & Schuler, 2012); such comparisons are used to evaluate an algorithm's 's performance given all assumptions are fulfilled (cf. Appendix A.1 for further details on the creation of the within-model objective functions). Here, TV-GP-UCB acts as the reference case with full information. It shows the best performance an UCB algorithm can obtain in this setting as it takes Assumption 2 explicitly into account by using the time-varying posterior updates in (5). R-GP-UCB is also parameterized using the true E for the periodic reset time as Neoust = (T, 12c 11411 (Bogunovic et al., 2016, Corollary 4.1). For ET-GP-UCB, we compare three different variants with different lower and upper bounds on & and set N = [T, 12g- 11/]] and N = [min[[, 12e-1 according to Corollary 1. For UI-TVBO, we set o% e (cf. Brunzema el al. (2022)). Table shows the normalized regret (median 975% ) for different e. We can ob- serve that in all cases the ET-GP-UCB variant with the largest flexibility of the event trigger outperforms all basclines with a time-invariant GP model (top). As expected, TV-GP-UCB is the best-performing algorithm if the rale of change is correct. However, if the rate of change is misspecified (right columns), ET-GP-UCB also outperforms the baselines with varying GP model as it does not rely exact knowledge of and can react to the realized changes of ft.
925%
6.2.2 Real-World Temperature Data
To benchmark the algorithms on real-world data, we use the temperature dataset collected from 46 sensors deployed at Intel Research Berkeley over eight days at minute intervals. 3 This dataset was also used as a benchmark in previous work on GP-UCB and TVBO (Srinivas et al., 2010; Krause & Ong, 2011; Bogunovic et al., 2016). The objective is to activate the sensor with the highest temperature each time step resulting in ppatio-tempora monitoring problem. In our TVBO setting, fo consists of all sensor readings at time step f, and an algorithm can query only single sensor every 10 minutes. Hence, the regret following Definition 1 is the difference between the maximum and the recorded temperature by the activated sensor.
For each experiment, two test days are selected, while the preceding days serve as the training set. Using this training set, all data is normalized to have a mean of zero and a standard deviation of one. The empirical covariance matrix is computed based on the training set and employed as the kernel for all algorithms as in Bogunovic et al. (2016). The sensor data of the days and 8 is shown in Figure 3 (a) and an overview of the remaining days is given in Appendix E. For TV-GP-UCB and UI-TVBO, we set = o² 0.03 and the
We thank the researchers at Intel for the publically available data set under; https: htae:///bb.cce wwt.b//.aal.....mmmmm.........ma wtt.b//sal.mi.......... wtt..ee//laaaaaaaaaaaaaaa hladaaaaaalaaaaamll
9

---

Published in Transactions on Machine Learning Research (01/2025)
GP-UCB
TV-GP-UCB
UI-TVBO
R-GP-UCB TV-GP-UCB (MLE) ET--G--CCB (ours)
3
Norm. Temperature (
Re/t (t)
2
2
Resets of R-GP-UCB
Resets of ETT-GP-UCB
0
20
Runs
12
24
48
12
36
100
200
300
Time t [b]
Time t [h]
Time [h]
Time steps t
(a) Temperature data
(b) Days 7 & 8
(c) Days 5 & 6
(d) Policy Search
Figure 3: Real-world examples. Subfigure (a) shows the temperature data from the test days used in Bogunovic et al. (2016). In (b), the performance of the algorithms on these two days (lop) and the resets of R-GP-UCB and ET-GP-UCH (bottom) are displayed. Subfigure (c) shows the performance on other test days. ET-GP-UCH displays superior performance improvement compared to all other algorithms. For these experiments on real-world data, we use the same parameter for the event trigger in all previous experiments whereas TV-GP-UCB and R-GP-UCB relied on estimating E. Lastly, in subfigure (d), ET-GP- UCB also outperforms all competing baselines on a policy search benchmark. For all methods, the median performance and the interquartile (25% and 75%) are displayed.
periodic reset time R-GP-UCB to Noonst = 15 according to Bogunovic et al. (2016). For this benchmark, we further compare against TV-GP-UCE with online maximum likelihood estimation (MLE) of æ as one might choose to do in practice if the true rate of change is unknown. Note that this significantly increases the computational overhead. For ET-GP-UCB, as the same parameter for the event trigger in all the previous experiments.
the best-performing variant form Sec. 6.2.1 as well
Figure 3 (b) presents the normalized regret (top) for each algorithm over 20 runs and the resets of R-GP- UCB and ET-GP-UCB (bottom). In this real-world benchmark, ET-GP-UCB outperforms all the competing basclines. highlights that using a constant rate of change in the GP model as in TV-GP-UCB can be problematic for real-world problems. ET-GP-UCB also shows the best performance on other test days as shown Figure 3 (c). considering the larger interquartile of TV-GP-UCB. While ET-GP-UCB and TV-GP- UCB have similar median performance, ET-GP-UCB displayed significantly smaller interquartile range, indicating more consistent and reliable results. From the bottom figure, of ET-GP-UCB. Considering the timing of resets of ET-CP-UCB, it appears that especially when changes in the objective function are more distinct (c.g., changes between night and day), the proposed event trigger framework is empirically beneficial compared to using an estimated constant rale of change.
observe the adaptive nature
6.2.3 Policy Search in Time-Varying Environments
We also compare all algorithms on four-dimensional policy search benchmark of a cart-pole system as proposed in (Brunzema al., 2022). Given the system dynamics of the cart-pole system, it is possible to caleulate the optimal policy and thus calculate the regret following Definition 1. To induce change in the objective function, increase the friction in the bearing at 100 by factor of 2.5. We expect the event trigger of ET-GP-UCB to detect this change in the objective function and converge to the new optimum after resetting the dataset. In contrast to previous examples, we also perform online hyperparameter optimization of the spatial lengthscales using gamma priors to guide the search (all parameters are listed Appendix H). For TV-GP-UCH
we choose e 0.03 and we pick the constant reset timing of R-GP- according Bogunovic et al. (2016, Corollary 4.1). For ET-GP-UCB, we again choose the same parameter for the event trigger in all previous examples.
10

---

Published in Transactions on Machine Learning Research (01/2025)
R-GP-UCB
ET-GP-UCB
R.GP-UCB (N
mean time of (T-GP-UCR)
Table 2: Online MLE hyperparameter optimiza- tion of the GP when using CT-G..UCB.
1.0
0.8
= 0.01
& = 0.03
= 0.05
RafT
Algorithm
RT/T (1)
RT/T (4)
RT/T (4)
0.6
GP-UCB 0.835
1.260
1.405
0.4
R-GP-UCB
0.776
0.998
1.123
0.01
0.02
0.03
0.04
0.05
ET-GP-UCB- 0.612
0.870
1.057
Rate of change
*with "learn-then-monion approach
Figure 4: Impact of Reset Frequency Timing.
The results over 20 runs displayed in Figure 3 (d). We can observe that ET-GP-UCB is the best performing algorithm since it can detect the change in the objective function and converge to the new optimal solution. We can further see that TV-GP-UCB and R-GP-UCB, which rely on an ongoing and smooth change in the objective function, empirically struggle to cope with such sudden changes resulting in significantly increased regret. While UI-TVBO can also recover from the change, ET-GP-UCB still outperforms this baseline. The example also highlights that a constant reset time that is too short can yield divergent behavior in digher-dimensional settings and non-smooth changes.
6.2.4 Ablation on the Impact of Reset Frequency vs. Reset Timing
To better understand the source of empirical improvement of ET-GP-UCB, we conducted an ablation study to Lest. whether the performance gains are due to resetting at the correct frequency or at the correct time steps. Specifically, we investigated the effect of using the average resetting time of ET-GP-UCB, derived from Monte-Carlo simulations of the event trigger, as the reset interval N for R-GP-UCB, Figure 4 shows mithin-model experiments across multiple rate of changes for 50 different seeds. The results demonstrate that resetting at the average frequency of the trigger improves regret performance at low rates of change. However, as the rate of change increases, adaptively resetting at the correct time steps, ie., considering the realized changes in the objective function, becomes crucial for minimizing regret.
6.3 Practical Extensions of ET-GP-UCB
Next, we discuss practical extensions to ET-GP-UCB that can enhance its performance in practice.
Upper Bound on the Noise Imposing a fixed upper bound on the noise term w., after a certain number of time steps can be practical. While the gradual increase of w, is crucial for Lemma to hold, it results in large thresholds in (11) over time. This might lead to undelected changes, especially when combined with large N and rare changes in the objective. Considering bounded noise in Assumption 1 instead of Gaussian noise might also be more suitable for some applications.
Online Hyperparameter Optimization Practitioners should be cautious when using naïve MLE for all hyperparameters, as our algorithm fits time-invariant GP to time-varying data. This can inflate the noise variance, again resulting in large thresholds in (11). The core issue lies in the dual use of the model mismatch: the trent-rrigger mechanism is designed to detect model mismatches, while hyperparameter optimization aims to minimize this mismatch. These two objectives conflict. We suggest estimating the noise on hold-out data, as demonstrated in Section 6.2 with the temperature dataset. If this is not feasible, we recommend using reasonable priors and constraints for all hyperparameters and then performing maximmmm-a-poseere estimation, as is common practice for most GP and BО libraries (GPy, 2012; Gardner et al., 2018; Balandat et al., 2020) and demonstrated in the policy search example. In the case of no prior knowledge, a simple "learn-then-monitoo"" approach, i.e., learning the parameters and freezing them afterward while monitoring for changes, can be helpful. To demonstrate this, we conducted experiments with online hyperparameter optimization for all basclines using only very coarse constraints. Specifically, we use a lower bound of 0.001 and an upper bound of 0.1 for the noise (true noise value 0.02) and a lower bound of 0.01 and an upper bound of 1 for the lenaths (true lengthscales are 0.2). For ET-GP-UCB,
the first. 2 d time steps
11

---

Published in Transactions on Machine Learning Research (01/2025)
Norm. cumulative value
R-GP-UCB ET-CP-UCB
R-GP-UCB
ET-GP-UCB
TV-GP-UCB ET-GP-UCB w. backtracking
TV-GP-UCB
ET-GP-UCB backtracking
Runs E, E,fiff)ft [
Consecutive triggers
3.0
2
ftx))/tt
Input dimension: d=8 8
1
Forgetting factor: = 0.01
2.0
Resets of respective algorithm
330
340
W
20
1.5
100
200
300
400
Time t
Input dimensions d (s = 0.01)
Figure 5: Lefl: Performance an eight-dimensional within-model objective function. Retaining portion of past data using Algorithm 2 leads to significant performance boost, nearly matching the results of T--GP- UCB, even without prior knowledge of Right: Final performance for different dimensions (d E {2, 4, 6, 8)) with a fixed E. As dimensionality increases, the benefit of retaining data becomes more pronounced.
after a trigger event learn the noise variance and individual lengthscales, and then freeze the parameters and start monitoring for changes using event trigger. Table 2 shows that this approach decouples the two conflicting objectives, yielding reasonable performance without prior knowledge of E, the spatial hyperparameters, nor the observation model.
Retaining Past Information Exploring alternative strategies to augment the data set instead of perform- ing a full reset every time the event. trigger activates is interesting to preserve the most recent information. In this paper, we demonstrated that even with a full reset we obtain superior performance on commonly used real-world benchmarks but preserving information can become crucial for higher-dimennsioall problems. One idea that arises directly from our event- trent-trigeeed setup is to implement a "backtracking" mechanism when change is detected: after detecting change, the algorithm backtracks to check how many of the most recent data points still satisfy the trigger bounds. These relevant observations are then retained for further optimization. By re-using the event. trigger (o determine which data to keep, we ensure that, in the event of sudden changes, there are no inconsistent data points in the data set. We cap the re-initialization size of the data set by 2 d; similar data set sizes are commonly used to initialize BO algorithms (Eriksson et al., 2019; Balandat et al., 2020; Müller et al., 2021). The backtracking procedure summarized in Algorithm To test this approach, we conduct within- model comparisons for different dimensions summarized in Figure 5. Retaining some of the past data proves to be very beneficial. In Figure 5 (left), we can see that by using Algo- rithm 2, ET-GP-UCB is almost able to match the performance of TV-GP-UCB even with- out prior knowledge of c. Notably, combin- ing data reuse and smooth changes can yield consecutive resets, occasionally transforming the method into adaptive sliding window approach. From a computational standpoint, this poses no significant overhead, as back- tracking through the dataset and evaluating the event trigger are computationally efficient.
A Algorithm Backtracking procedure to retain data
procedure BACKTRACKING(D. EVENT TRIGGER)
Droup t- 0
> IInitialzze an empty data set for [D downto do
(yi, v;) t D(i] Extract input-output pair
5:
Treset +- EVENT TRIGGER(DReg) (1s, m:))
if Presot [Droep 2-d d then
break
Stop backtracking
8: Dxeep + Dusep U (11,
return Dreep
Figure 5 (right) shows that as the dimensions increase, using a strategy that reuses some of the past data becomes more beneficial. The results for more forgetting factors in Appendix A.
12

---

Published in Transactions on Machine Learning Research (01/2025)
7 Conclusion and Future Work
This paper presented the novel concept of event- trint-triggered resets to optimize time-varying objective functions with TVBO. Our theoretical results show that algorithms that reset between lower and upper thresholds yield meaningful regret even if the rate of change is unknown. This allowed us to optimize the actual reset timing for increased empirical performance using probabilistic error bounds. The resulting algorithm, ET- GP-UCB, adapts to the realized changes in the objective function without relying on exact prior knowledge of these temporal changes, making it especially useful in real-world scenarios. We empirically showed that the adaptive resetting of ET-GP-UCB consistently outperforms periodic resetting and further demonstrated ET-GP-UCB'ss increased performance on real-world benchmarks. Lastly, we presented practical extensions to our algorithm. These extensions included an approach for practical online hyperparameter estimation within the event trigger framework and an approach to retaining past information, which further increases performance in higher-dimensional scttings. The latter leverages the event trigger not only to detect changes but also to determine how much recent data can be reused, thereby improving sample efficiency. Overall, ET-GP-UGE UCB offers an adaptive solution for optimizing time-varying objectives with minimal hyperparameter tuning and prior knowledge required, making it a valuable tool for many time-varying optimization problems.
An interesting direction for future work on ET-GP-UCH is its further theoretical analysis. While Corollary 1 ensures the desirable scaling of the regret for worst-case resets, it does not reflect the improved empirical regret when increasing the window size as the event trigger is not explicitly included in the analysis. Regret bounds that explicitly consider the choice of event trigger are a promising direction for future research and would bridge the current gap between theoretical analysis and empirical performance. For further discussion, see Appendix C. Another research direction would be to move away from a sctting solely focused on dynamic regret and consider e-g,, a setting with multiple comparators that are fixed for certain window lengths (Zhang et al., 2020). Here, we believe that adaptive approaches as proposed herein can be especially beneficial. Lastly, investigating different event triggers within the framework of Definition 2 is interesting, such as event triggers that consider multiple time steps in their test and threshold function to, c.g., identify slow trends in the objective. New and tailored data augmentation strategies may also arise with new event. triggers or event triggers tailored to specific applications.
Acknowledgments
The authors thank Pierre-François Massiani, Dominik Baumann, Christian Fiedler, and Noel Brindise for their helpful comments and discussions, as well as the reviewers and action editor for their constructive feedback, which helped improve the final version of the paper. This work is partially funded by the Deutsche Forschugsgemeinsccaatt (DFG, German Research Foundation)-RTT 2236/2 (UnRAVeL.). Simulations were performed with computing resources granted by RWTH Aachen University under project rwth1579.
References
Yasin Abbasi Yadkori, András György, and Nevena Lazič. A new look at dynamic regret for non-stationary stochastic bandits. Journal of Machine Learning Research, 24(288), 2023.
Karl Johan Aström and Björn Wittenmark. Adaptive control. In Mathematical System Theory: The Influence of RE Kalman. Springer, 1995.
Peter Auer, Pratik Gajane, and Ronald Ortner. Adaptively tracking the best bandit arm with an unknown number of distribution changes. Tn Conference on Learning Theory, pp. 138-158. PMLR, 2019.
Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy. BoTorch A framework for efficient Monte-Cari Carlo Bayesian optimization. In Advances in Neural Information Processing Systems, 2020.
Omar Besbes, Yonatan Gur, and Assaf Zeevi. Non-stationary stochastic optimization. Operations Research, 63(, 2015.
Iija Bogunovic, Jonathan Scarlett, and Volkan Cevher. Time-varying Gaussian process bandit optimization.
In International Conference on Artificial Intelligence and Statistics. PMLRR 2016.
13

---

Published in Transactions on Machine Learning Research (01/2025)
Paul Brunzema, Alexander von Rohr, and Sebastian Trimpe. On controller tuning with time-varying Bayesian optimization. In IEEE Conference on Decision and Control, 2022.
Roberto Calandra, André Seyfarth, Jan Peters, and Mare Peter Deisenroth. Bayesian optimization for learning gaits under uncertainty: An experimental comparison dynamic bipedal walker. Annals of Mathematics and Artificial Intelligence, 76:5 23, 2016.
Yifang Chen, Chung- Wei Lee, Haipeng Luo, and Chen-Yu Wei. A new algorithm for non-stationary con- textual bandits: Efficient, optimal and parameter-free. In Conference on Learning Theory, pp. 696 726. PMLR, 2019.
Yutian Chen, Aja Huang, Ziyu Wang, Ioannis Anlonoglou, Julian Schrittwieser, David Silver, and Nando de Freitas. Bayesian optimization in AlphaGo. vrXio:1812.06855, 2018.
Sayak Ray Chowdhury and Aditya Gopalan. On kernelized multi-armed bandits. In International Conference on Machine Learning. PMLR, 2017.
Lionel Colliandre and Christophe Muller. Bayesian optimization in drug discovery. High Performance Computing for Drug Discovery and Biomedicine, 2023.
Marcello Colombino, Emiliano Dall' Anese, and Andrey Bernstein. Online optimization feedback con-
troller: Stability and tracking, IEEE Transactions on Control of Network Systems, 7(1), 2019.
Thomas Dangl and Michael Halling. Predictive regressions with time-varying coefficients. Journal of Finan- cial Economics, 106(1):157 181, 2012.
Yuntian Deng, Xingyu Zhou, Backjin Kim, Ambuj Tewari, Abhishek Gupta, and Ness Shroff. Weighted Gaussian process bandits for non-stationary environments. In Proceedings of The 25th International Conference on Artificial Intelligence and Stalistics, volume 151. PMLR, 2022.
Matthew Ellis and Panagiotis D Christofides. Economic model predictive control with time-varying objective function for nonlinear process systems. ATCLE Journal, 60(2), 2014.
David Eriksson, Michael Pearce, Jacob Gardner, Ryan D Turner, and Matthias Poloczek. Scalable global optimization via local Bayesian optimization. Advances in Neural Information Processing Systems, 2019.
Peter Frazier and Jialei Wang. Bayesian optimization for materials design. Information Science for Materials Discovery and Design, 2016.
Jacob Gardner, Geoff Pleiss, Kilian Q Weinberger, David Bindel, and Andrew G Wilson. GPyTorch: Black- box matrix-matrix Gaussian process inference with GPU acceleration. In Advances in Neural Information Processing Systems, 2018.
Roman Garnett. Bayesian Optimization. Cambridge University Press, 2023.
Roman Garnett, Michael A Osborne, and Stephen J Roberts. Bayesian optimization for sensor set selection. International Conference on Information Processing in Sensor Networks, 2010.
Subhashis Ghosal and Anindya Roy. Posterior consistency of Gaussian process prior for nonparametric binary regression. The Annols of Statístics, 34(5):2413 2429, 2006.
GPy. GPy:
gaussian process framework in python. http://github. com wwl.ccoo//mSffee htt////effeedM//P......aa 2012.
Ryan-Rhys Griffiths and José Miguel Hornández-Lobato. Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chemical Science, 11(2), 2020.
Philipp Hennig and Christian J. Schuler. Entropy search for information--fffcient global optimization. Journal of Machine Learning Research, 13, 2012.
Antonia Holzapfel, Paul Brunzema, and Sebastian Trimpe. Event-triggered safe Bayesian optimization on quadcopters. In Learning for Dynamics and Control Conference. PMLR, 2024.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Andreas Krause and Cheng Ong. Contextual Gaussian process bandit optimization. In Advances in Neural Information. Processing Systems. Curran Associates, Inc., 2011.
Christopher König, Matleo Turchelta, John Lygeros, Alisa Rupenyan, and Andreas Krause. Safe and elficient model-free adaptive control via Bayesian optimization. In IEEE International Conference on Robotics and Automation, 2021.
Roman Marchant and Fabio Ramos. Bayesian optimisation for intelligent environmental monitoring. IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012.
Roman Marchant, Fabio Ramos, and Scott Sanner. Sequential Bayesian optimisation for spatial-temporal monitoring. In Conference on Uncerlainly in Artificial Intelligence. Citeseer, 2014.
Sarah Müller, Alexander von Rohr, and Sebastian Trimpe. Local policy search with Bayesian optimization. Advances in Neural Information Processing Systems, 34:20708 20720, 2021.
Matthias Neumann-Brosig, Alonso Marco, Dieter Schwarzmann, and Sebastian Trimpe. Data-efficient auto- tuning with Bayesian optimization: An industrial control study. IEEE Transactions on Control Systems Technology, 28(3), 2019.
Christos H Papadimitrion and John N Tsitsiklis. The complexity of optimal queueing network control. In Conference Struclure in Complexily Theory. IEEE, 1994.
Jack Parker-Holder, Vu Nguyen, and Stephen J Roberts. Provably elficient online hyperparameter optimiza- tion with population-based bandits. In Advances Neural Information Processing Systems, 2020.
Jack Parker- Parker-Holder, Vu Nguyen, Shaan Desai, and S Roberts. Tuning mixed imput hyperparameters on the fly for efficient population based AutoRD. In Advances in Neural Information Processing Systems, volume 34, pp. 15513 15528. Curran Associates, Inc., 2021.
Giorgio E Primiceri. Time varying structural vector autoregressions and monetary policy. The Review of Economic Studies, 72(3):821 852, 2005.
Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in Neural Information Processing Systems, 20, 2007.
Andrea Simonetto, Emiliano Dall' Anese, Santiago Paternain, Geert Leus, and Georgios B Giannakis. Time- varying convex optimization: Time-structured algorithms and applications. Proceedings of the IEFE, 108 (11), 2020.
Aleksandrs Slivkins and Eli Upfal. Adapting to changing enviromment: the Brownian restless bandits. In Conference on Learning Theory, 2008.
Jasper Snock, Hugo Larochelle, and Ryan P Adams. Practical Bayesian optimization of machine lcarning algorithms. Advances in Neural Information Processing Systems, 2012.
Friedrich Solowjow and Sebastian Trimpe. Event-triggered learning. Automatico, 117, 2020.
Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In International Conference Machine Learning, 2010.
Jie Su, Junfeng Wu, Peng Cheng, and Jiming Chen. Autonomous vehicle control through the dynamics and controller learning. IEEE Transactions on Vehiculor Technology, 6777):560)) 2018.
Joe Suk and Samory Kpotufe. Tracking most significant arm switches in bandits. In Conference on Learning Theory. PMLR, 2022.
Jonas Umlauft and Sandra Hirche. Feedback linearization based on Gaussian processes with trent-triggere online learning. IEFF Transactions on Automatic Control, 65(10), 2019.
15

---

Published in Transactions on Machine Learning Research (01/2025)
Chen-Yi Wei and Haipeng Luo. Non-stationary reinforcement learning without prior knowledge: An optimal black-box approach. In Conference on Learning Theory, pp. 4300-4354. PMLR, 2021.
Peter Whittle. Restless bandits: Activity allocation in changing world. Journal of Applied Probobility, 25 (A):287- 297, 1988.
Lijun Zhang, Shiyin Lu, and Tianbao Yang. Minimizing dynamic regret and adaptive regret simultaneously In International Conference on Artificial Intelligence and Statistics, pp. 309 319. PMLR, 2020.
Xingyu Zhou and Ness Shroff. No-regret algorithms for time-varying Bayesian optimization. In Annual Conference on Information Sciences and Systems, pp. 1 6. IEEE, 2021.
11

---

Published in Transactions on Machine Learning Research (01/2025)
Supplementary Material for " "Event-Triggered Time- Morring Bayesian Optimization"
A Additional Results and Details on Experimental Design
13
A.1 Creating Within-Model Objective Functions
13
A.2 Additional Results on Within-Model Objective Functions
13
A.3 Comparison to Performance-based Triggers
13
A.4 Performance on Standard Optimization Benchmarks
15
A.5 Comparisons to Algorithms in the Finite Variational Budget Setting
................................
B Sensitivity of ET-GP-UCB to the Event Trigger Hyperparameter 6B
19
C Discussion on the Theoretical Analysis and Future Directions
20
D Influence of the window size on the distribution of resets for different
21
E Overview of the Temperature Data
22
F Proof of Theorem and Corollary from Section
22
F.1 Proof of Theorem 1
22
F.2 Proof of Corollary
25
G Proof of Lemma 2 and Lemma
27
H Hyperparameters
27
Following is the technical appendix. Note that all citations here are the bibliography the main document, and similarly for many of the cross-references.
12

---

Published in Transactions on Machine Learning Research (01/2025)
A Additional Results and Details on Experimental Design
This section contains details about the design of the experiment for the results in Section 6.2 for repro- ducibility.
A.1 Creating Within-Model Objective Functions
To create the within-model objective functions, we take two different approaches. For the two-dimensional objectives in Section 6.2.1, we draw sample paths Ji from a GP with the corresponding hyperparameters on a dense equidistant grid of the compact set X = [0,1]² with 100 x 100 input locations. To create the time-varriii ame-varying objoctive function ft we then combine the sample paths following Assumption 2 with specifiod rale of change E and perform linear interpolation in the dense grid. For the higher-dimensional within-model objective functions in Section 6.3, this procedure is not feasible. Therefore, approximate prior samples ge with a M random Fourier features (RFFs) following Rahimi & Recht (2007). This yields set of parametric functions
M
9: (x)) = Σ
Wm 1m (x)) with @m(()) = M m + Tm))
V cos (0
(12)
Here, Om are sampled proportional to the kernel's spectral density and Tm ~ ((0022m) These approximate sample paths again can be combined following Assumption to yield the objective function f1(x) given e. In all experiments in Section 6.3, we use M = 1028 RFFs. This approach does not require interpolation in the input space since the approximate sample paths are parametric.
A.2 Additional Results on Wodhin-Model Objective Functions
GP-UCR R-GP-UCB
TV-GP-UCR ET-GP-UCB
Here, we list some of the additional results on within-model objective functions. On the right in Figure 6, we conducted an experiment with a change in the rate of change after 175 time steps (segment highlighted in gray). We initialize all algorithms that require epsilon as an input to their algorithm with the rate of change at time step O. In this exper- iment, the adaptive nature of our algorithm becomes evident. After the change in the rate of change E, ET-GP-UCB starts to increase the reset. frequency adjusting to the higher rate of change in the un- known objective. We can observe how TV-GP-UCB with the too small rate of change starts diverge over time. While the difference for 500 time steps is not as big, there is a clear diverging trend.
0.002
0.1
1.0
R,/t(())
0.5
0.0
Reseta of respective algorithm
50
Runs
100
200
300
400
500
Time
Figure 6: Within-model with change in E.
We next further visualize the results from Table 1 in Figure 7. Here, we only list the best-performing variant of ET-GP-UCB to increase the readability of the plot. Lastly, Figure 8 shows the results of using Algorithm 2 to keep some of the past information for different rate of changes. We can observe that using the backtracking procedure improves the performance for all dimensions for the given rate of changes.
A.3 Comparison to Performance- based Triggers
Another idea that arises naturally for an event trigger to consider reset strategy that is solely basod on an improvement in performance. This strategy is inspired by trust-region BO, TuRBO (Eriksson et al., 2019), where lack of improvement "triggers" a refinement of the search domain. We apply variant of this mechanism to the time-varying setting: If an algorithm is not able to improve its performance for Tiail N4 time steps in a row, the event trigger resets the data set. To set this threshold, we use the same heuristic proposed in the TuRBO paper and set Thail = [D/4] = D (c.f. Appendix D in Eriksson et al. (2019) and our batch size is q = given our problem setting). We refer to this event trigger as TuRBO Trigger. To test its performance, we ran the same benchmark from the previous scction, i.c., multiple within-model functions generated using RFFs (cf. Section A.1) with different rates of change and different dimensions. The results are in Figure 9.
13

---

Published in Transactions on Machine Learning Research (01/2025)
GP-UCB R-GP-UCB TV-GP-UCB
UI-TVBO ET-GP-UCB
0.01
1.0
Raji
0.5
Etrue = 0.05
Emizep. 0,2
0.0
Resets of R-GP-UCB
Resets of ET-GP-UCR
Runs
200 Time steps
400 0
200
400 0
200
100 0
200 Time steps é
100
Time steps
Time steps &
(a) Constant 8
(b) Constant
(c) Constant e
(d) Misspecified e
Top row: performance of the different algorithms in terms of normalized regret. The shaded areas show one standard deviation of the respective algorithms. Bottom row: reset timings of R-GP-UCB and ET-GP-UCB. In (a) to (c), rate of change is fixed for all time steps, and in (d), it is misspecified for TV-GP- UCB, UITVBBO, and R-GP-UCB. In (a) to (c), TV-GP-UCB is the full information reference, showing the best possible UCB performance with known constant E. We compare ET-GP-UCB to R-GP-UCB, the latter of which uses the true to determine its periodic reset time. For ET-GP-UCE, we use the same paramcter ōn in the event trigger for the test cases (a) to (d). In all cases, ET-GP-UCB outperforms R-GP-UCB. It also outperforms TV-GP-UCB and UI-TVBO if is misspecified, shown in (d), since ET-GP-UCB does not rely prior information The plots show the median performance and the interquartile.
Figure 7:
R-GP-UCB
ET-GP-UCB
UC-GP-UCB
ST-GP-UCB backtracking
3.0
2.5
2.0
ft (tx)/t
2.5
2.0
2.0
1.5
1.5
1.5
2
6
8
Input dimensions 0.01)
Input dimensions d (5 0.03)
Input dimensions (s 0.05)
Figure& 8: Benefit of reusing some of the past information through Algorithm for different e.
We can observe that for all rates of change, the TuRBO trigger performs worse than the other baselines. Unlike in TuRBO, a trigger event significantly impacts performance since all data discarded (while shrink- ing the trust region for TuRBO is less drastic). This is especially the case for dower-dimensional objective functions where only allow for few failures before the dataset is reset. An additional factor is that the objective function is noisy. Therefore, even consecutive evaluations at the optimum of non-changing function could yield a reset, which, in contrast to TuRBO, requires significant. new data lo recover from. While performance- based resets hold promise as they are not as tightly coupled to the prior assumptions of the GP, they require further in-depth investigation and algorithm design for the dynamic settings to yield well- cell-performing approaches by further including heuristics. Also, further tuning of Taail for each problem could improve its performance.
14

---

Published in Transactions on Machine Learning Research (01/2025)
R-GP-UCH TV-GP-UCB
ET-GP-UCH
TuRRO Trigger
ET-GP-UCB w. backtracking
3
fta))// (-)
2.0
2
2
1.5
1.0
1
0.5
2
4
8
2
8
2
6
8
Input dimensions d (s 0.01)
Input dimensions d (s = 0.03)
Input dimensions d (s 0.05)
Figure 9: Comparison to a performance-based event trigger (TuRBO Trigger).
A.4 Performance on Standard Optimization Benchmarks
Next, we will test our event trigger on some standard benchmarks, specifically Ackley (d= 2) and Hartmann (d ===) 6), which are popular for testing BO algorithms. The event trigger in ET-GP-UCB is tightly coupled the GP prior. We, therefore, expect some false positives of our event trigger on these benchmarks. Similar Scction 6.2.2 and Section 6.2.3, the comparisons in this section aim to test what extent resets affect the empirical performance. For Ackley, we set the time horizon to T = 60 and induce shift in the optimum after T/2 time steps. Similarly, we set Hartmann's time horizon to T 200 and induce the shift after T/2 time steps. To induce this shift, we perturb the input of the objective function by a constant factor.
We use the same UCB acquisition function stated in Section 6, run ten different seeds, and learn the lengthscales & online for all basclines. For the lengthscales, we use corse bounds as E [0.005, 4.] (Eriksson et al., 2019) and standardize the output data at each time step given the current data set which follows best practices in BO (Balandat et al., 2020). Since we learn the lengthscales online, theu-monitor" approach introduced Section 6.3 for the ETT-P-UCBB baselines and use the same On as in all previous experiments. For TV-GP-UCB, we set the rate of change to = 0.01 to yield good performance in the static setting but still allow for some adaptation.
apply the "learn-
Figure 10 shows the performance of all baselines. For Ackley, we can observe that at the beginning of optimization, there are rescts of both ET-GP-UCB variants. However, we also obscrve that these resets do not significantly impact the final performance (before the change) when considering simple regret. After the shift, we can observe that all the time-varye varying baselines can adjust to the change in the objective, and as expected, GP-UCB struggles to cope with the stale data in the data set.
On Hartmann, the event-triggered baseline without backtracking struggles and is outperformed by GP-UCB, Similar to Ackley we observe carly resets of both ET-GP-UCB variants. In this highsr-dimensional example, these resets are more costly for vanilla. ET-GP-UCB. For the backtracking variant, we can observe consecutive resets resulting in an event-triggen triggered sliding window (cf. Section 6.3). With this backtracking mechanism, the ET-GP-UCB variant outperforms GP-UCB significantly in terms of simple and cumulative regret. Similar to the Ackley example, most of the time-varying baselines can better adjust to the shift in the optimum compared to GP-UCB. Noteworthy is also that ET-GP-UCB with backtracking significantly outperforms GP-UCB in terms of simple regret even before the change. Removing some initial data allows this variant to achieve better GP hyperparamelers and converge to the optimum. This result raises further questions how classic BO algorithms should deal with past data for better convergence.
In summary, we observed that ET-GP-UCB does yield additional resets when optimizing an objective func- tion that not well explained by the GP prior, but the impact on final performance is relatively small for the considered test functions when leveraging the proposed backtracking mechanism. We further observed that this mismatch in prior and objective functions also harm static algorithms. Additionally, we observe
15

---

Published in Transactions on Machine Learning Research (01/2025)
GP-UCB ET-GP-UCB
GP-UCB TV-GP-UCB
ET-GP-UCH UCB
TV-GP-UCB ET-GP-UCB w. backtracking
-TT-GP-UCB backtracking
0
Hartmann (6D) Hartmann (6D)
Ackley (2D)
3
Best value
Best value
2
1
10
Shift
Shift
0
Ackley (2D)
15
Ra/t (+)
3
Re/t
10
Resets of respective algorithm
Resets of respective algorithm
Runs
10
Runs
10
10
20
30
40
50
60
50
100
150
200
Time
Time
Figure 10: Comparison on the standard benchmarks Ackley (left) and Hartmann (right). We show the best obtained value top and the normalized cumulative regret the bottom. We reset the best value algorithm as observed after the shift (indicated by the dashed line) to test how the algorithms cope with the shift in the optimum. All plots show the median performance as well as the interquantiles.
that the same algorithms developed to tackle time- airo-varying problems may also be helpful for static problems to overcome some problems with past data points, which is an exciting research direction.
A.5 Comparisons Algorithms in the Finite Variational Budget Setting
In this section of the appendix, we compare ET-GP-UCB to W-GP-UCB (Deng et al., 2022), and BR- GP-UCB and SW-GP-UCB (Zhou & Shroff, 2021). As discussed in Section 3, their underlying assump- tions differ from ours in that they assume fixed variational budget. Therefore, thcorctical compar- ison is not reasonable, but an empirical comparison, especially real-world data, is interesting. As the baseline that does not consider time-variations, we also compare against IGP-UCB (Chowdhury & Gopalan, 2017), which is an improved form of the agnostic GP-UCB of Srinivas et al. (2010). For the comparison, we use the code base submitted by Deng et al. (2022) the supplementary material of https://prooeeedingg wwweeediiinnmmm.m...mmmmeeeeeeee .wm.mmr.p...... aalr.prees//vl /11////nggd...... Their experiments consist of:
(a) an one-dimensional experiment with sudden changes at time steps = 100 and t 200, an one-dimensional experiment. with slowly changing objective function, an experiment on real-world stock market data.*
We refer to Deng et al. (2022) for the details on how the objective function for (a) and (b) composed. We implemented our algorithm ET-GP-UCB in their MATLAB code base and added random search as baseline.
While going through the implementation details, we noted that in the stock market data benchmark, the data not normalized and values of the objective function ranged from 19 to 250. This is problematic all algorithms use prior mean of zero and a prior variance of one. Thus, all observations will have a very small likelihood under the given model. Secondly, the noise variance was set to 300, which seems high for stock data where observation noise should be low. The high noise variance combined with the prior signal variance of one also explains the very high standard deviations all the algorithms in Deng et al. (2022,
*Not that stock market data docs not fit the fixed variational budget sctting stock prices constantly change. However, benchmark but wanted highlight that ongoing change the more
since this example used in prior work, suitable assumption.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Random
BR-GP-UCB SW-CP-UCB
W-GP-UCB ET-GP-UCB (ours)
ICP-UCB
x x10*
400
200
Regret R:
200
100
100
200
300
400
100
200
300
400
500
200
600
800
Time
Time
Time t
(a) Abruptly-change, SE kernel (b) Slowly-change, SE kernel
(c) Stock market data
Figure 11: Performance using the refactored GP prior in the experiments of Deng et al. (2022).
Figure 1 (c)). Overall, the stock market data has very low marginal likelihood for the model chosen in Deng et al. (2022). Therefore, we decided to improve the model and add preprocessing step to the stock market data example by normalizing the data to a mean of zero and variance of one to fit the prior model assumptions of all algorithms. Next, we chose a noise variance 0.01 to account for smaller short-term fluctuations in stock prices. We also changed the noise variance of the synthetic experiments (a) and (b) 0.1. It was previously set (same as the signal variance), amounting a 'signal noise ratio' of 1, which is very low for practical optimization problems. We then re-ran their experiments using 20 runs for the synthetic experiments and 5 runs for the stock market data as in Deng et al. (2022). For ST-GG--UCB, we use the same parameters for the event trigger in all previous experiments, as well as no hard upper and lower bounds, as this resulted in the best empirical performance in Section 6.2. Note that all other Lime-varying algorithms rely on specifying the amount of change By a priori, which hard to estimate in practice. The results are shown Figure 11 and regret at the last time step is listed in Table 3.
We can see that ET-GP-UCB can outperform baselines experiments Figure (a) (c). Especially Figure 11 (a), ET-GP-UCB can detect the sudden changes in the objective function and then quickly find the new optimum due to the simple one-dimensional objective function. In Figure 11 (b), ET-GP-UCB is the best performing algorithm, but W-GP-UCB is competitive. Considering the good performance of TGP-UCB, we can also conclude that the changes in the objective function over the time horizon are relatively small. In Figure 11 (c), ET-GP-UCB is again the best-performing algorithm. We also observe that in Figure 11 (c) all algorithms significantly improve in performance and yield a smaller standard deviation compared to Deng et al. (2022, Figure (c)) because the GP prior model more suitable for modeling the objective function. After approx. 500 time steps ET-GP-UCB, W-GP-UCB, SW-GP-UCB, and IGP-UCI (see Table 3), no longer accumulate regret. Looking at the objective function, this is expected the optimal bandit arm longer changes and the optimal values do not change significantly.
Table 3: Comparison on experiments in Deng al. (2022).
(a) Abruptly-change (b) Slowly-change (c) Stock market Regret (μ (u4o o)
Algorithm
Regret (#
Regret (o)
IGP-UCB
704.6+:238.9
136.0+332.2
11 1146.3+566.0
BR-GP-UCB
312.5-14.3
153.9+11.0
453362.214430.1
SW-GP-UCB
23..
111.6*+1.77
854.911277
W-GP-UCB
298.8+22..3
197. 197.0+:29.7
114...3+32.9
ET-GP-UCB (ours)
55.8
86.3
2483.8139.8
17

---

Published in Transactions on Machine Learning Research (01/2025)
Results with the Original Gaussian Process Prior
Below in Figure 12 are the results using the original GP prior from Deng et al. (2022). We can observe that for Figure (a), ET-GP-UCB can reliably detect. the first change in the objective function at = 100. However, due to the low tignal-to-noise ratio, ET-GP-UCB can only detect the second change approximately 50% of the time. It still outperforms periodic resctting and the sliding window approach. In Figure 12 (b); T1-GP-UCB shows similar performance W-GP-UCB. Both examples, however, highlight that ET-GP- UCB is not as dominant, in previous examples if the signal-to-noise ratio is too low since Wy will be large relative to 1y:- PD, (æt).. . For the stock market data, we observe the discussed large standard deviations. All algorithms outperform the random basclinc. However, we can observe that the ordering of algorithms has changed compared Deng al. (2022, Figure/ (c)) even though we used their implementations and only added ET-GP-UCB and the random baseline to the run script. Further, changing the random seed again changed this ordering of the algorithms, which indicates that the prior model assumptions yield random-like behavior the prior does not allow the GP model to make meaningful decisions.
Random
BR-GP-UCB
W-GP-UCB
IGP-UCB
SW-GP-UCB ET-GP-UCB (ours)
x10*
1000
Regret R
750
200
10
500
100
5
250
0
0
0
100
200
300
400
500
0
100
200
300
400
500
250
500
750
Time
Time
Time
(a) Abraptly-change, SE kernel (b) Slowly-change, SE kernel
(c) Stock market data
Figure 12: Performance using the original GP prior for the experiments of Deng et al. (2022).
18

---

Published in Transactions on Machine Learning Research (01/2025)
ög = 0.005 +
öø = 0.01
6g 0.05
+
ÓØ = 0.1
+
dB = 0.5
1.5
1.5
1.5
0.01
0.03
0.05
1.0
Reft
0.5
0.5
0.5
0.0
0.0
0.0
0
200
400
0
200
400
0
200
400
Time t
Time &
Time f
Figure 13: Scnsitivity of our event trigger from (11) with respect to its hyperparameter 5p.
B Sensitivity of ET-GP-UCB to the Event Trigger Hyperparameter dB
Table and Figure 13 show the sensitivity of ET-GP-UCB to the event trigger paramcter on Lemma To solely focus on the influence of őB with use the maximum window size as W T. It should be noted, that for different window sizes, different őB may be optimal for the best possible empirical performance both, the location of lower and upper reset bounds, and the hyperparameter бB, influence how much (and where) probability mass from the distribution over reset times is cut off. For each ōB, we ran 50 different seeds on within-model objective functions (cf. Section A.1). All hyperparameters are listed in Table 12. As ōg is inereased, the average number of resets increases as the probabilistic uniform error bound softened. We observe that for different rales of changes & different ôn may be optimal lo oblain the lowest possible empirical regret. From Figure 13, we can observe that for higher rates of change smaller dg can yield a smaller variance in the final regret. In summary, ET-GP-UCB displays low sensitivity to óg highlighting that our algorithm is readily applicable without the need for excessive hyperparameter tuning. In practice, we found dp = 0.1 to yield good empirical performance in all of our experiments.
Table 4: Sensitivity of ET-GP-UCB dB in Lemma 3.
Rate of Change ET-GP-UCB Hyperparameter óg Avg. No. of Resets Avg. Regret Rr
0.005
2.66
191.39
0.01
2.96
193,05
0.01
0.05
3.20
200.16
0.1
3.38
200.33
0.5
3.98
196.03
0.005
6.42
276.84
0.01
6.82
273.37
= 0.03
0.05
7.72
269.01
0.1
8.04
271.59
0.5
10.32
280.05
0.005
9.60
331.69
0.01
9.96
328.74
0.05
0.05
11.40
329.47
0.1
11.88
332.04
0.5
14.44
334.80
19

---

Published in Transactions on Machine Learning Research (01/2025)
C Discussion on the Theoretical Analysis and Future Directions
The current theoretical analysis does not explicitly include the event trigger (11), As stated in Section 7, we are interested in tighter regret bounds, tailored to the specific event trigger. This has the potential for additional insights into the design of new triggers. In the following, we list some of the challenges that may arise when conducting this theoretical investigation and give some pointers that may be helpful to derive tighter bounds.
Reset. times are stopping times. First, note that the activation time of the event trigger is a random variable 7 = min(t E N (xx) (x{)) This random variable is a stopping time, as ( is a discrete-time stochastic process and the trigger event is completely determined by the information known up to I. The stochastic process depends recursively on the acquisiiinn function, which makes an analytical st study of the stopping time difficult; even for simple stochastic processes, calculating expected values of stopping times is typically intractable.
The regret analysis should be specific for the event trigger. There is some information available on т: As long as T is finite, t is a bounded random variable as 7 E [1, T). This also means, that T is a sub-Gaussian random variable. One can use concentration inequalities of sub-Gaussial random variables bound the maximum of with high probability and then use this bound in the theoretical analysis. For the analysis to be specific for the event trigger, this concentration incquality has to depend the test and threshold functions of the event trigger which, stated above, is in itself very challenging: Random sampling block lengths within [1, have the same sub- Gabssinn property, i.e., with the same sub-Gaussian constant. However, the empirical performance would be very different compared to a sophisticated event trigger. Using standard bounds for sub-Gaussian random variables would therefore likely still result in a mismatch between empirical performance and theoretical analysis.
Bounding the random block sizes and number of block simultancously is promising. Once a event brigger specific bound on the stopping time is established, the theoretical analysis would still require fundamentally new way of quantifying the model mismatch compared Theorem 1. The current analysis bounds the maximum model mismatch by considering the maximum block length at every time step. Here, one could substitute in the bound on the stopping time 7 (which depends on T). However, the bound only holds with high probability and, therefore, for a valid probabilistic statement, a union bound over all time steps is required. This union bound can result in a super-linear scaling the regret bound. An interesting direction for future work would be instead of considering the maximum model mismatch at cach time step to bound the composition of the number of blocks and size of the blocks simultaneously.
20

---

Published in Transactions on Machine Learning Research (01/2025)
D Influence of the window size on the distribution of resets for different
In Figure 2 (left), displayed the influence of the window size the distribution of reset times. In Figure 14, we show this influence also for different e. We further add the empirical cumulative distribution in the right column.
Coustant reset time of R-GP-UCB
0
Window size W
27
Window size W
55
82
82
110
110
137
137
165
163
Constant reset time
of R- GP-UCB
193
193
50
100
150
200
50
100
150
200
Reset time (s = 0.03)
Cum. reset time (s = 0.03)
Constant reset time of R-GP-UCB
0
Window size W
28
Window size W
28
55
55
83
83
110
110
138
138
165
Constant reset time
of R- L-CP-UCB
193
193
50
100
150
200
50
100
150
200
Reset time (e 0.05)
Cum. reset time (e 0.05)
Constant reset time R-GP-UCB
0
Window size W
27
Window size W
27
55
55
82
110
110
137
137
165
165
Constant time
of R- R-GP-UCB
192
192
0
50
100
150
200
50
100
150
200
Reset time (s = 0.10)
Cum. reset time (s 0.10)
Figure 14: Left column: Influence of the window size on the empirical PDF of reset times. Right column: Influence of the window size on the empirical CDF.
21

---

Published in Transactions on Machine Learning Research (01/2025)
E Overview of the Temperature Data
In Figure 15, we show the normalized temperature data used for the empirical kernel in the results in Figure 3. The full pre-processing of the temperature data is described in the corresponding jupyter-nolebook in the provided code base
Day 3
Day 4
Day 5
Day
Norm. Temperatury [Pc]
2
24
48
Time [h]
Temperature data
Figure 15: Normalized temperature data used for the results in Figure 3. Each color resembles the measure- ment data from one sensor.
F Proof of Theorem 1 and Corollary 1 from Section 4
F.1 Proof of Theorem
Restated Theorem 1 Let the domain X rj, Rd be conver and compact with d E N4 and let fo follow Assumption with a kernel k(x, such that Assumption 3 is satisfied. Pick 5 E (0,1) and set 3, = 21n (2 (33)) + 2d block length Ñ E N4 with N < Ñ < T. Let yN be the marimum information gain over N points. Then, running any algorithm with block sizes between N and N satisfies
Pick lower block length N E N4 and upper
T +1) 7s +2+ Térle, N) Rr CTTBr
(13)
with probability al least - 8, where C₁ = 8/In( (n + 2) and defined
N) = 2/ Br + on (oon))) N + 2(0 on " + v20n) V In 4(ao +))21
(14)
Proof: The proof of Theorem is mainly based on the proof of Bogunovic et al. (2016, Theorem 4.2) which builds on arguments from Srinivas et al. (2010). The key distinction of our analysis that we will consider the block length not to be fixed but within an interval []. For completeness, we list the full proof here.
As in Bogunovic (d satisfying
(2016) and Srinivas et al. (2010), we first fix a discretization X, X C [0, r] of size
|| - [=]e|| S rd/0,,
E Xt
(15)
where [zr] denotes the closest point in X, to æ. Now fix a constant > and an incrcasing sequence of positive constants { satisfying Σ ,ं (cf. Remark 2). Next, we introduce some Lemmas
needed to bound the regret.
22

---

Published in Transactions on Machine Learning Research (01/2025)
Lemma 4 (Bogunovic et al. (2016, Proof of Theorem 4.2, cf. Appendix C.1)) Pick 8 (0.1)
and set B, = 2n 8, where Σiz1 "t" 1, > 0. Then we have
P((s((m))) (D, (at) s (arr 1I 12
(16)
Lemma 5 (Bogunovic et al. (2016, Proof of Theorem 4.2, cf. Appendix C.1)) Pick 8 E (0,1)
and set B, = 21n
= where Σ121 "T" P((i(()) irD (xx) S (Brō (2)), VI 2 Vo E X1 2 2
0. Then we have
p
(17)
Lemma 6 (Bogunovic et al. (2016, Proof of Theorem 4.2, cf. Appendix D)) Pick E (0, and = 1, > 0. Then, with Assumption 3 we have
set Lt = bi / In damm m
where E121 Tt
0f
P
sup xE
Lts
Vt 2 1, Vo Ye. Vj E fI.....,................. d3 $1
(18)
0z,
Lemma 7 Pick E (0, 1) and set Vt = (bo + v2on) In 2(an+))m have
where E121 Te = 1, Te > 0. Then we 6
P Pluell S Ve. vt 12112106
(19)
Proof: We have that Iedl s [xxe)) |wel given Assumption We can Lemma 2 (with instead of (,) with 8/2 to bound |wy| for all time steps. Combined with Assumption we have
PP (luel & b + w 2 og
( -1,7//6/ - 6/2.
(20)
Solving the remaining term with 6/2 and using the union bound, we get 5/ L5 + Wz == bo In 2ao
Wy Here, we can further bound boy In Zagxs with bo In 2(ao+ using the monotonicity of the logarithm (recall that on > 0). Since we have We = /2oa/In 2x, we can use the same trick to state that We S V2ony/n 2(a++ To end, we concatenate the terms obtaining T. Ue = (bo + v2o /n 2(a proving the claim.
Remark 3 Note that Lemma 7 slightly differs from the bound used in Bogunovic et al. (2016). We explicitly consider the variance of the measurement noise to bound the observations in probability.
For Theorem 1, we condition on four high probability events i.e., Lemma 4, Lemma 5, Lemma 6, and Lemma 7 cach with with 8/4. Conditioned on these events, the following Lemma holds.
Lemma 8 (Bogunovic et al. (2016, Appendix D)) Let Lemma 4, Lemana 5, and Lemma 6 each hold with probability 1 8/4. Pick 8 E (0, 1) and set
AT, + 2d In (rdtPbs / /I((AaaTre/0))
8, 2n
0
where Σi>1 - 1, > O. Let 0. = b b, /n /6) and let [x"), be the closest point in X, to x=.
Then we have
((iitar) - iTD - (1ti)e)
< Bar. (aoile) + p2
v2 422
]221]21- 21 21-
(21)
In the following, we will explicitly highlight the usage of cach Lemma. We first bound the instantancous regret at a time step as
rt f((()) f((e))
(22)
1 < üd, ([wi))) + BBOD (x+))) + (ax) + v (ir (Lemmna Lemina 8)
(23)
23

---

Published in Transactions on Machine Learning Research (01/2025)
This is identical to Bogunovic et al. (2016, Proof of Theorem 4.2) for R-GP-UCB. We now account for model mismatch between using the time-invariant model in the algorithm and the "true' time-varying posterior
DD, (a) I (DD, (x) + A (p)
with
A (P) sup (1ā,, (ax) po, (
D.
(24)
=
(o) (x) S oD. (a) + D.
with A (o) = sup {lōD (a)) oD,
(25)
We can thus further bound the regret as
Ti S WDD, ((e
A (1))
[ vBL OD, ([x;]1) A6 ((+)):))
(26)
- -DD (xx) + Dj
(at) + Be (z.) A (x+))
+ 12
(27)
Per definition of A Algorithm 1, we have that through the optimization of its UCB-type acquisition function PD, ((=t))) + VBIoD, ((xt))) $ uD. (xx) VBiaD, (ax,), and, therefore,
r 2 BIOD, (x,) + za + b. ((=)))) + (1) (xx) + Be (4)) (1x + Ab, (o) (a:))
(28)
What is left to bound is the model mismatch." 5 For this, we slightly adjust the bounds from Bogunovic et al. (2016, Lemma D.1.) to the setting where the block size can vary but is upper bounded by N.
Lemma 9 (Adjusted from Bogunovic et al. (2016, Lemma D.1.)) Conditioned on the event in Lemma 7, have for any block size N < N that the mazimum model mismatch in the mean and vari- ance between the time-varying GP model and the time-inuariane GP model is a.s. bounded as
(0) < V (3om 2 + on N3g
D.
(29)
(e) D.. S (o + tom No
(30)
Proof:
For establishing the bounds the model mismatch, refer to the proof of Bogunovic et al.
(2016, Lemma D.1.). If focuses on bounding (24) and (25) by explicitly bounding the difference in postcrior distributions of the time-invariant updates in (4) and time-varying updates (5). The bounds in (29) and (30)
are monotonic functions in the block length. Hence, the model mismatch is upper bounded by the maximum model mismatch with N.
With Lemma 9, we can summarize the bound on the model mismatch at each time step as
2 (1) + 2/33 s 2 (a + on NoE + 2 3. (3om² +o N3E (),
(31)
For completeness, we follow the steps in Srinivas et al. (2010, Proof of Theorem 2) and Bogunovic et al. (2016, Proof of Theorem 4.2 (R-GP-UCB)) to bound the regret following Definition as
T
1 2 (x))
R Σ rt S
Σ
22 + N),,
(32)
With 1/1 = < 2 and using Sauchy-Schwand with (Ei= 22)) 21)² < T TT x4 we obtain
PT S
T T M 4 (xt) + 2+ Σ (a + + Neg. + 2 (30m" -2 + on N38)
(33)
We noticed minor mistake Bogunovic al. (2016, Theorem 4.2). There, it should be 2 (N, c) in Eq. (17) model differenecs in and variance have be bounded twice it the for the variance proof (cf. Bogunovic et (2016, Proof of Theorem 4.2), Eq. (72)). There consequences from the missing constant but might be helpful future work state here explicitly.
24

---

Published in Transactions on Machine Learning Research (01/2025)
Furthermore, using the same arguments as in Bogunovic et al. (2016, Proof of Theorem 4.2 (R-GP-UCB)) and Srinivas et al. (2010, Proof of Lemma 5.3 (GP-UCB)) on the information gain within block of size M is Iyys fM) E In + o, (xx)) - 7M with 7M max [yMi JM) yields
4,,00 (ot) - 0 Σ
(o o (z()
(34)
T
S 48ro
Σ
In (1 or o (2:1)
(35)
with C2 =b=00//In(1+
o, 2 since $² s² C2In(1 + 52) for s [0, and on on 2, (x+) S on 2 2x)) S
on due the bounded kernel function in Assumption 2 (also see Srinivas al. (2010, Proof of Lemma 5.4), we get
T
T N NN-
48,0 (x)) & C3 N [(Ni C3 Σ
(36)
Here, the lower reset bound N becomes relevant with the following argument. With block lengths N E N], the maximum number of blocks one could obtain until T is bounded by N (assuming this is an integer for now). In each of the blocks, the maximum information gain is bounded by IN since the information gain is non decreasing. We assumed that T is an integer the number of block in T and we upper this with ( +1). We therefore have
T
Rr S CTTBT (T
T (1+1) YN +2+ Σ 2 20 +o,²) 2, No + +2 Be (30,² + oa N3E
NN
(37)
with C1 = 88/In(1 In(1 + 2) Further bounding the sum over maximum model mismatch yields
T +11 IN +2+ 2 + taat) No + 2 Br (30," + onr4) N dr(c,N
+1)12
RT < CiTBr
Theorem
F.2 Proof of Corollary
Restated Corollary Let the assumplions in Theorem 1 hold. Given a lower and upper bound on the
true such that = E E (0, 1), there exist A1 E (0, and X2 [1, such that g A1 and X2 For the SE kernel, set N = @ E=/T),))))))))) and N = (mmin(TTe 1114)). Then,
Rr Ô (max { (VT, 2/18 g 1/8T, A 3/8 1/8 x
3/4 (/1121117))
(38)
For the Matèrn kernel (v > 2), define & = d(++1) (mmin(T, -V/---))))))))) Then,
and set N = O(min(T,e (-/--1(4-))))) 1/14-5)1) 2+4ddd+1
and Ñ =
Rr = max VTI-E, X शा-क >
Le T, X शां म T, x; x + कيंي
(39)
Proof: First recall the bound from Theorem 1 and divide it into three terms to increase readability.
T (T CIT
NN +2 + 2T Br (3on" + on N3 + 2T (o + t No +
Rr
Term
Term 2
Term
25

---

Published in Transactions on Machine Learning Research (01/2025)
In the following, will first consider the SE kernel and afterwards the general Matérn kernel. We will bound each term separately for both kernels. The proof builds on Bogunovic et al. (2016, Appendix E) but extends it to the setting where only lower and upper bounds on E are known. In the following, let Zr(z) be the closest integer (1....,T) which is closest to 2 E RR
Squared Exponential Kernel
Term 1 For the SE kernel, Srinivas et al. (2010, Theorem 5) shows that YN O(In N) = Ō(1). Setting N = (/4))1/4, we obtain upon substitution
(T +1) CITT +1 7N ö
T 8-1/1 (VT=-M-=
=6 (>///8 0 (222881/8T) )
1
(40)
(A2e) 1144 1/4
Term For the second term, we have upon substitution of N
= Ir 1/4)
2I eTVer Br (3o + +on No = Ô (T (IVe-3/4g = (TV (TV(NIE) (MiE) 3/4
= =0 -0(x -X0AX33/2//0
(41)
Term 3 For the third term, we have upon substitution of N = -I
2T (on + to= NoesT - ô (T=- Ô (T(AIE) - =O(A7 (xT3)42//AT))
(42)
From these results, we can further see that for the scaling to be sub-linear & has to be sufficiently small relative the time horizon T and the lower and upper bounds on e have be sufficiently close. Specifically, X } obtain sub-linear scaling, ie., Ö(VT).
if min
ArT T }
Matérn Kernel (v m>>3 2)
Term For the Matérn Kernel kernel, Srinivas et al. (2010, Theorem 5) shows that Yv - O( In N) ((N) where & = 2v Setting N = (E-1(111(1-) and N = Ir (((e_)/( we have upon substitution
d(d+1)
T C1TBT
T
N (1+1)
+1
7x8 0
T E 1/(4 1/ 5/(4-5) (-((AA)))
(43)
(
(A2s) 11((4-0) (115) -E((2-8)
(44)
\ शरंيर कاتي & at--s)
(45)
Term For the second term, we have upon substitution of Ñ II(E 15r(5-1114 1/14-5)
2T Br (3om² + on N3g = θ 3
(46)
0 (TV (Ais) 3/(4 (1
(47)
= x 210-00 (xaaa Tls-6)e
(48)
Term 3 For the third term, have upon substitution of Ñ IT(E (1114 11(4-
2T (o -2 22200 t 4) N Eye θ Te's (T=-31//1 11/-33(444) 3/((--)) =1z)
3/14
(49)
δ (T()))) -3(())))
(50)
(2, (x ttt)
(11)
26

---

Published in Transactions on Machine Learning Research (01/2025)
Similar to the SE kernel, we also identify the sub-linear region for the Matérn kernel. Specifically, if 8 < <min
b p---5 -ー we obtain sub-linear scaling, i.e., ((VTI-5). We can furthermore nicely observe that for v + oc (therefore + 0), the bounds for Terms 1-3 as well as the sub-linear region converges to the bounds of the SE kernel.
x 2a 9)
G Proof of Lemma and Lemma 3
Restated Lemma Pick E (0,1) and set w7. 20 In F8, where Σx,21 Th, = 1, Te. > 0. Then, the noise sequence in Assumplion obtained since the last reset satisfies |we, S We, for all t 2 with probability at least 1- 6.
Proof: Since we, (O, (0), we can standard bound on the tail probability of the normal distribution I {1]] s) S f {20o)) with (O, o²) to obtain IP {|w,11 Wt, 2 1-exp { ////////2, /2o Solving for wh with 8/T,, using the same arguments as in the proof of Srinivas et al. (2010, Lemma 5.5) and taking the union bound over all time steps yields the results.
Restated Lemma Let (x) follow Assumption 2 with 0. Pick óB E (0,1) and set pt, In 25 BB
where E,,21 ni, =1, ne, > Also set wß 2 In 2%t.. Then, obseruations yt under Assumption satisfy on We - PD, (D)( I /Pt. DD. (xt) + wt, for all t, with probability at least 1 dB.
Proof: From Lemma 1 with 0B/2 and incorporating the noise, we have
(ft( + wt, - MDD. (xx+) I VD, (xx) |wel yt μ'D, (ax)) I VDL, (x:)) |wr.l-
(52)
with Pt, 21n 20tr oB
Using Lemma 2 with 0n/2 and laking the union bound yields the result.
H Hyperparameters
To ensure the reproducibility of our results, we give a list of all hyperparameters used in any simulation performed for this paper. All experiments in Section 6.2 were conducted 2021 MacBook Pro with an Apple M1 Pro chip and 16GB RAM. The Monte Carlo simulations in Scction 5 were conducted over 3 days on a compute cluster. The lemperature data set publicly available at http://db csail.mit. wwt.mmit..... edu/ Sabdata/labdaal html. The code will be published upon acceptance and is also part of the supplementary material of the submission.
Table 5; Hyperparameters of the Monte Carlo Simulation for ET-GP-UCH (cf. Figure 2).
Hyperparameters
Monte Carlo Simulation
dimension d
2
compact set X lengthscales time horizon T noise variance oi number of i..d. runs
00 1
0.2
200
0.02
000
event trigger parameter in Lemma 3 8g
0.1
rate of change 8
{0.03, 0.1}
27

---

all Published in Transactions on Machine Learning Research (01/2025)
Table 6: Hyperparameters for the within-model comparisons (cf. Figure 7).
Algorithm Hyperparameters
Correct rate of change Miss. rate of change
dimension d
2
compact set X lengthscales [ time horizon T noise variance oî
[0, 1]2
[0, 1|2
0.2
0.2
400
400
0.02
0.02
Br approximation parameters number of i.i.d runs
C1 = 0.4, 02 =
41 = 0.4, 02 =
50
UI-TVBO rale of change
(0.01, 0.03, 0.05)
{0.001,0.2
TV-GP-UCB rate of change &
(0.01, 0.03, 0.05)
(0.001,0:3) 0.2]
R-GP-UCB reset time Neonst
(38, 29, 26}
[68, 17}
ET-GP-UCB event trigger parameter 6B
0.1
0.1
Table 7: Hyperparameters for the temperature dataset (cf. Figure 3).
Algorithm Hyperparameters
Temperature Dataset
dimension d compact set X kernel k
46 arms empirical kernel
all
time horizon T noise variance of 3, approximation parameters number of i.idd runs
286
0.01
= 0.8, C2 = 0.4
50
UI-TVBO rate of change a%
0.03
TV-GP-UCB rate of change (see Bogunovic et al. (2016))
0.03
R-GP-UCB reset time Nconst (see Bogunovic et al. (2016))
15
ET-GP-UCB event trigger parameter in Lemma 3 6B
0.1
Table 8: Hyperparameters for the policy search experiment (cf. Figure 3 (c)).
Algorithm Hyperparameters
Policy scarch
dimension d
4
lower bound to [-3, -6, -5. -4]
compact set
upper bound to [-2, -4, -55, -2
scaling factors [1/8, 1 /4, 1/8)
lengthscales all
gamma prior ((15, 10/3)
time horizon T noise variance of
300
0.02
3, approximation parameters number of i.idd runs
0.8, = 4
50
UI-TVBO rate of change
0.03
TV-GP-UCB rate of change &
0.03
R-GP-UCB reset time Neonst induced by rate of change
29
UI-TVBO rate of change 82
0.03
ET-GP-UCB event trigger parameter in Lemma 3 6g
0.1
28

---

Published in Transactions on Machine Learning Research (01/2025)
Table 9: Ablation on Impact of Reset Frequency vs. Timing (cf. Figure 4).
Algorithm Hyperparameters
Sensitivity Analysis
dimension
compact
[0. 1]2
kernel &
fixed, = 0.2
all
time horizon T noise variance oî 6g approximation parameters number of i.i.d. runs
400
0.02
0.4, = 4
50
R-GP-UCB reset time Noonst
{38, 29, 26)
R-GP-UCB* reset time Noonst. as average from ET-GP-UCB
(100, 45, 32)
ET-GP-UCB event trigger parameter in Lemma 3 ōB
(0.1, 0.1,0.)
Table 10: Hyperparameters for the synthetic experiments in Deng et al. (2022).
Algorithm Hyperparameters
Synthetic Experiments
dimension
1
compact set kernel &
0, 1]2
fixed, [ 0.2
time horizon T noise variance of number of i.idd. runs
500
{0.01, 1)
20
ET-GP-UCB event trigger parameter in Lemma 6B
0.1
Table 11: Hyperparameters for the stock market data experiment in Deng al. (2022).
Algorithm Hyperparameters
Stock Market Data
dimension d compact set X kernel
48 arms empirical kernel
time horizon T noise variance of number of i.i.d. runs
823
(0.01,300)
20
ET-GP-UCB event. trigger parameter in Lemma 6B
0.1
Table 12: Hyperparameters for the sensitivity analysis (cf. Table 4).
Algorithm Hyperparameters
Sensitivity Analysis
dimension d compact set X kernel k
2
00, 1]2
fixed, / -= 0.2
all
time horizon T noise variance o² Rg approximation parameters number of i.i.d. runs
400
0.02
0.4, 02 = 4 50
ET-GP-UCB event trigger parameter in Lemma 3 6 {0.005, 0.01, 0.05, 0.1, 0.5}
29


Reviews:
[{"review_id": "YnaAxEfecL", "reviewer": "Reviewer_ANPd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. The abstract and introduction promise regret bounds that are adaptive to rate of change. But Theorem 1 and Corollary 1 require upper and lower bounds on the rate of change. The following papers show (truly) adaptive regret bounds for non-stationary multi-armed bandits (only one is cited):\n\nPeter Auer, Pratik Gajane, Ronald Ortner. Adaptively tracking the best bandit arm with an unknown number of distribution changes. COLT, 2019\nYifang Chen, Chung-Wei Lee, Haipeng Luo, Chen-Yu Wei. A new algorithm for nonstationary contextual bandits: Efficient, optimal and parameter-free. COLT, 2019.\nChen-Yu Wei and Haipeng Luo. Non-stationary reinforcement learning without prior knowledge: an optimal black-box approach. COLT, 2021.\nJoe Suk and Samory Kpotufe. Tracking most significant arm switches in bandits. COLT, 2022.\nYasin Abbasi Yadkori, Andras Gyorgy, Nevena Lazi. A New Look at Dynamic Regret for Non-Stationary Stochastic Bandits. JMLR, 2023.\n\n2. The Event-Trigger condition in Algorithm 1 seems to consider only one datapoint. Is this correct? Perhaps in this work, and given the Gaussian noise condition, such a condition might be enough to obtain sub-linear regret bounds. But more generally (e.g. Bernoulli observations) such a condition does not lead to sub-linear regret bounds. The above papers can show strong adaptive results only because they consider collection of observations over time intervals.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes an algorithm for non-stationary kernelized bandits. The algorithm uses a test to detect if the reward function has changed substantially. The paper shows sub-linear regret bounds for the proposed algorithm.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The abstract and introduction need to be updated to accurately state the quality of the performance bounds (not fully adaptive to the rate of change). \n\n2. How would your performance bounds look like in the multi-armed bandit case? \n\n3. Authors need to discuss the form of the Event-Trigger condition and why it can use only one datapoint.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "H3TKU0kViK", "reviewer": "Reviewer_P7Dp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. The algorithm is well motivated, and the problem is important. In most cases, parameters like $\\epsilon$ are rarely known ahead of time and removing them is always useful.\n2. ET-GP-UCB shows low sensitivity to the main parameter, $\\delta_{B}$, the threshold at which resets happen. The experiments also show ET-GP-UCB displays large robustness to $\\epsilon$ changes whereas previous algorithms (that requires precise knowledge of $\\epsilon$ do not.\n\nQuestions:\n1. How do we know $r$, $a_1$, and $b_1$ in Theorem 1?\n2. Why are we considering only the dynamic regret? Can we achieve sublinear regret if we allow for regret that tracks comparators which are fixed for certain windows? \n3. How do we know the upper and lower bounds on $\\epsilon$? Are they available for practical problems?\n4. We assume that $\\sigma_n$ is known in (11), but how do we know it?\n\nMinor points\n1. The $t^\\prime$ notation in p. 6 is a bit confusing, it might be better to be more explicit and name the reset timesteps at $t_1, t_2, \\ldots$ and consider $t \\in [t_i, t_{i+1}]$.\n2. The section on the empirical impacts of $\\underline{N}$ and $\\overline{N}$ on the reset times (p. 7) might be better moved to the experimental evaluation section.\n3. In Appendix D, there is a missing reference \"As stated in Sec. ??\".", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers the problem of time-varying Bayesian optimization (TVBO), where at each step $t$, a learner queries an oracle with a point $x_t$ and is given back a noisy observation $y_t$ of the function value $f_t (x_t)$, and the aim is to minimize the dynamic regret $\\min_{x_1, x_2, ...} \\sum_{t=1}^{T} [\\max_{x} f_t (x) - f_t (x_t)]$. The functions $f_t$ come from a smoothly-varying Gaussian process, as defined in Assumption 2, where a parameter $\\epsilon$ limits how much the functions $f_t$ differ from one step to another. The observations $y_t$ are corrupted with Gaussian noise (Assumption 1). The functions also satisfy some standard smoothness assumptions (Assumption 3). The main issue this paper considers is how to solve this problem without explicit knowledge of $\\epsilon$, here \"solve\" generally means obtain a regret rate that is worst-case bounded as $T \\phi(\\epsilon)$ for some function $\\phi$ of $\\epsilon$ such that $\\phi(\\epsilon) \\to 0$ when $\\epsilon \\to 0$. This is because in general, linear regret is unavoidable in this problem. In order to remove the knowledge of $\\epsilon$, the authors develop a new algorithm (Event-triggered GP-UCB, Algorithm 1) that adaptively resets the stored dataset whenever the data has become too stale, according to a certain threshold test. The paper also conducts extensive experimental evaluations for the algorithm.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please add more motivation and address the questions I mentioned in the previous section, if possible. This is not critical, I think the world is strong enough to recommend it for acceptance as-is.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sgFyJqME2z", "reviewer": "Reviewer_TDqt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n* The event-triggered idea is easy to understand.\n* The paper presents a theoretical analysis with clear regret bounds, providing a foundation for the algorithm’s effectiveness.\n* Many discussions including ablation study are provided.\n\n## Weaknesses\n* Although theoretical analysis is provided, it relies heavily on the posterior variance of a Gaussian process (GP) model, assuming exact adherence to GP priors. In practice, it is unlikely that data would align perfectly with such priors. Consequently, the event-trigger mechanism could signal unnecessary resets in static optimization scenarios, raising concerns about the algorithm’s robustness in real-world applications.\n*  Based on the aforementioned concern in the real-world scenarios, the hyperparameter settings of event trigger is important and should influence the performance. In Appendix B, the authors show how the hyperparameter $\\delta_B$ affect the regret performance. However, it does not provide enough details about the experiments, e.g. the benchmarks and the number of iterations. It is also important to provide line charts showing how the regret changes over time instead of only the final regret, as changing $\\delta_B$ might affect the convergence speed but has limited impact on final regrets.\n* The baselines selected for comparison are somewhat limited in complexity. A more robust comparison could include a simple and adaptive reset strategies based on actual optimization performance, such as the reset strategy in TuRBO [1], where resets are triggered when consecutive iterations fail to yield improvement.\n* The proposed method is validated only on synthetic and specific real data, lacking experimental support for a broader range of domains or more complex scenarios, such as those with dimensions greater than 5.\n\n[1] Scalable Global Optimization via Local Bayesian Optimization, Neurips 2019.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel event-triggered algorithm (ET-GP-UCB) to address the time-varying Bayesian optimization (TVBO) problem. Unlike existing approaches, ET-GP-UCB treats the optimization as static until a change in the objective function is detected, at which point it resets the dataset. This enables the algorithm to adapt dynamically without requiring prior knowledge of the objective’s rate of change. The authors provide theoretical analysis for the method and demonstrate its superior performance on synthetic and real-world datasets.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* The proposed event trigger strategy should be compared with performance-based reset strategies.\n* More benchmarks should be used for evaluation, e.g., synthetic functions and real-world applications with more dimensions or even a changing change rate ($\\epsilon$ is not a constant).\n* More details and more experiments about how event trigger hyperparameters affect the optimization process should be provided.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper describes an event-triggered algorithm to detect shift in the objective function for time-varying Bayesian optimization approaches.
During the review phase, reviewers raised several concerns about the paper:

Reviewer ANPd expressed doubts regarding whether the presented results could be directly translated to the multi-armed bandit (MAB) scenario, which can be viewed as a special case of the Gaussian process (GP)-based Bayesian optimization setting considered in the paper. I support the authors' argument that, while the two scenarios appear similar, translating the regret bounds derived from GP-UCB is non-trivial and beyond the scope of this work. The revised version of the paper also includes a discussion relating to MAB and highlights the key differences.

Reviewer TDqt pointed out that performance-based reset strategies are a practical tool for restarting optimization. In response, the authors included a comparison with a simple strategy inspired by prior work. Reviewer TDqt raised concerns about the evaluation of this baseline and proposed a more sophisticated approach. While I agree that the baseline could be further optimized, I believe this refinement would not fundamentally alter the main contributions of the paper, which center on providing a theoretical framework for detecting shifts in the objective function.

Secondly, Reviewer TDqt argued that the sensitivity analysis of hyperparameters was limited. However, my understanding is that this concern primarily pertains to the general modeling assumptions inherent in Gaussian processes, rather than specific assumptions made by the proposed methods. While I acknowledge the validity of these concerns, they have already been extensively addressed in the existing literature.

As discussed above, I do not believe that the paper overstates its claims and find that it provides both theoretical and empirical evidence to substantiate them. However, to minimize potential confusion for future readers, I recommend that the authors:

 -   Rephrase the abstract, which currently states that the proposed method outperforms state-of-the-art methods. Since the comparison is limited to GP-UCB, which is not considered the current state-of-the-art in Bayesian optimization, this claim should be adjusted.
 -  Add a sentence to Section A.3 noting that performance-based triggers could be further optimized, potentially leading to improved performance.
Decision: Accept (poster)