Safe Time-Varying Optimization based on Gaussian Processes with Spatio-Temporal Kernel
Authors: Jialin Li
Marta Zagorowska
Giulia De Pasquale
Alisa Rupenyan
John Lygeros
Venue: NeurIPS 2024 poster (2024)
Keywords: Safe learning
Bayesian optimization
Time-varying optimization
Primary area: active_learning

Abstract: Ensuring safety is a key aspect in sequential decision making problems, such as robotics or process control. The complexity of the underlying systems often makes finding the optimal decision challenging, especially when the safety-critical system is time-varying. Overcoming the problem of optimizing an unknown time-varying reward subject to unknown time-varying safety constraints, we propose TVSAFEOPT, a new algorithm built on Bayesian optimization with a spatio-temporal kernel. The algorithm is capable of safely tracking a time-varying safe region without the need for explicit change detection. Optimality guarantees are also provided for the algorithm when the optimization problem becomes stationary. We show that TVSAFEOPT compares favorably against SAFEOPT on synthetic data, both regarding safety and optimality. Evaluation on a realistic case study with gas compressors confirms that TVSAFEOPT ensures safety when solving time-varying optimization problems with unknown reward and safety functions.

Safe Time- Varying Optimization based on Gaussian Processes with Spatio-Temporal Kernel
Jialin Li
Marta Zagorowska NTNU (currently with TU Delft) a 2a.agoooossa@tuuel.ftt....
ETH Zürich (currently with UIUC) lijial@ethz.ch
Giulia De Pasquale
Alisa Rupenyan
Eindhoven Univeristy of Technology g de. www.paaaqualeeue.... nl
Zürich University of Applied Sciences alisa. rupenyadhaan.n ch
John Lygeros
ETH Zürich
jlygeros@ethz.ch ch
Abstract
Ensuring safety is a key aspect in sequential decision making problems, such as robotics process control. The complexity of the underlying systems often makes finding the optimal decision challenging, especially when the safety-critical system is time-varying. Overcoming the problem of optimizing an unknown time-varying reward subject unknown time-varying safety constraints, we pro- pose TVSAFEOPT, new algorithm built on Bayesian optimization with a spatio- temporal kernel. The algorithm is capable of safely tracking a time-varying safe region without the need for explicit change detection. Optimality guarantees are also provided for the algorithm when the optimization problem becomes stationary. We show that TVSAFEOPT compares favorably against SAFEOPT on synthetic data, both regarding safety and optimality. Evaluation on a realistic case study with gas compressors confirms that TVSAFEOPT ensures safety when solving time-varying optimization problems with unknown reward and safety functions.
Introduction
We seek to interactively optimize unknown time-varying reward function f X' is a finite set of decisions, and T == {0, 1, T), E N4 denotes the discretized time domain. We assume that the optimization problem is safety-critie critical, that is, there constraints that evaluated decisions must satisfy with high probability. Similar to the reward, the constraints are also unknown and potentially tire-varying, encoded through X' x T R, i E I. := {1, ..... m), where m E N4 denotes the number of safety constraints. The optimization problem at time is
R, where
max f(x,t) xEX
subject to e (x,x)) 2 0, IE
Both the reward function and the safety constraints are assumed to be unknown but can be evaluated. This is plausible setting, for example, for UAV that need to perform rescue missions in dangerous and poorly lit environments.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

safe set (0=30)
safe set (t=100)
safe set (t= 1t=)
(a)
(b)
(c)
-1
safe points maxim zers
current max miherr true musmieos
current thue masimizen
(d)
(e)
(
1-
1
intaal sofe soods safe peint
(9)
(h)
(1)
0-
-1-
poires Current max mile true maximbers
current bue masimicun
-2
2
Figure 1: Comparison of safe sets computed by TVSAFEOPT (top row), ETSAFEOPT (middle row), and SAFEOPT (bottom row) at f = 30, t = 100, and t = 170. Because TVSAFEOPT takes the possible changes in time into consideration, the safe sets computed by TVSAFEOPT are contained in the ground truth safe regions while those computed by ETSAFEOPT and SAFEOPT have multiple violations. The reason for the violations ETSAFEOPT is that the algorithm is unable to detect small changes in the constraints, confirming that the performance of ETSAFFOPT depends on the event detection algorithm.
1.1 Related Work
Bayesian Optimization (BO) is well-established approach for interactively optimizing unknown reward functions. Various BO based approaches have been proposed to solve a wide range of problems in robotics [1, 2]. combinatorial optimization [3], sensor networks [4], and automatic machine learning [5, 6]. However, Safe Bayesian Optimization in the time-varying setting is still under-explored.
Safe Bayesian Optimization To address safety requirements safety-critical applications, Safe Bayesian Optimization (SBO) [7] has been proposed to avoid unsafe decisions with high probability by interactively optimizing a reward function under safety constraints. SAFFOPT [7], one of the first SBO algorithms, expands an initial safe set iteratively based new evaluations and an

---

Table 1: Overview of safe learning methods based on BO for time-varying problems.
Handling Changes
Safety Guarantee
Optimality Guarantee
Sale Seed
in Time
A-GOOSE [21, 171
ppatio-temporal kemme
X
For all
C-SAFEOPT [8]
apatio-temporal kemee
XX X
For all
ETSAFEOPT [22]
Event detection
X
For
TVSAFEOPT (ours)
Spatio-temporal kemel
For initial
updated Gaussian Process (GP) model of safety functions. It calculates two subsets, maximizers and expanders, from the current safe set and selects the most uncertain decision within their union to balance maximizing the reward function and expanding the safe set. Subsequent algorithms extend SAFEOPT handle multiple constraints [8], decouple safe set expansion from optimization [9], and expand the safe set in a goal-oriented manner [10]. These methods also explore disconnected safe efficiency [13, 14]. They have been applied to
regions [11, 12] and enhance information-theone controller tuning for a ball-screw drive [15] and quadrupeds [16], and adaptive control on a rotational motion system [17]. However, SBO typically does not take into account changes with time.
Contextual Bayesian Optimization Contextual Bayesian Optimization (CBO) has been introduced to address the influence of external environmental factors reward and safety functions. Krause and Ong [18] extend the Gaussian Process Upper Confidence Bound (GP-UCB) algorithm [19] by in- corporating contextual variables into unconstrained BO, demonstrating sub-linear regret analogous to GP-UCB. An advancement of this framework is proposed in [20], with the Safe Contextual GP-UCB optimizing the contextual upper confidence bound within safe
manage room temperature via a PID controller. Berkenkamp et al. [8] present a contextual adaptation of SAFEOPT, discussing its safety and optimality guarantees by framing contextual SBO distinct SBO sub- babb-roblems. Additionally, König et al. [21] extends GOOSE [10] to the contextual domain for model-free adaptive control scenarios. Similarly to SBO, CBO does not explicitly consider time-varying problems.
Time- Varying Bayesian Optimization Varnin Bayesian Optimization (TVBO) addresses problems where the objective is lime-dependent, modeled with temporal kernel [23]. Methods in this setting include periodical resetting [23], change detection [24, 25]. sliding-window approaches using recent data [26], and discounting via exponentially decaying past observations [27]. However, these techniques have been developed for unconstrained BO and are unsuitable for safety-critical applications.
Time- Varying Safe Bayesian Optimization In the safety-critical time-varying setting, contextual lower confidence bounds can be optimized within the safe set [20], but it does not guarantee optimality theoretically. An event triggering mechanism is introduced SBO to restart exploration from backup policy [22]. but it may not trigger reliably during changes, posing safety risk. Extensions to SBO with contextual variables provide theoretical safety and optimality analyses [8, 16). treating contextual SBO as separate sub-problems for each contextual value, and assuming an initial safe set for each. However, ensuring optimality requires each contextual value to appear frequently, which is impractical in time-varying scenarios.
1.2 Methodology and Contributions
Methodology We propose the TVSAFEOPT algorithm to optimize an unknown time-varying reward subject to unknown time-varying safety constraints. The algorithm focuses on Time-Varying Safe Bayesian Optimization (TVSBO). TVSAFEOPT utilizes a spatio-temporal kernel and time Lipschitz constants as prior knowledge about how the problem depends time. The temporal part of the kernel encodes the continuity of the functions with time while the Lipschitz constants explicitly provide upper bounds on how fast the functions may change. Instead of considering safe sets at previous iteration as safe at the current iteration, which might lead to unsafe decisions, TVSAFEOPT robustly subtracts the safety margin when updating the safe sets (Figure 1). In this way, the algorithm is capable of adapting in real time and guarantees safety even when exploring the safe region of non-stationary problems.
Contributions Our contributions are threefold: a) We propose the TVSAFEOPT algorithm based on Gaussian processes with spatio-temporal kernels; b) We provide formal safety guarantees for TVSAFEOPT in the most general time- timevaryying setting and optimality guarantees for TVSAFEOPT for
3

---

locally stationary optimization problems; c) We show TVSAFEOPT performs well in the most general time-varying setting both on symthetic data and on a realistic case study on gas compressors. In Table we compare Adaptive GOOSE, Contextual SAFROPT, ETSAFROPT, and TVSAFFOPT in terms of how they handle changes in time, safety guarantees, optimality guarantees, and required safe seeds.
1.2.1 Expected societal impact
The TVSAFEOPT algorithm proposed in this paper extends the state of the art in Time-Vare Varying Safe Bayesian Optimization by enabling solving optimization problems with time-ng reward and constraints without pre-defining the time changes that can be compensated. Thmpensated.Thus, Thus, the algorithm can be used at the design stage operating strategies for safety-critical systems, such as medical dosage design [28] and controller design in robotics [17], or during online operation of chemical plants [29] or autonomous racing [30].
TVSAFEOPT Algorithm
The TVSAFEOPT algorithm builds upon SAFEOPT [7]. to handle time-varying reward function and safety functions. The key new feature of TVSAFEOPT is its capability of safely transferring the current safe set to the next time step. TVSAFEOPT achieves this with the help of the spatio-temporal kernel well the sequence of time Lipschitz constants. The approach is summarized in Algorithm 1.
2.1 Assumptions
Following [8], we incorporate the reward and safety functions into an auxiliary function 2 x T I R. where I == {0) U...
fxxx if (xxxt)
M(x,t,)) t,i
if I.
We model the auxiliary function using prior Gaussian Process (GP) with zero mean and spatio- temporal kernel k (X' T II x (X T II) + R. [31]. We require be Lipschitz continuous with respect to both x and t, and to have bounded norm in the Reproducing Kernel Hilbert Space (RKHS) [32] associated with the kernel x as formalized in the following.
Assumption 2.1. The spatio-temporal kernel
positive definite, and satisfies 1, for all E X,t E T,i E I. The function h(x, t) has bounded
((x, i,), ix) norm in the RKHS associated with kernel k. The function h(x, t, is Lxpischhitt continuous with respect x in the domain X' with respect to metric d X x X Rz0 for all N, i E I. There exists a sequence (L(f))ee,,sTT such that, for all x E X, E, [h(x,t+ 1, h(x,t, i)l L(t).
At each algorithm iteration k we make decision Xg, which we then apply to the system and get noisy measurements v of the reward function and safety functions during the iteration. We use the index & to refer the algorithm iteration. Even though k and I might differ in principle, in practice we run one algorithm iteration is for each time step L.
Assumption 2.2. Observations y = h(xx,t, t, i) + E Vi I, E N are perturbed by i.idd zero mean and o-sub-Gaussian noise.
Based on the measurements, we compute the posterior GP and make the decision for the next time step. To start the exploration, an initial set of safe decisions is assumed to be available to the algorithm. To ensure that the safe set remains non-empty after the first iteration, it is necessary that the initial safety function values at every decision within the initial safe set are positive.
Assumption 2.3. An initial set So C X' of safe decisions is known and for all decisions E So. we have (x, 0) > O, Vi E Ig.
Similar assumptions have also been made for the standard SAFEOPT algorithm [7] and are necessary to ensure fcasibility of the exploration steps and to be able to identify new safe decision.

---

2.2 Safety Updates
To ensure safety, based on Assumption 2.1 and 2.2, we extend the definition of the confidence intervals from [7] so that, with high probability, they contain f and q using the posterior GP estimate given the data sampled so far. The confidence intervals for l(xx, 1, i) given training samples until iteration & 2 1 are defined for all x E X' and for all i E I as
Qx 1) = ( i) + (xx)x-(xx
(3)
where 3& is scalar that determines the desired confidence interval, M (x, i) and 0 the posterior mean and standard deviation of h(x, 6, inferred with Dz. training samples till iteration k [31]. The probability of the true function value h lying within this interval depends on the choice of 3& [8]. We provide more details for this choice in Section 2.4.
(x, i) are
We now construct tighter confidence interval for h(x, t. by using the sequence {Q, (x. i) i)} instead of (x, i) alone. To this end, we recursively define for all x E X and for all e I the intersection
Cx == (Cx_1 i) @ (), (Q) (x,,
where @ denotes the Minkowski sum, Co(x, i) is [L()),, oc) for all x E So. i E Ie and R otherwise. We use the lower bound (x, == min Cx,x, i) and the upper bound us Mx(x,) == max Cx(x, i), to define the width of C2 (x
wi. (x, i) == un (x, i) 4 (x,
further used to update the safe set as well as pick the next decision to explore.
Based on the updated posterior and Lipschitz constants, we can update the safe set S. with the lower bounds la and the previous safe set Sk-1
S = NIEz. UXES,-, (x's E X -xddx)) L(t) 2 0}.
Lxd(x,
(6)
The set S& contains decisions that with high probability fulfill the safety constraints given the GP confidence intervals and the Lipschitz constants. contrast to SAFEOPT, the safe set of TVSAFOOPT allowed to shrink to adapt the potential change of the safe region given the time-varying setting. However, the safe set might even become empty after the update. This is either because the safe region indeed becomes empty because the updated safe set conservatively excludes all decisions with a lower bound of some safety function below L to guarantee safety. In all these cases, the updated safe set is empty, we terminate the algorithm.
2.3 Safe Exploration and Exploitation
With the safe set updated, the next challenge is to trade off between exploitation and expansion of the safe region. As in the standard SAFEOPT, the potential maximizers are those decisions, for which the upper confidence bound of the reward function is higher than the largest lower confidence bound
My = {x E Sx (x, 0) 2 max (x','0) 0)
x'ES,
To identify the potential expanders, GR, containing all decisions that could potentially expand the safe set, we first quantifly the potential enlargement of the current safe set after sampling new decision x. To do so, we define the function
(x) == Xx' E S' 3i E I, un (x, i) xx, Ltt 2 20}
(8)
where
refers to the cardinality of a set, and then update
G fx E Sx (x(x) > 0}.
At iteration k, TVSAFEOPT selects decision x within the union of potential maximizers (7) and expanders (9)
X = arg max wx((x,) i),
(10)
5

---

Algorithm TVSAFEOPT
1: Input: Sample set X' GP priors for f.G. c
Lipschitz constants Lx and (L(t))eN,,<T
Safe set seed Sy
2: Co(x, i) f- [L)),, oo), for all So. iE I
3: Co(x, i) + R, for all x E So, i EEI. In
4: Colx, 0) R
5: Query point Xn E So. yd + h(xn, 0, + t i E
6: Do = {o))
7: for k 1,2 do
8: Calculate (x, as in (3), Vx ViEiI
9: Cx(x, t (Cx-1 @ [ - 1), Ltt 1)1) (x,
S& NIz UXES.-1 (x' E & (xx Lx x)) ) L(t) 2 0}
11: if S& = Ø then
break end if
13:
MR + Ex St un (x(x,,) maxves, (x(x,0))
15:
GL t Ex (x(x) > 0} with (x) from (8)
16:
X + arg maSxEEEE..
(x,
17:
VE - h(Xk; 1, + 5k EEI
18:
Dj D (x)
19: end for
with we from (5). The objective of the greedy selection process in (10) is to take the most uncertain decision among the expanders Gş and the maximizers My. The decision x is then applied to the system and after making observations of the reward and safety functions, Yı (v 2e........... ................................ we add (x,y yx) to the training samples.
..... v(")
At any iteration, we can obtain an estimate for the current best decisions from
8 arg max 4 (x, 0),
(11)
xES,
which returns the maximizer of the lower bound of the reward function within the current safe set.
2.4 Safety Guarantee
To provide safety guarantees, we need the confidence intervals in (3) to contain the safety functions with high probability for all iterations. Note that the parameter 3& in (3) tunes the tightness of the confidence interval. The following lemma guides us to make a proper choice for Sg: This choice depends the information capacity % associated with the kemne namely is the maximal mutual information [33] we can obtain from the GP model of through i noisy measurements hx, at data points X := (xx, E X, T, I]]rck
max (hxx, h))
(12)
Lemma 2.4. Assume that h(xx, 1, has RKHS norm associated with k. bounded by B >> 0 and that measurements are perturbed by
noise. Let the variable 7h be defined as in (12).
For any 8 E 1), let VBn = B + o ( + + In(1/8)). then the following holds for all decisions x E X, function indices E I, and iterations k 2 jointly with probability at least
t, (x, S /BAT--IXx,1
Proof. This lemma is straightforward consequence of Lemma 1 of [16], a contextual extension of Lemma 4.1 of [8]. We can prove by selecting time as the context and picking 11) as the context sequence.
D

---

Lemma 2.4 indicates that, by selecting 3k properly, the confidence intervals Qs will w.h.p. contain the reward function and the safety functions. Due to this, they can be leveraged provide theoretical guarantees for safety and optimality.
The following theorem provides sufficient condition for safety of TVSAFEOPT.
Theorem 2.5. Let Assumptions 2.1 2.3 hold, and let 7/ be defined as in (12). For any å E (0, 1),
let V = B + v (7/pz + +1 In(1/5)). then TVSAFEOPT guarantees that with probability at least 1 1- for all E Ic and for all 2 0, and E Sy. it holds (x t) 2
The proof builds on Lemma 2.4 to show first that for all & 2 O, for all i E I and for all x E X, then h(x,t, E Cx(x, i) with high probability. Then using the recursive definition of the safe from (6), we obtain w.h.p. (x, t) 2 4xx', Lxd(x,x)) x) - L(t) 2 0, which concludes the proof. For details we refer the reader to Appendix B.
2.5 Near-Optimality Guarantee
In many safety critical real world applications, such as nuclear power plant operations, medical devices calibration, automated emergency response systems, the reward function is stationary most of the time. The problems are stationary until some changes happen and become stationary again when the systems reach new equilibria [34]. However, ensuring optimality is non-trivial even when the problem becomes stationary. Suppose the auxiliary function (2) becomes stationary in a time interval (ó, Φ). namely suppose there exist Φ > ø 2 such that vt1, t2 E [Ø, Φ), f(x, (1) = f(x, 12) =: f(x) and c(x, (1) = c(x, tz2) =: c(x), so that the optimization problem (1) becomes
max f(x)
x
(13)
subject (x) 0, I
We first define the largest safe set expanded from set S subject to a measurement error within
single time step:
Ra(S) SU
E (x4) Lxd(x, xx 2 0}
n time steps: (S) = Rs (Ra Rs (Re (S))...
times
arbitrary time steps: RS( = lim R
We also define L, as an upper bound of the sum of all time Lipschitz constants, that is, Σ ((T) L,
We find it reasonable that a tight upper bound L, can be provided when the underlying system slowly switches to the new stationarity condition.
Given these definitions, we are now in the position to provide optimality guarantees for TVSAFEOPT. In particular, we aim comparing the found reward value f(x;) with the optimal reward value within the largest safe set obtained in ideal conditions with no measurement error, (So). We also aim at providing TFSAFEOPT with an upper bound on the iterations needed find near-optimal solution. The following theorem states the optimality guarantee of TVSAFEOPT
Theorem 2.6. Let Assumptions 2.1 2.3 hold, let 7 be defined as (12) and, for any 5 (0, 1), let
VB.- = B + g (1h.p + +1+ In(1/6)). Define &g as in (11), and, for any e > 0, let k (e, 8) be
the smallest positive integer satisfying
k*
bi Ro (So) +1)
B 1
22
where 8/loo
log (1+0-2) Then, the TVSAFEOPT algorithm, applied (13), guarantees that, with probability at least 1 5, there exists ls < k.' such that
f (xx) 2
max
f(x) c.
x (So)
7

---

card. of sado
rum. of unsafe derisions cand. of sade
FUm. of safe decisions / card. of ground truth sade region
TVSateGPP
TVSateORT
[b)
TVSareOPT
SalcOPT
SafcOPT
SalsOPT
MMM
0.4-
My
40
50
20
60
days 00
days
days
Figure 2: Comparison between TVSAFEOPT, SAFEOPT, and approximate optimization the gas compressor case study, showing average 10 repetitions with different initial sets. (a): The cardinality of the safe sets, (b): The ratio between the number of unsafe decisions in the safe sets and the cardinality of the safe sets, (c): The ratio between the number of safe decisions in the safe sets and the cardinality of the ground truth safe regions. TVSAFEOPT robustly shrinks its safe sets based on its observations and thus maintains much less violations in its safe sets than SAFEOPT and approximate optimization, the cost of covering less of the ground truth safe region.
The proof consists in showing decaying upper bound of uncertainty we (x, i) S. e and exploiting local stationarity of (13) provide bounds the expansion of the safe set Sg. Details can be found in Appendix C.
3 Experiments
3.1 Synthetic Example
We first illustrate TVSAFEOPT on synthetic two-dimensional time-varying optimization problem
log(1 + y²) )+++ 0.01t
max 2.y
s.t.
r + 0.5 0.5 - 0.5 (1- cos
2π
Cos
π
+
y
0.3 - 0.5 (1- (1- cos
2T
sin
π
S 1.
50
6
50
Figure compares the safe sets computed by TVSAFEOPT, ETSAFEOPT, and SAFEOPT at t 30,
t = 100 and = 170. All algorithms start from the same singleton initial safe set S. = (.50.5, 0.0) Implementation details are described in Appendix A. Figure I illustrates that the safe sets computed by TVSAFEOPT are contained in the ground truth safe regions while those computed by SAFEOPT and ETSAFEOPT have multiple violations. Due to the dependence the initial safe set becomes unsafe at f = 30, and into consideration, TVSAFEOPT correctly identifies the possible unsafety of the initial safe set. In contrast, SAFEOPT always consider the initial safe set to be safe. Meanwhile, ATSAFEOPT correctly identifies the lack of safety of the initial safe set the event trigger is naturally insensitive to continuous changes. This toy example indicates that, in contrast SAFEOPT and ETSAFEOPT, TVSAFEOPT safely adapts the time changes of the optimization problem.
time of the example (Figure 3), 170. Taking the possible changes in time
30, but fails at 170. This is because
In this example, TVSAFEOPT and ETSAFEOPT overall find better reward function values than SAFEOPT, see Figure 3. The reward function value found by TVSAFEOPT is close to the optimal values when the reward function changes slowly, which supports Theorem 2.6.
Taking SAFEOPT as baseline, TVSAFEOPT and ETSAFEOPT achieves less violations, lower cumulative regret at the cost of covering less part of the safe region. Furthermore, TVSAFEOPT has little violations, and achieves larger coverage ratio than ETSAFEOPT. Mcanwhile, its cumulative regret is just slightly higher than that of ETSAFEOPT.
Quantitative metrics are listed in Table 2.

---

Table 2: Synthetic example: comparison TVSAFEOPT ETSAFEOPT with respect to SAFEOPT, showing the average and the standard deviation results from five runs with different initial safe sets (chosen randomly from the feasible space).
ETSAFEOPT
TVSAFROPT
Violations
-84,4% + 1.7 %
-99.99% 1 0.01%
Coverage Ratio
-30.9% +: 2.9 %
-21.0% + 1.3%
Cumulative Regret
-73.6% + 14.7%
-66.9% 14.4%
Table 3: Compressors case study: comparison of TVSAFEOPT and SAFEOPT with respect to Approximate Optimization, showing the average and the standard deviation results from 10 runs with different initial safe sets (chosen randomly from (zo 0.5/ / xo + 0.5/ /3] where 2o the initial safe seed and is the distance to the boundary of the feasible region). ETSAFEOPT is not included due to its high dependency event detection methods, which unavailable for compressor degradation.
SAFEOPT
TVSAFEOPT
Violations Coverage Ratio
-89.2% = 4.2 %
-9.8% = 1.0%
-35.7% + 2.6
-61.0% 1.3%
Cumulative Regret
+95.8% + 32.3%
+178.3% + 29.2%
3.2
Gas Compressor Case Study
3.2.1
Problem Setup
We show the performance of the proposed algorithm a compressor station with three identical compressors operating in parallel the time-varying compressor head H, with time-varying power consumption at time t (adapted from [35], details in Appendix A.3)
(a1 a2m, + azli, asm, + asmil + agll?)
min
(14)
dit
s.t.
my 2M
(15)
m: 2,3 H + 32 3a vi= l.....
(16)
2n : 72 H1 + 13 Vi 1,....,
<6, 82 Vi =i 1, I,...,N
m; Sa A. + 02 Fl, 03 Vi = 1,...,N,
(17)
where the objective (14) corresponds to the power to run the station with N compressors, here N = 3. affected by individual degradation du, 1,,..., N. The station must also satisfy time-varying demand M, in (15). In practice, is common to linearly approximate (16)-117) with respect to the compressor head H, (dashed lines Figure 4) [36].
3.2.2 Results
We compare the performance of TVSAFEOPT, SAFEOPT, and approximate optimization. ETSAFEOPT is not applicable to this case study because the magnitude of changes in the demand M,, compressor head H1, and degradation di keeps triggering the event trigger and thus the maintained safe set becomes empty very quickly. Implementation details are described in Appendix A. Figure compares the number of unsafe decisions in the safe sets calculated by TVSAFEOPT, SAFEOPT, and approximate optimization. We see that, by considering the uncertainty with respect to the decision variables. SAFEOPT maintains fewer unsafe decisions its safe sets than the approximate optimiza- tion. However, SAFEOPT tends to expand its safe sets regardless of external changes. TVSAFEOPT further improves this based on SAFEOPT by taking into consideration the time- varying safety func- tions. TVSAFEOPT robustly shrinks its safe sets based on its observations and thus maintains much

---

(synthetic)
reward function values (compressor)
TVSareOPT
SaleOFT
SetedeT
Approce
Oytimll
Optina
-30
00 333 340
300
100
60 50
Brne Indox
0ays
Figure 3: Comparison of reward functions from different methods with different initial safe sets, averaged over runs for the synthetic example (left) and 10 runs for the compressor case study (right, indicating power in MW obtained from maximization of (14)), with error bars, with respect to the optimal values (black). In the synthetic example, TVSAFEOPT finds better reward function values than SAFEOPT, and similar to these of ETSAFEOPT. In the compressor case study, TVSAFEOPT finds lower reward function values than SAFEOPT, but guarantees fewer violations (Table and 3) than either SAFEOPT or Approximate Optimization.
less violations in its safe sets than SAFEOPT (70.4%) and approximate optimization (96.8%). achieves this at the cost of covering less of the ground truth safe region than SAFEOPT (39.3%) and Approximate Optimization (61.0%).
The right-hand side of Figure 3 shows that TVSAFEOPT preserves safety at the expense of optimality. In the compressor case study, AFSAFF overall finds lower reward function values than SAFEOPT and approximate optimization, which is consistent with the fact that it covers lower fraction of the ground truth safe regions and the reward function changes significantly between iterations. Because of its strong focus on safety, TVSAFEOPT deviates more from the ground truth. The cumulative regret of TFSAFOPP is above the one of SAFEOPT by 42. %, and the one of approximate optimization by 178.3%. This illustrates the trade-off between safety and optimality in the presence of strong uncertainties due to the varying reward and safety constraints. Quantitative metrics using Approximate Optimization the baseline are listed in Table 3.
4 Limitations and Conclusion
Limitations The compressor case study demonstrated that TVSAFEOPT ensures safety at the expense of optimality if the stationarity assumption is not satisfied. The assumption about the local stationarity of the optimization problem (1) is thus the main limitation. Even though TVSAFEOPT demonstrates good empirical performance with respect safety even when the problem non-stationary, theoretical guarantee for its near-optimality in the non-stationary case warrant further investigation.
The need for obtaining the Lipschitz constants with respect to both x and in order compute the safe set S₃ in (6) may prove limiting in real applications. To overcome this limitation, we propose practical modifications in Appendix A.1.
Conclusions We propose TVSAFEOPT algorithm, which extends SAFEOPT to handle time-varying optimization problems. In conclusion, TVSAFFOPT outperforms SAFEOPT in terms of adaptation to changes in time and maintains fewer unsafe decisions in its safe sets for vime-narying problems. This is at the cost of covering less of the ground truth safe regions and may lead to poorer performance in terms of optimality.
We prove the safety guarantee for TVSAFEOPT in the general time-varying setting and prove its optriapptmallty guarantee for the case in which the optimization problem becomes stationary. The two theoretical results together guarantee that AFSAFE is capable of safely transferring safety of the decisions into the future and, based on the transferred safe sets, will find the near-optimal decision when the reward function stops changing. We show that TVSAFFOPT performs well in practice for the most general settings where both the reward function and the safety constraint are Lime-varying both on synthetic data and for real case study on gas compressor.
10

---

Acknowledgments and Disclosure of Funding
Research supported by NCCR Automation, National Centre of Competence in Research, funded by the Swiss National Science Foundation (grant no. 180545). Marta Zagorowska also acknowledges funding from the Marie Curie Horizon Postdoctoral Fellowship project RELIC (grant no 101063948) for writing, revisions, and the compressor case study. Alísa Rupenyan acknowledges also support from the Johann Jakob Rieter foundation.
References
[1] Daniel Lizotte, Tang Wao, Michael Bowling, and Dale Schuurmans. Automatic gait optimization with Gaussian process regression. International Joini Conference on Artificial Intelligence, pages 944-949, 2007.
[2] Ruben Cartinez-Cantinn Nando de Freitas, Arnaud Douchet, and José A. Castellanos. Active policy learning for robot planning and exploration under uncertainty. In Proceedings Roboties: Science and Systems, pages 321-328, 2007.
[3] Ziyu Wang, Masrour Zoghi, Frank Hutter, David Matheson, and Nando de Freitas. Bayesian optimization in high dimension via random embeddings. International Joint Conference on Artificial Intelligence, pages 1778-1784, 2013.
[4] Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. International Joint Conference on Muchine Learning, pages 1015-1022, 2010.
[5] Matthew Hoffman, Bobak Shahriari, and Nando de Freitas. On correlation and budget con- straints in model-based bandit optimization with application automatic machine learning. International Conference on Artificial Intelligence and Statistics, pages 365-374, 2014.
[6] Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. Proceedings of Advances in Neural Information Processing Systems, pages 2951-2959, 2012.
[7] Yanan Sui, Alkis Gotovos, Joel Burdick, and Andreas Krause. Safe exploration for optimization with Gaussian processes. In International conference on machine learning, pages 997-1000 PMLR, 2015.
[8] Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in roboties. Machine Learning, pages 1-35, 2021.
[9] Yanan Sui, Vincent Zhuang, Joel Burdick, and Yisong Yue. Stagewise safe Bayesian opti- mization with Gaussian processes. In International conference on machine learning, pages 4781-4789. PMLR, 2018
[10] Matteo Turchetta, Felix Berkenkamp, and Andreas Krause. Safe exploration for interactive machine learning. Advances in Neural Information Processing Systems, 32, 2019.
[11] Dominik Baumann, Alonso Marco, Matteo Turchetta, and Sebastian Trimpe. GoSate: Globally optimal safe robot learning. In 202/ IEEE International Conference on Robotics and Automation (ICRA), pages 4452 4458. IEEE, 2021.
[12] Bhavya Sukhija, Matteo Turchetta, David Lindner, Andreas Krause, Sebastian Trimpe, and Dominik Baumann. GoSafeOpt: Scalable safe exploration for global optimization of dynamical systems. Artificial Intelligence, page 103922, 2023.
[13] Alessandro G. Bottero, Carlos E. Luis, Julia Vinogradska, Felix Berkenkamp, and Jan Peters. Information-theoreficc safe Bayesian optimization, 2024.
[14] Jonas Hübotter, Bhavya Sukhija, Lenart Treven, Yarden As, and Andreas Krause. Information- based transductive active learning, arXiv preprint arXiv:2402. 15898, 2024.
11

---

[15] Marta Zagorowska, Efe C. Balta, Varsha Behrunani, Alisa Rupenyan, and John Lygeros. Efficient sample selection for safe learning. Pace-papesO 56(2):10107-00112 2023. 22nd IFAC World Congress.
[16] Daniel Widmer, Dongho Kang. Bhavya Sukhija, Jonas Hübotter, Andreas Krause, and Stelian Coros. Tuning legged locomotion controllers via safe Bayesian optimization. In 7th Annual Conference on Robot Learning (CoRL), 6-9 November, Atlanta, GA, 2023. URL. https:
//proceedings.. mml.imgssm..r.....mmmmm......... ptwps/v229 htt///idmme2222 html.
[17] Christopher König, Miks Ozols, Anastasia Makarova, Efe C. Balta, Andreas Krause, and Alisa Rupenyan. Safe risk-averse Bayesian optimization for controller tuning. IEEE Robotics and Automation Letters, 8(12):8208 8215, 2023.
[18] Andreas Krause and Cheng Ong. Contextual Gaussian process bandit optimization. Advances in neural information processing systems, 24, 2011.
[19] Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: no regret and experimental design. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML' 10, page 1015-1022, Madison, WI, USA, 2010. Omnipress.
[20] Marcello Fiducioso, Sebastian Curi, Benedikt Schumacher, Markus Gwerder, and Andreas Krause. Safe contextual bayesian optimization for sustainable room temperature PID control tuning. In Proceedings of the Ehrenty-Eightte International Joint Conference on Artificial Intelli- gence, IJCAI-19, pages 5850-5856. International Joint Conferences on Artificial Intelligence Organization, 2019.
[21] Christopher König, Matteo Turchetta, John Lygeros, Alisa Rupenyan, and Andreas Krause. Safe and efficient model-free adaptive control via Bayesian optimization. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9782-9788. IEEE, 2021.
[22] Antonia Holzapfel, Paul Brunzema, and Sebastian Trimpe. Event-triggered safe Bayesian optimization on quadcopters. In Alessandro Abate, Mark Cannon, Kostas Margellos, and Antonis Papachristodoulluu editors, Proceedings of the 6th Annual Leurning for Dynamics & Control Conference (L4DC), 15-17 July, Oxford, UK, volume 242 of Proceedings of Machine Learning Research, pages 1033-1045. PMLR, 15-17 Jul 2024.
[23] Ilija Bogunovic, Jonathan Scarlett, and Volkan Cevher. Time-varying Gaussian process bandit optimization. In Artificial Intelligence and Statistics, pages 314-323. PMLR, 2016.
[24] Paul Brunzema, Alexander von Rohr, Friedrich Solowjow, and Sebastian Trimee. Event- triggered time-varying Bayesian optimization. arXiv preprint arXiv:2208. 10790, 2022.
[25] Kihyuk Hong. Yuhang Li. and Ambuj Tewari. An optimization- algorithm for non- stationary kernel bandits without prior knowledge. In International Conference Artificial Intelligence and Statistics, pages 3048-3088 PMLR, 2023.
[26] Xingyu Zhou and Ness Shroff. No-regret algorithms for time-varying Bayesian optimization. In 2021 55th Annual Conference on Information Sciences and Systems (CISS), pages 1-6. IEEE, 2021.
[27] Yuntian Deng, Xingyu Zhou, Baekjin Kim, Ambuj Tewari, Abhishek Gupta, and Ness Shroff. Weighted Gaussian process bandits for non-stationary environments. In International Confer- ence Artificial Intelligence and Statistics, pages 6909-6932. PMLR, 2022.
[28] Dinesh Krishnamoorthy and Francis J Doyle. Safe Bayesian optimization using interior-point methods-and applied to personalized insulin dose guidance. IEEE Control Systems Letters. 2834-2839, 2022.
[29] Dinesh Krishnamoorthy and Francis Doyle III. Model-free real-time optimization of process systems using safe Bayesian optimization. AIChE Journal, 69(4), 2023.
12

---

[30] Lukas Hewing, Kim P Wabersich, Marcel Menner, and Melanie N Zeilinger. Learning- based model predictive control: Toward safe learning in control. Annual Review of Control, Robotics, and Autonomous Systems, 3:269-296, 2020.
[31] Carl Edward Rasmussen and Chris Williams. Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA, USA, January 2006.
[32] Bernhard Schölkopf and Alexander Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. The MIT Press, 2002.
[33] Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999.
[34] Michael Vogt and Holger Dette. Detecting gradual changes in locally stationary processes. The Annals of Statisties, 43(2):713-740, 2015.
[35] Marta Zagorowska and Nina F. Thornhill. Influence of compressor degradation on optimal load-sharing Computers and Chemical Engineering, 143(5):107104 104, 2020.
[36] Andrea Cortinovis, Joachim Ferreau, Daniel Lewandowski, and Mehmet Mercangöz. Experi- mental evaluation of MPC-based anti-surge and process control for electric driven centrifugal gas compressors. Journal of Process Control, 34:13-25, 23, 2015.
[37] Felix Berkenkamp, Angela P Schoellig, and Andreas Krause, Safe controller optimization for quadrotors with Gaussian processes. In 2016 IEEE International Conference on Robotics and Automation (ICRA), 16-21 May. Stockholm, Sweden, pages 491-496 IEEE, 2016.
[38] Rainer Kurz, Matt Lubomirsky, and Klaus Brun. Gas compressor station economic optimization. International Journal of Rotating Machinery, 2012:Article ID 715017, 9 pages, 2012.
[39] Predrag Milosavljevic, Alejandro G. Marchetti, Andrea Cortinovis, Timm Faulwasser, Mehmet Mercangöz, and Dominique Bonvin. Real-time optimization of load sharing for gas compressors in the presence of uncertainty. Applied Energy, 272:1 14883, 2020.
[40] Marta Zagorowska. Degradation modelling in process control applications. PhD thesis, Imperial College London, 2020. available at https::/spiree imperial. ac. wwk.handlee1004//// 105173, online 22 May 2024.
[41] Do Won Kang and Tong Scop Kim. Model-based performance diagnostics of heavy-duty gas turbines using compressor map adaptation. Applied Energy, 212:1345-1359, 2018.
[42] Andrea Cortinovis, Mehmet Mercangöz, Matteo Zovadelli, Diego Pareschi, Antonio De Marco, and Sergio Bittanti. Online performance tracking and load sharing optimization for parallel operation of gas compressors. Computers & Chemical Engineering, 88:145-156, 5 2016.
[43] Ayman Al Zawaideh, Khalifa Al Hosani, Igor Boiko, and Mohammad Luai Hammadih. Mini- mum energy adaptive load sharing of parallel operated compressors. IEEE Open Journal of Industry Applications, 1:178-191,202.
[44] Vibeke Stærkebye Nørstebø. Optimum Operation of Gas Export Systems. PhD thesis, Norwegian University of Science and Technology, 2008
[45] Rainer Kurz and Klaus Brun. Degradation of gas turbine performance in natural gas service. Journal of Natural Gas Science and Engineering. 1(3):95-100 -02, 2009.
[46] Yiuguang Li and Pannawat Nilkitsaranont. Gas turbine performance prognostic for condition- based maintenance. Applied Energy, 86(10):2152 2161, October 2009. ISSN 0306-2619,
[47] Matteo Cicciotti. Adaptive Monitoring of Health-state and Performance of Industrial Centrifugal Compressors. PhD thesis, Imperial College London, 2015.
[48] Marta Zagorowska, Frederik Schulze Spüntrup, Arne-Marius Ditlefsen, Lars Imsland, Erling Lunde, and Nina Thornhill. Adaptive detection and prediction of performance degradation off-shore turbomachinery. Applied Energy, 268:p. 114934, 2020.
13

---

[49] Yu-Zhi Chen, Xu-Dong Zhao, Heng-Chan Xiang, and Elias Tsoutsanis. A sequential model- based approach for gas turbine performance diagnostics. Energy, 220: 19657, 2021-04. ISSN 0360-5442.
[50] Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, and Colin Jones. Constrained efficient global optimization of expensive black-bor functions. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhard,, Sivan Sabato. and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 38485-38498. PMLR, 23-29 Jul 2023.
[51] Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, and Colin N Jones. Primal-dual contextual bayesian optimization for control system online optimization with time-average constraints. In 2023 62nd IEEE Conference Decision and Control (CDC), pages 4112-4117. IEEE, 2023.
14

---

A Experiment Details
Experiments are conducted on an Intel 17-11370H CPU using Python 3.8.5. The implementation utilizes the following libraries: GPy 1.12.0, NumPy 1.22.0, and Matplotlib 3.5.0.
A.1 Practical Modifications
In practice, Lipschitz constants are difficult to estimate. Thus, here provide Lipschitz-constant free version of TVSAFEOPT algorithm by modifying (6) and (8).
The safe set is updated as all decisions with non-negative lower confidence bounds for the safety functions at the current iteration k, that is,
S E (
(x 23.
(18)
Furthermore, the expanders are intuitively defined as decisions within the current safe set such that, by evaluating any of the decisions, at least decision outside the current safe set will be considered as safe, that is, G = {x E S₁ ex (x) > 0}. where (x) denotes the number of decisions outside Se that will be considered safe when evaluating x. Instead of using Lipschitz constants, define
ex (x) using lower bound of auxiliary GP similar to the method by Berkenkamp et al. [37]
ex(x) = 1xx E D Bi Ie (x., (x', k + 1, 2 0)}
where (x..) kx', i) denotes the lower bound of the function values and + if x is evaluated at the k-th iteration and the upper bound is observed.
A.2 Synthetic Example
The search space is X' = [- 2| uniformly quantized into 100 100 points. Both algorithms start with the singleton initial safe set ((-0.5, 0.0) The measurements are perturbed by i.i.d. Gaussian noise (0, 0.012).
The reward function is formulated as: f(x, - + + + 0.01/;
llog(1
The safety function formulated as: (x,,)) = 1 - [ + 0.5 0.5 (1 - cos [y 0.3 0.5 (1 cos 50 sin X
2x 50
cos T
The hyperparameters of GPs in the synthetic case study are modelled as follows,
TVSAFEOPT: The reward function and the safety function are modeled by independent GPs with zero mean and spatio-temporal kernel n((x, t), (x', t')) exp ||x 112
20
(-+1) where σ1 1.0, 02 25.0 for and 9 15.0 for C1 20
exp
SAFEOPT: The reward function and the safety function are modeled by independent GPs
with zero mean and 2d Gaussian kernel s(x, x') = exp
|x-x'|| where = 1.0.
20
ETSAFEOPT: Hyperparameters for GPs are the same as SAFEOPT. Besides, we choose the sentivity of the event trigger 6 as 0.01.
A.3 Compressor Case Study
Centrifugal compressors are often used in gas transport networks to deliver the required amount of gas by boosting the pressure in the pipelines. Organised compressor stations with N units, the compressors are often operated to minimise their power consumption P while satisfying the demand M, and operating constraints, capturing how compressor head H, depends on the mass flow through the compressor [38, 39]:
m, mm-157..4 157.4 A. 1.0166 3.210g4 01 1.979e7, 02 = 5.274e6, 03 5.375e6, 04
34.37
6.055e5, ag 5. 5.718e5, ag 3.319e5
15

---

H
H. 1.225e5
.1 -1.533, 32 16.86, 83 = 118.1
3 3.764e1
H, H, 22-66,11 6.1eee
-1.566 11.122 = 116.9
TOU2
H, H. H4-8.7000 70064
61 = 73.21, 82 = 183.7
6.289e4
H.
H. 1.572c5
-7.260, 29.65, 03 204.4
2.044e4
The compressor case study has been adapted from [35]. The data for the demand, compressor head, and degradation for the three compressors were obtained from [40] (Creative Commons Attribution NonCommercial Licence).
Individual characteristics of compressors in (16)-117) are called compressor maps (Figure 4). The operating area for compressor is defined by minimal and maximal speed the compressor and its mechanical properties. The operating area can be obtained from compressors maps delivered by the manufacturer of the compressor, or estimated during the operation [41]. However, estimation would require collecting datapoints close to the boundary of the operating area, which may be unavailable due to safety consideration [42, 43]. Using safe learning has the potential to improve the operation of the station because it enables safe exploration of the unknown operating area of individual compressors.
x10
Sunge
Manorure
Compressor head kg
1.8
Min. speed
1.8
1.4
Mass [kg s'1]
Figure 4: Ground truth (solid) and linear approximation (dashed) of the operating area from com- pressor maps, adapted from [44, 35]. For given compressor head at time / (dotted horizontal line for H, = 1 120000 J kg-1), the mass flow ma through the i-th compressor is required to be between minimum speed (red) and surge (blue) lines, and maximum speed (violet) and choke (yellow) lines
Furthermore, varying operating conditions and demand often lead to compressor degradation du (Figure 5). over time increasing power consumption (14) of the entire compressor station [45]. Capturing the tire-varying aspect compressor degradation a subject of research (c.g. [46-48]) but limited availability of measured degradation data presents challenge [49].
For convenience, the optimization variables are scaled by factor K 200, that is, x (m1, m2, ma)/K. K. The search space is X' [50.0/ 60 60 x 60 points. Both algorithms start with the singleton initial safe set ((Mg, /33K)) The measurements are perturbed by i.i.d. Gaussian noise (00, 0.01²).
10.0/K K, 750.0/K13 uniformly quantized into
The reward function is formulated as:
f(x,t) Σ i=1
du)) 107 (as + agm, agHH + aym; asm, + as ag
The safety functions are formulated cx(x,t) 2 i 1, ... with:
cx( Iı
16

---

1.2
Demand
145
Compressor Head
Degradation
(a)
0.00
Compressor
(c)
1.50
Campressor
Compressor
1:25
WWM m
0.03
0.25
0.00
days
days
days
Figure 5: Visualization of demand (a), compressor head (b), and degradation for the compressors (c) changing with time.
(x, = U2- E1
(x,,t) I2 Lt
(x,,t) = U1 I2
(x(x,t) = I3 4r
co(x, 1) U. J3
c7(x11) .ı 12 33g 0.67M./K
where L. m + 32H, Bs H + 12 + 7///K, Us min 82 H,
02 H, + 03)/K.
The hyperparameters of GPs in the compressor case study are modelled as follows,
TVSAFEOPT: The reward function and the safety functions are modeled by independent GPs with zero mean and spatio-temports kernel s((x, t), (x', f')) cxp x|x-x' 113
20
exp (-(--r))) where 01 = 1.0, 02 = 80.0 for f and e Co and 02 70.0 for C7. 20
SAFEOPT: The reward function and the safety functions are modeled by independent GPs
with zero mean and 3d Gaussian kernel x(x,x' x') exp
x |x-x'11 113 where 03 1.0.
20
As for approximate optimization, the r.h.s. of (16) (17) are linearly approximated as follows:
Surge line: F3 3, B3 = 4.481c 4 H, 59.76
Min. speed line: 71 H1 + 72 H. + 73 ~ -.333e - 3 H, + 193.3
Choke line: 6, H + 82 ~ 1.611e 3- H. + 46.77
Max. speed line: o, म H. + 03 1.667e 3 H, + 461.7
17

---

B Proof of Safety Guarantee
Note all following lemmas hold for any ó (0, 1), and S.. such that S So S X.
First, we want to show that the intersected confidence interval Ce (4) w.h.p. contains the reward function and safety functions h(x, t, i) in (2).
Lemma B.I. Let VB&- = B + ov (7k- ++ 1u(1/0)). with T defined as (12) and Cx(x, i)
defined as in (4), then the following holds with probability at least 1 - 6:
1(x,1,)) (x(,
0, Z
Proof by induction. If t = O, by Assumption 2.3 and the definition of C₁ in (4), then h(x, 0, E Co(x,i), for all i I and for all
X.
Suppose, for any t = r 2 O, that h(x, 7, E C, 1), then for = r + 1, from the Lipschitz continuity of h, |h(x,7 + 1, T,), v L(r), it holds that h(x, F + 1, Cx(x,)) t [-L(T)
Moreover, by Lemma 2.4 and (3) we have that 1(x, 1, (x-
Thus, h(x, - L, E (C, (x, i) @ ((),,, (T))) n nQ++1(x,)) (x,) Vie Vx X.
Therefore, for all 2 0, for all E I and for all x E X we have that /(x,, t, Cx(x, i), and this concludes the proof.
We are now ready to prove Theorem 2.5 that provides a sufficient condition for TVSAFEOPT to ensure safety embedded in the constraints c, (x, () 0, for all I,..
Proof of Theorem 2.5.
If 0, by definition of So, one has cxxx () - (x 0) > L(0) 20 VXT.,.XEE So.
For any 2 1, Vx E St. by recursive definition of S (6), Vi E Ie, there exists x' E St-1, s.t. 4xx',, i) Lx x(1 x') L(t) 0. Then, Vi E Ie
(x,,t)
2 2x f) - Lxd(x,x)
by Lipschitz continuity with x
24(x, i) xxd(x,x))
by Lemma B.I
(x - Ixd(xxxX x') - I(t))
20
and this concludes the proof.
18

---

C Proof of Near-Optimality Guarantee
The proof of near optimality consists in two parts: i) bounding the uncertainty and ii) bounding the expansion of the safe set.
C.1 Bounding the Uncertainty
We first derive decaying upper bound of uncertainty for TVSAFEOPT. In this way we can ensure the uncertainty of the reward function and safety functions to drop below a desired threshold.
Lemma C.I. Define bi 8/ log 3- E R. and 7 in (12). For any k > ko 2 1, there exists k' E (ko, k), such that the following holds all E I:
by 1 Tk
wy (xx, 2) S V
h: ko
Proof.
Let iy == arg max Wy (X,, i), where x, arg max max W. (x, For all i E ko k, there XEG,UM, VEI
iET
exists K' (kg.
We (XM'
wr (X
k kip
k=
(a) 2
(x,
k:
Ko
1o+1
2/Bk
k
0--1 (
h: Ko
1
43
Σ
kn
r=k
b
Σ
log + 02
ko 2
r=
b
Σ
log(1 + 2- (x,;,,
Ko 2
11
Br I (hx,;
k: ko
INS
b Bank
k: ko
(a): Definition of wa in (5),
(b): From the fact that the quadratic mean upper bounds the arithmetic mean,
(c): (x,,
& (1x,,,,ir)) (x,),, < 1 by Assumption 2.1, and the fact that < by log(1 + 2a), Va E [0, 1],
8
(d): (x, denotes the posterior standard deviation of 7,) inferred by observations X, = ((x),) Since ((x).).15))) & ((,,5))err x I, then 1 (7) S
o (x,,
19

---

(e): From [19, Lemma 5.3],
(f): Definition of 2 (12).
Corollary C.2. Given := 8 8/log (1 -1- E IR, lake Ta as the smallest positive integer satisfying
Ty. 2 2.. Then, there exists k' E (k, k + Ti], such that for any x E Gk U My, and for all
BE+Ty "K+Ty
i E I it holds that
wx,(x, 1) c.
C.2 Bounding the Expansion the Safe Set
All following lemmas hold for any d E 10, 0 and So. such that Ø & So C X.
To facilitate the theoretical analysis, define Vx E E
[xxx 1) = (x), ()(x, = (x(,
3 1x,,
k
(19)
Remember that, from (4), we can derive VX E X. Vi T:
4xx i) max
L(t - 1), 1k-1 1 i) 3362 Tk (x,
(20)
Therefore, La can be viewed as updating L/ with L(t) = 0. With slight abuse of notation, we omit arguments and when not ambiguous. Lemma C.3. The following holds for any k 2 1, VX E W.e
(i) (x i) 2 11-1 L(t 1) 1)
(ii) [x(x, i) 2 (xx,
(ii) (x(x,)) < (x, i)
(iv) [x(x,)) i) - L, S (x.
Proof.
(i) Direct consequence of (20).
(ii) Direct consequence (19).
(iii) We proceed by induction. Suppose 1, & ī, then 1, I(r) < ī,, thus according (20),
o, = i from which 4++1 = max(1, L(T), Hr 11/2 3 o,} I max(ī7, 1r - 0
follows (x( < ix.
T-1
(iv) We proceed by induction. Suppose 4 2 - Σ L(k).
k=0
If -31/2 > i., then 1-+1 (20)
/r B or ī+t1 2 [+t1
(19)
Σ L(k).
k=0
If #4 -31/3 0r L(T), then 4+1
(0) 4 L(7) i L(k) L(T)
T, Σ T(k).
Otherwise, 1+41 20 |z
11 E I(k).
-
T-1
k=
k=U
20

---

1=1 summarize, 4 24 Σ [(k) 2 Ia L
Lemma C.3 allows us define auxiliary safe sets based on l such that they are contained in Sk- Furthermore, due to the monotonicity of 4g, we can prove the auxiliary safe sets never shrink, which will play a fundamental role in studying their convergence property and provide near-optimality guarantee of TVSAFEOPT.
Based on (19), we further define:
S = [x E X Vi E Le, 3x St-1, 8.1. (x(,,,1) [xd((x, x() 2 0]
S. = (x X Vi E Ie, 3x s.t. (x(,,,1) Lxd(xxx xx) L, 2 0}
S, S. So
Remember S4 = fx E X Vi E Iz, Ex, E Stt1,8.t (x(,,)) Lxd(x, xx) L(t) 2 0}. Thus, S = S = Sy if and only if L(t) = O.
The following lemma proves that S. never shrinks, and that S. and SE are subset and a superset for Sk, respectively.
Lemma C.4. The following holds for any t 1:
(i) S C Sx
(ii) S S Su Sk
Proof.
(i) We refer the reader [8, Lemma 7.1].
(ii) We proceed by induction. Suppose S, S, S,
-x) Lxd(xx For S741, and for all i Ie. there exists x, S, S Stt 1)
(x(,, i) Txd((x, x() T(r) 0, hence S7+1 well. Therefore, S C S VT.
For x E S and for all E I., there exists x E S. C Sy S.L. L, (X,, 1) I x, x()
I(T) 2 (x(, 1) Σ L(k) Lxd(xx x() L(T) = (x(, i) Lxd(xx x)) Σ L(k)
k=
k=
(x(x, i) (x,( x() L 2 thus S741. Therefore, S, C Sr, Vr. From which
conclude S S, S, VT. VT.
Note: Where needed in the following lemmas, we assume by and T3 are defined as in Lemma C. and Corollary C.2
Lemma C.5 (Lemma 7.4 in [8)). For any k 2 a if R, (S6) SS, + Ø, then R2 (S+) SS + ø.
The following lemma provides sufficient condition for the expansion of the auxiliary safe set Sx- Lemma C.6. For any /. 2 1, if (So) + 2,, then, with probability at least 8, it holds
that Sa4Ty 2 Sx-
Proof.
Similar to the proof of [8, Lemma 7.5].
By Lemma C.5, we get that, Ri (Sx) SS * B. Equivalently, 3x E Ri (S+) Sr which implies that, for all E Ic,
3z, Sy (Z,)) Lxd(z,,)) L, -=20
21

---

Now assume, to the contrary, that SKTT, = k.. Thus, Ak' E (ke k Tal, x E D( and vi E Ic. Zj E Sg.
Up [z 1) Lxd(z,, - -(k')
(z( - Txx((2i x) I(K))
by Lemma B.1
(0((2i) Tuel(Zis x) In--
20
Therefore, by definition (8), ex (2:) > O, which implies 2, E Gk, AA E (k: h: Ta], Vi Therefore, we know that there exists k' E (ke, k + Ta], for all E Ies ww i) S c. (Corollary C.2) Hence, for all E Ic.
(z,, 1) - Lxd(zi, x)
(2())) WK (Z), - Lxd(2z,
by Lemma B.1
(z,))) lxxd(2s;)) x)
I
This means Sy Sx. which leads contradiction.
The following lemma gives a superset for the auxiliary safe S.. Lemma C.7. Se S RI, (So) with probability at least - 5.
Proof by induction.
S. = So S Rí, (So)
Suppose S, S (So).
For all x E St+1 and for all i E Io there exists x E S,, 24 (x,) - x,) - L. (4) 1x(x),)) Txdl(x, x() T. 2 0.
(a): Lemma B.I.
Thus, S,+1 C RI, (S,) C RL, (5o)
Lemma C.8 (Lemma 7.8 in [8]). Let k= be the smallest integer, such that k. Then, there exists ko & k., such that S = =Sko
Ri, (So) Tk.
Lemma C.8 together with Lemma C.6, and Lemma C.7 entail convergence of S within ke time steps, which ultimately leads us the near-optimality of TVSAFEOPT when the problem becomes stationary.
Lemma C.9. For any k 2 1, SatT. St then, with probability least 1 ô, there exists k' E (k, To] such that
[(xx)
f(x)
max
xe xE
Proof.
Let X == arg max f(x). Note that X E Mys since
XESy
(a)
uy (xp. 0)
f(x)))
IV
f ((t))
IV IV3
ler (Stx,, 0)
max (x, 0)
xES
22

---

(a) and (b): Lemma B.1,
(c): Definition of Ry (11).
We will first show that 3K' (k, k + Tal, 8.1. [(xx) ) 2 f(xx)) ) - c. Assume, to the contrary, that Wk' E (k, k + Tal, f(xw) < f(x()) -e
Then, have, 3K' E (k: k Tkl
(x(x),,0) 0)
(2)
2 W (RM,, 0)
S f(tw))
<f(x()) -
S (x 0);
which is contradiction.
(d): Definition of x (11),
(c): Lemma B.I.
(f): Corollary C.2, and x E Mr
Finally, RL.te (So) S St C Sg, by Lemma C.6 and Lemma C.4 (ii). Therefore, ANE E (k, + Tyl such that
max f(x) -
max f(x) xESp
x RR R 1. (86)
f(xx))
f(xx)
C.3 Near-Optimality Proof
Proof of Theorem 2.6. Theorem 2.6 is direct consequence of Corollary C.2, Lemma C.8, and Lemma C.9.
D Practical Considerations
D.1 Trade-off between Safety and Optimality
In this work, focus safety critical systems where satisfying the safety constraints has highest priority over finding the optima. Through pessimistically considering change with time in the decision- making process, TVSAFEOPT emphasizes safety non-stationary conditions at the inevitable expense of optimality, In practice, such sacrifice on optimality can be alleviated by tighter bound of rate of change.
Besides, in the case where safety can be to some extent comprised, which is beyond the focus of this work, constrained BO and its time-varying extension would be more suitable method apply. We refer the readers to [50, 51] for further information.
D.2 Scalability
Explicit considering time can be viewed roughly as adding dimension by 1, Therefore, TVSAFEOPT achieves time adaptation without adding much computational cost. With the increase in dimensionality of the problem, safety constraints might arise across multiple dimensions, from multiple directions at price of optimality. As our approach is suitable for safety critical conditions, the focus is put on maintaining safety under change, therefore safety considerations "dictate" the optima. Additionally, in practice, the safety functions are modeled independently with a GP, and thus the computational cost scales linearly with the number of constraints.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The paper set out to solve the problem optimizing an unknown time-varying reward subject to unknown time-varying safety constraints, stated in the Abstract and the Introduction (Section 1). In partial fulfilment of this goal, the paper develops a safe learning algorithm and provides safety guarantees for general time-varyy varying case, and near-optimality guarantees for when the time-varying problem becomes stationary.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper contains separate section on limitations, see Section 4
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer:
[Yes]
Justification: The paper provides the necessary assumptions in Section 2.1, the definitions in Section 2, and proofs in Appendices B and C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides the necessary information in Section 3 and the background details are in Appendix A. In particular, the modifications necessary for practical imple- mentation the proposed algorithm, for example regarding the Lipschitz constants, are provided in Appendix A. and the hyperparameters for the examples are in Appendix A.2 and Appendix A.3.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
25

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper provides the details necessary to reproduce the simulational results in Appendix A. The code accompanying the paper currently under review and will appear shortly at https://wwe www.eacch-clleeecceccccccc--- wwweeiinnnnttz.cccmmo ch/.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips. pttppppbbbcuudies...ppppppp Sttps//ooeSSuSesiieeeeee ssionsolicyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
reproducible, they
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides the necessary information in Section 3 and the background details are in Appendix A. In particular, the modifications necessary for practical imple- mentation of the proposed algorithm, for example regarding the Lipschitz constants, provided in Appendix A. and the hyperparameters for the examples are in Appendix A.2 and Appendix A.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
26

---

Answer: [Yes]
Justification: The main contribution of the paper is in theoretical guarantees for the proposed algorithm. The only source of uncertainty in the experiments in assumed mcasurement noise, with the standard deviation 0.01, making error bars negligible, as stated in respective experimental sections.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides the necessary information in Section 3 and the background details are in Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: /ttp:://///euueeee htt///nurrpp..// wwwpourips.cpppuullleeeeeeeceeee httpppubii./..iiccuuuieeeeeeaeee
Answer: [Yes]
The paper uses neither external datasets nor human subjects, and no data have been collected for the paper. The algorithm proposed in the paper is optimization algorithm and thus does not introduce additional biases or privacy and dual-use concerns. The necessary licences for fair use have been provided in Appendix A.3.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: The paper states the need for safe optimization algorithm with potential applications in the Introduction (Section 1, in particular Section 1.2.1).
The algorithm proposed in the paper is optimization algorithm and thus does not introduce additional biases or privacy and dual-use concerns. The algorithm does not use external datasets, and as such does require privacy or security considerations, Neither does the algorithm generate data that can be used for disinformation or discrimination.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
27

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: The paper does not contain data of high risk for misuse. The necessary licences for fair use have been provided in Appendix A.3.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification; All previously existing results used in the paper have been cited in References. including the necessary licensing information for the data for the compressor case study in Appendix A.3.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce additional assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper uses neither crowdsourcing nor human subjects.
Guidelines:
The answer NA means that the paper dlees not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not use human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
29

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "A4IEyk6Dur", "reviewer": "Reviewer_o5DK", "summary": "The paper introduces the TVSAFEOPT algorithm, which is based on Gaussian processes with spatio-temporal kernels, designed specifically for optimizing time-varying rewards under time-varying safety constraints. The algorithm provides formal safety guarantees in a general time-varying setting, ensuring safety even when exploring non-stationary safe regions. It robustly subtracts safety margins to prevent unsafe decisions, adapting in real-time to changing environments. Furthermore, they provide optimality guarantees for locally stationary optimization problems, ensuring near-optimal solutions when the optimization problem becomes stationary.", "questions": "-", "limitations": "-", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "They provide formal safety guarantees in dynamic environments, ensuring safe decision-making even in non-stationary settings. \n\nAdditionally, the algorithm offers optimality guarantees for stationary optimization problems, enhancing its reliability and performance\n\nExtensive numerical simulations were provided to validate the proposed approach.", "weaknesses": "They extend the Safeopt algorithm from literature. However, it is clear on what are the additional contributions and difference between these two different approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1:They extend the Safeopt algorithm from literature. However, it is clear on what are the additional contributions and difference between these two different approaches.**\n\nWe thank the reviewer for the positive assessment of our paper, and for their constructive feedback. We now provide a table (see Table 1 in the PDF and refer to References) to give a clear overview of the contributions of our proposed approach TVSafeOPT, compared  with other methods associated with time-varying safe optimization. In contrast to safe learning methods based on contextual Bayesian optimization, such as Contextual SafeOPT [1], which rely on using time as a context and thus require a safe seed for every context, our TVSafeOPT requires only an initial safe seed. Adaptive Goal Oriented Safe Exploration (A-GoOSE) can also handle a single initial seed, but without providing theoretical guarantees on safety across time [3,4]. Furthermore, compared to event-triggered methods such as ETSafeOPT [2], our method requires the initial safe set to be provided only once, at the initial time instant, $t=0$. In contrast, ETSafeOPT requires a new safe state initialisation after each iteration. Moreover, we are able to provide  both safety and convergence guarantees to the optimal solution within the safe set, while ETSafeOPT guarantees only safety.\n\n**References**\n\n    \n[1] F. Berkenkamp, A. Krause, A. P. Schoellig, \\emph{Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics},Machine Learning 2021;\n\n[2] A. Holzapfel, P. Brunzema, S. Trimpe, \\emph{Event-triggered safe Bayesian optimization on quadcopters}, L4DC 2024;\n\n[3] C. K\\\"onig, M. Turchetta, J. Lygeros, A. Rupenyan, and A. Krause, \\emph{Safe and efficient model-free adaptive control via Bayesian optimization}, ICRA, 2021;\n\n[4] C. K\\\"onig, M. Ozols, A. Makarova, E. C. Balta, A. Krause, and A. Rupenyan, \\emph{Safe risk-averse Bayesian optimization for controller tuning}, IEEE Rob. and Autom. Letters, 2023;\n\n[5] D. Widmer, D. Kang, B. Sukhija, J. H\\\"ubotter, A. Krause, and S. Coros. Tuning legged locomotion controllers via safe Bayesian optimiztion, CoRL, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "84Okown8ES", "reviewer": "Reviewer_g7A9", "summary": "This paper presents a safe Bayesian optimization algorithm TVSAFEOPT with a spatial-temporal kernel and time Lipschitz constants, which improves on SAFEOPT with time-varying reward and safety constraints. The optimality guarantee is proved for the stationary case and the safety guarantee for more general settings. The method is tested on a synthetic problem and gas compressors.", "questions": "1. How to find an initial safe set for complex problems?\n2. How to find the kernel parameter for each task?\n2. What is the computational complexity compared to other BO baselines?", "limitations": "The societal impact is discussed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The use of a spatio-temporal kernel in Bayesian optimization for time-varying safety constraints is novel.\n2. A formal proof of safety and optimality guarantee under certain assumptions.", "weaknesses": "1. More discussion on how to make a tradeoff between optimality and safety is encouraged. \n2. Will this conservatism in safety become too large in high-dimensional problems?\n2. The method to choose the proper initial safe set and kernel parameters is unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the useful feedback, and the positive assessment of the paper. We provide a point-by-point answers to all raised suggestions, comments, and questions.\n\n **W1**\n\nWe thank the reviewer for this suggestion. In the paper we are focused on safety critical systems where satisfying the safety constraints has highest priority over finding the optimum. The focus on safety is illustrated in Fig. 1 in the paper, where the proposed algorithm follows the safe set, even though the safe set changes over time. In particular, at time $t=170$, the algorithms takes decisions within a safe set that is smaller than the initial safe set, further emphasizing the safety at the expense of optimality. We will discuss these aspects in the Appendix C of the revised paper.\n    \n**W2**\n\nWe thank the reviewer for raising this question. As our approach is suitable for safety critical conditions, the focus is put on maintaining safety under change, therefore safety considerations ``dictate'' the optima. If the dimensionality of the problem increases, safety constraints might arise across multiple dimensions, from multiple directions at the price of optimality. This clarification will be included in Appendix C. \n\n\n   **W3**\n\n Indeed, the choice of the safe set and the kernel is part of ongoing research [6]  and typically requires some extended knowledge about the underlying functions[7.8]. A common choice for the kernel is a squared exponential kernel that intuitively puts emphasis on the points that are close to each other. The meaning of ``close'' is further defined by hyperparameters, which can be obtained for instance through maximum likelihood estimation [9]. \n    \n Regarding the choice of the safe set, often a single safe point is already sufficient [Ch. 16, 10]. One of our assumptions is that at least one safe point is available in the initial safe set, and this is most often the case in practice.  In most application it is reasonable to assume some prior knowledge is available, which in turn can be exploited in the initialization phase. This assumption is a direct consequence of the underlying principle that both the objective and the constraint functions can be measured. For example, in the compressor optimization case, the safe point is obtained from domain knowledge - an equal distribution of the gas is a safe point, but may be far from optimal if the compressors are dissimilar [10].\n\nWe further point out that most of the algorithms in this domain  require the safe set to be reinitialized after each iteration (see Table 1 in the PDF and the associated references at the end of this paragraph). In contrast, TVSafeOPT requires only one initial safe set at $t=0$. As indicated, we will dedicate a subsection in Appendix C to address these points.\n\n\n\n  **Q1**\n\n In complex setting it is still reasonable to assume that at least one single safe point is available. As we also mention in the answer to Weakness 3 raised from the reviewer, this would be already sufficient for TVSafeOPT to run and to guarantee optimality within the safe region that can either  enlarge or shrink along time (see Fig. 1 in the submitted manuscript).\n \n For example, in the compressor optimization case, the safe point is obtained from domain knowledge - an equal distribution of the gas is a safe point, but may be far from optimal if the compressors are dissimilar [11].\n\n      \n**Q2**\n    \nThe choice of the kernel and the hyperparameters of the algorithm is a part of ongoing research [6]  and typically requires some extended knowledge about the underlying functions [7,8]. A common choice for the kernel is a squared exponential kernel that intuitively puts emphasis on the points that are close to each other. The meaning of ``close'' is further defined by hyperparameters, which can be obtained for instance through maximum likelihood estimation [10]. One way to adjust the hyperparameters is to collect data in advance and optimize the hyperparameters via maximum likelihood optimization, following standard GP regression methods.\n\n**Q3**\n\nThrough observations during the simulations, the computational complexity of each iteration of TVSafeOpt is only slightly heavier than SafeOpt; The sample complexity of the algorithm is similar to SafeOpt [1] in the stationary case and scales as $\\mathcal{O}(\\varepsilon^{-2})$, where $\\varepsilon$ is the accuracy threshold for the optimization. In the non-stationary case, it is a still open problem to bound the sample complexity.\n\n\n\n\n\n**References** \n\n[1] F. Berkenkamp, A. Krause, A. P. Schoellig, \\emph{Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics},Machine Learning 2021;\n\n[2] A. Holzapfel, P. Brunzema, S. Trimpe, \\emph{Event-triggered safe Bayesian optimization on quadcopters}, L4DC 2024;\n\n[3] C. K\\\"onig, M. Turchetta, J. Lygeros, A. Rupenyan, and A. Krause, \\emph{Safe and efficient model-free adaptive control via Bayesian optimization}, ICRA, 2021;\n\n[4] C. K\\\"onig, M. Ozols, A. Makarova, E. C. Balta, A. Krause, and A. Rupenyan, \\emph{Safe risk-averse Bayesian optimization for controller tuning}, IEEE Rob. and Autom. Letters, 2023;\n\n[5] D. Widmer, D. Kang, B. Sukhija, J. H\\\"ubotter, A. Krause, and S. Coros. Tuning legged locomotion controllers via safe Bayesian optimiztion, CoRL, 2023.\n\n[6] C. Fiedler, J. Menn, L. Kreisk ̈other, S. Trimpe, On safety in safe Bayesian Optimization, arXiv, 2024\n\n\n[7] J. Bergstra, R. Bardenet, Y. Bengio, B. Kegl, Algorithms for hyper-parameter optimization, NIPS, 2021 \n\n[8] F. Berkenkamp,  A. P. Schoellig, A. Krause, Safe controller optimization for quadrotors with Gaussian processes, ICRA, 2016.\n\n[9] Y. Sui, A. Gotovos, J. Burdick, A. Krause, Safe exploration for optimization with Gaussian processes, PMLR, 2015.\n\n[10] Y. Li, P. Nikitsaranont, Gas turbine performance prognostic for condition-based manteinance. Applied Energy, 2009.\n\n[11] B. G. Liptak, Instrument Engineers' Handbook, vol. 2, 2005", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ae4td0N9sA", "reviewer": "Reviewer_yo4J", "summary": "The authors propose a time-varying extension of SAFEOPT to overcome the problems of time-varying rewards under time-varying safety constraints.\n\nUnder stationarity conditions, optimality guarantees are provided and the numerical simluation shows a (favorable) comparison to the SAFEOPT.", "questions": "1. *In the Appendix, a spatio-temporal SE kernel is introduced. How is this construction different from using an SE-ARD kernel with a composite variable $z = [x^T,t]^T$?* If I am not mistaken, for the SE-ARD kernel there would be no different than defining a single kernel with $z$. \n\n2. It is mentioned that the Lipschitz constants are to be known beforehand. However, while commonly assumed, *how do you get a hold of an RKHS norm bound $B$ (related to Assumption 2.1) to compute the UCB?* \n\n3. *Could you provide Figure 1 sooner in the manuscript?* It would be super helpful to see this central illustration already on page 2.", "limitations": "1. Practicality of the safety guarantee: requiring many Lipschitz constants for both for space and time while also requiring an RKHS norm bound.\n\n2. Theoretical and empirical impact: Lack of comparisons to TVSBO approaches makes the implications of the contribution unclear both theoretically and empirically.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is very well written and easy to follow.\n\n2. Based on related work, the problems of time-varying rewards under time-varying safety constraints are an open problem in literature, and his paper addresses that.\n\n3. The paper provides formal safety guarantees for their TVSAFEOPT algorithm.", "weaknesses": "1. *Some delineation to related work seems rather vague and requires stronger justification.* An example for TVSBO: the time-variable and temporal aspect of the kernel can just as well be interpreted as context using existing results. Perhaps a table would help here to highlight key aspects. \n\n2. *Lack of real-world data experiments and comparison to related work.* To support the downsides of existing approaches, an empirical comparison to existing TVSBO approaches mentioned in the related work section would be needed.\n\n3. The *empirical results could be more convincing* by adding a variety of initial safe sets and revised plots. The current plots/results are hard to parse. \n\n4. It would be beneficial if the *theoretical/technical challenge of extending safety to the time-varying case were more detailed*. This would streamline the presentation and help in assessing the impact of the contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1**\n\nWe thank the reviewer for the insight. We refer the reviewer to Table 1 in the pdf.  The main difference between our method and context-based methods lies in how time is handled in the safe sets. Context-based methods require a safe seed to be available for every context or at every iteration, which may be impractical, as time is not a decision variable. Conversely, out TVSafeOPT ensures safety by tightening the definition of the safe set, and thus even just an initial safe seed would be sufficient for the algorithm to find a safe optimum. \n\n\n\n   **W2**\n    \n\n We provide a comparison of the performance of our proposed algorithm, TVSafeOPT, with the solution of an approximate optimization problem used in practice (baseline) and SafeOPT, in the PDF. Contextual approaches with time as a context would require a safe seed for every context, which is unavailable for the unknown changes in a compressor. Similarly, detecting changes due to degradation in turbomachinery is notoriously difficult [10]. The performance of event-triggered SafeOPT relies on the chosen event detection method, thus making the comparison of performance in the compressor case study inaccurate.\n\nThe results of the comparison are shown in Fig. 3 in the attached PDF. \n    \n\n  **W3**\n  \nWe added a comparison of safe sets computed by TVSafeOPT (top row in Fig. 1) and SafeOPT (bottom row in Fig. 1). Because TVSafeOPT takes the possible changes in time into consideration, the safe sets computed by TVSafeOPT are contained in the ground truth safe regions while those computed by SafeOPT have multiple violations. .\n\n\n\n**W4**\n \nTechnically, the challenge arises in the non-stationary case from the fact that a safe/unsafe action at one moment might become unsafe/safe in the future. This means that a TVSBO algorithm should be aware of such information in order to find a sub-optimal solution while guaranteeing safety. ETSafeOPT [13] hopes to detect the change with an event trigger.  However, in general, when the reward or safety functions change continuously or frequently, ETSafeOPT suffers from poor performance and even loses empirical safety guarantee.  In contrast, context-based methods [1,3] well handle this issue with prior knowledge on the time change infused through the temporal kernel. However, existing context-based methods usually require an initial guess of safe set for each context value, which is often unrealistic. Our algorithm does not rely on change detection and overcomes the challenge related to the safe set initialization at every iteration. This is done by propagating the initial safe set to the future utilizing the temporal kernel.\nTheoretically, proving the safety and near-optimality guarantees for TVSBO algorithms is challenging even in the stationary case because the common proof techniques rely on the monotonicity of the lower bounds, upper bounds, and thus the safe sets, maintained by the algorithm. We lose such monotonicity by design of the algorithm. Instead we manage to circumvent this issue by proposing an lower bound of the safe set which is non-shrinking and converging. Furthermore, near-optimality guarantee in the non-stationary case is a very promising yet challenging problem. The first open-end question to answer is what kind of convergence property under what conditions we can prove for the non-stationary case.\n\n\n **Q1**\n\nIndeed, there is no technical difference in the two notations, we kept $x$ and $t$ separate to emphasise that $t$ indicates time and, as such, it is not a decision variable.\n\n**Q2**\n\n Indeed, the Lipschitz constant is usually unknown in realistic optimization scenarios. Its value can be obtained based on information about the capacity of the chosen kernel [7]. Such information, however,  may be unavailable, making the choice a challenge. Oftentimes, however, an educated guess allows overcoming the aforementioned challenge. The authors in [12] have shown that in practice, it is often sufficient to choose an upper bound on $B$ [7], $\\beta\\geq 2$. This is needed to approximate the Lipschitz constant by an optimistic upper bound. Given that our method relies on Gaussian processes, we made a localized approximation of what has been proposed in [8].\n\n**Q3**\n\nThanks. We  will apply this change.\n\n\n**L1**\n \nWe refer the reviewer to Q2.\n\n **L2**\n\n To make our contribution clear, we have now added an overview of the key aspects of existing methods for time-varying safe BO in Table 1 in the PDF. See also Fig, 2,3\n\n\n[1] F. Berkenkamp, A. Krause, A. P. Schoellig, \\emph{Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics},Machine Learning 2021;\n\n[2] A. Holzapfel, P. Brunzema, S. Trimpe, \\emph{Event-triggered safe Bayesian optimization on quadcopters}, L4DC 2024;\n\n[3] C. K\\\"onig, M. Turchetta, J. Lygeros, A. Rupenyan, and A. Krause, Safe and efficient model-free adaptive control via Bayesian optimization, ICRA, 2021;\n\n[4] C. K\\\"onig, M. Ozols, A. Makarova, E. C. Balta, A. Krause, and A. Rupenyan, {Safe risk-averse Bayesian optimization for controller tuning, 2023;\n\n[5] D. Widmer, D. Kang, B. Sukhija, J. H\\\"ubotter, A. Krause, and S. Coros. Tuning legged locomotion controllers via safe Bayesian optimiztion, CoRL, 2023.\n\n[7] J. Bergstra, R. Bardenet, Y. Bengio, B. Kegl, Algorithms for hyper-parameter optimization, NIPS, 2021 \n\n[8] F. Berkenkamp,  A. P. Schoellig, A. Krause, Safe controller optimization for quadrotors with Gaussian processes, ICRA, 2016.\n\n\n[10] Y. Li, P. Nikitsaranont, Gas turbine performance prognostic for condition-based manteinance, 2009.\n\n[12] C. K\\\"onig, M. Turchetta, J. Lygeros, A. Rupenyan, A. Krause, Safe and efficient model-free adaptive control via Bayesian optimization, ICRA, 2021\n\n[13] A. Holzapfel, P. Brunzema, S. Trimpe, Event-triggered safe Bayesian optimization on quadrotors, L4DC, 2024\n\n[14] M. Guptca, R. Wadhvani, A Rasool. Comprehensive analysis of change-point dynamics detection in time series data: A review,  2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Chairs, 

Dear Reviewers, 


Thank you for the thoughtful feedback on our manuscript. All three reviewers found our results of interest to the wide readership of NeurIPS. In particular, the reviewers appreciated the theoretical guarantees for safety and optimality of our proposed TVSafeOPT algorithm. 

The main reservation of the reviewers concerns the choice of the initial safe set and robustness of our algorithm with respect to  the initial safe set, together with a more clear explanation on how the proposed algorithm differs from the current time-varying safe learning algorithms with direct comparison. Other suggestions to further improve the manuscript ask for a clarification of the technical challenges of the time varying setting and a discussion on trading off safety and optimality.


We have now revised the manuscript to address all the comments from the reviewers, in particular:

1) We show via additional experiments how our algorithm is robust with respect to perturbation on the initial safe set;

2) We make it clear how our algorithm departs from the other time-varying optimization algorithms in the literature concerning optimization in safety critical settings, and provided Table 1 to make an overview about the state of the art and the contribution of our paper. 
    We also show, experimentally, the contribution of our algorithm with respect the the baseline SafeOPT and to a time-varying BO baseline, event-triggered BO;

3) We describe the technical challenges in providing theoretical guarantees for the case in which the objective function does not reach a steady state.

We hope that these revisions, as well as the individual answers provided to all comments and questions improve the presentation of our algorithm. We appreciate the opportunity to resubmit our manuscript for potential publication in NeurIPS and thank you in advance for your time.


Yours sincerely,

The authors

Meta-review: This paper extends the SafeOpt algorithm for online safe optimization when the safety-critical system is time-varying. The authors provide theoretical analysis on safe optimization with a spatio-temporal kernel, together with synthetic experiments and a realistic case study with gas compressors.

The meta-reviewer agrees with all reviewers that the paper has reached the bar for NeurIPS. The AC would recommend the authors to improve the paper towards a camera-ready version based on the rebuttals and discussions.
Decision: Accept (poster)