Title: Safe Time-Varying Optimization based on Gaussian Processes with Spatio-Temporal Kernel
Venue: NeurIPS 2024
Accepted: yes
Keywords: Safe learning

Bayesian optimization

Time-varying optimization
Abstract: Ensuring safety is a key aspect in sequential decision making problems, such as robotics or process control. The complexity of the underlying systems often makes finding the optimal decision challenging, especially when the safety-critical system is time-varying. Overcoming the problem of optimizing an unknown time-varying reward subject to unknown time-varying safety constraints, we propose TVSAFEOPT, a new algorithm built on Bayesian optimization with a spatio-temporal kernel. The algorithm is capable of safely tracking a time-varying safe region without the need for explicit change detection. Optimality guarantees are also provided for the algorithm when the optimization problem becomes stationary. We show that TVSAFEOPT compares favorably against SAFEOPT on synthetic data, both regarding safety and optimality. Evaluation on a realistic case study with gas compressors confirms that TVSAFEOPT ensures safety when solving time-varying optimization problems with unknown reward and safety functions.

Safe Time-Varying Optimization based on Gaussian Processes with Spatio-Temporal Kernel

Abstract

Ensuring safety is a key aspect in sequential decision making problems, such as robotics or process control. The complexity of the underlying systems often makes finding the optimal decision challenging, especially when the safety-critical system is time-varying. Overcoming the problem of optimizing an unknown time-varying reward subject to unknown time-varying safety constraints, we propose TVSAFEOPT, a new algorithm built on Bayesian optimization with a spatiotemporal kernel. The algorithm is capable of safely tracking a time-varying safe region without the need for explicit change detection. Optimality guarantees are also provided for the algorithm when the optimization problem becomes stationary. We show that TVSAFEOPT compares favorably against SAFEOPT on synthetic data, both regarding safety and optimality. Evaluation on a realistic case study with gas compressors confirms that TVSAFEOPT ensures safety when solving time-varying optimization problems with unknown reward and safety functions.

1 Introduction

We seek to interactively optimize an unknown time-varying reward function f : X × T → R , where X is a finite set of decisions, and T := { 0 , 1 , 2 , . . . , T } , T ∈ N + denotes the discretized time domain. We assume that the optimization problem is safety-critical, that is, there are constraints that evaluated decisions must satisfy with high probability. Similar to the reward, the constraints are also unknown and potentially time-varying, encoded through c i : X × T → R , i ∈ I c := { 1 , 2 , . . . , m } , where m ∈ N + denotes the number of safety constraints. The optimization problem at time t is

Both the reward function and the safety constraints are assumed to be unknown but can be evaluated. This is a plausible setting, for example, for UA V that need to perform rescue missions in dangerous and poorly lit environments.

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(a)

safe set (t=30)

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(b)

safe set (t=100)

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(c)

safe set (t=170)

initial safe seeds

safe points

maximizers

current maximizer

true maximizers

current true maximizers

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(d)

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(e)

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(f)

initial safe seeds

safe points

maximizers

current maximizer

true maximizers

current true maximizers

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(g)

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(h)

-2

-1

0

1

2

x

-2

-1

0

1

2

y

(i)

initial safe seeds

safe points

maximizers

current maximizer

true maximizers

current true maximizers

1.1 Related Work

Bayesian Optimization (BO) is a well-established approach for interactively optimizing unknown reward functions. Various BO based approaches have been proposed to solve a wide range of problems in robotics [1, 2], combinatorial optimization [3], sensor networks [4], and automatic machine learning [5, 6]. However, Safe Bayesian Optimization in the time-varying setting is still under-explored.

Safe Bayesian Optimization To address safety requirements in safety-critical applications, Safe Bayesian Optimization (SBO) [7] has been proposed to avoid unsafe decisions with high probability by interactively optimizing a reward function under safety constraints. SAFEOPT [7], one of the first SBO algorithms, expands an initial safe set iteratively based on new evaluations and an

updated Gaussian Process (GP) model of safety functions. It calculates two subsets, maximizers and expanders, from the current safe set and selects the most uncertain decision within their union to balance maximizing the reward function and expanding the safe set. Subsequent algorithms extend SAFEOPT to handle multiple constraints [8], decouple safe set expansion from optimization [9], and expand the safe set in a goal-oriented manner [10]. These methods also explore disconnected safe regions [11, 12] and enhance information-theoretic efficiency [13, 14]. They have been applied to controller tuning for a ball-screw drive [15] and quadrupeds [16], and adaptive control on a rotational motion system [17]. However, SBO typically does not take into account changes with time.

Contextual Bayesian Optimization Contextual Bayesian Optimization (CBO) has been introduced to address the influence of external environmental factors on reward and safety functions. Krause and Ong [18] extend the Gaussian Process Upper Confidence Bound (GP-UCB) algorithm [19] by incorporating contextual variables into unconstrained BO, demonstrating sub-linear regret analogous to GP-UCB. An advancement of this framework is proposed in [20], with the Safe Contextual GP-UCB optimizing the contextual upper confidence bound within a safe set to manage room temperature via a PID controller. Berkenkamp et al. [8] present a contextual adaptation of SAFEOPT, discussing its safety and optimality guarantees by framing contextual SBO as distinct SBO sub-problems. Additionally, König et al. [21] extends GOOSE [10] to the contextual domain for model-free adaptive control scenarios. Similarly to SBO, CBO does not explicitly consider time-varying problems.

Time-Varying Bayesian Optimization Time-Varying Bayesian Optimization (TVBO) addresses problems where the objective is time-dependent, modeled with a temporal kernel [23]. Methods in this setting include periodical resetting [23], change detection [24, 25], sliding-window approaches using recent data [26], and discounting via exponentially decaying past observations [27]. However, these techniques have been developed for unconstrained BO and are unsuitable for safety-critical applications.

Time-Varying Safe Bayesian Optimization In the safety-critical time-varying setting, contextual lower confidence bounds can be optimized within the safe set [20], but it does not guarantee optimality theoretically. An event triggering mechanism is introduced to SBO to restart exploration from a backup policy [22], but it may not trigger reliably during changes, posing a safety risk. Extensions to SBO with contextual variables provide theoretical safety and optimality analyses [8, 16], treating contextual SBO as separate sub-problems for each contextual value, and assuming an initial safe set for each. However, ensuring optimality requires each contextual value to appear frequently, which is impractical in time-varying scenarios.

1.2 Methodology and Contributions

Methodology We propose the TVSAFEOPT algorithm to optimize an unknown time-varying reward subject to unknown time-varying safety constraints. The algorithm focuses on Time-Varying Safe Bayesian Optimization (TVSBO). TVSAFEOPT utilizes a spatio-temporal kernel and time Lipschitz constants as prior knowledge about how the problem depends on time. The temporal part of the kernel encodes the continuity of the functions with time while the Lipschitz constants explicitly provide upper bounds on how fast the functions may change. Instead of considering safe sets at previous iteration as safe at the current iteration, which might lead to unsafe decisions, TVSAFEOPT robustly subtracts the safety margin when updating the safe sets (Figure 1). In this way, the algorithm is capable of adapting in real time and guarantees safety even when exploring the safe region of non-stationary problems.

Contributions Our contributions are threefold: a) We propose the TVSAFEOPT algorithm based on Gaussian processes with spatio-temporal kernels; b) We provide formal safety guarantees for TVSAFEOPT in the most general time-varying setting and optimality guarantees for TVSAFEOPT for

locally stationary optimization problems; c) We show TVSAFEOPT performs well in the most general time-varying setting both on synthetic data and on a realistic case study on gas compressors. In Table 1, we compare Adaptive GOOSE, Contextual SAFEOPT, ETSAFEOPT, and TVSAFEOPT in terms of how they handle changes in time, safety guarantees, optimality guarantees, and required safe seeds.

1.2.1 Expected societal impact

The TVSAFEOPT algorithm proposed in this paper extends the state of the art in Time-Varying Safe Bayesian Optimization by enabling solving optimization problems with time-varying reward and constraints without pre-defining the time changes that can be compensated.Thus, the algorithm can be used at the design stage of operating strategies for safety-critical systems, such as medical dosage design [28] and controller design in robotics [17], or during online operation of chemical plants [29] or autonomous racing [30].

2 TVSAFEOPT Algorithm

The TVSAFEOPT algorithm builds upon SAFEOPT [7], to handle time-varying reward function and safety functions. The key new feature of TVSAFEOPT is its capability of safely transferring the current safe set to the next time step. TVSAFEOPT achieves this with the help of the spatio-temporal kernel as well as the sequence of time Lipschitz constants. The approach is summarized in Algorithm 1.

2.1 Assumptions

Following [8], we incorporate the reward and safety functions into an auxiliary function h : X ×T × I → R , where I := { 0 } ∪ I c ,

We model the auxiliary function using a prior Gaussian Process (GP) with zero mean and spatiotemporal kernel κ : ( X ×T ×I ) × ( X ×T ×I ) → R , [31]. We require h to be Lipschitz continuous with respect to both x and t , and to have bounded norm in the Reproducing Kernel Hilbert Space (RKHS) [32] associated with the kernel κ as formalized in the following.

Assumption 2.1. The spatio-temporal kernel is positive definite, and satisfies κ (( x , t, i ) , ( x , t, i )) ≤ 1 , for all x ∈ X , t ∈ T , i ∈ I . The function h ( x , t, i ) has bounded norm in the RKHS associated with kernel κ . The function h ( x , t, i ) is L x -Lipschitz continuous with respect to x in the domain X with respect to some metric d : X × X → R ≥ 0 for all t ∈ N , i ∈ I . There exists a sequence { L ( t ) } t ∈ N ,t<T , such that, for all x ∈ X , i ∈ I , t ∈ N , t < T , | h ( x , t +1 , i ) -h ( x , t, i ) | ≤ L ( t ) .

At each algorithm iteration k , we make a decision x k , which we then apply to the system and get noisy measurements y i k of the reward function and safety functions during the iteration. We use the index k to refer to the algorithm iteration. Even though k and t might differ in principle, in practice we run one algorithm iteration k for each time step t .

Assumption 2.2. Observations y i k = h ( x k , t, i ) + ε i k , ∀ i ∈ I , t ∈ N are perturbed by i.i.d. zero mean and σ -sub-Gaussian noise.

Based on the measurements, we compute the posterior GP and make the decision for the next time step. To start the exploration, an initial set of safe decisions is assumed to be available to the algorithm. To ensure that the safe set remains non-empty after the first iteration, it is necessary that the initial safety function values at every decision within the initial safe set are positive.

Assumption 2.3. An initial set S 0 ⊆ X of safe decisions is known and for all decisions x ∈ S 0 , we have c i ( x , 0) > 0 , ∀ i ∈ I c .

Similar assumptions have also been made for the standard SAFEOPT algorithm [7] and are necessary to ensure feasibility of the exploration steps and to be able to identify new safe decision.

2.2 Safety Updates

To ensure safety, based on Assumption 2.1 and 2.2, we extend the definition of the confidence intervals from [7] so that, with high probability, they contain f and c i using the posterior GP estimate given the data sampled so far. The confidence intervals for h ( x , t, i ) given training samples until iteration k ≥ 1 are defined for all x ∈ X and for all i ∈ I as

where β k is a scalar that determines the desired confidence interval, µ k -1 ( x , i ) and σ k -1 ( x , i ) are the posterior mean and standard deviation of h ( x , t, i ) inferred with D k , training samples till iteration k [31]. The probability of the true function value h lying within this interval depends on the choice of β k [8]. We provide more details for this choice in Section 2.4.

We now construct a tighter confidence interval for h ( x , t, i ) by using the sequence { Q τ ( x , i ) } τ ≤ k instead of Q k ( x , i ) alone. To this end, we recursively define for all x ∈ X and for all i ∈ I the intersection

where ⊕ denotes the Minkowski sum, C 0 ( x , i ) is [ L (0) , ∞ ) for all x ∈ S 0 , i ∈ I c and R otherwise. We use the lower bound l k ( x , i ) := min C k ( x , i ) and the upper bound u k ( x , i ) := max C k ( x , i ) , to define the width of C k ( x , i )

further used to update the safe set as well as pick the next decision to explore.

Based on the updated posterior and Lipschitz constants, we can update the safe set S k with the lower bounds l k and the previous safe set S k -1 as

The set S k contains decisions that with high probability fulfill the safety constraints given the GP confidence intervals and the Lipschitz constants. In contrast to SAFEOPT, the safe set of TVSAFEOPT is allowed to shrink to adapt to the potential change of the safe region given the time-varying setting. However, the safe set might even become empty after the update. This is either because the safe region indeed becomes empty or because the updated safe set conservatively excludes all decisions with a lower bound of some safety function below L to guarantee safety. In all these cases, if the updated safe set is empty, we terminate the algorithm.

2.3 Safe Exploration and Exploitation

With the safe set updated, the next challenge is to trade off between exploitation and expansion of the safe region. As in the standard SAFEOPT, the potential maximizers are those decisions, for which the upper confidence bound of the reward function is higher than the largest lower confidence bound

To identify the potential expanders, G k , containing all decisions that could potentially expand the safe set, we first quantify the potential enlargement of the current safe set after sampling a new decision x . To do so, we define the function

where | · | refers to the cardinality of a set, and then update

At iteration k , TVSAFEOPT selects a decision x k within the union of potential maximizers (7) and expanders (9)

Algorithm 1 TVSAFEOPT

with w k from (5). The objective of the greedy selection process in (10) is to take the most uncertain decision among the expanders G k and the maximizers M k . The decision x k is then applied to the system and after making observations of the reward and safety functions, y k := ( y 0 k , y 1 k , . . . , y m k ) , we add ( x k , y k ) to the training samples.

At any iteration, we can obtain an estimate for the current best decisions from

which returns the maximizer of the lower bound of the reward function within the current safe set.

2.4 Safety Guarantee

To provide safety guarantees, we need the confidence intervals in (3) to contain the safety functions with high probability for all iterations. Note that the parameter β k in (3) tunes the tightness of the confidence interval. The following lemma guides us to make a proper choice for β k : This choice depends on the information capacity γ h k associated with the kernel κ , namely is the maximal mutual information [33] we can obtain from the GP model of h through k noisy measurements ˆ h X k at data points X k := { ( x τ ∈ X , τ, i τ ∈ I ) } τ<k

Lemma 2.4. Assume that h ( x , t, i ) has RKHS norm associated with κ bounded by B > 0 and that measurements are perturbed by σ -sub-Gaussian noise. Let the variable γ h k be defined as in (12) .

For any δ ∈ (0 , 1) , let √ β k = B + σ √ 2 ( γ h k ·|I| +1+ln(1 /δ ) ) , then the following holds for all decisions x ∈ X , function indices i ∈ I , and iterations k ≥ 1 jointly with probability at least 1 -δ :

Proof. This lemma is a straightforward consequence of Lemma 1 of [16], a contextual extension of Lemma 4.1 of [8]. We can prove it by selecting time as the context and picking { t } t ≥ 1 ,t ∈ N as the context sequence.

Lemma 2.4 indicates that, by selecting β k properly, the confidence intervals Q k will w.h.p. contain the reward function and the safety functions. Due to this, they can be leveraged to provide theoretical guarantees for safety and optimality.

The following theorem provides a sufficient condition for safety of TVSAFEOPT.

Theorem 2.5. Let Assumptions 2.1 - 2.3 hold, and let γ be defined as in (12) . For any δ ∈ (0 , 1) ,

h k √ √ least 1 -δ , for all i ∈ I c and for all t ≥ 0 , and x ∈ S k it holds c i ( x , t ) ≥ 0 .

The proof builds on Lemma 2.4 to show first that for all t ≥ 0 , for all i ∈ I and for all x ∈ X , then h ( x , t, i ) ∈ C k ( x , i ) with high probability. Then using the recursive definition of the safe set from (6), we obtain w.h.p. c i ( x , t ) ≥ l k ( x ′ , i ) -L x d ( x , x ′ ) -L ( t ) ≥ 0 , which concludes the proof. For details we refer the reader to Appendix B.

2.5 Near-Optimality Guarantee

In many safety critical real world applications, such as nuclear power plant operations, medical devices calibration, automated emergency response systems, the reward function is stationary most of the time. The problems are stationary until some changes happen and become stationary again when the systems reach new equilibria [34]. However, ensuring optimality is non-trivial even when the problem becomes stationary. Suppose the auxiliary function (2) becomes stationary in a time interval [ ϕ, ϕ ] , namely suppose there exist ϕ > ϕ ≥ 1 such that ∀ t 1 , t 2 ∈ [ ϕ, ϕ ] , f ( x , t 1 ) = f ( x , t 2 ) =: ¯ f ( x ) and c ( x , t 1 ) = c ( x , t 2 ) =: ¯ c ( x ) , so that the optimization problem (1) becomes

We first define the largest safe set expanded from a set S subject to a measurement error a within

a single time step:

arbitrary time steps: ¯ R a ( S ) := lim n →∞ R n a ( S )

Wealso define ¯ L t as an upper bound of the sum of all time Lipschitz constants, that is, T -1 ∑ τ =0 L ( τ ) ≤ ¯ L t .

We find it reasonable that a tight upper bound ¯ L t can be provided when the underlying system slowly switches to the new stationarity condition.

Given these definitions, we are now in the position to provide optimality guarantees for TVSAFEOPT. In particular, we aim at comparing the found reward value ¯ f ( x k ) with the optimal reward value within the largest safe set obtained in ideal conditions with no measurement error, ¯ R 0 ( S 0 ) . We also aim at providing TVSAFEOPT with an upper bound on the iterations needed to find a near-optimal solution. The following theorem states the optimality guarantee of TVSAFEOPT.

Theorem 2.6. Let Assumptions 2.1 - 2.3 hold, let γ h be defined as in (12) and, for any δ ∈ (0 , 1) , let k √ β k = B + σ √ 2 ( γ h k ·|I| +1+ln(1 /δ ) ) . Define ˆ x k as in (11), and, for any ϵ > 0 , let k ∗ ( ϵ, δ ) be

the smallest positive integer satisfying

where b 1 = 8 / log ( 1 + σ -2 ) . Then, the TVSAFEOPT algorithm, applied to (13) , guarantees that, with probability at least 1 -δ , there exists k ≤ k ∗ such that

0

20

40

60

80

days

t

0

1

2

3

4

5

6

7

8

1e4

(a)

card. of safe set

TVSafeOPT

SafeOPT

Approx. Opt.

0

20

40

60

80

days

t

0.0

0.1

0.2

0.3

0.4

0.5

(b)

num. of unsafe decisions / card. of safe set

TVSafeOPT

SafeOPT

Approx. Opt.

0

20

40

60

80

days

t

0.0

0.2

0.4

0.6

0.8

1.0

(c)

num. of safe decisions / card. of ground truth safe region

TVSafeOPT

SafeOPT

Approx. Opt.

The proof consists in showing a decaying upper bound of uncertainty w k ( x , i ) ≤ ϵ and exploiting local stationarity of (13) to provide bounds on the expansion of the safe set S k . Details can be found in Appendix C.

3 Experiments

3.1 Synthetic Example

We first illustrate TVSAFEOPT on a synthetic two-dimensional time-varying optimization problem

Figure 1 compares the safe sets computed by TVSAFEOPT, ETSAFEOPT, and SAFEOPT at t = 30 , t = 100 and t = 170 . All algorithms start from the same singleton initial safe set S 0 = { ( -0 . 5 , 0 . 0) } . Implementation details are described in Appendix A. Figure 1 illustrates that the safe sets computed by TVSAFEOPT are contained in the ground truth safe regions while those computed by SAFEOPT and ETSAFEOPT have multiple violations. Due to the dependence on time of the example (Figure 3), the initial safe set becomes unsafe at t = 30 , and t = 170 . Taking the possible changes in time into consideration, TVSAFEOPT correctly identifies the possible unsafety of the initial safe set. In contrast, SAFEOPT always consider the initial safe set to be safe. Meanwhile, ETSAFEOPT correctly identifies the lack of safety of the initial safe set at t = 30 , but fails at t = 170 . This is because the event trigger is naturally insensitive to continuous changes. This toy example indicates that, in contrast to SAFEOPT and ETSAFEOPT, TVSAFEOPT safely adapts to the time changes of the optimization problem.

In this example, TVSAFEOPT and ETSAFEOPT overall find better reward function values than SAFEOPT, see Figure 3. The reward function value found by TVSAFEOPT is close to the optimal values when the reward function changes slowly, which supports Theorem 2.6.

Quantitative metrics are listed in Table 2. Taking SAFEOPT as a baseline, TVSAFEOPT and ETSAFEOPT achieves less violations, lower cumulative regret at the cost of covering less part of the safe region. Furthermore, TVSAFEOPT has little violations, and achieves larger coverage ratio than ETSAFEOPT. Meanwhile, its cumulative regret is just slightly higher than that of ETSAFEOPT.

3.2 Gas Compressor Case Study

3.2.1 Problem Setup

We show the performance of the proposed algorithm in a compressor station with three identical compressors operating in parallel at the time-varying compressor head H t with time-varying power consumption at time t (adapted from [35], details in Appendix A.3)

m

m

i

i

β

γ

1

1

¯

H

¯

¯

H

2

t

2

t

+

+

β

γ

2

2

¯

H

¯

¯

H

t

+

β

3

,

∀

i

= 1

, . . . , N

t

+

γ

3

,

∀

i

= 1

, . . . , N

where the objective (14) corresponds to the power to run the station with N compressors, here N = 3 , affected by individual degradation d it , i = 1 , l . . . , N . The station must also satisfy time-varying demand M t in (15). In practice, it is common to linearly approximate (16)-(17) with respect to the compressor head H t (dashed lines in Figure 4) [36].

3.2.2 Results

We compare the performance of TVSAFEOPT, SAFEOPT, and approximate optimization. ETSAFEOPT is not applicable to this case study because the magnitude of changes in the demand M t , compressor head H t , and degradation d it keeps triggering the event trigger and thus the maintained safe set becomes empty very quickly. Implementation details are described in Appendix A. Figure 2 compares the number of unsafe decisions in the safe sets calculated by TVSAFEOPT, SAFEOPT, and approximate optimization. We see that, by considering the uncertainty with respect to the decision variables, SAFEOPT maintains fewer unsafe decisions in its safe sets than the approximate optimization. However, SAFEOPT tends to expand its safe sets regardless of external changes. TVSAFEOPT further improves this based on SAFEOPT by taking into consideration the time-varying safety functions. TVSAFEOPT robustly shrinks its safe sets based on its observations and thus maintains much

(16)

less violations in its safe sets than SAFEOPT (70.4%) and approximate optimization (96.8%). It achieves this at the cost of covering less of the ground truth safe region than SAFEOPT (39.3%) and Approximate Optimization (61.0%).

The right-hand side of Figure 3 shows that TVSAFEOPT preserves safety at the expense of optimality. In the compressor case study, TVSAFEOPT overall finds lower reward function values than SAFEOPT and approximate optimization, which is consistent with the fact that it covers a lower fraction of the ground truth safe regions and the reward function changes significantly between iterations. Because of its strong focus on safety, TVSAFEOPT deviates more from the ground truth. The cumulative regret of TVSAFEOPT is above the one of SAFEOPT by 42.1%, and the one of approximate optimization by 178.3%. This illustrates the trade-off between safety and optimality in the presence of strong uncertainties due to the varying reward and safety constraints. Quantitative metrics using Approximate Optimization as the baseline are listed in Table 3.

4 Limitations and Conclusion

Limitations The compressor case study demonstrated that TVSAFEOPT ensures safety at the expense of optimality if the stationarity assumption is not satisfied. The assumption about the local stationarity of the optimization problem (1) is thus the main limitation. Even though TVSAFEOPT demonstrates good empirical performance with respect to safety even when the problem is non-stationary, theoretical guarantee for its near-optimality in the non-stationary case warrant further investigation.

The need for obtaining the Lipschitz constants with respect to both x and t in order to compute the safe set S k in (6) may prove limiting in real applications. To overcome this limitation, we propose practical modifications in Appendix A.1.

Conclusions We propose TVSAFEOPT algorithm, which extends SAFEOPT to handle time-varying optimization problems. In conclusion, TVSAFEOPT outperforms SAFEOPT in terms of adaptation to changes in time and maintains fewer unsafe decisions in its safe sets for time-varying problems. This is at the cost of covering less of the ground truth safe regions and may lead to poorer performance in terms of optimality.

We prove the safety guarantee for TVSAFEOPT in the general time-varying setting and prove its near-optimality guarantee for the case in which the optimization problem becomes stationary. The two theoretical results together guarantee that TVSAFEOPT is capable of safely transferring safety of the decisions into the future and, based on the transferred safe sets, it will find the near-optimal decision when the reward function stops changing. We show that TVSAFEOPT performs well in practice for the most general settings where both the reward function and the safety constraint are time-varying, both on synthetic data and for real case study on a gas compressor.

Acknowledgments and Disclosure of Funding

Research supported by NCCR Automation, National Centre of Competence in Research, funded by the Swiss National Science Foundation (grant no. 180545). Marta Zagorowska also acknowledges funding from the Marie Curie Horizon Postdoctoral Fellowship project RELIC (grant no 101063948) for writing, revisions, and the compressor case study. Alisa Rupenyan acknowledges also support from the Johann Jakob Rieter foundation.

References

Daniel Lizotte, Tang Wao, Michael Bowling, and Dale Schuurmans. Automatic gait optimization with Gaussian process regression. International Joint Conference on Artificial Intelligence , pages 944-949, 2007.

Ruben Martinez-Cantin, Nando de Freitas, Arnaud Douchet, and José A. Castellanos. Active policy learning for robot planning and exploration under uncertainty. In Proceedings Robotics: Science and Systems , pages 321-328, 2007.

Ziyu Wang, Masrour Zoghi, Frank Hutter, David Matheson, and Nando de Freitas. Bayesian optimization in high dimension via random embeddings. International Joint Conference on Artificial Intelligence , pages 1778-1784, 2013.

Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. International Joint Conference on Machine Learning , pages 1015-1022, 2010.

Matthew Hoffman, Bobak Shahriari, and Nando de Freitas. On correlation and budget constraints in model-based bandit optimization with application to automatic machine learning. International Conference on Artificial Intelligence and Statistics , pages 365-374, 2014.

Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. In Proceedings of Advances in Neural Information Processing Systems , pages 2951-2959, 2012.

Yanan Sui, Alkis Gotovos, Joel Burdick, and Andreas Krause. Safe exploration for optimization with Gaussian processes. In International conference on machine learning , pages 997-1005. PMLR, 2015.

Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics. Machine Learning , pages 1-35, 2021.

Yanan Sui, Vincent Zhuang, Joel Burdick, and Yisong Yue. Stagewise safe Bayesian optimization with Gaussian processes. In International conference on machine learning , pages 4781-4789. PMLR, 2018.

Matteo Turchetta, Felix Berkenkamp, and Andreas Krause. Safe exploration for interactive machine learning. Advances in Neural Information Processing Systems , 32, 2019.

Dominik Baumann, Alonso Marco, Matteo Turchetta, and Sebastian Trimpe. GoSafe: Globally optimal safe robot learning. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 4452-4458. IEEE, 2021.

Bhavya Sukhija, Matteo Turchetta, David Lindner, Andreas Krause, Sebastian Trimpe, and Dominik Baumann. GoSafeOpt: Scalable safe exploration for global optimization of dynamical systems. Artificial Intelligence , page 103922, 2023.

Alessandro G. Bottero, Carlos E. Luis, Julia Vinogradska, Felix Berkenkamp, and Jan Peters. Information-theoretic safe Bayesian optimization, 2024.

Jonas Hübotter, Bhavya Sukhija, Lenart Treven, Yarden As, and Andreas Krause. Informationbased transductive active learning. arXiv preprint arXiv:2402.15898 , 2024.

Marta Zagorowska, Efe C. Balta, Varsha Behrunani, Alisa Rupenyan, and John Lygeros. Efficient sample selection for safe learning. IFAC-PapersOnLine , 56(2):10107-10112, 2023. 22nd IFAC World Congress.

Daniel Widmer, Dongho Kang, Bhavya Sukhija, Jonas Hübotter, Andreas Krause, and Stelian Coros. Tuning legged locomotion controllers via safe Bayesian optimization. In 7th Annual Conference on Robot Learning (CoRL), 6-9 November, Atlanta, GA , 2023. URL https: //proceedings.mlr.press/v229/widmer23a.html .

Christopher König, Miks Ozols, Anastasia Makarova, Efe C. Balta, Andreas Krause, and Alisa Rupenyan. Safe risk-averse Bayesian optimization for controller tuning. IEEE Robotics and Automation Letters , 8(12):8208-8215, 2023.

Andreas Krause and Cheng Ong. Contextual Gaussian process bandit optimization. Advances in neural information processing systems , 24, 2011.

Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: no regret and experimental design. In Proceedings of the 27th International Conference on International Conference on Machine Learning , ICML'10, page 1015-1022, Madison, WI, USA, 2010. Omnipress.

Marcello Fiducioso, Sebastian Curi, Benedikt Schumacher, Markus Gwerder, and Andreas Krause. Safe contextual bayesian optimization for sustainable room temperature PID control tuning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 , pages 5850-5856. International Joint Conferences on Artificial Intelligence Organization, 7 2019.

Christopher König, Matteo Turchetta, John Lygeros, Alisa Rupenyan, and Andreas Krause. Safe and efficient model-free adaptive control via Bayesian optimization. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 9782-9788. IEEE, 2021.

Antonia Holzapfel, Paul Brunzema, and Sebastian Trimpe. Event-triggered safe Bayesian optimization on quadcopters. In Alessandro Abate, Mark Cannon, Kostas Margellos, and Antonis Papachristodoulou, editors, Proceedings of the 6th Annual Learning for Dynamics & Control Conference (L4DC), 15-17 July, Oxford, UK , volume 242 of Proceedings of Machine Learning Research , pages 1033-1045. PMLR, 15-17 Jul 2024.

Ilija Bogunovic, Jonathan Scarlett, and Volkan Cevher. Time-varying Gaussian process bandit optimization. In Artificial Intelligence and Statistics , pages 314-323. PMLR, 2016.

Paul Brunzema, Alexander von Rohr, Friedrich Solowjow, and Sebastian Trimpe. Eventtriggered time-varying Bayesian optimization. arXiv preprint arXiv:2208.10790 , 2022.

Kihyuk Hong, Yuhang Li, and Ambuj Tewari. An optimization-based algorithm for nonstationary kernel bandits without prior knowledge. In International Conference on Artificial Intelligence and Statistics , pages 3048-3085. PMLR, 2023.

Xingyu Zhou and Ness Shroff. No-regret algorithms for time-varying Bayesian optimization. In 2021 55th Annual Conference on Information Sciences and Systems (CISS) , pages 1-6. IEEE, 2021.

Yuntian Deng, Xingyu Zhou, Baekjin Kim, Ambuj Tewari, Abhishek Gupta, and Ness Shroff. Weighted Gaussian process bandits for non-stationary environments. In International Conference on Artificial Intelligence and Statistics , pages 6909-6932. PMLR, 2022.

Dinesh Krishnamoorthy and Francis J Doyle. Safe Bayesian optimization using interior-point methods-applied to personalized insulin dose guidance. IEEE Control Systems Letters , 6: 2834-2839, 2022.

Dinesh Krishnamoorthy and Francis J Doyle III. Model-free real-time optimization of process systems using safe Bayesian optimization. AIChE Journal , 69(4):e17993, 2023.

Lukas Hewing, Kim P Wabersich, Marcel Menner, and Melanie N Zeilinger. Learning-based model predictive control: Toward safe learning in control. Annual Review of Control, Robotics, and Autonomous Systems , 3:269-296, 2020.

Carl Edward Rasmussen and Chris Williams. Gaussian Processes for Machine Learning . Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA, USA, January 2006.

Bernhard Schölkopf and Alexander J Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond . The MIT Press, 2002.

Thomas M Cover. Elements of information theory . John Wiley & Sons, 1999.

Michael Vogt and Holger Dette. Detecting gradual changes in locally stationary processes. The Annals of Statistics , 43(2):713-740, 2015.

Marta Zagorowska and Nina F. Thornhill. Influence of compressor degradation on optimal load-sharing. Computers and Chemical Engineering , 143(5):107104, 2020.

Andrea Cortinovis, Joachim Ferreau, Daniel Lewandowski, and Mehmet Mercangöz. Experimental evaluation of MPC-based anti-surge and process control for electric driven centrifugal gas compressors. Journal of Process Control , 34:13-25, 2015.

Felix Berkenkamp, Angela P Schoellig, and Andreas Krause. Safe controller optimization for quadrotors with Gaussian processes. In 2016 IEEE International Conference on Robotics and Automation (ICRA), 16-21 May, Stockholm, Sweden , pages 491-496. IEEE, 2016.

Rainer Kurz, Matt Lubomirsky, and Klaus Brun. Gas compressor station economic optimization. International Journal of Rotating Machinery , 2012:Article ID 715017, 9 pages, 2012.

Predrag Milosavljevic, Alejandro G. Marchetti, Andrea Cortinovis, Timm Faulwasser, Mehmet Mercangöz, and Dominique Bonvin. Real-time optimization of load sharing for gas compressors in the presence of uncertainty. Applied Energy , 272:114883, 2020.

Marta Zagorowska. Degradation modelling in process control applications . PhD thesis, Imperial College London, 2020. available at https://spiral.imperial.ac.uk/handle/10044/1/ 105173 , online 22 May 2024.

Do Won Kang and Tong Seop Kim. Model-based performance diagnostics of heavy-duty gas turbines using compressor map adaptation. Applied Energy , 212:1345-1359, 2018.

Andrea Cortinovis, Mehmet Mercangöz, Matteo Zovadelli, Diego Pareschi, Antonio De Marco, and Sergio Bittanti. Online performance tracking and load sharing optimization for parallel operation of gas compressors. Computers & Chemical Engineering , 88:145-156, 5 2016.

Ayman Al Zawaideh, Khalifa Al Hosani, Igor Boiko, and Mohammad Luai Hammadih. Minimum energy adaptive load sharing of parallel operated compressors. IEEE Open Journal of Industry Applications , 3:178-191, 2022.

Vibeke Stærkebye Nørstebø. Optimum Operation of Gas Export Systems . PhD thesis, Norwegian University of Science and Technology, 2008.

Rainer Kurz and Klaus Brun. Degradation of gas turbine performance in natural gas service. Journal of Natural Gas Science and Engineering , 1(3):95-102, 2009.

Yiuguang Li and Pannawat Nilkitsaranont. Gas turbine performance prognostic for conditionbased maintenance. Applied Energy , 86(10):2152-2161, October 2009. ISSN 0306-2619.

Matteo Cicciotti. Adaptive Monitoring of Health-state and Performance of Industrial Centrifugal Compressors . PhD thesis, Imperial College London, 2015.

Marta Zagorowska, Frederik Schulze Spüntrup, Arne-Marius Ditlefsen, Lars Imsland, Erling Lunde, and Nina F. Thornhill. Adaptive detection and prediction of performance degradation in off-shore turbomachinery. Applied Energy , 268:p. 114934, 2020.

Yu-Zhi Chen, Xu-Dong Zhao, Heng-Chao Xiang, and Elias Tsoutsanis. A sequential modelbased approach for gas turbine performance diagnostics. Energy , 220:119657, 2021-04. ISSN 0360-5442.

Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, and Colin Jones. Constrained efficient global optimization of expensive black-box functions. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 38485-38498. PMLR, 23-29 Jul 2023.

Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, and Colin N Jones. Primal-dual contextual bayesian optimization for control system online optimization with time-average constraints. In 2023 62nd IEEE Conference on Decision and Control (CDC) , pages 4112-4117. IEEE, 2023.

A Experiment Details

Experiments are conducted on an Intel i7-11370H CPU using Python 3.8.5. The implementation utilizes the following libraries: GPy 1.12.0, NumPy 1.22.0, and Matplotlib 3.5.0.

A.1 Practical Modifications

In practice, Lipschitz constants are difficult to estimate. Thus, here we provide a Lipschitz-constantfree version of TVSAFEOPT algorithm by modifying (6) and (8).

The safe set is updated as all decisions with non-negative lower confidence bounds for the safety functions at the current iteration k , that is,

Furthermore, the expanders are intuitively defined as decisions within the current safe set such that, by evaluating any of the decisions, at least one decision outside the current safe set will be considered as safe, that is, G k = { x ∈ S k | e k ( x ) > 0 } , where e k ( x ) denotes the number of decisions outside S k that will be considered safe when evaluating x . Instead of using Lipschitz constants, we define e k ( x ) using lower bound of auxiliary GP similar to the method by Berkenkamp et al. [37]

where l k, ( x ,u k ( x ,i )) ( x ′ , k +1 , i ) denotes the lower bound of the function values at x and t = k +1 if x is evaluated at the k -th iteration and the upper bound is observed.

A.2 Synthetic Example

The search space is X = [ -2 , 2] 2 , uniformly quantized into 100 × 100 points. Both algorithms start with the singleton initial safe set { ( -0 . 5 , 0 . 0) } . The measurements are perturbed by i.i.d. Gaussian noise N (0 , 0 . 01 2 ) .

The reward function is formulated as: f ( x , t ) = -e x 2 -log(1 + y 2 ) + 0 . 01 t ;

The safety function is formulated as: c 1 ( x , t ) = 1 -[ x +0 . 5 -0 . 5 ( 1 -cos 2 π 50 t ) cos π 6 ] 2 -[ y -0 . 3 -0 . 5 ( 1 -cos 2 π 50 t ) sin π 6 ] 2 .

The hyperparameters of GPs in the synthetic case study are modelled as follows,

TVSAFEOPT: The reward function and the safety function are modeled by independent GPs with zero mean and spatio-temporal kernel κ (( x , t ) , ( x ′ , t ′ )) = exp ( -∥ x -x ′ ∥ 2 2 2 σ 2 1 ) · exp ( -( t -t ′ ) 2 2 σ 2 2 ) , where σ 1 ≡ 1 . 0 , σ 2 = 25 . 0 for f , and σ 2 = 15 . 0 for c 1 .

SAFEOPT: The reward function and the safety function are modeled by independent GPs with zero mean and 2d Gaussian kernel κ ( x , x ′ ) = exp ( -∥ x -x ′ ∥ 2 2 2 σ 2 3 ) , where σ 3 ≡ 1 . 0 .

ETSAFEOPT: Hyperparameters for GPs are the same as SAFEOPT. Besides, we choose the sentivity of the event trigger δ as 0.01.

A.3 Compressor Case Study

Centrifugal compressors are often used in gas transport networks to deliver the required amount of gas by boosting the pressure in the pipelines. Organised as compressor stations with N units, the compressors are often operated to minimise their power consumption P while satisfying the demand M t and operating constraints, capturing how compressor head H t depends on the mass flow through the compressor [38, 39]:

˜ m i = m i -157 . 4 34 . 37 , ˜ H t = H t -1 . 016 e 5 3 . 210 e 4 , α 1 = 1 . 979 e 7 , α 2 = 5 . 274 e 6 , α 3 = 5 . 375 e 6 , α 4 = 6 . 055 e 5 , α 5 = 5 . 718 e 5 , α 6 = 3 . 319 e 5

¯ H t = H t -1 . 235 e 5 3 . 764 e 4 , β 1 = -1 . 953 , β 2 = 16 . 86 , β 3 = 118 . 1

¯ ¯ H t = H t -6 . 152 e 4 7002 , γ 1 = -1 . 516 , γ 2 = -11 . 12 , γ 3 = 116 . 9

˜ ˜ H t = H t -8 . 706 e 4 5 . 289 e 4 , δ 1 = 73 . 21 , δ 2 = 183 . 7

˜ ¯ H t = H t -1 . 572 e 5 2 . 044 e 4 , σ 1 = -7 . 260 , σ 2 = -29 . 65 , σ 3 = 204 . 4

The compressor case study has been adapted from [35]. The data for the demand, compressor head, and degradation for the three compressors were obtained from [40] (Creative Commons Attribution NonCommercial Licence).

Individual characteristics of compressors in (16)-(17) are called compressor maps (Figure 4). The operating area for a compressor is defined by minimal and maximal speed of the compressor and its mechanical properties. The operating area can be obtained from compressors maps delivered by the manufacturer of the compressor, or estimated during the operation [41]. However, estimation would require collecting datapoints close to the boundary of the operating area, which may be unavailable due to safety consideration [42, 43]. Using safe learning has the potential to improve the operation of the station because it enables safe exploration of the unknown operating area of individual compressors.

Max. speed app.

Choke app.

Min. speed app.

Surge app.

Surge

Minimum speed

Choke

Maximum speed

Furthermore, varying operating conditions and demand often lead to compressor degradation d it (Figure 5), over time increasing power consumption (14) of the entire compressor station [45]. Capturing the time-varying aspect of compressor degradation is a subject of research (e.g. [46-48]) but limited availability of measured degradation data presents a challenge [49].

For convenience, the optimization variables are scaled by a factor K = 200 , that is, x = ( m 1 , m 2 , m 3 ) /K . The search space is X = [50 . 0 /K, 250 . 0 /K ] 3 , uniformly quantized into 60 × 60 × 60 points. Both algorithms start with the singleton initial safe set { ( M 0 , M 0 .M 0 ) / 3 K } . The measurements are perturbed by i.i.d. Gaussian noise N (0 , 0 . 01 2 ) .

The reward function is formulated as:

The safety functions are formulated as c i ( x, t ) ≥ 0 , i = 1 , . . . , 7 , with:

c 1 ( x , t ) = x 1 -L t

0

20

40

60

80

days

t

0

1

2

3

4

5

6

7

8

kg ⋅ s -1

1e2

(a)

Demand

0

20

40

60

80

days

t

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

1e5

(a)

0

20

40

60

80

days

t

0.00

0.01

0.02

0.03

0.04

0.05

0.06

(c)

Degradation

Compressor 1

Compressor 3

c 2 ( x , t ) = U t -x 1

c 3 ( x , t ) = x 2 -L t

c 4 ( x , t ) = U t -x 2

c 5 ( x , t ) = x 3 -L t

c 6 ( x , t ) = U t -x 3

c 7 ( x , t ) = x 1 + x 2 + x 3 -0 . 67 M t /K ,

where L t = max { β 1 ¯ H 2 t + β 2 ¯ H t + β 3 , γ 1 ¯ ¯ H 2 t + γ 2 ¯ ¯ H t + γ 3 } /K , U t = min { δ 1 ˜ ˜ H t + δ 2 , σ 1 ˜ ¯ H 2 t + σ 2 ˜ ¯ H t + σ 3 } /K .

The hyperparameters of GPs in the compressor case study are modelled as follows,

TVSAFEOPT: The reward function and the safety functions are modeled by independent GPs with zero mean and spatio-temporal kernel κ (( x , t ) , ( x ′ , t ′ )) = exp ( -∥ x -x ′ ∥ 2 2 2 σ 2 1 ) · exp ( -( t -t ′ ) 2 2 σ 2 2 ) , where σ 1 ≡ 1 . 0 , σ 2 = 80 . 0 for f and c 1 -c 6 , and σ 2 = 70 . 0 for c 7 .

SAFEOPT: The reward function and the safety functions are modeled by independent GPs with zero mean and 3d Gaussian kernel κ ( x , x ′ ) = exp ( -∥ x -x ′ ∥ 2 2 2 σ 2 3 ) , where σ 3 ≡ 1 . 0 .

As for approximate optimization, the r.h.s. of (16) - (17) are linearly approximated as follows:

Surge line: β 1 ¯ H 2 t + β 2 ¯ H t + β 3 ≈ 4 . 481 e -4 · H t +59 . 76

Min. speed line: γ 1 ¯ ¯ H 2 t + γ 2 ¯ ¯ H t + γ 3 ≈ -1 . 333 e -3 · H t +193 . 3

Choke line: δ 1 ˜ ˜ H t + δ 2 ≈ 1 . 611 e -3 · H t +46 . 77

Max. speed line: σ 1 ˜ ¯ H 2 t + σ 2 ˜ ¯ H t + σ 3 ≈ -1 . 667 e -3 · H t +461 . 7

B Proof of Safety Guarantee

Note all following lemmas hold for any δ ∈ (0 , 1) , and S 0 , such that ∅ ⊊ S 0 ⊆ X .

First, we want to show that the intersected confidence interval C k in (4) w.h.p. contains the reward function and safety functions h ( x , t, i ) as in (2).

Lemma B.1. Let √ β k = B + σ √ 2 ( γ h k ·|I| +1+ln(1 /δ ) ) , with γ h k defined as in (12) and C k ( x , i ) defined as in (4), then the following holds with probability at least 1 -δ :

Proof by induction.

If t = 0 , by Assumption 2.3 and the definition of C k in (4), then h ( x , 0 , i ) ∈ C 0 ( x , i ) , for all i ∈ I and for all x ∈ X .

Suppose, for any t = τ ≥ 0 , that h ( x , τ, i ) ∈ C τ ( x , i ) , then for t = τ + 1 , from the Lipschitz continuity of h , | h ( x , τ + 1 , i ) -h ( x , τ, i ) | ≤ L ( τ ) , it holds that h ( x , τ + 1 , i ) ∈ C τ ( x , i ) ⊕ [ -L ( τ ) , L ( τ )] .

Moreover, by Lemma 2.4 and (3) we have that h ( x , τ +1 , i ) ∈ Q τ +1 ( x , i ) .

Therefore, for all t ≥ 0 , for all i ∈ I and for all x ∈ X we have that h ( x , t, i ) ∈ C k ( x , i ) , and this concludes the proof.

We are now ready to prove Theorem 2.5 that provides a sufficient condition for TVSAFEOPT to ensure safety embedded in the constraints c i ( x , t ) ≥ 0 , for all i ∈ I c .

Proof of Theorem 2.5.

If t = 0 , by definition of S 0 , one has c i ( x , t ) = c i ( x , 0) ≥ L (0) ≥ 0 , ∀ i ∈ I c , ∀ x ∈ S 0 .

For any t ≥ 1 , ∀ x ∈ S t , by recursive definition of S k in (6), ∀ i ∈ I c , there exists x ′ ∈ S t -1 , s.t. l k ( x ′ , i ) -L x d ( x , x ′ ) -L ( t ) ≥ 0 . Then, ∀ i ∈ I c

and this concludes the proof.

C Proof of Near-Optimality Guarantee

The proof of near optimality consists in two parts: i) bounding the uncertainty and ii) bounding the expansion of the safe set.

C.1 Bounding the Uncertainty

We first derive a decaying upper bound of uncertainty for TVSAFEOPT. In this way we can ensure the uncertainty of the reward function and safety functions to drop below a desired threshold.

Lemma C.1. Define b 1 := 8 / log ( 1 + σ -2 ) ∈ R , and γ h k as in (12) . For any k > k 0 ≥ 1 , there exists k ′ ∈ ( k 0 , k ] , such that the following holds for all i ∈ I :

Proof.

Let i τ := arg max i ∈I w τ ( x τ , i ) , where x τ = argmax x ∈ G τ ∪ M τ max i ∈I w τ ( x , i ) . For all i ∈ I , k 0 < k , there exists k ′ ∈ ( k 0 , k ] :

(a): Definition of w k in (5),

(b): From the fact that the quadratic mean upper bounds the arithmetic mean,

(c): σ 2 τ -1 ( x τ , i τ ) ≤ k (( x τ , τ, i τ ) , ( x τ , τ, i τ )) ≤ 1 by Assumption 2.1 , and the fact that a ≤ b 1 8 log(1 + σ -2 a ) , ∀ a ∈ [0 , 1] ,

(d): σ ′ τ -1 ( x , i ) denotes the posterior standard deviation of h ( x , τ, i ) inferred by observations at X τ := { ( x j , j, i j ) } j<τ . Since { ( x j , j, i j ) } j = <τ ⊊ { ( x j , j ) } j<τ × I , then σ τ -1 ( x τ , i τ ) ≤ σ ′ τ -1 ( x τ , i τ ) ,

(e): From [19, Lemma 5.3],

(f): Definition of γ h k (12).

Corollary C.2. Given b 1 := 8 / log ( 1 + σ -2 ) ∈ R , take T k as the smallest positive integer satisfying T k β k + T k γ h k + T k ≥ b 1 ϵ 2 . Then, there exists k ′ ∈ ( k, k + T k ] , such that for any x ∈ G k ′ ∪ M k ′ , and for all i ∈ I it holds that

C.2 Bounding the Expansion of the Safe Set

All following lemmas hold for any δ ∈ (0 , 1) , ϵ > 0 and S 0 , such that ∅ ⊊ S 0 ⊆ X .

To facilitate the theoretical analysis, we define ∀ x ∈ X , ∀ i ∈ I :

Remember that, from (4), we can derive ∀ x ∈ X , ∀ i ∈ I :

Therefore, ˜ l k can be viewed as updating l k with L ( t ) ≡ 0 . With a slight abuse of notation, we omit arguments x and i when not ambiguous.

Lemma C.3. The following holds for any k ≥ 1 , ∀ x ∈ X , ∀ i ∈ I :

Proof.

(i) Direct consequence of (20).

(ii) Direct consequence of (19).

(iii) We proceed by induction. Suppose l τ ≤ ˜ l τ , then l τ -L ( τ ) ≤ ˜ l τ , thus according to (20), l τ +1 = max { l τ -L ( τ ) , µ τ -β 1 / 2 τ +1 σ τ } ≤ max { ˜ l τ , µ τ -β 1 / 2 τ +1 σ τ } = ˜ l τ +1 , from which it follows l k ( x , i ) ≤ ˜ l k ( x , i ) .

(iv) We proceed by induction. Suppose l τ ≥ ˜ l τ -τ -1 ∑ k =0 L ( k ) .

Lemma C.3 allows us to define auxiliary safe sets based on ˜ l k such that they are contained in S k . Furthermore, due to the monotonicity of ˜ l k , we can prove the auxiliary safe sets never shrink, which will play a fundamental role in studying their convergence property and provide near-optimality guarantee of TVSAFEOPT.

Based on (19), we further define:

Remember S k = { x ∈ X | ∀ i ∈ I c , ∃ x ′ i ∈ S t -1 , s.t. l k ( x ′ i , i ) -L x d ( x , x ′ i ) -L ( t ) ≥ 0 } . Thus, S k = S k = S k if and only if L ( t ) ≡ 0 .

The following lemma proves that S k never shrinks, and that S k and S k are a subset and a superset for S k , respectively.

Lemma C.4. The following holds for any t ≥ 1 :

Proof.

(i) We refer the reader to [8, Lemma 7.1].

(ii) We proceed by induction. Suppose S τ ⊆ S τ ⊆ S τ .

For all x ∈ S τ +1 , and for all i ∈ I c , there exists x ′ i ∈ S τ ⊆ S τ , s.t. ˜ l τ ( x ′ i , i ) -L x d ( x , x ′ i ) ≥ l τ ( x ′ i , i ) -L x d ( x , x ′ i ) -L ( τ ) ≥ 0 , hence x ∈ S τ +1 as well. Therefore, S τ ⊆ S τ ∀ τ .

For all x ∈ S τ +1 , and for all i ∈ I c , there exists x ′ i ∈ S τ ⊆ S τ , s.t. l τ ( x ′ i , i ) -L x d ( x , x ′ i ) -

˜ l τ ( x ′ i , i ) -L x d ( x , x ′ i ) -¯ L t ≥ 0 , thus x ∈ S τ +1 . Therefore, S τ ⊆ S τ , ∀ τ . From which we conclude S τ ⊆ S τ ⊆ S τ , ∀ τ .

Note: Where needed in the following lemmas, we assume b 1 and T k are defined as in Lemma C.1 and Corollary C.2

̸

Lemma C.5 (Lemma 7.4 in [8]) . For any k ≥ 1 , a > 0 , if ¯ R a ( S 0 ) \ S k = ∅ , then R a ( S k ) \ S k = ∅ .

̸

̸

The following lemma provides a sufficient condition for the expansion of the auxiliary safe set S k . Lemma C.6. For any t ≥ 1 , if ¯ R ¯ L t + ϵ ( S 0 ) \ S k = ∅ , then, with probability at least 1 -δ , it holds that S k + T k ⊋ S k .

Proof.

Similar to the proof of [8, Lemma 7.5].

By Lemma C.5, we get that, R ¯ L t + ϵ ( S k ) \ S k = ∅ . Equivalently, ∃ x ∈ R ¯ L t + ϵ ( S k ) \ S k which implies that, for all i ∈ I c ,

̸

Now assume, to the contrary, that S k + T k = S k . Thus, ∀ k ′ ∈ ( k, k + T k ] , x ∈ D\ S k ′ , and ∀ i ∈ I c , z i ∈ S k ′ .

Therefore, by definition (8), e k ′ ( z i ) > 0 , which implies z i ∈ G k ′ , ∀ k ′ ∈ ( k, k + T k ] , ∀ i ∈ I c .

Therefore, we know that there exists k ′ ∈ ( k, k + T k ] , for all i ∈ I c , w k ′ ( z i , i ) ≤ ϵ . (Corollary C.2) Hence, for all i ∈ I c ,

This means x ∈ S k ′ = S k , which leads to a contradiction.

The following lemma gives a superset for the auxiliary safe set S k .

Lemma C.7. S k ⊆ ¯ R ¯ L t ( S 0 ) with probability at least 1 -δ .

Proof by induction.

S 0 = S 0 ⊆ ¯ R ¯ L t ( S 0 )

Suppose S τ ⊆ ¯ R ¯ L t ( S 0 ) .

For all x ∈ S τ +1 and for all i ∈ I c there exists x ′ i ∈ S τ , s.t. ¯ c i ( x ′ i ) -L x d ( x , x ′ i ) -¯ L t ( a ) ≥ ˜ l k ( x ′ i , i ) -L x d ( x , x ′ i ) -¯ L t ≥ 0 .

(a): Lemma B.1.

Lemma C.8 (Lemma 7.8 in [8]) . Let k ∗ be the smallest integer, such that k ∗ ≥ ∣ ∣ ¯ R ¯ L t ( S 0 ) ∣ ∣ T k ∗ . Then, there exists k 0 ≤ k ∗ , such that S k 0 + T k 0 = S k 0 .

Lemma C.8 together with Lemma C.6, and Lemma C.7 entail convergence of S k within k ∗ time steps, which ultimately leads us to the near-optimality of TVSAFEOPT when the problem becomes stationary.

Lemma C.9. For any k ≥ 1 , if S k + T k = S k , then, with probability at least 1 -δ , there exists k ′ ∈ ( k, k + T k ] such that

Proof.

Let x ∗ k ′ := arg max x ∈ S k ′ ¯ f ( x ) . Note that x ∗ k ′ ∈ M k ′ , since

(c): Definition of ˆ x k (11).

We will first show that ∃ k ′ ∈ ( k, k + T k ] , s.t. ¯ f (ˆ x k ′ ) ≥ ¯ f ( x ∗ k ′ ) -ϵ . Assume, to the contrary, that

Then, we have, ∃ k ′ ∈ ( k, k + T k ]

which is a contradiction.

(d): Definition of ˆ x k (11),

(e): Lemma B.1,

(f): Corollary C.2, and x ∗ k ′ ∈ M k ′

Finally, ¯ R ¯ L t + ϵ ( S 0 ) ⊆ S k ′ ⊆ S k ′ , by Lemma C.6 and Lemma C.4 (ii). Therefore, ∃ k ′ ∈ ( k, k + T k ] such that

C.3 Near-Optimality Proof

Proof of Theorem 2.6. Theorem 2.6 is a direct consequence of Corollary C.2, Lemma C.8, and Lemma C.9.

D Practical Considerations

D.1 Trade-off between Safety and Optimality

In this work, we focus on safety critical systems where satisfying the safety constraints has highest priority over finding the optima. Through pessimistically considering change with time in the decisionmaking process, TVSAFEOPT emphasizes safety in non-stationary conditions at the inevitable expense of optimality. In practice, such sacrifice on optimality can be alleviated by tighter bound of rate of change.

Besides, in the case where safety can be to some extent comprised, which is beyond the focus of this work, constrained BO and its time-varying extension would be a more suitable method to apply. We refer the readers to [50, 51] for further information.

D.2 Scalability

Explicit considering time can be viewed roughly as adding dimension by 1 , Therefore, TVSAFEOPT achieves time adaptation without adding much computational cost. With the increase in dimensionality of the problem, safety constraints might arise across multiple dimensions, from multiple directions at the price of optimality. As our approach is suitable for safety critical conditions, the focus is put on maintaining safety under change, therefore safety considerations 'dictate' the optima. Additionally, in practice, the safety functions are modeled independently with a GP, and thus the computational cost scales linearly with the number of constraints.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The paper set out to solve the problem of optimizing an unknown time-varying reward subject to unknown time-varying safety constraints, as stated in the Abstract and the Introduction (Section 1). In a partial fulfilment of this goal, the paper develops a safe learning algorithm and provides safety guarantees for a general time-varying case, and near-optimality guarantees for when the time-varying problem becomes stationary.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper contains a separate section on limitations, see Section 4

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper provides the necessary assumptions in Section 2.1, the definitions in Section 2, and proofs in Appendices B and C.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper provides the necessary information in Section 3 and the background details are in Appendix A. In particular, the modifications necessary for practical implementation of the proposed algorithm, for example regarding the Lipschitz constants, are provided in Appendix A.1 and the hyperparameters for the examples are in Appendix A.2 and Appendix A.3.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The paper provides the details necessary to reproduce the simulational results in Appendix A. The code accompanying the paper is currently under review and will appear shortly at https://www.research-collection.ethz.ch/ .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The paper provides the necessary information in Section 3 and the background details are in Appendix A. In particular, the modifications necessary for practical implementation of the proposed algorithm, for example regarding the Lipschitz constants, are provided in Appendix A.1 and the hyperparameters for the examples are in Appendix A.2 and Appendix A.3.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The main contribution of the paper is in theoretical guarantees for the proposed algorithm. The only source of uncertainty in the experiments is in assumed measurement noise, with the standard deviation 0.01, making error bars negligible, as stated in respective experimental sections.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper provides the necessary information in Section 3 and the background details are in Appendix A.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The paper uses neither external datasets nor human subjects, and no data have been collected for the paper. The algorithm proposed in the paper is an optimization algorithm and thus does not introduce additional biases or privacy and dual-use concerns. The necessary licences for fair use have been provided in Appendix A.3.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The paper states the need for safe optimization algorithm with potential applications in the Introduction (Section 1, in particular Section 1.2.1).

The algorithm proposed in the paper is an optimization algorithm and thus does not introduce additional biases or privacy and dual-use concerns. The algorithm does not use external datasets, and as such does not require privacy or security considerations. Neither does the algorithm generate data that can be used for disinformation or discrimination.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: The paper does not contain data of high risk for misuse. The necessary licences for fair use have been provided in Appendix A.3.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All previously existing results used in the paper have been cited in References , including the necessary licensing information for the data for the compressor case study in Appendix A.3.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not introduce additional assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper uses neither crowdsourcing nor human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not use human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a time-varying extension of SAFEOPT to overcome the problems of time-varying rewards under time-varying safety constraints.\n\nUnder stationarity conditions, optimality guarantees are provided and the numerical simluation shows a (favorable) comparison to the SAFEOPT.\n\nSTRENGTHS:\n1. The paper is very well written and easy to follow.\n\n2. Based on related work, the problems of time-varying rewards under time-varying safety constraints are an open problem in literature, and his paper addresses that.\n\n3. The paper provides formal safety guarantees for their TVSAFEOPT algorithm.\n\nWEAKNESSES:\n1. *Some delineation to related work seems rather vague and requires stronger justification.* An example for TVSBO: the time-variable and temporal aspect of the kernel can just as well be interpreted as context using existing results. Perhaps a table would help here to highlight key aspects. \n\n2. *Lack of real-world data experiments and comparison to related work.* To support the downsides of existing approaches, an empirical comparison to existing TVSBO approaches mentioned in the related work section would be needed.\n\n3. The *empirical results could be more convincing* by adding a variety of initial safe sets and revised plots. The current plots/results are hard to parse. \n\n4. It would be beneficial if the *theoretical/technical challenge of extending safety to the time-varying case were more detailed*. This would streamline the presentation and help in assessing the impact of the contribution.\n\nQUESTIONS:\n1. *In the Appendix, a spatio-temporal SE kernel is introduced. How is this construction different from using an SE-ARD kernel with a composite variable $z = [x^T,t]^T$?* If I am not mistaken, for the SE-ARD kernel there would be no different than defining a single kernel with $z$. \n\n2. It is mentioned that the Lipschitz constants are to be known beforehand. However, while commonly assumed, *how do you get a hold of an RKHS norm bound $B$ (related to Assumption 2.1) to compute the UCB?* \n\n3. *Could you provide Figure 1 sooner in the manuscript?* It would be super helpful to see this central illustration already on page 2.\n\nLIMITATIONS:\n1. Practicality of the safety guarantee: requiring many Lipschitz constants for both for space and time while also requiring an RKHS norm bound.\n\n2. Theoretical and empirical impact: Lack of comparisons to TVSBO approaches makes the implications of the contribution unclear both theoretically and empirically.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

comments: SUMMARY:
This paper presents a safe Bayesian optimization algorithm TVSAFEOPT with a spatial-temporal kernel and time Lipschitz constants, which improves on SAFEOPT with time-varying reward and safety constraints. The optimality guarantee is proved for the stationary case and the safety guarantee for more general settings. The method is tested on a synthetic problem and gas compressors.

STRENGTHS:
1. The use of a spatio-temporal kernel in Bayesian optimization for time-varying safety constraints is novel.
2. A formal proof of safety and optimality guarantee under certain assumptions.

WEAKNESSES:
1. More discussion on how to make a tradeoff between optimality and safety is encouraged. 
2. Will this conservatism in safety become too large in high-dimensional problems?
2. The method to choose the proper initial safe set and kernel parameters is unclear.

QUESTIONS:
1. How to find an initial safe set for complex problems?
2. How to find the kernel parameter for each task?
2. What is the computational complexity compared to other BO baselines?

LIMITATIONS:
The societal impact is discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

comments: SUMMARY:
The paper introduces the TVSAFEOPT algorithm, which is based on Gaussian processes with spatio-temporal kernels, designed specifically for optimizing time-varying rewards under time-varying safety constraints. The algorithm provides formal safety guarantees in a general time-varying setting, ensuring safety even when exploring non-stationary safe regions. It robustly subtracts safety margins to prevent unsafe decisions, adapting in real-time to changing environments. Furthermore, they provide optimality guarantees for locally stationary optimization problems, ensuring near-optimal solutions when the optimization problem becomes stationary.

STRENGTHS:
They provide formal safety guarantees in dynamic environments, ensuring safe decision-making even in non-stationary settings. 

Additionally, the algorithm offers optimality guarantees for stationary optimization problems, enhancing its reliability and performance

Extensive numerical simulations were provided to validate the proposed approach.

WEAKNESSES:
They extend the Safeopt algorithm from literature. However, it is clear on what are the additional contributions and difference between these two different approaches.

QUESTIONS:
-

LIMITATIONS:
-
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-13