Meta-Learning Priors for Safe Bayesian Optimization
Authors: Jonas Rothfuss
Christopher Koenig
Alisa Rupenyan
Andreas Krause
Venue: CoRL 2022 Oral (2022)
Keywords: Safety
Meta-Learning
Bayesian Optimization
Gaussian Processes

Abstract: In robotics, optimizing controller parameters under safety constraints is an important challenge. Safe Bayesian optimization (BO) quantifies uncertainty in the objective and constraints to safely guide exploration in such settings. Hand-designing a suitable probabilistic model can be challenging however. In the presence of unknown safety constraints, it is crucial to choose reliable model hyper-parameters to avoid safety violations. Here, we propose a data-driven approach to this problem by em meta-learning priors for safe BO from offline data.
We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner via empirical uncertainty metrics and a frontier search algorithm. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learnt priors accelerate convergence of safe BO approaches while maintaining safety.


Meta-Learning Priors for Safe Bayesian Optimization
Jonas Rothfuss
Christopher Koenig inspire AG, ETH Switzerland chkoenigdethz.co ch
Alisa Rupenyan inspire AG, ETH Switzerland ralisa@ethz.ch ch
Andreas Krause ETH Zurich Switzerland krauseaßethz.co ch
ETH Zurich Switzerland roj rojonas rojonas@ethz.co ch
Abstract: In roboties, optimizing controller parameters under safety constraints is important challenge. Safe Bayesian optimization (BO) quantifies uncertainty in the objective and constraints to safely guide exploration in such settings. Hand- designing a suitable probabilistic model can be challenging however. In the pres- ence of unknown safety constraints, it is crucial to choose reliable model hyper- parameters to avoid safety violations. Here, we propose a data-driven approach to this problem by meta-learning priors for safe BO from offline data. We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, develop novel framework for choosing safety-compliant priors in data-riven manner via empir- ical uncertainty metrics and a frontieer search algorithm. On benchmark functions and high- precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches while maintaining safety.
Keywords: Meta-learning, Safety, Controller tuning, Bayesian Optimization
Introduction
Optimizing a black-box function with as few queries as possible is a ubiquitous problem in science and engineering. Bayesian Optimization (BO) a promising paradigm, which learns a probabilistic surrogate model (often Gaussian process, GP) the unknown function to guide exploration. BO has been successfully applied for optimizing sensor configurations [1, 2] or tuning the parameters of robotic controllers [3, 4, 5, 6, 7]. However, such world applications are often subject safety constraints which must not be violated in the process of optimization, e.g., the robot not getting dam- aged. Often, the dependence of the safety constraints on the query inputs is a priori unknown and can only be observed by measurement. To cope with these requirements, safe BO methods [8, 9, 10] model both the objective and constraint functions with GPs. The uncertainty in the constraint is used to approximate the feasible region from within, to guarantee that no safety violations occur. Therefore, critical requirement for safe BO is the reliability of the uncertainty estimates of our models. Typically, it is assumed that correct GP prior, upon which the uncertainty estimates hinge, is exogenously given [e.g.. 8, 9, 10]. In practice, however, appropriate choices for the kernel variance and lengthscale are unknown and typically have to be chosen very conservatively or hand-tuned by trial and error, a problematic endeavour in a safety-critical setting. A too conservative choice dra- matically reduces sample efficiency, whereas overestimating smoothness may risk safety violations.
Addressing these shortcomings, we develop an approach for meta-learning informative, but safe, GP priors in data-driven way from related offline data. We build on the F-PACOH meta-learning method [11] which is capable of providing reliable uncertainty estimates, even in the face of data-scarcity and out-of-distribution data. However, their approach still relies on an appropriate kernel choice on which it falls back in the absence of sufficient data. We propose a novel framework for choosing safety-compliant kernel Syyer--arameteen in data-driven manner based on calibration and sharpness metrics of the confidence intervals. To optimize these uncertainty metrics, we devise frontier search algorithm that efficiently exploits the monotone structure of the problem. The result- ing Safe Meta- -Byyeian Optimization (SAMBO) approach can be instantiated with existing safe BO
6th Conference Robot Learning (CoRL. 2022), Auckland, New Zealand.

---

methods and utilize the improved, meta-learned GPs to perform safe optimization more efficiently. In our experiments, we evaluate and compare our proposed approach on benchmark functions as well as controller tuning for a high-precision motion system. Throughout, SAMBO significantly improves the query efficiency of popular safe BO methods, without compromising safety.
Related Work
Safe BO aims to efficiently optimize black-box function under safety-critical conditions, where un- known safety constraints must not be violated. Constrained variants of standard BO methods [12. 13, 14] return feasible solutions, but do not reliably exclude unsafe queries. In contrast, SAFEOPT [8, 9] and related variants [15] guarantee safety at all times and have been used to safely tune controllers in various applications [e.g., 16, 17]. While SAFFOPT explores an undirected manner, GoOSE [10, 18] does so by expanding the safe set a goal-oriented fashion. All mentioned methods rely on GPs to model the target and constraint function and assume that correct kernel
given. Our work is complementary: We show how to use related offline data to obtain informative and safe GP priors in a data-driven way that makes the downstream safe BO more query efficient.
Meta-Learning. Common approaches in meta-learning amortize inference [19, 20, 21J, learn a shared embedding space [22, 23, 24, 25] or good neural network initialization [26, 27, 28, 29]. However, when the available data is limited, these approaches prone to overfit the meta- level. A body of work studies meta-regularization to prevent overfitting in meta-learning [30, 31, 32, 33, 34, 35J. Such meta-regularizztion methods prevent meta-overfitingg for the mean predictions, but not for the uncertainty estimates. Recent meta-learning methods aim at providing reliable confidence intervals even when data is scarce and non-i.i.d [11, 36, 37J. These methods ex- tend meta-learning to interactive and life-long settings. However, they either make unrealistic model assumptions or hinge on paremeerrseterss whose improper choice is critical in safety constraint set- tings. Our work uses F-PACOH [11] to meta-learn reliable GP priors, but removes the need for hand- specifying correct Slype--proor by choosing its parameters in data-driven, safety-aware manner.
3 Problem Statement and Background
3.1 Problem Statement
We consider the problem of safe Bayesian Optimization (safe BO), seeking the global minimizer
arg min f(x) s.t. q(x) 0
xEX
of a function f: X' + R over bounded domain subject to a safety constraint g(x) S 0 with constraint function q X' + R. For instance, we may want to optimize the controller parameter of robot without subjecting it to potentially damaging vibrations collisions. During the optimization, we iteratively make queries X1, .... XT E X' and observe noisy feedback fi.... fr and q1, .... qr, t, e.g., via fe [(x)) + ff and In our setting, performing query is assumed to be costly, e.g., running the robot and observing relevant measurements. Hence, we want to find a solution as close to the global minimum in as few iterations as possible without violating the safety constraint with any query we make.
7(x)) lq where cf, fq is o² sub-Gaussian noise [38, 39].
Additionally, we assume to have access datasets D D1,;; Dr,,,, with observations from sta- tistically similar but distinct data-generating systems, e.g., data of the same robotic platform under different conditions. Each dataset D,,,, = 1, q..1..... (x,, ft.,. q..T:) consists of T measurement triples, where fr,+ fx(,),)) + cy, and t., (xx,,) + Cu. are the noisy target and constraints observations. We assume that the underlying functions fi, fn and qı,..., Zn. which generated the data are representative of our current target and constraint functions f and e.g., that they are all i.i.d. draws from the same stochastic process. However, the data within each dataset may be highly dependent (i.e., non-i.i.d)). For instance, each Da.1; may correspond to the queries and observations from previous safe BO sessions with the same robot under different conditions.
In this paper, we ask the question of how can harness such related data sources to make safe BO on our current problem of interest more query efficient, without compromising safety.
2

---

3.2 Safe Bayesian Optimization Methods
Safe BO methods construct Bayesian surrogate model of the functions f and based on previous
observations D, = {(xe, fer q Deee.. Typically, a Gaussian Process GP (f))))((x)) k(x, x')) with mean m((x and kernel function k(x, x') is employed to form posterior beliefs p (x) Dz) =
N(4 (x), (of (x))²) and (D;) = . (x](x) (of (x))²) over function values [40]. Based on
the predictive posterior, we can form confidence intervals (CIs) to the confidence level a E [0.
(((() = J (x) = 3 (a)of (xx]
(2)
where B (a), the scaling of the standard deviation, is set such that / f(x) is in the CI with probability a. For BO, often employ shift invariant kernel k(x,x') = [ x'I1/t), where v is its vari- ance, the lengthscale and Φ positive function, e.g., squared exponential (SE) Φ(z) = exp(-2²).
BO methods typically choose their query points by maximizing an acquisition function based on p()()))P;), trading-off exploration and exploitation [41, 42, 43, 44). When we have safety con- straints, we need to maintain a safe set Si(a) {x E X1 (x) + 3 (a)o (x) < 0} which contains parts of the domain we know to be safe with high-probability a. To maintain safety, we can only query points within the current Sa(a). In addition, need to explore w.r.t. q so that can expand S. E.g., SAFFOPT [8, 9, 16] computes safe query candidate that optimizes the acquisition function for f as well as a query candidate that promises the best expansion of S, Then, it selects the one with the highest uncertainty. While SAFEOPT expands S, undirectedly, GOOSE [10. 18] does so in a more directed manner to avoid unnecessary expansion queries, irrelevant for minimizing f. In ad- dition to the pessimistic St, it also maintains an optimistic safe set which is used to calculate query candidate x x that maximizes the acquisition function for f. If x2 xPt is outside of S,, it chooses safe query points aiming at expanding S, in the direction of x. See Appx. A for more details.
3.3 Meta-Lea GP Priors
In meta-learning [45, 46], we aim to extract prior knowledge (i.e., inductive bias) from a set of related learning tasks. Typically, the meta-learner is given such learning tasks in the form of datasets D1.7, ...,, D, with Dr,,, {(xr,,, v1,)) and outputs a refined prior distribution or hypothesis space which can then be used to accelerate inference on a new, but related, learning task.
Prior work proposes to meta-learning GP priors [47, 48, 34), tough, fails to maintain reliable uncertainty estimates when data scarce and/or non-i.i.d.. The recently introduced F-PACOH method [11] overcomes this issue, by using regularization approach in the function space. As previous work [e.g., 49, 48], they use learnable GP prior pa(h) = GP(h(x)) ((xxx)mm(x)) mx , ko (x, x')) where the mean and kernel function are neural networks with parameters 0 and employ the marginal log- aigllikelihood to fit the (h) to the meta-training data. However, during the meta-learning, they regularize pa towards a Vanilla GP hyper-prior p(h) = GP(h(x)|0, k(x, x)) with the SE kernel.
L.d.
To do so, they uniformly sample random measurement sets X = [x1...., X.n] ~
U((2( from the
domain and compare the GPs finite marginals po (h> = Po (h(x)) ); ...., l(xm)) )) and p(h*) through
their KL-divergence. The resulting meta-learning loss with the functional KL regularizer
( Vni
(()) ==
T1 Σ
T: ID po)
+
Ex [KL (1xx |]lp(b>))] (10)0o( T.T
(3)
marginal log. likelibond
functional KL-divergence
makes sure that, in the absence of sufficient meta-training data, the learned GP behaves like Vanilla GP. Overall, this allows to meta-leam more informative GP prior which still yields reliable confidence intervals, even if the meta-training data was collected via safe BO and is thus not i.i.d.
4 Choosing the Safe Kernel Hyper-Parameters
Important for safe BO is the reliability of the uncertainty estimates of our objective and constraint models. For GPs, the kernel hyper-parameteed with the biggest influence on the Cls. If the kernel variance y is chosen too small and/or the lengthscale to large, our models become over-confident
3

---

and the corresponding BO unsafe. In the reverse case, the CIs become too conservative and safe BO requires many queries to progress. Despite the assumption commonly made earlier work, [e.g., 8, 9, 10], appropriate choices for v and are unknown. In practice, they are typically chosen conserva- tively or hand-tuned by trial and error, problematic safety-critical settings, Aiming to address this issue, we develop framework for choosing the kernel saper-parameters in data-driven manner.
4.1 Assessing kernel hyper- paper-parameters: Calibration and sharpness
Our approach is based on the calibration and sharpness of uncertainty estimates [see e.g. 50, 51, 52, 53]. Naturally, if construct CIs to the confidence level a, we want that at least an a percentage of (unseen) observations to fall within these Cls. If this holds in expectation, estimates are calibrated. If the empirical percentage is less than a, it indicates that our model's un- certainty estimates are over-confident and we are likely to underestimate the risk of safety violations. To empirically assess how calibrated probabilistic regression model with hyper--aramann parameters w, con- ditioned on a training dataset D is, we compute its calibration frequency on test dataset Diet:
say that the uncertainty
1 calib-freq(D". Diou, w) = LA
Σ DEA
1
Σ (1 (x/D", (w)D 2 (x.Y)EDIM
(4)
Diesi
Here, A C. [0, 1] is a set of relevant confidence levels (in our case 20 values equally spaced between 0.8 and 1.0). Since the CIs of our model need be calibrated at any iteration during the BO and for any task we may face, we choose the best empirical estimate we can. We compute the average calibration frequency across all meta-training datasets and for any sub-sequence of points within dataset. In particular, for any task i = 1,..., n and t 1,..., T; - we condition/train our model on the data points Di,gt = {(x,r' B.rr) and use the remaining data points Dist = {(x,,'' Vi ) to compute the calibration frequency, Overall, this gives us
1 avg-calib((P.n)) M. 71 _
calib-freq (Di, Di,se,w).
(5)
While the calibration captures how reliable the uncertainty estimates are, does not reflect how useful the confidence intervals are for narrowing down the range of possible function values. For instance, predictor that always outputs mean with sufficiently wide confidence intervals is calibrated, but useless for BO. Hence, we also consider the sharpness of the uncertainty estimates which we empirically quantify through the average predictive standard deviation. Similar to (5), we average over all tasks and data sub-sequences within each task:
a
T.-1
evg-std([2) w) Σ
Σ ((((,,
(6)
T -
Di.e
(x.))E,,,
The avg-std measures how concentrated the uncertainty estimates are and, thus, constitutes a natural complement calibration which can be simply achieved by wide/loose confidence intervals.
4.2 Choosing good prper-parameters via Frontier search
Based on the two empirical quantities introduced above, we can optimize the hyper-paramettrn w of our model as to maximize sharpness (i.e., minimize the avg-std) subject to calibration [50]:
min avg-std( (D.,,:
s.t. avg-calib ((Di, w) 2
Since computing avg-std( ( (Pi,r; ) w) and wwg.calib(Pi..r. w) requires solving the the GP inference problem many times, each query is computationally demanding. Hence, we need an opti- mization algorithm for (7) that requires as few queries as possible to get close to the optimal solution.
We develop efficient frontier search (FS) algorithm that exploits the monotonicity properties of this optimization problem. Both avg-std and avg-calib are monotonically increasing in the kernel variance v and decreasing in the lengthscale I1. By setting = (-1, and writing
Note that the monotonicity of the calibation frequency in / is only an empirical heuristic that holds in almost all cases if is at least as big the variance of the targets y in dataset.

---

Algorithm FRONTIERSEARCH (details in Appendix C)
Input: Domain bounds z', z" s.t. z' <x S z"
1: Q" < {z"), Q' + {z'}
2: for 1, LARGESTMAXMINRICT. 2 Q4)
do
3:
(Z,; z,,) &
Largest max-min rect betw. frontiers
4:
f- DESTWORSTCASEQUURRYYC (r, by, O Q")
I/ Best query point to split rectangle
5:
if ((2)))) 2 I then
6:
2u + PRUNE {2+})
7:
else
8:
2 f PRUNE( O' U (za})
Return: arg min Q" s(z)
iter
iter 5
iter 10
iter 20
4.0
4.0
4.0
4.0
kerell Variance
upper front.
lower front.
2.0
2.0
2.0
2.0
safety barder
solution
optimum
1.00
1.00
1.00
1.00
kernel longthscaae
kornel lengthscale
kornel lengthscale
kernel longthscale
Figure I:
Frontier search (FS) on the kernel lengthscale and variance for the constraint model Argus robot. Red: areas ruled out, because unsafe. Green: Safe areas that ruled out since dominated by better safe queries. After a few iterations, FS has already shrunk the set of possible optima (white area between fronts) points close the safety boarder and picked nearly optimal solution (cross).
s(2) avg-std( (Di, 1i 1, v) and c(z) avg-calib( |i v), we can turn (7) into
min s(z) s.t. c(z) 2 where s(z) R + R and c(z) IR? IR are monotone.
(8)
We presume upper and lower bound (z" zz) such that resulting search domain Z |2/ a1' (2l, = contains the optimal solution z = arg min ze/ s().. Since both s(2) and c(z) are mono- tone we know that zt must lie on or directly above the constraint boundary c(z) = 1 (see Lemma 2).
In each iteration & of Algorithm we query point zi E 2 and observe the corresponding objective and constraint values s(z)) and c(z)). We separate the queries into two sets Q" and Q! based on whether they lie above below the constraint boundary. That is, add z to Q" if c(z)) 2 and to Q' otherwise. Since the optimal solution lies on the constraint boundary and c(z) is monotone, we can rule out entire corners of the search domain: For each 2 E Q" we can rule all points z' zi as candidates for the optimal solution and, similarly for all z E Q', we can rule out all z' S z This also allows us to prune the sets Q" and Q' by removing all the points from them that can be ruled out by a new query results. To keep track which parts of Z have not been ruled out yet, we construct upper and lower frontiers, here expressed functions 21 + 22 and 22 + 21
P (21; 2*) min 2/2 2 2 z. 2*), F2 2 = max max 21 221 C)) (( = max (2/ 22 2 d Q') (10)
(22 Q") min ( 2/ 2*) (9)
such that the points frontiers still plausible candidates for the optimal solution. For notational brevity, define J = {z Z|FT (22; Q") V FF Q") 22) and F' analogously the set of points that lie on the upper and lower frontier respectively.
2(= Z Q)) < 22 S PTr:) C Z between the
At any point, the best solution to (8) that we can give, is the best query we have made so far that fulfills the constraint, i.c.,
arg min Q= s(z). If we assume Lipschitz continuity for s, which holds in our case holds since Z is bounded and the avg-std is differentiable in and v, can bound how much our best solution is away from the optimum, i.c., s(z"):
Lemma 1. Let s(z) be L Lipschitz and F") max minze ||z z'|l the max-min distance between the frontiers. Then, the sub-optimality is bounded by s(2) s(z*) E L d() F").
5

---

Algorithm 2 Safe Meta-BO (SAMBO)
Input: Safe BO problem with flar and qla", set of datasets (D Mi=1, (Di )
1: for h E { g) do
2: (In, 1h)) < FRONTIERSEARCH(
3: po,, (h) <- (PACOH((D),.) YE=1 Pia (h))
Return: 2² < SAFEBO(FLa, qlar, po, (f), pa. (4))
Here, the key insight is that we can bound the sub-optimality 8(2) - s(2") with the maximum distance of any point between the frontiers from its closest point on the upper frontier instead of 2. This is the case because 2 dominates any point on the upper frontier (i.e., s(2) S s(z') Vz E F").
Hence, we want to choose the next query that we can shrink the max-min distance between the frontiers the most. For this purpose, we select the largest max-min rectangle between the frontiers (see Definition 5). Then, we choose the query point, that reduce the max-min distance within the rectangle the best. For this we need to consider two scenarios that will affect the max max-min distance differently: either the query point satisfies the constraint (c(z²) 2 1) or it does not (c(z²) <
1). We compute the rectangle's max- mii distance for both scenarios and choose the query-point point that gives us the lowest max-min distance in the less-favorable (worst-case) scenario. For more details refer Appendix C. Finally, we provide worst-case rates for the proposed frontier search algorithm:
Theorem 1. Under the assumptions of Lemma 1, Algorithm I needs no more than k S 3 (lgg)(1/)) O ((1/c)1.00) iterations to have a sub-optimality of less than s(2) s(z") ) S L 1 - z (1/c)-
The O ((1/e)1-39) 1559) query complexity is more efficient than those of similar algorithms that do not make use of the monotonicity properties, c.g., that of grid search: O ((L/E+1))))
5 Safe BO with meta-learned GP priors
In Sec. 4 we discuss how to efficiently choose the kernel variance and lengthscale, so that we obtain calibrated and yet sharp uncertainty estimates. Now, we one step further and use the related datasets DI.T..... Dn,T, to meta-learn GP priors, aiming to give the downstream safe BO algorithm more prior knowledge about the optimization problem at hand so that it can be more query efficient.
For this, we the F-PACOH [11] meta-learning approach, introduced in Sec 3. We choose this method since it regularizes the meta-learned model in the function space and, thus, is able to maintain reliability of the uncertainty estimates, even for out-of-distribution data. F-PACOH requires Vanilla GP p(h) = (P k(x,xx)) as hyper-prior, towards which it regularizes the meta-learned GP po (h) = ()(((()))mm()))) (x, x'). Hence, we face a similar kernel hyper- parameter choice problem as addressed in Section 4: the kernel parameters of the hyper-prior are chosen such that the Vanilla GP itself yields over-confident uncertainty estimates, safe BO with the meta-learned prior will most likely turn out to be unsafe well. Hence, we use the calibration- sharpness based frontier search to choose the kernel variance and lengthscale of the hyper-priee -rierr
Since safe BO maintains separate models for the target f'x) and constraint 7(x), we meta-learn a prior for each of the functions. We split the sets of data triplets DI,1; = {(x,,,, Jrs, tr.)) ?
into separate datasets D',;; = {xx.,, Fi.))]. and D {xx:,, f First, we use frontier search to find kernel parameters (Ly, and (4y, vyy on the respective set of datasets. Then, we meta-learn GP priors pa, (f) and Pa, (4) for f and with F-PACOH. while using the Vanilla GPs pl,,,, (f)) and Pi,,,,()) with the chosen kernel hyper-parameters as hyper-prior. Finally, we run a safe BO algorithm (either SAFEOPT or GOOSE) with the meta-learned GP priors and perform safe Bayesian optimization on our target problem of interest. This procedure is summarized in Algorithm 2. We refer our Safe MO (SaMBO) algorithm with GOOSE, SAMBO-G and, when instantiated with SAFEOPT as SAMBO-S.
that give good sharpness subject to calibration

---

calibration-Fi freq.
avg.sta
max. safety constr.
cum. inferenceeeeeee regret
1,0000
0.9995
Komal varaance
komel varlanca
safety barder
solution
0.9990
75
0
kereel
0.9905
50
-50
kemel
0.9980
25
-00
0.5
1.0
kemel lengthsccale
kernel lengthscake
lengthscale
lengthsrale
Figure 2: Left: calib and avg-std across grid of kernel lengthscales b and variances Vy for the constraint model. Right: max safety constraint value and cumulative regret for 200 iterations of safe BO. Calibration and sharpness are good proxies for safety and efficiency of the downstream safe BO.
Experiments
First, we investigate if the ciibrationn/shaappess--beeee frontier search chooses good and safe kernel parameters. Second, we evaluate the the full SAMBO algorithm in range of safe BO environments.
6.1 Experiment Setup
In the following, we describe our experiment setup and methodology. See Appx. F for more details.
Synthetic safe BO benchmark environments. We consider two synthetic environments, based on popular benchmark functions with two-dimensional domain. The first is based the Camelback function [54] overlaid with products of sinusoids of varying amplitude, phase and shifts. The constraint function is similar, with an additional random quadratic component to ensure connection between most of the safe regions of the domain. The second enviromment is based on the challenging Eggholder function [55] with many local minima. The function's shape can be varied by random sampling of three parameters. The constraint is a quadratic function, overlayed with sinusoids of varying frequencies. task corresponds to a pair of randomly drawn target and constraint functions.
Controller Tuning for high-precision linear robot. As robotic case study, we tune the controller of linear axis in an Argus linear motion system by Schneeberger Linear Technology, a high-pprecisionsspeed robot for wafer inspection. To goal is to tune the three gain parameters of a cascaded PI controller to achieve minimal position error. We minimize the total variation (TV) of the position error signal, while constraining maximum frequency in the FFT which measures (potentially damaging) instabiliiess/iibraioons in the system. Different tasks correspond to different step sizes, ranging from 10jm to 10mm. At different scales, the robot behaves differently in response to the controller parameters, resulting in different target and constraint functions. The experiments are conducted with simulation of the robot, so we can explore unsafe kernel hyper-parameters- - key element for finding and visualizing the safety boundary in Figure 2.
Meta-training data. We generate the benchmark meta-training by running SAFEOPT with conser- vative kernel hyper-parameters choices on each task. The resulting datasets are non-i.i.d. and only consist of observations where (zz) < 0. much more realistic than sampling data uniformly and i.i.d.. With this, we aim to mimic practical scenario where we have performed various safe BO on related tasks in the past and now want to harness the corresponding data. For the synthetic environments, we use n = 40 tasks with T; = 100 samples in case of the Camelback-Sin and T; = 200 samples for the Random Eggholder environment. For the Argus controller tuning, we use n = 20 and T, = 400.
Evaluation metrics. To evaluate the performance of various methods, run safe BO with them on at least 4 unseen test tasks with each 5 seeds. To measure method's 's query efficiency, report the (safe) inference regret = f(ī) -x(x") where x = arg min zes, (x) is method's 's best current guess for the safe minimizer of (1). In Fig. we also report the cumulative regret Ri : :- Σ'= Tt-
6.2 Choosing the kernel parameters via calibration & sharpness based frontier search
We investigate how well the calibration and sharpness based frontier search (FS) for finding kernel hyper-parametert works in practice. For that, consider the Argus controller tuning problem. Generally, perform FS in the log-space of I and since we found this to work better in practice.
7

---

Camelback Function + Random Sinusoids
Random Eggholder Functions
10a
inference negret
10a
Sdttel constrains
Inferunce regeut
safety corstrains
-100
10-1
-200
-300
400
101
50 Iteration
100
50
100
100
200
100
200
SaMBO-G
SaMBO-S
GoDSE
SafeOpt
safety constr.
Figure 3: Safe BO regret and safety constraint the synthetic benchmarks. SAMBO converges significantly faster towards the global optimum than the baselines, without violating the constraints.
Fig. displays the frontier search at various iterations. FS quickly shrinks the of possible safe optima (area between upper frontier (green) and lower frontier (red)) to points close to the safety border. After only 20 iterations, it already found solution that is very close to the safe optimum. This showcases the efficiency we gain thanks to taking into account the monotonicity of the problem.
Furthermore, we investigate how well calibration and sharpness reflect what we care about: 1) No safety constraint violations and 2) query efficiency. We compute the maximum constraint value across 200 iterations with GoOSE as well the cumulative regret across grid of kernel length- scales and variances. Fig. 2 holds the results, together with the calib and stg-std. Overall, calibration and sharpness of the GPs' uncertainty estimates are good proxy for the downstream safety during BO and, respectively, the regret. Importantly, all parameters that fulfill the calibration constraint calib( (D,,T, ) // W) 2 lead to safe BO runs without constraint violations. Finally, the kernel pa- rameters, chosen by FS, are both safe and lead a small cumulative regret. Overall, this empirically supports the validity of our data-driven upproach for choosing good, but safe, kernel parameters.
6.3 Safe Bayesian Optimization Benchmark & Controller Tuning
We compare the two SAMBO instantiations, SAMBO-S and SAMBO-G, with their corre- sponding safe BO baseline methods SAFFOPT and GoOSE. For the baselines, we use the GP kernel parameters found by FS. Fig. 3 displays the results for the synthetic benchmark func- tions and Fig. 4 for the Argus controller tuning. Note that, in case of the regret, the shaded areas correspond to confidence intervals while for the safety constraint values they correspond to the entire range of values (i.e., max- min). Overall, SAMBO converges to near optimal solutions much faster than the baselines without meta-learning. Across all environments, there are no safety violations which demonstrates that 1) the kernel pa- rameters by FS are safe and 2) SAMBO maintains safety thanks to principled regularization in the function space during meta-learning. The improved query efficiency without compromising safety in the controller tuning setting, where the constraint (maximum frequency in the signal) is highly non-smooth, demonstrates the applicability of SAMBO to challenging real-worle robotics problems.
infersnce rigret
0.04
fele constraint
-100
-150
100
200
0
100
200
iteration
iteraton
SaMBO-G
SaMBO
GoOSE
SafeOpt
Figure 4: Safe controller tuning for the Argus robot. SAMBO safely finds good controller pa- rameters faster than safe BO baselines.
7 Summary and Discussion of Limitations
We have introduced data-driven framework for choosing kernel parameters even meta-learning GP priors that are both informative and reliable. When combined with a safe BO algorithm, the resulting SAMBO speeds up the optimization of, e.g., controller parameters, without compromising safety. Except for the observation noise variance, which is often known or easy to estimate, our framework makes safe BO free of typpe-parameters and, thus, more robust and practical. However, relies on the availability of offline data that is both sufficient in quantity and representative of the target task. As approach relies empirical estimates of the calibration, it may fail to ensure safety when given too little data or tasks that are systematically different to the target task.

---

Acknowledgments
This research was supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program grant agreement 815943, the Swiss National Science Foundation under NCCR Automation, grant agreement 51NF40 180545, and the Swiss Innovation Agency (Innosuisse), grant number 467 16. Jonas Rothfuss was supported by an Apple Scholars in AI/ML fellowship. We thank Parnian Kassraie for proofreading the manuscript and making useful suggestions how to improve it.
References
[1] R. Garnett, M. A. Osborne, and S. J. Roberts. Bayesian optimization for sensor set selection. In Proceedings of the 9th ACM/IEEE international conference on information processing in sensor networks, pages 209-219, 2010.
[2] E. Cishani, A. Del Dotto, Fanelli, M. Williams, M. Alfred, F. Barbosa, L.. Barion, V. Berd- nikov, W. Brooks, T. Cao, al. Ai-optimized detector design for the future electron-ion col- lider: the dual-radiator rich case. Journal of Instrumentation, 15(05):P05009, 2020.
[3] A. Marco, P. Hennig, J. Bohg, S. Schaal, and S. Trimpe. Automatic lqr tuning based on gaussian process global optimization. In 2016 IEEE international conference on robotics and automation (ICRA), pages 270-277. IEEE, 2016.
[4] M. Neumann-Brosig A. Marco, D. Schwarzmann, and S. Trimpe. Data-efficient autotuning with bayesian optimization: An industrial control study. IEEE Transactions Control Systems Technology, 28(3):730-749 2019.
[5] M. Rowold, A. Wischnewski, and B. Lohmann. Constrained bayesian optimization of linear feed-forward controller. PaAC-PaperOOnL.iee 52(29):1 2019. ISSN 2405-8963. 13th IFAC Workshop on Adaptive and Learning Control Systems ALCOS 2019.
[6] M. Khosravi, V. Behrunani, R. S. Smith, A. Rupenyan, and J. Lygeros. Cascade control: Data- driven tuning approach based on bayesian optimization. IFAC-PapersOnLine, 53(2):382-387, 2020.
[7] A. Rupenyan, M. Khosravi, and J. Lygeros. Performance-basse trajectory optimization for path following control using bayesian optimization. In 60th IEEE conference on Decision and Control (CDC 2021), 2021.
[8] Y. Sui, A. Gotovos, J. W. Burdick, and A. Krause. Safe exploration for optimization with Gaussian processes. JCML, 2015.
[9] Berkenkamp, A. P. Schoellig, and A. Krause. Safe controller optimization for quadro- tors with gaussian processes. In IEEE International Conference on Robotics and Automation (ICRA), pages 491 2016.
[10] M. Turchetta, F. Berkenkamp. and A. Krause. Safe exploration for interactive machine learn- ing. In Advances in Neural Information Processing Systems, 2019.
[11] Rothfuss, D. Heyn, J. Chen, and A. Krause. Meta-learning reliable priors in the function space. Advances in Neural Information Processing Systems, 34, 2021.
[12] J. M. Hernández-Lobato, M. A. Gelbart, M. W. Hoffmmn, R. Adams, and Z. Ghahramani. Predictive entropy search for Bayesian optimization with unknown constraints. In JCML, 2015.
[13] M. Khosravi, C. Koenig, M. Maier, R. Smith. J. Lygeros, and A. Rupenyan. Safety-aware cascade controller tuning using constrained bayesian optimization. IEEE Transactions on In- dustrial Electronics, 2022.

---

[14] M. Fiducioso, S. Curi. B. Schumacher, M. Gwerder, and A. Krause. Safe contextual bayesian optimization for sustainable room temperature pid control tuning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 5850-5856. AAAI Press, De- cember 2019.
[15] Y. Sui, vincent Zhuang, J. Burdick, and Y. Yue. Stagewise safe Bayesian optimization with Gaussian processes. In ICML, volume 80 of Proceedings of Machine Learning Research, pages 4781-4789. PMLR, 10-15 Jul 2018.
[16] F. Berkenkamp, A. Krause, and A. P. Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics. Machine Learning, pages 1-35, 2021.
[17] M. Khosravi, Eichler, N. A. Schmid, R. S. Smith, and P. Heer. Controller tuning by bayesian optimization an application to a heat pump. 2019 18th European Control Conference (ECC), pages 1472-147, 2019.
[18] C. König, M. Turchetta, J. Lygeros, A. Rupenyan, and A. Krause. Safe and efficient model- free adaptive control via bayesian optimization. 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9782-9788. IEEE, 2021.
[19] S. Hochreiter, A. S. Younger, and P. R. Conwell. Learning To Learn Using Gradient Descent. In International Conference on Artificial Neural Networks, 2001.
[20] M. Andrychowicz, M. Denil, S. G. Colmenarejo, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. De Freitas. Learning to learn by gradient descent by gradient descent. arXiv, 2016.
[21] Y. Chen, M. W. Hoffman, S. G. Colmenarejo, M. Denil, T. P. Lillicrap. M. Botvinick, and N. De Freitas. Learning to Learn without Gradient Descent by Gradient Descent. In Interna- tional Conference on Machine Learning, 2017.
[22] Baxter. model of inductive bias learning. Journal of Artificial Intelligence Research, 2000.
[23] J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, 2017.
[24] O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al. Matching networks for shot learning. In Advances in Neural Information Processing Systems, 2016.
[25] Harrison, Sharma, and M. Pavone. Meta-leamingg priors for efficient online bayesian regression. In International Workshop on the Algorithmic Foundations of Roboties, pages 318-337. Springer, 2018
[26] C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. International Conference on Machine Learning, 2017.
[27] J. Rothfuss, D. Lee, I. Clavera, T. Asfour, and P. Abbeel, ProMP: Proximal Meta-Policy Search. In International Conference on Learning Representations, 2019.
[28] A. Nichol, J. Achiam, and J. Schulman. On First-Order Meta-Learning Algorithms. arXiv, 2018.
[29] Kim, J. Yoon, Dia, S. Kim, Y. Bengio, and S. Ahn. Bayesian model-agnostic meta- learning. In Advances in Neural Information Processing Systems, 2018.
[30] A. Pentina and C. Lampert. A PAC-Bayesian bound for lifelong learning. In International Conference on Machine Learning, 2014.
[31] R. Amit and R. Meir. Meta-learning by adjusting priors based on extended PAC-Bayes theory. In International Conference Machine Learning, 2018.
10

---

[32] Y. Balaji, S. Sankaranarayanan, and R. Chellappa. Metareg: Towards domain generalization using meta-regularivation. In Advances in Neural Information Processing Systems, volume 31, 2018.
[33] M. Yin, G. Tucker, M. Zhou, S. Levine, and C. Finn. Meta-learning without memorization, In International Conference on Learning Representations, 2020.
[34] J. Rothfuss, V. Fortuin, M. Josifoski, and A. Krause. PACOH: Bayes-optimal meta-learning with PAC-guarantees. In International Conference for Machine Learning (ICML), 2021.
[35] A. Farid and A. Majumdar. Generalization bounds for meta-learning via pac-bayes and uniform stability. In Advances in Neural Information Processing Systems, volume 34, pages 2173- 2186, 2021.
[36] L. Cella, K. Lounici, and M. Pontil. Meta representation learning with contextual linear ban- dits. arXiv preprint 2rXiv:2205.15100 2022.
[37] Kassraie, Rothfuss, and Krause. Meta-learning hypothesis spaces for sequential decision-makinn making. In International Conference on Machine Learning. 2022.
[38] B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas. Taking the Human Out of the Loop: Review Bayesian Optimization. Proceedings of the IEEE, 104(1):148-175, 2016. doi: 10. 1000//PO...211.22492222222
[39] I. Frazier. A Tutorial on Bayesian Optimization. arXiv preprint 1eXiv:1807.0285) 1, 2018.
[40] C. E. Rasmussen and C. K. I. Williams. Gaussian processes in machine learning. MIT Press, 2006.
[41] P. Auer, N. Cesa- Baaa-Bianchi, and P. Fischer. Finite-tima timee-ttmmmeee analysis of the multiarmed bandit prob- lem. Machine learning, 47(2):235-255 256, 2002.
[42] N. Srinivas, A. Krause, S. Kakade, and M. Seeger. Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design. In International Conference Machine Learning, pages 1015-1022, 07 2010.
[43] W. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4):285-299) 1933. ISSN 00063444. URL http://wwal http://ww.jjtrttooo/oo/o.o jstor. www/////////ao arg/sstablee2332266.
[44] Z. Wang and S. Jegelka. Max-value entropy search for efficient bayesian optimization. In International Conference Machine Learning, pages 3627-3635. PMLR, 2017.
[45]
Schmidhuber. Evolutionary principles in self-referential learning. On learning how learn: The meta-meta-.. hook. PhD thesis, Technische Universitaet Munchen, 1987.
[46] S. Thrun and I.. Pratt, editors. Learning to Learn. Kluwer Academic Publishers, 1998.
[47] V. Perrone, R. Jenatton, M. W. Seeger, and C. Archambeau. Scalable Hyperparameter Trans- fer Learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gareett, editors, Advances in Neural Information Processing Systems, volume 31, 2018.
[48] V. Fortuin and G. Rätsch. Deep mean functions for meta-learning in gaussian processes. arXiv preprint 19X1v::900.00000 2019.
[49] A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. P. Xing. Deep kernel learning. In International Conference Artificial Intelligence and Statistics, pages 370-378. PMLR, 2016.
[50] Т. Gneiting, F. Balabdaoui, and A. E. Raftery. Probabilistic forecasts, calibration and sharp- ness. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2):243- 268, 2007.

---

[51] T. Gneiting and A. E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-37, 2007.
[52] V. Kuleshov, N. Fenner, and S. Ermon. Accurate Uncertainties for Deep Learning Using Cali- brated Regression. In International Conference Machine Learning, 2018.
[53] M.-O. Pohle. The murphy decomposition and the calibration-resolutiin principle: new perspective on forecast evaluation. arXiv preprint arXiv:2005.01833 2020.
[54] M. Molga and C. Smutnicki. Test functions for optimization needs. Test functions for opti- mization needs, 101:48, 2005.
[55] M. Jamil and X.-S. Yang. A literature survey of benchmark functions for global optimisation problems. International Journal of Mathematical Modelling and Numerical Optimisation, 4 (2):150-194, 2013.
[56] Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan, and I. Stoica. Ray: A distributed framework for emerging ai applications. In Pro- ceedings of the 13th USENIX Conference Operating Systems Design and Implementation, OSDI' 18, page 561-577,2011
12

---

A
Safe Bayesian Optimization Algorithms
A.1 SafeOpt
SAFEOPT [9, 16] make use of the GP confidence intervals, by defining safe set discretized domain X':
S((a) = (x E X (x) + ((a)of(x) < 0}.
(11)
where we write μ (x) and of (x) short for μ (x( and o (x(xx((xx (x().). Based on the safe set, SAFEOPT constructs two additional sets. First, the set of potential optimizers
M2(a)= {x E Se(a) uf (x) o( (x) (xt) + (xa))
(12)
where xf is the best observed input so far. Second, the set of possible expanders
Ge(a) {x E (((a) g(x) > 0),
(13)
where ge (x) is defined as the additional number of inputs that become safe if we would query x and observe an hypothetical optimistic constraint value q = u (x') - (x)). ge xx X \ S.(a) 1²(X' U (x, 7)) + ((a)o"(x' D, (x, < 0}}.
(14)
In each iteration of the optimization the sets are updated, w.r.t. the posterior belief of the underlying GPs a query candidate is selected from each the set of optimizers and the set of expanders. From the set of optimizers the GP-LCB sample
opt
arg min (x(x
(15)
zEM((a)
is selected. From the set of possible expanders
arg max 9n (x) (c)
(16)
is selected. Finally, SAFEOPT selects
x"
arg max max (x)
(17)
xc oppt x
as the next query.
A.2 GoOSE
In SAFEOPT the expansion of the safe is traded off against the optimization by the uncertainty of the prediction. This can lead to part-wise exploration of the safe set without consideration of the objective function. To cope with this [10] proposes goal oriented safe exploration (GoOSE) algorithm. The idea is that the safe set is expanded only if it is also beneficial to the optimizaton of the objective. Like SAFEOPT it builds a GP model of the objective and constraints from noisy evaluations based on GP regression. The models of the constraints are used to construct two sets: First, the pessimistic safe set
=x) = X' ((( (a)))) 0},
(18)
which only contains domain points that are with high probability fulfilling the constraint. Sec- ond, we construct an optimistic safe set which includes all points we can optimistically expand the pessimistic safe set to. Rather than the version of Turchetta al. [10] which requires an a-priori known Lipschitz constant, we use the version of König et al. [18] which only uses quantities that provided by our GP model of the constraint function. For instance, the Lipschitz constant is ap- proximated by the sup-norm (x)]oo of the gradient of the GP's posterior mean. This nicely accommodates non non-stationary kernels which might are arise during meta-learning. We first make sure that we don't 't query the same point over and over by excluding all points within S. (a) whose confidence intervals for g standard deviation become narrower than a alleatoric noise threshold e. The remaining points are: We (z (f)) (z) >
(19)
13

---

Then, we check for any x E A' whether we still fulfill the safety constraint when we add the distance
times the Lipschitz constant estimate to the optimistic lower bound μ4 (x) (3(a)) of the constraint CI, formally:
(x z) = 11 [n] (x) - (x))o((( + V (x)]oo) |x z|/2 < <0]
(20)
Based on this indicator function, we can construct the optimistic safe set:
5 = (x X
3 2 E W. gx(x, 2) =
(21)
Within the optimistic safe set (a)) we now find the optimizer of (standard) BO acquisition function. We use the the UCB aquisition function [42], i.. aq(x) == pf (x) - (a)) (x), to find the next query candidate x arg millxesy(a) aq(x).
X is inside St (a.) it is evaluated. If not we query the point in W2 which is closest to x and fulfills the expander criterion (20). After querying a point and observing the corresponding f and te, the posteriors of the GPs updated and thus the sets which defined above. This repeated until X* is inside (a) and can be evaluated x" is no longer in (a and compute new query candidate.
The described procedure is summarized in Algorithm Note that in comparison to Turchetta et al. [10], König et al. [18], Algorithm not exclude points from W, that lie the periphery of the domain because checking whether point lies at the periphery is hard when working with uniformly sampled domain points instead of grid.
Algorithm 3 GoOSE algorithm
Input: Initial safe set So
Input: GP models f ~ GP(w), 9
1: for t = 1,....' T' do
2:
S% + {x E X'
(x) 3(a))(( <0}
|/ pessimistic safe set // expanders optimistic safe set // UCB candidate within optimistic safe set
3:
W, t { S
(3(a))((z) >
4:
SP(a) + fx 3z Wi: (x, =]
5:
X t arg mis aq(x)
6:
x (a. then
7:
evaluate f(x"), (x*),
8:
else
9:
X.. + arg millxew,
XEW, ||x - x 1112 (x)x =
// expand S in direction of x*
evaluate ((xx), ), g(xx))))
Return: X* <- arg me /4/ (x)
// return safe point with best posterior mean
B Meta-Learning reliable priors with F-PACOH
The F-PACOH method of Rothfuss et al. [11] uses a set datasets D1,7, Dn,I, to meta- leamn a GP prior. For that, it requires a parametric family {po E )) of GP priors (h) GP(h(x))mol (x), (x, x')). Typically the mean and kernel function of the GP are parameterized by neural networks. In addition, presumes Vanilla GP p(h) GP(((x)) J0, k(x, x')) stochas- tic process hyper-prior. In our case, the hyper-prior GP has zero-mean and SE kernel. During the meta-training the marginal log-likelihood In Z(D.,, Pe) In p(y?
X. 0) is used to fit the (h) to the meta-training data. Here we write Di,T. = (XP D y? for the matrix of function inputs and vector of targets of the respective dataset. Since we use GPs, the marginal log- likeliood can be computed in closed form as
Inp(y D 1 0) - (y--myp -mx T Kxt (y mx .) In [ el In 2 (22)
1
T 2
where Kxp,0 = Kx Ex0_0to0 12 with kernel matrix Kxp,0 = [ko(xi, xx))ii- likelihood variance 0², and mean vector mxx 00 [mo(x:), me [x(x);)]]
14

---

Algorithm F-PACOH [11]
Input: Datasets ..., P. parametric family {ee 0 E )) of priors, learning rate a
Input: Stochastic process hyper-prior with marginals p(-)
1: Initialize the parameters 0 of prior po
2: while not converged do
3: for L,.., do
4: X, = X 1.s x, where 1.5 C C X.M fid
~ ((2)
1/ Sample measurement set
5:
Estimate or compute Vn P PO) and Vok L (1
6: VoJr.
Voln 2(D,,, + 1/ +
nT. V
1
Voo )lpp(h*))]
7: 0 0 -a E VoJr.
// Update prior parameter
At the same time, during meta-training, po(h) is regularized towards the hyper-prior p(h). We can only tractably assess the stochastic processes (0(h) and p(h) in finite set of (measurement) points X ==
x E x h N through their finite marginal distributions of function values p(h*) = p(h(x)). h(xx)) and pe (e = pe (h(x),, h(xx)) respectively. In particuar, for earch task, we construct measurement sets X; = - XMM by selecting a random subset X
of the meta-training inputs X well as random points XM U(X) sampled independently and uniformly from the bounded domain X.2 In each iteration, we compute the KL-divergence be- tween the marginal distributions of the stochastic processes the sampled measurement sets. Since both stochastic processes are GPs, their finite marginals are Gaussians p(h:²) = NO Kx,) and po (h,")) = (m .. o and thus the KL-divergence available in close form. expectation, over many iteration, we effectively minimize Ex, [KL [(lxx)))(h>>>)]]]]
Overall, the loss with the functional KL. regularizer which we minimize in F-PACOH reads as:
2
+ (
(())= -
Σ
(P)
Ex. [L (h)lpo(>>)ll((x>)))
(23)
T;
nT:
marginal likelibood
lunctional KL-divereence
The stochastic minimization procedure for (23) which we have described summarized Algo- rithm
C Frontier Search
We aim to solve the constraint optimization problem
min s(z) s.t. c(z) 2 1
(24)
where s IR? IR and IR2 are monotonically increasing functions. Formally, we define the monotonicity to the partial order on R.2 Definition 1 (Monotone Function). A function lzz)) 1R? R is said to be monotone if for all (21, 22), (21, zf) R
31 S 2 22 S 2/ h( 22) S h(=),
(25)
For brevity we write S z' short for 21 S o 22 2/ and z' 21
For our algorithm, we assume knowledge of an upper bound z and lower bound z of the optimal so- lution z" such that the rectangle Z = =2/ x 122. 231 that is spanned by the bounds and contains z". Assumption 1 (Valid Search Domain). The search domain Z |=|, x |=/, =3] is valid if it contains the optinum z" arg min s(z) of the constraint optimization problem.
In Section 3.3, we have portrayed the measurement sets only the uniform domain points x for brevity of the exposition. However, implementation X; [xP X.1 ), described here in the appendix.
15

---

Due to the monotonicity of s(z) and c(z) we know that the optimal solution must lie on or imme- diately above the constraint boundary. The latter case is rather a technical detail which is due to the fact that we do not assume continuity of c. Formally we have:
Lemma (z" lies on or directly above the constraint boundary). Let z² arg min s(z) be the unique minimizer, then there exists no 22 E [2)] =2] with 1 S c(2 22) < clz").
Proof. The proof of Lemma 2 follows by reduction: Assume 3žg E [ = with 1 S cz 22) <
c(z*). Since e is monotonically increasing c(2t, 22) c(z*) implies that 22 < 25. Thus, by the monotonicity of s, we have that 22) s(z") and x² cannot he the unique constrained minimizer.
In each iteration A of Algorithm 1 we query point 2 E Z and observe the corresponding objective and constraint values s(z)) and c(zg). Due to Lemma 2 and the monotonicity of y we can rule out an entire corner of the search domain. In particular, if c(z 2 O, we can rule all points z' > z as candidates for the optimal solution and, similarly, if c(z)) < 0, we can rule out all z' S x
To keep track of all the areas of the search domain we can rule out, we construct upper and lower frontier such that all ruled-out points lie either above the upper and below the lower frontier. To construct these frontiers, we separate all queries based on whether they fulfill the constraint or not. Initially, we set Q" = }(²) and Q' = (z'), since Assumption I together with Lemma 2 imply that c(z") 2 and c(z²) S 1. Then, for every query, we add zg to Q if c(z9) 2 and to Q' otherwise.
We define the upper and lower frontiers maps 21 + 22 and 22 + 21:
F (21; 2*) min a/ z 2 z z' E 2"), FF;;;; Q*) = min {
22 2 222 z' E Q*) (26)
Ea ) = max(z/) | æ1 S o z' @'). F( Q') = max{=1 22 S 222 x'
0') (27)
For convenience, we define the sets of points that lie on the frontiers as
F²((") = fz E Z|F (22; Q*) = 21 V F2 (21: Q*) = 22
(28)
F(C')= (z E (22 Q¹) = 21 V (2 Q') = 22
(29)
If we assume Lipschitz continuity for s, we can bound how much our best solution is away from the optimum, i.c., s(z"):
Lemma 3 (Long version of Lemma 1). Lei 2² = arggmnn zi(e)) s(z) be the solution of the con-
straint optimization problem where s : 122 + IR is monotone and L Lipschitz, and e : R² + R is monotone constraint. Let
r(O', Q*) - ((E1, Z (2:: 2¹) S 22 < F (21; 2n)}
(30)
be the set of points that lie between the frontiers and â arg arg s(29) the current best solution. Then, always have that
s(2) s(z") S. L max min
||z - z'|l
(31)
e ==
d(F
where F") maxy er mil ||z zz|| is the max-min distance between the frontiers.
Proof. By Assumption 1, we know that z" E Z. Due to Lemma 2, and the construction of the upper
and lower Frontiers (i.e. F (21) < 22 = c(21, 02) 2 1 and (z)))) > 22 = c(21, (2) 2 1) we
always have that F (zf) S z S (zt), ) that is, z² E r. Due to the monotonicity of s, we have
zz E F" that
s(2) - s(z²) = (s(2) - s(2)) -ss(2") - sZ))
(32)
50
S s(z) s(z")
(33)
< L z zz| ||
(34)
16

---

where the last step follows from the Lipschitz property of s. Finally, as z' E I we have take the maximum over z' E I so that the bound holds in the worst case. However, at the same time, we can take the minimum over z E F since (34) holds for all points 2 on the upper frontier. Both steps yield the final result
s(z) s==s((>) S L max min z -zzzl-
(35)
x'el ZEJ
which concludes the proof.
The key insight of Lemma is that we can bound the sub-optimality =2)) -z(²)) with the maximum
distance of any point between the frontiers from its closest point on the upper frontier instead of z. This is the case because å dominates any point on the upper frontier (i.e., s(2) S s(z') Vz F"). In fact, we can further reduce the set of points which we have to consider for computing the max-min distance d(T, F") to the outer corner points of T, as defined in the following:
Definition 2 (Outer Corner Points of T). Let b' (F1() 2)) be the left upper and by ( F((2))) the right outer corner points of r. Let Q O' U (b', b,} and (0...... its ordering such that 21,;, S 21,441- We define
I oo ( =
2,.,, 1 23 U (b',
(36)
as the the outer corner points of
This allows to compute the max-min distance d") by maximizing over finite set of outer corner points, instead of the whole inter-frontier area P:
Lemma 4 (Version of the max-min distance that is easier to compute). Let I ou be the outer corner points of L, as defined in (40). Then, we have that
d(I,F") = max min ||z z'|l
(37)
=E Z'E
Proof. Due the construction of the frontiers, for every uplle (z,z') I x F that the optimal
solution of the max-min problem in (31) there must be (ž,z') E Tl out x Fu that is the optimal solution of the max-min problem on the RHS of (37). Again, we can show this by reduction:
Assume this is not the case, and z E r rV then, by the geometrical properties of I there exists
2 E T which we can obtain by reducing 21 or 22 of z = (#1, 22), such that || z'|1 > ||z z' Thus (z, zz) cannot the optimal solution of the max-min problem.
As we only have to consider a finite set of outer corners, instead of the whole r, this makes the max-min distance easier to implement in practice.
Since the max-min distance bounds how sub-optimal our current best solution can be, we want to choose the next query so that we can shrink the max-min distance d(T', F²)) between the frontiers. For this purpose, we find a rectangle
R2 {ž Z min 2/1 & S max(21 21, 2f} A min(22 2) s 2e S max 22, 2611 (38)
within that has the largest max-min distance
dr (T, d(T n n F"
F n
Rux)
(39)
To make finding this rectangle tractable, we narrow down the points on the upper frontier which we have to consider to its corner points, defined in the following: Definition (Boundary Points of Upper Frontier). Let Z [=], Then
211 x [=/ 23} be the search domain.
by = (z"', min (22((2), 22) E F))). (minf (2), 22) 2
the points of the upper frontier that intersect the vertical and horizontal domain boundary.
17

---

Definition 4 (Corner Points of F"). Let (21 ((u1+22) the ordering of Q" U (b., bj} such that 21,6 S 21,44 and 2 2 22,1+1- Then
F (2") [(2 Fx.2) li= 2, [ U
(40)
are the corner points of the upper frontier F
Now, we consider all outer corners of the inter frontier area 1 (o)) and corners of the upper frontier For Q")) find the largest max-min rectangle:
Definition 5 (Largest Max-Min Rectangle). The largest max-min rectangle, defined by its corner
points (z. z') E I x Foor the rectangle in I' with the largest max-min distance such that it fulfills
condition (3), formally:
LARGESMAXMINRECTQQ Q²)= (2, (2)x)
arg max
dR,, (T. s.t. (2)
rectangle lies in T, ie. (21, ET (21, E
(2) rectangle has a positive area, iee. |21 =11 122 2 >0
(3) we cannot obtain a smaller rectangle (z, 2) with a upper frontier evaluation & E Q" that matches (z, in one side of the rectangle but has a smaller second side, i.e.
-32 E @ (21 < &σ < o A2 = i2) (22 < in 2/ A = 21)
(41)
Given the largest max-min rectangle Rz,x', want choose the next query point that reduce the max-min distance dR,,, (T F") within this rectangle as efficiently possible. We consider the set of query candidates
2n (21/2+ + x 22/2 2 2/2 + 2212) 2// (2)
(42)
middle right side
middle of upper side
consisting of the center point, together with the middle points of its right/upper sides. From this query set, we choose the candidate that minimizes the worst-case max-min distance. In particular, query point 2 there are two possible scenarios that will affect the max-min distance differently: either the point satisfies the constraint (c(29) 1) it does not (c(x9) < 1). We compute the rectangle's 's max-min distance for both scenarios and choose the query-point that gives us the lowest max-min distance in the less-favorable (worst-case) scenario:
BESTWORSTCASEEUERRY z', Q¹, Q4)
arg min 2, 6 (..e..//(FFFFF
max dR.,x (T(@' U (*),
(43)
dR.,, Q" Uz')
one of the query candidates already lies on one of the frontiers, it cannot expand the frontiers and thus also not improve the worst-case distance. Hence, we directly exclude such query candidates by removing them from the argmin in (43).
Theorem (Appendix version of Theorem 1). Under the assumptions of Lemma 3 the Algorithm, needs more than k < 3 [loge(1/e) O ((1/c)]1.59) iterations get
s(Z)) s(z") ) & -||z"=z''|1 (1/e)
-z
(44)
close to the optimal solution.
Proof. In the worst case, requires Algorithm no more than three queries to half the max-min distance within max-min rectangle. The process of doing so maximally segments the inter frontier area I' within the rectangle into three max-min -in rectangles. This can be checked by going though the possible cases of how max-min rectangle is split by Algorithm 1. When a max-min rectangle
18

---

Case 1
Case 2
Case 3
Case 4
Case 5
iter
1
iter
iter
max-min rect.
upper front.
lower front.
Figure 5: All possible cases how Algorithm halfs the max min distance within a max-min rectangle when only the upper right corner of the rectangle belongs the upper frontier.
split up in three iterations, there are, in principle, 2 cases since each query either lie above below the constraint boundary. However, when the first query fulfills the constraint, it immediately halfs the max-min distance. Thus no further queries are necessary and we effectively only have to consider cases. Fig. 5 illustrates these five split-up cases for max-min rectangle where only the upper right corner belongs to F=, Similarly, Fig. 6 and illustrate the split-up cases when either half or the entire upper side of the rectangle belongs to F=. The cases when half or entire right side of the rectangle belong to Fu are analogous. two further queries are necessary half the max-min distance. Finally, the case where both the upper and the right side of the rectangle belong half full F is trivial, it only requires one more query. Thus, to shrink the max-min distance of all max-min rectangles by a factor of n, need l < 3 [1og2(n)] iterations at most. If we set 1/e it follows by Lemma that we need k < (1/e)) to obtain d(I', < 2|"" - z z/e/e. Finally, the result in the theorem follows from combining this with Lemma 3.
these scenarios, only
D Implementation Details of SaMBO
Calibration & Sharpness based Frontier Search. To compute the calibration (5) and sharpness (6) across multiple datasets efficiently, we distribute the computation across individual processes for each datasets, using the ray distributed computing framework [56]. A model that always gives valid confidence intervals should always be calibrated, matter what is the ordering of the datasets D, which split into train sets Di,tt and test sets Di,e to compute (5). Based on this principle, we compute avg-std and calib based on the given ordering as well as the reversed ordering of each dataset, and average both results. In principle, more permutations can be done, but come with additional computational cost.
We perform frontier search search for the kernel lengthscale and variance parameter for the safety constraint calib( 1 Vg) 2 (i.e. calib 1 since calib [0, For the
both the target and constraint function model. For the constraint function model,
19

---

Case
Case 2
Case 3
Case 4
Case 5
1
iter
0
1
iter
max-min upper front.
lower front.
Figure 6: All possible cases how Algorithm halfs the max- max-min distance within a max-min min rectangle when the upper side of the rectangle belongs to the upper frontier.
Case 1
Case 2
Case 3
Case 4
Case 5
1
iter
0
1
iter
0
0
max min rect.
frant.
lower front.
Figure 7: All possible cases how Algorithm halfs the max-min distance within max-min rectangle when half the upper side of the rectangle belongs to the upper frontier.
target function model, the calibration requirements are safety related and thus as strict in case of the constraint function. Hence, only calibration frequency constraint calib( (abb((D/,))) /n (e vy) 2 0.95 of 95 % when performing frontier search on ly and vy. For both models, we run frontier search for 20 iterations as this gives already good solution without incurring to much computational cost. For the variances, we us an lower and upper boundary of 1.0 and 6.0 whereas for the lengthscale we consider the range [0.01, 5.0]. The frontier search is per- formed in the log-space to the base of both variance and lengthscale, i.e., we transform the both variance and lengthscale with t(2) log, 0(z) during the frontier search and in the end transform back the result by (=) 10
Parallelizing the sharpness and calibration computations. Computing the calibration and sharp- ness metrics (5) and (6) is computationally expensive since it requires computing the calib-freq and atg over all n tasks and T; subsequences within each task. Fortunately, each these computations can done independently. Thus, we use compute each of the summands in (5) and (6) in parallel processes and aggregate the results in the end. This significantly reduces the compu- tation time for the calibration and sharpness which allows to perform 20 steps of frontier search relatively fast.
20

---

The NN-based GP prior. Following [49, 11], we parameterize the GP prior Po(h) GP (kima (o) (x x'),, particularly the mean me and kernel function kig as neural networks (NN). Here, the parameter vector @ corresponds to the weights and biases of the NN. To ensure the positive-defiiteness of the kernel, we use the neural network as feature map $x(x) : X + Re that maps a d-dimensional real-values feature space in which we apply squared exponential kernel. We choose the dimensionality of the feature space the same the domain. Accordingly, the parametric kernel reads as
(2r x²) = up exp (- (x
(45)
Both mx(x) and (x(x) are fully-connected neural networks with 3 layers with each 32 neurons and tanh non-linearities. The kernel variance up and lengthscale Ip also learnable parameters which appended to the NN parameters 0. Since lp and of need to be positive, represent and optimize them in log- Unlike Rothfuss et al. [11], we set the variance r² of the Gaussian likelihood (x)) = (Ny; o²) as a fixed parameter rather than meta-learning it. This is, because, in order to ensure safety, the likelihood variance needs to be the same as the one that was used during frontier search, when computing then calibration and sharpness.
The hyper-prior. We Vanilla GP GP(O, k( x²)) as hyper- prrir-proorroor stochastic process. In that, k(æ, x²) = V/ exp (-1|= (111/((())
(46)
a SE kernel with variance and lengthscale chosen by the frontier search procedure, discussed in Sec. 4.2.
Minimization of the F-PACOH objective. To estimate the F-PACOH objective in (23), we use measurement sets of size & = 20, i.e., considering subset of 10 training points and 10 uniformly sampled domain points per iteration. We minimize the loss by performing 5000 iterations with the Adam optimizer with learning rate of 0.001.
Code and Data. We provide implementations of the SaMBO components as well as the experi- ment scripts and data under ptttps:///tnyurrl// cwmm mamet//ae.mmm
E Further discussions about SaMBO
Discussion on the applicahility to higher-dimensional problems. Similar to other BO/ / safe BO methods that are based on GPs, the sample complexity grows exponentially with the number of dimensions. The meta-learning in our case alleviates this issue to some degree by making the GP prior more informed about our environment tasks in areas where meta-training data is available. However, to maintain safety, we regularize the leareee GP prior towards a Vanilla GP with SE kernel (ef. Eq. 3) in areas without little meta-training data. The higher-dimensional our safe BO problem, the sparser is the meta-training data in the domain. Thus, the areas where our meta-learned GP resembles Vanilla GP become larger in proportion and we face again the general issue of poor sample complexity in high dimensions. Generally, without additional assumption that, e.g., there exists some lower-dimensional sub-space in which the functions we are optimizing lie, we do not believe that the 'curse dimensionality" problem can be solved while, at the same time, assuring safety. Generally, we not aware of any safe BO method that has been successfully applied to a high-dimensional optimization problem with safety constraint.
Discussion the monotonicity of the calibration-shappaaa constraint optimization problem. As discussed in Section 4.2, the introduced frontier search algorithm aims to exploit the mono- tonicity properties of the clibrattion-saappesss constraint optimization problem in 7. For large ranges of and v, the avg-std and avg-calib are monotonically increasing in the kernel variance and decreasing in the lengthscale While the monotonicity for the avg-std provably holds across the spectrum, the monotonicity of the calibation frequency in is only an empirical heuristic that holds in almost all cases if // is at least big as the variance of the targets y in a dataset. If the kernel variance is chosen smaller than the variance of the data, significant parts of the function
21

---

that underlie the data are outside of the high-probabiln probability regions of the GP prior and the GPs predictive distribution systematically underestimates the corresponding variance in the absence of close-by data points. such cases, larger lengthseale can actually improve the calibration by increasing the sphere of local influence of data points on the predictive distribution and, thus, slowing down the reversion of the posterior towards the prior with too small variance.
Calibration frequency
3.0
vertence (con-rriin)
-0999
2.0
0.97
1.0
0.96
0.5
0.35
Figure 8 displays the calibration frequeney in response to 1 and in the extended range where the kernel variance becomes smaller than (data is standardized) and thus smaller than the variance of the data. The lower left corner where v < illustrates the described breakdown monotonicity.
0.5
1.0
3.5
kernel longhhocale (constroint)
setting Figure 2, but with
Figure 8: varying kernel lengthscales / and vari- ances v. The data underlying the cal- ibration computations has been stan- dardized. While avg-calib is monotone in / for v 1, the monotonicity breaks down if the kernel variance is smaller than the variance of the data.
Calibration frequency for
In practice, this not an issue since we can easily we standardize data and then choose the lower bound of kernel variance in the frontier search to be 1. In the re- sulting search space, our constraint optimization problem is monotone and the frontier search converges expected.
Experiment Details
F.1
SafeBO environments
F111
The Camelback Random Sinusoids environment
The environment objective/target function corresponds to Camelback function [54]
g( .2) max (-(4 2.1 of + x1/3) of x 2 - (4 x - 4) 222
- -2.5)
(47)
plus random sinusoid functions, defined over the 2-dimensional cube X' 1-2 x [-1, as
domain. Specifically, the target function is defined as
f(1 g x2) + sin(w, (x1 p)) sin(wy (x2 p))
(48)
wherein the parameters are sampled independently as
(.0..,..5). W/ ((0.2, 2.0). p N.0,
(49)
The constraint function q(x) is linear combination of the camelback function g(x), a product of sinusoids along the two dimensions and quadratic component that ensures that the safe regions sufficiently connected so that they can be reached:
q x2) = 3 s0.4 म Wy 2) sin(2m w.) b. (1 + x2)) + 1.2 ((11, - 0.7 (50)
Here the parameters wig and are sampled follows:
Wq ~ (0.44,.0.5) ~ U(0.3, 0.5)
(51)
The optimization domain is X 2- [-,, 1]. As initial safe point we use So {(-1.5, -..5) Figure 9 displays an example task of the Camelback + Random Sinusoids en- vironment.
F.1.2 The Random Eggholder environment
The random Eggolder environemnt is based on the Eggholder function [55], popular benchmark function used in global optimization:
[ 42) = -xx2 + e) sin (Vlarz 21/2 + 471) - b. x1 sin (Vluz x22 477) (52)
22

---

Larget N2)
constraint 4iX,,
1.00
1.00
Giobal optimum
0.50
1.8
0.50
0.55
1.2
0.25
-0.25
0.0
-0.5
-0.50
-0.6
-050
-0.75
-1.2
-0.75
-.0
-15
-1.0
0.0
0.5
2.0
2.5
Figure 9: Example task of the Camelback Random Sinusoids environment. Left: target function. Right: Safe regions in green and unsafe regions in red.
target function fin xx)
constraint function X s
400
800
400
Global optimum
350
350
Initial safe point
230
250
200
200
-200
400
50
50-
-1000
0
100
200
300
400
50
100
150
200
250
300
350
400
Figure 10: Example task of the Random Eggholder environment. Left: target function. Right: Safe regions in green and unsafe regions in red.
We randomly sample its parameters a, c as follows:
~ L(066 1.4). b~ U(066, 1.4), e~ (44,55))
(53)
to obtain different tasks. The corresponding constraint function is defined
( x2) 300 V . + 2n3 s0 ((w).) + w2#22/20)
(54)
where the frequencies are sampled independently as W1, W2 ~ (4(.88 1.2). The optimization domain X' = [0, 400) x [0, 400]. As initial safe point we use Sg = {(880, 50)). Figure 10 displays an example task of the Random Eggholder environment. The environment particularly challenging since the Eggholder function has many local minima.
F11. Tuning controller parameters for the Argus linear robotic platform
The high-precision motion system Argus from Schneeberger Linear Technology is shown on Fig- 11). It is a 3 axes positioning system with orthogonal linear axes and rotational axes on top. The system has an accuracy of =10pm, bidirectional repeatability of 0.7pm and 30 position stability of < 1nm. In our experiments we focus on the upper linear axis of the system.
The exact optimization problem
d T. peltmoove tead] dt
arg min x= [KKP VKP, VKI)
(55)
s.t.
FFT max (x)
max wE|0.03,0.07]
[fft [fft(ve))(w) + max [fft(v)](ww) - -x < 0 00(0.08.011
(56)
23

---

Figure 11: Argus motion system
Feadoorwrdd
Feedforward
Peel
Position Controlior C1
Yan
Controllar C2
Velceity Pl
Plant G
Figure 12: Argus controller structure
where PKP is the proportional gain of the position control loop, VKP is the proportional gain of the velocity control loop, VKI is the integral gain of the velocity control loop, T, is the settling time, pe is the position eтror measurement, ve is the velocity eгror measurement, move is the move- time defined as the timepoint where the reference movement is finished, fend is the time endpoint of the movement measurement, set to I 1.2s, w the frequency, fft the fast Fourier transform given frequency window, b is a scaling factor, and the constraint limit, dependent on the reference stepsize. The position reference trajectory step-wise constant jerk based s-curve with jerk 200 3. maximum acceleration
20 ga and maximum velocity 1
The simulation model of the Argus used in section 6.3 contains cascade controller and model of the Argus system (plant). The cascaded controller (see figure 12) was rebuilt from the real controller design and the plant (see figure 13) was modelled by fitted linear transfer function G(s), con- 12mnst22m//aaa
taining a double integrator, the five most dominant resonances of form T(s) 22 22/war+ + /wart1 and dead time, using frequency domain data and 2) nonlinear position-dependee model (p), based linear interpolation of lookup table for 16000 points at positions between 2200mm of the axis using discretization of 0.025mm, where p is position, is velocity, and F, is the force reference (proportional to current) applied to the motor of the axis. The linear transfer function has order 13. The resonant and anti-resonant frequencies are displayed in table 1, with
cogging
Sn/d Wn/d V n/ and dead time /dead 2ms
G
G(s)
Figure 13: Argus controller structure

---

fn Hz
An
in Hz
390
0.1
400
0.1
475
0.03
500
0.05
690
0,03
800
0.06
870
0.03
900
0.04
1050
0.03
1100
0.06
Table 1: Argus simulation linear transfer function parameters
Environment
# meta-train tasks
# points per task
lengthscale constr model
Camelback Random Sin
40
100
0.5
Random Eggolder
40
200
0.4
Argus Controller Tuning
20
400
0.4
Table 2: Specifications of the meta-train data
The optimization domain is 3-dimensional and corresponds to the three controller gains PKP, VKP, VKI, restricted to the following ranges: = [100, 400] x [300, 1200] x [500, 4000). As initial safe set, we take So = {(200, 800, 1000)}.
F.2 Environment Normalization and Data Collection
Environment Normalization. The three environments, specified in Appx F.1 have vastly different scales, both in x, f and q. To alleviate the problems arising from different value ranges we stan- dardize the environment data such that the value ranges are roughly those of a standard normally distributed random variable, before we pass on the data the GP model. Hence, the kernel variances and lengthscales displayed in Fig. 6.2 correspond to the standardized value ranges.
To determine the standardization statistics (i.e., mean and standard deviation (std)) for x we use the domain ranges ((4,, x for dimension i 1,..., d) of the environments. Assuming a uniform
distribution over the domain, we use μs, +f mean and σz, = conservative, because do not want underestimate their std which could lead to getting stuck in local optima or safety violations. In particular, we consider the respective minimum and maximum values of f and observed in the union of the meta-train datasets, i.e. fimin, frax, aminn gmmax. For f, we use H pais pmin and oj
For f and
prss - For the constraint values, we set max (17""(_11""" as std
Mq = O because we do not want to distort the safety threshold and use og to re-scale the constraint values.
2
Collection of meta-train data. We collect the meta-training data running SAFEOPT on each task. Table 2 holds the specifications for each of the three environments, i.c. the number of tasks n, the number of points/iterations T; per task and the constraint model lengthscale that is used for SAFEOPT. In all cases, we use a conservative lengthscale of 0.2 for the target model and a likelihood std 0. Conservative parameter choices like these ensure that we continue exploring (within the confines of SaFEOPT) throughout the data collection and don not start to over-exploit, e.g. by querying the same data point many times.
F.3 Parameter Choices for the Experiments
Domain discretization. Since both SAFEOPT and GOOSE require finite domains, we discretize the continuous domains of the safe BO tasks with 40000 uniform points each.
The standard deviation of the Gaussian likelihood of the GP is the only hyper- parameter have to choose when employing proposed frontier search in combination with Vanilla GPs. However, the observation noise is often known or easy estimate by querying the
Likelihood Std.
25

---

10
5
0
-5
-10
15
20
Figure 14: Monotone optimization problem in Eq. 57
same point multiple times and observing the variation of responses. We use the latter approach to determine the likelihood std in experiments, obtaining the following settings: 0.02 for the Camelback Random Sin environment,
0.05 for the Random Eggholder environment and 0.1 for the Argus Controller parameter tuning.
GoOSE expander sets. For computing the GoOSE expander sets W,, we always use = 0.2. Note, that this applies after the environment standardization.
G Further Experiment Results
G.1 Frontier Search
In this section, we aim to investigate how fast the proposed Frontier Search algorithm (c.f. Algorithm 1) approaches the optimal solution montone optimization problem like the one in (24). To this end, we use the following monotone, constraint optimization problem to test the algorithm:
min s(z) c(z)
with s(z) = + 2.²
(57)
((z) == 5 + 0.5 * 23 33 3
Figure 14 visualizes the s(z) and the constraint boundary g(z) = 0. In Figure 15 we visualize the first 33 iterations of Frontier Search on the optimization problem in (57). As we can see, Algorithm quickly shrinks the area between the fronticrs and returns solutions close to the optimum after only a handful of queries.
Finally, Figure 16 visualizes the convergence of Frontier Search, both in terms of the max-min distance and the sub-optimality s(²*) s(2") ) of the solution, together with the bounds provided in Theorem I.
G.2 Compute Time Analysis
Here, we provide a computational analysis of various components of the proposed SaMBO algo- rithm. Table 3 reports the time takes finish the computations associated with the frontier search well F-PACOH for different sizes of the meta-training data. Each experiment has been 16 processor cores of an Intel Xcon 8360Y 2.4GHz CPU. As in all the experiments, we use 20 itera- tions of frontier search and 5000 iterations of stochastic gradient descent (with the Adam optimized) on the F-PACOH objective.
As already discussed in Appendix D, we can use parallelization for the computation of the avg-calib and avg-std metric which makes frontier search relatively fast. For instance, for 20 tasks and T 200 samples, the frontier search part of Algorithm 2 finishes in ca. 35 seconds. The compu- tational complexity of the calibration and sharpness computations grows with T' since the number
26

---

iter
iter 3
iter
iter 9
-5
-5
-5
-10
0
-10
-10
-10
iter 15
iter 21
5
0
-5
-5
-5
5
iter 24
iter 27
Iter 30
iter 33
5
0
-10
-10
-10
-10
upper
front.
X
solution
optimum
safety border
Figure 15: Frontier Search on the monotone optimization problem in in Eq. 57. Algorithm quickly shrinks the between the frontiers and returns solutions close the optimum after only handful of queries.
101
Sub-optimallty
max-min
max-min dist bound
max-min dist
sub-optimality bound
sub optimality
100
10-1
0
40
60
20
60
Figure 16: Convergence of Frontier Search for the optimization problem in Eq. 57.
27

---

T
Duration Frontier Search
Duration F-PACOH
10
14.19 + 0.52
169.55 1.99
10
100
18.83 = 0.62
183.55 2.21
10
200
33.82 = 1.26
230.90 4.57
10
400
75.23 + 10.42
247.89 + 11.05
20
50
13.96 = 0.87
178.47 + 4.93
20
100
22.14 = 0.68
187.57 = 1.84
20
200
37.44 + 2.54
224.54 8.75
20
400
95.15 + 11.65
243.12 8.94
Table 3: Compute time in seconds for Frontier Search and F-PACOH for different number of tasks and samples per task T.. Reported is the mean and standard deviation over 5 sees / repetitions.
of sub-sequences of each dataset, as well the maximum size of sub-sequence used for GP in- ference grows. While the former be parallelized, GP inference cannot be easily parallelized. The meta-training of the GP priors, typically takes ca. 3-4 minutes. Finally, the compute time for the safe BO part strongly varies depending and how time intensive/costly it is to query the f and q. In the best case, when we use simulated toy functions that are very cheap to evaluate and run GoOSE for 200 steps, the safe BO takes ca. 500 seconds (8-9 min). case of SafcOpt, the compute time is highly variable depending on the expander computation and ranges from 15 minutes 1.5 hours. Generally, when employed with real system expensive simulation, the time for the safe BO outweights the compute time for the frontier search and F-PACOH steps of SaMBO by order of magnitude.
the employed algorithm (e.g. GoOSE SafeOpt)
G.3 Study on amount of meta-train data
In this section, we aim to study how varying amounts of meta-training data affect the overall per- formance of SaMBO. For this purpose, we employ SaMBO-G on the the Camelback Function Random Sinusoid environment with a varying number of meta-training tasks and data points per task. Figure 17 displays the terminal inference regret at (T 100) for a varying number of tasks with each 100 data points (left) and varying number of tasks with each 100 data points (right).
In all cases, the performance of SaMBO-G is substantially better than the inference regret of > 0.1 we obtain when running GoOSE with a Vamilla GP, as done in Figure 4. This demonstrates that we can also successfully perform positive transfer when the amount of meta-training data is smaller. Finally, we observe that the performance improvement we gain by doubling the number of tasks is much bigger than when double the number of points per task. This is consistent with learning theory in e.g. Pentina and Lampert [30] and Rothfuss et al. [34].
28

---

Inference regret at T==100
0.0175
inference regret at T=100
0.020
0.0150
0.0125
0.015
0.0100
0.010
0.0075
0.0050
0.005
0.0025
0.0000
0.000
50
200
20 number of tasks
number points per task
inference regret at last step (T' 100) of SaMBO-G in the Camelback Function + Random Sinusoid environ- ment. Left: Varying number data points per task m for n = 20 tasks. Right: varying number of tasks with each 100 data points. Doubling the number of tasks leads to more performance improvements than doubling the number of points per tasks.
Figure 17:
Overall performance for different amounts of meta-training. Displayed
29


Reviews:
[{"review_id": "ewkzEQiXuF", "reviewer": "Reviewer_iFMD", "summary": "The authors present a method for meta-learning priors for safe bayesian optimization (BO) from data. The authors contribute a method for finding safety-compliant kernel hyper-parameters by developing an optimization problem which maximizes sharpness while maintaining calibration at all times. By exploiting monotonicity properties of the optimization problem, the authors develop an efficient search algorithm, “Frontier Search” to converge to optimal hyper-parameters. They contribute a safe BO algorithm “SaMBO”, which uses the frontier search in combination with F-PACOH and existing safe BO methods. The authors show that their method requires fewer iterations to converge as compared with data-free methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Section 3 problem statement and background is informative and clear. It introduces the background thoroughly and in an accessible way.\n- The methods and contributions are well presented.\n- The contribution is useful, and as far as I can tell, novel. The proposed method has the potential to significantly reduce computation time for safe BO without significantly complicating implementation. \n- Results (Fig 3, Fig 4) show clear improvement in number of iterations required in BO as compared with baseline, data-free, methods.\n\nWeaknesses\n- If I am understanding the examples correctly, you only compare SaMBO-G/S to the data-free baselines. It would be important to compare SaMBO to methods which can use the calibration data which SaMBO uses. \n- The authors make safe BO more efficient by spending computation time on the Frontier Search. It’s not immediately clear that the overall result is more efficient. Nor are there any results which discuss or compare overall efficiency (in terms of e.g. wall-clock time of the entire BO algorithm including the Frontier Search).\n- Some assumptions are not entirely clear or well justified. E.g. the authors say that safe BO methods assume that correct kernel hyper-parameters are given (L52). However, their assumption that upper and lower bounds on the correct hyper-parameters (L167) is not well justified.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The challenge of hyper-parameter selection is ubiquitous and the authors present a practical, data-based, method for selecting kernel hyper-parameters. When combined with existing safe BO methods, the authors show their method requires fewer iterations to converge when compared with data-free methods. However, it is not entirely clear that the method has better computation time (i.e. in terms of wall-clock time) since no results for this are presented. I would also like to see a fairer comparison (i.e. to methods which also use data for calibration) such as a hand-tuned baseline or naive approach. However, as far as I am aware, no other method for this kind of safe kernel hyper-parameter selection exists. The methods are convincing and well presented. Therefore, I recommend weak acceptance.\n\n---\n\n**Update**: After reading the replies, the authors have clarified or improved upon all major issues and weaknesses. I have increased my recommendation to strong accept. ", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "Content issues/questions: \n- Related work: I believe there are a number of related works in the area of PAC-bayes meta-learning regularization which could be added. E.g.: Pentane & Lampert NeurIPS 2015, Amit & Mier ICML 2018, Balaji et al. NeurIPS, 2018, Yin et al. ICLR 2020, Farid & Majumdar NeurIPS 2022. I think meta-regularization would be fair to mention as it relates to both uncertainty estimation and PAC-Bayes meta-learning which are already discussed in the related work.\n\n- L137 What exactly about eq. (5) and (6) are different from [47, 48, 49, 50]? If the authors reference [47-50] purely to refer the reader to information about calibration and sharpness, it would be more clear to say “see e.g. [47, 48, 49, 50]”.\n\n- L167: The assumption that the optimal (and even safe) kernel hyper-parameters have known upper and lower bounds is a key assumption used to prove Lemma 1 and Theorem 1. However, I do not see this assumption as well motivated/argued. Why can we assume that we known a region of safe hyper-parameter? If we choose a very large range to ensure the box includes the optimal solution, is that damaging to the sample efficiency of the Frontier Search?\n\n- Lemma 1, Theorem 2: I would appreciate a proof outline/intuition in the body of the paper. At least add a reference to the proofs in the appendix so the reader can find them more easily.\n\n- Section 6 Experiments: I think it is important to report computation of the proposed method SaMBO in comparison to methods which do not use the frontier search. The authors mention that Equations (6) and (7) are expensive. Since this is additional computation not usually performed in Safe BO methods, it is not clear to me that one is preferable in terms of total processing time. \n\n- L253: It is interesting that the method works with non-i.i.d. within-task data. Is there a significant improvement in performance if the method is provided with i.i.d data? If there is not a significant difference, this would be a major positive of the approach. \n\n- L265: Do the authors have any intuition as to why log-space of l and v work better? It would be interesting to see a comparison of log and linear search. \n\n- Figure 3: Which hyper-parameter selection do the authors use for GoOSE/SafeOpt? Can the hyper-parameters be tuned in any (potentially naive) way by manually looking at the reference data you give to SaMBO-G/S to provide a fairer comparison. \n\n- L302 “our framework makes safe BO free of hyper-parameters”. I believe $z_1^l, z_1^u, z_2^l, z_2^u$ (L167-L168) should be considered hyper-parameters in addition to the standard deviation of the Gaussian likelihood of the GP (L751). I agree that the proposed method is more practical in general, but it is not correct to say that SaMBO is free of hyper-parameters. \n\n- L302-L305: I believe the authors could expand on the discussion of the limitations. Can discuss possible time to compute Equations (5) and (6), are there cases where (5) and (6) are intractable? Similarly, are there cases where specifying l and v are straightforward, and therefore the frontier search methods is not necessary? Also, no longer needing to specify l and v moves the problem of hyper-parameter specification for Safe BO to other parameters — are there cases in which these other parameters are just as difficult to specify? \n\nEditorial issues: \n- Handful of minor typos: L10 and L37: “data-riven” -> “data-driven”, L51: “GoOSE by expanding” -> “GoOSE explores by expanding”, L234 “fist” -> “first \n- I am not certain the authors are using “c.f.” correctly. E.g. in L169 “Since both s(z) and c(z) are monotone we know that z* must lie on or directly above the constraint boundary c(z) = 1 (c.f. Lemma 2).”. I would interpret “(c.f. Lemma 2)” as “compared to Lemma 2”, but it seems that the authors mean “see Lemma 2”. Indeed, Lemma 2 states that “z* lies on or directly above the constraint boundary”. I believe the authors make this mistake on L194 and L762 as well."}}, {"review_id": "aEJs0NmmyLlM", "reviewer": "Reviewer_jWPh", "summary": "This paper presents a meta-learning framework to optimize controllers with safety constraints with bayesian optimization. To this end, the authors build on recent work for tuning gaussian process parameters and kernels with meta-learning. The authors test this approach on several bayesian optimization benchmarks and for tuning a controller for a linear robot system. The results demonstrate how optimization with meta-learning priors converges faster than previous paradigms while satisfying safety constraints.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper offers a strong technical analysis of different meta-learning paradigms for gaussian-processes\n2. The paper runs thorough experiments to demonstrate the capabilities of this approach on several benchmarks.\n\nWeaknesses:\n\n1. The limitations section of this approach does not discuss algorithmic complexity or implications when applied to larger optimization problems.\n2. Theorems and Lemmas are presented but not proven in the main paper, which forces the reader to look over appendices.\n3. Several acronyms are used repeatedly over the paper and not explicitly defined. The presentation could be improved.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper offers an interesting contribution and is written in a fashion that walks the unfamiliar reader through the key concepts that build this algorithm. The results are impressive and seem like a helpful contribution to the field of meta-learning for robotics. I recommend acceptance, but the authors can significantly improve the paper by improving the presentation and clarifying on the limitations of the approach.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "2: The reviewer is willing to defend the evaluation, but it is quite likely that the reviewer did not understand central parts of the paper"}, "extra_text": {"issues": "Section 1.\n- F-PACOH is an acronym that is very important to this paper, yet the authors do not describe its meaning explicitly.\nSection 4.2.\n- The frontier search is hard to follow over long paragraphs and can, at times, be confusing to the reader. \n- the functions LARGESTMAXMINRECT() and BESTWORSTCASEQUERY() are not defined in the paper, is up to the reader to infer their meaning.\n- It would be much more helpful if the authors described the functions LARGESTMAXMINRECT() and BESTWORSTCASEQUERY() under an equation or a separate algorithm environment.\n- If Theorem 1 is not proven in the main paper it is very hard to assess its validity or impact. \nSection 6.1.\n- Why is regret the best evaluation metric? It would be helpful to comment on this\nSection 7.\n- What are the limitations of this method when applied to larger problems?\n- What kind of BO problems are hard to represent with the F-PACOH modeling?\n- What are future lines of work where this algorithm could be impactful?"}}, {"review_id": "VzWRDHBXFmQ", "reviewer": "Reviewer_wnHy", "summary": "This paper presents a meta-learning method for model selection in the safe BO setting. The method selects the hyperparameters of the target and constraint GP models such that the BO procedure respects constraints while avoiding too conservative hyperparameters.\n\nThis is achieved by the maximizing calibration and sharpness of the uncertainty estimates over a set of datasets collected from related tasks.\n\nThe best hyperparameters are employed in a meta-learning procedure (F-PACOH) to learn informative and safe priors for the GP models used in BO with the goal of improving efficiency. This approach mitigates the risk of overconfident priors when data are scarce.\nOnce the GP priors are meta-learned, they can be employed for efficient Safe BO. \n\nThe approach demonstrates improved query efficiency with respect to baselines and respects constraints. Experiments are carried out in simulation in low-dimensions (2 benchmark functions and a PID tuning problem).\n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The problem of selecting hyperparameters for safe BO so as to ensure constraint compliance while at the same time avoiding excessively conservative exploration is a highly relevant one, since it can hamper actual applications of Safe BO to parameter tuning for safety-critical and costly-to-operate systems. This paper represents a significant step forward to tackle this problem, providing means to reliably estimate hyperparameters and informative GP priors in the case in which related task data are available.\n\nThe paper elegantly and coherently brings together tools and methods from model calibration and meta-learning, presenting them in a coherent way, and proposes a novel frontier search algorithm to select safe hyperparameters for GP priors, leveraging monotonicity of the calibration frequency (heuristically) and sharpness measures.\n\nFrom the technical point of view, the presented method seems sound. The improvement in terms of query efficiency is shown both experimentally and in Theorem 1. \n\nThe frontier search algorithm assumes monotonicity of the calibration frequency in the kernel length scale, which the authors state is a reasonable empirical heuristic in most cases. Still, it would be informative to elaborate more on possible edge cases and counterexamples.\n\nA more in-depth discussion on the applicability of the method in real-world scenarios would also be beneficial. In particular, is it common to have access to offline data from similar tasks on a real system like a PID-controlled robot? How can we assess if the target task can be considered to belong to the same meta-distribution as the previously-collected data?\n\nExperiments are carried out in low-dimensional parameter spaces, i.e., two 2-D benchmark functions and a 3-D PID gain tuning problem. The latter is not carried out on a physical system, and it is not clear how SaMBO would scale to higher dimensions. Yet, it represents a promising demonstration of the capabilities of SaMBO for many potential robotics and automation applications. The number of repetitions of the experiments appears adequate.\n\nThe quality of presentation is high. The paper is well-structured. Concepts are introduced at the right granularity and notation is not too heavy. Figures are informative and successful in clarifying the rationale of the frontier search algorithm and presenting the experimental results.\n\nFew comments on potential improvements:\n-\tConsider harmonizing the notation in the figures and text, which appear to vary slightly in some cases (e.g., $Q_l$ vs. $Q^l$, define $f^{tar}$, $q^{tar}$)\n-\tI may be wrong, but, in Algorithm 1, wouldn’t the Prune function need to take as input the intersection between Q and z instead of the union?\n-\tThe definition of average predictive std in Eq. (6) may be worth an additional short description in a way similar to Eq. (5) to better appreciate the conceptual difference\n\nOthers:\n\n- L36: on which it falls back to $\\rightarrow$  on which it falls back\n- L37: data-riven $\\rightarrow$ data-driven\n- L51: GOOSE [10, 18] by expanding $\\rightarrow$ GOOSE [10, 18] does so by expanding\n- L60: non-i.i.d $\\rightarrow$ non-i.i.d.\n- L61: settings, however, $\\rightarrow$ settings. However,\n- L77: e.g., data of the same robotic platform under different conditions. $\\rightarrow$  For example?\n- L85: we ask the question how $\\rightarrow$ we ask the question of how\n- L93 where $\\beta$ *is* the… *and* is set such…\n- L100: w.r.t. $\\rightarrow$  with respect to\n- L114: Prior work *proposing* to *meta-learn* GP priors [44, 45, 32], *though*, fail…\n- L117, missing a closing parenthesis in the inline expression\n- Consider using the definition symbol in Eqq. (2), (3), (4), (5), (6)\n- Add full stop after Eq. (4)\n- Consider redefining “calib” by “avg-calib” in Eq. (5), since in the tect (L147) it is referred to as average calibration frequency and it is averaged in a similar way as “avg-std”\n- L229 calibration- and sharpness-based\n- L259 to measure a method’s\n- L287 Note that, in case of the regret, the …", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, the paper presents a novel method tackling a well-known relevant problem in Safe BO. To the best of my knowledge, relevant literature is exhaustively covered. Presentation quality is good. The method employs tools from calibration analysis and meta learning in an innovative way, and presents a novel frontier search method to safely select informative priors and improve query efficiency of BO procedures when data from similar tasks is available. \n\nThe efficiency and safety of the method is demonstrated with two different BO algorithms on three low-dimensional tasks in simulation. Although not demonstrated on real hardware, the results seem promising and the method may be applied to a wide range of robot control and automation problems in which gains need be tuned under strict safety constraints. Scaling properties of this approach to higher dimensions remain to be investigated.\n\nIn my view, the presented method and results are timely and relevant to the robot learning community.\n\n", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "Consider addressing the weaknesses listed above, i.e., \n- Comment upon scaling to higher-dimensional settings (or even show an experimental stress-test if time permits)\n- Comment and provide examples of potential real-world scenarios in which similar tasks may typically be available or could be conveniently acquired, and how to quantify if a task is “similar enough” to another one for it to be employed for meta learning\n- Discuss/show counter-examples and edge-cases for the monotonicity of the average calibration w.r.t. the kernel length scale\n- Notation consistency between text and Algorithm environments  \n- Add short description of avg-std\n- Fix the listed typos\n"}}, {"review_id": "2TY8LteMhw0", "reviewer": "Reviewer_SuCp", "summary": "In this paper, the authors present an improved safe Bayesian optimization method by exploiting existing offline data for the meta-learning of priors. By introducing safety-compliant priors via empirical uncertainty metrics and a frontier search algorithm, the proposed framework provides a reliable uncertainty quantification which leads to more trustworthy safety guarantees in contrast to hand-tuned vanilla safe BO. A simulation and a robotic experiment show the benefit of the framework.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n+ The paper is very well written and clear in all its steps\n+ The topic is of timely interest and important for the robotics community\n+ The method allows to exploit eventually existing offline data.\n+ The experiment/simulation are convincing\n\nCons:\n- The footnote \"the monotonicity of the calibration frequency in l is only an empirical heuristic that holds in almost all cases if ν is at least as big as the variance of the targets y in a dataset\" seems to be quite important as the monotonicity is key for the success of the optimization. I would like to see some more discussion and evaluations here.\n\n- The proposed framework seems to be computational expensive. Some more details on that would be of interest.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I really enjoyed reading this paper as it is well written and technical sound. The idea is interesting and can be definitely helpful in scenarios where offline-data is existing. However, I'm not sure if such a rich offline-data set is often available such that a major impact in robotics is questionable.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "Please address the cons mentioned above.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (oral)