SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Key of regret analysis. To analyze the regret, we start by evaluating the growth rate of the number of successful observations  $n_t$  in the algorithm. By noting that  $\theta_t b(t)$  is monotonically decreasing, we can find that the failure points of F-GP-UCB are separated by at least a distance  $\theta_t b(t)$  apart from each other. Therefore,  $n_t$  can be evaluated via  $\theta_t b(t)$ -packing number of  $F_c$ . The following Lemma 4.1 is based on the standard argument for the packing number, which gives the lower bound for the number of successful observations  $n_t$ .
Lemma 4.1. When running Algorithm 1,  $n_t \geq t - \mathcal{P}(\mathcal{X}, \| \cdot \|_{\infty}, \theta_t b(t) / 4)$  holds for any  $t \in \mathbb{N}_+$ , where  $\mathcal{P}(\mathcal{X}, \| \cdot \|_{\infty}, \epsilon)$  denotes the  $\epsilon$ -packing number over the set  $\mathcal{X}$  with respect to the norm  $\| \cdot \|_{\infty}$ .
Using the above Lemma 4.1, designing the choice of  $b$  so that  $t - \mathcal{P}(\mathcal{X},\| \cdot \|_{\infty},\theta_t b(t) / 4) = \Theta (t)$  holds leads to  $n_t = \Omega (t)$ , which means that the number of successful observations can be secured at the same rate as in the standard GP optimization setting. By using the fact  $\mathcal{P}(\mathcal{X},\| \cdot \|_{\infty},\epsilon) = \lceil 1 / \epsilon \rceil^d$  (Lemma A.4), the choice of  $b(t) = o(t^{-1 / d})$  is sufficient for  $n_t = \Omega (t)^3$ . While our result holds for any  $b$  satisfying  $b(t) = o(t^{-1 / d})$ , we give results for  $b$  satisfying  $b(t) = t^{-\alpha}$  with  $\alpha \in (0,1 / d)$  in this section for simplicity. In Appendix A, the result for general  $\mathcal{X}$  and  $b$  is also provided.
Regret bound. The following Theorem 4.2 gives the regret bound of the F-GP-UCB algorithm.
Theorem 4.2. Fix  $\delta \in (0,1)$ ,  $\alpha \in (0,d^{-1})$ ,  $\mathcal{X} = [0,1]^d$ , and  $f \in \mathcal{H}_k$  such that  $\| f \|_{\mathcal{H}_k} \leq B$ . Let  $\beta_t^{1/2} = B + \sigma \sqrt{2(\gamma_{t-1};\mathcal{X} + 1 + \ln(1/\delta))}$  and  $b(t) = t^{-\alpha}$ . In addition, suppose that there exists  $\eta > 0$  such that  $N_{\mathbf{x}^*;\eta} \subset S_c$ , where  $N_{\mathbf{x}^*;\eta} = \{\mathbf{x} \in \mathcal{X} \mid \| \mathbf{x} - \mathbf{x}^* \|_{\infty} < \eta\}$ . When applying Algorithm 1 under the above conditions, the estimated solution  $\hat{\mathbf{x}}_T$  defined in (8) exists for all  $T \geq \tilde{T}_s := \max \{(10/\overline{\theta}^d)^{1/(1-d\alpha)}, 2\} + 1$ , where  $\overline{\theta} = \min \{\theta_0, \eta/2\}$ . Moreover, the following holds with probability at least  $1-\delta$ :
$$
\forall T \geq \tilde {T} _ {s}, r _ {T} \leq \frac {2}{T} \left[ 2 B \left(\tilde {T} _ {*} - 1\right) + \sqrt {C _ {1} \beta_ {T} T \gamma_ {T ; S _ {c}}} \right], \tag {9}
$$
where  $\tilde{T}_{*} = (\theta_{0} / \eta)^{1 / \alpha} + 1$  and  $C_1 = 8 / \ln (1 + \sigma^{-2})$ .
In the analysis above,  $\eta$  is interpreted as the complexity of the failure function  $c$ , and  $\overline{\theta}$  corresponds to the lower bound on  $\theta_t$  while running the algorithm. The constants  $\tilde{T}_s$  and  $\tilde{T}_*$  are important quantities that characterize the regret and can be respectively interpreted as follows.
- The constant  $\tilde{T}_s$  is the upper bound on the number of steps until the first successful observation.
- The constant  $\tilde{T}_{*}$  is the upper bound on the number of steps needed until the search space  $\mathcal{X}_t$  of the algorithm always contains the optimal solution  $\boldsymbol{x}^*$ .
The regret bound in Theorem 4.2 can be intuitively seen as evaluating the algorithm execution in terms of two stages along the time axis. The first term of (9) represents the regret generated when the algorithm's search space  $\mathcal{X}_t$  does not contain  $\pmb{x}^*$ . Specifically, the worst-case regret incurred during this phase is bounded by  $2B$  using the fact that  $\sup_{\pmb{x}\in \mathcal{X}}|f(\pmb{x})|\leq B$  from the kernel normalization
3Strictly speaking, we use the fact that  $\theta_t$  becomes constant for sufficiently large  $t$ . See Lemma A.8 in Appendix A.
condition. The second term is the regret incurred during the process of identifying  $\pmb{x}^*$  after  $\mathcal{X}_t$  includes  $\pmb{x}^*$  and is similar to the regret bound of standard GP-UCB. The only difference is that, in our setup, the observations of GP are only on  $S_c$ , so the maximum information gain is defined over  $S_c$ .
By noting that  $\gamma_{T;S_c} \subset \gamma_{T;\mathcal{X}}$  in the second term and using the known results concerning the upper bound on  $\gamma_{T;\mathcal{X}}$ , the more explicit form of regret upper bound can be obtained on a per-kernel basis. For example, in the case of the Gaussian kernel, we have that  $\gamma_{T;\mathcal{X}} = \mathcal{O}((\ln T)^{d+1})$  and  $r_T = \mathcal{O}\left(T^{-1}B\tilde{T}_* + T^{-1/2}(\ln T)^{(d+1)/2}\left(B + \sigma\sqrt{(\ln T)^{d+1} + \ln(1/\delta)}\right)\right)$ . Thus, indeed we obtain  $r_T \to 0$  (as  $T \to \infty$ ). The F-GP-UCB algorithm is guaranteed to converge when the cumulative regret for ordinary GP-UCB (corresponding to the second term in (9)) becomes sub-linear under the chosen setup and the kernel<sup>4</sup>.
The case where  $\boldsymbol{x}^*$  exists on the boundary of  $S_{c}$ . The case where  $\boldsymbol{x}^*$  exists on the boundary  $S_{c}$  is not covered in Theorem 4.2, which is the case that often appears in a real-world application. If undesirable Lipschitz-style dependencies are allowed to emerge in the regret upper bound, our F-GP-UCB algorithm is also guaranteed to have convergence in such cases (Theorem B.2). The details of the result and discussion about its limitation are in Appendix B.
5 Practical considerations
In this section, we discuss several issues that arise in practical situations and provide their solutions. Due to the space limitation, we only give brief descriptions. The details are described in Appendix E.
Computation of  $\theta_{t}$ . The computation of  $\theta_{t}$  requires us to know whether the union of all the neighborhoods of the failure points covers  $\mathcal{X}$ . A simple solution might be to partition  $\mathcal{X}$  into a sufficiently fine grid by subdividing each axis and look for a feasible solution in each grid cell by brute force. However, as the step size  $t$  and dimension  $d$  become large, the number of grid cells grows rapidly. Furthermore, even a very fine grid will not be able to detect the case where a feasible solution exists on the cell boundary. Another heuristic approach is to use a constrained optimization solver which does not require a feasible solution as part of the initial points (e.g., penalty function method and augmented Lagrangian method [17]) and try to solve the problem (7) by using  $\theta_{t-1}$ . In case that the solver is unable to find a feasible solution, it is decided that  $\tilde{\mathcal{X}}_t \neq \emptyset$ , multiply  $\theta_{t-1}$  by  $1/2$ , then we try to solve problem (7) again. This is our chosen approach. A drawback of this approach is that, depending on the scale of  $\theta_{t}$ , the constrained optimization problem must be solved many times. We show that, by performing an appropriate preprocessing, a feasible solution can be obtained within two rounds of constrained optimization in the worst case. We give the details in Appendix E.1.
Selection of  $b$ . As stated in Sec. 4, the F-GP-UCB algorithm is valid for an arbitrary monotonically decreasing function  $b$  such that  $t - \mathcal{P}(\mathcal{X},\| \cdot \|_{\infty},\theta_{t}b(t) / 4) = \Theta (t)$ . However, if the rate is slower than  $1 / t^{\alpha}$  ( $\alpha \in (0,1 / d)$ ), for example,  $b(t) = 1 / \ln (t + 1)$ , then the dominant term of our regret upper bound can become  $\mathcal{O}(1 / \ln T)$  (See Theorem B.3 in Appendix B and Corollary F.1 in Appendix F). This intuitively means that an exponentially large sample is needed to obtain an arbitrarily small regret and is not the desirable behavior. In this paper, we use  $b$  in the form of  $b(t) = 1 / t^{\alpha}$  with  $\alpha = 1 / (2d)$  as a practical choice. (Additional discussion about  $\alpha$  is given in Appendix E.2.)
Adaptive tuning of  $\theta_0$ . Although the choice of  $\theta_0$  does not affect the no-regret guarantee, it has a large impact on the practical behavior when the number of steps is small. For example, if  $\theta_0$  is set to a large value when the failure region  $F_{c}$  forms a relatively small region, then there is a risk of excessively avoiding the feasible region that should be searched. It is practically difficult to find a desirable choice of  $\theta_0$  as it depends on some conditions that are unknown to the user, such as the failure region  $F_{c}$ . In order to alleviate this issue while retaining the theoretical convergence, we use the strategy to adaptively choose  $\theta_t$  based on the posterior standard deviation  $\sigma_{t-1}(\boldsymbol{x}_t)$  of the observed points. We note that such a strategy using  $\sigma_{t-1}(\boldsymbol{x}_t)$  to adaptively select an unknown parameter is also developed in [54] to set the unknown kernel hyperparameters for ordinary GP optimization.
As with the standard GP-UCB,  $\beta_{t} = \mathcal{O}(\sqrt{\gamma_{T;\mathcal{X}}})$  in F-GP-UCB, which leads to the regret of  $\tilde{\mathcal{O}} (\gamma_{T;\mathcal{X}} / \sqrt{T})$ . In Appendix D, we discuss the possibility of addressing this issue based on several existing works [48, 30].
Algorithm 4 in Appendix E.3 shows the pseudo-code of our modified strategy. In Algorithm 4, the user specifies the possible range for  $\theta_t \in [\theta_{\min}, \theta_{\max}]$ , the threshold for the posterior standard deviation  $h_\sigma > 0$ , the threshold for the step number  $q \in \mathbb{N}_+$ , and a scaling factor  $w \in (0,1)$ . As the initial value, we set  $\theta_0 = \theta_{\max}$ . After that, if the observed point is lower than the posterior standard deviation for  $q$  consecutive numbers of times,  $\theta_t$  is multiplied by  $w$  to lower its value, but no less than the minimum value  $\theta_{\min}$ . Intuitively, suppose the posterior standard deviation is repeatedly excessively small. In that case, there exists a possibility that  $\theta_0$  was set to an excessively large value, which may lead to the unwanted exclusion of the feasible region from the search space. In this procedure, by setting  $\theta_0$  to a large value of  $\theta_{\max}$  at the start, we expect it to get adjusted towards an appropriate neighborhood scale. In our simulation experiments, we set  $w = 0.75$ ,  $h_\sigma = 0.02$ ,  $q = 3$ ,  $\theta_{\min} = 0.0001$ , and  $\theta_{\max} = 0.5$ . We emphasize that this procedure does not affect our convergence guarantee (Theorems F.2 in Appendix E).
Lastly, the pseudo-code, including all the considerations provided in this section, is described in Algorithm 5 of Appendix E.
In this section, we show the performance of F-GP-UCB through numerical experiments. The detailed settings of the experiments and additional results can be found in Appendices G and H, respectively. First, for comparison, we use EI [37] and GP-UCB [42] as the baseline algorithms which do not consider failures. In each of these algorithms, the GP model is constructed using only the successful observations; the failure observations are not used. We also compare F-GP-UCB with the EFI-GPC-EP and EFI-GPC-Sign algorithms that leverage GPC models [2]. EFI-GPC-EP is the algorithm that uses the classic GPC model with probit likelihood, and posterior approximation is based on Expectation Propagation (EP). EFI-GPC-Sign uses a variant of GPC proposed in [2]. Both EFI-GPC-EP and EFI-GPC-Sign choose the next input based on the posterior success probability of GPC and the EI value of  $f$  as in [2] (Details are in Appendix H).
In EFI-GPC-EP and EFI-GPC-Sign, as the prior for the latent function of GPC, we use zero-mean GP with Gaussian kernel  $k_{c}(\boldsymbol{x},\boldsymbol{y}) \coloneqq \sigma_{c}^{2}\exp(-\|\boldsymbol{x}-\boldsymbol{y}\|^{2}/(2l_{c}^{2}))$ . Before the experiment, we fix the kernel parameters of GPC  $\sigma_{c}$  and  $l_{c}$  by marginal likelihood maximization using observations at a randomly generated Sobol sequence. All the kernel hyperparameters of the GPC fixed by this procedure are given in Appendix H. Furthermore, it is well-known that the theoretical choice of  $\beta_{t}$  in GP-UCB is excessively conservative. We set  $\beta_{t} = 2\ln (2t)$  as in [24] for GP-UCB and F-GP-UCB. The other parameters in F-GP-UCB are fixed as described in Sec. 5. For the evaluation of the algorithms, we confirm the behavior of the regret at each step  $r_{t} = f(\boldsymbol{x}^{*}) - f(\hat{\boldsymbol{x}}_{t})$ . We note that  $\hat{\boldsymbol{x}}_{t}$  is the estimated solution  $\hat{\boldsymbol{x}}_{t} = \boldsymbol{x}_{\hat{t}}$  where  $\hat{t} = \operatorname{argmax}_{i\in\mathcal{T}_{t}^{(s)}}\operatorname{lcb}_{t}(\boldsymbol{x}_{i})$ , which is slightly
modified from the theoretical value as discussed in Sec. 3 If we get that  $\mathcal{I}_t^{(S)} = \emptyset$ , then we set  $r_t = f(\pmb{x}^*) - \min_{\pmb{x} \in \mathcal{X}} f(\pmb{x})$ . This definition is the same as the utility gap metric for constrained GP optimization [19]. In each experiment, we report the average performance of 20 numbers of optimization trials with different seeds. In each trial, we generate one point uniformly at random over  $\mathcal{X}$  and use it as the initial point. Lastly, we add an artificial noise  $\epsilon_t \sim \mathcal{N}(0, \sigma^2)$  with  $\sigma^2 = 0.0001$  to the observation of  $f$ . We also fix the noise variance hyperparameter of the GP model for  $f$  to 0.0001.
Synthetic experiments with the Branin function. We perform synthetic benchmark experiments with the Branin function whose input space is scaled to  $[0,1]^2$ . The failure function used in these experiments is shown in Fig. 2. This failure function has a feasible region on the upper right side which is easily identifiable. In addition, the failure function also contains multiple small isolated feasible regions, one of which contains the optimal solution. We use the Gaussian kernel  $k(\pmb{x},\pmb{y})\coloneqq \sigma_f^2\exp (-\| \pmb {x} - \pmb {y}\| _2^2 /(2l_f^2))$  to construct a GP model of  $f$ . The parameters  $\sigma_{f}$  and  $l_{f}$  are fixed beforehand by marginal likelihood maximization over a Sobol sequence of 1024 points. The subsequent experiments also use the Gaussian kernel; and the parameters are fixed as described above.
Figure 2 shows the result. Under this benchmark setting, F-GP-UCB has a better performance. We also give the detailed behavior of F-GP-UCB in Appendix G. We note that, in EI and GP-UCB, the regret stops decreasing in the early stage. This is because these algorithms get stuck on the same failure point.
Figure 2: The left plot shows the Branin function whose input space is scaled to  $[0,1]^2$ . The shaded regions represent the failure regions. The right plot shows the regret in the synthetic problem using the Branin function, where the average of 20 experiments with different random seeds is shown. The error bars correspond to two standard errors.
Figure 3: Experiment results of benchmark function of the constrained optimization problems (left and middle) and quasicrystal simulation function (right). The left and middle plots show the result of Gardner and Hartmann test functions, respectively. The error bars correspond to two standard errors.
Test functions for constrained optimization. We adopt the two benchmark settings which are used in existing GP-based constrained optimization literature. Gardner [10] is a 2D test problem whose objective and constraint functions are defined as a combination of sine and cosine functions. Hartmann [29] is a 3D test problem whose feasible region becomes a unit hypersphere. To adapt the constrained problem to our settings, the feasible region in each benchmark is defined as the success region, while the rest is defined as the failure region. Fig. 3 shows the result. Within the compared algorithms, F-GP-UCB has the best performance for the Gardner function, while EFI-GPC-Sign has the best performance for the Hartmann function. This can be understood from the fact that the feasible region of the Hartmann function, which is a unit hypersphere, and can be modeled easily by GPC. However, F-GP-UCB performs better compared to EFI-GPC-EP, which uses a classic GPC model. Moreover, the regret of F-GP-UCB continues to decrease as the number of iterations increases.
Numerical experiments with quasicrystals. We perform a numerical experiment involving quasicrystals in the Al-Cu-Mn system. We consider the optimization of the phonon thermal conductivity for quasicrystals. In this setting, the input space is the relative composition of the three elements, restricted to be around the feasible region of quasicrystal formation. The feasible region is the composition values that form quasicrystals based on data [16]. Outside of this region, quasicrystals do not form and so we consider it as a failure region, as the material property could not be probed with methods designed for quasicrystals. For the objective function, we use an empirical formula for the phonon thermal conductivity which we extract from [46]. The details of the setting are described in
5The original paper [29] considers a 6D problem and this setting leads to an excessively large failure region. We modify the setting by using a 3D version of the Hartmann function with unit hypersphere constraint.
Appendix H.2. In Fig. 3, the right plot shows the result of the numerical experiments for quasicrystals. It shows that F-GP-UCB performs the best among the compared algorithms.
In this paper, we propose a novel GP-based optimization algorithm in the presence of unknown failure regions in the search space. Our algorithm only requires a very mild assumption of the failure function that the optimal solution lies on an interior of a feasible region. We show that our algorithm achieves a convergence with high probability and provides the first regret upper bound by appropriately adjusting the search space. Its effectiveness is verified empirically through numerical experiments, including the heuristic simulation experiment motivated by the material research of quasicrystals.
This work was partially supported by RIKEN Center for Advanced Intelligence Project.
[1] Yasin Abbasi-Yadkori. Online learning for linearly parametrized control problems. PhD thesis, University of Alberta, 2013.
[2] François Bachoc, Céline Helbert, and Victor Picheny. Gaussian process optimization with failures: classification and convergence proof. Journal of Global Optimization, 78(3):483-506, 2020.
[3] Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: A framework for efficient Monte-Carlo Bayesian optimization. Advances in Neural Information Processing Systems, 33:21524-21538, 2020.
[4] James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyperparameter optimization. In Advances in Neural Information Processing Systems, volume 24, 2011.
[5] Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics. Machine Learning, pages 1-35, 2021.
[6] Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, and Volkan Cevher. Adversarially robust optimization with Gaussian processes. Advances in Neural Information Processing Systems, 31, 2018.
[7] Romain Camilleri, Kevin Jamieson, and Julian Katz-Samuels. High-dimensional experimental design and kernel bandits. In International Conference on Machine Learning, pages 1227-1237. PMLR, 2021.
[8] Sayak Ray Chowdhury and Aditya Gopalan. On kernelized multi-armed bandits. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pages 844–853, 2017.
[9] Thomas Desautels, Andreas Krause, and Joel W. Burdick. Parallelizing exploration-exploitation tradeoffs in Gaussian process bandit optimization. Journal of Machine Learning Research, 15 (119):4053-4103, 2014.
[10] Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunningham. Bayesian optimization with inequality constraints. Proceedings of the 31st International Conference on Machine Learning, 32(2):937-945, 22-24 Jun 2014.
[11] Michael A. Gelbart, Jasper Snoek, and Ryan P. Adams. Bayesian optimization with unknown constraints. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI'14, page 250-259, Arlington, Virginia, USA, 2014. AUAI Press. ISBN 9780974903910.
[47] Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Sequential and parallel constrained max-value entropy search via information lower bound. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 20960-20986, 17-23 Jul 2022.
[48] Sattar Vakili, Nacime Bouziani, Sepehr Jalali, Alberto Bernacchia, and Da-shan Shiu. Optimal order simple regret for gaussian process bandits. Advances in Neural Information Processing Systems, 34:21202-21215, 2021.
[49] Sattar Vakili, Kia Khezeli, and Victor Picheny. On information gain and regret bounds in Gaussian process bandits. In International Conference on Artificial Intelligence and Statistics, pages 82–90. PMLR, 2021.
[50] Sattar Vakili, Jonathan Scarlett, and Tara Javidi. Open problem: Tight online confidence intervals for RKHS elements. In Conference on Learning Theory, pages 4647-4652. PMLR, 2021.
[51] Michal Valko, Nathan Korda, Rémi Munos, Ilias Flaounas, and Nello Cristianini. Finite-time analysis of kernelised contextual bandits. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, pages 654-663, 2013.
[52] Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge University Press, 2018.
[53] Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient Bayesian optimization. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 3627-3635, 06-11 Aug 2017.
[54] Ziyu Wang, Frank Hutter, Masour Zoghi, David Matheson, and Nando De Feitas. Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55:361-387, 2016.