SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We investigated simple extensions of TS-PSQ, UCB-PSQ, and EI-PSQ (i.e., the classic expected improvement algorithm [20] adapted for BOPSQ, see Appendix C for details) for the cost-varying problem by dividing the acquisition score of a control set by its cost in a manner similar to Snoek et al. [31, Sec. 3.2]. Fig. 3 shows the mean and standard error of the simple regret incurred over 10 RNG seeds for one set of experiments. We found that these naive methods generally do not work well. For TS per unit cost and UCB-PSQ per unit cost, if a suboptimal control set is very cheap, its acquisition score may remain artificially high throughout, and the algorithm fails to converge. EI per unit cost was slightly more promising, but suffered from the inverse problem: the suboptimal control sets had 0 expected improvement very quickly and dividing by the cost had no effect. This algorithm thus fails to encourage exploration with cheaper control sets. Furthermore, the EI algorithm was computationally expensive due to the double Monte Carlo expectation computation. In general, we see that the UCB-CVS algorithm is able to use the cheaper control sets much more effectively for exploration and hence find better solutions.
Figure 3: Mean and standard error (over 10 RNG seeds) of the simple regret (lower is better) incurred against cost spent (budget)  $C$  by all algorithms including TS-PSQ per unit cost, UCB-PSQ per unit cost, and EI per unit cost, with samples from the GP prior as the objective function, moderate cost set, and all variances. A diamond indicates the average budget after which an algorithm only chooses the optimal control set.
All experiments use a squared exponential kernel with ARD lengthscales that depend on the objective function,  $k(\mathbf{x},\mathbf{x}^{\prime}) = 1$ , Gaussian observation noise with  $\sigma = 0.01$ , and 5 initial query-observation pairs with queries drawn uniformly at random. All expectations are approximated with Monte Carlo sampling with 1024 samples. All acquisition maximizations are performed with L-BFGS-B with random restarts. All query sets are  $[0,1]^d$ .
C.1 Objective functions
The control sets described here are given in an order corresponding to their costs given in Sec. 5. For example, for the GP samples objective, under the cheap cost set, control set  $\{1\}$  has cost 0.01, control set  $\{1,2\}$  has cost 0.1, and control set  $\{1,2,3\}$  has cost 1.
Samples from GP prior (3-D): We use samples from the same kernel  $k$  used to model the GP posteriors during learning. We use a kernel lengthscale of 0.1 and control sets  $\{\{1\}, \{2\}, \{3\}, \{1,2\}, \{1,3\}, \{2,3\}, \{1,2,3\}\}$ .
Hartmann (3-D): We use a kernel lengthscale of 0.1 and control sets  $\{\{1\}, \{2\}, \{3\}, \{1, 2\}, \{1, 3\}, \{2, 3\}, \{1, 2, 3\}\}$ .
Plant growth simulator (5-D): The plant growth simulator is a GP built from private data collected on the maximum leaf area achieved by Marchantia plants depending on input variables Ca, B,  $\mathrm{NH}_3$ , K, and pH. We use min-max feature scaling to scale all input variables to [0, 1] and standardize the output values. We use the posterior mean of the GP as the objective function. We use a kernel lengthscale of 0.2 and control sets  $\{\{1,2\}, \{3,4\}, \{4,5\}, \{1,2,3\}, \{2,3,4\}, \{3,4,5\}, \{1,2,3,4,5\}\}$ .
Airfoil self-noise (5-D): We use the airfoil self-noise dataset from the UCI Machine Learning Repository [9]. To scale all input variables to  $[0,1]$ , we first take the natural logarithm of variables 1 and 5, then do min-max feature scaling on all input variables. We also standardize the output values. We then feed the data into a default SingleTaskGP from BoTorch and use the posterior mean as the objective function. We use a kernel lengthscale of 0.2 and control sets  $\{\{4,5\}, \{2,5\}, \{1,4\}, \{2,3\}, \{3,5\}, \{1,2\}, \{3,4\}\}$ .
C.2 Algorithms
UCB-PSQ and UCB-CVS: For the experiments, we set  $\beta_{t} = 2$  for all  $t$ .
TS-PSQ: Following [13], we use random Fourier features (RFF) [28] to approximately sample from a GP posterior. We use RFF with 1024 features.
EI-PSQ: We adapt the BoTorch acquisition NoisyExpectedImprovement to the BOPSQ problem setting. To evaluate the acquisition score of a partial query, we first sample 32 fantasy models of  $f$  from the GP posterior. For each fantasy model, we compute the expected value of the partial query and take the best value as the value of the best observation so far (assuming the full query control set is available). We then compute the improvement score as the expected value minus the best value, and then average the improvement score over all fantasy models.
C.3 Implementation
The experiments were implemented in Python. The major libraries used were NumPy [12], SciPy [40], PyTorch [26], GPyTorch [10] and BoTorch [3]. For more details, please refer to the code repository.
C.4 Compute
The following CPU times in seconds were collected on a server running Ubuntu 20.04.4 LTS with  $2 \times$  Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz and 256 GB of RAM. We measure the CPU time for 1 iteration of TS-PSQ and UCB-CVS with a dataset of 100 observations. In general, none of the algorithms in the settings tested in this paper require a significant amount of compute.
GP sample	Hartmann	Plant	Airfoil
TS-PSQ	6.27	4.14	8.96	232.27
UCB-CVS	37.92	52.34	61.96	87.09
A limitation of our work is that the theoretical guarantees of UCB-CVS rely on a few assumptions that may not hold in practice. For example, the regularity assumption that assumes the objective function  $f$  resides in some RKHS may not be true in some problems. The kernel corresponding to this RKHS may not be known either. The work also assumes that the probability distributions governing each variable are independent and fixed. In practice, these assumptions may be violated, if the probability distributions have some dependence on one another, or may change over time.