SlowGuess/NeurIPS_2023_OCR huggingface dataset:
According to Figure 8, we find that 1) with sufficient computation power, the MMDGP-raw-L can obtain the best performance by using a large sample size. 2) However, with limited complexity, the performance MMDGP-raw-S degrades obviously while the MMDGP-nystrom performs much better. This suggests that the Nyström approximation can significantly improve the efficiency with a mild cost of performance degradation. 3) All the MMDGP-based methods are better than the vanilla GP-UCB.
2For a fair comparison, all the methods in this test use a UCB acquisition.
Figure 9: Optimization regret on 10D bumped-bowl problem.
Figure 10: The input GMM distribution.
Figure 11: Simulation results of the push configurations found by different algorithms.
D.3 Optimization on 10D Bumped-Bowl Problem
To further evaluate AIRBO's optimization performance on the high-dimensional problem, we employ a 10-dimensional bumped bowl function from [27, 19]:
$$
f (\boldsymbol {x}) = g \left(\boldsymbol {x} _ {1: 2}\right) h \left(\boldsymbol {x} _ {3:}\right), \text {w h e r e} \left\{ \begin{array}{l} g (\boldsymbol {x}) = 2 \log \left(0. 8 \| \boldsymbol {x} \| ^ {2} + e ^ {- 1 0 \| \boldsymbol {x} \| ^ {2}}\right) + 2. 5 4 \\ h (\boldsymbol {x}) = \sum_ {i} ^ {d} 5 x _ {i} ^ {2} + 1 \end{array} \right. \tag {33}
$$
Here,  $x_{i}$  is the i-th dimension of  $\pmb{x}$ ,  $x_{1:2}$  represents the first 2 dimensions for the variable, and  $x_{3}$  indicates the rest dimensions. The input uncertainty is designed to follow a concatenated distribution of a 2D circular distribution  $(r = 0.5)$  and a multivariate normal distribution with a zero mean and diagonal covariance of 0.01.
Figure 9 shows the mean and std values of the optimization regrets. We note that 1) when it comes to a high-dimensional problem and complex input distribution, the misassumption of Gaussian input uncertainty renders the  $\text{skl}$  and  $\text{ERBF}$  fail to locate the robust optimum and get stuck at local optima. 2) Our method outperforms the others and can find the robust optimum efficiently and stably, while the  $uGP$  with a similar inference cost suffers the instability caused by insufficient sampling and stumbles over iterations, which can be evidenced by its high std values of optimization regret.
D.4 Robust Robot Pushing
This benchmark is based on a Box2D simulator from [31], where our objective is to identify a robust push configuration, enabling a robot to push a ball to predetermined targets under input randomness. In our experiment, we simplify the task by setting the push angle to  $r_a = \arctan \frac{r_y}{r_x}$ , ensuring the robot is always facing the ball. Also, we intentionally define the input distribution as a two-component Gaussian Mixture Model as follows:
$$
\left(r _ {x}, r _ {y}, r _ {t}\right) \sim G M M \left(\mu = \left[ \begin{array}{l l l} 0 & 0 & 0 \\ - 1 & 1 & 0 \end{array} \right], \boldsymbol {\Sigma} = \left[ \begin{array}{l l l} 0. 1 ^ {2} & - 0. 3 ^ {2} & 1 e - 6 \\ - 0. 3 ^ {2} & 0. 1 ^ {2} & 1 e - 6 \\ 1 e - 6 & 1 e - 6 & 1. 0 ^ {2} \end{array} \right], w = \left[ \begin{array}{l} 0. 5 \\ 0. 5 \end{array} \right]\right), \tag {34}
$$
where the covariance matrix  $\Sigma$  is shared among components and  $w$  is the weights of mixture components. Meanwhile, as the SKL-UCB and ERBF-UCB surrogates can only accept Gaussian input distributions, we choose to approximate the true input distribution with a Gaussian. As shown in Figure 10, the approximation error is obvious, which explains the performance gap among these algorithms in Figure 5b.
Apart from the statistics of the found pre-images in Figure 6, we also simulate the robot pushes according to the found configurations and visualize the results in Figure 11. In this figure, each black hollow square represents an instance of the robot's initial location, the grey arrow indicates the push direction and duration, and the blue circle marks the ball's ending position after the push. We can find that, as the GP-UCB ignores the input uncertainty, it randomly pushes to these targets and the ball ending positions fluctuate. Also, due to the incorrect assumption of the input distribution, the SKL-UCB and ERBF-UCB fail to control the ball's ending position under input randomness. On the contrary, AIRBO successfully recognizes the twin targets in quadrant I as an optimal choice and frequently pushes to this area. Moreover, all the ball's ending positions are well controlled and centralized around the targets under input randomness.