Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization
Abstract
Bayesian optimization (BO) is a class of popular methods for expensive black-box optimization, and has been widely applied to many scenarios.
However, BO suffers from the curse of dimensionality, and scaling it to high-dimensional problems is still a challenge.
In this paper, we propose a variable selection method MCTS-VS based on Monte Carlo tree search (MCTS), to iteratively select and optimize a subset of variables.
That is, MCTS-VS constructs a low-dimensional subspace via MCTS and optimizes in the subspace with any BO algorithm.
We give a theoretical analysis of the general variable selection method to reveal how it can work.
Experiments on high-dimensional synthetic functions and real-world problems (i.e., NAS-bench problems and MuJoCo locomotion tasks) show that MCTS-VS equipped with a proper BO optimizer can achieve state-of-the-art performance.
Introduction
In many real-world tasks such as neural architecture search (NAS) [39] and policy search in reinforcement learning (RL) [6], one often needs to solve the expensive black-box optimization problems.
Bayesian optimization (BO) [2, 11, 23, 32] is a sample-efficient algorithm for solving such problems.
It iteratively fits a surrogate model, typically Gaussian process (GP) and maximizes an acquisition function to obtain the next point to evaluate.
While BO has been employed in a wide variety of settings, successful applications are often limited to low-dimensional problems.
Recently, scaling BO to high-dimensional problems has received a lot of interest.
Decompositionbased methods [13, 15, 17, 26, 31] assume that the high-dimensional function to be optimized has a certain structure, typically the additive structure.
By decomposing the original high-dimensional function into the sum of several low-dimensional functions, they optimize each low-dimensional function to obtain the point in the high-dimensional space.
However, it is not easy to decide whether a decomposition exists as well as to learn the decomposition.
Other methods often assume that the original high-dimensional function with dimension D has a low-dimensional subspace with dimension d ≪ D , and then perform the optimization in the low-dimensional subspace and project the low-dimensional point back for evaluation.
For example, embedding-based methods [20, 27, 40] use a random matrix to embed the original space into the lowdimensional subspace.
Another way is to select a subset of variables directly, which can even avoid the time-consuming matrix operations of embedding-based methods.
For example, Dropout [21] selects d variables randomly in each iteration.
Note that for both embedding and variable selection methods, the parameter d can have a large influence on the performance, which is, however, difficult to set in real-world problems.
In this paper, we propose a new Variable Selection method using Monte Carlo Tree Search (MCTS), called MCTS-VS.
MCTS is employed to partition the variables into important and unimportant ones, and only those selected important variables are optimized via any black-box optimization algorithm, e.g., vanilla BO [32] or TuRBO [10].
The values of unimportant variables are sampled using historical information.
Compared with Dropout-BO, MCTS-VS can select important variables automatically.
We also provide regret and computational complexity analyses of general variable selection methods, showing that variable selection can reduce the computational complexity while increasing the cumulative regret.
Our regret bound generalizes that of GP-UCB [36] which always selects all variables, as well as that of Dropout [21] which selects d variables randomly in each iteration.
The results suggest that a good variable selection method should select as important variables as possible.
Experiments on high-dimensional synthetic functions and real-world problems (i.e., NAS and RL problems) show that MCTS-VS is better than the previous variable selection method Dropout [21], and can also achieve the competitive performance to state-of-the-art BO algorithms.
Furthermore, its running time is small due to the advantage of variable selection.
We also observe that MCTS-VS can select important variables, explaining its good performance based on our theoretical analysis.
Background 2.1 Bayesian Optimization
We consider the problem max x ∈X f ( x ) , where f is a black-box function and
X ⊆
R D is the domain.
The basic framework of BO contains two critical components: a surrogate model and an acquisition function.
GP is the most popular surrogate model.
Given the sampled data points { ( x i , y i ) } t − 1 i =1 , where y i =
f ( x i )
+ ϵ i and ϵ i ∼ N (0 , η 2 ) is the observation noise, GP at iteration t seeks to infer f ∼ GP ( µ ( · ) , k ( · , · ) + η 2 I ) , specified by the mean µ ( · ) and covariance kernel k ( · , · ) , where I is the identity matrix of size D .
After that, an acquisition function, e.g., Probability of Improvement (PI) [19], Expected Improvement (EI) [16] or Upper Confidence Bound (UCB) [36], is used to determine the next query point x t while balancing exploitation and exploration.
2.2 High-dimensional Bayesian Optimization
Scaling BO to high-dimensional problems is a challenge due to the curse of dimensionality and the computation cost.
As the dimension increases, the search space increases exponentially, requiring more samples, and thus more expensive evaluations, to find a good solution.
Furthermore, the computation cost of updating the GP model and optimizing the acquisition function will be very time-consuming [30].
There have been a few common approaches to tackle high-dimensional BO with different assumptions.
Decomposition.
Assuming that the function can be decomposed into the sum of low-dimensional functions with disjoint subspaces, Kandasamy et al.
[17] proposed the Add-GP-UCB algorithm to optimize those low-dimensional functions separately, which was further generalized to overlapping subspaces [26, 31].
Wang et al.
[41] proposed ensemble BO that uses an ensemble of additive GP models for scalability.
Han et al.
[13] constrained the dependency graphs of decomposition to tree structures to facilitate the decomposition learning and optimization.
For most problems, however, the decomposition is unknown, and also difficult to learn.
Embedding.
Assuming that only a few dimensions affect the high-dimensional function significantly, embedding-based methods embed the high-dimensional space into a low-dimensional subspace, and optimize in the subspace while projecting the point back for evaluation.
REMBO and its variants use a random matrix to embed the search space into a low-dimensional subspace [3, 4, 40].
Nayebi et al.
[27] used a hash-based method for embedding.
Letham et al.
[20] proposed ALEBO, focusing on several misconceptions in REMBO to improve the performance.
The VAE-based approaches were also employed to project a structured input space (e.g., graphs and images) to a low-dimensional subspace [12, 22].
Variable Selection.
Based on the same assumption as embedding, variable selection methods iteratively select a subset of variables to build a low-dimensional subspace and optimize through
BO.
The selected variables can be viewed as important variables that are valuable for exploitation, or having high uncertainty that are valuable for exploration.
A classical method is Dropout [21], which randomly chooses d variables in each iteration.
When evaluating the sampled point, the values of those unselected variables are obtained by random sampling or using historical information.
Note that variable selection can be faster than embedding, because the embedding cost (e.g., matrix inversion) is time-consuming for high-dimensional optimization.
Both embedding and variable selection methods need to specify the parameter d , i.e., the dimension of low-dimensional subspace, which will affect the performance significantly, but is not easy to set.
There are also some methods to improve the basic components of BO directly for high-dimensional problems.
For example, DNGO [35] uses the neural network as an alternative of GP to speed up inference; BO-PP [29] generates pseudo-points (i.e., data points whose objective values are not evaluated) to improve the GP model; SAASBO [9] uses sparsity-inducing prior to perform variable selection implicitly, making the coefficients of unimportant variables near to zero and thus restraining over-exploration on these variables. Note that different from Dropout and our proposed MCTS-VS, SAASBO still optimizes all variables, and also due to its high computational cost of inference, it is very time-consuming as reported in [9].
These methods can be combined with the above-mentioned dimensionality reduction methods, which may bring further improvement.
2.3 Monte Carlo Tree Search
MCTS [5] is a tree search algorithm based on random sampling, and has shown great success in high-dimensional tasks, such as Go [33, 34].
A tree node represents a state, describing the current situation, e.g., the position in path planning.
Each tree node X stores a value v X representing its goodness, and the number n X that it has been visited.
They are used to calculate UCB [1], i.e., [Equation]
where C p is a hyper-parameter, and n p is the number of visits of the parent of X .
UCB considers both exploitation and exploration, and will be used for node selection.
MCTS iteratively selects a leaf node of the tree for expansion.
Each iteration can be divided into four steps: selection , expansion , simulation and back-propagation .
Starting from the root node, selection is to recursively select a node with larger UCB until a leaf node, denoted as X .
Expansion is to execute a certain action in the state represented by X and transfer to the next state, e.g., move forward and arrive at a new position in path planning.
We use the child node Y of X to represent the next state.
Simulation is to obtain the value v Y via random sampling.
Back-propagation is to update the value and the number of visits of Y ’s ancestors.
To tackle high-dimensional optimization, Wang et al.
[38] proposed LA-MCTS, which applies MCTS to iteratively partition the search space into small sub-regions, and optimizes only in the good subregions.
That is, the root of the tree represents the entire search space Ω , and each tree node X represents a sub-region Ω X .
The value v X is measured by the average objective value of the sampled points in the sub-region Ω X .
In each iteration, after selecting a leaf node X , LA-MCTS performs the optimization in Ω X by vanilla BO [32] or TuRBO [10], and the sampled points are used for clustering and classification to bifurcate Ω X into two disjoint sub-regions, which are “good” and “bad”, respectively.
Note that the sub-regions are generated by dividing the range of variables, and their dimensionality does not decrease, which is still the number of all variables.
Wang et al.
[38] have empirically shown the good performance of LA-MCTS.
However, as the dimension increases, the search space increases exponentially, and more partitions and evaluations are required to find a good solution, making the application of LA-MCTS to high-dimensional optimization still limited.
MCTS-VS Method
In this section, we propose a Variable Selection method based on MCTS for high-dimensional
BO, briefly called MCTS-VS.
The main idea is to apply MCTS to iteratively partition all variables into important and unimportant ones, and perform BO only for those important variables.
Let [ D ] = { 1 , 2 , . . .
, D } denote the indexes of all variables x , and x M denote the subset of variables indexed by M ⊆
[ D ] .
We first introduce a D -dimensional vector named variable score , which is a key component of
MCTS-VS.
Its i -th element represents the importance of the i -th variable x i .
During the running process of MCTS-VS, after optimizing a subset x M of variables where M ⊆
[ D ] denotes the indexes of the variables, a set D of sampled points will be generated, and the pair ( M , D ) will be recorded into a set D , called information set .
The variable score vector is based on D , and calculated as
[Equation]
where the function g : 2 [ D ] → { 0 , 1 } D gives the Boolean vector representation of a variable index subset M ⊆
[ D ] (i.e., the i -th element of g ( M ) is 1 if i ∈ M , and 0 otherwise), and / is the elementwise division.
Each dimension of (cid:80) ( M , D ) ∈ D (cid:80) ( x i ,y i ) ∈D y
i · g ( M ) is the sum of query evaluations using each variable, and each dimension of (cid:80) ( M , D ) ∈ D |D| · g ( M ) is the number of queries using each variable.
Thus, the i -th element of variable score s , representing the importance of the i -th variable x
i , is actually measured by the average goodness of all the sampled points that are generated by optimizing a subset of variables containing x i .
The variable score s will be used to define the value of each tree node of MCTS as well as for node expansion.
In MCTS-VS, the root of the tree represents all variables.
A tree node X represents a subset of variables, whose index set is denoted by A X ⊆
[ D ] , and it stores the value v X and the number n X of visits, which are used to calculate the value of UCB as in Eq.
The value v X is defined as the average score (i.e., importance) of the variables contained by X , which can be calculated by s · g ( A X ) / |
A X | , where g ( A X ) is the Boolean vector representation of A X and | A X | is the size of A X , i.e., the number of variables in node X .
At each iteration, MCTS-VS first recursively selects a node with larger UCB until a leaf node (denoted as X ), which is regarded as containing important variables.
Note that if we optimize the subset x A X of variables represented by the leaf X directly, the variables in x A X will have the same score (because they are optimized together), and their relative importance cannot be further distinguished.
Thus,
MCTS-VS uniformly selects a variable index subset M from A X at random, and employs BO to optimize x M as well as x A X \ M ; this process is repeated for several times.
After that, the information set D will be augmented by the pairs of the selected variable index subset M (or A X \ M ) and the corresponding sampled points generated by BO.
The variable score vector s will be updated using this new D .
Based on s , the variable index set A X represented by the leaf X will be divided into two disjoint subsets, containing variables with larger and smaller scores (i.e., important and unimportant variables), respectively, and the leaf X will be bifurcated into two child nodes accordingly.
Finally, the v values of these two children will be calculated using the variable score vector s , and backpropagation will be performed to update the v value and the number of visits of the nodes along the current path of the tree.
MCTS-VS can be equipped with any specific BO optimizer, resulting in the concrete algorithm
MCTS-VS-BO, where BO is used to optimize the selected subsets of variables during the running of MCTS-VS.
Compared with LA-MCTS [38], MCTS-VS applies MCTS to partition the variables instead of the search space, and thus can be more scalable.
Compared with the previous variable selection method Dropout [21], MCTS-VS can select important variables automatically instead of randomly selecting a fixed number of variables in each iteration.
Next we introduce it in detail.
3.1 Details of MCTS-VS
The procedure of MCTS-VS is described in Algorithm 1.
In line 1, it first initializes the information set D .
In particular, a variable index subset M i is randomly sampled from [ D ] , and the Latin hypercube sampling [24] is used to generate two sets (denoted as D i and D ¯ i ) of N s points to form the two pairs of ( M i , D i ) and ( ¯ M i , D ¯ i ) , where ¯ M i =
[ D ] \ M i .
This process will be repeated for N v times, resulting in the initial D
= { ( M i , D i ) , ( ¯ M i , D ¯ i ) } N v i =1 .
The variable score vector s is calculated using this initial D in line 3, and the Monte Carlo tree is initialized in line 4 by adding only a root node, whose v value is calculated according to s and number of visits is 0.
MCTS-VS uses the variable t to record the number of evaluations it has performed, and thus t is set to 2 × N v × N s in line 5 as the initial D contains 2 × N v × N s sampled points in total.
In each iteration (i.e., lines 7–28) of MCTS-VS, it selects a leaf node X by UCB in line 10, and optimizes the variables (i.e., x A X ) represented by X in lines 13–23.
Note that to measure the relative importance of variables in x A X , MCTS-VS optimizes different subsets of variables of x A X instead of x A X directly.
That is, a variable index subset M is randomly sampled from A X in line 14, Let A X denote the indexes of the subset of variables represented by X and the corresponding subset x M of variables is optimized by BO in lines 15–16.
The data points { ( x i M , y i ) } ti =1 sampled-so-far is used to fit a GP model, and N s (called sample batch size ) new points { x t + i M } N s i =1 are generated by maximizing an acquisition function.
Note that this is a standard BO procedure, which can be replaced by any other variant.
To evaluate x t + i M , we need to fill in the values of the other variables x t
+ i
[ D ] \ M , which will be explained later.
After evaluating x t + i =
[ x t + i M , x t + i
[ D ] \ M ] in line 18, the information set D is augmented with the new pair of ( M , { ( x t
+ i , y t + i ) }
N s i
=1 ) in line 19, and t is increased by N s accordingly in line 21.
For fairness, the complement subset x ¯ M of variables, where ¯ M =
A X \ M , is also optimized by the same way, i.e., lines 15–21 of Algorithm 1 is repeated for ¯ M .
The whole process of optimizing x M and x ¯ M in lines 14–22 will be repeated for N v times, which is called batch size of variable index subset .
To fill in the values of the un-optimized variables in line 17, we employ the best- k strategy, which utilizes the best k data points sampled-so-far, denoted as { ( x ∗ j , y ∗ j ) } kj =1 .
That is, { y ∗ j } kj =1 are the k largest objective values observed-so-far.
If the i -th variable is un-optimized, its value will be uniformly selected from { x ∗ ji } kj =1 at random.
Thus, MCTS-VS needs to store the best k data points in line 2 after initializing the information set D , and update them in line 20 after augmenting D .
Other direct “fill-in” strategies include sampling the value randomly, or using the average variable value of the best k data points. The superiority of the employed best-
k strategy will be shown in the experiments in Appendix D.
After optimizing the variables x A X represented by the selected leaf X , the variable score vector s measuring the importance of each variable will be updated using the augmented D in line 24.
If the number | A X | of variables in the leaf X is larger than a threshold N split (i.e., line 25), A X will be divided into two subsets.
One contains those “important” variable indexes with score larger than the average score of x A X , and the other contains the remaining “unimportant” ones.
The leaf X will be bifurcated into a left child Y and a right child Z in line 26, containing those important and unimportant variables, respectively.
Meanwhile, v Y and v Z will be calculated according to s , and the number of visits is 0, i.e., n Y = n Z = 0 .
Finally, MCTS-VS performs back-propagation in line 28 to re-calculate the v value and increase the number of visits by 1 for each ancestor of Y and Z .
MCTS-VS will run until the number t of performed evaluations reaches the budget N e .
Note that as the Monte Carlo tree may be built improperly, we use a variable n bad to record the number of visiting a right child node (regarded as containing unimportant variables), measuring the goodness of the tree.
In line 5 of Algorithm 1, n bad is initialized as 0.
During the procedure of selecting a leaf node by UCB in line 10, n bad will be increased by 1 once visiting a right child node, which is updated in line 12.
If n bad is larger than a threshold N bad (i.e., line 7), the current tree is regarded as bad, and will be re-initialized in line 8.
Furthermore, the frequency of re-initialization can be used to indicate whether MCTS-VS can do a good variable selection for the current problem.
For ease of understanding, we also provide an example illustration of MCTS-VS in Appendix A.
Theoretical Analysis
Although it is difficult to analyze the regret of MCTS-VS directly, we can theoretically analyze the influence of general variable selection by adopting the acquisition function GP-UCB.
The considered general variable selection framework is as follows: after selecting a subset of variables at each iteration, the corresponding observation data (i.e., the data points sampled-so-far where only the selected variables are used) is used to build a GP model, and the next data point is sampled by maximizing GP-UCB.
We use M t to denote the sampled variable index subset at iteration t , and let | M t | = d t .
Regret Analysis.
Let x ∗ denote an optimal solution.
We analyze the cumulative regret R T =
(cid:80) Tt =
( f ( x ∗ ) − f ( x t )) , i.e., the sum of the gap between the optimum and the function values of the selected points by iteration T .
To derive an upper bound on R T , we pessimistically assume that the worst function value, i.e., min x [ D ] \ M t f ([ x M t , x [ D ] \ M t ]) , given x M t is returned in evaluation.
As in [21, 36], we assume that X ⊂ [0 , r ] D is convex and compact, and f satisfies the following Lipschitz assumption.
Assumption 4.1.
The function f is a GP sample path.
For some a, b > 0 , given L > 0 , the partial derivatives of f satisfy that ∀ i ∈
[ D ] , ∃ α i ≥ 0 , [Equation]
Based on Assumption 4.1, we define α ∗ i to be the minimum value of α i such that Eq. (3) holds, which characterizes the importance of the i -th variable x i .
The larger α ∗ i , the greater influence of x i on the function f .
Let α max = max i ∈
[ D ] α ∗ i .
Theorem 4.2 gives an upper bound on the cumulative regret R T with high probability for general variable selection methods.
The proof is inspired by that of GP-UCB without variable selection [36] and provided in Appendix B.1.
If we select all variables each time (i.e., ∀ t : M t =
[ D ] ) and assume ∀ i : α ∗ i ≤ 1 , the regret bound Eq.
(4) becomes R T ≤ (cid:112) C
1 Tβ ∗ T γ T
+ 2 , which is consistent with [36].
Note that ∀ t : | M t | =
d t =
D in this case, which implies that β t increases with t , leading to β ∗ T = β T .
We can see that using variable selection will increase R T by 2 (cid:80)
Tt =1 (cid:80) i
∈
[ D ] \ M t α ∗ i Lr , related to the importance (i.e., α ∗ i ) of unselected variables at each iteration.
The more important variables unselected, the larger R T .
Meanwhile, the term (cid:112) C 1 Tβ ∗ T γ T will decrease as β ∗ T relies on the number d t of selected variables positively.
Ideally, if the unselected variables at each iteration are always unrelated (i.e., α ∗ i =0 ), the regret bound will be better than that of using all variables [36].
Theorem 4.2.
∀ δ ∈ (0 , 1) , let β t
=
2 log(4 π t /δ )
+ 2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) and
L = b (cid:112) log(4 Da/δ ) , where r is the upper bound on each variable, and { π t } t ≥ 1 satisfies (cid:80) t ≥ 1 π − 1 t = and π t > 0 .
Let β ∗ T = max 1 ≤ i ≤ T β t .
At iteration T , the cumulative regret
[Equation]
holds with probability at least 1 − δ , where C 1 is a constant, γ T =max |D| =
T I ( y D , f D ) , I ( · , · ) is the information gain, and y D and f D are the noisy and true observations of a set D of points, respectively. By selecting d variables randomly at each iteration and assuming that r = 1 and ∀ i : α ∗ i ≤ 1 , it has been proved [21] that the cumulative regret of Dropout satisfies[-R T ≤ √ C Tβ T γ T +
 +2 TL-]
[Equation]
In this case, we have d t = | M t | =
d , r = 1 and ∀ i : α ∗ i ≤ 1 .
Thus, Eq.
(4) becomes [Equation]
Note that β ∗ T = β T here, as β t increases with t given d t = d . This implies that our bound Eq.
(4) for general variable selection is a generalization of Eq. (5) for Dropout [21]. Computational Complexity Analysis. The computational complexity of one iteration of BO depends on three critical components: fitting a GP surrogate model, maximizing an acquisition function and evaluating a sampled point.
If using the squared exponential kernel, the computational complexity of fitting a GP model at iteration t is O
( t 3 + t 2 d t ) .
Maximizing an acquisition function is related to the optimization algorithm.
If we use the Quasi-Newton method to optimize GP-UCB, the computational complexity is O ( m ( t 2 + td t + d 2 t ))
[28], where m denotes the Quasi-Newton’s running rounds. The cost of evaluating a sampled point is fixed. Thus, by selecting only a subset of variables, instead of all variables, to optimize, the computational complexity can be decreased significantly. The detailed analysis is provided in Appendix B.2.
Insight.
The above regret and computational complexity analyses have shown that variable selection can reduce the computational complexity while increasing the regret.
Given the number d t of variables to be selected, a good variable selection method should select as important variables as possible, i.e., variables with as large α ∗ i as possible, which may help to design and evaluate variable selection methods.
The experiments in Section 5.1 will show that MCTS-VS can select a good subset of variables while maintaining a small computational complexity.
Experiment
To examine the performance of MCTS-VS, we conduct experiments on different tasks, including synthetic functions, NAS-bench problems and MuJoCo locomotion tasks, to compare MCTS-VS with other black-box optimization methods.
For MCTS-VS, we use the same hyper-parameters except C p , which is used for calculating UCB in Eq.
For Dropout and embedding-based methods, we set the parameter d to the number of valid dimensions for synthetic functions, and a reasonable value for real-world problems.
The hyper-parameters of the same components of different methods are set to the same.
We use five identical random seeds (2021–2025) for all problems and methods.
More details about the settings can be found in Appendix C.
Our code is available at https://github.com/lamda-bbo/MCTS-VS .
5.1 Synthetic Functions
We use Hartmann ( d = 6 ) and Levy ( d = 10 ) as the synthetic benchmark functions, and extend them to high dimensions by adding unrelated variables as [20, 27, 40].
For example, Hartmann 6 _ 300 has the dimension D = 300 , and is generated by appending 294 unrelated dimensions to Hartmann.
The variables affecting the value of f are called valid variables .
Effectiveness of Variable Selection.
Dropout [21] is the previous variable selection method which randomly selects d variables in each iteration, while our proposed MCTS-VS applies MCTS to automatically select important variables.
We compare them against vanilla BO [32] without variable selection.
The first two subfigures in Figure 1 show that Dropout-BO and MCTS-VS-BO are better than vanilla BO, implying the effectiveness of variable selection.
We can also see that MCTS-VS-BO performs the best, implying the superiority of MCTS-based variable selection over random selection.
We also equip MCTS-VS and Dropout with the advanced BO algorithm TuRBO [10], resulting in
MCTS-VS-TuRBO and Dropout-TuRBO.
The last two subfigures in Figure 1 show the similar results except that MCTS-VS-TuRBO needs more evaluations to be better than Dropout-TuRBO.
This is because TuRBO costs more evaluations than BO on the same selected variables, and thus needs more evaluations to generate sufficient samples for an accurate estimation of the variable score in Eq. [Figure]
[Figure]
Comparison with State-of-The-Art Methods.
We compare MCTS-VS with the state-of-the-art methods, including TuRBO [10], LA-MCTS-TuRBO [38], SAASBO [9], HeSBO [27], ALEBO [20] and CMA-ES [14].
TuRBO fits a collection of local models to optimize in the trust regions for overcoming the homogeneity of the global model and over-exploration.
LA-MCTS-TuRBO applies MCTS to partition the search space and uses TuRBO to optimize in a small sub-region.
SAASBO uses sparsity-inducing prior to select variables implicitly.
HeSBO and ALEBO are state-of-the-art embedding methods.
CMA-ES is a popular evolutionary algorithm.
We also implement VAE-BO by combining VAE [18] with vanilla BO directly, as a baseline of learning-based embedding.
For MCTS-VS, we implement the two versions of MCTS-VS-BO and MCTS-VS-TuRBO, i.e., MCTS-VS equipped with vanilla BO and TuRBO.
As shown in Figure 2, MCTS-VS can achieve the best performance except on Levy 10 _ 100 , where it is a little worse than TuRBO.
For low-dimensional functions (e.g., D = 100 for Levy 10 _ 100 ), TuRBO can adjust the trust region quickly while MCTS-VS needs samples to estimate the variable score.
But as the dimension increases, the search space increases exponentially and it becomes difficult for TuRBO to adjust the trust region; while the number of variables only increases linearly, making
MCTS-VS more scalable.
SAASBO has similar performance to MCTS-VS due to the advantage of sparsity-inducing prior.
HeSBO is not stable, which has a moderate performance on Hartmann but a relatively good performance on Levy.
Note that we only run SAASBO and ALEBO forevaluations on Hartmann functions because it has already taken more than hours to finish one iteration when the number of samples is large. More details about runtime are shown in Table 1.
VAE-BO has the worst performance, suggesting that the learning algorithm in high-dimensional BO needs to be designed carefully.
We also conduct experiments on extremely low and high dimensional variants of
Hartmann (i.e., Hartmann 6 _ 100 and Hartmann 6 _ 1000 ), showing that MCTS-VS still performs well, and perform the significance test by running each method more times.
Please see Appendix E.
[Figure]
Next, we compare the practical running overheads of these methods.
We run each method forevaluations independently using 30 different random seeds
[Figure]
and calculate the average wall clock time.
The results are shown in Table 1.
As expected, when using variable selection (i [Figure] e., Dropout and
MCTS-VS)
, the time is less than that of Vanilla BO or TuRBO, because we only optimize a subset of variables.
MCTS-VS is a little slower than Dropout, which is because MCTS-VS needs to build the search tree and calculate the variable score, while Dropout only randomly selects variables.
MCTSVS is much faster than LA-MCTS-TuRBO, showing the advantage of partitioning the variables to partitioning the search space. SAASBO optimizes all variables instead of only a subset of variables [Table] [Table]
and uses No-U-Turn sampler (NUTS) to inference, consuming × 500 – × 1000 time.
HeSBO and ALEBO consume × 10 – × 500 time compared with the variable selection methods.
CMA-ES is very fast because it does not need to fit a GP model or optimize an acquisition function.
The reasons for the small running overhead of MCTS-VS can be summarized as follows: 1) it only optimizes a selected subset of variables; 2) the depth of the search tree is shallow, i.e., O (log D ) in expectation and less than D in the worse case; 3) the variable score vector in Eq.
(2) is easy to calculate for bifurcating a tree node.
Why MCTS-VS Can Perform Well.
The theoretical results have suggested that a good variable selection method should select as important variables as possible.
Thus, we compare the quality of the variables selected by MCTS-VS and Dropout (i.e., random selection), measured by the recall d ∗ t /d , where d is the number of valid variables, and d ∗ t is the number of valid variables selected at iteration t .
Dropout randomly selects d variables at each iteration, and thus, the recall is d/D in expectation.
For MCTS-VS, we run MCTS-VS-BO for 600 evaluations on five different random seeds, and calculate the average recall.
As shown in Table 2, the average recall of MCTS-VS is much larger than that of Dropout, implying that MCTS-VS can select better variables than random selection, and thus achieve a good performance as shown before.
Meanwhile, the recall between 0 .and 0 .
433 of MCTS-VS also implies that the variable selection method could be further improved.
5.2 Real-World Problems
We further compare MCTS-VS with the baselines on real-world problems, including NAS-Bench101 [43], NAS-Bench-201 [7], Hopper and Walker2d.
NAS-Bench problems are popular benchmarks in high-dimensional BO.
Hopper and Walker2d are robot locomotion tasks in MuJoCo [37], which is a popular black-box optimization benchmark and much more difficult than NAS-Bench problems.
The experimental results on more real-world problems can refer to Appendix E.
NAS-Bench Problems.
NAS-Bench-101 is a tabular data set that maps convolutional neural network architectures to their trained and evaluated performance on CIFAR-10, and we create a constrained problem with D = 36 in the same way as [20].
NAS-Bench-201 is an extension to NAS-Bench-101, leading to a problem with D = 30 but without constraints.
Figure 3 shows the results with the wall clock time as the x -axis, where the gray dashed line denotes the optimum.
The results using the number of evaluations as the x -axis are provided in Appendix E, showing that the performance of BO-style methods is similar, as already observed in [20].
This may be because there are many structures whose objective values are close to the optimum. But when considering the actual runtime, MCTS-VS-BO is still clearly better as shown in Figure 3, due to the advantage of variable selection.
We also provide results on more NAS-Bench problems, including NAS-Bench-1Shot1 [44],
TransNAS-Bench-101 [8] and NAS-Bench-ASR [25] in Appendix E.
[Figure]
[Figure]
MuJoCo Locomotion Tasks.
Next we turn to the more difficult MuJoCo tasks in RL.
The goal is to find the parameters of a linear policy maximizing the accumulative reward.
Different from previous problems, the objective f (i.e., the accumulative reward) is highly stochastic here, making it difficult to solve.
We use the mean of three independent evaluations to estimate f [Figure] and limit the evaluation budget to 1200 due to expensive evaluation.
Note that we do not run SAASBO, ALEBO, and VAE-BO because SAASBO and ALEBO are extremely time-consuming, and VAE-BO behaves badly in previous experiments.
The results are shown in Figure 4.
TuRBO behaves well on Hopper with a low dimension D = [Figure] and MCTS-VS-TuRBO, combining the advantage of variable selection and TuRBO, achieves better performance, outperforming all the other baselines.
On Walker2d with a high dimension D = 102 , MCTS-VS-BO performs the best, because of the good scalability.
Most methods have large variance due to the randomness of f .
For HeSBO, we have little knowledge about the parameter d , and use 10 and 20 for Hopper and Walker2d, respectively.
Its performance may be improved by choosing a better d , which, however, requires running the experiment many times, and is time-consuming.
Note that on the two MuJoCo tasks, Hopper and Walker2d, each variable is valid.
The good performance of MCTS-VS may be because optimizing only a subset of variables is sufficient for achieving the goal and MCTS-VS can select them. For example, the Walker2D robot consists of four main body parts: a torso, two thighs, two legs and two feet, where adjacent ones are connected by two hinges. The goal is to move forward by optimizing the hinges, each of which is valid. But even locking the hinges between legs and feet, the robot can still move forward by optimizing the other hinges.
This is similar to that when the ankles are fixed, a person can still walk.
Further Studies. We further perform sensitivity analysis about the hyper-parameters of MCTS-VS, including the employed optimizer, “fill-in” strategy, C p for calculating UCB in Eq. (1), number 2 × N v × N s of sampled data in each iteration, threshold N bad for re-initializing a tree and N split for splitting a tree node.
Please see Appendix D. We conduct additional experiments in Appendix E, including experiments on synthetic functions depending on a subset of variables to various extent and with increasing ratio of valid variables, examination of combining MCTS-VS with SAASBO (which can be viewed as a hierarchical variable selection method), and comparison with other variable selection methods (e.g., LASSO).
Conclusion
In this paper, we propose the MCTS-VS method for variable selection in high-dimensional BO, which uses MCTS to recursively partition the variables into important and unimportant ones, and only optimizes those important variables.
Theoretical results suggest selecting as important variables as possible, which may be of independent interest for variable selection.
Comprehensive experiments on synthetic, NAS-bench and MuJoCo problems demonstrate the effectiveness of MCTS-VS.
However, MCTS-VS relies on the assumption of low effective dimensionality, and might not work well if the percentage of valid variables is high.
The amount of hyper-parameters might be another limitation, though our sensitivity analysis has shown that the performance of MCTS-VS is not sensitive to most hyper-parameters.
The current theoretical analysis is for general variable selection, while it will be very interesting to perform specific theoretical analysis for MCTS-VS. Acknowledgement
The authors would like to thank reviewers for their helpful comments and suggestions.
This work was supported by the NSFC (62022039, 62276124) and the Fundamental Research Funds for the
Central Universities (0221-14380014).
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See the end of Section 5.1 and the last paragraph of the paper.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 4.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix B. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Appendix C, and the code is provided in GitHub.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Appendix C.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We show error bars by the length of vertical bars in the figures.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[Yes] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
[Figure]
Figure 5 gives an example of how MCTS-VS bifurcates a leaf node.
Assume that we are to optimize a problem with dimension D = 9 , and the variables are denoted as x 1 , x 2 , . . .
, x 9 .
The Monte
Carlo tree shown in the middle of Figure 5 now has three nodes, i.e., A , B and C , denoted as the solid circles.
The root A contains all the nine variables.
The current variable score vector s =
[8 . 5 , 8 , 5 , 7 , 3 , 3 , 7 , 10 . 7 , 4 . 5] , which is represented by the bar graph as shown in the left of
Figure 5.
For each i , the value of s i represents the importance of the corresponding variable x i .
The blue and gray bars denote the important and unimportant variables, respectively, which are contained by the leaf nodes B and C , respectively.
That is, the leaf B contains x 1 , x 2 , x 4 , x 7 , x 8 , and C contains the remaining x 3 , x 5 , x 6 , x 9 .
The current v values (i.e., the average scores of the contained variables) of the three nodes A , B and C are v A = (8 .
+ 8 + 5 + 7 + 3 + 3 + 7 + 10 .
7 + 4 . 5) / 9 = 6 . 3 , v B = (8 . 5 + 8 + 7 + 7 + 10 . 7) / 5 = 8 .
24 and v C = (5 + 3 + 3 + 4 . 5) / 4 = 3 . 875 , respectively.
For the number that they have been visited, we have n A = 1 , n B = 0 and n C = 0 .
MCTS-VS starts from the root node A at one iteration and recursively selects a node with a larger
UCB value until a leaf node.
According to the way of calculating UCB in Eq. (1), the UCB values of the leaf nodes B and C are both ∞ as n B = n C = 0 .
In this case, MCTS-VS will select B and C randomly.
Assume that B is selected.
The variables (i.e., x 1 , x 2 , x 4 , x 7 and x 8 ) contained by B will then be optimized by BO with A B = { 1 , 2 , 4 , 7 , 8 } , as in lines 13–23 in Algorithm 1.
After that, the variable score vector s will be re-calculated, which is assumed to be [9 , 8 . 5 , 5 , 11 , 3 , 3 , 11 , 11 . 2 , 4 . 5] , as shown in the right of Figure 5.
The average score of the five variables in B is denoted as the orange horizontal line, calculated by (9 + 8 .
5 + 11 + 11 + 11 . 2) / 5 = 10 .
We can see that the variables x 4 , x 7 and x 8 have score larger than the average value 10 .
14 , which are regarded as more important variables in B .
We use the left child D to represent these variables.
The scores of variables x 1 and x 2 are smaller than the average, which are regarded as less important variables in B .
We use the right child E to represent them.
Thus, the node B has been partitioned into two children D and E , denoted as the dashed circles in Figure 5.
The v values and the number of visits of these two new leaf nodes are then calculated.
The v value of a node is the average score of the contained variables.
Thus, v D is the average score of x 4 , x 7 and x 8 , i.e., (11 + 11 + 11 . 2) / 3 = 11 .
067 , and v E is the average score of x 1 and x 2 , i.e., (9 + 8 . 5) / 2 = 8 .
For the number of visits, obviously n D = n E = 0 .
Finally, back-propagation is performed to update the v value and the number of visits of the nodes along the path from the root A to the node B .
v A is the average score of all the variables, i.e., (9 + 8 .
5 + 5 + 11 + 3 + 3 + 11 + 11 .
2 + 4 . 5) / 9 = 7 . 356 .
v B is the average score of x 1 , x 2 , x 4 , x 7 and x 8 , i.e., (9 + 8 .
5 + 11 + 11 + 11 . 2) / 5 = 10 .
Their number of visits will be increased by 1.
That is, n A = 2 and n B = 1 .
By far, one iteration of MCTS-VS has been finished, and this process will be performed iteratively.
B Details of Theoretical Analysis B.1 Detailed Proof of Theorem 4.
The proof is inspired by [36].
To prove the upper bound on the cumulative regret R T in Theorem 4.2, we analyze the instantaneous regret r t = f ( x ∗ ) − f ( x t M t ) , i.e., the gap between the function values of the optimal point x ∗ and the sampled point x t M t at iteration t .
Note that R T =
(cid:80) Tt =1 r t .
Let µ t − 1 ( · ) and σ 2 t − 1 ( · ) denote the posterior mean and variance after running t − 1 iterations, respectively.
Lemma B.1 gives a confidence bound on f ( x t M t ) , leading to a lower bound on f ( x t M t ) , i.e., f ( x t M t ) ≥ µ t − 1 ( x t M t ) − β 1 / 2 t σ t − 1 ( x t M t ) .
Note that M t denotes the sampled variable index subset at iteration t , and | M t | = d t .
Lemma B.1.
∀ δ ∈ (0 , 1) , ∀ t ≥ 1 , let β t = 2 log( π t /δ ) , where (cid:80) t ≥ 1 π − 1 t = 1 , π t > 0 .
Then, ∀ t ≥ 1 ,
[Equation]
holds with probability at least 1 − δ , where x t M t is the point obtained at iteration t .
Proof.
At iteration t , f ( x t M t ) ∼ N ( µ t − 1 ( x t M t ) , σ 2 t − 1 ( x t M t )) , and thus, Y = f
( x t M t ) − µ t − 1 ( x t M t ) σ t − 1 ( x t M t ) ∼
N (0 , 1) .
We have
[Equation]
Using the union bound for all t ∈ N , we have
[Equation]
where the equality holds by (cid:80) t ≥ 1 π − 1 t = 1 .
Thus, the lemma holds.
Next we are to analyze the upper bound on f ( x ∗ ) , which can be represented as ( f ( x ∗ )
− f ( x ∗ M t ))
+ f ( x ∗ M t ) , where x ∗ M t denotes the point obtained by projecting x ∗ onto M t .
The first term f ( x ∗ ) − f ( x ∗ M t ) can be upper bounded by Assumption 4.1.
To upper bound the second term f ( x ∗ M t ) , we need to discretize the decision space X M t at iteration t into ˜ X M t , where | ˜ X M t | = ( τ t ) d t , i.e., we divide each variable of X M t into τ t parts equally.
Let ˜ x ∗ M t denote the point closest to x ∗ M t in the discretized space ˜ X M t .
Then, we can write f ( x ∗ M t ) as ( f ( x ∗ M t ) − f (˜ x ∗ M t ))
+ f (˜ x ∗ M t ) .
The first term f ( x ∗ M t ) − f (˜ x ∗ M t ) again can be upper bounded by Assumption 4.1.
Lemma B.2 gives a confidence bound on f (˜ x M t ) for any discretized point
˜ x
M t ∈ ˜
X M t , leading to an upper bound on f (˜ x ∗ M t ) , i.e., f (˜ x ∗ M t ) ≤ µ t − 1 (˜ x ∗ M t )
+ β 1 / 2 t σ t − 1 (˜ x ∗ M t ) .
Lemma B.2.
∀ δ ∈ (0 , 1) , ∀ t ≥ 1 , let β t = 2 log( | ˜ X M t | π t /δ ) , where (cid:80) t ≥ 1 π − 1 t = 1 , π t > 0 .
Then, ∀ t ≥ 1 , ∀ ˜
x M t ∈ ˜
X M t ,
[Equation]
σ
[Equation]
) holds with probability at least 1 − δ .
Proof.
Similar to Lemma B.1, we can derive
[Equation]
Using the union bound for all t ∈ N and ˜ x
M t ∈ ˜
X M t , we have
[Equation]
Thus, the lemma holds.
Now, we can upper bound f ( x ∗ M t ) based on Assumption 4.1 and Lemma B.2, as shown in Lemma B.3.
Note that x ∗ M t denotes the point obtained by projecting x ∗ onto M t , and ˜ x ∗ M t denotes the point closest to x ∗ M t in ˜ X M t .
Lemma B.3.
∀ δ ∈ (0 , 1) , t ≥ 1 , let β t = 2 log(2 π t /δ )
+ 2 d t log (cid:16) d t t 2 br (cid:113) log 2 Daδ (cid:17) , where (cid:80) t ≥ 1 π − 1 t = 1 , π t > 0 .
Set τ t
=
d t t 2 br (cid:113) log 2 Da δ and L = b (cid:113) log 2 Da δ .
Then, ∀ t ≥ 1 ,
[Equation]
holds with probability at least 1 − δ .
Proof.
First, we have
[Equation]
By Assumption 4.1 with L = b (cid:113) log 2 Daδ , we have ∀ x , y ∈ X , with probability at least 1 − D · ae − ( L/b ) 2 = 1 − δ/ 2 ,
[Equation]
where the second inequality holds by X ⊂ [0 , r ] D , and the last inequality holds by α max =
max i ∈
[ D ] α ∗ i .
Thus, it holds with probability at least 1 − δ/ 2 that
[Equation]
By Lemma B.2 with β t = 2 log(2( τ t )
d t π t /δ )
= 2 log(2 | ˜
X M t | π t /δ ) , we have, with probability at least 1 − δ/ 2 ,
[Equation]
[Equation]
Applying Eqs.
(9) and (10) to Eq.
(7), it holds with probability at least 1 − δ that
[Equation]
where the second inequality holds by | M t |
= d t and the way of discretization (i.e., each variable is discretized into τ t parts equally), and the last inequality holds by the definition of τ t and L .
Thus, the lemma holds.
Lemma B.3 implies an upper bound on f ( x ∗ M t ) , i.e., f ( x ∗ M t ) ≤ µ [Equation] − 1 (˜ x [Equation]
M [Equation] + β [Equation]
/ 2 t σ
[Equation]
+ α max /t 2 + (cid:80) i ∈
[ D ] \ M t α ∗ i
Lr .
Combining this upper bound on f ( x ∗ M t ) with f ( x ∗ ) − f ( x ∗ M t ) (which can be upper bounded by Assumption 4.1), we can derive an upper bound on f ( x ∗ ) .
Together with the lower bound on f ( x t M t ) given by Lemma B.1, we can derive an upper bound on the instantaneous regret r t .
Thus, we are now ready to prove the upper bound on the cumulative regret
R T in Theorem 4.2, which is re-stated in Theorem B.4 for clearness.
Theorem B.4.
∀ δ ∈ (0 , 1) , let β t
=
2 log(4 π t /δ )
+ 2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) and
L = b (cid:112) log(4 Da/δ ) ,
where { π t } t ≥ 1 satisfies (cid:80) t ≥ 1 π − 1 t = 1 and π t > 0 .
Let β ∗ T = max 1 ≤ i ≤ T β t .
At iteration T , the cumulative regret
[Equation]
Lr holds with probability at least 1 − δ , where C 1 > 0 is a constant, γ T = max |D| =
T I ( y D , f D ) , I ( · , · ) denotes the information gain, and y D and f D are the noisy and true observations of a set D of points, respectively.
Proof.
For all t ≥ 1 , we have
[Equation]
By Eq.
(8), we have
[Equation]
Note that L = b (cid:112) log(4 Da/δ ) here, and thus Eq.
(12) holds with probability at least 1 − δ/ 4 .
By
Lemma B.3 with β t = 2 log(4 π t /δ )
+ 2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) and
L = b (cid:112) log(4 Da/δ ) , setting τ t =
d t t 2 br (cid:112) log(4 Da/δ ) leads to that
[Equation]
holds with probability at least 1 − δ/ 2 .
By Lemma B.1 with β t = 2 log(4 π t /δ )
+ 2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) ≥ 2 log(4 π t /δ ) , it holds with probability at least 1 − δ/ 4 that [Equation]
[Equation]
Applying Eqs.
(12), (13) and (14) to Eq.
(11), it holds with probability at least 1 − δ that ∀ t ≥ 1 , r t ≤ (cid:88) i
[Equation]
where the second inequality holds because x t M t is generated by maximizing GP-UCB, and thus µ t − [Equation] x [Equation]
+
[Equation]
By summing up r t from t = 1 to T , we have with probability at least 1 − δ that, ∀ T ≥ 1 , R T =
T (cid:88)
[Equation]
(cid:88)
[Equation]
(cid:88)
[Equation]
(cid:88)
[Equation]
(cid:88)
[Equation]
(cid:88)
[Equation]
(cid:88)
i
[Equation]
(15) where the second inequality holds by (cid:80) Tt =1 1 /t≤
π 2 /≤ 2 .
Furthermore, let
C 1 = 8 / log(1 + η − 2 ) , and Lemma 5.4 in [36] has shown that (cid:80) Tt =1 2 β 1 / 2 t σ t [Equation] ( x
[Equation]
t [Equation] ≤ (cid:113)
[Equation]
Finally, by applying this inequality to
Eq.
(15), the theorem holds.
We also summarize the main idea of the above proof.
The proof is inspired by [36], i.e., to derive the upper bound on the gap r t = f ( x ∗ ) − f ( x t M t ) between the function values of the optimal point x ∗ and the sampled point x t M t at iteration t .
Let x ∗ M t denote the point obtained by projecting x ∗ onto M t , and ˜ x ∗ M t denote its closest discretized point.
By utilizing the posterior mean µ t − 1 ( · ) and variance σ 2 t − 1 ( · ) of f ( x t M t ) and f (˜ x ∗ M t ) , we can have f ( x t M t ) ≥ µ t − 1 ( x t M t ) − β 1 / 2 t σ t − 1 ( x t M t ) and f ( x ∗ ) =
( f ( x ∗ ) − f ( x ∗ M t ))
+ ( f ( x ∗ M t ) − f (˜ x ∗ M t ))
+ f (˜ x ∗ M t ) ≤ (cid:80)
i ∈
[ D ] \ M t α ∗ i Lr + α max /t 2 + (cid:80) i
∈
[ D ] \ M t α ∗ i Lr + µ t − 1 (˜ x ∗ M t ) + β 1 / 2 t σ t − 1 (˜ x ∗ M t ) , where the terms (cid:80) i ∈
[ D ] \ M t α ∗ i
Lr
and α max /tare led by variable selection and discretization, respectively.
As x t M t is generated by maximizing
GP-UCB, we have µ t − 1 (˜ x ∗ M t ) + β 1 / 2 t σ t − 1 (˜ x ∗ M t ) ≤ µ t − 1 ( x t M t ) + β 1 / 2 t σ t − 1 ( x t M t ) .
Thus, r t ≤ 2 β 1 / 2 t σ t − 1 ( x t M t ) + α max /t 2 + 2 (cid:80) i ∈
[ D ] \ M t α ∗ i
Lr .
Finally, summing up r t from t = 1 to T and using Lemma 5.4 in [36] can lead to Theorem 4.2.
The main difference from the proof of GP-UCB [36] is that variable selection brings some uncertainty introduced by the unselected variables.
Based on the Lipschitz condition in Assumption 4.1, the uncertainty by the i -th unselected variable can be upper bounded by α ∗ i Lr , leading to the additional regret 2 (cid:80) Tt =1 (cid:80) i ∈
[ D ] \ M t α ∗ i Lr in Eq.
B.2 Details of Computational Complexity Analysis
The computational complexity of one iteration of BO depends on three critical components: fitting a
GP surrogate model, maximizing an acquisition function and evaluating a sampled point.
Assume that the kernel function is squared exponential kernel.
At iteration t , the number of selected variables is d t .
When fitting a GP model, we calculate the marginal likelihood [30] and gradient as follows:
[Equation]
where y t =
[ y 1 , . . .
, y t ] T , X t =
[ x 1 , . . .
, x t ] , θ are the kernel parameters, K t is the covariance matrix, | · | and tr( · ) denote the determinant and trace of a matrix, respectively.
Then, we can use the gradient-based methods to optimize the likelihood function.
Therefore, the computational complexity of calculating the kernel parameters is O ( t 3 + t 2 d t ) .
Note that θ has been ignored, because its dimension is much smaller than d t and t .
When calculating the mean µ t ( x ) and variance σ 2 t ( x ) , the computational complexity is O ( t 3 + t 2 d t ) , due to the calculation of the kernel matrix and its inverse.
Thus, the total computational complexity of fitting the GP model is O ( t 3 + t 2 d t ) .
Maximizing an acquisition function is related to the optimization algorithm.
If we use the Quasi-Newton method to optimize GP-UCB, the computational complexity is O ( m ( t 2 + td t + d 2 t ))
[28], where m denotes the Quasi-Newton’s running rounds.
We note that in BO setting, t will not grow very large.
The running rounds m , however, will grow with d t .
Thus, the complexity of optimizing the acquisition function can be much larger than the square of d t .
The cost of evaluating a sampled point is fixed.
Thus, by selecting only a subset of variables, instead of all variables, to optimize, the computational complexity can decrease significantly.
C Method Implementation and Experimental Setting
We use the authors’ reference implementations for TuRBO 1 , LA-MCTS 2 and SAASBO.
For HeSBO and ALEBO, their implementations in Adaptive Experimentation Platform (Ax 4 ) are used.
We use the pycma library for CMA-ES.
Their hyper-parameters are summarized as follows.
• Vanilla BO.
We use the GP model in Scikit-learn 6 and the qExpectedImprovement acquisition function [42].
For the optimization of acquisition function, we randomly generate numerous points and select some ones with the maximal expected improvements, which is similar to the implementation in TuRBO [10], LA-MCTS [38], and HeSBO [27].
• MCTS-VS.
For the “fill-in” strategy, we use the best- k strategy with k = 20 .
The hyperparameter C p for calculating UCB in Eq.
(1) varies on different problems, as shown in Table 3.
We set all the other parameters to be same on different problems, where the batch size N v of variable index set is 2 , the sample batch size N s = 3 , the threshold N bad for re-initializing a tree is 5 , and the threshold N split for splitting a node is 3 .
When using TuRBO as the optimizer, we limit the maximal number of evaluations in TuRBO to 50 .
[Table]
• Dropout.
We set the parameter d to the number of valid dimensions for synthetic functions, and use the same “fill-in” strategy as MCTS-VS.
• TuRBO.
We use the default parameter setting in the authors’ reference implementation.
• LA-MCTS-TuRBO.
We use the same TuRBO setting as MCTS-VS.
The parameter C p is recommended between 1% and 10% of the optimum in LA-MCTS [38].
Because all our selected values of C p for MCTS-VS have belonged to the recommended range for LA-MCTS, we use them directly.
The RBF kernel is used for SVM classification.
• SAASBO.
We use the default parameter setting in the authors’ reference implementation, but modify the acquisition function optimization to the same as other methods for fair comparison.
• HeSBO and ALEBO.
We set the parameter d to the number of valid dimensions for synthetic functions.
For real-world problems, we do not know the number of valid dimensions, and thus we just set a reasonable value, i.e., d = 10 for NAS-Bench, d = 10 for Hopper, and d = 20 for Walker2d.
• CMA-ES.
We only adjust the step-size parameter σ for different problems, because the default setting σ = 0 .
01 leads to extremely poor performance.
We set σ = 0 .
8 for Hartmann problems, σ = 10 for Levy problems, σ = 0 .
1 for NAS-Bench, and σ = 0 .
01 for MuJoCo tasks.
We set the population size to 20 and maintain all the other parameters to default.
• VAE-BO uses VAE for embedding.
That is, VAE-BO uses the encoder to embed the original high-dimensional space into a low-dimensional subspace, then optimizes via BO in the subspace and uses the decoder to project the new sampled point back for evaluation.
We set the learning rate to 0 . 01 and the interval of updating VAE to 30 . The experiments of comparing wall clock time are conducted on Intel(R) Core(TM) i7-10700 CPU @
2.90GHz and use single thread.
D Sensitivity Analysis of Hyper-parameters of MCTS-VS
We provide further studies to examine the influence of the hyper-parameters of MCTS-VS, including the employed optimization algorithm for optimizing the selected variables in each iteration, the “fillin” strategy, the hyper-parameter k used in the best- k strategy, the hyper-parameter C p for calculating
UCB in Eq. (1), the number 2 × N v × N s of sampled data in each iteration, the threshold N bad for re-initializing a tree, and the threshold N split for splitting a tree node.
The optimization algorithm is employed by MCTS-VS to optimize the selected variables in each iteration.
We compare three different optimization algorithms, i.e., random search (RS), BO and
TuRBO.
First, we conduct experiments similar to “Effectiveness of Variable Selection” in Section 5.1, to show the effectiveness of MCTS-VS even when equipped with RS.
Figure 6 shows that MCTSVS-RS is better than Dropout-RS and RS, revealing the advantage of MCTS-VS. [Figure] [Figure] [Figure]
[Figure]
[Figure]
Next we compare the performance of MCTS-VS equipped with RS, BO and TuRBO, by experiments on the Hartmann functions with increasing ratio of valid variables.
Hartmann 6 _ 500 has 6 valid variables.
Hartmann 6 _ 5 _ 500 is generated by mixing 5 Hartmann 6 functions as Hartmann 6( x 1:6 )
+
Hartmann 6( x 7:12 ) + · · · + Hartmann 6( x 25:30 ) , and appending 470 unrelated dimensions, where x i : j denotes the i -th to j -th variables.
Hartmann 6 _ 10 _ 500 is generated alike.
Thus, Hartmann 6 _ 5 _and Hartmann 6 _ 10 _ 500 have 30 and 60 valid variables, respectively.
The results in Figure 7 show that as the ratio of valid variables increases, MCTS-VS-TuRBO gradually surpasses MCTS-VS-RS and MCTS-VS-BO, while MCTS-VS-RS becomes worse and worse.
This is expected.
If the ratio of valid variables is high, MCTS-VS is more likely to select the valid variables, so it is worth to use the expensive optimization algorithm, e.g., TuRBO, to optimize the selected variables.
If the ratio is low, unrelated variables are more likely to be selected most of the time, so using a cheap optimization algorithm would be better.
These observations also give us some guidance on selecting optimization algorithms in practice.
“Fill-in” strategy is a basic component of variable selection methods, which influences the quality of the value of unselected variables.
We compare the employed best-
k strategy ( k = 20 ) with the average best-
k strategy and the random strategy.
The average best-
k strategy uses the average of the best k data points for the unselected variables, and the random strategy samples the value of an unselected variable from its domain randomly.
As shown in Figure 8(a), the random strategy leads to the poor performance of MCTS-VS-BO, which may be because it does not utilize the historical information and leads to over-exploration.
The best- k strategy utilizes the historical points that have high objective values to fill in the unselected variables, thus behaving much better.
The performance of the average strategy is between the best- k and random strategies.
We recommend using the best- k strategy in practice.
The hyper-parameter k used in the best-
k strategy controls the degree of exploitation for the unselected variables.
As shown in Figure 8(b), a smaller k encourages exploitation, which results in better performance in the early stage, but easily leads to premature convergence. A larger k [Figure]
encourages exploration and behaves worse in the early stage, but may converge to a better value.
We recommend using a larger k if allowing enough evaluations
[Figure]
The hyper-parameter C p for calculating UCB in Eq. (1) balances the exploration and exploitation of MCTS.
As shown in Figure 9, a too small C p leads to relatively worse performance, highlighting the importance of exploration.
A too large C p may also lead to over-exploration.
But overall MCTSVS is not very sensitive to C p .
We recommend setting C p between 1% and 10% of the optimum (i.e., max f ( x ) ), which is consistent with that for LA-MCTS [38].
[Figure]
The number 2 × N v × N s of sampled data in each iteration depends on the batch size N v of variable index subset and the sample batch size N s , and will influence the accuracy of estimating the variable score vector in Eq.
If we increase N v and N s , we can calculate the variable score more accurately, but also need more evaluations.
Figure 10(a) shows that given the same number of evaluations, MCTS-VS-BO achieves the best performance when N v = 2 and N s = 3 .
Thus, this setting may be a good choice to balance the accuracy of variable score and the number of evaluations, which is also used throughout the experiments.
The threshold N bad for re-initializing a tree controls the tolerance of selecting bad tree nodes (i.e., nodes containing unimportant variables).
A smaller N bad leads to frequent re-initialization, which can adjust quickly but may cause under-exploitation of the tree.
A larger N bad can make full use of the tree, but may optimize too much on unimportant variables.
Figure 10(b) shows that MCTS-VS achieves the best performance when N bad = 5 .
Thus, we recommend to use this setting, to balance the re-initialization and exploitation of the tree.
The threshold N split for splitting a node.
If the number of variables in a node is larger than
N split , the node can be further partitioned.
That is, the parameter N split controls the least number of variables in a leaf node and thus affects the number of selected variables, which has a direct influence on the wall clock time.
Note that MCTS-VS selects a leaf node and optimizes the variables contained by this node in each iteration.
The smaller N split , the shorter the time.
Figure 10(c) shows that N split has little influence on the performance of MCTS-VS-BO, and thus we recommend to set N split =to reduce the wall clock time.
[Figure]
Influence of the hyper-parameters on the runtime of MCTS-VS.
We also provide some intuitive explanation about the influence of the hyper-parameters on the runtime.
The threshold N split for splitting a node has a direct impact on the runtime, because it controls the least number of variables to be optimized in a leaf node.
That is, the runtime will increase with N split .
Other parameters may affect the depth of the tree and thus the runtime.
For the threshold N bad for re-initializing a tree, if it is set to a small value, MCTS-VS will re-build the tree frequently and the depth of the tree is small.
The shallow nodes have more variables, leading to more runtime to optimize.
For the hyper-parameter C p for calculating UCB, if it is set to a large value, the exploration is preferred and
MCTS-VS will tend to select the right node (regarded as containing unimportant variables).
The tree thus will be re-built frequently, leading to more runtime.
For the number 2 × N v × N s of sampled data at each iteration, if N v and N s are set to large values, the depth of the tree will be small given the total number of evaluations, and thus lead to more runtime.
E Additional Experiments
Detailed results on NAS-Bench-101 and NAS-Bench-201.
Figure 11 shows the performance of the compared methods on the task of NAS-Bench-101 and NAS-Bench-201 when using the number of evaluations and wall clock time as the x -axis, respectively.
Though most of their performance is similar in the left two subfigures, it can be clearly observed from the right two subfigures that
MCTS-VS-BO uses the least time to achieve the best accuracy.
Note that we only show the subfigures with the wall clock time as the x -axis in the main paper due to the space limitation.
Besides, we also run a longer time here (i.e., in the right two subfigures) to provide a more complete observation.
Experiments on more NAS-Bench problems.
We also conduct experiments on NAS-Bench1Shot1 [44], TransNAS-Bench-101 [8] and NAS-Bench-ASR [25].
NAS-Bench-1Shot1 is a weightsharing benchmark based on one-shot NAS methods, deriving from the large architecture space of
NASBench-101.
TransNAS-Bench-101 is a benchmark dataset containing network performance across seven vision tasks, e.g., object classification, scene classification and so on.
We use the scene classification task with cell-level search space in our experiments.
NAS-Bench-ASR is a benchmark for Automatic Speech Recognition (ASR) and trained on the TIMIT audio dataset.
For
NAS-Bench-ASR, we use Phoneme Error Rate (PER) on the validation dataset as the metric.
In the same way as [20], we create problems with D = 33 , D = 24 and D = 30 for NAS-Bench- [Figure] 1Shot1, TransNAS-Bench-101 and NAS-Bench-ASR, respectively.
The results in Figure 12 show that MCTS-VS-BO still uses the least time to achieve the best performance
[Figure]
Experiments on extremely low and high dimensional problems.
We also evaluate the compared methods for extremely low and high dimensional problems by testing on Hartmann 6 _ 100 and
Hartmann 6 _ 1000 .
We only run MCTS-VS, TuRBO, LA-MCTS-TuRBO and HeSBO here, because they behave well in the previous experiments.
As expected, the right subfigure of Figure 13 shows that MCTS-VS-BO has a clear advantage over the rest methods on the extremely high dimensional function Hartmann 6 _ 1000 .
The left subfigure shows that on Hartmann 6 _ 100 , TuRBO behaves the best and MCTS-VS is the runner-up, implying that MCTS-VS can also tackle low dimensional problems to some degree.
Experiments on synthetic functions depending on a subset of variables to various extent.
In the experiments, the synthetic functions are generated by adding unrelated variables directly.
For example, Hartmann 6 _ 500 has the dimension D = 500 , and is generated by appendingunrelated dimensions to Hartmann with 6 variables.
Here, we test the performance of MCTS-VS on a synthetic function whose dependence on a subset of variables is more various.
For this purpose, we generate Hartmann 6 _ 5 _ 500 _ v by mixing five Hartmann 6 functions as 0 .
5 0 Hartmann 6( x 1:6 ) + 0 .
5 1 × Hartmann 6( x 7:12 ) + · · · + 0 .
5 4 Hartmann 6( x 25:30 ) , and appending 470 unrelated dimensions, where x i : j denotes the i -th to j -th variables, and different coefficients represent various degrees of dependence.
The results in Figure 14 show that MCTS-VS-BO performs the best.
Experiments with increasing ratio of valid variables.
We also examine the performance of MCTSVS when the ratio of valid variables increases.
We use the synthetic function Hartmann 6 _ 500 , and generate the variants with more valid variables by mixing multiple Hartmann 6 functions as in Appendix D. For example, Hartmann 6 _ 5 _ 500 is generated by mixing five Hartmannfunctions as Hartmann 6( x 1:6 ) + Hartmann 6( x 7:12 ) + · · · + Hartmann 6( x 25:30 ) , and appending 470 unrelated dimensions.
We have compared MCTS-VS-TuRBO with LA-MCTS-TuRBO and
TuRBO on Hartmann 6 _ 500 , Hartmann 6 _ 5 _ 500 , Hartmann 6 _ 10 _ 500 , . . .
, Hartmann 6 _ 30 _ 500 , and
Hartmann 6 _ 83 _ 500 , which has the largest number (i.e., 6 × 83 = 498 ) of valid variables.
The results are shown in Figure 15.
It can be observed that LA-MCTS-TuRBO performs the worst.
As expected, when the percentage of valid variables is low (e.g., in Hartmann 6 _ 500 , Hartmann 6 _ 5 _and Hartmann 6 _ 10 _ 500 ), MCTS-VS-TuRBO can be better than TuRBO; but as the percentage of [Figure] TuRBO [Figure]
[Figure]
valid variables increases, TuRBO becomes better
[Figure]
because a leaf node of MCTS can contain only a small fraction of valid variables
[Figure]
Hierarchical variable selection for extremely high-dimensional problems.
We attempt to combine
MCTS-VS and SAASBO (i.e., MCTS-VS-SAASBO) to handle extremely high-dimensional problems.
MCTS-VS-SAASBO can be viewed as a hierarchical variable selection method, i.e., MCTS-VS first performs an efficient but rough variable selection to select some variables, and then SAASBO performs a time-consuming but precise variable selection under the relative low-dimensional space, to further select the important variables.
We run MCTS-VS-SAASBO and SAASBO on Hartmann 6 _ 500 .
The results are shown in Figure 16.
The performance of MCTS-VS-SAASBO and SAASBO is similar.
But when considering the runtime, the time of 200 iterations of MCTS-VS-SAASBO is about 6000 s, while the time of SAASBO is about 45000 s. That is, MCTS-VS-SAASBO can achieve more thantimes acceleration.
The curves of using the wall clock time as the x -axis in the right sub-figure of
Figure 16 clearly show the advantage of MCTS-VS-SAASBO over SAASBO.
MCTS-VS-SAASBO selects the variables containing important ones by MCTS and then uses SAASBO to optimize the selected variables, which reduces the dimension and thus costs much less time than using SAASBO directly.
The combination of MCTS-VS and SAASBO may be a potential solution for BO to handle extremely high-dimensional optimization problems, where it is difficult to select important variables directly.
Comparison with LASSO-VS.
There are other variable selection methods (e.g., LASSO), which are not designed for high dimensional BO but can be used directly.
We have implemented the
LASSO-based variable selection method, named LASSO-VS.
We compare MCTS-VS, LASSO-VS and Dropout on the synthetic function Hartmann 6 _ 300 .
When using LASSO-VS, the d variables with the largest absolute values of the regression coefficients are selected at each iteration.
The results are shown in Figure 17.
When equipped with either BO or TuRBO, the proposed MCTS-VS always performs the best.
We can also observe that when equipped with BO, LASSO-VS can even [Figure] be worse than Dropout.
This may be because many of existing variable selection methods (e [Figure] g.,
LASSO) usually require a large number of samples to fit the linear regression model well, while in
BO scenarios, only a limited number of samples can be evaluated
[Figure]
Statistical tests.
As most of the previous works, we have conducted experiments using 5 random seeds (2021–2025).
Here, we also conduct statistical tests on Hartmann and Levy functions by running the methods for 50 times (random seeds 2021–2070), to make a more confident comparison.
Considering the performance and runtime of the methods we have observed in Figure 2, we only compare MCTS-VS with LA-MCTS-TuRBO and TuRBO, which achieve good performance in acceptable time.
The results are shown in Table 4.
MCTS-VS-BO achieves the best average objective value on all the synthetic functions except Levy 10 _ 100 where the dimension is relatively low and
TuRBO performs the best.
By the Wilcoxon signed-rank test with confidence level 0.05, MCTS-VSTuRBO is significantly better than LA-MCTS-TuRBO on all the synthetic functions, showing the advantage of MCTS-VS over LA-MCTS for variable selection.
Compared with TuRBO, MCTS-VSTuRBO is only significantly better on Hartmann functions, which may be because the ratio of valid variables of Hartmann 6 _ 300 and Hartmann 6 _ 500 is lower than that of Levy 10 _ 100 and Levy 10 _ 300 , and thus the advantage of performing variable selection by MCTS-VS is more clear.
Note that the observations about the performance rank of the compared methods are consistent with that observed in Figure 2, which plot the results of the compared methods by running five times.
F Enlargement of Some Figures in the Main Paper
Due to space limitation, Figures 1 and 2 in the main paper are a little small.
Here, we also provide their enlarged versions, i.e., Figures 18 and 19.
[Table]
Vanilla BO
Dropout-BO
[Table] [Figure]
[Figure]
[Figure]
MCTS-VS-BO
MCTS-VS-TuRBO TuRBO
LA-MCTS-TuRBO 0.
SAASBO
HeSBO
ALEBO CMA-ES VAE-BO
[Figure]