Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization
Abstract
Bayesian optimization (BO) is a class of popular methods for expensive black-boxoptimization, and has been widely applied to many scenarios.
However, BO suffersfrom the curse of dimensionality, and scaling it to high-dimensional problems is stilla challenge.
In this paper, we propose a variable selection method MCTS-VS basedon Monte Carlo tree search (MCTS), to iteratively select and optimize a subset ofvariables.
That is, MCTS-VS constructs a low-dimensional subspace via MCTS andoptimizes in the subspace with any BO algorithm.
We give a theoretical analysis ofthe general variable selection method to reveal how it can work.
Experiments onhigh-dimensional synthetic functions and real-world problems (i.e., NAS-benchproblems and MuJoCo locomotion tasks) show that MCTS-VS equipped with aproper BO optimizer can achieve state-of-the-art performance.
Introduction
In many real-world tasks such as neural architecture search (NAS) [37] and policy search in reinforcement learning (RL) [5], one often needs to solve the expensive black-box optimization problems.
Bayesian optimization (BO) [10, 22, 30] is a sample-efficient algorithm for solving such problems.
It iteratively fits a surrogate model, typically Gaussian process (GP) and maximizes an acquisitionfunction to obtain the next point to evaluate.
While BO has been employed in a wide variety ofsettings, successful applications are often limited to low-dimensional problems.
Recently, scaling BO to high-dimensional problems has received a lot of interest.
Decompositionbased methods [12, 14, 16, 25, 29] assume that the high-dimensional function to be optimized hasa certain structure, typically the additive structure.
By decomposing the original high-dimensionalfunction into the sum of several low-dimensional functions, they optimize each low-dimensionalfunction to obtain the point in the high-dimensional space.
However, it is not easy to decide whethera decomposition exists as well as to learn the decomposition.
Other methods often assume that the original high-dimensional function with dimension D hasa low-dimensional subspace with dimension d ≪ D , and then perform the optimization in thelow-dimensional subspace and project the low-dimensional point back for evaluation.
For example,embedding-based methods [19, 38] use a random matrix to embed the original space into the lowdimensional subspace.
Another way is to select a subset of variables directly, which can even avoidthe time-consuming matrix operations of embedding-based methods.
For example, Dropout [20]selects d variables randomly in each iteration.
Note that for both embedding and variable selectionmethods, the parameter d can have a large influence on the performance, which is, however, difficultto set in real-world problems.
In this paper, we propose a new Variable Selection method using Monte Carlo Tree Search (MCTS),called MCTS-VS.
MCTS is employed to partition the variables into important and unimportant ones, and only those selected important variables are optimized via any black-box optimization algorithm,e.g., vanilla BO [30] or TuRBO [9].
The values of unimportant variables are sampled using historicalinformation.
Compared with Dropout-BO, MCTS-VS can select important variables automatically.
We also provide regret and computational complexity analyses of general variable selection methods,showing that variable selection can reduce the computational complexity while increasing thecumulative regret.
Our regret bound generalizes that of GP-UCB [34] which always selects allvariables, as well as that of Dropout [20] which selects d variables randomly in each iteration.
Theresults suggest that a good variable selection method should select as important variables as possible.
Experiments on high-dimensional synthetic functions and real-world problems (i.e., NAS and RLproblems) show that MCTS-VS is better than the previous variable selection method Dropout [20],and can also achieve the competitive performance to state-of-the-art BO algorithms.
Furthermore, itsrunning time is small due to the advantage of variable selection.
We also observe that MCTS-VS canselect important variables, explaining its good performance based on our theoretical analysis.
Background2.1 Bayesian Optimization
We consider the problem max x ∈X f ( x ) , where f is a black-box function and
X ⊆
R D is the domain.
The basic framework of BO contains two critical components: a surrogate model and an acquisitionfunction.
GP is the most popular surrogate model.
Given the sampled data points { ( x i , y i ) } t − 1 i =1 ,where y i =
f ( x i )
+ ϵ i and ϵ i ∼ N (0 , η 2 ) is the observation noise, GP at iteration t seeks to inferf ∼ GP ( µ ( · ) , k ( · , · ) + η 2 I ) , specified by the mean µ ( · ) and covariance kernel k ( · , · ) , where I isthe identity matrix of size D .
After that, an acquisition function, e.g., Probability of Improvement(PI) [18], Expected Improvement (EI) [15] or Upper Confidence Bound (UCB) [34], is used todetermine the next query point x t while balancing exploitation and exploration.
2.2 High-dimensional Bayesian Optimization
Scaling BO to high-dimensional problems is a challenge due to the curse of dimensionality and thecomputation cost.
As the dimension increases, the search space increases exponentially, requiringmore samples, and thus more expensive evaluations, to find a good solution.
Furthermore, thecomputation cost of updating the GP model and optimizing the acquisition function will be verytime-consuming [28].
There have been a few common approaches to tackle high-dimensional BOwith different assumptions.
Decomposition.
Assuming that the function can be decomposed into the sum of low-dimensionalfunctions with disjoint subspaces, Kandasamy et al.
[16] proposed the Add-GP-UCB algorithm tooptimize those low-dimensional functions separately, which was further generalized to overlappingsubspaces [25, 29].
Wang et al.
[39] proposed ensemble BO that uses an ensemble of additive GPmodels for scalability.
Han et al.
[12] constrained the dependency graphs of decomposition to treestructures to facilitate the decomposition learning and optimization.
For most problems, however, thedecomposition is unknown, and also difficult to learn.
Embedding.
Assuming that only a few dimensions affect the high-dimensional function significantly,embedding-based methods embed the high-dimensional space into a low-dimensional subspace, andoptimize in the subspace while projecting the point back for evaluation.
REMBO and its variants use arandom matrix to embed the search space into a low-dimensional subspace [2, 3, 38].
Nayebi et al.
[26]used a hash-based method for embedding.
Letham et al.
[19] proposed ALEBO, focusing on severalmisconceptions in REMBO to improve the performance.
The VAE-based approaches were alsoemployed to project a structured input space (e.g., graphs) to a low-dimensional subspace [11, 21].
Variable Selection.
Based on the same assumption as embedding, variable selection methodsiteratively select a subset of variables to build a low-dimensional subspace and optimize through
BO.
The selected variables can be viewed as important variables that are valuable for exploitation, orhaving high uncertainty that are valuable for exploration.
A classical method is Dropout [20], whichrandomly chooses d variables in each iteration.
When evaluating the sampled point, the values ofthose unselected variables are obtained by random sampling or using historical information.
Note that variable selection can be faster than embedding, because the embedding cost (e.g., matrix inversion)is time-consuming for high-dimensional optimization.
Both embedding and variable selection methods need to specify the parameter d , i.e., the dimensionof low-dimensional subspace, which will affect the performance significantly, but is not easy to set.
There are also some methods to improve the basic components of BO directly for high-dimensionalproblems.
For example, DNGO [33] uses the neural network as an alternative of GP to speedup inference; SAASBO [8] uses sparsity-inducing prior to perform variable selection, makingunimportant variables not selected to optimize and thus restraining over-exploration.
They can becombined with the above dimensionality reduction methods, which may bring further improvement.
2.3 Monte Carlo Tree Search
MCTS [4] is a tree search algorithm based on random sampling, and has shown great success inhigh-dimensional tasks, such as Go [31, 32].
A tree node represents a state, describing the currentsituation, e.g., the position in path planning.
Each tree node X stores a value v X representing itsgoodness, and the number n X that it has been visited.
They are used to calculate UCB [1], i.e.,v X + 2 C p (cid:112) 2(log n p ) /n
X , where C p is a hyper-parameter, and n p is the number of visits of theparent of X .
UCB considers both exploitation and exploration, and will be used for node selection.
MCTS iteratively selects a leaf node of the tree for expansion.
Each iteration can be divided into foursteps: selection , expansion , simulation and back-propagation .
Starting from the root node, selectionis to recursively select a node with larger UCB until a leaf node, denoted as X .
Expansion is toexecute a certain action in the state represented by X and transfer to the next state, e.g., move forwardand arrive at a new position in path planning.
We use the child node Y of X to represent the nextstate.
Simulation is to obtain the value v Y via random sampling.
Back-propagation is to update thevalue and the number of visits of Y ’s ancestors.
To tackle high-dimensional optimization, Wang et al.
[36] proposed LA-MCTS, which applies MCTSto iteratively partition the search space into small sub-regions, and optimizes only in the goodsub-regions.
That is, the root of the tree represents the entire search space Ω , and each tree node
X represents a sub-region Ω X .
The value v X is measured by the average objective value of thesampled points in the sub-region Ω X .
In each iteration, after selecting a leaf node X , LA-MCTSperforms the optimization in Ω X by vanilla BO [30] or TuRBO [9], and the sampled points are usedfor clustering and classification to bifurcate Ω X into two disjoint sub-regions, which are “good" and“bad", respectively.
Note that the sub-regions are generated by dividing the range of variables, andtheir dimensionality does not decrease, which is still the number of all variables.
Wang et al.
[36]have empirically shown the good performance of LA-MCTS.
However, as the dimension increases,the search space increases exponentially, and more partitions and evaluations are required to find agood solution, making the application of LA-MCTS to high-dimensional optimization still limited.
MCTS-VS Method
In this section, we propose a Variable Selection method based on MCTS for high-dimensional
BO, briefly called MCTS-VS.
The main idea is to apply MCTS to iteratively partition all variablesinto important and unimportant ones, and perform BO only for those important variables.
Let[ D ] = { 1 , 2 , . . .
, D } denote the indexes of all variables x , and x M denote the subset of variablesindexed by M ⊆
[ D ] .
We first introduce a D -dimensional vector named variable score , which is a key component of
MCTS-VS.
Its i -th element represents the importance of the i -th variable x i .
During the runningprocess of MCTS-VS, after optimizing a subset x M of variables where M ⊆
[ D ] denotes the indexesof the variables, a set D of sampled points will be generated, and the pair ( M , D ) will be recordedinto a set D , called information set .
The variable score vector is based on D , and calculated as
[Equation]
where the function g : 2 [ D ] → { 0 , 1 } D gives the Boolean vector representation of a variable indexsubset M ⊆
[ D ] (i.e., the i -th element of g ( M ) is 1 if i ∈ M , and 0 otherwise), 1 D is a D -dimensional vector with all 1s, ⊙ and / are the element-wise product and division, respectively.
Each dimensionof (cid:80) ( M , D ) ∈
D g ( M ) ⊙ ( (cid:80) ( x i ,y i ) ∈D
y i · 1 D ) is the sum of query evaluations using each variable,and each dimension of (cid:80) ( M , D ) ∈ D |D| · g ( M ) is the number of queries using each variable.
Thus,the i -th element of variable score s , representing the importance of the i -th variable x
i , is actuallymeasured by the average goodness of all the sampled points that are generated by optimizing a subsetof variables containing x i .
The variable score s will be used to define the value of each tree node of
MCTS as well as for node expansion.
In MCTS-VS, the root of the tree represents all variables.
A tree node X represents a subset ofvariables, whose index set is denoted by A X ⊆
[ D ] , and it stores the value v X and the number n X ofvisits, which are used to calculate the value of UCB.
The value v X is defined as the average score (i.e.,importance) of the variables contained by X , which can be calculated by s · g ( A X ) /
|
A X | , whereg ( A X ) is the Boolean vector representation of A X and | A X | is the size of A X , i.e., the number ofvariables in node X .
At each iteration, MCTS-VS first recursively selects a node with larger UCB until a leaf node (denotedas X ), which is regarded as containing important variables.
Note that if we optimize the subset x A Xof variables represented by the leaf X directly, the variables in x A X will have the same score (becausethey are optimized together), and their relative importance cannot be further distinguished.
Thus,
MCTS-VS uniformly selects a variable index subset M from A X at random, and employs BO tooptimize x M as well as x A X \ M ; this process is repeated for several times.
After that, the informationset D will be augmented by the pairs of the selected variable index subset M (or A X \ M ) and thecorresponding sampled points generated by BO.
The variable score vector s will be updated usingthis new D .
Based on s , the variable index set A X represented by the leaf X will be divided into twodisjoint subsets, containing variables with larger and smaller scores (i.e., important and unimportantvariables), respectively, and the leaf X will be bifurcated into two child nodes accordingly.
Finally,the v values of these two children will be calculated using the variable score vector s , and backpropagation will be performed to update the v value and the number of visits of the nodes along thecurrent path of the tree.
MCTS-VS can be equipped with any specific BO optimizer, resulting in the concrete algorithm
MCTS-VS-BO, where BO is used to optimize the selected subsets of variables during the runningof MCTS-VS.
Compared with LA-MCTS [36], MCTS-VS applies MCTS to partition the variablesinstead of the search space, and thus can be more scalable.
Compared with the previous variableselection method Dropout [20], MCTS-VS can select important variables automatically instead ofrandomly selecting a fixed number of variables in each iteration.
Next we introduce it in detail.
3.1 Details of MCTS-VS
The procedure of MCTS-VS is described in Algorithm 1.
In line 1, it first initializes the informationset D .
In particular, a variable index subset M i is randomly sampled from [ D ] , and the Latinhypercube sampling [23] is used to generate two sets (denoted as D i and D ¯ i ) of N s points to formthe two pairs of ( M i , D i ) and ( ¯ M i , D ¯ i ) , where ¯ M i =
[ D ] \ M i .
This process will be repeated for N vtimes, resulting in the initial D
= { ( M i , D i ) , ( ¯ M i , D ¯ i ) } N v i =1 .
The variable score vector s is calculatedusing this initial D in line 3, and the Monte Carlo tree is initialized in line 4 by adding only a rootnode, whose v value is calculated according to s and number of visits is 0.
MCTS-VS uses thevariable t to record the number of evaluations it has performed, and thus t is set to 2 × N v × N s inline 5 as the initial D contains 2 × N v × N s sampled points in total.
In each iteration (i.e., lines 7–28) of MCTS-VS, it selects a leaf node X by UCB in line 10, andoptimizes the variables (i.e., x A X ) represented by X in lines 13–23.
Note that to measure therelative importance of variables in x A X , MCTS-VS optimizes different subsets of variables of x A Xinstead of x A X directly.
That is, a variable index subset M is randomly sampled from A X in line 14,and the corresponding subset x M of variables is optimized by BO in lines 15–16.
The data points{ ( x i M , y i ) } t i =1 sampled-so-far is used to fit a GP model, and N s (called sample batch size ) new points{ x t + i M } N s i =1 are generated by maximizing an acquisition function.
Note that this is a standard BOprocedure, which can be replaced by any other variant.
To evaluate x t + i M , we need to fill in the valuesof the other variables x t
+ i
[ D ] \ M , which will be explained later.
After evaluating x t + i =
[ x t + i M , x t + i
[ D ] \ M ]in line 18, the information set D is augmented with the new pair of ( M , { ( x t
+ i , y t + i ) }
N s i
=1 ) in line 19,
Let A X denote the indexes of the subset of variables represented by X and t is increased by N s accordingly in line 21.
For fairness, the complement subset x ¯ M of variables,where ¯ M =
A X \ M , is also optimized by the same way, i.e., lines 15–21 of Algorithm 1 is repeatedfor ¯ M .
The whole process of optimizing x M and x ¯ M in lines 14–22 will be repeated for N v times,which is called batch size of variable index subset .
To fill in the values of the un-optimized variables in line 17, we employ the best- k strategy, whichutilizes the best k data points sampled-so-far, denoted as { ( x ∗ j , y ∗ j ) } kj =1 .
That is, { y ∗ j } kj =1 arethe k largest objective values observed-so-far.
If the i -th variable is un-optimized, its value willbe uniformly selected from { x ∗ ji } kj =1 at random.
Thus, MCTS-VS needs to store the best k datapoints in line 2 after initializing the information set D , and update them in line 20 after augmenting 195 D .
Another direct “fill-in" strategy is to sample the value randomly, while the superiority of theemployed best-
k strategy will be shown in the experiments.
After optimizing the variables x A X represented by the selected leaf X , the variable score vector smeasuring the importance of each variable will be updated using the augmented D in line 24.
If thenumber | A X | of variables in the leaf X is larger than a threshold N split (i.e., line 25), A X will bedivided into two subsets.
One contains those “important" variable indexes with score larger thanthe average score of x A X , and the other contains the remaining “unimportant" ones.
The leaf Xwill be bifurcated into a left child Y and a right child Z in line 26, containing those important and unimportant variables, respectively.
Meanwhile, v Y and v Z will be calculated according to s , and thenumber of visits is 0, i.e., n Y = n Z = 0 .
Finally, MCTS-VS performs back-propagation in line 28 tore-calculate the v value and increase the number of visits by 1 for each ancestor of Y and Z .
MCTS-VS will run until the number t of performed evaluations reaches the budget N e .
Note thatas the Monte Carlo tree may be built improperly, we use a variable n bad to record the number ofvisiting a right child node (regarded as containing unimportant variables), measuring the goodnessof the tree.
In line 5 of Algorithm 1, n bad is initialized as 0.
During the procedure of selecting aleaf node by UCB in line 10, n bad will be increased by 1 once visiting a right child node, which isupdated in line 12.
If n bad is larger than a threshold N bad (i.e., line 7), the current tree is regarded asbad, and will be re-initialized in line 8.
Furthermore, the frequency of re-initialization can be used toindicate whether MCTS-VS can do a good variable selection for the current problem.
For ease ofunderstanding, we also provide an example illustration of MCTS-VS in Appendix A.
Theoretical Analysis
It is difficult to analyze the regret of MCTS-VS directly, but we can theoretically analyze the influenceof general variable selection by adopting the acquisition function GP-UCB.
The considered generalvariable selection framework is as follows: after selecting a subset of variables at each iteration, thecorresponding observation data (i.e., the data points sampled-so-far where only the selected variablesare used) is used to build a GP model, and the next data point is sampled by maximizing GP-UCB.
We use M t to denote the sampled variable index subset at iteration t , and let | M t | = d t .
Let x ∗ denote an optimal solution.
We analyze the cumulative regret R T =
(cid:80) Tt =
( f ( x ∗ ) − f ( x t )) ,i.e., the sum of the gap between the optimum and the function values of the selected points by iteration
T .
To derive an upper bound on R T , we pessimistically assume that the worst function value, i.e.,min x [ D ] \ M t f ([ x M t , x [ D ] \ M t ]) , given x M t is returned in evaluation.
As in [20, 34], we assume that
X ⊂ [0 , r ] D is convex and compact, and f satisfies the following Lipschitz assumption.
Assumption 4.1.
The function f is a GP sample path.
For some a, b > 0 , given L > 0 , the partialderivatives of f satisfy that ∀ i ∈
[ D ] , ∃ α i ≥ 0 , P (sup x ∈X | ∂f/∂x i | < α i L ) ≥ 1 − ae − ( L/b ) 2 .
Based on Assumption 4.1, we define α ∗ i to be the minimum value of α i such that above equationholds, which characterizes the importance of the i -th variable x i .
The larger α ∗ i , the greater influenceof x i on the function f .
Let α max = max i ∈
[ D ] α ∗ i .
Theorem 4.2 gives an upper bound on the cumulative regret R T with high probability for generalvariable selection methods.
The proof is inspired by that of GP-UCB without variable selection [34]and provided in Appendix B.1.
If we select all variables each time (i.e., ∀ t : M t =
[ D ] ) and assume∀ i : α ∗ i ≤ 1 , the regret bound Eq.
(2) becomes R T ≤ (cid:112) C
1 Tβ ∗ T γ T
+ 2 , which is consistent with [34].
Note that ∀ t : | M t | =
d t =
D in this case, which implies that β t increases with t , leading toβ ∗ T = β T .
We can see that using variable selection will increase R T by 2 (cid:80)
Tt =1 (cid:80) i
∈
[ D ] \ M t α ∗ i Lr ,related to the importance (i.e., α ∗ i ) of unselected variables at each iteration.
The more importantvariables unselected, the larger R T .
Meanwhile, the term (cid:112) C 1 Tβ ∗ T γ T will decrease as β ∗ T relies onthe number d t of selected variables positively.
Ideally, if the unselected variables at each iteration arealways unrelated (i.e., α ∗ i =0 ), the regret bound will be better than that of using all variables [34].
Theorem 4.2.
∀ δ ∈ (0 , 1) , let β t
=
2 log(4 π t /δ )
+ 2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) and
L =b (cid:112) log(4 Da/δ ) , where r is the upper bound on each variable, and { π t } t ≥ 1 satisfies (cid:80) t ≥ 1 π − 1 t = 1and π t > 0 .
Let β ∗ T = max 1 ≤ i ≤ T β t .
At iteration T , the cumulative regret
[Equation]
holds with probability at least 1 − δ , where C 1 is a constant, γ T =max |D| =
T I ( y D , f D ) , I ( · , · ) is theinformation gain, and y D and f D are the noisy and true observations of a set D of points, respectively.248 By selecting d variables randomly at each iteration and assuming that r = 1 and ∀ i : α ∗ i ≤ 1 , it hasbeen proved [20] that the cumulative regret of Dropout satisfies R T ≤ √ C
Tβ T γ T +
+2 TL
( D − d ) .
In this case, we have d t = | M t | =
d , r = 1 and ∀ i : α ∗ i ≤ 1 .
Thus, Eq.
(2) becomes R T ≤ 251 (cid:112)
C 1 Tβ ∗ T γ T + 2 + 2 TL ( D − d ) , where β ∗ T = β T , as β t increases with t
By selecting only a subset of variables, instead of all variables, to optimize, the computationalcomplexity of fitting a GP surrogate model and maximizing an acquisition function in each iterationof BO will decrease significantly, whose detailed analysis is provided in Appendix B.2.
Thus, giventhe number d t of variables to be selected, a good variable selection method should select as importantvariables as possible, i.e., variables with as large α ∗ i as possible, which may help to design andevaluate variable selection methods.
The experiments in Section 5.1 will show that MCTS-VS canselect a good subset of variables while maintaining a small computational complexity.
Experiment
To examine the performance of MCTS-VS, we conduct experiments on different tasks, includingsynthetic functions, NAS-bench problems and MuJoCo locomotion tasks, to compare MCTS-VS withother black-box optimization methods.
For MCTS-VS, we use the same hyper-parameters except C p ,which is used for calculating UCB.
For Dropout and embedding-based methods, we set the parameterd to the number of valid dimensions for synthetic functions, and a reasonable value for real-worldproblems.
The hyper-parameters of the same components of different methods are set to the same.
We use five identical random seeds (2021–2025) for all problems and methods.
More details aboutthe settings can be found in Appendix C.
The codes are provided in the supplementary.
5.1 Synthetic Functions
We use Hartmann ( d = 6 ) and Levy ( d = 10 ) as the synthetic benchmark functions, and extend themto high dimensions by adding unrelated variables as [19, 26, 38].
For example, Hartmann 6 _ 300 hasthe dimension D = 300 , and is generated by appending 294 unrelated dimensions to Hartmann.
Thevariables affecting the value of f are called valid variables .
Effectiveness of Variable Selection.
Dropout [20] is the previous variable selection method whichrandomly selects d variables in each iteration, while our proposed MCTS-VS applies MCTS toautomatically select important variables.
We compare them against vanilla BO [30] without variableselection.
The first two subfigures in Figure 1 show that Dropout-BO and MCTS-VS-BO are betterthan vanilla BO, implying the effectiveness of variable selection.
We can also see that MCTS-VS-BOperforms the best, implying the superiority of MCTS-based variable selection over random selection.
We also equip MCTS-VS and Dropout with the advanced BO algorithm TuRBO [9], resulting in
MCTS-VS-TuRBO and Dropout-TuRBO.
The last two subfigures in Figure 1 show the similar resultsexcept that MCTS-VS-TuRBO needs more evaluations to be better than Dropout-TuRBO.
This isbecause TuRBO costs more evaluations than BO on the same selected variables, and thus needs moreevaluations to generate sufficient samples for an accurate estimation of the variable score in Eq.
[Figure]
Comparison with State-of-The-Art Methods.
We compare MCTS-VS with the state-of-the-artmethods, including TuRBO [9], LA-MCTS-TuRBO [36], HeSBO [26], ALEBO [19] and CMAES [13].
TuRBO fits a collection of local models to optimize in the trust regions for overcoming thehomogeneity of the global model and over-exploration.
LA-MCTS-TuRBO applies MCTS to partitionthe search space and uses TuRBO to optimize in a small sub-region.
HeSBO and ALEBO are stateof-the-art embedding methods.
CMA-ES is a popular evolutionary algorithm.
We also implement
VAE-BO
by combining VAE [17] with vanilla BO directly, as a baseline of learning-based embedding.
For MCTS-VS, we implement the two versions of MCTS-VS-BO and MCTS-VS-TuRBO, i.e.,
MCTS-VS equipped with vanilla BO and TuRBO.
As shown in Figure 2, MCTS-VS can achieve the best performance except on Levy 10 _ 100 , where it isa little worse than TuRBO.
For low-dimensional functions (e.g., D = 100 for Levy 10 _ 100 ), TuRBO can adjust the trust region quickly while MCTS-VS needs samples to estimate the variable score.
But as the dimension increases, the search space increases exponentially and it becomes difficultfor TuRBO to adjust the trust region; while the number of variables only increases linearly, making
MCTS-VS more scalable.
HeSBO is not stable, which has a moderate performance on Hartmann buta relatively good performance on Levy.
Note that we only run ALEBO on Hartmann 6 _ 300 , becauseit has already taken more than weeks to finish 600 evaluations.
VAE-BO has the worst performance,suggesting that the learning algorithm in high-dimensional BO needs to be designed carefully.
We also conduct experiments on extremely low and high dimensional variants of Hartmann (i.e.,
Hartmann 6 _ 100 and Hartmann 6 _ 1000 ), showing that MCTS-VS still performs well, and perform thesignificance test by running each method more times.
Please see Appendix E.
[Figure]
Next, we compare the practical running overheads of these methods.
We run each method for 100evaluations independently using 30 different random seeds, and calculate the average wall clock
[Figure]
time.
The results are shown in Table 1.
As expected, when using variable selection (i.e., Dropout and
MCTS-VS)
, the time is less than that of Vanilla BO or TuRBO, because we only optimize a subset ofvariables.
MCTS-VS is a little slower than Dropout, which is because MCTS-VS needs to build thesearch tree and calculate the variable score, while Dropout only randomly selects variables.
MCTSVS is much faster than LA-MCTS-TuRBO, showing the advantage of partitioning the variables topartitioning the search space.
HeSBO and ALEBO consume × 10 – × 200 time compared with thevariable selection methods.
CMA-ES is very fast because it does not need to fit a GP model oroptimize an acquisition function.
The reasons for the small running overhead of MCTS-VS can besummarized as follows: 1) it only optimizes a selected subset of variables; 2) the depth of the searchtree is shallow, i.e., O (log D ) in expectation and less than D in the worse case; 3) the variable scorevector in Eq.
(1) is easy to calculate for bifurcating a tree node.
Why MCTS-VS Can Perform Well.
The theoretical results have suggested that a good variableselection method should select as important variables as possible.
Thus, we compare the quality of thevariables selected by MCTS-VS and Dropout (i.e., random selection), measured by the recall d ∗ t /d ,where d is the number of valid variables, and d ∗ t is the number of valid variables selected at iteration t .
Dropout randomly selects d variables at each iteration, and thus, the recall is d/D in expectation.
For
MCTS-VS, we run MCTS-VS-BO for 600 evaluations on five different random seeds, and calculatethe average recall.
On Levy 10 _ 100 , Levy 10 _ 300 , Hartmann 6 _ 300 and Hartmann 6 _ 500 , the recallfor Dropout is 0 . 100 , 0 .
020 and 0 .
012 , respectively, while the average recall for MCTS-VS ismuch larger, which is 0 . 429 , 0 .
352 and 0 .
350 , respectively.
Thus, MCTS-VS can select bettervariables than Dropout, and thus achieve a good performance as shown before.
Meanwhile, the recallbetween 0 .
350 and 0 .
433 of MCTS-VS implies that the method could be further improved.
5.2 Real-World Problems
We further compare MCTS-VS with the baselines on real-world problems, including NAS-Bench101 [41], NAS-Bench-201 [6], Hopper and Walker2d.
NAS-Bench problems are popular benchmarksin high-dimensional BO.
Hopper and Walker2d are robot locomotion tasks in MuJoCo [35], which isa popular black-box optimization benchmark and much more difficult than NAS-Bench problems.
The experimental results on more real-world problems can refer to Appendix E.
NAS-Bench Problems.
NAS-Bench-101 is a tabular data set that maps convolutional neural networkarchitectures to their trained and evaluated performance on CIFAR-10, and we create a constrainedproblem with D = 36 in the same way as [19].
NAS-Bench-201 is an extension to NAS-Bench-101,leading to a problem with D = 30 but without constraints.
Figure 3 shows the results with thewall clock time as the x -axis, where the gray dashed line denotes the optimum.
The results usingthe number of evaluations as the x -axis are provided in Appendix E, showing that the performance
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Figure]
e w a r d
Walker2D of BO-style methods is similar, as already observed in [19].
This may be because there are manystructures whose objective values are close to the optimum.
But when considering the actual runtime,
MCTS-VS-BO is still clearly better as shown in Figure 3, due to the advantage of variable selection.
MuJoCo Locomotion Tasks.
Next we turn to the more difficult MuJoCo task in RL.
The goal is tofind the parameters of a linear policy maximizing the accumulative reward.
Different from previousproblems, the objective f (i.e., the accumulative reward) is highly stochastic here, making it difficultto solve.
We use the mean of three independent evaluations to estimate f , and limit the evaluationbudget to 1200 due to expensive evaluation.
Note that we do not run ALEBO and VAE-BO because
ALEBO is extremely time-consuming and VAE-BO behaves badly in previous experiments.
Theresults are shown in Figure 4.
TuRBO behaves well on Hopper with a low dimension D = 33 ,and MCTS-VS-TuRBO, combining the advantage of variable selection and TuRBO, achieves betterperformance, outperforming all the other baselines.
On Walker2d with a high dimension D = 102 ,
MCTS-VS-BO performs the best, because of the good scalability.
Most methods have large variancedue to the randomness of f .
For HeSBO, we have little knowledge about the hyper-parameter d , anduse 10 and 20 for Hopper and Walker2d, respectively.
Its performance may be improved by choosinga better d , which, however, requires running the experiment many times, and is time-consuming.
Further Studies.
We further perform sensitivity analysis about the hyper-parameters of MCTS-VS,including the employed optimizer, “fill-in" strategy, C p for calculating UCB, number 2 × N v × N sof sampled data in each iteration, threshold N bad for re-initializing a tree and N split for splitting atree node.
We also conduct experiments on synthetic functions depending on variables to variousextent.
These results are provided in Appendix D and E.
Conclusion
In this paper, we propose the MCTS-VS method for variable selection in high-dimensional BO,which uses MCTS to recursively partition the variables into important and unimportant ones, andonly optimizes those important variables.
Theoretical results suggest selecting as important variablesas possible, which may be of independent interest for variable selection.
Comprehensive experimentson synthetic, NAS-bench and MuJoCo problems demonstrate the effectiveness of MCTS-VS.
However, MCTS-VS relies on the assumption of low effective dimensionality, and might not workwell if the percentage of valid variables is high.
The amount of hyper-parameters might be anotherlimitation, though our sensitivity analysis has shown that the performance of MCTS-VS is notsensitive to most hyper-parameters.
The current theoretical analysis is for general variable selection,while it will be very interesting to perform specific theoretical analysis for MCTS-VS.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See the end of Section 5.1 and
the last paragraph of the paper.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 4.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix B.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See Appendix C,and the code is provided in the supplementary material.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Appendix C.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] We show error bars by the length of vertical bars in the
figures.
(d) Did
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
[Figure]
Figure 5 gives an example of how MCTS-VS bifurcates a leaf node.
Assume that we are to optimizea problem with dimension D = 9 , and the variables are denoted as x 1 , x 2 , . . .
, x 9 .
The Monte
Carlo tree shown in the middle of Figure 5 now has three nodes, i.e., A , B and C , denoted asthe solid circles.
The root A contains all the nine variables.
The current variable score vectors =
[8 . 5 , 8 , 5 , 7 , 3 , 3 , 7 , 10 . 7 , 4 . 5] , which is represented by the bar graph as shown in the left of
Figure 5.
For each i , the value of s i represents the importance of the corresponding variable x i .
Theblue and gray bars denote the important and unimportant variables, respectively, which are containedby the leaf nodes B and C , respectively.
That is, the leaf B contains x 1 , x 2 , x 4 , x 7 , x 8 , and C containsthe remaining x 3 , x 5 , x 6 , x 9 .
The current v values (i.e., the average scores of the contained variables)of the three nodes A , B and C are v A = (8 .
+ 8 + 5 + 7 + 3 + 3 + 7 + 10 .
7 + 4 . 5) / 9 = 6 . 3 ,v B = (8 . 5 + 8 + 7 + 7 + 10 . 7) / 5 = 8 .
24 and v C = (5 + 3 + 3 + 4 . 5) / 4 = 3 . 875 , respectively.
Forthe number that they have been visited, we have n A = 1 , n B = 0 and n C = 0 .
MCTS-VS starts from the root node A at one iteration and recursively selects a node with a larger
UCB value until a leaf node.
According to the way of calculating UCB, the UCB values of theleaf nodes B and C are both ∞ as n B = n C = 0 .
In this case, MCTS-VS will select B and Crandomly.
Assume that B is selected.
The variables (i.e., x 1 , x 2 , x 4 , x 7 and x 8 ) contained by B willthen be optimized by BO with A B = { 1 , 2 , 4 , 7 , 8 } , as in lines 13–23 in Algorithm 1.
After that, thevariable score vector s will be re-calculated, which is assumed to be [9 , 8 . 5 , 5 , 11 , 3 , 3 , 11 , 11 . 2 , 4 . 5] ,as shown in the right of Figure 5.
The average score of the five variables in B is denoted as theorange horizontal line, calculated by (9 + 8 .
5 + 11 + 11 + 11 . 2) / 5 = 10 .
We can see that thevariables x 4 , x 7 and x 8 have score larger than the average value 10 .
14 , which are regarded as moreimportant variables in B .
We use the left child D to represent these variables.
The scores of variablesx 1 and x 2 are smaller than the average, which are regarded as less important variables in B .
Weuse the right child E to represent them.
Thus, the node B has been partitioned into two children Dand E , denoted as the dashed circles in Figure 5.
The v values and the number of visits of thesetwo new leaf nodes are then calculated.
The v value of a node is the average score of the containedvariables.
Thus, v D is the average score of x 4 , x 7 and x 8 , i.e., (11 + 11 + 11 . 2) / 3 = 11 .
067 , andv E is the average score of x 1 and x 2 , i.e., (9 + 8 . 5) / 2 = 8 .
For the number of visits, obviouslyn D = n E = 0 .
Finally, back-propagation is performed to update the v value and the number ofvisits of the nodes along the path from the root A to the node B .
v A is the average score of all thevariables, i.e., (9 + 8 .
5 + 5 + 11 + 3 + 3 + 11 + 11 .
2 + 4 . 5) / 9 = 7 . 356 .
v B is the average score ofx 1 , x 2 , x 4 , x 7 and x 8 , i.e., (9 + 8 .
5 + 11 + 11 + 11 . 2) / 5 = 10 .
Their number of visits will beincreased by 1.
That is, n A = 2 and n B = 1 .
By far, one iteration of MCTS-VS has been finished,and this process will be performed iteratively.
B Details of Theoretical AnalysisB.1 Detailed Proof of Theorem 4.2
The proof is inspired by [34].
To prove the upper bound on the cumulative regret R T in Theorem 4.2,we analyze the instantaneous regret r t = f ( x ∗ ) − f ( x t M t ) , i.e., the gap between the functionvalues of the optimal point x ∗ and the sampled point x t M t at iteration t .
Note that R T =
(cid:80) Tt =1 r t .
Let µ t − 1 ( · ) and σ 2 t − 1 ( · ) denote the posterior mean and variance after running t − 1 iterations,respectively.
Lemma B.1 gives a confidence bound on f ( x t M t ) , leading to a lower bound on f ( x t M t ) ,i.e., f ( x t M t ) ≥ µ t − 1 ( x t M t ) − β 1 / 2 t σ t − 1 ( x t M t ) .
Note that M t denotes the sampled variable indexsubset at iteration t , and | M t | = d t .
Lemma B.1.
∀ δ ∈ (0 , 1) , ∀ t ≥ 1 , let β t = 2 log( π t /δ ) , where (cid:80) t ≥ 1 π − 1 t = 1 , π t > 0 .
Then,∀ t ≥ 1 ,
[Equation]
holds with probability at least 1 − δ , where x t M t is the point obtained at iteration t .
Proof.
At iteration t , f ( x t M t ) ∼ N ( µ t − 1 ( x t M t ) , σ 2 t − 1 ( x t M t )) , and thus, Y = f
( x t M t ) − µ t − 1 ( x t M t ) σ t − 1 ( x t M t ) ∼
N (0 , 1) .
We have
[Equation]
Using the union bound for all t ∈ N , we have
[Equation]
where the equality holds by (cid:80) t ≥ 1 π − 1 t = 1 .
Thus, the lemma holds.
Next we are to analyze the upper bound on f ( x ∗ ) , which can be represented as ( f ( x ∗ )
− f ( x ∗ M t ))
+f ( x ∗ M t ) , where x ∗ M t denotes the point obtained by projecting x ∗ onto M t .
The first term f ( x ∗ ) −f ( x ∗ M t ) can be upper bounded by Assumption 4.1.
To upper bound the second term f ( x ∗ M t ) , weneed to discretize the decision space X M t at iteration t into ˜ X M t , where | ˜ X M t | = ( τ t ) d t , i.e., wedivide each variable of X M t into τ t parts equally.
Let ˜ x ∗ M t denote the point closest to x ∗ M t in thediscretized space ˜ X M t .
Then, we can write f ( x ∗ M t ) as ( f ( x ∗ M t ) − f (˜ x ∗ M t ))
+ f (˜ x ∗ M t ) .
The first termf ( x ∗ M t ) − f (˜ x ∗ M t ) again can be upper bounded by Assumption 4.1.
Lemma B.2 gives a confidencebound on f (˜ x M t ) for any discretized point
˜ x
M t ∈ ˜
X M t , leading to an upper bound on f (˜ x ∗ M t ) , i.e.,f (˜ x ∗ M t ) ≤ µ t − 1 (˜ x ∗ M t )
+ β 1 / 2 t σ t − 1 (˜ x
[Equation]
Lemma B.2.
∀ δ ∈ (0 , 1) , ∀ t ≥ 1 , let β t = 2 log( | ˜ X M t | π t /δ ) , where (cid:80) t ≥ 1 π − 1 t = 1 , π t > 0 .
Then,∀ t ≥ 1 , ∀ ˜ x
M t ∈ ˜
X M t ,
[Equation]
holds with probability at least 1 − δ .
Proof.
Similar to Lemma B.1, we can derive
[Equation]
Using the union bound for all t ∈ N and ˜ x
M t ∈ ˜
X M t , we have
[Equation]
Thus, the lemma holds.
Now, we can upper bound f ( x ∗ M t ) based on Assumption 4.1 and Lemma B.2, as shown in Lemma B.3.
Note that x ∗ M t denotes the point obtained by projecting x ∗ onto M t , and ˜ x ∗ M t denotes the point closestto x ∗ M t in ˜ X M t .
Lemma B.3.
∀ δ ∈ (0 , 1) , t ≥ 1 , let β t = 2 log(2 π t /δ )
+ 2 d t log (cid:16) d t t 2 br (cid:113) log 2 Daδ (cid:17) , where(cid:80) t ≥ 1 π − 1 t = 1 , π t > 0 .
Set τ t
=
d t t 2 br (cid:113) log 2 Da δ and L = b (cid:113) log 2 Da δ .
Then, ∀ t ≥ 1 ,
[Equation]
f (
[Equation]
M
[Equation]
−
[Equation]
− 1 (
[Equation]
| ≤ β
[Equation]
holds with probability at least 1 − δ .
Proof.
First, we have
[Equation]
By Assumption 4.1 with L = b (cid:113) log 2 Daδ , we have ∀ x , y ∈ X , with probability at least 1 − D ·
[Equation] [Equation]
where the second inequality holds by X ⊂ [0 , r ] D , and the last inequality holds by α max =
max i ∈
[ D ] α ∗ i .
Thus, it holds with probability at least 1 − δ/ 2 that
[Equation]
By Lemma B.2 with β t = 2 log(2( τ t )
d t π t /δ )
= 2 log(2 | ˜
X M t | π t /δ ) , we have, with probability atleast 1 − δ/ 2 ,
[Equation]
Applying Eqs.
(5) and (6) to Eq.
(3), it holds with probability at least 1 − δ that
[Equation]
where the second inequality holds by | M t |
= d t and the way of discretization (i.e., each variable isdiscretized into τ t parts equally), and the last inequality holds by the definition of τ t and L .
Thus, thelemma holds.
Lemma B.3 implies an upper bound on f ( x ∗ M t ) , i.e., f ( x ∗ M t ) ≤ µ t − 1 (˜ x ∗ M t ) + β 1 / 2 t σ t − 1 (˜ x ∗ M t ) +α max /t 2 + (cid:80) i ∈
[ D ] \ M t α ∗ i
Lr .
Combining this upper bound on f ( x ∗ M t ) with f ( x ∗ ) − f ( x ∗ M t ) (whichcan be upper bounded by Assumption 4.1), we can derive an upper bound on f ( x ∗ ) .
Togetherwith the lower bound on f ( x t M t ) given by Lemma B.1, we can derive an upper bound on theinstantaneous regret r t .
Thus, we are now ready to prove the upper bound on the cumulative regret
R T in Theorem 4.2, which is re-stated in Theorem B.4 for clearness.
Theorem B.4.
∀ δ ∈ (0 , 1) , let β t
=
2 log(4 π t /δ )
+ 2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) and
L =b (cid:112) log(4 Da/δ ) ,
where { π t } t ≥ 1 satisfies (cid:80) t ≥ 1 π − 1 t = 1 and π t > 0 .
Let β ∗ T = max 1 ≤ i ≤ T β t .
At iteration T , the cumulative regret
[Equation]
(cid:88) t =
[Equation]
(cid:88) i ∈
[Equation]
\ M
[Equation]
α
[Equation]
Lr holds with probability at least 1 − δ , where C 1 > 0 is a constant, γ T = max |D| =
T I ( y D , f D ) , I ( · , · )denotes the information gain, and y D and f D are the noisy and true observations of a set D of points,respectively.
Proof.
For all t ≥ 1 , we have
[Equation]
By Eq.
(4), we have
[Equation]
Note that L = b (cid:112) log(4 Da/δ ) here, and thus Eq.
(8) holds with probability at least 1 − δ/ 4 .
By
Lemma B.3 with β t = 2 log(4 π t /δ )
+ 2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) and
L = b (cid:112) log(4 Da/δ ) ,setting τ t =
d t t 2 br (cid:112) log(4 Da/δ ) leads to that
[Equation]
holds with probability at least 1 − δ/ 2 .
By Lemma B.1 with β t = 2 log(4 π t /δ )
+2 d t log( d t t 2 br (cid:112) log(4 Da/δ )) ≥ 2 log(4 π t /δ ) , it holds with probability at least 1 − δ/ 4 that
[Equation]
Applying Eqs.
(8), (9) and (10) to Eq.
(7), it holds with probability at least 1 − δ that ∀ t ≥ 1 ,r t ≤ (cid:88)
[Equation]
where the second inequality holds because x t M t is generated by maximizing GP-UCB, and thusµ t − 1 (˜ x ∗ M t ) + β 1 / 2 t
[Equation]
By summing up r t from t = 1 to T , we have with probability at least 1 − δ that, ∀ T ≥ 1 ,
[Equation]
(cid:88)
[Equation]
(cid:88)
[Equation]
(cid:88)
[Equation]
(cid:88)
[Equation]
t
[Equation]
(cid:88)
[Equation]
(11) where the second inequality holds by (cid:80) Tt =1 1 /t≤
π 2 /≤ 2 .
Furthermore, let
C 1 = 8 / log(1 + η − 2 ) , and Lemma 5.4 in [34] has shown that (cid:80) Tt =1 2 β 1 / 2 t σ t − 1 ( x t M t
[Equation]
≤
[Equation]
Finally, by applying this inequality to
Eq.
(11), the theorem holds.
We also summarize the main idea of the above proof.
The proof is inspired by [34], i.e., to derivethe upper bound on the gap r t = f ( x ∗ ) − f ( x t M t ) between the function values of the optimal pointx ∗ and the sampled point x t M t at iteration t .
Let x ∗ M t denote the point obtained by projecting x ∗onto M t , and ˜ x ∗ M t denote its closest discretized point.
By utilizing the posterior mean µ t − 1 ( · ) andvariance σ 2 t − 1 ( · ) of f ( x t M t ) and f (˜ x ∗ M t ) , we can have f ( x t M t ) ≥ µ t − 1 ( x t M t ) − β 1 / 2 t σ t − 1 ( x t M t ) andf
( x ∗ ) =
( f ( x ∗ ) − f ( x ∗ M t ))
+ ( f ( x ∗ M t ) − f (˜ x ∗ M t ))
+ f (˜ x ∗ M t ) ≤ (cid:80)
i ∈
[ D ] \ M t α ∗ i Lr + α max /t 2 +(cid:80) i
∈
[ D ] \ M t α ∗ i Lr + µ t − 1 (˜ x ∗ M t ) + β 1 / 2 t σ t − 1 (˜ x ∗ M t ) , where the terms (cid:80) i ∈
[ D ] \ M t α ∗ i
Lr
and α max /t
2are led by variable selection and discretization, respectively.
As x t M t is generated by maximizing
GP-UCB, we have µ t − 1 (˜ x ∗ M t ) + β 1 / 2 t σ t − 1 (˜ x ∗ M t ) ≤ µ t − 1 ( x t M t ) + β 1 / 2 t σ t − 1 ( x t M t ) .
Thus, r t ≤2 β 1 / 2 t σ t − 1 ( x t M t ) + α max /t 2 + 2 (cid:80) i ∈
[ D ] \ M t α ∗ i
Lr .
Finally, summing up r t from t = 1 to T andusing Lemma 5.4 in [34] can lead to Theorem 4.2.
The main difference from the proof of GP-UCB [34] is that variable selection brings some uncertaintyintroduced by the unselected variables.
Based on the Lipschitz condition in Assumption 4.1, theuncertainty by the i -th unselected variable can be upper bounded by α ∗ i Lr , leading to the additionalregret 2 (cid:80) Tt =1 (cid:80) i ∈
[ D ] \ M t α ∗ i Lr in Eq.
B.2 Details of Computational Complexity Analysis
The computational complexity of one iteration of BO depends on three critical components: fitting a
GP surrogate model, maximizing an acquisition function and evaluating a sampled point.
Assumethat the kernel function is squared exponential kernel.
At iteration t , the number of selected variablesis d t .
When fitting a GP model, we calculate the marginal likelihood [28] and gradient as follows:
[Equation]
where y t =
[ y 1 , . . .
, y t ] T , X t =
[ x 1 , . . .
, x t ] , θ are the kernel parameters, K t is the covariancematrix, | · | and tr( · ) denote the determinant and trace of a matrix, respectively.
Then, we can use thegradient-based methods to optimize the likelihood function.
Therefore, the computational complexityof calculating the kernel parameters is O ( t 3 + t 2 d t ) .
Note that θ has been ignored, because itsdimension is much smaller than d t and t .
When calculating the mean µ t ( x ) and variance σ 2 t ( x ) , thecomputational complexity is O ( t 3 + t 2 d t ) , due to the calculation of the kernel matrix and its inverse.
Thus, the total computational complexity of fitting the GP model is O ( t 3 + t 2 d t ) .
Maximizing anacquisition function is related to the optimization algorithm.
If we use the Quasi-Newton method tooptimize GP-UCB, the computational complexity is O ( m ( t 2 + td t + d 2 t ))
[27], where m denotesthe Quasi-Newton’s running rounds.
We note that in BO setting, t will not grow very large.
Therunning rounds m , however, will grow with d t .
Thus, the complexity of optimizing the acquisition function can be much larger than the square of d t .
The cost of evaluating a sampled point is fixed.
Thus, by selecting only a subset of variables, instead of all variables, to optimize, the computationalcomplexity can decrease significantly.
C Method Implementation and Experimental Setting
We use the authors’ reference implementations for TuRBO 1 and LA-MCTS.
For HeSBO and
ALEBO, their implementations in Adaptive Experimentation Platform (Ax 3 ) are used.
We use thepycma library for CMA-ES.
Their hyper-parameters are summarized as follows.
• Vanilla BO.
We use the GP model in Scikit-learn 5 and the qExpectedImprovement acquisi-
tion function [40].
• TuRBO.
We use the default parameter setting in the authors’ reference implementation.
• MCTS-VS.
For the “fill-in" strategy, we use the best- k strategy with k = 20 .
The hyper-
parameter C p for calculating UCB varies on different problems, as shown in Table 2.
We
set all the other parameters to be same on different problems, where the
batch size
N
v
of
variable index set
is 2 ,
the sample batch size N s = 3 , the threshold N bad for re-initializinga tree is 5 , and the threshold N split for splitting a node is 3 .
When using TuRBO as theoptimizer, we limit the maximal number of evaluations in TuRBO to
[Table]
• Dropout.
We set the parameter d to the number of valid dimensions for synthetic functions,and use the same “fill-in" strategy as MCTS-VS.
• LA-MCTS-TuRBO.
We use the same TuRBO setting as MCTS-VS.
The parameter C pis recommended between 1% and 10% of the optimum in LA-MCTS [36].
Because all
our selected values of C p for MCTS-VS have belonged to the recommended range for
LA-MCTS, we use them directly.
The RBF kernel is used for SVM classification.
• HeSBO and ALEBO.
We set the parameter d to the number of valid dimensions for synthetic
functions.
For real-world problems, we do not know the number of valid dimensions, and
thus we just set a reasonable value,
i
e., d = 10 for NAS-Bench, d = 10 for Hopper, and
d = 20 for Walker2d.
• VAE-BO uses VAE for embedding.
That is, VAE-BO uses the encoder to embed the original
high-dimensional space into a low-dimensional subspace, then optimizes via BO in the
subspace and uses the decoder to project the new sampled point back for evaluation.
We set
the learning rate to 0 .
01 and the interval of updating VAE to 30 .
• CMA-ES.
We only adjust the step-size parameter σ for different problems, because the
default setting σ = 0 .
01 leads to extremely poor performance.
We set σ = 0 .
8 for Hartmann
problems, σ = 10 for Levy problems, σ = 0 .
1 for NAS-Bench, and σ = 0 .
01 for MuJoCotasks.
We set the population size to 20 and maintain all the other parameters to default.
The experiments of comparing wall clock time are conducted on Intel(R) Core(TM) i7-10700 CPU@ 2.90GHz and use single thread
.com/uber-research/TuRBO
com/
com/CMA-ES/pycmagithub
com/scikit-learn/scikit-learn
D Sensitivity Analysis of Hyper-parameters of MCTS-VS
We provide further studies to examine the influence of the hyper-parameters of MCTS-VS, includingthe employed optimization algorithm for optimizing the selected variables in each iteration, the “fillin" strategy, the hyper-parameter k used in the best- k strategy, the hyper-parameter C p for calculating
UCB, the number 2 × N v × N s of sampled data in each iteration, the threshold N bad for re-initializinga tree, and the threshold N split for splitting a tree node.
The optimization algorithm is employed by MCTS-VS to optimize the selected variables in eachiteration.
We compare three different optimization algorithms, i.e., random search (RS), BO and
TuRBO.
First, we conduct experiments similar to “Effectiveness of Variable Selection" in Section 5.1,to show the effectiveness of MCTS-VS even when equipped with RS.
Figure 6 shows that MCTSVS-RS is better than Dropout-RS and RS, revealing the advantage of MCTS-VS.
[Figure]
Next we compare the performance of MCTS-VS equipped with RS, BO and TuRBO, by experimentson the Hartmann functions with increasing ratio of valid variables.
Hartmann 6 _ 500 has 6 validvariables.
Hartmann 6 _ 5 _ 500 is generated by mixing 5 Hartmann 6 functions as Hartmann 6( x 1:6 )
+
Hartmann 6( x 7:12 ) + · · · + Hartmann 6( x 25:30 ) , and appending 470 unrelated dimensions, where x i : jdenotes the i -th to j -th variables.
Hartmann 6 _ 10 _ 500 is generated alike.
Thus, Hartmann 6 _ 5 _ 500and Hartmann 6 _ 10 _ 500 have 30 and 60 valid variables, respectively.
The results in Figure 7 showthat as the ratio of valid variables increases, MCTS-VS-TuRBO gradually surpasses MCTS-VS-RSand MCTS-VS-BO, while MCTS-VS-RS becomes worse and worse.
This is expected.
If the ratio ofvalid variables is high, MCTS-VS is more likely to select the valid variables, so it is worth to use theexpensive optimization algorithm, e.g., TuRBO, to optimize the selected variables.
If the ratio is low,unrelated variables are more likely to be selected most of the time, so using a cheap optimizationalgorithm would be better.
These observations also give us some guidance on selecting optimizationalgorithms in practice.
[Figure]
“Fill-in" strategy is a basic component of variable selection methods, which influences the qualityof the value of unselected variables.
We compare the employed best-
k strategy ( k = 20 ) with the average best-
k strategy and the random strategy.
The average best-
k strategy uses the average ofthe best k data points for the unselected variables, and the random strategy samples the value of anunselected variable from its domain randomly.
As shown in Figure 8(a), the random strategy leadsto the poor performance of MCTS-VS-BO, which may be because it does not utilize the historicalinformation and leads to over-exploration.
The best- k strategy utilizes the historical points that havehigh objective values to fill in the unselected variables, thus behaving much better.
The performanceof the average strategy is between the best- k and random strategies.
We recommend using the best- kstrategy in practice.
The hyper-parameter k used in the best-
k strategy controls the degree of exploitation for theunselected variables.
As shown in Figure 8(b), a smaller k encourages exploitation, which resultsin better performance in the early stage, but easily leads to premature convergence.
A larger kencourages exploration and behaves worse in the early stage, but may converge to a better value.
Werecommend using a larger k if allowing enough evaluations.
[Figure]
The hyper-parameter C p for calculating UCB balances the exploration and exploitation of MCTS.
As shown in Figure 9, a too small C p leads to relatively worse performance, highlighting theimportance of exploration.
A too large C p may also lead to over-exploration.
But overall MCTS-VSis not very sensitive to C p .
We recommend setting C p between 1% and 10% of the optimum (i.e.,max f ( x ) ), which is consistent with that for LA-MCTS [36].
[Figure]
The number 2 × N v × N s of sampled data in each iteration depends on the batch size N v ofvariable index subset and the sample batch size N s , and will influence the accuracy of estimatingthe variable score vector in Eq.
If we increase N v and N s , we can calculate the variable scoremore accurately, but also need more evaluations.
Figure 10(a) shows that given the same number ofevaluations, MCTS-VS-BO achieves the best performance when N v = 2 and N s = 3 .
Thus, thissetting may be a good choice to balance the accuracy of variable score and the number of evaluations,which is also used throughout the experiments.
The threshold N bad for re-initializing a tree controls the tolerance of selecting bad tree nodes (i.e.,nodes containing unimportant variables).
A smaller N bad leads to frequent re-initialization, whichcan adjust quickly but may cause under-exploitation of the tree.
A larger N bad can make full use ofthe tree, but may optimize too much on unimportant variables.
Figure 10(b) shows that MCTS-VS achieves the best performance when N bad = 5 .
Thus, we recommend to use this setting, to balancethe re-initialization and exploitation of the tree.
The threshold N split for splitting a node.
If the number of variables in a node is larger than
N split , the node can be further partitioned.
That is, the parameter N split controls the least number ofvariables in a leaf node and thus affects the number of selected variables, which has a direct influenceon the wall clock time.
Note that MCTS-VS selects a leaf node and optimizes the variables containedby this node in each iteration.
The smaller N split , the shorter the time.
Figure 10(c) shows that N splithas little influence on the performance of MCTS-VS-BO, and thus we recommend to set N split
= 3to reduce the wall clock time.
[Figure]
VS-BO on Hartmann 6 _ 300 .
Influence of the hyper-parameters on the runtime of MCTS-VS.
We also provide some intuitiveexplanation about the influence of the hyper-parameters on the runtime.
The threshold N split forsplitting a node has a direct impact on the runtime, because it controls the least number of variablesto be optimized in the leaf node.
That is, the runtime will increase with N split .
Other parametersmay affect the depth of the tree and thus the runtime.
For the threshold N bad for re-initializing atree, if it is set to a small value, MCTS-VS will re-build the tree frequently and the depth of thetree is small.
The shallow nodes have more variables, leading to more runtime to optimize.
For thehyper-parameter C p for calculating UCB, if it is set to a large value, the exploration is preferred and
MCTS-VS will tend to select the right node (regarded as containing unimportant variables).
The treethus will be re-built frequently, leading to more runtime.
For the number 2 × N v × N s of sampleddata at each iteration, if N v and N s are set to large values, the depth of the tree will be small giventhe total number of evaluations, and thus lead to more runtime.
E Additional Experiments
Detailed results on NAS-Bench-101 and NAS-Bench-201.
Figure 11 shows the performance ofthe compared methods on the task of NAS-Bench-101 and NAS-Bench-201 when using the numberof evaluations and wall clock time as the x -axis, respectively.
Though most of their performanceis similar in the left two subfigures, it can be clearly observed from the right two subfigures that
MCTS-VS-BO uses the least time to achieve the best accuracy.
Note that we only show the subfigureswith the wall clock time as the x -axis in the paper due to the space limitation.
Besides, we also run alonger time here (i.e., the right two subfigures) to provide a more complete observation.
Experiments on more NAS-Bench problems.
We also conduct experiments on NAS-Bench1Shot1 [42], TransNAS-Bench-101 [7] and NAS-Bench-ASR [24].
NAS-Bench-1Shot1 is a weightsharing benchmark based on one-shot NAS methods, deriving from the large architecture space of
NASBench-101.
TransNAS-Bench-101 is a benchmark dataset containing network performanceacross seven vision tasks, e.g., object classification, scene classification and so on.
We use thescene classification task with cell-level search space in our experiments.
NAS-Bench-ASR is abenchmark for Automatic Speech Recognition (ASR) and trained on the TIMIT audio dataset.
For
NAS-Bench-ASR, we use Phoneme Error Rate (PER) on the validation dataset as the metric.
Inthe same way as [19], we create problems with D = 33 , D = 24 and D = 30 for NAS-Bench1Shot1, TransNAS-Bench-101 and NAS-Bench-ASR, respectively.
The results in Figure 12 showthat MCTS-VS-BO still uses the least time to achieve the best performance.
[Figure] [Figure]
More explanation for the good performance of MCTS-VS in the MuJoCo experiments.
Figure 4has shown that MCTS-VS can perform well on the two MuJoCo tasks, Hopper and Walker2d, whereeach variable is valid.
This may be because optimizing only a subset of variables is sufficient forachieving the goal and MCTS-VS can select them.
For example, the Walker2D robot consists of fourmain body parts: a torso, two thighs, two legs and two feet, where adjacent ones are connected bytwo hinges.
The goal is to move forward by optimizing the hinges, each of which is valid.
But evenlocking the hinges between legs and feet, the robot can still move forward by optimizing the otherhinges.
This is similar to that when the ankles are fixed, a person can still walk.
Experiments on extremely low and high dimensional problems.
We also evaluate the comparedmethods for extremely low and high dimensional problems by testing on Hartmann 6 _ 100 and
Hartmann 6 _ 1000 .
We only run MCTS-VS, TuRBO, LA-MCTS-TuRBO and HeSBO here, becausethey behave well in the previous experiments.
As expected, the right subfigure of Figure 13 showsthat MCTS-VS-BO has a clear advantage over the rest methods on the extremely high dimensionalfunction Hartmann 6 _ 1000 .
The left subfigure shows that on Hartmann 6 _ 100 , TuRBO behaves thebest and MCTS-VS is the runner-up, implying that MCTS-VS can also tackle low dimensionalproblems to some degree.
Experiments on synthetic functions depending on a subset of variables to various extent.
In the experiments, the synthetic functions are generated by adding unrelated variables directly.
For example, Hartmann 6 _ 500 has the dimension D = 500 , and is generated by appending 494unrelated dimensions to Hartmann with 6 variables.
Here, we test the performance of MCTS-VS ona synthetic function whose dependence on a subset of variables is more various.
For this purpose,we generate Hartmann 6 _ 5 _ 500 _ v by mixing five Hartmann 6 functions as 0 .
5 0 Hartmann 6( x 1:6 )
+0 .
5 1 × Hartmann 6( x 7:12 ) + · · · + 0 .
5 4 Hartmann 6( x 25:30 ) , and appending 470 unrelated dimensions,where x i : j denotes the i -th to j -th variables, and different coefficients represent various degrees ofdependence.
The results in Figure 14 show that MCTS-VS-BO performs the best.
Experiments with increasing ratio of valid variables.
We also examine the performance of
MCTS-VS when the percentage of valid variables increases.
We use the synthetic function
Hartmann 6 _ 500 , and generate the variants with more valid variables by mixing multiple Hartmann 6functions as in Appendix D.
For example, Hartmann 6 _ 5 _ 500 is generated by mixing five Hartmann 6functions as Hartmann 6( x 1:6 )+ Hartmann 6( x 7:12 ) + · · · + Hartmann 6( x 25:30 ) , and appending470 unrelated dimensions.
We have compared MCTS-VS-TuRBO with LA-MCTS-TuRBO and
[Figure]
TuRBO
HeSBO
[Figure]
[Figure]
TuRBO on Hartmann 6 _ 500 , Hartmann 6 _ 5 _ 500 , Hartmann 6 _ 10 _ 500 , . . .
, Hartmann 6 _ 30 _ 500 , and
Hartmann 6 _ 83 _ 500 , which has the largest number (i.e., 6 × 83 = 498 ) of valid variables.
Theresults are shown in Figure 15.
It can be observed that LA-MCTS-TuRBO performs the worst.
Asexpected, when the percentage of valid variables is low (e.g., in Hartmann 6 _ 500 , Hartmann 6 _ 5 _ 500and Hartmann 6 _ 10 _ 500 ), MCTS-VS-TuRBO can be better than TuRBO; but as the percentage ofvalid variables increases, TuRBO becomes better, because a leaf node of MCTS can contain only asmall fraction of valid variables.
[Figure]
Comparison with LASSO-VS.
There are other variable selection methods (e.g., LASSO), whichare not designed for high dimensional BO but can be used directly.
We have implemented the
LASSO-based variable selection method, named LASSO-VS.
We compare MCTS-VS, LASSO-VSand Dropout on the synthetic function Hartmann 6 _ 300 .
When using LASSO-VS, the d variableswith the largest absolute values of the regression coefficients are selected at each iteration.
Theresults are shown in Figure 16.
When equipped with either BO or TuRBO, the proposed MCTS-VSalways performs the best.
We can also observe that when equipped with BO, LASSO-VS can evenbe worse than Dropout.
This may be because many of existing variable selection methods (e.g.,
LASSO) usually require a large number of samples to fit the linear regression model well, while in
BO scenarios, only a limited number of samples can be evaluated.
[Figure]
tests.
As most of the previous works, we have conducted experiments using 5 randomseeds (2021–2025).
Here, we also conduct statistical tests on Hartmann and Levy functions byrunning the methods for 50 times (random seeds 2021–2070), to make a more confident comparison.
Considering the performance and runtime of the methods we have observed in Figure 2, we onlycompare MCTS-VS with LA-MCTS-TuRBO and TuRBO, which achieve good performance inacceptable time.
The results are shown in Table 3.
MCTS-VS-BO achieves the best average objectivevalue on all the synthetic functions except Levy 10 _ 100 where the dimension is relatively low and
TuRBO performs the best.
By the Wilcoxon signed-rank test with confidence level 0.05, MCTS-VS-
[Figure]
TuRBO is significantly better than LA-MCTS-TuRBO on all the synthetic functions, showing theadvantage of MCTS-VS over LA-MCTS for variable selection.
Compared with TuRBO, MCTS-VSTuRBO is only significantly better on Hartmann functions, which may be because the ratio of validvariables of Hartmann 6 _ 300 and Hartmann 6 _ 500 is lower than that of Levy 10 _ 100 and Levy 10 _ 300 ,and thus the advantage of performing variable selection by MCTS-VS is more clear.
Note that theobservations about the performance rank of the compared methods are consistent with that observedin Figure 2, which plot the results of the compared methods by running five times.
Problem
[Table] [Table]
[Table] [Table]
F Enlargement of Some Figures in the Main Paper
Due to space limitation, Figures 1 and 2 in the main paper are a little small.
Here, we also providetheir enlarged versions, i
[Table]
e., Figures 17 and 18.
[Figure] [Figure]
[Figure]
Vanilla BO
Dropout-BO
[Figure]
[Figure] [Figure] [Figure] [Figure]
[Figure]
MCTS-VS-BO
MCTS-VS-SAASBO
SAASBO [Figure] [Figure] MCTS-VS-BO
MCTS-VS-SAASBO
SAASBO
[Figure]