## 3 Methodology for Model Solving
### Kernel Regression UCT for Truss Layout Design  
The UCT algorithm has been proven to be effective for truss generation [20]; however, there remains a problem dealing with continuous action space. Notice that the coordinates of newly added nodes and cross-sectional areas of bars are often continuous variables. The strategy for uniformly discretizing the continuous decision space would cause exponential growth of candidate nodes in 3D cases; therefore, it is worth looking for some method to discretize the action space while controlling the size of candidate sets and the value of each action in the collection. That is, a continuous action set, which is of finite size and heuristically generated, is required. To control the size of the candidate set, one direct approach is to gradually expand the width of the search tree as the number of loop iteration grows. More concretely, the \(n\)th candidate action is added to the search tree after \(T(n)\) loops of MCTS have proceeded. This  
Figure 1: Truss layout MDP model represented by the search tree. approach was independently introduced in [27] named progressive widening. Commonly, \(T(n)\) can be set to a power function. In this paper, \(T(n)\) is set to be \(3n^{2}\). When adding a new action to the candidate set, the previous search information should be considered to give a reference for choosing high-value candidate actions. This paper uses the Gaussian Radial Basis Function [28] to evaluate the neighborhood information, which is a nonparametric fit to the existing search experience. More concretely, when sampling an action with a continuous variable vector \(\overrightarrow{\mathbf{x}}\) once, another action \(\overrightarrow{\mathbf{y}}\) is treated to be sampled  
\[K\left(\overrightarrow{\mathbf{x}},\ \overrightarrow{\mathbf{y}}\right)=\ \exp\left(- \frac{\|\overrightarrow{\mathbf{x}}-\overrightarrow{\mathbf{y}}\|^{2}}{2\sigma^{2}}\right) \tag{6}\]  
times. Assume \(m\) simulations are processed, vector \(\overrightarrow{\mathbf{x}}_{i}\) is sampled with a reward of \(r_{i}\) in \(i\)th simulation and the vector \(\overrightarrow{\mathbf{x}}_{i}\) is sampled \(n_{\overrightarrow{\mathbf{x}}_{i}}\) times. For an action vector \(\overrightarrow{\mathbf{y}}\), the weighting function can be defined as  
\[W\left(\overrightarrow{\mathbf{y}}\right)=\sum_{i=1}^{m}K\left(\overrightarrow{ \mathbf{x}}_{i},\ \overrightarrow{\mathbf{y}}\right)n_{\overrightarrow{\mathbf{x}}_{i}}. \tag{7}\]  
Similarly, the average reward of an action vector \(\overrightarrow{\mathbf{y}}\) can be approximately evaluated by  
\[\overline{v}\left(\overrightarrow{\mathbf{y}}\right)=\frac{W\left(\overrightarrow {\mathbf{y}}\right)r_{i}}{W\left(\overrightarrow{\mathbf{y}}\right)}=\frac{\sum_{i=1 }^{m}K\left(\overrightarrow{\mathbf{x}}_{i},\ \overrightarrow{\mathbf{y}}\right)n_{\overrightarrow{\mathbf{x}}_{i}}r_{i}}{\sum_{i= 1}^{m}K\left(\overrightarrow{\mathbf{x}}_{i},\ \overrightarrow{\mathbf{y}}\right)n_{ \overrightarrow{\mathbf{x}}_{i}}} \tag{8}\]  
using Equations (7) and (8), the adjusted upper confidence bound in our approach can be written as  
\[U_{\overrightarrow{y}}=\alpha\times\overline{v}\left(\overrightarrow{\mathbf{y }}\right)+(1-\alpha)\times vbest_{\overrightarrow{\mathbf{y}}}+C\times\sqrt{ \frac{\ln\left(\sum_{\mathbf{\widetilde{x}}}W\left(\overrightarrow{\mathbf{x}} \right)\right)}{W\left(\overrightarrow{\mathbf{y}}\right)}} \tag{9}\]  
Notice that the constraints in the generation process are strict, indicating that a small difference in node position or bar area may let the structure fail to pass a constraint. Therefore, the optimal structure found by the algorithm cannot be evaluated around the neighborhood. When \(\overrightarrow{\mathbf{y}}\) has not been simulated, \(vbest_{\overrightarrow{\mathbf{y}}}\) is initially set to 0. The selection strategy of a new action should discover the unknown area of the continuous space with a high expected reward. To discover the unknown area, the weight function of the new action should be small.