## 1 Introduction  
Generative design is an intelligent design method that automatically explores and generates architectural design layouts on the premise of satisfying multiple design requirements [1]. It can be realized through structural optimization methods such as size, shape, and topology optimization [2; 3; 4]. As for truss structures, the optimal layout design is often represented as a combinational structural optimization problem, simultaneously considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars [2]. The design variables involve both continuous and discrete types since node locations are commonly continuous variables and connections between nodes are typical discrete variables, which greatly increases the complexity of the solution space. The ground structure method first proposed by [PERSON] [5] is an effective and classical strategy to limit the solution space of such truss layout optimization problems and has been widely incorporated into many automatic optimization algorithms of truss layout [6; 7; 8; 9]. The ground structure is formed by a finite number of predefined nodes and members in the design domain and the optimal truss layout can be generated by eliminating unnecessary bars and nodes in the ground structure [10]. Usually, node locations of the ground structure are uniformly sampled in the design domain; however, in order to reach a good auto-design performance, it is often necessary to discretize the design domain with a small step size. This leads to limitations in a large range of two-dimensional design domains or three-dimensional design domains for ground-structure-based methods [11]. For example, when uniformly generating nodes of the ground structure (each dimension is discretized into \(k\) nodes), the number of the node candidate set is \(k^{2}\) for two-dimensional design cases and increases to \(k^{3}\) in three-dimensional design cases. Moreover, the total number of design variables becomes larger since the bar connections and cross-sectional areas also need to be considered. This makes the design task challenging to find the optimal truss layout [12]. The engineering design problems often need to consider constraints related to structural performance, which usually makes such optimization tasks suffer from non-convexity and non-differentiability. Many previous studies adopted search-based methods [7; 9; 13; 14; 15] to find an approximate global optimal solution instead of mathematical programming methods. To solve such a huge solution space of the truss layout design, this optimal design task can be regarded as a decision-making problem and formulated into a Markov Decision Process (MDP) [16]. In the MDP, it is not necessary to consider all the decision variables at the same time; however, according to the decision logic of engineers, the generation of the optimal truss layout design can be gradually completed by sequential decisions [17; 18]. By defining three kinds of sequential actions, which are adding nodes, adding bars, and selecting the cross-section of bars, the optimal sequence of generating actions can be obtained via Upper Confidence Trees (UCT) [19] search method, which is a reinforcement learning algorithm based on Monte Carlo Tree Search (MCTS) [20]. This UCT method has been successfully applied to the generation of planar trusses; however, there are two main problems to be solved in the generation of 3D structures by using this tree search strategy. First, the UCT algorithm also discretizes continuous variables (node location, cross-sectional area of bars); thus, the number of candidate variables increases exponentially. Although the two-stage gradual discretization strategy can be adopted, the number of variables is still a challenging problem restricting the search efficiency in 3D cases. Second, for the discretization method, the general strategy is to adopt a uniform distribution.