## 2 Materials and Methods
### The Proposed BO-Stacking-TabNet Model
#### 2.5.3 Stacking-Encoder Architecture  
This study builds upon the TabNet model and integrates the idea of Stacking ensemble learning to improve the Encoder architecture, thereby further enhancing the performance of the TabNet model. Additionally, the Bayesian optimization algorithm is used to optimize the hyperparameters of both the base learners and the meta-learner [47]. Stacking is a technique that improves prediction performance by combining multiple different models. The core idea is to employ a two-layer model consisting of base learners and a meta-learner to achieve a hierarchical structure. The outputs of multiple models are used as new inputs to construct a model with improved performance. Figure 5 illustrates the overall architecture of the Stacking-Encoder in the BO-Stacking-TabNet model. In this architecture, the BO-Stacking-TabNet model is composed of two Encoder structures with similar architecture but different hyperparameters, serving as the base learners and meta-learner in the Stacking structure. These two Encoder architectures are optimized for hyperparameters using the Bayesian optimization algorithm prior to training, ensuring that the model achieves optimal training performance. Regarding the Encoder architecture, the original numerical features are fed directly into the model without any global feature normalization, and only batch normalization (BN) is applied to the input features. The same-\(D\)-dimensional original input features \(f_{\text{base}}\in R^{B\times D}\) are passed to each decision step of the base learner Encoder. The category probability distribution output by the base learner Encoder serves as the input features for the meta-learner Encoder \(f_{meta}\in R^{B\times L}\), where \(B\) represents the batch size and \(L\) represents the number of categories. The encoding process in TabNet is based on \(N_{\text{steps}}\) sequential multi-step processing, where information processed in step \((i-1)^{th}\) is used as input for step  
Figure 4: Module structure of the Attentive Transformer. Figure 5: Overall architecture of the Stacking-Encoder. \(i^{th}\), to determine which features to use and output the processed feature representations, which are then aggregated into the overall decision. The data processing flow of the Stacking-Encoder is roughly divided into two stages: feature selection and feature processing. In the feature selection stage, the model sets learnable masks to perform soft selection of significant features. The base learner Encoder mask is denoted as \(M_{base}[i]\in R^{B\times D}\), and the meta-learner Encoder mask is denoted as \(M_{meta}[i]\in R^{B\times L}\). By sparsely selecting the most significant features, the learning capacity of the decision steps is not wasted on irrelevant features, thus improving the parameter efficiency of the model. The mask is multiplicative \(M[i]\cdot f\), and at the same time, an attentive transformer is used to obtain the mask based on the features processed in the previous step, i.e., \(a[i-1]:M[i]=\ sparsemax(P[i-1]\cdot h_{i}(a[i-1]))\). Moreover, \(P[i]\) is the prior scale term, which represents how many specific features were previously used, i.e., \(P[i]=\prod\limits_{j=1}^{i}(\gamma-M[j])\), where \(\gamma\) is the relaxation parameter. When all values are initialized to 1, that is, \(1^{B\times D}\) or \(1^{B\times L}\), there is no prior on the masked features. Additionally, the model uses sparsity regularization to further control the sparsity of the selected features, i.e., \(L_{sparse}=\sum\limits_{i=1}^{N_{steps}}\sum\limits_{b=1}^{B}\sum\limits_{ j=1}^{D}\frac{-M_{bj}[i]\log\left(M_{b,j}[i]+\varepsilon\right)}{N_{steps}\cdot B}\) or \(L_{sparse}=\sum\limits_{i=1}^{N_{steps}}\sum\limits_{b=1}^{B}\sum\limits_{ j=1}^{L}\frac{-M_{bj}[i]\log\left(M_{b,j}[i]+\varepsilon\right)}{N_{steps}\cdot B}\), where \(\varepsilon\) is a numerical stability factor. The sparsity regularization is then added to the overall loss function, weighted by a sparsity coefficient \(\lambda\).