# Enhancing Heart Disease Prediction with Federated Learning and Blockchain Integration
## 3 Model Construction and Training Setup
### TabNet Model Architecture  
For tasks using tabular data, tree-based models have better performance than other models, and Boosting tree-based models like XGBoost [24] and LightGBM [25] have become the standard in data mining competitions nowadays. In this project, we need to keep updating the local models by online learning. However, the tree-based model can only be retrained with the whole dataset because the decision tree structure will be destroyed and rebuilt in the training process. Consequently, we chose TabNet, a DNN model with a fixed model structure that also features tree-based models. It has the advantages of interpretability and sparse feature selection of tree-based models while retaining the end-to-end and representation learning features of DNNs. (1) Building a decision tree using a DNN. Figure 2 illustrates a DNN structured similarly to a decision tree. This architecture processes inputs through masked, fully connected (FC) layers that isolate specific features. Non-linearity and classification are achieved through ReLU and softmax layers, respectively. The diagram also details how weight (\(W\)) and bias (\(b\)) influence decision paths, where both \(W\) and \(b\) are represented as four-dimensional vectors, corresponding to different layers of the network. Initially, input features \(x_{1}\) and \(x_{2}\) are filtered by mask layers, then passed through fully connected layers with explicitly set weights and biases. Variables like \(C\) and \(D\) represent feature transformation and decision layers, while \(a\) refers to the attention mechanism applied during feature selection. Weights and biases are key to transforming and combining input features to make predictions. Weights, learned during training, adjust the strength of connections between neurons, while biases shift activation functions for better data fitting. In TabNet, the attention mechanism uses weights to emphasize essential features and suppress irrelevant ones, while biases further refine layer outputs, ensuring the model captures complex relationships. These parameters enable the TabNet model to process tabular data and make accurate predictions efficiently. The ReLU activation function sums up the outputs of the two FC layers and passes through a softmax activation function as the final output. Let us compare it with the decision tree process. We can find that each layer of this neural network corresponds to the corresponding step of the decision tree: The mask layer corresponds to feature selection in the decision tree, which is well understood; FC layer+ReLU corresponds to the threshold judgment. Take \(x_{1}\) as an example:  
After passing through the FC layer, followed by ReLU activation, the model ensures that only one element of the output vector remains positive (i.e., greater than zero). In contrast, the other elements are set to zero. This process mirrors the conditional decisions made in a decision tree, where a single decision path is chosen based on feature thresholds. This corresponds to the conditional judgment of the decision tree; finally, the results of all the conditional judgments are added up, and the final output is obtained through a softmax layer. (2) The model structure of TabNet is illustrated in Figure 3. This figure outlines the architecture of TabNet for tabular data processing, showcasing the sequence of operations, including batch normalization, feature transformation, attentive transformation, masking, and aggregation, which culminate in the output through ReLU activation and a fully connected layer. This model shares a framework similar to traditional neural networks, functioning as an additive model with multiple processing steps. The input to the model consists of features with dimensions \(B\times D\), where \(B\) represents the batch size and \(D\) denotes the dimension of the features. The output of the model is a vector representing the classification result. Figure 3: TabNet model architecture. Figure 2: Decision tree with DNN architecture. BN denotes the batch normalization layer.