# FFNN-TabNet: An Enhanced Stellar Age Determination Method Based on TabNet
## 3 Method
### The FFNN-TabNet
#### 3.2.2 The FFNN-TabNet Architecture  
The FFNN-TabNet framework operates through a streamlined process, wherein the final feature vector is meticulously constructed via a series of continuous feature selections and the aggregation of pertinent information, ultimately facilitating the execution of decision-making tasks. The architecture of the FFNN-TabNet's encoder and decoder is delineated in Figure 2.  
In this methodology, the red giant dataset, comprised of \(M\) data instances each with \(D\) features as an input to TabNet, R is the vector set. The model processes the feature  
Figure 2: The architecture of the FFNN–TabNet’s encoder and decoder. In contrast to the original TabNet by [PERSON] (2021) [12], our implementation has made improvements in activation function and Feature Transformer modules. vector \(Y\in R^{M\times D}\) at each stage. The detailed feature-selection mechanism encompasses several phases:  
1. Initially, the primal feature vector \(Y\) undergoes normalization in a batch normalization (BN) [58] layer, followed by computational processing in the Feature Transformer layer. Within this layer, data traverses a shared parameter layer constituted of two FC + BN + GLU units, connected via a residual network and subsequently normalized by a factor of 5.0. This layer's chief purpose is to deduce commonalities among features. Subsequent to this layer, data proceeds to an independent parameter layer, mirroring the preceding layer's architecture [12]. The feature transformer layer architecture is depicted in Figure 3.
2. Subsequent to its processing through the Feature Transformer layer, the red giant dataset advances into the Split module. This module functions to segregate the output derived from the initial Feature Transformer, thereby isolating and acquiring the features \(a[i-1]\) in the first step, specifically when \(i=1\). The formula is shown in Equation (3). \[\left[d[i],a[i]\right]=f_{i}(M[i]\cdot f)\] (3) In Equation (3), the term \(d[i]\) denotes the current decision step, while \(a[i]\) denotes the input of the next decision step. 3. Subsequent to traversing the Split module, the dataset proceeds into the Attentive Transformer layer, analogous to the Mask layer depicted in Figure 1. The primary role of this layer is the computation of the current step's Mask, which is essential for feature selection. The Attentive Transformer layer is intricately composed of an FC layer, BN layer, and a Sparsemax layer, along with a weighted scaling factor as a foundational scale component [12]. Owing to the intricate non-linear interrelations among the features of red giant star data, in the current research, we have strategically integrated a FFNN upstream of the BN layer. The attentive transformer architecture is illustrated in Figure 4.  
Figure 4: The attentive transformer architecture. In contrast to the original TabNet by [PERSON] (2021) [12], our implementation has added the FFNN module to improve performance. Figure 3: The feature transformer layer architecture. (1) Feedforward Neural Network (FFNN)  
As illustrated in Figure 4, in the architecture of the attentive transformer, the FFNN module is positioned between the FC layer and the BN layer. FFNN module consists of two sequences of FC and PReLU layers. The formula is shown in Equation (4). \[\text{FFNN}(x)=\text{PReLU}(\text{FC}_{2}(\text{PReLU}(\text{FC}_{1}(x)))) \tag{4}\]  
The integration of the FFNN module significantly enhances the TabNet model's proficiency in comprehending and leveraging the intrinsic characteristics of features, thereby substantially amplifying its capacity to discern and interpret complex nonlinear interrelationships. This architectural innovation plays a pivotal role in advancing the model's analytical capabilities. (2) Sparsemax [59]  
The Sparsemax layer serves to induce sparsity in the output of the Attentive Transformer. It accomplishes this by directly mapping the feature vector onto a simplex, thereby effectuating sparsification. The formula is delineated in Equation (5).