# FFNN-TabNet: An Enhanced Stellar Age Determination Method Based on TabNet
## 3 Method
### The FFNN-TabNet
#### 3.2.1 TabNet Neural Network Decision Process  
The TabNet [12] neural network architecture employs a strategically devised fully connected (FC) [57] layer, coupled with the ReLU function, to adeptly simulate the decision-making paradigm characteristic of tree models. This innovative approach enables the network to replicate the intricate decision processes inherent in these models, thereby enhancing its analytical capabilities. In this research, the standard ReLU activation function has been supplanted by the more versatile PReLU. Decision process is delineated in Figure 1.  
This modified model structure, as illustrated in Figure 1, initiates the decision process by inputting a feature vector [\(x_{1}\), \(x_{2}\)] into the TabNet neural network. After traversing  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Description** \\ \hline \(T_{\text{eff}}\) (K) & Effective temperature \\ \(\log\)\(g\) (dex) & Surface gravity \\ mass & Red giant mass \\ alpha & \(\alpha\) elements \\ \multicolumn{2}{c}{Chemical abundances, logarithm of the ratio of iron,} \\ Fe\_H, N\_H, Ca\_H, V\_H, C\_H, P\_H, & nitrogen, calcium, vanadium, carbon, phosphorus, \\ Ni\_H, Na\_H, Mg\_H, K\_H, Co\_H, & nickel, natrium, magnesium, potassium, cobalt, titanium, \\ Ti\_H, Al\_H, Mn\_H, O\_H, Cr\_H & aluminum, manganese, oxygen, chromium to \\  & hydrogen atoms \\ Age (Gyr) & red giant age \\ \hline \hline \end{tabular}
\end{table}
Table 1: Red giant stars data parameters.  
Figure 1: TabNet neural network decision process. In contrast to the original TabNet by [PERSON] and [PERSON] (2021) [12], which employs ReLU as the activation function, our implementation has utilized PReLU for improved performance.  
the Mask layer, which selectively filters features \(x_{1}\) and \(x_{2}\) from the vector, these features undergo a linear transformation via a fully connected layer, custom-tailored with distinct weights and biases. Activation functions are pivotal in augmenting the learning capabilities of neural networks, serving as a fundamental component that significantly influences the network's ability to process complex data patterns and learn from them effectively. The ReLU activation function traditionally manipulates each vector element, ensuring a singular positive element per vector, designating the chosen feature and assigning zero to all other elements, denoting their non-selection. The formula for ReLU is shown in Equation (1).  
\[f(x)=\max(0,x) \tag{1}\]  
However, this approach can induce gradient degradation or disappearance, particularly when inputs turn negative, resulting in the deactivation of neurons and their unresponsiveness to subsequent data variations. To mitigate these limitations, the study introduces the PReLU activation function. PReLU introduces a learnable parameter, \(\alpha\), which ensures a consistent non-zero gradient, even for negative input values. This innovative approach effectively addresses the issue of gradient vanishing, thereby significantly enhancing the model's training efficiency and facilitating improved gradient flow throughout the network. Additionally, PReLU's allowance for minor activation values in negative inputs addresses the issue of \"dead neurons\". The decision process culminates with the fully connected layer and PReLU activation function performing conditional judgments, linearly amalgamating the decision outcomes of both features, with the final output determined by the Softmax function. The formula for PReLU is shown in Equation (2).  
\[f(x)=\max(0,x)+\min(0,x) \tag{2}\]