## 4 Results and Discussion
### Feature Importance Analysis  
Feature importance analysis (Table 5) revealed distinct patterns in how each architecture weighted the contribution of water quality parameters to potability classification. The EnhancedTabTransformer identified chloramines (0.102632) as the most significant parameter, followed by conductivity (0.075384), and trihalomethanes (0.070908). This aligns with established water quality standards, as chloramines serve as disinfection agents with strict regulatory thresholds due to their potential health impacts at elevated concentrations.  
The SAINT architecture assigned the highest importance to pH (0.194173), followed by sulfate (0.188918) and TDS solids (0.172967), suggesting that this model places greater emphasis on the chemical balance aspects of water quality. The FITransformer displayed a more uniform distribution of feature importance values (ranging from 0.454429 to 0.564652), with sulfate (0.564652) receiving marginally higher importance. This more balanced feature weighting may partially explain the FITransformer's lower overall performance, as it might not sufficiently prioritize the most discriminative parameters.  
The lower importance of features such as turbidity, total dissolved solids (TDS), and organic carbon across models suggests that these parameters may have less discriminatory power in this dataset. This could be attributed to their relatively narrow range of variation (e.g., turbidity: 1.45-6.49 NTU, Table 5) or their indirect relationship to potability compared to chloramines and trihalomethanes, which are directly linked to health risks. These insights  
\begin{table}
\begin{tabular}{l l l l l} \hline  & **Precision** & **Recall** & **f1-Score** & **Support** \\ \hline Non-drinkable & 0.96 & 0.96 & 0.96 & 240 \\ Drinkable & 0.94 & 0.94 & 0.94 & 163 \\ Accuracy & & & 0.95 & 403 \\ Macro avg & 0.95 & 0.95 & 0.95 & 403 \\ Weighted avg & 0.95 & 0.95 & 0.95 & 403 \\ \hline \end{tabular}
\end{table}
Table 4: Classification report of the EnhancedTabTransformer model across two distinct classes.  
\begin{table}
\begin{tabular}{l c c c c} \hline  & \multicolumn{4}{c}{**Feature Importances**} \\ \multicolumn{2}{c}{**Features**} & \multicolumn{2}{c}{**Transformer-Based Deep Neural Network Models**} & \\ \multicolumn{2}{c}{**Features**} & \multicolumn{2}{c}{**ETabTransformer**} & **TAPE** & **SAINT** & **FITransformer** \\ \hline Chloramines (ppm) & 0.102632 & & 0.163053 & 0.476386 \\ Conductivity (MikroS/cm) & 0.075384 & & 0.152160 & 0.462036 \\ Trihalomethanes (mikroS/L) & 0.070908 & & 0.139151 & 0.517876 \\ ph & 0.067879 & & 0.194173 & 0.519935 \\ Hardness (mg/L) & 0.056796 & & 0.169163 & 0.502482 \\ Sulfate (mg/L) & 0.056418 & & 0.188918 & 0.564652 \\ Turbidity (NTU) & 0.054839 & & 0.140685 & 0.454429 \\ TDS solids (ppm = mg/L) & 0.046426 & & 0.172967 & 0.465583 \\ Organic\_carbon (ppm) & 0.045725 & & 0.156996 & 0.507146 \\ \hline \end{tabular}
\end{table}
Table 5: Feature importances derived from top transformer-based deep neural network models.  
can guide sensor prioritization in water quality monitoring systems, focusing resources on measuring high-impact parameters to optimize cost and efficiency. Unfortunately, feature importance values could not be extracted for the TAPE architecture due to its complex autoencoder structure, which does not contain a suitable layer for direct weight interpretation. This highlights a limitation in model interpretability for certain deep learning architectures, particularly those employing unsupervised pretraining methodologies.