## 3 Materials and Methods
### Enhanced Tabular Transformer Framework  
The enhanced tabular transformer architecture represents a specialized adaptation of the canonical transformer framework [28] specifically optimized for tabular data analysis. This architecture incorporates the following:  
Feature embedding layer: transforms the input features into a high-dimensional representation space through a non-linear projection:  
\[E(x)=\delta(\mathrm{LN}(W_{e}x+b_{e})) \tag{3}\]  
where \(\delta\) represents the rectified linear unit activation function [28], LN denotes layer normalization, and \(W_{e}\in R^{d\times h}\) and \(b_{e}\in R^{h}\) are learnable parameters mapping the \(d\)-dimensional input to an \(h\)-dimensional hidden representation. Self-attention mechanism: Enables the model to capture complex relationships between different water quality parameters through multi-head attention. This embedding is \begin{table}
\begin{tabular}{p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Model** & \begin{tabular}{c} **Column-Wise** \\ **Input Handling** \\ \end{tabular} & \begin{tabular}{c} **Inter-Sample** \\ **Attention** \\ \end{tabular} & \begin{tabular}{c} **Pretraining** \\ **Strategy** \\ \end{tabular} & \begin{tabular}{c} **Core Strengths** \\ \end{tabular} \\ \hline Enhanced tab transformer (ETT) & Column-wise embedding with ReLU & None & None & \begin{tabular}{c} Captures complex feature interactions, robust to \\ high-variance attributes. \\ \end{tabular} \\ \hline FTTransformer & Feature-specific tokenization & None & None & \begin{tabular}{c} Effective for attribute \\ interactions, handles \\ class imbalance. \\ \end{tabular} \\ \hline SAINT & \begin{tabular}{c} Column-wise embedding with dual-attention \\ \end{tabular} & Yes & \begin{tabular}{c} Contrastive pretraining \\ \end{tabular} & \begin{tabular}{c} Leverages sample-level \\ patterns, suited for \\ moderate datasets. \\ \end{tabular} \\ \hline TAPE & \begin{tabular}{c} Multi-layer encoder \\ for attributes \\ \end{tabular} & None & \
\begin{tabular}{c} Unsupervised \\ autoencoder \\ \end{tabular} &
\begin{tabular}{c} Learns intrinsic data \\ structures, effective for \\ environmental data. \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of deep learning architectures for water potability classification. followed by a scaled dot product attention mechanism, which computes attention scores to focus on relevant features:  
\[\text{Attention}(Q,K,V)=\text{softmax}\bigg{(}\frac{QK^{T}}{\sqrt{d_{k}}}\bigg{)}V \tag{4}\]  
where \(Q,\ K,\text{ and }V\) represent query, key, and value projections, respectively, and \(d_{k}\) is the dimensionality of the key vectors. These components, introduced by [PERSON] et al. [28], form the basis of the transformer encoder block used in ETT and FTTransformer. This formulation allows the model to attend differently to various feature interactions through multiple attention heads. \[\text{MultiHead}(Q,K,V)=\text{Concat}(head_{1},head_{2},\ldots,head_{h})W^{ \text{O}} \tag{5}\]  
where \(head_{i}=\text{Attention}\Big{(}QW_{i}^{Q},KW_{i}^{K},VW_{i}^{V}\Big{)}\), with projection matrices \(W_{i}^{Q},W_{i}^{K},W_{i}^{V}\in R^{h\times d_{k}}\) and \(W^{\text{O}}\in R^{hd_{k}\times h}\). Here, \(d_{k}=h/h_{heads}\) represents the dimension per attention head. The feed-forward network (FFN) applies a two-layer transformation with GELU activation:  
\[\text{FFN}(x)=\text{GELU}(xW_{1}+b_{1})W_{2}+b_{2} \tag{6}\]  
where GELU denotes the Gaussian error linear unit activation function [29], providing smoother gradients compared to ReLU.