# Enhancing Software Quality with AI: A Transformer-Based Approach for Code Smell Detection
## 3 Methodology
### Data Preparation  
The training of the AI system began with a structured Java class dataset consisting of 1000 entries divided into large class (contains code smell) and not a large class (does not contain code smell) groups. A total of 20 metrics form the dataset, which had been chosen specifically to identify elements that define code structure and maintainability. The preprocessing pipeline consisted of the following steps:  
1. Data cleaning: We cleaned the dataset by replacing missing values in numerical data with the median of the corresponding feature. This prevented the loss of information while training the models.
2. Normalization: We normalized all features using the min-max scaling method so that the feature values range between 0 and 1. This normalization was done to have features with large magnitudes not to take the center stage in modeling.
3. Feature selection: Features with a Pearson correlation coefficient > 0.2 of the target labels were chosen. This approach eliminated noise and retained features most likely to contain smell indicators.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Study** & **Year** & **Methodology** & **Key Findings** & **Reference** \\ \hline \multirow{4}{*}{Alazba (CoRT)} & \multirow{4}{*}{2022} & Transformer-based model & Significantly improved & \multirow{4}{*}{[5]} \\  & & using self-supervised & detection of code smells by & \\  & & learning for semantic and & learning semantic and & \\  & & structural analysis. & structural features. & \\  & & Integrated pretrained & Enhanced accuracy compared & \\  & & transformers with & to traditional methods by & \\  & & stacking techniques. & leveraging textual features in & \\  & & & code analysis. & \\  & & & Demonstrated strong & \\  & & Graph neural networks & performance on tasks & \\  & & (GNNs) for learning & involving relationships in code & \\  & & semantic representations. & structures but struggled & \\  & & & with scalability. & \\  & & & Improved performance by & \\  & & Captured hierarchical & leveraging hierarchical & \\  & & relationships in code (e.g., & structures but faced & \\  & & methods and classes). & computational challenges with & \\  & & & large-scale data. & \\  & & Relation-aware & & \\  & & embeddings in & (90.0\%) and precision (91.0\%), & \\  & & transformer-based & highlighting the effectiveness & \\  & & architecture for & of relational embeddings. & \\ \hline \hline \end{tabular}
\end{table}
Table 1: _Cont._4. Train test split: in the current study, the stratified sampling procedure was employed to divide the collected dataset into training and testing sets in a proportion of 4:1, respectively, and the class distribution in both partitions was balanced.  
The dataset and preprocessing code used in this research are available in the GitHub repository [42]. The Relation-Aware BERT (RABERT) transformer model processed this dataset through fine-tuning with relations embeddings which connected software metric dependencies. The model gained improved detection capabilities for large class smells when this enhancement was added to traditional machine-learning models.