# Enhancing Software Quality with AI: A Transformer-Based Approach for Code Smell Detection
## 3 Methodology
### Model Architectures
#### 3.2.5 Relational Embeddings for Code Smell Detection
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline Logistic Regression & 84.5\% & 73.0\% & 58.5\% & 64.9\% \\ Support Vector Machine & 85.2\% & 74.8\% & 60.8\% & 67.1\% \\ Decision Trees & 89.5\% & 79.5\% & 63.8\% & 70.7\% \\ Gradient Boosting & 89.5\% & 80.0\% & 65.0\% & 71.9\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance of classical machine-learning models. ### Per-Class Performance Evaluation

To gain deeper insight into the model's behavior, we conducted a per-class performance analysis for RABERT. Table 5 below shows the precision, recall, and F1 scores for both classes (\"Not LargeClass\" and \"LargeClass\") based on the classification report. As Table 5 shows, while the model achieves high precision for both classes, the recall for the \"LargeClass\" is considerably lower at 0.53. This imbalance indicates that nearly half of the actual large class instances are not detected by the model. In addition, the macro average recall of 0.76 confirms that the performance disparity between classes significantly affects the overall evaluation. ### Statistical Validation

To ensure the reliability of results, statistical significance tests were conducted using paired \(t\)-tests. The differences in performance between gradient boosting and transformer-based models were statistically significant (\(p\) < 0.05), confirming the superiority of transformer-based approaches. ### Comparative Analysis

Figure 2 visually compares the F1 scores of all models. Transformer-based models, especially RABERT, outperformed classical approaches in terms of accuracy and precision. However, the precision-recall trade-off observed in RABERT highlights the need for further optimization to improve recall without compromising precision. Figure 3 provides a side-by-side comparison of four classical machine-learning models (logistic regression, SVM, decision tree, and gradient boosting) and three transformer-based architectures (Feature-Aware BERT, Hierarchical BERT, and RABERT) in terms of accuracy, precision, recall, and F1 score. Each group of bars corresponds to a single model, with the individual bars indicating performance on these four metrics. Notably, RABERT attains the highest accuracy (90%) and precision (91%), illustrating its strong ability to minimize false positives. However, it experiences a lower recall (53%), suggesting that many instances of the \"large class\" smell may go undetected. Hierarchical BERT offers a more balanced performance between precision and recall, whereas classical models like gradient boosting \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Class** & **Precision (\%)** & **Recall (\%)** & **F1 Score (\%)** \\ \hline Not LargeClass & 0.89 & 0.99 & 0.94 \\ LargeClass & 0.91 & 0.53 & 0.67 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Per-Class Metrics for RABERT. Figure 2: F1 scores of all models. remain competitive (89.5% accuracy) with shorter training and inference times. Overall, the figure underscores the trade-off between precision and recall across different methods and highlights the promise of transformer-based models for advanced code smell detection. ### Precision Recall Curves

Each of the following figures provides a precision-recall (PR) curve for different transformer models, showing their ability to handle positive and negative predictions. Figure 4 shows the FABERT precision-recall curve. It shows that FABERT maintains a relatively balanced precision-recall trade-off. It also indicates that precision remains high, but recall is moderate, suggesting FABERT is better at avoiding false positives but may miss some true positive cases. Figure 5 shows the Hierarchical BERT precision-recall curve. It demonstrates that Hierarchical BERT achieves a better balance between precision and recall than FABERT and RABERT. It suggests that hierarchical structure embeddings help in capturing relations between different software components (e.g., methods, classes).