# Enhancing Software Quality with AI: A Transformer-Based Approach for Code Smell Detection
## 3 Methodology
### Model Architectures
#### 3.2.5 Relational Embeddings for Code Smell Detection
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Experiment ID** & **Model Variant** & **Removed Component/Change** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline Exp-1 & Full RABERT & (All Components Active) & 90.0 & 91.0 & 53.0 & 67.1 \\ Exp-2 & RABERT-RelEmb & Removed Relational Embeddings & 88.3 & 87.5 & 51.0 & 64.5 \\ Exp-3 & RABERT-FE & Removed Feature Embeddings & 87.8 & 86.5 & 50.0 & 63.5 \\ Exp-4 & RABERT-Norm & Removed Feature Normalization & 87.0 & 85.0 & 48.5 & 62.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Ablation test for RABERT. ### Cross-Language Validation

To assess the generalizability of the proposed RABERT model across programming languages, we performed a cross-language validation experiment. The model was originally trained on Python-based code samples. For validation, we tested the model on a separate Java dataset containing similar code smells (large class smell) without any retraining. As shown in Table 8, the model achieved an accuracy of 90.0% on the original Python dataset. However, when evaluated on the Java dataset, the accuracy dropped to 89.3%, and the F1 score was reduced to 66.5%. This performance degradation is expected due to the inherent structural and syntactic differences between Python and Java. Despite this reduction, the model maintained acceptable accuracy and precision, demonstrating its potential applicability across programming languages with possible fine-tuning or domain adaptation techniques. ### Feature Selection Using Mutual Information and Embedded Methods (Lasso)

To further evaluate the impact of feature selection techniques on model performance, we conducted additional experiments using mutual information and lasso regularization. The objective was to explore alternative methods to the previously used Pearson correlation approach for identifying the most relevant features contributing to code smell detection. Mutual information measures the mutual dependence between features and the target variable, capturing both linear and non-linear relationships. Lasso regularization (least absolute shrinkage and selection operator), on the other hand, is an embedded method that penalizes the absolute size of feature coefficients, effectively eliminating less relevant features during model training. Table 9 presents the comparative analysis of the different feature selection methods applied to the RABERT model. ### Explainability Analysis Using SHAP and LIME

To enhance the interpretability of our proposed RABERT model, we conducted an explainability analysis using SHAP (SHapley Additive exPlanations) and LIME (local interpretable model-agnostic explanations). These methods provide insights into the contribution of individual features in the decision-making process of the model, particularly for predicting the large class code smell. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Exp-ID** & **Language** & **Observation** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline Exp-Python & Python (Original) & Best performance as model trained & 90.0 & 91.0 & 53.0 & 67.1 \\  & Java & Performance drop observed due to & 89.3 & 88.5 & 52.0 & 66.5 \\  & (Cross-Language) & syntax and structural differences & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Cross-language validation report. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Feature Selection** & **Selected Features** & **Observation** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline Pearson Correlation & 15 & Baseline feature selection using & 90.0 & 91.0 & 53.0 & 67.1 \\  & & Better recall and balanced feature & & & & \\ Mutual Information & 17 & importance based on & 89.8 & 90.7 & 54.5 & 68.0 \\  & & Information gain & & & & \\ Lasso Regularization & 14 & & Lasso eliminated more features, & & & \\  & & leading to a slight drop in accuracy & 89.2 & 90.0 & 55.2 & 68.5 \\  & & but improved recall & & & & \\ \hline \hline \end{tabular}
\end{table}