# Enhancing Software Quality with AI: A Transformer-Based Approach for Code Smell Detection
## 3 Methodology
### Model Architectures
#### 3.2.5 Relational Embeddings for Code Smell Detection
Lightweight models: exploring efficient transformer architectures, such as DistilBERT or MobileBERT, can reduce computational overhead while maintaining high accuracy. Broader applications: extending these methodologies to related tasks like defect prediction, effort estimation, and code refactoring recommendations can broaden their impact. ## 7 Conclusions

This study presents Relation-Aware BERT (RABERT), a novel transformer-based model for code smell detection that leverages relational embeddings to capture dependencies in software metrics. The results demonstrate that RABERT outperforms classical ML models, achieving an accuracy of 90% and a precision of 91%, making it a strong candidate for integration into software quality assurance tools. However, challenges remain, particularly in handling imbalanced datasets, where RABERT shows lower recall (53.0%). Addressing this trade-off through data augmentation, cost-sensitive learning, or hybrid approaches is a promising direction for future research. Moreover, extending this framework to detect multiple types of code smells across different programming languages could enhance its applicability in diverse software projects. Consequently, the paper's results are of considerable importance for both academic and industry contexts. Researchers stand to benefit from this work by gaining insights into how they may apply contextual and relational embeddings to transformer models to aid in software quality assurance problems. To the practitioners, these models offer reliable information on the state of the code to enhance the recognition of structural defects and ways of responding to them. At this point, areas for future work include developing better models for domain-specific pretraining to improve semantic comprehension, improving optimization techniques to minimize computational costs, and generalizing these models to other software engineering problems, such as smell prediction and refactoring suggestions. Furthermore, techniques such as explainable AI could be adopted to enhance the interpretability of transformer models and thus enhance their usage in industrial applications. This work contributes to the growing field of AI-driven software engineering by demonstrating how relational embeddings improve contextual understanding in software analysis. Future work should explore explainable AI (XAI) methods to make transformer-based models more interpretable for developers and industry practitioners. In summary, this study makes two pivotal contributions to the field of software engineering. First, by introducing Relation-Aware BERT (RABERT), we demonstrate that integrating relational embedding into transformer architectures can significantly enhance code smell detection, particularly for complex patterns such as the large class smell. Second, our comprehensive analysis comparing classical machine-learning models with transformer-based approaches highlights the potential of advanced deep-learning methods to overcome limitations posed by traditional feature engineering. These findings not only drive forward the development of more accurate and robust automated quality assurance tools but also lay the groundwork for future research aimed at addressing challenges such as recall imbalance and computational overhead. The field is poised to benefit from these innovations, paving the way for more effective software maintenance and improved reliability in large-scale systems. [PERSON] contributed to the conceptualization and development of the TabNet hybrid transformer architecture, including the integration of TabNet and transformer modules for code smell detection. He also designed the experiments, analyzed the results, and drafted the manuscript. [PERSON] provided guidance on the methodology, ensured the technical rigor of the proposed architecture, and contributed to the interpretation of the experimental results. He also reviewed and revised the manuscript critically for important intellectual content.