# Enhancing Software Quality with AI: A Transformer-Based Approach for Code Smell Detection
## 2 Literature Review
### Recent Transformer-Based Models for Code Analysis  
Recent advances in transformer-based models have led to the development of specialized architectures for code representation. For instance, CodeBERT [3] is pre-trained in both natural and programming language corpora using masked language modeling, which enables it to capture the subtle semantic nuances of code. Building on this, GraphCodeBERT [4] augments the CodeBERT framework by integrating data flow information into its representations, thereby enhancing its ability to model the structural and relational aspects of code.  
While these models have demonstrated impressive performance in tasks such as code summarization, code search, and defect detection, they typically rely on standard self-attention mechanisms without incorporating explicit modeling of inter-feature relationships. In contrast, our proposed Relation-Aware BERT (RABERT) introduces relational embeddings directly into the transformer's self-attention mechanism to capture dependencies among software metrics. This architectural innovation is designed to improve the precision of code smell detection by focusing on the intricate relationships between metrics, such as the interplay between lines of code and Halstead volume.  
Thus, although CodeBERT and GraphCodeBERT offer valuable insights into code representation and structure, RABERT's explicit relational modeling provides a complementary approach that is particularly beneficial for detecting complex code smells like the large class smell.  
Recent research in transformer-based models has significantly enhanced software engineering tasks, particularly defect prediction, code analysis, and software quality assurance. Models like those presented by [PERSON] [24] have effectively utilized structural embeddings to evaluate code quality. Optimizing code embeddings for defect prediction has also gained traction, as demonstrated by [PERSON] [25], who fine-tuned transformer architectures to identify software defects more accurately. [PERSON] [26] leveraged pre-trained code embeddings specifically targeting defect detection tasks, demonstrating substantial improvements in accuracy compared to classical methods.  
Additionally, [PERSON] [27] explored the intersection of machine learning with traditional software quality concerns, emphasizing transformers' ability to handle semantic intricacies in code smells. [PERSON] and [PERSON] [28] presented hierarchical deep learning strategies for code modeling, integrating transformer architectures to manage complex software hierarchies effectively. [PERSON] [29] similarly illustrated transformers' applicability to software bug prediction tasks, significantly outperforming previous non-transformer models.  
Hybrid approaches combining transformers with traditional machine learning techniques, as described by [PERSON] [30], have also shown promise, balancing computational efficiency and accuracy. Furthermore, [PERSON] [31] employed graph-based transformer models to analyze code metrics effectively, highlighting the utility of graph neural networks in conjunction with transformer architectures. [PERSON] [32] emphasized the benefits of relation-aware transformer architectures in defect prediction, closely aligning with the relational embedding concept used in this study.  
Moreover, recent systematic reviews by [PERSON] [33] summarized the broad applicability and ongoing innovations of transformer-based techniques in various software engineering domains, reinforcing the transformative impact these methods have had across diverse software analysis tasks.