## 2 Related Work  
The primary objective of research in the field of automated program repair (APR) is to automatically generate efficient patches for bugged code in order to significantly reduce the time and expense involved in manual debugging. The three main categories of APRtechniques currently in use are traditional APR and NMT-based APR, in addition to the more recent use of large language models of code (LLMCs) for bug fixing.  
Traditional APR techniques are divided into three main categories: heuristic-based methods [19; 20; 21], constraint-based methods [22; 23; 24; 25; 26; 27], and template-based methods [28; 29; 30; 31]. Among the traditional APR methods, optimal performance can be achieved through the use of template-based APR methods, in which each template is predefined based on the experience of the researcher and is designed to fix a specific type of bug. For example, Tbar [31] systematically summarizes commonly used repair templates in the literature and applies them to fix bugs in programs. Although template-based APR demonstrates significant capability in program repair, it cannot fix bugs that fall outside the operational scope of the repair templates.  
To address the limitations of template-based APR approaches, researchers have leveraged neural machine translation (NMT) techniques to develop APR methods [3; 4; 17]. These methods treat APR as a translation task, that is, translating code with bugs into fixed code. For example, NMT-based APR methods such as CoCoNut [3], SequenceR [4], and RewardRepair [17] employ supervised training of bug-fix code pairs (BFPs) and utilize an encoder-decoder architecture. In this architecture, the encoder is responsible for encoding the bug code and its context into intermediate vectors, whereas the decoder is responsible for decoding these intermediate vectors into a repair patch. However, they remain constrained by the dependency on bug-fix code pairs as training data. This indicates that NMT-based APR methods face challenges in repairing bug patterns not present in their training datasets.  
Compared to NMT-based APR techniques, large language models of code achieve state-of-the-art bug-fixing performance. Large language models of code, which comprise billions of parameters trained on open-source code repositories, demonstrate exceptional proficiency in understanding programming languages [9; 10]. These models can subsequently be transferred to downstream APR tasks through fine-tuning with limited training data. Some researchers have begun to explore the use of LLMCs for program repair [7; 8; 13], and these models outperform both traditional and NMT-based techniques. For example, [PERSON] et al. [13] made the first attempt to address the problem of single-line bug fixing using a fine-tuned CodeBERT. However, the fine-tuning process for applying LLMCs is straightforward and simplistic. Existing NMT-based APR techniques incorporate specific designs tailored for APR tasks: Recoder [12] leverages programming language knowledge, while CoCoNut [3] utilizes contextual information surrounding buggy statements. In contrast, current LLMCs lack such specialized architectural designs. CURE [16] enhances the repair capability of LLMCs by integrating them with NMT-based APR techniques.  
In this paper, a new approach is proposed that utilizes the CodeBERT model to learn the buggy code context representation and then incorporates it into the NMT architecture to enhance repair capabilities.