Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks
Authors: Hongyuan Tao
Ying Zhang
Zhenhao Tang
Hongen Peng
Xukun Zhu
Bingchang Liu
Yingguang Yang
Ziyin Zhang
Zhaogui Xu
Haipeng Zhang
Linchao Zhu
Rui Wang
Hang Yu
Jianguo Li
Peng Di
Venue: NeurIPS 2025 poster ()
Keywords: Code LLM
Repository-Level
Agentless
Open-source
Multimodal
Graph
Primary area: deep_learning
TL;DR: Code Graph Models (CGMs) innovatively integrate both semantic and structural information from code repositories into LLMs, enabling effective repository-level coding tasks without relying on agents or closed-source models.

Abstract: Recent advances in Large Language Models (LLMs) have shown promise in function-level code generation, yet repository-level software engineering tasks remain challenging. Current solutions predominantly rely on proprietary LLM agents, which introduce unpredictability and limit accessibility, raising concerns about data privacy and model customization. This paper investigates whether open-source LLMs can effectively address repository-level tasks without requiring agent-based approaches. We demonstrate this is possible by enabling LLMs to comprehend functions and files within codebases through their semantic information and structural dependencies. To this end, we introduce Code Graph Models (CGMs), which integrate repository code graph structures into the LLM's attention mechanism and map node attributes to the LLM's input space using a specialized adapter. When combined with an agentless graph RAG framework, our approach achieves a 43.00% resolution rate on the SWE-bench Lite benchmark using the open-source Qwen2.5-72B model. This performance ranks first among open weight models, second among methods with open-source systems, and eighth overall, surpassing the previous best open-source model-based method by 12.33%.

Code Graph Model (CGM):
A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks
Hongyuan Tao Ying Zhang 12+ Zhenhao Tang
Hongen Peng'
Xukun Zhu Bingchang Liu
Yingguang Yang'
Ziyin Zhang 14 Zhaogui Xu'
Haipeng Zhang
Linehao Zhu
Rui Wang'
Hang Yu Jianguo Li Peng Di
1 Ant Group, 2ShanghaiTech University,
*Zhejiang University, 4Shanghai Jiaotong University
[lyu.hugo.liggzzrdddpnn..... @antgroup.com
Abstract
Recent advances in Large Language Models (LLMs) have shown promise in function-level code generation, yet repository-level software engineering tasks main challenging. Current solutions predominantly rely on proprietary LLM agents, which introduce unpredictability and limit accessibility, raising concerns about data privacy and model customization. This paper investigates whether open-source LLMs can effectively address repository-Ievel tasks without requiring agent- based approaches. We demonstrate this is possible by enabling LLMs to comprehend functions and files within codebases through their semantic information and struc- tural dependencies. To this end, we introduce Code Graph Models (CGMs), which integrate repository code graph structures into the LLM's attention mechanism and map node attributes to the LLM'' input space using specialized adapter. When combined with an agentless graph RAG framework, our approach achieves a 43.00% resolution rate the SWE-bend' Lite benchmark using the open-source Qwen2.5-72B model. This performance ranks first among open weight models, second among methods with open-source systems, and eighth overall, surpassing the previous best open-source model-based method by 12.33%
Introduction
The dream of automating software engineering (SE) has long captivated both the SE and artifi- cial intelligence (Al) communities [ Recent advancements in Large Language Models (LLMs) have shown promising results, particularly in code generation the function level, with models achieving resolution rates above 90% on benchmarks such as HumanEval 4 Unfortu- nately, real-world SE tasks extend far beyond isolated functions or self-contained code files. This exemplified by repository-level issue resolution [5] 66, which encompasses not only software maintenance-addrer
bugs and technical debt-buu also software evolution, which involves introducing new features and enhancements
"qqual contribution.
Corresponding authors.
'The code is available http:p/p///.httaa.ooooooooeeeeea
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

The complexity of repository-level coding tasks has led researchers and practitioners to assume that sophisticated strategies are necessary for their completion [8. Indeed, current leading approaches typically utilize L.M agents powered by proprietary models like GPT-4/4o [ and Claude 3.5 Sonnet [10]. These agents are designed to leverage tools, execute commands, observe environmental feedback, and plan subsequent actions [1] Nevertheless, these methods suffer from two problems. First, the agent-driven mechanism introduces unpredictability in decision-making [2]. As the reasoning processes become intricate in tackling complex problems, the accumulation of errors can hinder the generation of optimal solutions [2]. Second, the reliance on closed-source models creates substantial barriers for the broader SE community [13] including limited accessibility, inability to enhance or customize models for specific tasks, and serious security concerns regarding the privacy of sensitive code repositories when interacting with external API services.
The above two challenges lead to a bold question: Can open-source LLMs be employed in an agentless manner to complete repository-level coding tasks? At first glance, this seems improbable. flosed-source agent-based approaches can resolve up to 55% of issues on the popular SWE-bench Lite benchmark for issue fixing, whereas existing methods using open-source models have only achieved a maximum resolution rate of 30.67% as of May 2025 [15)
Despite these initial reservations, we posit that the answer is "Yes", and the key lies in empowering the open-source LLMs to fully comprehend code repositories, not just the information within individual functions and files, but also the dependencies across functions and files. To move forward to this goal. we propose Code Graph Models (CGMs), to jointly model the semantic and structural information of code repositories. Specifically, we first construct code graph for each repository, which characterizes the hierarchical and reference dependencies between code entities. We then develop a method to integrate this graph into the LLM through two key mechanisms. (i) Semantic Integration: Node attributes (containing code or comments) are first encoded by a pretrained text encoder and then mapped to the LLM's input space via an adapter, enabling the model to understand the semantic information of all nodes. (ii) Structural Integration: The graph structure is incorporated into the LLM through the attention mask, allowing direct message passing only between neighboring nodes in each layer of the LLM, similar to spatial Graph Neural Networks the text encoder, adapter, and LLM decoder-is is then
(GNNs) [16). The entire prrstem-compree fine-tuned using Low Rank Adaptation (LoRA) [17]. The resulting CGM can tackle repository-level coding tasks by using both the code graph and user instructions (text format). To further augment the abilities of the CGM, we develop specially designed Graph Retrieval-Augmented Generation (RA framework, consisting of four modules: Rewriter, Retriever, Reranker, and Reader (i.e., CGM). The first three modules focus the CGM on the subgraph that is most pertinent to the user's query or issue.
Our approach has demonstrated remarkable results on the SWE-bench Lite benchmark, reaching a 43.00 % resolution rate using the open-source Qwen2.5-72B model and our agentless RAG framework. As of May 2025, this performance ranks first among methods utilizing open-source models, second among methods with open-source code implementations (the underlying model may still be closed-soures sourco) and eighth overall. Notably, our approach surpasses the previous best method based on open-source models (Moatless+ DeepSeee- V3 (155) by 12.33%. 3,, despite that method employing DeepSeek-V3. V3, which shows stronger performance than Qwen2.5-72B.
The main contributions of this work are as follows:
We propose CGMs, a novel architecture that seamlessly integrates repository code graphs with open-souree LLMs through semantic and structural integration.
We develop an agentless Graph RAG framework that enhances the CGM's performance by focusing on the most relevant subgraphs for user queries.
Our CGM, armed with the Graph RAG, achieves 43.00% resolution rate on SWE-bench Lite, surpassing most agent-based approaches. We also demonstrate its effectiveness on other repository- level tasks such as code completion.
2 Related Work
Large Language Models for Code Recent advancements in L.MSs have shown remarkable success in generating code at self-contained function or file levels [3]. This includes powerful closed- source models like GPT-4/4o (9) Gemini-2.0 [18. and Claude 3.5 Sonnet [10], as well as open-
source alternatives such as Llama 3.1 19 Qwen 2.5 [20]. and DeepSeek-VV V3 2T]. Additionally,
httpp::/www... cttp:/wwww..eebbec.com

---

23 24 Code Llama [25], StarCoder 27], DeepSeek-Coder Coder [4] 28,. and Qwen-Coder 29 However, these models struggle with repository-level coding tasks that better reflect practical software development scenarios. Even the most capable closed-source models achieve only modest success rates on the SWE--bnch Lite benchmark [5] for real-world issue fixing, while open-source models lag further behind with a maximum resolution rate of 26% [30]. Although closed-source models show superior performance, their limited accessibility and data privacy concerns hinder widespread adoption in the SE community. Furthermore, their proprictary nature prevents fine-tuning on task-specific data to improve performance, if even such data is available.
sede-specialized spen-s models have also emerged, including CodeFuse 122
LLMs to better handle repository-level tasks, they must develop a comprehensive understanding of both semantic and structural information within codebases. DeepSeek- -ooder [14] has attempted to address this challenge by pre-training models on topologically sorted repository codes. However, this approach faces two major limitations: real-world repositories often contain more code than can fit within the model's maximum context length; and the conversion of repository structure into text format tends to obscure explicit dependencies that exist in the codebase.
For
To overcome these challenges, we propose representing repositories as text-rich graphs and aligning them with LLMs via self-supervised continual pre-training. This approach preserves code repository structure while enabling more effective processing and understanding of complex dependencies.
The integration of graph structures into code language models can be classified into three main approaches BT: (1) attention mask modification, (2) graph-to-text conversion, and (3) positional encoding augmentation. In the first approach, models like GraphCode- BERT 332 and StructCoder 333 modify attention masks capture relationships between code tokens in Abstract Syntax Trees (ASTs) and Data Flow Graphs (DFGs). The second approach, demonstrated by TreeBERT [34] and UniXcoder [35), transforms ASTs or node paths into textual sequences that can be processed by language models. The third approach, exemplified by TPTrans [36], leverages relative positional encodings represent structural relationships within ASTs.
Graphs in Code Language Models
While these approaches have shown promise, they primarily focus on Transformer encoders and small-scale language models (such BERT CodeT5) and are limited to file- or function-level tasks. In contrast, our work enhances decoder-only LLMs to handle repository-level tasks. We construct text-rich code graphs for entire codebases, moving beyond simple ASTs DFGs. Inspired by GraphCodeBERTT and StructCoder, we incorporate graph structures through attention masks in LLMs. However, due the text-rich nature of the graphs, each node's 's text semantic information is processed by a pretrained text encoder and then projected onto the LLM's input space via an adapter.
Agent-drive Methods for Software Engineering LLM-based agents like Devin (37 have shown the potential to solve real-world SE problems through their reasoning 38 39
and interactive capabilities 40 41 42 I1] Along this direction, researchers have worked to enhance LLM agents through various approaches, including specialized agent-computer interfaces (ACI) 43
45).
fine-grained search [46, 12 III, and expanded action spaces [47)
However, these agent-based approaches face several drawbacks. First, they typically delegate decision- making to the agents, allowing them to determine both the timing and nature of actions. While agents base their decisions on previous actions and environmental feedback, the expansive action space and complex feedback mechanisms can lead to repetitive behaviors or accumulating errors, ultimately resulting in suboptimal solutions [12]. Second, resolving a single issue often requires 30-40 interaction turns, making the process time-consuming and complicating the identification of specific turns that resulted in unsatisfactory outcomes (2) Third, the inherent unpredictability of agent behavior and reliance closed models creates obstacles for leveraging data to improve performance, despite the abundance of such data in practice, such as issuc-patch pairs for issue fixing [5). While SWE-Gym [48] attempts to address trainability, it may introduce bias by only training with the trajectories that lead the SWE agent to correct answers. As a remedy, we propose the CGM, built open-source LLMs and enhanced through an agentless Graph RAG framework.
Agentless Methods for Software Engineering Agentless models offer more controlled approach to simulating real-world SE processes by following well-defined, fixed steps rather than relying on LLM agents to make autonomous decisions or use complex tools. They help avoid the issues of unpredictability and lengthy interaction chains. These methods typically operate in two main stages: localization and editing [4).. The localization stage identifies relevant code snippets within repository, while the editing stage generates or modifies code based on these identified sections.
3

---

contains
imports
extends
RLC
REFO
GASS
calls
contains
calls
T-PLE
Figure 1: An example of our repository-Ievel code graph. "PKG", "FUNC", and "T-FILE" represent "PACKAGE", "FUNCTION", and "TEXTFILE", respectively. Solid and dashed lines indicate hierarchical (contains) and reference dependencies (calls/ipporsseextennd))) respectively.
This framework is particularly effective for repository-level code completion tasks, especially when combined with RAG [50 51 For more complex tasks like issue fixing, enhanced approaches with additional steps exist 22 I. For instance, Agentless (2] implements a comprchensive ten-step pipeline, dedicating four steps improving localization accuracy. This method has achieved promising resolution rate of 40.67% on SWE-bench Lite, comparable to state-of-the-art (SOTA) agent-based based methods, though relies on the closed-source model Claude-3.5 Sonnet.
Recent research has also focused on enhancing code understanding by incorporating structural information through graph-enhanced repository modeling [52] 49]. However, even when graph structures are used during retrieval, existing methods typically flatten the retrieved code snippets into linear text sequences for downstream model prompting. This flattening process fails preserve the inherent heterogeneity between graph and text modalities. As remedy, we propose the CGM that explicitly aligns these distinct modalities, enabling better preservation and utilization of structural information throughout the entire process.
3 Code Graph Construction
Before delving into the CGM, it is crucial to understand the repository-level code graph that CGM utilizes and the process of its construction. The primary aim this code graph to offer structured representation of the structural and semantic information inherent in complex codebases.
We represent each repository as directed graph G (V.E). E). where V is the set of distinct entities in the codebase and E is the set of edges between these entities. To be specific, the code graph includes up seven types of nodes and five types of edges (details are provided in Appendix[B B The node types vary in granularity, ranging from the repository level (REPO) to fine-grained attributes. The edge types comprise both hierarchical (i.e., contains) and reference dependencies (callsiimports/extends)),
As shown in Figure T the hierarchical dependencies (i.e., the solid edges) span the code graph. In other words, all nodes interconnected by edges reflecting hierarchical dependencies, establishing a top-down tree structure. This structure mirrors the organization of code entities as dictated by file systems and programming language syntax rules. Building this tree graph begins with AST parsing [52.. During this phase, code entities and their hierarchical dependencies are identified in recursive manner: the root node (i.e., REPO) is added to the graph first, followed by its children (i.e., PKG and T-FILE), until all nodes without descendants (i.e., FUNC) are processed. With each recursion, directed edges added from parents to children.
On the other hand, reference dependencies (i.e., the dashed edges) capture interactions between different entities, such as class inheritance, function calls, and module imports. Unlike hierarchical edges, which maintain a vertical hierarchy, reference edges create horizontal connections that may introduce cycles, such those caused by recursive calls. These edges typically not part of an AST. To derive them, we conduct a lightweight semantic analysis to resolve symbols, such as references or calls to classes and attributes. Once target symbol is identified, an edge is added from the source node to the target node in the code graph.
Concerning node attributes, we retain the original content and line range of each node. This approach enables explicit graph traversal and retrieval and facilitates training models with enhanced semantic understanding capabilities. During post-processing, we remove the text contained in the child nodes from the parent nodes within the tree graph derived from the hierarchical dependencies. The resulting code graph is text-rich graph (54) in which each node encapsulates corresponding code snippet.

---

c ISSUE; Modeling's 'ygurnhiityyaaae maric does not compate reparalilty cornecty for sesaed Conpooudd Models Causidee the felliiing model pyrboe from carogiannlelngg ieport moddef.....
(b) Retrievor
Graph Model (CGM)
Estrader Nodea
Adjanany Matris
Alsurian Musk
Graphanaare
Test Tolen
Insioner Noaeen
Nado Fosked
Strusaani Intepreton
Nade Tokee
Trainabl
(a) Rowrttor
Extructer Rrddo
Sereentic Vntegratee
Code Exily & Keyosvis
Edloer Text
Tect
Adagii
Kg woode sporeiilyy sunts
Adggrssh
(c) Reranker
Tosb
Salcted Ailes
Gäven the cantenttc (nods token] [quers]
Extradter
follnweing files way
Fanctionr aeclhal Ausaling...
Stage
Stage
ansdl/ieet: Jedected. fiiss
Answer
Name Rank
Pio Skatetion Rack
Prompt for "ses"
Given issue, (a) Rewriter extracts code entities and keywords from the issue (Extractor), and modifies the original issue into more detailed queries (Inferer). Based on this, (b) Retriever locates relevant nodes from the corresponding code graph; then expands these nodes to a connected subgraph by including neighboring and upstream nodes. (c) Reranker ranks retrieved results in two stages: File Name Rank and File Skeleton Rank, selecting the most relevant files for modification. Finally, (d) Reader (CGM) takes the retrieved graph and selected files input. Each node's 's code content is encoded by an Encoder E, producing node token via the Adapter A. Node tokens are then concatenated with text tokens in the prompt before entering the LLM decoder D, where the adjacency matrix replaces its original attention mask.
Figure 2: Architecture of CGM and its Graph RAG extension:
4 Code Graph Models (CGMs)
In this section, we elaborate on the architecture of the Code Graph Model (CGM), the training strategy we adopted, and how we enhance the CGM via the Graph RAG framework.
4.1
Model Architecture
The architecture of the CGM illustrated in Figure d). CGM takes the code graph as inputs, enhancing the LLM's comprehension of both semantic and structural information within the graph. Below, we detail how CGM integrates both aspects into the LLM.
Semantic Integration:
The code graphs are text-rich, with semantic information only residing in the textual contents of the nodes. As shown in Figure we integrate the node information into the LLM decoder 'D by transforming node text into node tokens through an encoder & and an adapter A.
Specifically for the encoder, utilize the pretrained encoder from CodeT5+ [53], chosen for its proven effectiveness in processing both source code and text (comments and documentation). For nodes containing lengthy text, we segment the content into chunks of 512 tokens each. These chunks are processed independently by the encoder. To maintain graph consistency, we duplicate the source node for each chunk, preserving identical connections other nodes. The chunks within node are fully connected, and their sequential order is maintained through position embeddings in the LLM decoder D. We fine-tune the encoder using Low-Rank Adaptation (LoRA) [1 optimize its performance for downstream tasks.
The adapter A serves as bridge between the encoder and LLM, projecting encoder outputs into the LLM's input embedding space. Following successful practices in Vision Language Models (VLMs) [56 57), we implement the adapter two-layer MLP with GELU activation [58]. The adapter is trained from scratch with random initialization.
Unlike VL.Ms, which bridge different modalities, CGM's encoder & and decoder DD are of the same modality, simplifying the alignment process. Furthermore, we compress each 512-token chunk (shown as gray tokens in Figure d)) into single node token (black tokens in Figure 2 d)) for the LLM decoder. This compression effectively extends the LLM's context length by a factor of 512, enabling the processing of extensive code repository contexts. Similar techniques, referred to as soft prompt compression, have been shown to enhance long-context modeling in recent studies [59 60 61].
Besides node information, another challenge is integrating the graph struc- ture into the LLM decoder D. While LLMs excel at processing sequential data, they are not inherently designed to capture graph structures 54 Traditional approaches have attempted to incorporate repository-level structural information by simply linearizing code snippets into sequences [14 (4).
Structural Integration:
but this transformation often fails to preserve the explicit relationships between code entities.
5

---

To better maintain structural relationships, we introduce graph-aware attention mask to replace the causal attention mask solely between node tokens in the LLM. This mask is derived from the code graph's ss adjacency matrix, taking into account the node duplication process described earlier. We then fine-tune the I.M with LoRA to adapt to both the new attention mechanism and the node tokens from the adapter A. This approach ensures that attention occurs only between neighboring nodes in the code graph, mimicking the message passing mechanism frequently used in spatial GNNs [62 63
4.2
Training Strategies
Given the pretrained encoder & and decoder D, the training of the CGM consists of two main phases:
Subgraph Reconstruction Pre-training: This phase focuses on training the CGM effectively capture both the semantic and structural aspects of code graphs. To achieve this, we introduce a novel pre-training task that requires the model to reconstruct code content from its corresponding code graph, a process we refer to as Graph-to-Code.
In this task, the inputs are subgraphs randomly sampled from large-scale code graphs, with a limited number of nodes. This constraint ensures that the corresponding output code remains below 8,000 tokens, allowing for computational efficiency and manageable context sizes during training. To enhance the meaningfulness of the output code, we employ a hierarchical approach that preserves the inherent dependencies within the code graphs as they are translated into text. Concretely, for higher-level nodes (e.g., REPO and PACKAGE), we position them at the beginning of the output sequence or their respective files to maintain hierarchical consistency. We then utilize the approach from DeepSeek- Coder [14] to perform topological sorting on all file nodes, thereby establishing structured order for the code content. Lastly, intra-file nodes (e.g., CLASS and FUNCTION) are sorted by line numbers and concatenated within their respective files, culminating in coherent text sequence that accurately represents the sampled subgraph.
This phase fine-tunes CGM on real-world issue-patch pairs [ adapting it to practical software debugging and code editing tasks. As displayed in Figure 2 d), the model Iearns to generate code patches based on two inputs: (i) a subgraph and (ii) text prompt that indicates the "oracle" flls-files that require modification according to the ground-truth patch. The subgraph is constructed by combining the oracle files, their downstream nodes, and one-hop neighbors from the repository-level code graph. To improve model robustness, we intentionally introduce noise into the prompts: 10% include an irrelevant file that doesn't 't require modification, while another 10% omit at least one oracle file. This controlled noise exposure helps the model better generalize to real-world scenarios where inputs may be incomplete or contain irrelevant information.
Noisy
4.3
The Graph RAG Framework
This section presents our Graph RAG framework, a streamlined extension of CGM designed for automated resolution of real-world repository tasks. The framework consists of four core modules: Rewriter, Retriever, Reranker, and Reader (the proposed CGM). This compact architecture stands in contrast to the SOTA agentless method, which requires ten distinct steps (2).
As illustrated in Figure 2, the framework operates sequentially. First, Rewriter enhances the original issue description to help Retriever identify relevant nodes in the code graph. Retriever then constructs a connected subgraph using both lexical and semantic search techniques. This subgraph serves as input for both Reranker and Reader. Reranker analyzes the subgraph identify the Top K files likely to be modified. Finally, Reader (CGM) generates the code patch using both the subgraph from Retriever and the selected files from Reranker. Rewriter and Reranker are implemented by prompting the open-source Q -iinstructt (20). while the semantic search in Retriever utilizes the open-source CGE-Large model (64 In Appendix [ we provide case study on how CGM solve a specific issue from scratch. Meanwhile, we report the computational costs of our framework, including the cost of code graph construction, in Appendix C.4
Rewriter comprises two subcomponents: Extractor and Inferer, as illustrated in Figure a). Ex- tractor identifies key code elements from the user query, including file names, function names, and relevant keywords. Inferer then enriches the query's 's semantics by providing more detailed functional descriptions. The specific prompts for both components are detailed in Appendix G
Retriever generates a connected subgraph from the code graph for subsequent modules. As shown in Figure bb), Extractor nodes (blue nodes) are first identified through string matching with the code elements and keywords extracted earlier. Next, Inferer nodes are located (red nodes) through semantic search, comparing the Inferer's output with each node's 's textual information. These anchor nodes are

---

Table 1: Performance comparison of open source system on SWE-bench Lite and Verified. CS-3.5
denotes 3.5-e-----Soneet 200mett20241022, DS- represents DeepSeek-V3. V3, Q2.5C-32B means Qwen2.5-
Coder-32B and Q2.5-72B stands for Qwen2.5-72B-IInsrucc The icons closed-source models, respectively.
+ and denote open and
(a): SWE-bench Lite
(b): SWE-bench Verified
Method
L.M
Agent
Rt
Rank
Method
LLM
Agent
%
Rank All
DARS Agent
CSS-3.5
Yes 47.00
1
OpenHands
NA
Yes
65.80
1
CGM-SWE-PY
Q2255-72B
Na
43.00
PatchPilot-vl.11
NA
NA
64.60
2
5
Lingxi
42.67
SWE-Agent
CS-3.7
Yes
62.40
3
10
CodeAct-v2.1
CCS33.5
Yes 41.67
Agentless-yL.5
CS-3355
No 50.80
25
PatchKitty-0.9
CSS-355
41.33
CGM-SWE-PY
Q2.5-72B
No 50,40
26
Composio SK
BCS-3.5
Yes 41.00
Composio SK
NA
Yes 48.60
6
31
Agentless-v1.5
CS-3355
No
40.67
Agentless Lite
03-mini
No 42.40
8
39
Moatless
GSS-3.5
Yes 39.00
8
Composio SK
CS-3.5
Yes 40.60
10
47
Patched.
CS-3.5
Yes 37.00
SWE-Agent
Q2.5C-32B
No
40,20
11
48
CGM-Multi
Q2.5-72B
No 36.67
10
Agentless-v1.5
GPT-4o
No
38.80
12
50
AppMap
CS-3.5
Yes 36.00
11
24
SWE-Fixer
Q2.5-72B
Yes 32.80
14
54
Agentless Lite
a3-mini
No
32.33
13
31
Lingma SWE-GPT
QQ25-722B
No 30,20
15
58
Agentless-v1.55
GPT-4o
No 32.00
14
32
Lingma Agent
Q2.5-72B
Yes 28.80
16
60
Moatless
DS-V3
Yes 30.67
16
35
Lingma SWE-GPT
Q2.5-72B
No
25,40
18
64
SWE-Fixer
Q2.5-72B
Yes 24.67
26
51
SWE-Agent
GPT-4o
Yes
23,20
16
67
Lingma SWE-GPT
Q225-72B
No
22.00
28
57
SWE-Agent
GPT-4
Yes
22.40
17
68
then expanded to include their one-hop neighbors, capturing local programming dependencies [65] To ensure connectivity and incorporate upstream information, these expanded nodes connected to the Root node (REPO in Figure [). Finally, each FILE node in the subgraph is expanded to include all its internal nodes, aligning with Reranker's file-level output. The result is repository-enhancee context subgraph representing the user query, asdenoted by the shaded nodes in Figure b
Reranker further refines the subgraph generated by Retriever, selecting only the Top K files deemed most likely to be revised. This refinement is necessary because Retriever's 's output includes files that may only be referenced and not modified. Reranker operates in two steps: first, it selects K 10 files based on the original user query and file names; next, it narrows this selection down to K = files by individually scoring each according to how relevant its file skeleton 2 is to the user query. The specific prompt for Reranker can be found in the Appendix
Reader receives two inputs: the subgraph from Retriever node tokens (black tokens) and the selected files with their full contents as text tokens (gray tokens), as depicted in Figure[[d), These inputs are combined using the prompt template in the white box on the left of the figure. The graph and text tokens complement each other by providing global and local information related to the user query. Using this comprehensive information, Reader (i.c., the CGM) generates the final response.
5 Experiments
In this section, we assess the performance of the CGM on two primary tasks: repository-leve issue resolution and code completion, for both Python and Java programming languages. We also conduct series of ablation studies to validate the effectiveness of the model design and training strategies.
5.1 Repository-Level Issue Fixing
This section evaluates the proposed CGM against other SOTA methods in resolving real-world software issues. We use three benchmark datasets: SWE-bench Lite, containing 300 issues from 11 Python repositories, SWE-bench Verified, containing 500 issues from 12 Python repositories, and SWFE-bench-jvaa Verified, comprising 91 issues from 6 Java repositories. All benchmarks utilize developer-wring written unit tests to verify the correctness of model sadel-geeerated patches, ensuring rigorous evaluation. Performance is measured using the resolution rate (% R), defined the percentage of successfully resolved issue instances. We present results for two variants of our model: CGM-Multi, trained for both issue resolution and code completion tasks across Python and Java repositories, and CGM-SWE-PY, specifically optimized for Python issue resolution. Detailed information regarding the datasets and implementations can be found in Appendix C.5
As shown Table [(a) our CGM-SWE-PY model achieves a 43% resolution rate on SWE-bench Lite, placing it first among methods utilizing open-source models, second among those that implement open-source methods but use closed-source source models, and eighth overall. Notably: (i) When compared to other methods based on open-source models, CGM-SWE-PY outperforms Moatless+DeepSeee
7

---

Table Performance evaluation on SWE-bench-java Verified. DS-V2 denotes DeepSeek-Chaid V2, DSC-V2 V2 represents DeepSeek-Coderwa GPT-4o refers to GPT-4--2024-05-11 for Doubao-Pro- 128k, and GPT-40-MINI indicates GPT-40--MIN--024-007111 The icons t and f denote open-source and closed-source methods or models, respectively.
DB-128K stands
METHOD
LLM
AGENT
% R
RANK
CGM-MULTI
Q2.55-72B
No
14.29
SWE-AGENT
DS-V2
YES
9.89
2
SWE-AGENT
DDSC-V
YES
7.69
SWE-AGENT
GPT-40
YES
6.59
SWE-AGENT
DB-128K
YES
1.10
5
SWE-AGENT
GPT-40-MINI
YES
1.10
Table 3: Performance comparison CrossCodefival and ComplexCodefival benchmarks. DeepSeek- 236B represents -
V2.5-236B, Mixtral-123I denotes Bastral-aagge-lnsttttt--444 and Qwen2.5-72B refers to 1wre2255-22B-nnstuue Baseline models are evaluated using FIM (Fill-in- Middle) and one-hop expansion.
CROSSCODEEVAL
COMPLEXCODEEAAL
МETHOD
JAVA EM
PYTHON ES
JAVA
PYTHON
ES
EM
ES
MIXTRAL. 123B
47.17
82.23
53.92
82.42
37.00
64,81
31.00
62.48
DEEFSEEK-2366
44.74
83.81
58.54
85.03
36.00
63.08
32.00
60.60
QWEN2,5-72B
37.31
78.78
58.50
81.56
26.00
54.14
28.00
$7.12
CGM-MULII-72B
50.21
80.76
61.20
84.30
47.00
78.86
43.00
72.60
V3 by 12.33% [b]. despite DeepSeek- V3's generally superior performance in various coding benchmarks compared to LLM decoder Qwen2.5-72B (21). Furthermore, it exceeds Lingma SWE-GPT by 21, even though the latter employs carefully curated COT (chain-of-thouge boost Owen2.5-72BBs effectiveness in issue resolution. (ii) In relation to other agentless frameworks, by 2.33% % and significantly outperforms GPPT by 11.00% This achievement is particularly noteworthy given that Agentless leverages a complex ten-step pipeline with more powerful closed-source models, while CGM-SWE-PY operates a simpler four-step Graph RAG framework. We attribute this success to CGM's enhanced capacity to interpret both semantic and structural information within repositories. (iii) While the top methods on SWE-bench Lite are entirely closed-source regarding both models and implementations, CGM-SWE-PYS results are within 10% of these systems. This indicates that CGM-SWE-PY has the potential compete with leading agent-based methodologies. Compared to other open-sourced model-based methods, CGM significantly narrows the gap between open-source models and closed-source methods in issue-fixing scenarios. (iv) Our multi-task model, CGM-Multi, achieves resolution rate of 36.67% % on SWE-bench Lite, ranking 23rd overall. The relatively lower performance compared CGM-SWE-PY can be attributed to its broader focus, which encompasses both issue fixing and code completion tasks across Python and Java repositories. (v) We further apply CGM-SWE-PY to a larger Python Benchmark-SSW--beree Verified in Table [(b) where CGM-SWE-PY ranks first again among open weight models, and fifth among methods with open-source system.
data to
CGM-SWE-PY slightly surpasses
In the SWE-bench-jare java evaluation for Java repositories as shown in Table 2 CGM-Multi records resolution rate 14.29%, significantly outclassing SWE-Agent build upon both closed-source and open-source models. These findings further substantiate the effectiveness of our proposed GCM and the specially designed Graph RAG framework.
5.2 Repository-Lerv Code Completion
In this section, we evaluate the CGM's performance on code completion tasks at the repository level for both Python and Java programming languages. Our evaluation uses two benchmarks: ErossCodeE- val and ComplexCodeEval. Concretely, CrossCodeEval focuses on cross-file file code completion, while ComplexCodeEvval encompasses more intricate tasks, including API recommendations and test case generation. Performance is measured using two metrics: Edit Match (EM) and Edit Similarity (ES), evaluating how similar the generated code is to the ground-truth code. Detailed information regarding the datasets, metries, and the implementation of baseline models can be found in Appendix C.6
8

---

Table Comparison of CGM with RAG variants on CrossCodeEval. Results are reported for Java and Python across multiple base models. Evaluation metrics include EM and ES.
CODELLAMA-7B
SEarSEEK-CODER.
METHOD
JAVA
PYTHON
JAVA
PYTHON
EM
ES
EM
EM
ES
EM
ES
NORAG
20.60
54.50
13.70
44.10
24.20
59.30
19.40
52.50
BM25
23.42
66.13
21.76
69.09
22.49
66.78
23.30
70.84
REPOFUSE
24.80
71.05
27.92
73.09
RLCODER
26.23
67.61
26.60
72.27
26.09
67.31
30.28
74.42
R2C2
35.60
58.50
23.60
42.90
41.60
64.60
32.70
54.00
CGM-MOLTI
36.42
75.28
31.03
73.90
41.65
74.76
33.88
71.19
STARCODER-7B
QWEN2.5
METHOD
JAVA EM
PYTHON
JAVA
PYTHON
NORAG
21.60
55.90
17.00
49.50
37.31
78.78
78
33.6
73.19
BM25
22.16
67.80
22.33
69.60
49.37
82.63
43.15
78.66
REPOFUSE
24.20
70.82
RLCODER
24.73
69.08
25.82
72.11
R2C2
38.10
63.60
30.90
51.90
CGM-MOLTI
37.44 73.77 31.00
71.66 51.61 84.62 46.23 82.16
Table 5: Impact of each RAG (Retrieval-Augmentee Generation) component on the performance of CGM for issue fixing, Results are reported as the resolve rate (% R) on SWE-bench Lite, demonstrating the contribution of rewriter, retriever, and reranker modules.
SETTING
%
W/O REWRITER
34.67
W/O RETRIEVER
31.67
w/O RERANKER
18.33
-W/ R3
9.67
W/O CGM READER (FLATGRAPH)
5.33
as its LLM decoder. We compare with similarly sized large language models, including Mistral-Large-Instruet DeepSeek-V2..5 V2.5-236B, and the standalone Dwen2.5-72B innstructt For all models, context retrieval for code completion performed by identifying one-hop neighbors of the target file (that requires completion) in the code graph. While CGM-Mull processes the entire subgraph as input, baseline models only receive the textual content from the nodes. Results show that CGM-Multi performs on par with or exceeds other models on CrossCodeEval. More importantly, it greatly outperforms the baseline models on ComplexCodeEvar demonstrating its superior capability in handling complex tasks through comprehensive subgraph analysis.
3 the results for CGM-Multi, which uses
Next, we evaluate CGM against other RAG methods for CrossCodeEvall The comparison includes several established systems: BM25, the default retrieval method in CrossCodeEval [66;; RLcoder [67], which employs reinforcement learning for retrieval optimization; RepoFuse 668,, which integrates code graphs during retrieval but converts retrieved code snippets into linear text sequences; and R2C2 169), which combines retrieved code snippets with Tree-sitter-generaie the input to the LLM. In our CGM implementation, we still construct input subgraphs by combining target files with their one-hop neighbors. We evaluate these methods using various base models for StarCoder-7B: DeepSeek- PeepSeek-Coder-75 and Qwen2.5-Coder.. BB. This diverse set of comparison methods enables a comprehensive evaluation of CGM's effectiveness in long-context retrieval and understanding. As shown in Table 4 CGM typically outperforms other RAG methods, regardless the base model used, suggesting that graph- based context retrieval is more effective for code completion tasks. Moreover, CGM's superiority over RepoFuse, which also uses code graphs for retrieval, can be attributed to CGM's s explicit integration of structural information within the subgraph. whereas RepoFuse flattens node context into text sequences, obscuring the explicit dependencies among code entities.
abstract context as
generation, including CodeLlama
5.3 Ablation Studies
In this section, we present key findings from our ablation studies, with detailed analysis available in Appendix C.7 Our investigation reveals four crucial insights: (i) Graph RAG: Our assessment of

---

Table 6: Impact of training strategies on CGM's performance. Results are reported for CrossCodeEval (Java and Python) in terms of EM and ES. Here, & denotes the encoder, A denotes the adapter, D denotes the LLM,, and "combined" refers to the full CGM setup (w/ mask, Recon, A + LoRA w/ E + D).
SETTING
CROSSCODEEVAL
JAVA
PYTHON
EM
ES
EM
ES
MODULE
FREEZE
17.91
58.28
11.78
51.60
A
41.70
77.25
34.90
74.54
LORA W/
46.84
82.06
38.76
76.02
-A+ LORA W/
49.51
83.21
43.15
79.84
w/O
48.71
83.40
42.21
80.46
w/O RECON
42.78
80.29
39.77
75.87
COMBINED
51.61
84.62
46.23
82.16
Table 7: Performance of CGM with different LLM backbones. Results are reported as the resolve rate (% R) on SWE--Bnchh Lite, demonstrating the model's ability to generalize across various sizes and architectures.
BACKBONE
SWE-BENCH LITE (% R)
CGM
-WEN2.5-72B--NSTEE
43.00
MAAMA3.1-70B-INSTII
25.33
QWEN2.5-CODER CODDER--- -28--UTT
28.67
OWEN2.5-CODER INSTRUCT
4.00
Table 8: Analysis of the test-time scaling (TTS) performance using the Pass@K metric. Results reported as the resolve rate (% R) on both SWE-Bench Lite and SWE-Bench Verified, demonstrating the benefits of leveraging additional test-time computation.
PASS@K
SWE-BENCH LITE (% R)
SWE-BENCH VERIFIED (%
K=1
43.00
50.40
K=2
44.33
51.40
K=3
46.67
53.20
the Graph RAG modules in Table shows that the presence of Rewriter, Retriever, and Reranker is essential for achieving optimal performance on the SWE-bench Lite benchmark. Notably, Reranker plays a pivotal role it dictates which files should be modified. (ii) Semantic Integration: Joint fine-tuning of all three components in Table eenlode- &, the adapter A. and the decoder D-yields superior performance compared keeping any component fixed. (iii) Structural Integration: The integration of graph structural information through attention masking is essential for optimal performance. (iv) Training Strategies: The subgraph reconstruction task, as described in Section 4.2 significantly contributes to improving the CGM's 's overall performance. (v) Backbone Generalization: Moreover in Table 7 CGM can also be generalized on backbones with different sizes, demonstrating its potential for sssource-constriigg test-time scaling strategy implemented via Pass@ K sampling significantly improves the performance of CGM on both SWE-Bendh benchmarks.
scenarios. (vi) Test-Time Scaling: As detailed in Table 8 the
Conclusion
In this paper, we present CGM, novel graph-enhanced LLM architecture designed for comprehen- sive repository-level code understanding. By seamlessly integrating both semantic and structural information from codebases through a specialized enoodrr-adapter framework and graph-aware atten- tion mechanisms, CGM demonstrates that sophisticated agent-based approaches and closed-source models are not necessarily required for complex SE tasks. When combined with our custom-designed Graph RAG framework, CGM achieves remarkable 43.00% resolution rate in real-world issue- issue-fixing scenarios on SWE-bench Lite, using only open-source models. Our work establishes new direction for developing powerful, transparent, and accessible tools for automated SE.
10

---

7 Acknowledgement
This work was supported by Ant Group Research Intern Program. Prof. Haipeng Zhang was supported by Science and Technology Commission of Shanghai Municipality (252R1401256).
References
[1] Yingwei Ma. Rongyu Cao, Yongchang Cao, Yue Zhang, Jue Chen, Yibo Liu, Yuchen Liu, Binhua Li, Fei Huang, and Yongbin Li. Lingma swe-gpt: An open pwveloopmentppoccsss-cooioi language model for automated software improvement. arXiv preprint arXiv:2411.00622, 2024.
[2] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying Ilm-based software engineering agents. arXiv preprint #rXiv:2407.01489 2024.
[3] Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, and Rui Wang. Unifying the perspectives of nlp and software engineering: A survey on language models for code. arXiv preprint arXiv:2311.07966 2023.
[4] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107 003344, 2021.
[5] Carlos E Jimenez, John Yang. Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024.
[6] Daoguang Zan, Zhirong Huang, Ailun Yu, Shaoxin Lin, Yifan Shi, Wei Liu, Dong Chen, Zongshuai Qi, Hao Yu, Lei Yu, et al. twe-bench-java: A github issue resolving benchmark for java. arXiv preprint arXiv:2408. 14354, 2024.
[7] Yizhou Liu, Pengfei Gao, Xinchen Wang, Jie Liu, Yexuan Shi, Zhao Zhang, and Chao Peng. Marscode agent: Ai-native automated bug fixing, arXiv preprint arXiv:2409.0005: 2024.
[8] John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. arXiv preprint MrXXi:2405.157933 2024.
[9] OpenAl. Hello gpt-4o. http:://pppaaacccommmemee//llllp 2024.
[10] Anthropic. Claude 3.5 sonnet. https:/www.nnhrrppic.com http:///wwwcclaaad------------/- sonnet 2024.
[11] Yingwei Ma, Qingping Yang, Rongyu Cao, Binhua Li, Fei Huang, and Yongbin Li. How to understand whole software repository? arXiv preprint arXiv:2446.0144 01422, 2024.
[12] Antonis Antoniades, Albert Örwall, Kexun Zhang, Yuxi Xie, Anirudh Goyal, and William Wang. Swe-search: Enhancing software agents with monte carlo tree search and iterative refinement. arXiv preprint arXhh:2410.20285 2024.
[13] Jiya Manchanda, Laura Boettcher, Matheus Westphalen, and Jasser Jasser. The open source advantage in large language models (Ilms). arXiv preprint arXiv:2412.12004 2024.
[14] Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Wu, Yukun Li, Huazuo Gao. Shirong Ma, al. Deepseck-coder-vv. Breaking the barrier of closed-source models in code intelligence. arXiv preprint a0X1::22611:: 2024.
[15] Aor. Aoatless-tools, https: http:p//ggghhooooo//////////aaap
2024.
[16] Ines Chami, Sami Abu-El-Haipp Bryan Perozzi, Christopher Ré, and Kevin Murphy, Machine learning on graphs: model and comprehensive taxonomy. Journal of Machine Learning Research, 23(89):1-64, 2022.

---

[17] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen -hen, Yuanzhi Li. Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint uxiv:22106.09685 2021.
[18] Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. Introducing gemini 2.0: our new ai model for the agentic era. https:://ble woogg///eechnooog//gggggggggg -tw.ooooggggggoggeeeeeeeemaollla 2024 2024.
[19] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
[20] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. 2024.
[21] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
[22] Peng Di, Jianguo Li, Hang Yu, Wei Jiang, Wenting Cai, Yang Cao, Chaoyu Chen, Dajun Chen, Hongwei Chen, Liang Chen, et al. Codefuse- A pretrained multi-lingual code large language model. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, pages 418-429. 2024.
[23] Bingchang Liu, Chaoyu Chen, Zi Gong, Cong Liao, Huan Wang, Zhichao Lei, Ming I Liang, Dajun Chen, Min Shen, Hailian Zhou, et al. Mftcoder: Boosting code llms with multitask fine-tuning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5430-5441, 2024.
[24] Zi Gong, Hang Yu, Cong Liao, Bingchang Liu, Chaoyu Chen, and Jianguo Li. Coba: Conver- gence balancer for multitask finetuning of large language models. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pages 8063-8077, 2024.
[25] Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, al. Code Ilama: Open foundation models for code. arXiv preprint arXiv:2308. 12950, 2023.
[26] Raymond Li. Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Mare Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06611 20..3000.666.,. 2023.
[27] Anton Lozhkov, Raymond Li. Loubna Ben Allal, Federico Cassano, Joel Lamy-Poiries Noua- mane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. Starcoder and the stack v2: The next generation. arXiv preprint 2xXv:2242.11977. 2024.
[28] Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepscek-coder: When the large language model meets programming the rise of code intelligence. arXiv preprint arXiv:2401. 2024.
[29] Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jia- jun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. S-coder technical report. arXiv preprint 1iX1:2240..12186, 2024.
[30] Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, and Kai Chen. Swe-fixer: Training open-source Ims for effective and efficient github issue resolution. arXiv preprint arXiv:2501.05040, 2025.
[31] Ziyin Zhang, Hang Yu, Shijie Li, Peng Di, Jianguo Li, and Rui Wang. Galla: Graph aligned large language models for improved source code understanding. arXiv preprint arXiv:2409.00 04 083, 2024.
12

---

[32] Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, al. Grapheodebert: Pre-training code representations with 2020.
data flow. arXiv preprint arXiv:2009.0885
[33] Sindhu Tipimeni, Ming Zhu, and Chandan K Reddy. Structcoder: Structure-aware transformer for code generation. ACM Transactions on Knowledge Discovery from Data, 18(3):1-20, 2024.
[34] Xue Jiang, Zhuoran Zheng, Chen Lyu, Liang Li, and Lei Lyu, Treebert: A tree-based pre-trained model for programming language. In Uncertainty in Artificial Intelligence, pages 54-63. PMLR, 2021.
[35] Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. Unixcoder: Unified cross- cross-modal pre-training for code representation. arXiv preprint arXiv:2203.0388 03855, 2022.
[36] Han Peng, Ge Li, Wenhan Wang, Yunfei Zhao, and Zhi Jin. Integrating tree path in transformer for code representation. Advances in Neural Information Processing Systems, 34:9343-999 9354, 2021.
[37] Cognition. Introducing devin.
2023.
[38] Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, et al. Repoagent: An llm-powered open-source framework for repository-level code documentation generation. arXiv preprint arXiv:2402. 16667, 2024.
[39] Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better I1m agents. arXiv preprint arXiv:2402.01030, 2024.
[40] Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308. 00352, 2023.
[41] Jiaolong Kong, Mingfei Cheng, Xiaofei Xie, Shangqing Liu, Xiaoning Du, and Qi Guo. Contrastrepair: Enhancing conversation-base automated program repair via contrastive test case pairs. arXiv preprini arXiv:2403.01977 2024.
[42] Yifan Xie, Zhouyang Jia, Shanshan Li, Ying Wang, Jun Ma, Xiaoling Li, Haoran Liu, Ying Fu, and Xiangke Liao. How pet two-headed snake? solving cross-repository compatibility issues with hera. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, pages 694-705, 2024.
[43] Zhipeng Xue, Zhipeng Gao, Xing Hu, and Shanping Li. Acwrecommender: A tool for validating actionable warnings with weak supervision. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 1876-1880. IEEE, 2023.
[44] Cheryl Lee, Chunqiu Steven Xia, Longji Yang, Jen-tse Huang, Zhouruixin Zhu, Lingming Zhang, and Michael R Lyu. A unified debugging approach via lIm-based multi-agent synergy. arXiv preprint Xrx 17153, 2024.
[45] Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. Codeagent: Enhancing code generation with integrrtett agent systems for real-world repo-level coding challenges. arXiv preprint arXiv:2401.07333 2024.
[46] Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. Autocoderover: Au- tonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. pages 1592-1604, 2024.
[47] Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. Opendevin: An open platform for ai software developers as generalist agents. arXiv preprint arXiv:2407. 16741, 2024.
[48] Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with swe-gym. arXiv preprint 241.:224 2024.
13

---

[49] Siru Ouyang, Wenhao Yu, Kaixin Ma, Zilin Xiao, Zhihan Zhang, Mengzhao Jia, Jiawei Han, Hongming Zhang, and Dong Yu. Repograph: Enhancing ai software engineering with repository- level code graph. arXiv preprint arXiv:2410. 14684, 2024.
[50] Disha Shrivastava, Hugo Larochelle, and Daniel Tarlow. Repository-Ieve prompt generation for large language models of code. In International Conference on Machine Learning, pages 3 31693-31715. PMLR, 2023.
[51] Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian- Guang Lou, and Weizhu Chen. Repocoder: Repository-levet code completion through iterative retrieval and generation. arXiv preprint wrXiv:2303./25577 2023.
[52] Yangruibo Ding, Zijian Wang, Wasi Uddin Ahmad, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, and Bing Xiang. Cocomic: Code completion by jointly modeling in-file and cross-file context. arXiv preprint arXiv:2212 2022.
[53] Wei Liu, Ailun Yu, Daoguang Zan, Bo Shen, Wei Zhang, Haiyan Zhao, Zhi Jin, and Qianxiang Wang. Grapheoder: Enhancing repository-leve code completion via code context graph-based retrieval and language model. arXiv preprint arXiv:2406.07003. 2024.
[54] Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. Large language models on graphs: A comprehensive survey. IEEE Transactions Knowledge and Data Engineering, 2024.
[55] Yue Wang. Hung Le, Akhilesh Deepak Gotmare, Nghi DQ Bui, Junnan Li, and Steven CH Hoi. Codet5+: Open code large language models for code understanding and generation. arXiv preprint WxXiv23305.07922 2023.
[56] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jac Lec. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[57] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing vision- Laiio-la model's perception of the world at any resolution. arXiv preprint 11111:2409.12191, 2024.
[58] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv: 08415, 2016.
[59] Zihan Liao, Jun Wang. Hang Yu, Lingxiao Wei, Jianguo Li, and Wei Zhang, E2Ilm: Encoder elongated large language models for long-context understanding and reasoning. arXiv preprint arXiv:2409. 06679, 2024.
[60] Sijun Tan, Xiuyu Li, Shishir Patil, Ziyang Wu, Tianjun Zhang, Kurt Keutzer, Joseph Gonzalez, and Raluca Ada Popa. Lloco: Learning long contexts offline. arXiv preprint arXiv:2404.07979, 2024.
[61] Zhenyu Li, Yike Zhang, Tengyu Pan, Yutao Sun, Zhichao Duan, Junjie Fang, Rong Han, Zixuan Wang, and Jianyong Wang. Focusllm: Scaling I1m's 's context by parallel decoding. CoRR, 2024.
[62] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
[63] Ahsan Shehzad, Feng Xia, Shagufta Abid, Ciyuan Peng, Shuo Yu, Dongyu Zhang, and Karin Verspoor. Graph transformers: A survey. arXiv preprint 4xXiv.240..09777, 2024. 2024.
[64] CodeFuse. Codefuse-cge. httpp://ggtaab...ooccccceeeaaaaa cctaa///ooeeeuuooaaa/eeeeeeeeeee
2024.
[65] Zhiyuan Pan, Xing Hu, Xin Xia, and Xiaohu Yang. Enhancing repositooyyoll code generation with integrated contextual information. arXiv preprint arXiv:2406.03283, 2024.
[66] Yangruibo Ding, Zijian Wang, Wasi Ahmad, Hantian Ding, Ming Tan, Nihal Jain, Murali Kr- ishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, et al. Crosscodeeval: diverse and multilingual benchmark for cross-file code completion. Advances in Neural Information Processing Systems, 36, 2024.
14

---

[67] Yanlin Wang, Yanli Wang, Daya Guo, Jiachi Chen, Ruikai Zhang, Yuchi Ma, and Zibin Zheng. Rlcoder: Reinforcement learning for repository-level code completion. arXiv preprint arXiv:2407. /9487, 2024.
[68] Ming Liang, Xiaoheng Xie, Gehao Zhang, Xunjin Zheng, Peng Di, Hongwei Chen, Chengpeng Wang, Gang Fan, et al. Repofuse: Repository-lever code completion with fused dual context. arXiv preprint arXi::202.. 14323, 2024.
[69] Ken Deng. Jiaheng Liu, He Zhu, Congnan Liu, Jingxin Li, Jiakai Wang, Peng Zhao, Chenchen Zhang, Yanan Wu, Xueqiao Yin, et al. R2c2-coder: Enhancing and benchmarking real-world repository-level code completion abilities of code large language models. arXiv preprini Stli::2006.013552 2024.
[70] Wenhua Li, Quang Loc Le, Yahui Song, and Wei-Ngan Chin. Incorrectness proofs for object- oriented programs via subelass reflection. Asian Symposium Programming Languages and Systems, pages 289. Springer, 2023.
[71] Jason Sawin and Atanas Rountev. Assumption hierarchy for cha call graph construction algorithm. In 2011 IEEE 11th International Working Conference on Source Code Analysis and Manipulation, pages 35-44. IEEE, 2011.
[72] Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, Daniel Haziza, Luca Wehrstedt, Jeremy Reizenstein, and Grigory Sizov. xformers: A modular and hack- able transformer modelling library. http:///tt...ccoooooooooeer/t/// 2022.
[73] OpenAL. Introducing swe-bench verified. introducing- swe-
https://openaa..com/
mntroducing-swe--becch-oco wwwe-bench--errfieeee 2024.
[74] Jia Feng, Jiachen Liu, Cuiyun Gao, Chun Yong Chong, Chaozheng Wang, Shan Gao, and Xin Xia. Complexcodeeval: benchmark for evaluating large code models on more complex code. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, pages 1895-1906, 2024.
[75] A model fluent in 80+ programming languages. languages.htttp:: httpg///mmaeraal.a//arrr/ddddddt 2024.
[76] Stephen Robertson, Hugo Zaragoza, al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 3(4):333-389, 2009.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: All statements in the abstract and introduction are aligned with the main contribution of the paper: provide method innovatively integrating both semantic and structural information from code repositories into LLMs, enabling effective repository-level coding tasks without relying on agents closed-source models.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We included a section about the limitations of the work in Appendix E.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
16

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We upload the codes and instructions the cover the results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: For anonymity reasons, we provide an anonymous link that contains code with instructions, including README file and detailed code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification: the training and model details are specified Appendix Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due the resource limitation, we do not report error bars. Please note that in Section 5, we spent numerous resources training open-sourced LLMs on different backbones under different scenarios, which makes prohibitively run each experiments for multiple times.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the cost analysis in Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: /ate:://neuiippe wttpipss..cppublppe httppppbiic.ttiiccccccciiaiaaaaa
Answer: [Yes]
Justification:
We followed the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This work based on a publicly-available Github code repositories. This work is not related any private personal data, and there's explicit negative social impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification:
We do not foresee any high risk for misuse of this work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
Yes, we credited them in appropriate way.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our core method does not involve LLMs as any important, original, or non- standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hutpep//hearipssccc//offrrrrreee for what should or should not be described.
22

---

A Case in Issue Fix Scenario
ISSUE: The last checkpoint is being automatically restored when checkpoint exists. This is issue when the model has previously been trained with different settings I want train network from scratch...
enalaation Juce
collbank complgur
TrainerEvaluuiinn www.iraaninooopiiiii...
a/pyturch tranniicak config-y
Traitert TraaserCalllaeee CnllbackCafffg
paralled py
+-+ hipytecch, lighting trainericallback canfig.yy
TDMiiinDPMMisin
888
TninerCallion CallbaciCoCofiiiiiiii the proper values imiiialisation should done in child class default save path::
Daiinor.py
alsuns piens
Trainer
conovis eslends calls
TrainerFPMMisin
logger Union] iggnnng agges
Rgse Rose, bool]
weights save puth: sir
ckpt path: str checkpoint callback: ModelChookgeiin.....
ddp cnin))
Class
1pC, truit)))
Function
Figure 3: Illustration of real-world issue from pytorch-lightning codebase, where user wants to disable automatic checkpoint loading. Given the original issue, the corresponding liff-formatted patch (bottom left) shows all the code modifications in a linear fashion. Compared to code sequences, the relationships between them can be more clear if we represent the code graph (bottom right), where containment (solid lines), inheritance (dashed lines), and function calls (dotted lines) explicitly demonstrate the connections between different code snippets.
B
Details of Code Graph
B.1 Node and Edge Types in Code Graph
This section provides the node and edge types defined in our code graph. Table Table T0 detail the categories of nodes and edges, respectively. For now, code graph supports two bojected-orientee programming language (Python and Java).
Table 9: Node Types in Code Graph.
Node Type
Description
REPO
Virtual node, represents the root of the repository
PACKAGE
Virtual node, acts representation of directory in the file system Files ending with "-py"
FILE
TEXTFILE
Files that do not end with ".py", such as Markdown (.md) files and text (.txt) files In object-oriented programming, a class is blueprint for creating objects Refers to the function within classes standalone function
CLASS
FUNCTION
ATTRIBUTE
Includes global variables, member variables, and local variables
Table 10: Edge Types in Code Graph.
Edge Type
Description
contains
Indicating a hicrarchical relationship in which one entity contains another entity This type of edge captures the dynamic invocation relationship Representing inheritance relationship, where class extends another class Represent dependencies where one file imports another class/function
calls
extends
imports
implements
This edge is exclusively applicable to Java, denoting the relation where class implements an interface
B.2 Handling of Complex Dependiences
During the construction of code graph, we explicitly address both dynamic calls and multiple inheritance in the following way.
23

---

Dynamic Calls: We employ conservative resolution approach following the aprropapro
principle 770. When encountering base class method calls (e.g., (ese.) we include all
possible overriding implementations from subclasses in the calls set. This ensures we don't 't miss any potential execution paths.
We utilize the Class Hierarchy Analysis (CHA) algorithm 771 to properly handle inheritance relationships, including cases where classes inherit from multiple parent classes.
Multiple Inheritance:
B.3 Search on Code Graph
Graph search can be easily implemented in code graph. The first step usually begins with finding the source node. This can be achieved by many ways, such as indexing, keyword and embedding matching. Starting from the source node, different strategies can be applied, such as one-hop neighborhood, depth-first search (DFS), breadth- search (BFS), random walk, etc. is up to the application scenarios decide which search algorithm is the best. The result of graph search could be a sub-graph of the whole repository-level graph, containing the most relevant context for specific problems.
C Implementation Details
C.1 Details of Training CGM
This section details how we train CGM-Multi (multi-language version), CGM-SWE-PY (tailored for Python issue fixing), and CGM 7B series (based on different 7B base models).
C.1.1 Training Data
construct training data for different training phase of CGM respectively. Meanwhile, to enhance the model's 's ability in code completion, we also construct code completion samples for fine-tuning of code completion task. To explore the performance of CGM across different programming languages, our data includes both Python and Java. When constructing the training data, we filter out the repositories involved in the sets for testing avoid data leakage.
As mentioned in section
Data for Subgraph Reconstruction Pre-training:
We obtain 500k Python and 360k Java subgraphs (with the maximum length of 8k tokens) from a total of 20k high-star Github repositories.
Data for Issue Fixing: We collect 200k issue- issue-patch pairs (100k per language), from GitHub pull- requests. Among the 100k Python pairs, 14k are sourced from the SWEBench training set [8].
Data for Code Completion: The code completion samples self-constructed from the above repositories, 250k per language.
C.1.2 CGM-Multi
We initialize CGM-Multi with Swen2.5-72B-Instruu (00 as the base L.M.. Then pre-train it using subgraph reconstruction data and fine-tuning data (issue-fixing and code completion) both two languages (Python and Java). To ensure balance between different languages, we use 360k subgraph reconstruction data for both languages. Training uses 4-bit QLoRA on 64 A100s (batch=32, Ir=le4, and epoch=2). The encoder combines CodeT5+ (55) with LoRA (rank=64, and alpha=32), and the adapter uses two-layer MLP with GELU activation. The first layer of the adapter maps the CodeT5+ output dimension of 256 to 8192, and the second layer maintains the dimension of 8192, which aligns with the LLM's hidden dimension. We adopt Xformers 772 for efficient attention computation.
C.1.3 CGM-SWE-PY
CGM-SWE-PY, as model specifically designed for SWE-bench Lite, pre-trained using python subgraph reconstruction data (the entire 500k) and fine-tuned on specific python issue-fixing data (the 14k sourced from SWEBench training set). Besides, all details of training and parameters are set the same as CGM-Multi.
24

---

Table 11: Recall performance of each Graph RAG module on SWE-bench Lite and SWE-bench-jave Verified. The table shows the recall percentage for Retriever, Reranker Stage 1, and Reranker Stage components.
MODULE
SWE-RENCH LITE Si RECALL
SWE-BENCH-JVV VERIFIED
% RECALL
RETRIEVER
94
87
RERANKER STAGE
89
74
RERANKER STAGE
60
C.1.4
CGM 7B Series
We train several small-scale CGMs based on the existing 7B base models compare with small- scale models on code completion benchmarks. Specifically, we trained CGMs in seperate language based CodeI_lama-7B, StarCoder-7B, SeepSeek-Coder-1e
and -reen25-Coder-7B-Immee respectively. For each model each language, we use training data in the target language during both pre-training and fine-tuning stages. For example, train CodeLlama-7B with 500k Python subgraph reconstruction data and 250k Python code completion samples, and then evaluate it on the Python test of crosscodeeval.
Except for modifying the parameters in LoRA (set rank=32, and alpha=16), other training/parameter settings are consistent with CGM-Multi.
C.2 Recall Results for the Graph RAG Framework
We provide the recall of each component of our Graph RAG framework (in the file level), as shown in Table [1] The recall of each component on SWE-bench-java Verified lower than those SWE-bench Lite. One possible reason may be that the issues in SWE-bench Lite usually requires modifying one file, while the issues on the SWE-bench-jav: Verified sometimes need modify multiple files.
C.3 Hyperparameters for Inference
We use the same parameter settings for inference with LLMs (CGMs and Qwen2.5-72B-Instnn in the Graph RAG framework), setting tops 20, top, = 0.8, temperature = 0.7, and repetition penalty=
C.4
Cost Analysis
In this section, present cost analysis of the overall process, including the time required for code graph construction and computational expenses.
C.4.1 Code Graph Construction
The construction of a repository-level code graph usually takes 3 minutes or more depending on the complexity of the code repository (such as the implementations of different classes and the calling relationships between codes). Since code graph construction can be performed offline, does not impact real-time inference workflows. Additionally, optimizations such as incremental updates and parallel processing can further reduce latency for large-scale repositories.
C.4.2 Cost of Each Module
We analyze the runtime and resource requirements of each key module in system, focusing LLM inference overhead, memory consumption, and latency scaling.
Rewriter:
Requires two sequential LLM calls (Qwen2.5-72B-Insttuu)
Retriever:
25

---

Anchor node matching and subgraph generation take 3-7 seconds per issue.
Lightweight CPU operation.
Reranker:
Requires two sequential LLM calls (Qwen2.5-72H5 1-Itt-nstt
Latency additive Rewriter (x single-call time).
Reader (CGM):
Table reports the memory consumption and inference latency,
Latency increases by 0.5-0.7s per 1k tokens (Ik+8K:: 3.6s-).66).
Table 12: Inference Time and Memory Cost of CGM (Qwen2.5-72B).
# Input Tokens
Time (s)
Memory (GB)
1,000
3.934
68.79
2,000
4.408
68.98
3,000
5.055
69.43
4,000
5.808
70.26
5,000
6.432
70.77
6,000
7.163
70.98
7,000
7.838
71.44
8,000
8,553
72.02
C,5 Experimental Setup of Issue Fixing
C.5.1
Implementation Details of CGM
To adapt CGM in the issue-fix scenario, extend CGM GraphRAG framework (as described in section 4.. In this scenario, the inputs of CGM are the corresponding subgraph and prompt generated by R3 (Rewriter, Retriever, Reranker). In the experiment, we compare two pre-trained CGMs, CGM-Multi and CGM-SWE-PY (see Appendix C.T for training details), as Reader in our Graph RAG framework.
C.5.2 Datasets
The following three benchmarks, which focus on repository-level issue fixing, are all evaluated using the Docker executable environment.
SWE-bench Lite [5]: SWE-bench Lite contains 300 self-contained repository-Ieve issues from 11 repositories, designed to test the model's understanding of repository- code changes and its ability to generate correct patches primarily focused on Python. provides realistic software engineering environment, including execution contexts, to evaluate the model's ability to resolve real-world issues.
SWE-bench Verified 733 SWE-bench Verified contains 500 self-contained repository-
level issues from 12 repositories. This dataset contains samples that have been verified to be non pon-problematic by human annotators.
SWE-bench-java Verified (6): This dataset include Java issues from repositories, en- abling lanss-lagguage evaluation. Like SWE-bench Lite, it provides execution environments to validate the correctness of generated patches.
C.5.3 Evaluation Metrics
Resolve Rate (% R): The metrics used in the above benchmarks is Resolve Rate, which evaluates the correctness of generated patches for the issue-fix task. patch is considered resolved if correctly addresses the issue and is superset of the ground-truie truth edits.
26

---

C.6 Experimental Setup of Code Completion
C.6.1 Implementation Details of CGM
As simpler scenario than issue fixing, the files that need to be modified are given in the code completion tasks. Therefore, we obtain the input subgraph of CGM through a heuristic method, rather than the Graph RAG framework. To be specific, we take the given incomplete file as the center node, obtaining its one-hop ego graph from the repository-leve code graph. Note that nodes that need to be completed are not considered in this process. The resulting subgraph (graph modalities), and the incomplete files (text modalities), form the inputs to the CGM.
In this experiment, CGM-Multi (72B) and ssall-scale CGM 7B series can be found in the Appendix C.I
two size of CGMs for evaluation. Training details for the large-scale
C.6.2 Datasets
CrossCodeEval [66] an emerging benchmark for cross-file code completion, which is constructed from a wide range of real-world repositories from GitHub in four popular programming languages: Python, Java, TypeScript, and C#. our experiments, we evaluate the model's code completion ability only two languages, Java and Python. As shown in Table 13] we provide the dataset statisties of the CrossCodeEval benchmark for Java and Python.
ComplexCodeEval 744 is new benchmark for evaluating the performance of large code models in complex development scenarios. It includes 3,897 Java samples from 1.055 Java code repositories and 7,18 184 Python samples from 2,107 Python code repositories. Following the original setup of this benchmark, we randomly selected 100 samples each in Python and Java for the evaluation. Table 14 and Table 13 show the information of the selected samples. When evaluating the code completion capability, unlike the original setup which requires the model to complete the second half of the function, we ask the model to complete the middle line of the function given the contextual information.
Table 13: The statistics of the CrossCodeFyyal for Java and Python.
Feature
Java
Python
Repositories
239
471
# Files
745
1368
Examples
2139
2665
C.6.3 Evaluation Matries
When evaluating prediction code y in comparison to the reference ground truth y", the above benchmarks utilize the following two metrics: the exact match accuracy (EM) and the Levenshtein edit similarity (ES).
EM: The exact match accuracy (EM) is determined by an indicator function. This function takes a value of when the prediction y is exactly equal the reference y² and 0 otherwise.
ES: The Levenshtein edit similarity (ES) is calculated using the formula
Lev(y, v*)
ES 1
max(Ilvll, ll
Here, || is used to compute the length of a string, and Lev() employed to calculate the Levenshtein distance between the two strings y and y*
C.6.4 Baselines: Base Model
Given the subgraph same to CGM (see Appendix C.6. I) we textualize the graph into text sequnce and
input into the following baselines based on the prompt templates provided by each benchmark [66] 74) Since these base models have limitations on context length, we perform truncation on text inputs larger than 8k.
27

---

Mistral-Large-2 [75] is a model developed by Mistral AI with 123 billion parameters. It stands out with remarkable 128k tokens context length, proficiently handling dozens of languages and over 80 programming languages, excelling in code generation, mathematics, and reasoning. To be specific, we chose
as the latest version of Mistral-Large to compare with CGM.
DeepSeek-V24 [4] is a strong Mixture-of-Experts (MoE) language model characterized by econom- ical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token.
Qwen2.5 20 is a decoder-only LM series whose size varies from 0.5B to 72B trained on 18 trillion tokens. Its context length support up to 128K tokens and can generate up to 8K tokens.
Qwen2.5-Coder [29 is series of code-specific language models developed by Alibaba. Derived from Qwen2.5, comes in six sizes, trained on a vast 5.5-trillion-token corpus, and excels in various code-related tasks, outperforming many models of the same or larger size. Since it uses specific format and tokens for training on the fill-in-middee code completion task, we followed this prompt setting during the evaluation to obtain its true performance.
For the inference of the baseline model, we all deployed the above models using the VLLM framework with the model's 's default settings. All models inference on 4 A100s with 80G VRAM, except for DeepSeek- -225 which requires 8 A100s.
C.6.5 Baselines: RAG Method
BM25 76 is a classic information-retriess algorithm based on the probabilistic model. Its core idea is to rank documents based the relevance between query and documents. It serves as traditional retrieval method that does not regard the structural information naturally existing in the coding task, and only performs similarity matching based on word frequency and text length. It was used in the original CrossCodeFyyl dataset to search for cross-file information hased on the code snippets. In our experiments, we directly use the BM25 results provided by CrossCodeEvall.
R2C2-Coder 669 is a method that aims to enhance and benchmark the real-world repository-level code completion abilities of code Large Language Models. In particular, 1 C2CCh-Ene reduces cross-file information to skeletor 5) text by syntactically analyzing the content of code files. The cross-file file context is retrieved using BM25 after forming a candidate retrieval pool together with the context obtained from semantic- based retrieval. It takes into account structural information of the code but does not establish graph relations across code files.
RepoFuse 68 is a solution for the Context-Laaencyy Conundrum repository-level code completion. It constructs Code Knowledge Graph by analyzing the code graph dependencies in the repository and uses the repository-level graphs for retrieval. It integrates the Rationale Context obtained by analyzing the repository code structure and the Analogy Context based on the retrieval of similar code blocks, and filtering the context by scoring function.
framework for repository-level code completion, which can effectively improve code completion performance and has good generalization ability. During training, the RLRetriever is trained with a reward mechanism based on weighted perplexity to learn retrieval, while a stop-signal mechanism introduced filter candidate codes. inference, the trained Retrriiever retrieves useful candidate codes from the code repository and inputs them together with the incomplete code into the generator to complete code generation.
RLCoder [67] is reinforcement- leaanrreemeee--atnnngggenee
C.7 Details of Ablation Study
In this section, we first conduct an ablation study on our Graph RAG framework to verify the effectiveness of each component by SWE-bench Lite (Table Then, conduct the other ablation study on CGM itself to evaluate the effectiveness of model design by CrossCodeEval dataset 6T.
'The file skeleton is a hierarchical structure of the contents of a code file, containing class and function declarations, without specific definitions and comments.
28

---

C.7.1 Variants of Graph RAG Framework
The Graph RAG framework, comprising Rewriter, Retriever, Reranker, and Reader, extends CGM to real-world issue fixing. In Table we verify the effectiveness of each component in our Graph RAG framework by removing them. Here, we use CGM-SWE-PY (see Appendix C.1 for training details) as Reader.
w/o Rewriter: We directly perform semantic search based on the original issue descriptions, obtain the anchor nodes from the code graph, and provide them to Retriever. Removing Rewriter results in an 8.33% performance drop, which proves its effectiveness in enhancing the original issue descriptions.
Since there is no Retriever to provide filtered files and subgraphs, we input all the files in the original codebase into Reranker's Stage for selection, and the same time append the key information output by Rewriter into Reranker's s prompts. Based on the files output by Reranker, we build a subgraph using these files and their one-hop neighbors, as the graph modality input of CGM. The exclusion of Retriever results in an 11 11.33% performance degradation, a more severe drop than removing Rewriter, highlighting its importance in providing issue-related subgraph.
w/o Retriever:
w/o Reranker:
We use the top 5 files that are most similar to the query in embedding space (during semantic search) from the FILE node obtained by Retriever and provide them to Reader the files to be modified. Removing Reranker results in the largest performance drop (decreased by -24.67%), emphasizing its importance in improving the precision of retrieval results and providing the right, relevant files to Reader.
w/o R3: To evaluate the effectiveness of the RAG module, we create a baseline which removes the first three modules (Rewriter, Retriever, and Reranker) and feed the entire (truncated when the length exceeds the context length of the base model) repository graph as input to Reader during fine-tuning. Removing the RAG module leads to a poor performance (decreased by 33.33%), possibly due to excessive noise from the unfiltered repository graph and information loss from context-window truncation.
w/o CGM Reader (FlatGraph): To verify the effectiveness of CGM Reader in jointly modeling semantics and structure, we create a naive graph-based baseline which flattens code snippets based on topological structure [28), representing an alternative Reader with structure- structure-enhanced fine-tuning. The naive graph-based Reader only achieves 5.33% on SWE-bench Lite, far behind the proposed CGM (decreased by 37.67%).
C.7.2 Variants of CGM
In T we compare CGM with its variants in the following three aspects. The CGM we use here is trained on Qwen2.5-Coder-77 7B Instruct (see Appendix C.I for training details).
Semantic Integration:
To verify the design of CGM in understanding semantic information,
we compare with four types of variants: (1) freeze all parameters (include Encoder, Adapter, and LLM Decoder) (2) training the Adapter A (3) training the LLM Decoder D (4) training both the Adapter A and LLM Decoder D. Table 6 demonstrates that training the adapter A alone leads to significant improvements in the EM performance: a 22.26% increase for Java and a 21.43% increase for Python when comparing CGM-A with GGM- Freeze. Additionally, further training the LLM decoder D in conjunction with the adapter A esults in further enhancements, yielding a 5.33% improvement for Java and 4.80% improvement for Python. Finally, when the encoder &, the adapter A, and the decoder 'D are all trained together, we observe an additional increase of 5.71 % for Java and 6.12% for Python. This data illustrates that fine-tuning the encoder E, the adapter A, and the decoder is essential to effectively align the graph and code modalities.
Structural Integration: To verify the design of CGM integrating structural information, we remove the graph- aware attention mask during training, and use the original causal mask (denoted as "w/o MASK"). As shown in Table substituting the graph- attention mask in the CGM with standard causal mask results in a drop of 8.61% % in EM performance for Java and 5.56% for Python. This demonstrates the necessity of incorporating the structural information from the code graph into the CGM to maintain optimal performance.
29

---

Training Strategy: We remove the subgraph reconstruction pre- -re-training task to verify the effectiveness of this task, denoted "w/o RECON". Subgraph reconstruction pre-training plays crucial role, contributing 7.65% to the overall EM improvements.
C.8
Generalization of CGM on Different Backbones
To evaluate CGM with different backbones, we trained CGM using Llama3.1-700- 1-70B-Instrrtt,, Qwen2.5- Moder-32B-Instruee and Siven2.5-Code--1 Cooer--B-Inntrucce I-Ittuutt,t, in addition to Qwen2.5-72B. The results are summarized in Table 7
We find that the performance of CGM positively correlates with the LLM decoder's inherent cod- CGM's performance
ing and instruction-following abilities. For example, Llama3. decreased 17.67% compared to Qwen2.5-72B, possibly due to weaker inherent coding abilities (see Table 2 in (2)). Still, it surpassed Lingma-SWECG ( built on Llama3.1 1-000-1-s--r--CC by 18.33%, demonstrating CGM's power in improving open-source LLMs.
C.9 Test-Time Scaling Analysis
To further investigate the impact of inference-time computation, analyze the performance of CGM under the test-time scaling (TTS) strategy, using the standard Pass K metric. This approach generates K independent solutions for each issue and considers the issue resolved if least one of the solutions passes the unit tests.
The results are summarized in Table 8 We observe that increasing the number of attempts (K) leads to a consistent and substantial improvement in the resolve rate (% R) across both benchmarks. Specifically, by increasing K from (no scaling) to 3, the performance from 43.00% to 46.67%, an improvement of 3.67%. Similarly, on more challenging SWE- Bench Verified subset, the resolve rate increases from 50.40% to 53.20%, gain of 2.80%. This analysis demonstrates that allocating additional compute during inference through parallel sampling significantly enhances the model's s ability generate and select correct solution for complex software engineering tasks, confirming the benefits of leveraging this robust decoding strategy.
SWE-Bench Lite rises
30

---

Table 14: The Repositories and funcitons selected from ComplexCodeeEval-Pyyho
Repository
Function
lateol.bss/oachh
validate validate_output
transform
boto/boto3
fiink-extendedlaiilloo
get_conn
indico/indico
_process
aleju/imgaug
Iucparssons(OpennOveriigg
send_email load frames
williamfzcc/tugesspxx
dj- jj-stripe/fj-stripe
resync _insttnceeenss
biosustain/potion
parse_ carserrequest
MI.Bazaar/BTB
_fit from_json
mljar/mljar- supervisse
archesproject/arches
save
uber/causalml
causalsens request interpolate check baeck_ccomponnt
digiteinnotechh/aiirn
DeepLabCCu/DeepLLabbe
WeblateOrg//webatte
datteeasssssssoeesddaaatalssssss dataclasses
to internal value
etsy/boundary-layy
load authenticate process grad get index fit_
grafana/oncall
Syypromptyy/LLLMSttaaa
weihuayi/fealpy
django-ras-ng/djaanoocaa-ss
locii//jukebox
LAMMA--NUUDDeep-TTNn
transform
jazzband/django-iipll--itttt
cestory_fomm,
fabfuel/ecs-deploy
assume role log select_action action render sample_nodd popple_ndeesseee
daterdipauidattacheoks
pfnct/pfrll
bhch/ajjagoo-soonform
allenai/OI.Mo
A14Finance-
cnit_before__rart
somnenginnerrig//iixivetoor
parse_args run create_tensorboant fit probe init
ssube/onnx-web
IntelA//nauta
seikit-lear//scckitleees
awwlabs/aws--embeddee. metricee-e-eeeeeeeee
metrics-python
amundsen-io/amundsen
MatuCannaall/DeepTaablll
diyan/pywinrm
build, session set_ set_and_save fit compute load
adanchainz//jaagg-perf
ihmeuw-
google-reeercch//catteeeeeeee
langroid/langroid
jina-ai/jcloud
get_post_params
calldmmnn//oraniiee
from-string
georgia tech-db/evadb
exec
sibson/redbeat
due
bread-and-
betooeallneidd/Shhillleah
supports
kakeoenteepriis/JJRRLDY
sample
openstack/neutrrn
get_total total reservations
mobiusml/hqg
quantize
dang.
nasaharvest/presto
had__masked_ _ooendstoksssse
locuslab/mpe..ytooed
grad input
31

---

Lightning-Universee
Universe/lightnnn--lane openxriabxrioccaiizttoo
transform
knn_ knn_ratio _atch from hoom_ymml__gg file
bentoml//entooML
bayesiains/nflows
inverse resize run
open-mmlab/mmmev
threaat/routersploit
hscspring/hcgf
train
martenlicnen/torcheee
from k
arthurmensch/modl
split
pyg teaam/yytors toooooorrrammeee frame
forward
DjangoGGins/djanggiirrs
save
DytaCannassOOHHypemetts
create
randovania//aaddvaniaa
format run run_task generate
materfalpprojeechhreewors
LinkedinnAtiic/naarad
gift-surg/NiftyMICC
read_ aid_iiitlaiities
Projer-MOONA//MONAT...aa
entropy_3d_volle vol_volume
griffibblab/pVAAloolls
execute run get_cookie_lic list _save save_model model clean merge_code
Giskard-AA/giskard
Zemo@99/cchaGPP1-iisooooooo
aueeliign--mmaahiieleeerinnnnnee
api-ky
save save_mme
GhostMaangge/CCootwiirre
allwefantasylautoocoderr
caktus/dfango-treenay
save eval_deriv maake_schedule
simpeg/simpeg
arcee-ai/mergekitt
alex-petrenne
petrenkk//sampll-faceeeo
_save submit_payout _addrsss
RoboSats/robosats
pallets/quart
_create_ __reate_requess tat_rrruuuus_rfotee
michael-lazar/rtt
get_mimetype fromm_iil file read
router
drivendataorr/deonn
element element-hg/synapp ha/synapse
generate snneerateconfiiee coofionnoccacceeeee config_section section is is_aws_pod_v2
gquasecuritykuuhe-uunte.
CarterBBain/AlephNuul
simulate optimize_swan write ditte_datafrmee
metauto-ai/GPTS warm
m16team//fonaant
pytorchbearer//orchbbeerr
save_checkpoint
intelow
putelow/projjec//nnel/oo
chainer/chainerrl
initialize
petuum/adaptil
optimize
regel/loudm
forecast
ansible/ansible
construct mapping
Table 15: The Repositories and funcitons selected from ComplexCodeEVvllJava.
Repo apache/tajo
Function
findScalarFunccions
spring-proe projectssspriiggbbiaeee
Proerrrrpettt addAlias ToSql listAssets
fencentmnuscc/sppersoiii
tmobile/pacbot
microcks/microcks
CreateGGeneriiRResoureesse show New.NeeeQueeeee Quesstoonaoee execute SecuteWithStickyyComeecic from getXml
jralks-org/coommune
spring-prov
projects/sprinn--ataae data-redis
seache/james-projpee project apache/hop
32

---

expandLisPParametter commit
apache/archiva
Alfrescofalfrecc--rppsiito
check
52North/SOS
init index
kubernetes--Chenffavaa
xwikkxxwiki-platffrr
getFileltems
ctripcorp/x-pipe
analyze
prggtallpeeservaiionndoodd
gedAvaalableeiinnauueiiiis
Iridumldeenityyrrdiimm
generate
sofastack/sofaaaa
ParseGenTTblleDatas
Programme Vitam/vitan /
switchIndex
revelc/formatter-miie maantterr-maee--luugi pnuggnn
init unmarshalIXml
Hack23/cia
inmutables/immutables
oneLiner
pentaho/pennahh--latfoor
startup
ORCID/ORCID-Soume
getWorkInfo
88250/latke
resolve
mybatis/guice
get hashCode
gbil/ipt
add
jhy/jsoup
submit
neo4j/neo4j
nodeApplyChanger
PaladinCloud/CE
getAssetLists
alibaba/SREWorks
execute
installedPlugins
apache/syneope
apache/hadoop
checkAllVolumes
Qihoo360/Quickse
distinctList
openlookeng/hetu-coon
updateRows
canaaa/zanatt-pllaffoo
getLocales
AutoMQ/automq
OctoPerf/kraken
list
metamx/druid
run startSolver bind recordFrame childEvent
oceanbase/odc
lemaarkooopnann/nzyymee
Stratio/Decisionn
hilbaba//ebocily-spinn-boooo
getMatchOutcome
Aiven-Open/Klaw
gelConsuumrrGroupDetaiis
apache//doris-manager
create'Table init distinct
apacheshardiiespper-eciiiioo
apache/rya
ixrjog/opscloud4
queryMy WorkRole
google/nomulus
korakookkeaam-ccodenneejjaaaa
rconExec
wikimeddaa/ikkidata---eeer
load getSeckillList
techa03//oodskiill
runelite/runelite
onChatMessage
jenkinscibluuoocaan plugin
validtteAccesssTokeeSpppp.
formatProperties
jenkinsci/gitea-ppe plugin
getFileLink
gentics/mesh
getUid fromHttpRequest
twilio/twilio-jaan
ppdaicorp/das
checkSql
insideapp-oss/sonar--tie
define linkToo
dschulten/hyydrajjvaa
alibaba/fastjson2
of
opencast/opencast
multiTrimConcat
33

---

spring-projfee
spriing-roieta/sspiidddaa---iip
removeSubqueries
jline/jline3
open list
star-whale/starwhale
javaparser/javaparee
solveSymbolln
datavane/datasophoon
syneUserToHHsts
sakaiproject/sakai
upgradeRoleString
alswl/yugong
parseGlossaryFiie
jaayuu//llyuu--o---jaa-a--a
try tryAppend
google/mug
forDoubles
apache/druid
wrap equals
ExpediaGrroup/styx
apache/kylin
encrypt
dCache/dcache
map
AsqatassunAAsqatasuuu
findByAudiAAndUrll
mybatis/mybatis-3
register
apache/poi
setArrayooruua
SeareaDD--CCotttuuuuu-r------aee Saaaa--Spinn-Seeverr
parse
dianping/puma
copyFromLocal
alturkovic/disttrbuuclllo/
refresh
twitter/hraven
getAppid
OpenOLAAT/OpenOL..7
iaseetffflashcaadEEiiiing
apache/rockeimq
add[raansaciiooSuusciiiio
RIPE-NCC/whois
parse
odpi/egeria
MuilGGlossaryTermoonnee
ShifuMIL/shifu
exec
brimow//pring--oot-emaiii--eee
merge mergeeTemplatelnooString
Valion/lScurilyAggnyyylawwww
from
spring
addProperty
openeetworkinglaboonss
parse
Graylog2/graylog2-seeee
authenticate
opennrs/opennts--ore
handle
webx/citrus
getFastConstructor
34

---

D CGM for Issue Fixing: A Case Study
In this section, we take a real issue from the django/django repository as example to show how CGM solves a specific problem. Figure provides the original issue description and the intermediate outputs produced at each stage of our Graph RAG framework, and Figure S gives the generated patches along with the gold patch. To evaluate the effectiveness of graph modality assisting solving practical issues, we also compare the patches generated by CGM with and without code graph (as shown in 55. For the latter, the input of CGM is only the context files provided by Reranker, and does not include the subgraph generated by Retriever.
Issue:
Admin verbose, name as allows specification a verbose name and verbose plural for Inline classes in admin views. However, verhose name plura for Inline not currently based on specified verbose Instead, continues be based the model name, in the madel's Meta class. This was confusing to me initialy (1 didn't undersland why I had specity both name forms for an Inline illl wanted overuuee the default name), and seems incomsistent with approach
plural Description Django
verbose
specified
model's Meta class (which does auternaicalll base plural form specified verbose name). I propose that verbese name name_plural for an Inline class should by default be based verbose for implement this, including tests. Would be happy submit that.
specified.
written a patch
Rewriter:
"rewriter inferrnffeeereee output": [ "File names containing 'admin' AND classes related to 'Inline" to locate the definition of Inline classes.", "Functions or methods handling setting 'verbose name' directory to find where these attributes are processed.", "Code snippets where "InlineWodeIAdnin" is defined or used to identify the base class for admin inlines.", ", "File get "gget_verree carrbose_nane_puue implemented."" "Functions or nethods handling the initialization Inline classes within 'admin' directory find where default values set." "rewriter extractor
"admin' 'verbose nane_plural'
containing 'options.py AND functions related to find where the pluralization logic is currently
"rewriter pawtactorroupuutto.o cettput_entity"" [ http://cntrr///nnn/ooootttt
autput keyward": [ "verbose perbbse_name", "verbose paapnnaneeuuueeeee pane_plural",
"Inline"
Retriever:
Key Results django.util.text
"nodeType": "File",
"id": 52874,
"fileName": "text.py",
"filePath": "django/utilss"
"text": "import html, etnl.entitien iintitieslnimmon tags
"nodeType": "File",
"id": 53224, "FileNane": init_....y.
"filePath:: "django/utilsstransiii
"text": "VV'VWiittematioolzzaaioooo support
Reranker:
"FineRanker": {
djjngo//ontrib/ scccttrr//ddnnn/ggttoons.su
pyysten_prompt"" "You experienced software developer ...">
"user :scr_pproppt: soreppssttorssnndaannee hatpa//go///ooo///////o heapo//nnneossorry......
"response": "[start_of aww.oo.aaaallsssssiiiiiii issue describes
Figure 4: The given issue and the intermediate outputs produced by Rewriter, Retriever, and Reranker, respectively.
35

---

CGM with Code Graph
CGM w/o Code Graph
sddjammp//nttibb/eeeeee anddin/pptions py
a/djange/
*** wwspaann////ot/////////a/
py
b/ajjang/contt Ub/ada optiiuns..p
28 -7,6 +7,8 BB fron urllib. perse perse ieport quote as urlquote fron django teport forss
60 -1235,6 +1235,9 80 class InLineVodeladeiin BascHldde
self perbose_nane nane
fron django. cont trport settings
self, model.. Leeell verbese, nare
jaago.contrio Laport rassages
XT selt ver.oreeeeeeeeeeee cerrose__nn______eeeee paunep is None:
+fron django.utt ut..s........................... text usport formet_lary
Self verbose name, plural
fron dj jaano.uttls... translat issort ngettext
self www.oodd.........
plurat.
fron django. www.tttttttoo.o adain teport helpers, widgets
eltf self wer.tverrrreeeeeee
not
from www..oooorbbbaiiin.... www.ii.aaainnnhcccccc.. checks leport
use the verbose, naso basts plural
BasoModelad
form
inchinhh
self.verbose_neee namee plural self, verbose, name
-2842,
InLineModelAAne BaseePddelAddiii
welf.vertsose_nee nane None:
Gold Patch
self, verbovee_eeeeee name
selr selr.radel.. www.do.dd..ccec metll..nttttaaeeeeeeee www.nntaavvrree. verbese, nane
--git ardjange/ http:///ottrrt///aaatttttooo ht//ataropption...
Lf self. ver.vveeeeeeeeeee nane_plurat is None:
b/django/con:e cercrrtt//ae tb//daln////
self. wwt..veeeeeaaaeaeeeoo pat.nnnnnoauaatttaaaapaa
janga wt/p///t//ggttooooo By
self Lodel__neta. www.veeeeeeneeeee
If carpe_nnee__pprr_eee
b/django/ /oo/rrttttb httpt///////tttaaaaaaco
explicitly
-2837. +2837,13 de
lset. parert_sedel. model,
verbosc_ _ww.c_ptuuaatttt..c.c..... vasn't,
use Daango's plutaltzation of verbose, name Lf self verbose, nave is not None:
regi steredt .model)
fron di jnngo ut ils. text import forret_lary fron django, utils. translation trporr
plural self wwrbooereeeee.oo.. Nonel
sulf
ngettect
sarroos__nnn_sarre seLf.wodel weooel...ett...... verbose_nane__pteeeeeeeee pturat
solf.verbese, www.nnnnunrrtttc.ooooo
lazyt
ngettext(selff.to verbose_ nane, self, verbose_rare 's', 21)
vew.ose. www._eeteettttttt.....o formst, 'Is', set".veriose__ee
self verbboer
Nones
self verbose, hwwpeeeeeeee_ppeeeeeee none_ptural
selt. filf..odel... wete,
selr mnddl... mettat inta. verbose, caassss__nnn__iiieeee plural
self wwrbove.eeooo.... plural None:
salf. vertoserossooo. www.seeanwesseeeeeoooooo nww.._turaaaaa............. self.ecdel. moto. www.ose.eeoee..o .nan, plural
cet sedia)self):
eproperty
daf selis)
Patches generated by CGM (with without code graph), along with the gold patch. Green boxes represent successful patches and the red box represents unsuccessful one.
Figure 5:
E Error Analysis
In issue-fixing task, errors can be classified into two types: execution errors, where the generated code cannot be successfully executed, and unresolved cases, where the code is executable but does not fix the issue.
Notably, the vast majority ( 80%) of CGM's failures are unresolved cases, not execution errors. This highlights that our architecture's 's high fidelity in generating syntactically correct and executable code, even when the semantic logic for the fix not perfect.
Then we manually inspect around 20% (33/17) 171) of the failure cases (6 from execution errors and 27 from unresolved ones) profile the failure patterns and identify potential development directions.
The main reasons for execution errors are: (1) being misled by complicated issue descriptions (60%) and (2) occasional mistakes in syntactic generation (40%). For example, in the instance django__dy ango- model directly copies a diff snippet from the lengthy input issue as output. Breaking down complicated issues into clearer instructions for CGM Reader might alleviate this. As for the second reason, we observe occasional missing functionality, such as a missing return clause in instance sympy su__syymp10000 characteristic of code large language models.
The appearance of such errors, while infrequent, is known
As more major error, the unresolved ones mainly caused by: (1) limited reasoning ability for complex issues (55%), (2) knowledge gap (26%), and (3) cascading errors from the RAG module (19%).
We demonstrate the first reason by instance scikit cociktt-eemm_sskktt--eeeerr does locate and fix the user-reported vulnerable class, NearestNeighbow turally superior solution, provided by the golden patch, was to fix its parent class, NeighborsBase, from which NearestNeighhon there exist an edge connecting NearestNei ighbors with NeighborsBase. However, the L.M de- coder in CGM fails to leverage the edge to modify NeighborsBase instead of NearestMeeighorr..
Here, CGM However, the architec-
inherits. This distinction is not trivial. In fact, in the code graph,
36

---

For reason (2), knowledge gap is exemplified by LLM's unawareness of the current internal imple- mentation of third-pat hard-party packages: in the instance attempts an unsupported operation on NumPy array, which could be mitigated by including such details directly into the Code Graph.
les--eemmsiiii-eeaaaaa 10949,, model
Finally, the errors caused by RAG means the files needed to be modified have not been retrieved by previous module (Retriever or Reranker), thus leading to the failure of CGM. In other words, improving the Recall of the GraphRAG module can further improve the final performance of CGM.
Limitations
We have limited this work to Python and Java two popular object-oriented languages- so the current code graph schema is untested other paradigms. Although these languages cover a large part of real-world issue-fixing scenarios, extending our framework to other paradigms (e.g., multi-paradigm languages like Rust, or functional languages such Haskell) will require re-examining how code graphs are built to capture paradigm-specific structures.
G Prompt Template Example
This section shows the prompt templates used by Rewriter (Figure and Figure 7 and Reranker (Figure 8 and Figure in our Graph RAG framework.
37

---

Prompis:
<issue>
(ISSUE TEXT]
</issues
This issue related repository (REPO NAME)"
Instructions:
Ananlysis:
Analyze the provided issue description. Identify the relevant File, Class, or Function involved.
Determine the specific problem or error encountered and note any clues that may assist locating the relevant or problematic area.
Extraction:
After the anslysis, extract ALL the mentioned code eatities (File, Class, Function), especially Files.
Then extract three potential and meuningful keywords, respunding in the following format:
[start of analysis]
<detailed analyyiss>
[end] analysis]
(start of related code cutities]
centity name with path> [ond of related code antities)
[start related keywords]
<keywords>
[end_ of related
]eywords keywords]
Pay attention to the information in the error logs (if exists).
The buggy code exists solely the project deseribed in the issue (c.g.. django, skleam). Buggy location usually not in the tosts files or external packages.
Your extracted enities should be CONCISE, ACCURATE and INFORMATIVE.
Provide the relative puth for code entities specified (e g. package/foo.py) Relative path is relative the repository itself, do not 'home usernamee, 'etccservice" /ttre/mmaterr.
include
Do not include additional information such as line numbers explanations in extraction result.
Preferred extraction Examples Entities:
repoicart.py
Class User()
getData()
Preferred extraction Examples of Keywords:
train loop
hooks
docker
Unpreferred extraction Examples keywords:
something
imput validation
TypeError
Figure 6: Prompt for Extractor in Rewriter.
38

---

Prompts:
<issue>
(ISSUE TEXT)
</issuc>
This is an issue related to repository 'REPO NAMEJ".
Task:
Based the issue deseription provided, identify the characteristics of code entities (files, functions, class) that might need be modified.
For cach characteristie, generate scarch query that could help locate relevant code entities codcbase.
Instructions:
First, analyze the issue description and identify keywords, features, and functionalities that likely relevant to the modification of code entities.
Then, queries that capture these characteristics, focusing
File that may implement relevant functionalities.
Functions methods that related the features described
Any patterns or structures that might relevant the functionalities mentioned.
For example:
related the initialization of a neural network.
Function related to the training process.
Code used to configure the service.
Please answer in the following format:
[start of analysis]
<dctailed analysis
[end of analysis]
[start of relateee_gguuee queries]
query 1:
query 2:
[end of related queries]
Notes:
Your queries should DETAILED, ACCURATE and INFORMATIVE.
Your queries should complete sentences do include additional explanation.
focus the important charactcristies. Your queries should focus on repository code ilself, rather than information like commit history.
number queries is up
Pay attention information error logs exists).
Preferred Query Examples:
Look for references to "tædm" or "progress within the training loop files to where progress bars are currently
updated.
Code snippets where getthostynname' function from 'socket' module is called.
File name containing mmysqlppy' AND functions related to MySQLSStaemmentaamaee
initialization.
Functions or methods handling hostname resolution or encoding within 'dttadog checks' directory.
Find all occurrences of "carly_sttaaa stopping" within files that also mention "Traincr" to identify where carly stopping logic is implemented and potentially needs adjustment for non-default 'val scheck_interse
Figure 7: Prompt for Inferer Rewriter.
39

---

Prompts:
You experienced software developer who specializes in extracting the most relevant files for solving issues from reforence liles,
Task:
information recoived
a repository, find likely among may able
resolve the issuc.
Instractions:
I.nallysis:
Analyze the provided issue deseription and files, and pay attention the relevance of the provided files with the given issue, especially
those might medified during fixing the issue.
Deternine the specific problem or error mentioned in the issue and note any clues that could help your judgment.
Extmation::
Based on your analysis, choose the Top **10** relevant files which might be used in fixing the issue.
You should choose files from the provided files, and should modify their
Respood the following format:
[stari amalysis]
edetailed analysis-
[end] analysis]
[start relevant Giles]
1. <tile with its path
2- with its path--
3...
[end selevant files]
You can refer to to the information in the ercer logs (if exists).
in project deserihed in issue geg., django, skleam). File modification is usually the files or external packages.
The relevant file usually
The file choose should be contained in provided files.
Provide file path with files. De not include redundant suffix like "homme'ssrmmmme" 'iet isevvicee or /treeimaster"
include additional information such line numbers explarations extraction
Files for initialization and configuration might be medified during changing the code.
Preferred extruction Examples Related Files:
I. srctutils/Ell handlerpy
conolserv conelservoceeeseee iocalsorvico ice manager.py 3....
<repository
NAME)
<repository>>
<issue>>
[ISSUE TEXT)
<Hsssae>
creference python file lists
IREFERENCE PYTHON FILES)
<heference python file hist>
<other reference file list
(OTHER REFERENCE FILES)
<lother reforence file
Figure 8: Prompt for Reranker in Stage 1.
40

---

Prompis:
You experienced software developer who specializes assessing relevance file for solving the issme software repositories.
Task:
For a file provided, evaluate the likelibood that modifying this file would resolve the given issue, and assign score based on specific criteria.
Instructions:
Analysis:
content single file, pay attention any keywords, enur messages, specific functionalities mentioned that relute the file.
Analyze provided description
Deternine how closely the contents and functionality of the file are tied to the problem or eruor described the issue.
Consider the role of the file the overall project structure (e.g.. configuration files, core logic files versus test files, or utility scripts).
Scoring:
Based on your analysis, ussign score from that represents the relevance of mondifynn the given file in order to sulve the issue.
Specifications:
1. **Score [**; The file is almost certainly unrelated to the issuc, with no apparcnt conncction the functionality or ervor deseribed in
the ismue.
issue directly; possible edge **Score 3*a; The file has relevance to the issue; it might internet with the affected functionality indirectly and tweaking could be part of'a broader fix.
25a; file may Langentially related, but modifying unlikely
""**ooor 400; The file likely related the issuc;
code that interacts directly with the functionality in question and could
plausibly contain bugs that lead issue. **Seore 5*e; The file is very likely the root cause or heavily involved in the issue and modifying should directly address the error or problem mentioned.
Respond in the following format:
(start analysis]
<detailed analysig-
[end_ of analysis]
[start scure]]
Score <number>
Jend of score]
Notes:
The content of the file shows only the structure of this Gile, including the names of the classes and functiion defined in this file.
You can refer to information ervor logs (if exists).
<repository-
(REPO NAME) </repositor>>
<issue>
JISSUE TEXT;
</issug>
<file names
(FILE NAME;
</flle name>
<file
(FILE CONTENT;
<iflle content>
Figure 9: Prompt for Reranker in Stage 2.
41


Reviews:
[{"review_id": "Y40uQ9Yaxw", "reviewer": "Reviewer_7ALZ", "summary": "This paper presents the Code Graph Model (CGM), a framework that integrates code graph structures with open - source Large Language Models (LLMs) to address repository - level software engineering tasks. Traditional approaches relying on proprietary LLM agents face issues like unpredictability and limited accessibility. CGM overcomes these by enabling LLMs to understand codebase functions, files, and their dependencies through semantic and structural integration. It constructs a code graph for each repository, encoding node attributes and incorporating graph structures into the LLM's attention mechanism. Coupled with an agentless Graph Retrieval - Augmented Generation (RAG) framework, CGM achieves a 43.00% resolution rate on the SWE - bench Lite benchmark using the open - source Qwen2.5 - 72B model. This performance ranks highly among open - source models and demonstrates the feasibility of open - source LLMs in handling complex repository - level tasks without agents.", "questions": "1. Please submit the results or statistical curves of multiple TTS (e.g., different temperature/top-k) sampling runs, showing the potential improvement and stability of the model’s performance under best-of-n sampling. If CGM demonstrates significant improvement under TTS strategies, this will directly raise my evaluation of the model’s capability and the experimental thoroughness of the paper.\n2. If the authors can provide a detailed analysis of failure cases—including failure statistics, attribution of failure patterns, and concrete discussion of possible remedies—it will greatly increase my confidence in the research depth and the prospects for future improvement of this work.\n3. The resolution rate of CGM-Multi on Java repositories is much lower than on Python , but the reasons were not thoroughly analyzed. Is this due to insufficient support for Java-specific syntax (such as annotations or generics) in the code graph construction process, or is it because the Java sample proportion in training data is low? If possible, could you supplement experiments with other languages in the multi SWE bench[1]? If the current approach cannot be applied to other languages, please clearly state this limitation in the discussion, and provide a road map if available.\n4. Have the authors conducted any analysis of how the amount of training data affects repair performance? If so, please consider presenting this analysis to demonstrate that the capabilities of the CGM approach can continue to improve with increased data.\n\n[1]Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## **Strengths：**\n\n1. **Innovative Architectural Design:** Introduces Code Graph Models (CGM), achieving deep integration of code repository graph structures with open-source LLMs through semantic encoders and structure-aware attention mechanisms, solving the structural information loss problem caused by code graph linearization in traditional methods.\n\n2. **Efficient Agent-free Framework:** The developed Graph RAG framework requires only 4 modules  significantly reducing interaction complexity while achieving a 43% problem-solving rate on SWE-bench Lite using open-source models, surpassing closed-source model approaches like Claude 3.5 Sonnet.\n\n3. **Multimodal Information Integration Advantages:** Extends model context length by 512 times through a \"chunk encoding-adapter projection\" mechanism that compresses long text nodes into LLM-processable node representations, enabling simultaneous processing of graph structures (global dependencies) and complete code text (local details).\n\n4. **Strong Empirical Support:** Validates effectiveness in Python/Java cross-language scenarios, exceeding similar open-source model methods by 12.33% on SWE-bench Lite, while improving ES metrics by 19.8% in code completion tasks, demonstrating method generalization capability.\n\n5. **Interpretability Improvements:** Maintains stable performance even with 10% noisy inputs through controlled noise fine-tuning and structured attention masking, addressing the error accumulation pain point of traditional agent-based methods.\n\n## Weaknesses\n\n1. **No Analysis of TTS Performance:** Many frameworks (such as Agentless) employ test-time scaling to explore the diversity of model outputs. In the experimental setup of CGM, only a single sample was performed, without multiple sampling to explore the upper bound of the model’s capabilities.\n2. **Severe Performance Degradation for Small Models:** When using the Qwen2.5-Coder-7B-Ins model, only a 4% resolution success rate was achieved. In comparison, models of similar size such as Seed-Coder-7B-Ins can achieve a score of about 20%.\n3. **Limited Language Paradigms:** Experiments and tool design are conducted only on Python and Java (object-oriented languages). There is no exploration of the feasibility for functional, scripting, or more complex paradigms, and cross-paradigm transfer remains unaddressed.\n4. **Long Execution Time:** Constructing a repository-level code graph takes about 3 minutes. Combined with model inference time, this is significantly longer compared to other workflow-based frameworks.\n5. **Lack of Bad Case Analysis:** The paper lacks in-depth analysis of failure cases. Understanding why CGM fails in certain situations, such as incorrect handling of complex dependencies or loss of structural information, is crucial for further improvement.", "was_revised": true, "final_justification": "Because the authors provided a detailed analysis of TTS performance, a bad‐case analysis, and an explanation of why Java underperforms Python, along with the necessary clarifications, I believe the paper’s validity has been further strengthened and my concerns have been well resolved. Therefore, my final judgment is **5: Accept**.", "rebuttal": "_W1&Q1: analysis of TTS performance_\n\nThanks for your insightful suggestion! We have evaluated CGM's performance from Pass@1 to Pass@3 on both SWE-bench Lite and Verified dataset. Our results below show a clear and positive trend: multiple sampling boosts the resolution rate by approximately 2-3% on both datasets compared to single sampling (Pass@1).\n| **Pass@K** | **SWE-Bench Lite** | **SWE-Bench Verified** |\n| :---: | :---: | :---: |\n| K=1 | 43.00% | 50.40% |\n| K=2 | 44.33% | 51.40% |\n| K=3 | 46.67% | 53.20% |\n\nThis confirms CGM's performance scales with multiple samples. We will add this analysis to the final version.\n\n_W2: severe performance drop for small models_\n\nFirst, we want to clarify that the comparison between our 4% result and the ~20% from Seed-Coder is indirect. Seed-Coder's 19.2% result [R1] was achieved on the SWE-bench **Verified** dataset, which is known to yield higher scores than the more challenging SWE-bench **Lite** dataset used for our Table 12 ablation.\n\nTo create a direct and meaningful comparison, we established two crucial baselines for the **Qwen2.5-Coder-7B** model in an agentless setting:\n\n1. Performance on **Verified**: The Seed-Coder report itself shows that the base **Qwen2.5-Coder-7B** model, when used with the agentless workflow from [2], scores only **4.2%** on SWE-bench **Verified**.\n2. Performance on **Lite**: We further replicate this exact agentless workflow [2] with the **Qwen2.5-Coder-7B** model and tested it directly on SWE-bench **Lite**, achieving a resolution rate of **3.67%**.\n\nIn this direct, like-for-like comparison on SWE-bench **Lite**, our **CGM with the same backbone achieves 4.0%**, showing a slight performance improvement over this SOTA agentless baseline.\n\nThese results strongly suggest the **primary limitation is the intrinsic capability of the Qwen2.5-Coder-7B model itself**, limiting the performance across all frameworks built upon it. Moreover, our findings highlight that the **CGM architecture benefits from stronger LLM decoders**. As shown in Table 12, scaling the backbone from Qwen2.5-Coder-7B to Qwen2.5-72B-Instruct substantially increases CGM’s resolution rate from 4.0% to 43.0%.\n\n_W3: limited language paradigms_\n\nWe agree that expanding our graph-based framework to additional paradigms—such as functional, scripting, or other complex programming styles—is an important direction for future research.\n\nOur focus on Python and Java was determined primarily by the availability of high-quality benchmarks with testing environments. At the time of our experiments, the main multilingual benchmark initiative was an early release of what has since become **Multi-SWE-bench** [2]. **Before its formal release in April 2025, only the Java component (SWE-bench-java Verified [1]) was accessible for comprehensive evaluation, which is why it served as the basis for our Java experiments.** The full Multi-SWE-bench, with expanded language and paradigm coverage, was released too close to the submission deadline to allow for the extensive data collection and model training required for robust experimentation.\n\n**We have acknowledged this object-oriented focus as a limitation in Appendix E** and will expand this discussion in the final version. We are eager to extend CGM to cover a broader range of languages and paradigms by using the new Multi-SWE-bench in our future work.\n\n_W4: long execution time_\n\nWe would like to clarify that our current graph construction tool is a prototype, and there are several clear avenues for optimization. For example, we can parallelize the code parsing phase—followed by a final, serialized step for node relation establishment—to significantly accelerate graph generation. Moreover, after initial construction, the graph can be maintained through lightweight, incremental updates: when code is modified, affected nodes and edges are removed, and only the changed code is re-parsed and reinserted. \n\nWith these engineering improvements, we anticipate reducing graph generation time to about 10 seconds for small projects (under 100K lines of code) and 30 seconds for large projects (over 100K LOC).\n\nAccording to SWE-Gym [44], agent-based methods require at least 20 interaction turns for comparable performance, whereas our approach uses fewer than 5 LLM calls. Therefore, its overall runtime is substantially shorter, even accounting for the optimized graph construction and intermediate processing.\n\n**We will add this point in Appendix B.**\n\n_W5&Q2: Bad case analysis_\n\nThanks for your insightful input! Here we provide the **evaluation report** (generated by the official SWE-bench toolkit) for CGM-SWE-PY on SWE-bench Lite, followed by an **error analysis** to diagnose CGM's limitations..\n\n**Evaluation Report**\n\nThe toolkit shows CGM achieves an overall 43.00% (129/300) resolution rate. The detailed results by **repository** (e.g., performance on django or sympy) and **year** (year of issue origin) are below.\n\n| **Repository** | **Resolved Rate** |\n| :---: | :---: |\n| astropy | 1/6 |\n| django | 62/114 |\n| matplotlib | 8/23 |\n| seaborn | 2/4 |\n| flask | 0/3 |\n| requests | 3/6 |\n| xarray | 1/5 |\n| pylint | 3/6 |\n| pytest | 6/17 |\n| scikit-learn | 12/23 |\n| sphinx | 3/16 |\n| sympy | 28/77 |\n\n\n| **Year** | **Resolved Rate** |\n| :---: | :---: |\n| 2012 | 1/1 |\n| 2013 | 1/3 |\n| 2014 | 0/1 |\n| 2015 | 1/4 |\n| 2016 | 5/16 |\n| 2017 | 6/21 |\n| 2018 | 26/59 |\n| 2019 | 28/66 |\n| 2020 | 22/42 |\n| 2021 | 26/57 |\n| 2022 | 13/30 |\n| 2023 | 1/1 |\n\n\nCGM's performance is stable over time, indicating no data contamination, while its varied success across repositories suggests the potential influence of project-specific characteristics.\n\n**Error Analysis**\n\nFirstly, the vast majority (~80%) of CGM's failures are **unresolved cases** (i.e., executable but incorrect), not **execution errors**, highlighting its **high fidelity in generating syntactically valid code**, even when the semantic logic for the fix is not perfect.\n\nThen we manually inspect 33 failure cases (20% of total, 6 from execution errors and 27 from unresolved ones) to profile the failure patterns.\n\n1. The main reasons for **execution errors** are **(1) being misled by complicated issue** (60%) and **(2) occasional mistakes in syntactic generation** (40%). For example, in the _'django__django-12113'_, model directly copies a diff snippet from the issue description; breaking down issue into clearer instructions could alleviate this. As for the second, we observe occasional missing functionality like a missing return clause in _'sympy__sympy-13043'_. The appearance of such errors, while infrequent, is a known characteristic of code LLMs [R3].\n2. As a more major error, **the unresolved ones** are mainly caused by: **(1) limited reasoning ability for complex issues** (55%), **(2) knowledge gap** (26%), and **(3) cascading errors from the graph RAG module** (19%).\n    1. **Limited Reasoning:** This is demonstrated in _'scikit-learn__scikit-learn-11040'_, where CGM does locate and fix the user-reported vulnerable class, NearestNeighbors. However, the architecturally superior solution, provided by the golden patch, was to fix its parent class, _**KNeighborsMixin**_, from which _NearestNeighbors_ inherits. This distinction is not trivial. In fact, **in the code graph, there exists an edge connecting _NearestNeighbors_ with _KNeighborsMixin_**. Despite it, the current LLM decoder in CGM fails to leverage the edge to modify _KNeighborsMixin_ instead of _NearestNeighbors._\n    2. **Knowledge gap:** This is exemplified by **LLM's unawareness of the current implementation of third-party packages**. In _'scikit-learn__scikit-learn-10949'_, model attempts an unsupported operation on a NumPy array, which could be mitigated by including such details directly in the code graph.\n    3. **Cascading errors**: These errors occur when necessary files have not been retrieved by the previous RAG module, thus leading to the failure of CGM. **Improving the Recall of the GraphRAG module** can further improve CGM's final performance.\n\nThis analysis will be added to the final paper.\n\n_Q3: java is worse than python_\n\nWhile the performance of CGM-Multi is lower on the Java benchmark compared to the Python benchmark, we believe this reflects **the benchmark's higher inherent difficulty**, rather than a specific weakness of our model in processing Java.\n\nThis difficulty gap is not unique to our method; it is a consistent trend across other SOTA models in the field. To illustrate this, we can look at the performance of a powerful baseline from the official leaderboards:\n\n+ On the **Python-centric SWE-bench Lite**, **SWE-Agent + GPT-4o** achieves a resolution rate of **32.00%** (as seen in Table 1a).\n+ On the **SWE-bench-java Verified** benchmark, the **same** **SWE-Agent + GPT-4o** system's performance drops sharply to just **6.59%** (as seen in Table 2).\n\nThis demonstrates that the Java benchmark poses a substantially greater challenge for current methods. In this difficult context, our **CGM-Multi achieves a 14.29%** resolution rate, **more than doubling the performance** of the SWE-Agent+GPT-4o baseline. Therefore, while lower than our Python results, our Java performance is SOTA and significantly outperforms other leading methods on that specific, more difficult benchmark.\n\n_Q4: performance vs data amount_\n\nThanks for pointing this out! To analyze the influence of training data volume, we have trained CGM on subsets of the training data. The results below show a clear positive trend:\n\n| **Training Data** | **SWE-Bench Lite** |\n| :---: | :---: |\n| 10% | 29.67% |\n| 30% | 36.67% |\n| 50% | 39.33% |\n| 100% | 43.00% |\n\nThis demonstrates CGM's potential to improve further as more issue-fix pairs are available.\n\n[R1] Seed-Coder: Let the Code Model Curate Data for Itself\n\n[R2] Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving\n\n[R3] LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation"}, {"review_id": "qtGsaDUGjN", "reviewer": "Reviewer_2ymT", "summary": "This paper introduces Code Graph Models (CGMs), which is a graph-enhanced Large Language Model (LLM) architecture, for comprehensive repository-level code understanding. This architecture integrates repository code graph structures into the LLM’s attention mechanism and map node attributes to the LLM’s input space using a specialized adapter. When combined with an agentless graph RAG framework, this approach achieves a 43.00% resolution rate on the SWE-bench Lite benchmark using the open-source Qwen2.5-72B model.", "questions": "1. Can you add more in-depth analysis in the ablation study? Also, I would suggest adjusting the main content and adding the results of ablation study to the main content, as they are important.\n2. Can you use 2-3 sentences to summarize your strengths that are superior to previous works like RepoGraph and GraphCoder?", "limitations": "1. This work might provide more results using different open-sourced LLMs to demonstrate the effectiveness of the proposed approach.\n2. This paper might add a new section to discuss the broader impact of this approach to give more insights.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This work proposes an agentless approach to effectively address repository-level code tasks. \n2. When combined with an agentless graph RAG framework, this approach achieves a 43.00% resolution rate on the SWE-bench Lite benchmark using the open-source Qwen2.5-72B model, which ranks first among open weight models, second among methods with open-source systems, and eighth overall, surpassing the previous best open-source model-based method by 12.33%.\n3. The codebase is released for reproducibility. \n\nWeaknesses:\n1. Although there are engineering contributions in this work, the proposed approach seems too complicated to follow. Detailed ablation study and in-depth analysis are required in the main content to help readers gain more insight on the architecture design. \n2. This work mentions some related work such as RepoGraph[1] and GraphCoder[2] also leverage a graph-based approach to address repository-level code tasks, however, this work neither conducts a comprehensive performance comparison nor gives sufficient discussions on why the proposed approach is better.\n3. The experimental results are too limited to confirm the effectiveness of the proposed approach. We are not clear whether the effectiveness of CGM comes from the open-sourced code LLM Qwen2.5-72B or the architecture itself, since this paper only presents results from Qwen2.5-72B. More code LLMs such as DeepSeek-Coder and CodeLlama can be used for comparisons. \n4. Although this work claims the approach outperforms a couple of agentless baselines, there is an obvious gap between this approach and those agent-based baselines. The paper does not offer a compelling justification for why its proposed method should be adopted over agent-based approaches in applications, especially given that the latter demonstrate superior performance.\n\n[1] Ouyang et al., RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph, ICLR2025.\n[2] Liu et al., GraphCoder: Enhancing Repository-Level Code Completion via Coarse-to-fine Retrieval Based on Code Context Graph. ASE 2024.", "was_revised": true, "final_justification": "The rebuttal addresses my main concerns. Thus I have raised my ratings.", "rebuttal": "_W1&Q1: only engineering contributions & detailed ablation study_\n\nFirst of all, we would like to clarify that, beyond engineering contributions, we propose a novel method to integrate the graph modality with the language modality (i.e., LLMs). Specifically,\n\n> (i) **Semantic Integration**: Node attributes (containing code or comments) are first encoded by a pretrained text encoder and then mapped to the LLM's input space via an adapter, enabling the model to understand the semantic information of all nodes. \n>\n\n> (ii) **Structural Integration**: The graph structure is incorporated into the LLM through the attention mask, allowing direct message passing only between neighboring nodes in each layer of the LLM, similar to spatial Graph Neural Networks (GNNs).\n>\n\nMoreover, we have conducted comprehensive ablation studies in Appendix C.7, listed as follows:\n\n+ **Ablation of the high-level Graph RAG Framework**: Verifying the contribution of each pipeline component.\n    - **Rewriter**: Removing it dropped performance by 8.33%, confirming its value in clarifying issue descriptions.\n    - **Retriever**: Its removal caused an 11.33% performance drop, highlighting its role in finding relevant subgraphs.\n    - **Reranker**: Proved the most critical component; its removal led to a 24.67% performance drop.\n    - **Full RAG Pipeline (w/o R3)**: Removing all three modules caused a 33.33% drop, showing the RAG pipeline is essential to filter noise.\n+ Ablation of the Core CGM Reader: Analyzing the model's architecture, training, and generalization.\n    - **Model Architecture**:\n        * **Semantic Integration**: Progressively fine-tuning the model's Adapter, Decoder, and Encoder showed that training all components is essential for maximum performance.\n        * **Structural Integration**: Replacing the graph-aware attention mask with the original causal mask hurt performance by 8.61% (Java) and 5.56% (Python).\n        * **Graph Modality**: Replacing the graph modality input with pure text input (by flattening the text-rich graph as in RepoGraph and GraphCoder) resulted in a 37.67% performance decrease, validating CGM's superior design.\n    - **Training Strategy**:\n        * **Pre-training Task**: Removing the subgraph reconstruction task reduced performance by 7.65%, confirming its importance.\n    - **Generalization**:\n        * **Backbone Models**: CGM’s performance scales with the underlying model's coding ability across Llama3.1-70B-Instruct, Qwen2.5-Coder-32B/7B-Instruct, and Qwen2.5-72B, consistently enhancing each backbone.\n\nDue to space constraints in the initial version, we were only able to include the most essential ablation findings in Section 5.3 of the main paper. For the camera-ready version (which allows one additional page), **we will move more comprehensive ablation results and analyses into the main content, as you suggested.**\n\n_W2&Q2: discussion and comparison with RepoGraph and GraphCoder_\n\nWe would like to clarify that we have discussed the differences between our proposed CGM and previous graph-based methods such as RepoGraph [45] and GraphCoder [49] in Section 2 (Lines 141–147):\n\n> Recent research has also focused on enhancing code understanding by incorporating structural information through graph-enhanced repository modeling [45,48,49]. However, even when graph structures are used during retrieval, existing methods typically **flatten the retrieved code snippets into linear text sequences for downstream model prompting**. This flattening process fails to preserve the inherent heterogeneity between graph and text modalities. As a remedy, we propose the CGM that explicitly aligns these two distinct modalities, enabling better preservation and utilization of structural information throughout the entire process.\n>\n\nTo address performance comparison, RepoGraph achieves 29.67% on SWE-bench Lite (using GPT-4o as its base model), as reported in their paper. Our CGM attains 43.00% on the same benchmark using the open-source Qwen2.5-72B, demonstrating a significant improvement. **We will update Table 1 and the corresponding discussion** to include these direct comparisons for greater clarity.\n\nTo summarize the strengths of CGM over RepoGraph and GraphCoder in two sentences:  \n**CGM enables decoder-only LLMs to explicitly understand and attend over graph structures, going beyond simple textual flattening of code graphs. Additionally, by compressing node text into compact embeddings, CGM can efficiently encode much larger repository graphs than previous approaches, leading to superior performance and scalability.**\n\nFinally, as further evidence, our ablation study (Lines 810–814) shows that flattening the graph into plain text for LLM input leads to a 37.67% performance decrease, underscoring the importance of our explicit graph integration.\n\n_W3 & L1: effectiveness of the achitecture itself & more backbones_\n\nTo clarify, Table 1(a) shows that even when using the **same Qwen2.5-72B backbone**, methods such as SWE-Fixer and Lingma SWE-GPT achieve issue resolution rates of only 24.67% and 22.00% respectively on SWE-bench Lite, while our proposed CGM achieves a significantly higher 43.00%. This large margin demonstrates that **the gains stem primarily from our architecture, not just the choice of backbone**. \n\nFurthermore, as discussed in Appendix C.8, we have also trained CGM on other LLM backbones, including Llama3.1-70B-Instruct, Qwen2.5-Coder-32B-Instruct, and Qwen2.5-Coder-7B-Instruct, to test the generalizability of our approach. Notably, CGM improves the performance of Llama3.1-70B-Instruct from 7% (as reported in Lingma SWE-GPT [1]) to 25.33% on SWE-bench Lite—an absolute improvement of 18.33%. These results further confirm that the effectiveness of CGM comes from its architectural innovations, independent of the underlying LLM.\n\n_W4: justification for use against agent-based methods_\n\nWe would like to clarify the positioning and advantages of our approach compared to both open-source and closed-source agent-based methods.\n\nFor agent-based methods using open-source models, their performance is generally lower than CGM’s. As shown in Table 1(a) on SWE-bench Lite, CGM outperforms the strongest open-source agent-based baseline (Moatless+DeepSeek-V3) by 12.33%, despite DeepSeek-V3 itself being a stronger model than Qwen2.5-72B, which we use as our backbone. This highlights the advantage of our architecture over even the best open-source agent frameworks (see Lines 298–303, Page 8).\n\nFor agents based on closed-source model, the reliance on closed-source models creates substantial barriers for the broader SE community [13,14], including limited accessibility, inability to enhance or customize models for specific tasks, and serious security concerns regarding the privacy of sensitive code repositories when interacting with external API services. This point is mentioned in Lines 33-36. A more comprehensive discussion regarding agent-based methods and their practical challenges and limitations is provided in Lines 112-128 (Agent-drive Methods for SE in Section 2).\n\nAt the time of submission, our method ranks eighth overall on SWE-bench Lite, making it highly competitive even compared to the latest agent-based approaches. Importantly, CGM delivers these results using a fully open-source, agentless workflow, ensuring transparency, extensibility, and real-world deployability without the constraints of commercial API access. We believe this is meaningful for both AI and SE communities. Furthermore, the CGM framework is inherently flexible and **can be further improved as stronger encoders, adapters, and LLM backbones become available**, which is likely to further narrow (or even close) the gap to the very top results dominated by proprietary agents.\n\n_L2: broader impact_\n\nFollowing your suggestion, we will add the following discussion to the paper:\n\n> This research introduces a powerful new paradigm for reasoning about complex, interconnected information by deeply integrating text-rich graph structures directly into Large Language Models. Instead of perceiving a software repository as a flat, linear sequence of text, our approach endows the LLM with an explicit, structured understanding of the codebase's architecture. This is crucial for advancing AI in software engineering, as the model can now explicitly reason about the intricate web of dependencies—how functions call each other, classes inherit properties, and files are interconnected. Consequently, the model can better anticipate the downstream impact of a proposed code change, a reasoning ability previously exclusive to experienced human developers. Furthermore, the code graph itself serves as a valuable asset for human oversight, providing a clear visualization that helps developers grasp complex repository structures and validate the model’s logic.\n>\n\n> The implications of this approach, however, extend far beyond software engineering, offering a blueprint for enhancing LLM reasoning in any domain that can be represented as a text-rich graph. In academic research, for instance, a citation network where papers are nodes and citations are edges could be ingested. This would allow a model to not only summarize a paper's content but also understand its intellectual lineage, identify seminal works, and even forecast emerging research trends by analyzing its structural position within the scientific community. Similarly, in e-commerce, a graph of products and user interactions would enable hyper-personalized recommendation engines that understand the nuanced relationships between items based on both their descriptions and their connections in purchasing patterns. By fusing the structural context of a graph with the deep semantic understanding of an LLM, this methodology unlocks a new tier of contextual intelligence applicable across countless fields.\n>"}, {"review_id": "Xwqku5s56Y", "reviewer": "Reviewer_Fgkk", "summary": "Summary\n\nThe paper proposes a code graph-based model to resolve repository-level issue resolution problems like SWE Bench tasks. \n\n1. Their technique is a fixed workflow-based approach and does not depend on the agent to take autonomous steps. \n2. They model the entire repository as a graph structure, including the code repository structure, inheritance, and call graph structure.\n3. They have multiple steps in their procedure - \n\nThey comprehend and extract relevant entities from the natural language issue.\n\nThey perform a search on the code graph structure and extract the relevant files that include the nodes in the graph.\n\nThey perform re-ranking to pick the top most k files from the retrieved files. \n\nThey provide the selected graph structure as input to the LLM and modify the attention matrix supplied to the LLM. They then pass the actual file contents to the LLM. And ask the LLM to generate the final patch file.", "questions": "In Table 4, for the purposes of retrieval, were all the components used except for the final Reader? As the CGM reader is trained to output the final answer? If the Reader was also used during retrieval, how is the Reader used for the purposes of retrieval?", "limitations": "Yes, for limitations.\n\nNo, for potential societal impacts. The implications of open source models on the issue of automation should be considered in the paper.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The writing of the paper can be improved; it would have been nice to start with a motivating example and to explain the methodology using that example. As the proposed method is very involved and complicated, understanding the paper was challenging. \n\nStrengths\n\n1. The work is significant as it proposes an open-source trained LLM that achieves on-par performance with closed-source models.\n\n2. The paper introduces a novel technique to integrate code graph structures into an LLM; they explicitly train the LLM to understand the code graph structure and to reconstruct the file contents given the code graph structure. \n\n3. They show impressive performance on the SWE Bench Lite benchmark as well as the CrossCodeEval benchmark.\n\nWeakness\n\n1. There are a lot of moving components in the proposed approach. It is not exactly clear what components of the approach can be used for tasks other than issue resolution. Please also see my question on what component is used for retrieval-based generation.", "was_revised": true, "final_justification": "I appreciate the authors’ thoughtful and timely rebuttal. It clarified several points from my review and addressed some questions at a high level (like the role of the \"Reader\"). Balancing the helpful clarifications against the remaining limitations, my overall evaluation is unchanged, and I maintain my original score.", "rebuttal": "_W1 & Q1: Graph RAG for retrieval-based generation._\n\nThe core of our contribution is the CGM, which is a powerful graph-aware generator (or \"Reader\"). The Graph RAG framework (Rewriter, Retriever, Reranker) is a flexible front-end designed to supply the CGM with the most relevant context, and its components can be adapted or simplified depending on the task's complexity.\n\nTo clarify how this works in practice for our two experimental tasks:\n\n1. **For Complex Issue Fixing (Section 5.1):** This task starts with an ambiguous natural language query (an issue report). Therefore, we employ the **full Graph RAG pipeline (Rewriter, Retriever, Reranker)** to progressively refine the query and localize the most relevant subgraph to feed into the CGM. The CGM's role here is purely as the final **Reader/Generator**.\n2. **For Code Completion (Section 5.2):** This task is simpler as the location for code generation is already known. Consequently, the full, sophisticated RAG pipeline is unnecessary. We instead use a simplified retrieval strategy, as explicitly described in **Section 5.2 (Lines 334-336)** and **Appendix C.6.1**. Specifically, we directly build the input subgraph by retrieving the **one-hop neighbors** of the target function from the code graph. This subgraph, along with the incomplete file, is then fed into the **CGM, which acts solely as the generator.** \n\nTo ensure our comparisons in the code completion experiments were fair and informative, we designed them as follows:\n\n1. **Comparison against other LLMs (Table 3):** To isolate the power of the CGM as a generator, we **fixed the retrieval method for all models**. We used the same one-hop graph retrieval for CGM and all baseline LLMs (Mixtral, DeepSeek, etc.). For the baseline models that cannot process graphs, we flattened the retrieved subgraph into text. The results, which show CGM's superior performance on ComplexCodeEval, demonstrate its enhanced capability as a generator, even when the retrieval strategy is held constant.\n2. **Comparison against other RAG methods (Table 4):** To assess our end-to-end performance, we compared our simplified pipeline (one-hop graph retrieval + CGM generator) against other specialized RAG baselines (e.g., BM25, R2C2, RepoFuse). As detailed in **Section 5.2 (Lines 342-347)** and **Appendix C.6.4**, each baseline uses its own distinct retrieval mechanism. The results show that our graph-based approach typically outperforms these methods, highlighting the effectiveness of our method.\n\nTo directly answer your final question: **The CGM is exclusively used as the final generator in all experiments. It was never used for retrieval.** The components of the Graph RAG framework are responsible for retrieval, and their application is adapted to the specific task. We will ensure this distinction is further clarified in the final version of the paper.\n\n_L1: The implications of open source models on the issue of automation_\n\nOur work is motivated by the significant challenges that the current reliance on proprietary, closed-source models poses to the automation of software engineering. We have explicitly discussed these implications in several key sections:\n\n1. **Problem Formulation (Introduction, Lines 33-36 & 85-87):** We directly address the limitations of closed-source models, which currently dominate the field. We highlight that their use creates substantial barriers for the research and engineering communities, including:\n    - **Limited Accessibility & High Cost:** Preventing widespread adoption and experimentation.\n    - **Lack of Customization:** Prohibiting researchers and organizations from fine-tuning models on proprietary or domain-specific codebases.\n    - **Data Privacy & Security Risks:** Making it infeasible for many companies to use these models with their sensitive, internal code repositories.\n2. **Core Research Question (Introduction, Lines 37-41):** Based on these implications, we frame our entire investigation around the central question: **\"Can open-source LLMs be employed in an agentless manner to complete repository-level coding tasks?\"** This question positions our work as a direct response to the need for more accessible and transparent automation tools.\n3. **Contribution and Impact (Conclusion, Lines 376-377):** Our conclusion reinforces this theme by stating that our work \"establishes a new direction for developing powerful, transparent, and accessible tools for automated SE.\" By demonstrating that a fully open-source framework can achieve state-of-the-art results, we argue that our approach democratizes access to powerful SE automation, fosters a more transparent and extensible ecosystem, and provides a practical solution for real-world deployment where security and customization are paramount.\n\nIn summary, our paper is not merely an application of an open-source model; it is a dedicated effort to demonstrate its viability and to provide the community with a practical, powerful, and transparent pathway for future research in automated software engineering. We will ensure this narrative is further emphasized in the final version."}, {"review_id": "pCpUrv6DtN", "reviewer": "Reviewer_HU7w", "summary": "This paper offers a code graph model (CGM) system for repo-level SWE automation. It addressed a critical gap of open source LLMs in handling repo level tasks.", "questions": "Is it possible for the authors to reorganize the paper? I believe the content is quite deep and possibly with immense value. But, the presentation and organization perhaps need improvement.", "limitations": "The paper is Appendix-heavy. I think the authors should reconsider the paper organization.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper addresses a critical need. The methodology section is particularly strong, especially the Code Graph Construction and Code Graph Model components.\n\nHowever, I find that the Experiment and Results sections are inadequate and lack clearly defined research questions. Currently, the paper proposes a method and immediately presents Experiment and Results without sufficient context. I recommend authors to address following questions:\n\n1. What is the baseline?\n2. Which factors contributed most significantly to the overall success of the proposed approach?\n3. What are the fundamental research questions?\n4. I would like to see a clear assessment of the differential contribution of the graph-aware attention mask. Specifically, what would the performance be using a vanilla attention mask instead? The same applies to the graph-based retriever. These two artifacts are among the most intriguing elements of the paper.\n\nThat said, there is a wealth of valuable information in the Appendix, especially Figures 4 and 5. I recommend the authors carefully review their paper and present it following a clear scientific structure: Hypotheses → Experiments → Results → Conclusion.", "was_revised": false, "final_justification": "", "rebuttal": "_W1: what is the baseline_\n\n\n\nThere are two scenarios in our experiments - repository-level issue fixing (Section 5.1) and repository-level code completion (Section 5.2).\n\n**1. For Repository-Level Issue Fixing (Section 5.1, Tables 1 & 2):**\n\nOur primary baselines are the **SOTA methods from the official SWE-bench Lite leaderboards (as of May 2025)**. This ensures we compare our work against the most competitive and relevant systems in the field. To provide a fair and nuanced assessment of our open-source, agentless approach, we categorize our comparisons as follows:\n\n+ **Against Open-Source Systems:** We compare CGM directly with other fully open-source systems, regardless of the underlying LLM they use. As shown in **Table 1(a)**, our method (CGM-SWE-PY) ranks 2nd overall among all open systems on SWE-bench Lite.\n+ **Against Methods Using Open-Weight Models:** This is our most direct comparison group. Here, CGM-SWE-PY ranks **1st**, outperforming the next-best method (Moatless+DeepSeek-V3) by a significant margin of 12.33%.\n+ **Against All Systems (Including Proprietary Agents):** We also situate our performance in the broader context of all published results, including those using powerful, closed-source agentic frameworks. Our method achieves a top-8 overall rank, demonstrating its competitiveness even against proprietary systems.\n\n**2. For Repository-Level Code Completion (Section 5.2, Tables 3 & 4):**\n\nHere, we compare against two categories of strong baselines to isolate the benefits of our CGM architecture:\n\n+ **SOTA LLMs:** In **Table 3**, we compare CGM against powerful standalone language models (e.g., DeepSeek-236B, Mistral-123B) using a standard retrieval method. This demonstrates that the architectural innovations of CGM provide a substantial performance boost beyond what can be achieved by simply using a powerful LLM backbone.\n+ **SOTA RAG Methods:** In **Table 4**, we conduct a head-to-head comparison against established RAG and context-retrieval methods for code, such as BM25, R2C2, and RepoFuse. This is a critical comparison, as it shows that CGM's method of directly integrating graph structure into the model is more effective than methods that also use graphs for retrieval but then flatten the context into linear text.\n\nThese baseline selections were deliberately chosen to rigorously test our central hypotheses and clearly demonstrate the unique advantages of the CGM framework. Full details of all baseline implementations are provided in **Appendix C.6.4 and C.6.5**.\n\n\n\n\n\n_W2 & W4 & L1: most significantly contributed factors & the contribution of the graph-aware attention mask and the graph-based retriever & appendix heavy_\n\n\n\nTo clarify, we have conducted comprehensive ablation studies in Appendix C.7, listed as follows:\n\n+ **Ablation of the high-level Graph RAG Framework**: Verifying the contribution of each pipeline component.\n    - **Rewriter**: Removing it dropped performance by 8.33%, confirming its value in clarifying issue descriptions.\n    - **Retriever**: Its removal caused an 11.33% performance drop, highlighting its role in finding relevant subgraphs.\n    - **Reranker**: Proved the most critical component; its removal led to a 24.67% performance drop.\n    - **Full RAG Pipeline (w/o R3)**: Removing all three modules caused a 33.33% drop, showing the RAG pipeline is essential to filter noise.\n+ Ablation of the Core CGM Reader: Analyzing the model's architecture, training, and generalization.\n    - **Model Architecture**:\n        * **Semantic Integration**: Progressively fine-tuning the model's Adapter, Decoder, and Encoder showed that training all components is essential for maximum performance.\n        * **Structural Integration**: Replacing the graph-aware attention mask with the original causal mask hurt performance by 8.61% (Java) and 5.56% (Python).\n        * **Graph Modality**: Replacing the graph modality input with pure text input (by flattening the text-rich graph as in RepoGraph and GraphCoder) resulted in a 37.67% performance decrease, validating CGM's superior design.\n    - **Training Strategy**:\n        * **Pre-training Task**: Removing the subgraph reconstruction task reduced performance by 7.65%, confirming its importance.\n    - **Generalization**:\n        * **Backbone Models**: CGM’s performance scales with the underlying model's coding ability across Llama3.1-70B-Instruct, Qwen2.5-Coder-32B/7B-Instruct, and Qwen2.5-72B, consistently enhancing each backbone.\n\n\n\nFrom the above results, we can see that **among all high-level graph RAG components, the Reranker has the most significant impact**—removing it results in a 24.67% performance drop. This highlights the importance of precise file selection for effective issue resolution.\n\n\n\nWithin the core CGM Reader, as shown in Table 11, **joint fine-tuning of the adapter, encoder, and decoder (i.e., semantic integration) brings the largest performance improvement**. Notably, the adapter’s ability to map encoded node representations to the LLM input space is crucial for maximizing effectiveness.\n\n\n\nRegarding the **graph-aware attention mask (i.e., the ablation study of structural integration)**, using vanilla causal attention mask instead of the graph-aware attention mask leads to performance degradation on both Java and Python dataset (EM performance drops by 8.61% and 5.56%, respectively), as shown in Table 11.\n\n\n\nOmitting the **graph-based Retriever**—by relying solely on the Rewriter and Reranker for file selection—resulted in an 11.33% reduction in issue fixing performance on SWE-bench Lite (see Table 10). This underlines the Retriever’s role in supplying contextually relevant, structurally coherent subgraphs for downstream modules.\n\n\n\nDue to space constraints in the initial version, we were only able to include the most essential ablation findings in Section 5.3 of the main paper. For the camera-ready version (which allows one additional page), **we will move more comprehensive ablation results and analyses into the main content, as you suggested.**\n\n\n\n_W3: fundamental research questions_\n\n\n\nOur research is driven by observing the current landscape of automated software engineering, which is dominated by complex, agent-based systems that rely on proprietary LLMs (as discussed in Lines 26-36). This paradigm, while powerful, introduces significant challenges related to accessibility, cost, data privacy, and the inherent unpredictability of multi-step agentic reasoning.\n\n\n\nThis context gives rise to our primary research question, stated explicitly in the Introduction (**Lines 37-38**):  \n\"**Can open-source LLMs be employed in an agentless manner to effectively complete repository-level coding tasks?**\"\n\n\n\n\n\n_W5&Q1: reorganize the paper_\n\n\n\nWe agree that a clear presentation following the **Hypothesis → Experiments → Results → Conclusion** framework is essential for conveying our contributions effectively. We believe our paper already adheres to this logical structure, and we appreciate the opportunity to clarify this flow.\n\nHere is how the current sections of our paper map to the scientific structure suggested by the reviewer:\n\n1. **Hypotheses (Section 1: Introduction):**  \nOur primary hypothesis is explicitly stated as a research question in the introduction: **\"Can open-source LLMs be employed in an agentless manner to complete repository-level coding tasks?\"** (Lines 37-38). We further posit a core technical hypothesis that this is achievable **if the LLM is empowered to comprehend the repository’s structural dependencies via a code graph** (Lines 42-45, \"the key lies in empowering the open-source LLMs to fully comprehend code repositories... but also the dependencies across functions and files\").\n2. **Methodology to Test the Hypotheses (Sections 3 & 4):**  \nTo test our hypotheses, we designed and built the **Code Graph Model (CGM)** and the accompanying **Graph RAG framework**.\n    - **Section 3** details the construction of the code graph, the fundamental data structure required by our hypothesis.\n    - **Section 4** describes the novel CGM architecture and the Graph RAG framework, which are the instruments we use to conduct our experiments. This section explains _how_ we integrate semantic and structural information to test if this approach is effective.\n3. **Experiments & Results (Section 5):**  \nThis section is dedicated to the empirical validation of our hypotheses.\n    - We test our CGM on challenging repository-level tasks (Issue Fixing in Sec 5.1, Code Completion in Sec 5.2) and present the outcomes in **Tables 1, 2, 3, and 4**.\n    - Crucially, the **Ablation Studies (Section 5.3 and Appendix C.7)** directly test the core components of our technical hypothesis by measuring the performance impact of our structural integration (the graph-aware attention mask) and other key design choices. The results presented there provide strong evidence supporting our claims.\n4. **Conclusion (Section 6):**  \nThis section summarizes our findings, directly answering our initial research question. We conclude that our graph-integrated, agentless approach using open-source models is not only viable but also highly competitive, thus validating our central hypotheses.\n\nWhile we have not used explicit \"Hypothesis\" or \"Results\" subheadings, we believe this logical progression is inherent in the paper's current organization. We hope this clarification helps illustrate the scientific rigor of our paper's structure, and we thank the reviewer again for prompting this helpful discussion."}]

Author rebuttal:

Meta-review: This work argues that LLMs can more effectively process large, structured data when this structure is exposed to them. To establish this, it integrates the graph structure of software repositories into an LLM's inputs via a custom adapter. This setup helps it achieve strong performance on SWE-Bench with a large open-source model.

Reviewers commended the effectiveness of the approach and novelty of the technique. At the same time, they expressed concerns around the writing being hard to follow, which complicated the methodology.

In the rebuttal phase, most of these concerns were addressed through extensive clarifications from the authors. Given this, we recommend this paper for acceptance. We look forward to seeing the textual improvements incorporated in the final version.
Decision: Accept (poster)