From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph
Authors: Junfeng Gong
Zhiyi Wei
Junying Chen
Cheng Liu
Huawei Li
Venue: ICLR 2026 Poster (2026)
Keywords: Small Language Models
CUDA Code Generation
Reasoning Graph
MCTS
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Despite significant evolution of CUDA programming and domain-specific libraries, effectively utilizing GPUs with massively parallel engines remains difficult. Large language models (LLMs) show strong potential in generating optimized CUDA code from sequential code. However, using LLMs in practice faces two major challenges: cloud-based APIs pose risks of code leakage, and local deployment is often computationally expensive and inefficient. These drawbacks have spurred interest in small language models (SLMs), which are more lightweight and privacy-friendly. Encouragingly, recent studies show that SLMs can achieve performance comparable to LLMs on specific tasks. While SLMs can match LLMs on domain-specific tasks, their limited reasoning abilities lead to suboptimal performance in complex CUDA generation according to our experiments.
To bridge this gap, we propose ReGraphT, a training-free, retrieval-augmented generation framework that transfers LLM-level reasoning to smaller models. ReGraphT organizes CUDA optimization trajectories into a structured reasoning graph, modeling the combined CUDA optimizations as state transitions, and leverages Monte Carlo Graph Search (MCGS) for efficient exploration.
We also present a CUDA-specific benchmark with difficulty tiers defined by reasoning complexity to evaluate models more comprehensively. Experiments show that ReGraphT outperforms HPC-specific fine-tuned models and other retrieval-augmented approaches, achieving an average 2.33× speedup on CUDAEval and ParEval. When paired with DeepSeek-Coder-V2-Lite-Instruct and Qwen2.5-Coder-7B-Instruct, ReGraphT enables SLMs to approach LLM-level performance without the associated privacy risks or excessive computing overhead.

Published as a conference paper at ICLR 2026
FROM LARGE TO SMALL: TRANSFERRING CUDA OP- TIMIZATION EXPERTISE VIA REASONING GRAPH
Junfeng Gongl. Zhiyi Wei Junying Chen Cheng Liu
Huawei Li'
Institute of Computing Technology. Chinese Academy of Sciences
University of Chinese Academy of Sciences
South China University of Technology
guongjunfeng23s Liucheng)@@in acc.a.cccccccccccc
ABSTRACT
Despite significant evolution of CUDA programming and domain-specific li- braries, effecttively utilizing GPUs with massively parallel engines remains dif- ficult. Large language models (LLMs) show strong potential in generating op- timized CUDA code from sequential code. However, using LLMs in practice faces two major challenges: cloud-based APIs pose risks of code leakage, and local deployment is often computationally expensive and inefficient. These draw- backs have spurred interest in small language models (SLMs), which are more lightweight and privacy-friendl Encouragingly, recent studies show that SLMS can achieve performance comparable to LLMs specific tasks. While SLMs can match LLMs on comaain-specifee tasks, their limited reasoning abilities lead suboptimal performance in complex CUDA generation according to periments. To bridge this gap, we propose ReGraphT, a training-fie free, retrieval- augmented generation framework that transfers LLM-level reasoning to smaller models. ReGraphT organizes CUDA optimization trajectories into a structured reasoning graph, modeling the combined CUDA optimizations as state transi- tions, and leverages Monte Carlo Graph Search (MCGS) for efficient exploration. We also present a CUDA-specific benchmark with difficulty tiers defined by rea- soning complexity to evaluate models more comprehensively. Experiments show that ReGraphTT outperforms HPC-specific fine-tuned models and other retrieval- augmented approaches, achieving average 2.33x speedup CUDAEval and ParEval. When paired with DeepSeek- Cooeeee--Coe---eee V2-Lite-Insttee Instruct and Qwen2.5- Coder-7B BB-ntrrett ReGraphT enables SLMs approach LLM-level performance without the associated privacy risks or excessive computing overhead.
INTRODUCTION
The continuous performance improvement of NVIDIA GPUs (Dally et al., 2021; Lindholm et al., 2008 Nickolls & Dally 2010 Owens et al. 2008 has solidified CUDA as dominant program- ming model for high-peeffrmance computing tasks, including Al and scientific computing. How- ever, writing efficient CUDA code that fully exploits the massively parallel processing capabilities of GPUs remains significant challenge. To alleviate the burden of CUDA programming, prior search has proposed domain-specinn libraries, programming frameworks, and even domain-specific Tillet et al. 2019 Chen al., 20188 Che al., 2008 Bell & Hoberock] 2012; Hong et al.) 2019). While these approaches significantly enhance productivity and deliver competitive performance, they often demand substantial engineering effort, are restricted to specific application domains, and suffer from compatibility issues with frequent NVIDIA software updates.
languages (Brahmakshatriya & Amarasinghe 2022
Recently, large language models (LLMs) have shown remarkable potential in code generation tasks(Qiu et al., (020) across a wide range of programming languages languages-including including Python, C/C++, Verilog, and even high-level synthesis code for IPGAs--dmonstrrainngg new opportunities for au-
tomatic CUDA code generation from sequential code (Li et al.) 2023 Rozière et al. 2024 Luo
"Corresponding author.

---

Published as a conference paper at ICLR 2026
et al. 2023: Zheng et al. 2023 Zhang et al. 2024 Xiong et al. 2025 Zhang al. 2026). En couraging progress has already been observed in this direction(Bendi Ouis et al. 2025 Yan et al, Miranda et a 2025). Nevertheless, deploying LLMs such as DeepSeek locally is highly resource-intensive due to their large-scale architecture. On the other hand, using cloud-based APIs raises concerns over potential code leakage and privacy violations. These limitations have fueled interest in small language models (SLMs), which are significantly more lightweight, support conve- nient local deployment, and mitigate privacy risks. Notably, recent studies have shown that SLMs can achieve performance on par with LLMs in certain domain-specine code generation tasks(Brown al. 2020b).
2024
Despite this promise, training SLMs from scratch remains extremely challenging due to limited training data and convergence difficulties. Consequently, fine-tuning has emerged as a practi- cal means to produce compact, domainn-ppccialzzed and compute-efficien SI.Ms. For example, HPC-Coder-Vi V1 and V2 leverage curated parallel-code datasets to fine-tune large L.MSs and substan- tially improve their ability to generate high-performance parallel programs (Nichols et al. 2024b Chaturvedi et al. 2025), while RLPF employs reinforcement learning to further align LLM outputs with performance objectives (Nichols al. 2024c). However, find that the efficacy of these fine- tuned SLMs degrades markedly on problems demanding deeper, multi-step step reasoning. To quantify this effect, we sampled 20 benchmarks from ParEval and applied chain-of-though((() (Wei et al., 2023) (CoT) prompting to both the 671B DeepSeek-RI model and smaller 7B/14B code-specific SLMS. Figure I each model's average number of reasoning steps alongside the performance of the generated code. The results reveal that, while SLMs match LLMs on simpler tasks, they take sig- nificantly fewer reasoning steps and yield lower code quality on more complex benchmarks. This gap underscores the need for new techniques that can extend the reasoning capacity of lightweight models without sacrificing their deployment advantages.
In addition to fine-tuning, -tgmente- generation (RAG) is another widely adopted strat- egy for enhancing SI.M performance by injecting external information directly into the model's context. Prior works such as EVOR and Repoformer (Su et al., (0244) Wu et al. (024) have success- fully applied RAG to general code generation tasks, demonstrating notable improvements in output quality, especially for code involving recurring patterns known structures. However, while RAG effectively enriches contextual knowledge, it does not directly improve the model's reasoning capa- bilities. As result, EAG-nnnhnced SLMs still struggle with generation tasks that require multi-step logical reasoning, leaving a critical gap in handling more complex coding problems.
To enhance the reasoning capabilities of SLMS in CUDA code generation, propose ReGraphTT, training-fii ffree framework that augments SLMs with a structured reasoning process of CUDA- specific optimizations. ReGraphT lever- ages the reasoning strength of LLMs to collect step-by-step CUDA optimiza- tion trajectories, which are then ag- gregated into a unified CUDA reason- ing graph. This graph captures the intermediate states and transitions in- volved in transforming sequential code into efficient CUDA implementations. ReGraphT formulates the CUDA code generation task for SLMs as graph- based reasoning problem and incorpo- rates Monte Carlo Graph Search (MCGS) to guide the search over the graph efficiently. In addition, to support systematic evaluation, we also introduce CUDAEval, a benchmark suite specifically de- signed to assess CUDA code generation. CUDAEval organizes tasks into multiple difficulty levels based on the complexity of their underlying reasoning trajectories, enabling fine-grained analysis of model performance across different levels of challenge.
Averae Trajecury Length bifferert Modelx Using Oule Thought for CIDA Coda
truectory kirtth
1.5
35
Cuuncc S-Codee Thobetmat
DeepGert-Code- D-----tentacc
Danp Serb-4
Trajectory
7
--QeriS-Cokee Coder 7b4rstectt
Figure 1: Average number of reasoning steps and the per- formance of the generated code with SLMs and LLMss
Our contributions are summarized as follows:
2

---

Published as a conference paper at ICLR 2026
We propose ReGraphT, novel, training-free framework designed to mitigate the limited reasoning ability of SLMs in CUDA code generation. ReGraphT employs a CUDA Rea- soning Graph to encode optimization trajectories extracted from LLMs, thereby enabling SLMs to benefit from the rich multi-step reasoning encoded by larger models. The frame- work is open sourced Git
We formulate CUDA code generation as graph-based state transition problem and apply Monte Carlo Graph Search (MCGS) to efficiently navigate the CUDA Reasoning Graph. This formulation enables effective decision maaisiion-miinggg. at each optimization stage, enhancing the quality of generated CUDA code.
We design CUDAEval, a -UDA-specific benchmark suite that categorizes code generation tasks into levels of reasoning difficulty. Experimental results show that ReGraphT signif- icantly improves the reasoning and code generation performance SLMs, narrowing the gap between lightweight and large models in CUDA optimization tasks.
2 RELATED WORK
To support efficient CUDA programming, NVIDIA has developed a suite of CUDA Toolkit Libraries such as cuBLAS, cuDNN, and cuFFT (NVIDIA Corporation [2023a blc). which offer optimized implementations for common parallel kernels. For effective CUDA code generation for domain- specific applications such as Al and multimedia, several compilation- based methods (NVIDIA Cor- poration 2023d Chen et al. 2018 Ragan- Kelley et al. 2013 Tillet et (2019) have been proposed.
Beyond compiler-based methods, recent research has explored improving code generation via lan- guage models. LLMs have demonstrated strong capabilities in code generation across various do- mains, including cenerol-purpose programming. hardware design, and high-performance comput- ing. However, practical deployment of LLMs presents two major challenges: cloud-based APIs raise concerns over potential code leakage, while local deployment is often computationally expen- sive and inefficient. These limitations have driven interest in small language models (SLMs), which offer lightweight alternatives suitable for local use.
Supervised fine-tuning (SFT) has shown promise in domain-specifer tasks tasks-e.g. uuks--e...(attee (Fatema & Hu 2023) fine-tunes smaller LLMs for financial sentiment analysis with competitive results. HPC- Coder (Nichols al. 2024b Chaturvedi al. 2025) enhances L.M performance in generat- ing high-performance computing (HPC) code through fine-tuning with high-quality synthesized datasets. However, SFT has limited effectiveness in boosting multi-step reasoning capabilities in SLMs and often suffers from poor generalization (Ghosh et al. 2024). Knowledge distillation from L.LM-generaeed synthetic data has emerged as an alternative for improving SLM reasoning ability (DeepSeek-Al et al. 2025 Wang et al.) (0255. While effective, this approach relies heavily on care- fully crafted data recipes, making the distillation process challenging and sensitive to dataset com- position. The generalization of LLMSs can also be improved by injecting relevant external knowledge through RAG, but it may also introduce hallucinations or irrelevant information (Sun et al.) 2025) Gao et al. 2024). It becomes problematic particularly for CUDA code generation which typically combines multiple optimization techniques.
3 THE PROPOSED REGRAPHT FRAMEWORK
To address the reasoning limitations of SLMs in CUDA code generation, we propose ReGraphT, lightweight, training-free framework that augments SLMs with structured reasoning guidance. As shown in Figure? 2 ReGraphT first leverages LLMs extract multi-step CUDA optimization trajectories from sequential code, organizing them into CUDA Reasoning Graph. This graph encodes the step-by-step transformation paths and serves as a reasoning scaffold for SLMs. Then, it frames CUDA optimization graph traversal problem and applies Monte Carlo Graph Search (MCGS) for guided exploration, which enables SLMs to generate higher-quality CUDA code with improved multi-step reasoning capabilities.
httt:://ggthooooo////aaaahotttrr hta.:iiiiahiiielle//eeeeaaaha R/KGrapph
3

---

Published as a conference paper at ICLR 2026
ReGraph Construction Based LLM
Arallli
Shonds Sercery
Fast wath teteaaace
MCGS ReGraph
1600 Hardlan)
Repeated N Times
Arciflox
MCG6 Nidd
Okurgeor
ReGraph Ninde
reward
Rollout Fath hode
1rt Stete
Shandd
X6.7
x9.4
Menery
Opienistion Droryeser
Figure 2: Overview of the proposed ReGraphT framework.
3.1 REASONING GRAPH (REGRAPH) CONSTRUCTION
CoT (Wei et al. 2023) improves the ability of LLMs to perform complex reasoning through series of intermediate reasoning steps, allowing SLMs with limited intelligence to emulate the reasoning process of LLMs, boosting their performance on tasks involving planning and reasoning. Figure shows how CoT works CUDA optimization and produces a reasoning trajectory. To efficiently utilize the intermediate reasoning traces generated by LLMs, propose to organize the CUDA op- timization expertise in a novel graph structure called ReGraph. Prior to discussing the construction of ReGraph, we formally present its definition.
Definition 1. ReGraph can be defined G (V,)), E), referring to directed graph-based abstrac- tion derived from CUDA optimization expertise. ReGruph G, each represents an identified CUDA optimization technique. each E represents the link between two optimization methods.
According to Definition 1 ReGraph adopts directed graph representation that permits cycles. In ReGraphT, formulate CUDA code optimization state transition process on the graph. As all optimization processes start with sequential codes, there exists an initial state Vinit in G, which stands for the starting point of optimization. At initialization, ReGraph consists exclusively of vertex Vinits devoid of any edges. Building upon this, ReGraph completes the construction of the entire graph by merging CUDA optimization trajectories. Algorithm illustrates the complete process of ReGraph construction.
To acquire CUDA optimization trajectories, we prompt L.M to perform CUDA optimization step by step, thus yielding CUDA optimization trajectory. For each intermediate step of the trajectory, we instruct LLM to provide CUDA optimization method used, optimized CUDA code and correspond- ing reasoning process. Due to the stochastic nature of LLM outputs, identical CUDA optimization methods may be expressed differently by the LLM during different steps. Therefore, during the con- struction process, ReGraphT systematically records the existing CUDA optimization methods and prompts LLM to consolidate the current optimization trajectory with documented methods, thereby ensuring the consistency in the representation of each optimization method.
4

---

Published as a conference paper at ICLR 2026
LUM reasoning for trajectory
Legends
Optimization
Stone
Forawwton Shord Memayy for mit WONDES Wapp cheoggnce
CUDA Optimization State
CUDA Oprimoation Trjjectory
relabel
Merge ReGraph
Shaind Mentory math gutembation Gkergeooe
Relabel optimization method
ReGraph
merge
Unediig
Merged ReGraph
teck Dershayy
Miry flw nlergeerr
Coalnoing
Veruayy Conmscy
Phared Hersrr
Tol rut Cotveiviic
Figure 3: ReGraph construction based on LLM optimization trajectory.
Algorithm 1: ReGraph Generation
Require:
Sequential code dataset, Large language model, LLM
Require:
Output: ReGraph g = EV,E)
Initialize the set of CUDA optimization methods O :
2 Initialize the nodes of CUDA Reasoning Graph V = {(innet
Initialize the edges of CUDA Reasoning Graph & 0
for k E D do
Trajectory of CUDA optimization LLM(() (l:)
T + (LLM,,
CUDA Reasoning Graph merge-
Vinit for e E do
10
Get optimization method Method(e)
12
o O then
13
Find the node e corresponding if u E Succ(s) then
14
Find the edge u between s and v
Append optimization example to
17
else
u Edge(s, v) Append optimization example to u Append new edge u 3
19
20
22
else
23
v - Node(o)
24
Edge(s,
25
Append optimization example to u
26
Append node v to 1> Append new edge u to E
27
29
return CUDA Reasoning Graph G (V,E)
5

---

Published as a conference paper at ICLR 2026
After the CUDA optimization trajectory produced, ReGraphT merges the new trajectory into ReGraph. As mentioned in Definition [I CUDA Reasoning Graph composes of CUDA optimization method nodes and edges representing the link between different optimization methods. From this perspective, a CUDA optimization trajectory can be regarded as a specific state transition trajectory. The detailed state transition process is described by Algorithm [o lines 8 29. For the current trajectory 7 state s is initialized to Vinit- For each intermediate step in F ReGraphT determines whether its corresponding optimization method is already incorporated in CUDA Reasoning Graph first. If incorporated, processes separately based on whether the state transition represents exists (lines 13 22); otherwise, adds the method to the CUDA Reasoning Graph (lines 24 29). Afterwards, the current state 8 will be updated, which means moving to the node corresponding to the optimization method. More detailed construction steps of ReCraaph are provided in Appendix.
3.2 REASONING GRAPH (REGRAPH) EXPLORATION
Once ReGraph is constructed, ReGraphT leverages it to achieve the transfer of reasoning capabilities to SLMs via graph search, which means treating CUDA optimization state transitions ReGraph and determining the next optimization method used following a predefined strategy. A feasible search strategy is to enumerate all possible combinations of CUDA optimization methods based on ReGraph. Specifically, for each optimization state, all subsequent viable optimization methods are attempted until no more methods can be applied. However, despite the pruning of paths due to certain optimization methods being inapplicable. the time complexity of the enumeration search can still reach On where n is the number of nodes in CUDA Reasoning Graph and A represents the average of subsequent optimization methods for node.
Repeated N Times
Selection
Expansion
Rollout
Backpropagation
Leap Uerafing
Leop Devaling
Racilrize
Wayp Dierpeee
sices Bumeeeae
Swaypenee
Atrmn Costrchi
Menery Coatrrcirg
Shewe Meavey
skaved
Faxt eant Spasiietionn
Juave Minnayy
Fost mate Colniortbe
Fast Optiracrice
Fast resh Optiesiaiio
reward
vedyy
Figure 4: An overview of MCGS ReGraph.
To tackle the complexity of enumeration search, we propose Monte Carlo Graph Search (MCGS), combining Monte Carlo Tree Search (MCTS) with ReGraph, leveraging rollout feedback from future states to inform the subsequent choice of optimization methods. To enable MCTS on the graph structure, we introduce some adaptations to the standard MCTS. As shown in Figure we customize the key operations of MCGS on CUDA Reasoning Graph follows:
Selection: As MCGS progresses, nodes and edges from ReGraph are incrementally added to form new graph, which stands as sub- uub-graph of ReGraph. In the current iteration, MCGS select nodes from the existing graph based on UCB (Upper Confidence Bound):
2In(NN
P-UCB(s) (({) +
(1)
N(s)
Expansion: Unlike MCTS which decomposes problems at thought-levellChen et al. 2024 Xie
et al. 2024 et al. 2024 Hu et al. 2025). since MCGS method operates on a fixed graph ReGraph, the successor nodes of the current
the action space in each expansion step is also lxxed-specfiiclll optimization state within the ReGraph. If the node selected in the previous step has not been visited before, all successors will be expanded in MCGS to extend the entire search scope.

---

Published as a conference paper at ICLR 2026
Rollout: A rollout refers to simulating from the current state to evaluate it. Unlike MCTS which performs estimations on the tree, ReGraph contains cycles, which may cause simulations to fail to terminate. To facility the problem, made certain adjustments to the simulation strategy.
To avoid repeated visits to the same node, incorporated regularization term based the current visit count the simulation, balancing exploitation and exploration more effectively than standard e-greedy:
arg max [Q(s,a) (s,
with probability 1 -
n (als)
(2)
random action
with probability
We set a maximum step limit for each rollout to prevent non-termination. What's more, will also terminate if the optimization fails at any node.
CUDA optimization requires error-free compilation while maximizing performance. As result, each step of the rollout process, the optimized CUDA code undergoes compilation verification, functional validation, and performance benchmarking, yielding the following hierarchical reward design:
-1,
if S less 1,
reward
speedup - 1, if jest and speedup < 1, speedup,
(3)
if jest = and speedup 2 1.
In MCGS, each node be treated as terminal state, generating the final optimized code. The rollout's final reward is defined as the maximum reward observed on its trajectory.
Backpropagation To enable the rewards obtained in the current iteration guide subsequent pro- cesses, MCGS backpropagates the rewards along all nodes traversed in the selection path, updating their Q-values. After backpropagation, MCGS progresses to the next iteration.
THE PROPOSED CUDA EVALUATION BENCHMARK (CUDAEVAL)
verlly
Rule based fiteinng
</>
Sarne Extraction
dipandercy completion
Sequertial generation
CUDA
CUDA kernels
<CPP CUDA>
Figure 5: CUDAEval curation process.
Existing benchmarks such as HumanEval and MBPP (Chen et al.) 2021 Austin et al. 202 l) pri-
marily evaluate the functional correctness of LLM-generated code. Parbval Nichols et al.) (2024)), while designed for assessing parallel code generation, focuses mainly on various parallel paradigms and includes only 60 CUDA-specific instances lemitees-llimied in both scale and diversity. Moreover, lacks fine-grained classification scheme that reflects the complexity of real-world CUDA develop- ment. To address these limitations, we present CUDAEval, a dedicated benchmark for evaluating LLM performance in CUDA code generation across varying levels of reasoning complexity. Un- like prior benchmarks that start from sequential code, CUDAEval is built from real-world CUDA files. Specifically, we sample 10K CUDA files from the Stack v2. cuda hip dataset, which comprises 21.7K CUDA files collected from practical development scenarios.
The benchmark is curated by first applying heuristic rules (c.g., filtering files with local headers) to remove incomplete or unbuildable samples, followed by LLM-based extraction of CUDA kernels and completion of missing dependencies. I.LMs also generate corresponding CPU serial code and

---

Published as a conference paper at ICLR 2026
driver functions, with conrectness verified via compilation and execution of both parallel and serial versions. Only <C++, CUDA> pairs that pass build and output consistency checks are retained.
After validation, obtain 3,126 high-quality CUDA code pairs. Using DeepSeck-RI (DeepSeek- AI et al. 2025), we derive optimization trajectories and classify each sample into one of three dif- ficulty levels based on reasoning complexity. Specifically, samples with trajectory lengths of 1-2 are assigned to the easy-tier, those with lengths of 3-5 to the medium-tier, and those with longer trajectories to the hard-tier. Under this definition, the dataset comprises 1,783 easy, 791 medium, and 552 hard instances. We selected 10% of the tasks for final evaluation while the left are used for ReGraph construction. To further increase the challenge of the benchmark, we deliberately included a relatively larger proportion of the harder samples. As a result, the final CUDAEval contains 313 evaluation tasks, distributed across 106 easy, 105 medium, and 102 hard samples. Full details of CUDAEval pipeline are provided in the Appendix.
5 EXPERIMENTS
We conduct our experiments on single A100-80GB with Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz. For inference, we deploy LLMs using vLLM al. 2023 under BF16 precision.
5.1 EXPERIMENT SETUPS
Benchmarks: We have both CUDAEval and an established benchmark Parhhol(Nichols et al. 2024a) to evaluate the CUDA generation performance. ParEval covers 12 different computational problems and parallel models, but only 60 problems are available for CUDA code generation.
Baselines:
We compare ReGraphT with prior prompting and RAG methods. For prompting meth- ods, we compare with standard Brown et al. 2020a) and CoT Prompting/We Wei et al., 2023). For RAG methods, since there no RAG methods specifically designed for CUDA optimization, construct several RAG variants with the same CUDA optimization corpus used in ReGraphT. Specif- ically, we adopted a code similarity-base
retrieval approach the RAG baseline, which employ CodeBERTScore(Zhoo et al. 2023) as the embedding model to retrieve relevant CUDA optimiza- tion examples based on embedding similarity. In addition to this similarity- -based RAG baseline, we further include two stronger baselines-RethhnkMMCCe RethinkMCTS et al.) (2024) and MCTS-RAG(Hu et al.) 2025 to provide a more comprehensive comparison within the RAG family.
Hyperparameters: ReGraphT and BGGrapht-MCGS are evaluated under the same search budgets of 200. ReGraphT adopts a random sampling with max attempts of 5. The varying rollout configu- ration N in Figure 4 is set to 10. More details about Hyperparameter settings are in Appendix.
Metrics: To quantify the correctness of generated CUDA code, we adopt pass@k k introduced in (Chen et al.) (02T), while for optimization performance, we use speedup@ N(chhll et al. 2024a) to evaluate the performance improvement over the original sequential code.
5.2 EXPERIMENT RESULTS
Table presents the CUDA optimization performance under different methods with various code- specific SLMs DeepSeek-Codee Biieeek-Cooer-222.0 V2-Lite- Qwen2.5. - Yang et al.) 2024] et al. 2025), and the SOTA general LLMS 2024) and DeepSeek-R (DeepSekkkt et al., (025). In CUD- AEval, ReGraphT-MCGS achieves 73.3% in pass@1 on average with three code-specific SLMs,
Hui et al.] 2024), and
surpassing by +1..0% +9.0% and +6.8% compared to Standard, CoT and RAG in pass@) 1, +9.2%, +7.1% and +5.5% in pass@ 10. While ensuring the correctness of generated code, ReGraphT-MCGS also demonstrates superior quality in CUDA code generation, achieving at least x 1.84 speedup in speedup@1 and x 1.83 in speed compared to other baselines. For the search-based variants RethinkMCTS and MCTS-RAG although they achieve some improvement over other baselines, they still fall short of our ReGraphT-MCGS method. On the overall more challenging ParEval, ReGraphT-MCGS also demonstrates outstanding performance.
Beyond baseline comparisons, we further verify the efficacy of the MCGS strategy on ReGraph. According to Table I under the fixed search budgets of 200, ReGraph-MCGS achieves higher per- formance to ReGraphT, with +2.2% @nss@n, +1 +1.33% speedup@n n increase on average in CUDAE- val and + +1.7% pass @ +0.34% speedup@n in ParEval.Our experiments demonstrate that, under
8

---

Published as a conference paper at ICLR 2026
Table 1: CUDA generation performance on CUDAEval and ParEval benchmarks.
Model
paso
specdupen
pracee
puset
spoodapp011
p
spostlape
Spasstppi
Standaed
63.9
0.71
4.82 0.67
Coll [el 2023
0.70
4.97 0.67
DoppSock- V3-1 itc-lastract
Rethisk Mic
7.76 0.93
ReGraphT
10.78 9.82
6.32
Wereral
=
a
4.77
Qwen2.5 7B
RAG zznou
7.24
Rethoskkiry MiC
MCTS-RAG Hi
7.51
DeTinapiT-MCT
=
CiT Wis al 2023
67.1
0.70
HPC-Cole--V
64.8
65.5
0.69
RethinkhiC IZ AU1
7,48 1.26
41.3
5.01 =
MCTS-RAG Tiree
68.4
7.15 0.97
40.7
4.92 0.88
ReGrapn ReGraapt MEGS
7006
13:26 0.84
5000
10.21 0.800
14.39
53.3
10.01 0.82
Standard
63.9
64.9
7.52 = 0.67
2.771
43.9
45.3
5.08 a: 1165
0.67
Cut Wei etal. M023
67.1
69.3
8.16 0.69
8.16 0.68
48.1
50.0
5.40 1165
2.59
DeepSeek Dialli Qwen
RAGizaoi Zaou d Puzei
66.5
65.4
8.43
0.66
0.62
48.3
50.0
5.73
0.67
0.67
RethiskiCTT FE KiC
71.6
7:92 = 0.89
51.7
6.34 0.82
MCTS MCTS-RAG Fire aie sures
71.6
8:06 = 0.9
53.3
6.57 = 0.85
Ratraphi
75.8
14:15 0.77
55.0
10.92 0.73
ReGiagTTMMES
76.4
14,72 0.73
55.01
11.25 0.66
Standted
79.6
808
18:71
0.64
18.86
0
63.3
63.8
11:40
0.60
10.99 1009--661
GiT WWi cial 2023
80.2
81.5
18.58
0.57
18.45 0
61.7
62.1
11.53
+ 0.62
11.77
DeepSeek-R1
GiT WWi ctal. 2023
Standted
82.1
80.5
81.5
83.1
19.14
19:00
0:75
0:73
+021
19.22+0.00
63.
58.3
63.6
58.2
11.53 0.69
12.00 0.68
12.13 0.68
11:57-075
the same search budget constraints, ReGraphT-MCGS enables a more efficient exploration over ReGraph compared ReGraphT.
Table 2: CUDA Generation Performance Across Three Difficulty Levels in CUDAEval.
Model
Method
pass medium
spezdup@n medium
easy
hard
easy
hard
Standard
81.1
65.7
44.1
8.90 = 0.69
6.32 + 0.72
3.34 0,72
CoT
86.8
68.6
46.1
9.65 0.67
6,51 0.71
4.31 0.72
DeepSock-Coder-VV -or-------------nssstee
RAG
91.5
73.3
47.1
1.78
6.98 0.74
4.82 0.73
ReGraphT
90.6
76.2
52.0
15.86 0.80
12.38 = 0.87
8.84 : 0.86
ReGraph
90.6
79.0
54,9
17.82 a 0.80
13.79 a 0.80
9.69 = 0.82
Standard
81.1
65.7
43.1
8.51 + 0.73
5.62 + 0.76
3 0,80
CoT
82.1
66.7
43.1
8.47 + 0.75
5,54 0.79
3.46 0.78
Qwen2..5 5-Collr--.d B-Bntt7-tttttttt
RAG
85.8
69.5
45.1
9.54 0,80
6,23 + 0.81
4,29 0.84
ReGraphT
85.8
73.3
49.0
15.67
0.81
12.26 0.86
8.80 0.88
ReGraphT-MCGG
88.7
76.2
51.0
17.48 a 0.83
13.64 0.83
9.61 0.84
Standard
83.0
66.7
44.1
8.78 0.63
6.17 : 0.67
2.69 = 0.66
CoT
86.8
68.6
45.1
9.31 + 0.72
6.13 + 0.69
3.83 + 0.70
HPC-Coder-V2
RAG
81.1
66.7
44.1
8.82 + 0.69
6,22 + 0.69
3.08 + 0.68
ReGraphT
88.7
71.4
51.0
0.82
12,45 0.85
8.70 0.83
ReGruphT-MCGS
90.6
74.3
52.0
0.81 17.66
13.58 0.86
9.66 0.83
Standard
93.4
87.6
60.8
18.54 0.52
12.36 0.61
DeepSeck-V3 V3-0324
CoT
93.4
88.6
61.8
0.52 22.66
18.38 = 0.56
11.94 + 0.57
Standard
94.3
87.6
61.8
24.01
0.68
18.73 0.71
13.26 + 0.71
DeepScck-R1
CoT
95.3
89.5
63.7
24.24 a 0.69
18.96 = 0.72
13.40 + 0.70
To analyze the impact of reasoning capability on CUDA code optimization performance, we further investigated the relationship between the length of reasoning trajectories and corresponding perfor- mance based on performances across different difficulty levels in CUDAEval. As show in Table 2. ReGraph demonstrates varying performance across different difficulty levels. On the easy level which requires minimal reasoning, ReGraph shows no significant gap compared to other baselines, and occasionally underperforns CoT and RAG approaches. However, task difficulty escalates to medium and hard levels demanding more advanced reasoning, ReGraph begins to demonstrate marked advantages over alternative methods.

---

Published as a conference paper at ICLR 2026
Average Trajectory Length Across Different Methods
Trijectory Length Impact on Parformane Under Varying Difficulty
20
Length
15
Standard
COT
ReGophT
TeGrophT-MMGSS
2
IN every
I mediau
hard
sewbard
Figure 6: Normalized performance of the generated CUDA code under various difficulty levels
To further demonstrate ReGraphT's enhancement SLMs' reasoning capabilities in CUDA gen- eration tasks, we perform deeper examination regarding the correlation between the complexity of reasoning trajectories and optimization performance. As observed in Figure 6 limited by the reasoning capacity of SLMs, the length of CoT reasoning trajectories exhibits minimal variation across difficulty levels, while for ReGraphT series, the difference in the average length of reasoning trajectories between the easy and hard difficulty levels can reach up to 4.8, thus demonstrating that ReGraph can boost SLMs reasoning in CUDA generation process. What's more, in comparison to ReGraphT, ReGraphT-MCGS exhibits longer average reasoning trajectories, highlighting advan- tage in search efficiency. In addition to reasoning boosting, Figure 6 further demonstrates the role of reasoning enhancing performance for CUDA optimization tasks. From the figure, we observe positive correlation between CUDA generation performance and reasoning chain length across all difficulty levels, until the reasoning steps reach certain threshold. Notably, different difficulty tasks exhibit distinet thresholds, which generally show positive correlation with task difficulty. Beyond the threshold, the performance gap becomes statistically insignificant.
CONCLUSION
In this paper, we propose ReGraphT, training-free framework which transfers the CUDA opti- mization reasoning capability of LLMs to SLMs via Reasoning Graph. According to experiment results, ReGraphT has demonstrated significant effectiveness in enhancing SLM's reasoning capa- bility for CUDA -gDeerrtt content and improving generation quality. This work demonstrates that the proposed reasoning graph can transfer the reasoning capability of LLMs SLMs effectively and ReGraphTT can be potentially applied to more code generation scenarios that require complex or long reasoning procedures. We will investigate this approach in our future work.
ACKNOWLEDGMENTS
This work was partially supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant No. XDB0660103) and the State Key Laboratory of Processors, Insti- tute Computing Technology, Chinese Academy of Sciences (Grant No. CLQ202403).
REFERENCES
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021. / http//awww...g..oaaaaaaaoa abs
2108.011 07732
Carlo Baronio, Pietro Marsella, Ben Pan, Simon Guo, and Silas Alberti. Kevin: Multi-turn for generating cuda kernels, 2025. WRRLhttss///axxtt http://.rggabbbb///b/ba/e 11944.1..111.
Nathan Bell and Jared Hoberock. Chapter 26 thrust: A productiiitty-orieneed library for cuda. In
Wen mei W. Hwu (ed.), GPU Computing Gems Jade Edition, Applications of GPU Computing
Series, pp. 359-371. Morgan Kaufmamn, Boston, 2012. ISBN 978-0-12-385963-11 doi: https:
//doi.org/ 1tt..org/00.011//9977-44 1tt.:///878800-----8------111000
URL https: /ww. http:h://www...eeeeeeeeeeecceee sti..ncdieiicc.....t.....
c com/scciance//rticllee.. hatp://cccanceerraaaaaaa///////2
10

---

Published as a conference paper at ICLR 2026
Yannis Bendi-Ouis, Dan Dutartre, and Xavier Hinaut. Deploying spunc large language mod- els: A performance analysis, 2025. URL htttps://aaxii......aa org/ abs /es//2499.- 14887
Ajay Brahmakshatriya and Saman Amarasinghe. Graphit to cuda compiler in 2021 loc: A case for pigh-peeformmnce dsl implementation via staging with buildsl. 2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), pp. 53-65. IEEE, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Gretchen Krueger. Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler. Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., ceereeeeeeddddnnee..ee.oee..eeee neurins ca/pprr http:pppeerrieeeaeeeeeeee www.s//papeeeee//////// 2020a. URL 0tle://577cbbrrc0311110000001122
File/
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Verbert-Vosss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020b. URL htt.httpss///rxi.oorr///sss//
Aman Chaturvedi, Daniel Nichols, Siddharth Singh, and Abhinav Bhatele. Hpe-coder-v2: Studying code lims across low-resource parallel languages. In Proceedings of the ISC High Performance Conference, 2025.
Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W. Sheaffer, and Kevin Skadron. A performance study of general-purpose applications on graphics processors using cuda. Journal of Parallel and Distributed Computing, 13110)3377-13880 2008 ISSN 0743-7315. doi: https://doi. org/10.1016/j.jj.. 11w.110066....c.0000... 05.014. URL https: //wp:://wwwww// sci encedi com/ sc wacieeece ence/ General-Purpose Processing using Graphics Pro-
cessing Units
Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: Process supervision without process. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural In- formation Processing Systems, volume 37, pp. 27689-27724. Curran Associates, Inc., 2024. URL
.ddf
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss. William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mc- Grew, Dario Amodei. Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. 2021.
Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. Tym: an automated end-to-end optimizing compiler for deep learning. In Proceedings the 13th USENIX Conference on Operating Systems Design and Implementation, OSDI'18, pp. 579-594, USA, 2018. USENIX Association. ISBN 97819319714777 1478.

---

Published as a conference paper at ICLR 2026
William Dally, Stephen W Keckler, and David B Kirk. Evolution of the graphics processing unit (gpu). IEEE Micro, 41(6):42-55 51, 2021.
DeepSeek-AL. Deepseek-v3 v3 technical report, 2024. URL 19437
De.pSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai. Deli Chen, Dongjee Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang. Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingehuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang. Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, Wang. Wangding Zeng. Wanjia Zhao, Wen Liu, Wenfeng Liang. Wenjun Gao, Wenqin Yu, Wentao Zhang. W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li. Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang. Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha. Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song. Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning. 2025. URLpttts:: wttp://aaxvvvvv.. www.g//s/////oooorrrooooosaaaeee 2501 12948
Sorouralsadat Fatemi and Yuheng Hu. A comparative analysis of fine-tuned ilms and few-shot learning of I1ms for financial sentiment analysis, 2023. URL aattps://arxiss org/ abs/
2312. 08725
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. Retrieval-augmmented generation for large language models: A survey, 2024. URLbttps::
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, and Dinesh Manocha. A closer look at the limitations of instruction tuning. In Proceedings of the 41st International Conference on Machine Learning, ICML'24 JMg,. 2024.
Changwan Hong, Aravind Cestummrran-Rjjaan Israt Nisa, Kunal Singh, and P. Sadayappan. Adaptive sparse tiling for sparse matrix multiplication. In Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming, PPoPP '19, pp. 300-314, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 4781450362252. doi: 10.1145/3933883.2995711 11 3295712
URL
Yunhai Hu, Yilun Zhao, Chen Zhao, and Arman Cohan. Mcts-rag: Enhancing renireeeeeedreeellenneeeeeeeed generation with monte carlo tree search, 2025. htw.:htpss//aax....r//////////// 20757
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
12

---

Published as a conference paper at ICLR 2026
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng. Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Qingyao Li, Wei Xia, Kounianhua Du, Xinyi Dai, Ruiming Tang, Yasheng Wang, Yong Yu, and Weinan Zhang. Rethinkmets: Refining erroneous thoughts in monte carlo tree search for code generation, 2024. UtL/attps://axxii/i org/ abs / 2409. 09584
Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Mare Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhski,, Terry Yue Zhuo, Thomas Wang. Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Armel Zebaze, Ming-Ho Yee, Lo- gesh Kumar Umapathi, Jian Zhu, Benjamin Lipkin, Muhtasham Oblokulov, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Wenhao Yu, Swayam Singh, Sasha Luc- cioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Tri Dao, Mayank Mishra, Alex Gu, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, and Harm de Vries. Starcoder: may the source be with you!, 2023. URL htttt:t/tt/tiiiii org/ abs 02.305006665
Erik Lindholm, John Nickolls, Stuart Oberman, and John Montrym. Nvidia tesla: A unified graphics and computing architecture. IEEE Miero, 28(2):39-55 55, 2008
doi: N0.1109/MM.2008.0
Database Mart LLC. Rtx4090 vilm benchmark:
Best gpu for 1lms below 8b
on hugging face.
Blog post, 2025.
URL https: /www www. databasemart.
GBTVuk www.-aggggubeeenaaaa-ttt-------- oa http:///o...ooooooooo....o......
Accessed: 2025-09-24.
Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma. Qingwei Lin, and Daxin Jiang. Wizardeoder: Empowering code large language models with evol-instruct, 2023.
08568)
Michele Miranda, Elena Sofia Ruzzetti, Andrea Santilli, Fabio Massimo Zanzotto, Sébastien Bratières, and Emanuele Rodolà. Preserving privacy in large language models: A survey on current threats and solutions, 2025.
/ttp::///axx////aa http:/axxiv....gag./ abs ///////a///2///.. 2400. 05212
Daniel Nichols, Joshua H. Davis, Zhaojun Xie, Arjun Rajaram, and Abhinav Bhatele. Can large language models write parallel code?, 2024a.
Daniel Nichols, Aniruddha Marathe, Harshitha Menon, Todd Gamblin, and Abhinav Bhatele. Hpe- coder: Modeling parallel programs using large language models. ISC High Performance 2024 Research Paper Proceedings (39th International Conference), pp. 1-12, 2024b. doi: 10.23919/ ISC.2024. 10528929.
Daniel Nichols, Pranav Polasam, Harshitha Menon, Aniruddha Marathe, Todd Gamblin, and Abhi- nay Bhatele. aefformance-alignee Ilms for generating fast code, 2024c.
John Nickolls and William J. Dally. The gpu computing era. IEEE Micro, 30(2):56-69 2010. doi: 10.1109/NM.2010.411
NVIDIA Corporation. cuBLAS
index.htee 2023a. Version X.X, Accessed: [Insert Date).
NVIDIA Corporation. cuDNN: Deep neural network library. nttps://docs.nviin deepleaming/cudnn/ 2023b. Version X.X, Accessed: [Insert Date).
NVIDIA Corporation. cuFFT library. ltttps://occs nwidia. wtt.ddiaa..ooomcccccccttttttt/et
html 2023c. Version X.X, Accessed: [Insert Date].
NVIDIA Corporation. TensorRT: High performance deep learning inference. https://
developer nvi dia www/tensorrt.
Bervel/gerr.vlllaa..cc////eeeere 2023d. Version X.X, Accessed: [Insert Date).
13

---

Published as a conference paper at ICLR 2026
John D. Owens, Mike Houston, David Luebke, Simon Green, John E. Stone, and James C. Phillips. Gpu computing. Proceedings of the IEEE, 96(5):879- 899, 2008. doi: 10.1109/JPROC.2088. 917757.
XiPeng Qiu, TianXiang Sun, YiGe Xu, YunFan Shao, Ning Dai, and XuanJing Huang. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63 doi: 10.1007/s114310000--166 1007 101.:0000000111111 -011-16111131-2222-7--------
(10):1822-1897, September 2020. ISSN 1869-1900.
URL
http://dx: doi.
wwg.://.oo....r/.....ooo org.
Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. Halide: language and compiler for optimizing parallelism, locality, and recom- putation in image processing pipelines. SIGPLAN Not., 48(6): 530, June 2013. ISSN 0362- 1340. doi: 00.1145/24993702442177
URL https:/ /ww. http://p.or.r..111///../....////
2462178 76
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Ev- timov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. Code llama: Open foundation models for code, 2024.
URL BRRLhtts://aaet /arxivv org/ aataaass/200 12950
Hongjin Su, Shuyang Jiang, Yuhang Lai, Haoyuan Wu, Boao Shi, Che Liu, Qian Liu, and Tao Yu. Evor: Evolving retrieval for code generation, 2024. URL httpp:://axii...rrrrrtreteett
2402. 12317
Zhongxiang Sun, Xiaoxue Zang, Kai Zheng, Yang Song, Jun Xu, Xiao Zhang, Weijie Yu, Yang Song. and Han Li. Redeep: Detecting hallucination in retrieval-augmentee generation via mech- anistic interpretability, 2025. URL httpp/////a.....r./aabae.e.
Philippe Tillet, H. T. Kung, and David Cox. Triton: intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, MAPL 2019, pp. 10-19, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450367 196. doi: 11411 331 32555/31155..... 3329973
00.1145/3315508.333
URL ftttp:://oi..c
Haotian Wang. Han Zhao, Shuaiting Chen, Xiaoyu Tian, Sitong Zhao, Yunjie Ji, Yiping Peng, and Xiangang Li. Leveraging reasoning model answers to enhance non-reasoning model capability, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Lc, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. URL hatpps: :/arx/i/// org/ aaa.aaaaaaa/ 11903
Di Wu, Wasi Uddin Ahmad, Dejiao Zhang, Murali Krishna Ramanathan, and Xiaofei Ma. Re- poformer: selective retrieval for repository-Ievel code completion. In Proceedings of the s1st International Conference Machine Learning, ICML'22. JMLR.org, 2024.
Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shich. Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint 2xXiv:2405.00455 2024.
Chenwei Xiong, Cheng Liu, Huawei Li, and Xiaowei Li. Hispilot: Llm-based high-level synthe- sis. In Proceedings of the 43rd JEEE/ACM International Conference on Computer-Aided De- sign, ICCAD '24, New York, NY, USA. 2025, Association Computing Machinery. ISBN URL
9798400710773. doi: 10.114577676336337777111
1tt.://00....//0.111...... 1145/
3676536. 3676781
Biwei Yan, Kun Li, Minghui Xu, Yueyan Dong, Yue Zhang, Zhaochun Ren, and Xiuzhen Cheng. On protecting the data privacy of large language models (Imms): survey, 2024. URL https:
7/arxiva wwt.://gg/abaaaaeee 05156
14

---

Published as a conference paper at ICLR 2026
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei 7hang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. 2024.
Jiazheng Zhang, Cheng Liu, Long Cheng, Xiaowei Li, and Huawei Li. Understanding and mitigating errors of Ilm-generated rtl code. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2026. doi: ILL.0//TCAD.222..36661
Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, and Rui Wang. Unifying the perspectives of NLP and software engineering: A survey on language models for code. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https:
7 //openeeviw net / formmm hannddd--knnnnnnnan.
Qinkai Zheng. Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang. Yang Li, Teng Su, Zhilin Yang, and Jie Tang. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5673-5684. 2023.
Shuyan Zhou, Uri Alon, Sumit Agarwal, and Graham Neubig. Codebertscore: Evaluating code generation with pretrained models of code. 2023. URL http:p///aaaivi..r/////////0//0/ 05527
A CUDAEVAL CURATION
CPP compile verification
CPP progaam
speedup
Dependency
T
dd
correctness verification
CUDA compile verification
CUDD program
Figure 7: CUDAEval verification process.
Rule-based Preprocession The main goal of our filtering rules is reduce the cost of LLM api calling. To achieve this goal, we designed the following heuristic filtering rules:
Remove CUDA files that include local header files
Retain only files where code functions contain between 50 to 500 lines.
Filter out files containing dependencies on CUDA third-party libraries (including but not limited to cuDNN, cuBLAS).
Kernel Extraction and Dependency Completion After heuristic rule filtering, we employ prompts to instruct the LLM to extract CUDA kernels and their corresponding dependencies from the remain- ing files. Since these CUDA files were collected from repositories, the extracted kernels may still exhibit issues such missing macro definitions, absent class definitions, and similar deficiencies.
To address the aforementioned issues, we employ the LLM to attempt dependency completion for these kernels. The specific prompt used for this purpose is illustrated in the accompanying figure.
15

---

Published as a conference paper at ICLR 2026
Sequential Code and Driver Generation After kernel extraction and dependency completion, generate their corresponding serial codes based on the parallel codes and construct the main func- tions to call them respectively in preparation for the subsequent verification phase.
Verification Pipeline To maintain data correctness and improve quality, we implemented compre- hensive validation, specifically examining both accuracy and performance metrics. First, will compile and verify the two code segments separately. After confirming successful compilation, we execute both programs using the same test data and compare their outputs to validate correctness. Once correctness is ensured, we evaluate their runtime performance and select the code that demon- strates acceleration effects. The complete verification process illustrated in Figure
B REGRAPH CONSTRUCTION
LLM reasoning for CUDA Optimization trajectory As shown in Figure S. We instruct LLM to carry out CUDA optimizations stepwise in order to derive optimization trajectories using prompt However, LLM lacks the ability to verify either the correctness or the effectiveness of its own CUDA optimizations. As result, prior to merging the CUDA optimization trajectories into Re- Graph, we need to validate the results and performance of every optimization step. The verification approach follows the same methodology in Figure 7
prompt:
You an excellent high- performance computing engineer, skilled in optimizing CPP code using CUDA. Now, the user will provide you with CPP code, and you need to optimize it step by step using CUDA
think: The primary tasks include converting key loop structures into kernels and managing memory
think: Optimize memory patterns to reduce global memory operations. Observations reveal frequent global memory employ shared memory to cache intermediate
think: Optimize the sequential search fillkemaininnKerne by employing atomic operations for parallel vacancy filling ... replaced with parallel atomic operations.
allocation. -- emplaying thread layout parallelize element-wise processing.
code:
method:
method: Parallelism code:
c1/c2 data segments.
Atomic Operations
method:
code:
Shared Memory code:
Figure 8: Process of LLM reasoning for CUDA optimization.
Optimization trajectory relabel Following verification, it remains necessary to employ the LLM to re-annotate every step in the optimization trajectory according established CUDA optimization techniques. Specifically, using the prompt illustrated in G.4 we instruct the LLM to determine whether each optimization method employed in the current trajectory aligns with existing CUDA optimization methods. When a match is found, the corresponding optimization method is renamed accordingly.
C ANALYSIS OF REGRAPH DISTRIBUTION
Figure 9 illustrates the relationship between the distribution of ReGraph and the number of sam- ples used for its construction. Our experiments show that the reasoning graph converges after ap- proximately 500 samples. This is reasonable given that the space of effective CUDA optimization strategies is inherently limited. Therefore, ReGraph is able capture most of the commonly used optimizations using only a limited number of samples.
Moreover, further investigated the impact of ReGraph size on code generation quality by inte- grating different ReGraphs into Qwen2.5-Coder Booder--B-Inseeee Instructt, with 200 search budgets and rollout 10. According to Table 3 we observe that the performance continues to improve with increas-
16

---

Published as a conference paper at ICLR 2026
Dynamic Growth of Graph Nodes and Edges Throughout ReGraph Construction
400
Nodes
Idgee
100
Graph Size
130
50
200
400
600
800
1000
8Samples
Figure 9: ReGraph Distribution Across Different Sample Numbers.
ing ReGraph size before convergence, demonstrating the effectiveness of the CUDA optimization search space exposed by ReGraph. However, ReGraph has converged, its size longer has significant impact on performance.
Table 3: Ablation study the impact of ReGraph size code generation performance.
ReGraph ID
4
Graph Statistics
#Samples
50
100
300
500
600
1000
#Nodes
20
29
59
149
223
227
235
#Edges
25
38
87
225
384
388
399
Performance Metrics
@ 10
61.3
60.1
63.9
64,9
70.0
68.4
71.2
speedup@1 10
9.43
9.74
11.29
12.63
14.15
14.09
14.21
D ANALYSIS OF OVERHEAD FOR REGRAPHT
We provide both a theoretical analysis and empirical wall-time measurements for the overhead of the ReGraphT framework, encompassing ReGraph construction as well as the MCGS process. While ReGraph construction incurs LLM-related costs, we break down its three main stages, assuming N samples:
LLM Reasoning for Trajectories: Each sample requires single LLM invocation, resulting in total of N calls. With parallelism degree of C, the time complexity is O(N/C).
Relabeling Optimization Methods: This stage also involves N LLM calls, but they must be cuted sequentially, yielding a complexity O(N).
Merging into ReGraph: Each merge operation has complexity O(M), where M denotes the aver- age trajectory length, resulting in an overall complexity of O
When accounting for parallelism, the dominant cost becomes ONNM/C). Importantly, this repre- sents a one-time overhead, as the ReGraph can be preconstructed and reused. As illustrated in Figure 9 ReGraaph typically converges with approximately 500 samples, further enhancing cost efficiency.
17

---

Published as a conference paper at ICLR 2026
In addition to the asymptotical complexity analysis of ReGraph construction, also provide em- pirical measurements of the time overhead for ReGraph construction under different sample scales. Figure T0 illustrates the runtime required to construct ReGraph with 500 samples, as well as the overhead distribution across different components the construction process. 'he tests were con- ducted with parallelism level of 5, meaning that five parallel threads optimization trajectories. The results show that the time consumption scales approximately linearly with the size of ReGraph. On the other hand, analyzing the overhead distribution across compo- nents reveals that the main bottleneck in ReGraph construction lies in verifying the correctness of generated results, which accounts for 48% of the total runtime. In addition, the trajectory generation and relabeling steps, which require model-based generation, contribute 32% and 19% of the total overhead, respectively. In contrast, the cost associated with merging is negligible.
used extract CUDA
The Relationship Between the Seale ReCrapp the Runtime
Overhead Propertion for Component construction
merge
1.%
Verification
Timaho
19.0%
102
201
300
500
relnbel
#Samples
Figure 10: ReGraph Overhead.
ReGraphT models optimization as state transitions, making the search space grow linearly with the number of edges rather than nodes. A naive enumeration explores every edge with multiple attempts, yielding a time complexity of OCCE), where E is the number of edges and C is the number of attempts per node. Since converged ReGraph typically has hundreds of edges, this approach may require thousands of attempts. In contrast, MCGS distributes trials during the rollout phase, avoiding repeated edge attempts. Its time complexity O(Nb(d+()), where N is the number of iterations, is branching factor and d,/ represents the depth of selection and rollout respectively. Thus MCGS offers more efficient exploration of ReGraph search space.
Furthermore, we provide empirical data on the code generation overhead of ReGraphT. Inference tests were conducted on single A100-80GB GPU paired with an Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60 GHz, utilizing the Swee2.5-Coodr-7B-Inee a batch size of 16. Under these conditions, generating results for the 80 CUDAEval samples takes approximately 6.02 hours. Moreover, based on benchmark tests of various sub- 8B models running on a single RTX LLC comparable results can be achieved on consumer grade hardware in about 7.53 hours.
model, with a search budget of 100 and
E ABLATIONS ON MCGS TRAVERSAL
We conduct ablation study the varying number of rollouts and different reward strategies to explore the impact during MCGS traversal. For ReGraphT-MCGS, max attempts the same max rollouts. As shown in Table 4 under fixed search budgets and varying configurations, ReGraph-MCGS outperforms traversal-base methods, demonstrating both higher search efficiency and effectiveness. What's more, the maximum number of rollouts increases, ReGraphT-MCGS demonstrates sustained performance gains.
To investigate the effect of reward formulation, considered three different reward strategies:
18

---

Published as a conference paper at ICLR 2026
Table Performance Different Search Methods under Varying Budgets
Search Budgets
Search Methods
Max Attempts
Avg. Trajectory Length
pass@n
speedup@n
ReGraphT
3.1
64.9
7.91
10
2.3
63.9
7.76
100
ReGraphT-
2.5
64.9
8.13
MCGS
10
2.9
64.9
8.46
20
3.6
65.5
8.91
ReGraphTT
4.9
70.0
11.62
10
4.6
68.7
11.30
200
ReGraphT- MCGS
66.1
11.99
10
5.5
70.0
12.88
20
6.3
71.2
13.43
ReGraphT
7.4
68.7
12.89
10
70.0
12.45
300
ReGraphTT MCGS
7.8
69.6
13.01
10
8.7
72.5
14.76
20
9.1
74.1
14.98
strict reward The reward is defined as the average performance speedup of designs that pass all unit tests:
Ratrict
Σ p(d)
A pass (d, t)
pPass-all
dED
LET
where D plsssall denotes the subset of designs that successfully pass every test in T.
partial-credit reward Compared to strict reward, partial-credit reward does not require all unit tests be passed. Instead, it allocates rewards proportionally to the fraction of unit tests that successfully passed:
R Reprtial
1 EET
Σ
Σ p(d) (pass(d, t)],
m
De
dED
where D; is the set of designs passing test t.
rollout-based reward Similar to Baronio et al. (0025) ) rollout-based reward models the reward as a Markov decision process (MDP), setting the reward of given response the discounted sum of scores the current kernel and all subsequent ones and provides fine- one-grained feedback during generation:
R
Σ ('r(ss,
where r(st, 0z) denotes the performance gain associated with the design choice at step f, and y [0, is the discount factor.
Under different reward strategies, We conduct experiments using Qwen2.5-Codes Codee------nstruuee Instruct the SLM and ReGraph built from 500 samples Figure 9 The search budgets and varying rollout configurations are fixed to 200 and 10. As shown in Figure [1] we observe that strict reward and rollout-based reward have similar performance, while partial-credit reward leads to a slightly lower pass rate and speedup performance.
LIMITATIONS
During the construction of CUDAEval, we employed a multi-stage pipeline to carefully filter and select high-quality CUDA samples. While this process ensured the reliability and consistency of the benchmark, it resulted in the exclusion of a substantial portion the original dataset. Con- sequently, CUDAEval may not fully capture the diversity and complexity of real-world CUDA
19

---

Published as a conference paper at ICLR 2026
Performance Comparison Across Different Reward Strategies
100
16
80
1890
100
speodup.aaa
passiön
60
40
8
20
0
strict
partial-oredit
rollout- ioluut-based roward
strict
partial-credit
rollout- based roward
Figure 11: Ablation Study on Different Reward Strategies.
programs, particularly those with uncommon patterns, intricate dependency structures, or uncon- ventional optimization strategies. This selective filtering could limit the evaluation of models on edge cases or rare optimization scenarios, potentially underrepresenting certain challenging aspects of CUDA code generation and optimization. Future work may focus on incorporating broader variety samples to create a more comprehensive and representative benchmark.
G PROMPTS
G. PROMPT FOR KERNEL EXTRACTION
(a) Prompt for CUDA Kernel Extraction
CUDA kernel process prompt**
*+Roe
You are a professional high performance comput inpputing (HPC) engineer, skilled in optimizing C+ serial code using CUDA.
ARResponsibllityy.
You are supposed to extract the CUDA kernels from the given CUDA code file and identify the optimization techni used in them. If the provided CUDA code file contains multiple CUDA kernels, you should extract all them and for each them analyzc all optimizations used and corresponding code snippet.
**Response Format**:
1 'jsoon
"kernels":
"name": <extracted cuda kernel name>, "content"; rected cuda kernel content>
},
"opt ioptimization"..
"opt impizaiiiinn <the optimization methed used>, "snippet": <correspending code snippet>
]
20

---

Published as a conference paper at ICLR 2026
1.
#Precautions
1. You must only return the kernels that exist within this file, not those imported from other files and merely called here.
2. For each kernel, you must include its complete content without any omissions or abbreviated formatting.
3. Ensure that in the returned JSON content, the length of kernels matches the length of zations, meaning each kernel corresponds to list optimizations.
G.2 PROMPT FOR DEPENDENCY
(a) Prompt for CUDA Kernel Dependency Completion
You are an HPC engineer proficient in using CUDA. The CUDA kernel is extracted from the code file, so it may lack some relevant dependencies.
Now for the CUDA kernel provided the user, you need to determine whether this CUDA kernel lacks relevant dependencies.
1. lacks standard library dependencies, please supplement. them. 2. If it lacks user file dependencies, for example, user-defined classes, user-deiined functions, user-defined macros, etc., attempt to rewrite it in simple manner resolve the dependency issues.
Please return whether the rewrite was successful. If the rewrite is successful, return the rewritten code. If you are unable te rewrite the required user dependencies, return None for this item.
Note: that the user's s code where this kernel resides unavailable. Therefore, if you think some definitions are likely defined in the user's s code, you are also supposed to attompt to supplement them part of the rewritten code.
Prompt format
The user will provide you JSON dictionary in the follewing format:
"'json
"kernel" : <The CUDA kernel provided by user>
111
Response format
You will respond with JSON dicticnary in the following format:
''Json
"success": "<yes/no>",
"reason": "<Your reasoning process>",
"rewrite: "<The rewritten coce that doesn't lack relevant dependencies//onnee
21

---

Published as a conference paper at ICLR 2026
G.3 PROMPT FOR CUDA REASONING
(a) Prompt for CUDA Optimization Reasoning
You are an excellent pigh-performance computing engi neer, skilled in optimizing CPP code using CUDA. Now, the user will provide you with CPP code, and you neea to optimize it step step using CUDA.
Notes
1. Please optimize CUDA step by step. In each step of the optimization process, you need to provide the reasoning behind the optimization, explain the optimization methods used, and describe how these methods applied. Finally, provide the optimized code. Optimization methods refer to CUDA optimization techniques such as shared memory, warp divergence elimination etc. 'How the optimization methods are used' refers to how these CUDA optimizatien techniques are applied to optimize the code.
2. The optimization process should be returned as 1ist.
3. The function name must remain the same as the initial function after each optimizatinn step.
Prompt Format
The user will provide JSON dicticnary in the following format:
"Jjson
"kernel": "<The CPP code provided by user>",
}
Response Format
You should respond in the following JSON format:
''jjon
* "<The thought process for this optimization step>",
"method"; "<The optimization method used>", "detail"; "<How the optimizaiion methods are used>", "code": "<The optimizes code obtained in this step>"
11
G.4 PROMPT FOR RELABEL
(a) Prompt for CUDA Optimization Relabel
You are an excellent high-performance computing engincer, skilled in optimizing CPP code using CUDA. Now, the user will provide you with step-by-step optimization process for CPP code along with some existing CUDA optimization methods. You need determine whether each CUDA imtimizaiion method used this step-by-step process falls within the scope of the existing CUDA opt methods.
If the method used is part of the existing methods, renane it to the corresponcing method name from the existing ones; otherwise, keep the optimization method's s name unchanged.
22

---

Published as a conference paper at ICLR 2026
Notes
1. The user Input is a jeon dict incluing lists, 'metheds' represents the existing CUDA optimization methods, and 'prccess' represents the optimization process, where each item represents optimization step.
2. For each optimization step, you need to make judgment.
3. The CUDA optimization method used in each step is indicated in the 'method' field.
4. You should return list in JSON format, with the Length the input list.
Prompt Format
The will provide JSON dicticnary in the following format:
Jsoon
"mothods: [<CUDA optimization methods existed>),
"process":
"think": "<The thought process for this optimi cotimization zation step>",
"method": "<The optimization method used>", "detail"; "<How the optimization methods are used>", "code"; "<The optimized code obtained in this step>"
]
Response Format
You should respond the following JSON format:
j'on
"existed": "cyes/no>*,
"method"; "<If yes, the corresponding method nane from the existing methods; if no, keep the original methed name>"
11
G.5
PROMPT FOR STANDARD
(a) Prompt for Standard
You are an excellent bigh-pperformacce computing engineer, skilled in optimizing CPP code using CUDA. Now, the user will provide with CPP code, and you need to optimize it using CUDA.
Notes
1. You need to use CUDA to optimize the CPP code provided by user.
2. The optimized Function name needs to remain consistent with the original function. You need to handle the data transfer between host (CPU) memory and device (GPU) memory, as well as the invocation
CUDA kernels, within the function.
3. You must provide the complete code without any omissicns.
23

---

Published as a conference paper at ICLR 2026
Prompt Format
The user will provide JSON dicticnary in the following format:
'jjson
"kernel": "<The CPP code provided by user>",
}
Response Format
You should respond in the following JSON format:
Jjjoon
"think": "<The thought process for this optimizaticn>", "code": "<The optimized code using CUDA>"
G.6
PROMPT FOR CoT
(a) Prompt for CoT
You excellent high-performance computing engineer, skilled in optimizing CPP code using CUDA. Now, the user will provide you with CPP code, and you need to optimize it step by step using CUDA.
Notes
1. Please optimize CUDA step by step. each step of the aptimizaiion process, you need to provide the reasoning behind the optimization, explain the optimization methoas used, and descrice how these methods are applied. Finally, provide the optimized code. Optimization methods refer to CUDA optimization techniques such as shared memory, warp divergence elimination etc. 'How refers to how these CUDA optimization
optimization methods are used'
techniques are applied to optimize the code.
2. The optimization process should be returned as JSON list.
3. The function name must remain the same the initial function after each optimization step. You need
to handle the data transfer between host (CPU) memory and device (GPU) memory, as well as the invocation of CUDA kernels, within the function.
4. You must provide the complete code without any omissicns.
Prompt Format
The user will provide JSON dicticnary in the following format:
"'json
"kernel"; "<The CPP code provided by user>",
}
Response Format
You should respond in the following JSON format:
json
24

---

Published as a conference paper at ICLR 2026
"think"; "<The thought process for this optimi zatien step>", "method": "<The optimization method used>", "detail": "<How the optimization methods are used>", "code": "<The optimized code obtained in this step>"
G.7 PROMPT FOR CODERAG
(a) Prompt for CodeRAG
You are coaing expert that writes very fast code. You write parallel and C++
using CUDA and always strive to make the code fast as possible. The user will give you code and you will provide a modified version of the user's s code that as fast as possible using CUDA. At the same time, the user will also provide an optimization example, including the original program and the optimized program using CUDA. You refer this optimization example for your optimiztiinn.
Frompt format
The user will provide you JSON dictienary in the follewing format:
jsoson
"source_code" <lnitial code>, "example "example_orginaa"" <Example original program>, "example
optimized;; "example_optimizeed
<Example optimized program
Response format
You will respond with JSON dicticnary in the following format:
''json
"upcatea_code" <Optimized code>
1.1
G.8 PROMPT FOR REGRAPHT
(a) Prompt for ReGraphT
You are a coding expert that writes very fast code. You write parallel C and C++ code using CUDA and always strive to make the code as fast as possible. The user will give you code and you will provide a modified version of the user's code that is fast possible using
CUDA.
At the same time, the user will alse provide an optimi zation example,
25

---

Published as a conference paper at ICLR 2026
including opt imizaataio example consisted of the original program and the optimized program using CUDA, and the CUDA optimization method used.
This optimization example may not necessarily apply to the current code to be optimized, provided optimization method is suitable.
also need to determine whether the
Prompt format
The user will provide you JSON dicticnary the follewing format:
jjjon
"source_code" <Initial code>, exxmple"; {
"originn: <The original program in the optimization example>, roptmiized"; <The optimized program using CUDA in the optimization example>, "methoa"; <The CUDA optimization method used the optimization example>
1.1
Response format
You will respond with JSON dicticnary in the following format:
jsoon
"suitable"; <If the provided optimization method is suitable, yes/no>,
"optimization"; <The optimized code using CUDA>
H LLM USAGE
In preparing this manuscript, employed large language model (LLM) solely for grammar rection and stylistic polishing of the text. The LLM was not used for developing research ideas, designing methodologies, conducting experiments, or analyzing results. All scientific contributions, including problem formulation, theoretical analysis, experimental design, implementation, and eval- uation, were carried out entirely by the authors.
26


Reviews:
[{"review_id": "BrdN3ScWfL", "reviewer": "Reviewer_2bKR", "summary": "Summary\n\nThe paper proposes ReGraphT, a training‑free framework that lets small code LMs generate fast CUDA by borrowing multi‑step optimization “reasoning” from larger models. The authors first use an LLM to produce step‑by‑step CUDA optimization trajectories for many programs, then merge those steps into a CUDA Reasoning Graph (ReGraph) whose nodes are optimization techniques and whose edges are validated transitions accompanied by code examples. At inference, an SLM treats CUDA generation as graph traversal and uses Monte Carlo Graph Search (MCGS) with compilation, correctness checks, and runtime speedup as rewards to select an optimization path and produce code. Experiments on a new benchmark (CUDAEval) and on ParEval show that pairing ReGraphT with SLM backbones (DeepSeek‑Coder‑V2‑Lite‑Instruct, Qwen2.5‑Coder‑7B, HPC‑Coder‑V2) increases pass rates and roughly doubles speedup versus prompting and RAG baselines, narrowing the gap to strong LLMs. \n\nAccording to the *overview diagram on page 4*, ReGraphT has two phases: building ReGraph from LLM trajectories and exploring it with MCGS during generation. *Figure 5 on page 7* outlines the CUDAEval curation and verification pipeline, and *Table 1 on page 8* reports gains such as pass@1 moving into the low‑70% range and speedup@1 around 14× for SLMs with ReGraphT‑MCGS, compared with ~6–8× for standard prompting and RAG. The *difficulty analysis in Table 2 on page 9* shows larger benefits on medium and hard tasks, consistent with the method’s focus on multi‑step reasoning. \n\nContributions\n\n* ReGraphT framework: a training‑free method that transfers LLM‑derived optimization know‑how to SLMs via a reusable CUDA Reasoning Graph built from verified stepwise trajectories. \n* Graph‑based search for CUDA generation: formulation of CUDA optimization as state transitions on ReGraph and a tailored MCGS procedure with reward design that combines compilation success, functional correctness, and measured speedup. \n* CUDAEval benchmark: a curated set of 3,126 CUDA/CPU pairs with 313 held‑out evaluation tasks, stratified by reasoning complexity using trajectory length, plus a reproducible build/execute verification pipeline. \n* Empirical evidence: consistent improvements over standard prompting, CoT, and code‑similarity RAG on CUDAEval and ParEval; ablations that attribute gains to MCGS and to longer, more effective reasoning trajectories; analysis showing ReGraph converges after ~500 samples.", "questions": "1. P‑UCB on graphs with cycles\n\n   * Question: In Eq. (1) you use (N(s')) inside the log term. What is (s') when the same child has multiple parents, and how are visit counts disambiguated across different incoming edges? Please spell out the precise counters updated during selection/expansion/backprop when the structure is a directed cyclic graph rather than a tree. Suggestion: provide short pseudo‑code for selection/backprop on a general digraph and define all visitation statistics. A diagram keyed to *Figure 4 (page 6)* would prevent misimplementation. \n\n2. Rollout policy and hyperparameters\n\n   * Question: What values did you use for the rollout policy in Eq. (2) (λ, ϵ), step caps, and termination conditions, and how sensitive are results to these? Suggestion: add a small sensitivity sweep and report variance bars for pass@1 and speedup@1 to show robustness. Cite where these values appear in the code or Appendix. *Section 3.2 (page 6).* \n\n3. Reward shaping choice\n\n   * Question: You define the rollout’s final reward as the maximum reward along the trajectory (Eq. 3). Why prefer max over last‑state or discounted sums given the risk of favoring early “flashy” optimizations? Suggestion: include an ablation comparing max vs last‑state vs discounted‑sum reward, and report resulting trajectory lengths and speedups by difficulty tier. *Equation (3) and text on page 7.* \n\n4. What is “ReGraphT” vs “ReGraphT‑MCGS”?\n\n   * Question: In *Table 1 (page 8)* you evaluate “ReGraphT” and “ReGraphT‑MCGS,” but the text only briefly says ReGraphT does random sampling with max attempts 5. Please provide explicit pseudo‑code for the non‑MCGS traversal and clarify budgets, stopping rules, and how conflicts/inapplicable edges are handled. This will make the comparison interpretable. \n\n5. Failure handling during rollouts\n\n   * Question: A rollout “terminates if the optimization fails at any node.” Do you treat that as a hard terminal with negative reward only for that leaf, or do you propagate a penalty to the ancestor action? Suggestion: clarify how compilation/test failures affect backprop and whether you allow recovery by skipping the failed technique later in the path. *Section 3.2 (page 6).*", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Strengths by dimension\n\nOriginality\n\n* Introduces a training‑free way to transfer multi‑step optimization “know‑how” from an LLM to small code models by turning LLM‑generated optimization trajectories into a reusable CUDA Reasoning Graph and casting CUDA generation as graph traversal. This graph abstraction (nodes as optimization techniques, edges as validated transitions with examples) and the merge procedure are clearly new in the CUDA‑code LLM literature. The formal definition, Algorithm 1, and the relabeling routine that canonicalizes technique names make the idea concrete. \n* Adapts Monte Carlo Tree Search to a cyclic graph (MCGS) with an explicit P‑UCB selection rule, a rollout policy that avoids non‑termination on cycles, and a hierarchical reward mixing compilation success, test correctness, and measured speedup. Modeling CUDA optimization this way is a creative combination of known search ideas with domain‑specific constraints. The equations and rollout details on pages 6–7 are specific and principled. \n* Proposes CUDAEval, a benchmark built from real‑world CUDA files rather than synthetic sequential code, and stratifies tasks by reasoning complexity (trajectory length) rather than only algorithmic category. This reframes “difficulty” in code generation around reasoning depth, which is a fresh and useful lens. The curation and verification pipeline in Figure 5 and the easy/medium/hard tiers on page 7 ground this originality. \n\nQuality\n\n* Methodological rigor is high: the paper provides a full pipeline from ReGraph construction to MCGS exploration with precise definitions, pseudo‑code, and formulas (Definition 1, Algorithm 1, Figures 2–4). This makes the approach reproducible and falsifiable. \n* Evaluation is careful and multi‑faceted. Results are reported on two benchmarks (CUDAEval, ParEval) using three SLM backbones (DeepSeek‑Coder‑V2‑Lite‑Instruct, Qwen2.5‑Coder‑7B, HPC‑Coder‑V2) with consistent budgets and metrics (pass@k for correctness and speedup@k for performance). Table 1 shows substantial and consistent gains for ReGraphT‑MCGS over standard prompting, CoT, and a code‑similarity RAG baseline. \n* Analysis links mechanism to effect. Table 2 and Figure 6 demonstrate that gains grow with task difficulty and that longer reasoning chains correlate with better speedups until a saturation point, supporting the claim that the framework injects multi‑step reasoning into SLMs. \n* Solid ablations and practicality checks: the paper studies search budgets and rollout counts (Table 4), compares reward formulations (Figure 11), examines graph growth and convergence (~500 samples; Table 3 and Figure 9), and breaks down construction and inference overheads (Figure 10), including wall‑clock estimates on A100 and consumer GPUs. These details increase confidence the method is not a one‑off. \n\nClarity\n\n* The narrative is easy to follow: the overview (Figure 2) cleanly separates graph construction and graph‑guided generation; Algorithm 1 steps through merging trajectories; Figure 4 maps MCGS phases to the CUDA setting; Figure 5 explains data curation at a glance. The consistent use of page‑local figures and definitions reduces ambiguity. \n* Experimental reporting is structured and legible. Table 1 compares methods across models and benchmarks in a single view; Table 2 breaks down difficulty tiers; subsequent figures and tables isolate the impacts of graph size, reward design, and search parameters. The paper also explicitly states hardware, precision, budgets, and metrics, which helps replication. \n* The appendix includes prompt templates and verification workflows, which is unusually transparent for a systems‑plus‑LLM paper and lowers the barrier for re‑use. \n\nSignificance\n\n* The empirical gains are large enough to matter in practice. For example, on CUDAEval with DeepSeek‑Coder‑V2‑Lite‑Instruct, pass@1 rises from 61.7% (standard) to 75.1% and speedup@1 from 6.54× to 14.46×; similar boosts appear for Qwen2.5‑Coder‑7B and HPC‑Coder‑V2. On ParEval, pass@1 improves from 40.0% to 55.0% with speedup@1 from 4.61× to 10.78×. These are not marginal deltas; they more than halve the gap to strong LLMs while preserving the deployability of SLMs. \n* The framework’s training‑free nature and the reported convergence of ReGraph after a few hundred samples suggest a reusable artifact that can be shipped with SLMs to unlock privacy‑preserving, local CUDA generation. The one‑time cost and search efficiency analysis further support practical adoption. \n* By defining difficulty via reasoning rather than only algorithm class, CUDAEval may nudge future work to measure and improve multi‑step optimization ability explicitly. The positive trajectory‑length/performance correlation (Figure 6) gives the community a concrete target and a measurement recipe. \n* The abstraction is likely portable: encoding verified optimization trajectories as a graph and exploring with MCGS is a general recipe that could extend to other performance‑critical domains where correctness gating and speedups are measurable, not just CUDA. The paper’s discussion hints at this broader applicability. \n\nBottom line: This is a well‑executed systems contribution that combines a novel representation (reasoning graph), an effective search procedure (MCGS with domain‑aware rewards), and a carefully curated benchmark. The work meaningfully advances the practicality of small code models for GPU programming, with strong evidence and unusually transparent documentation.", "weaknesses": "1. Positioning vs prior search‑based reasoning is underdeveloped.\n   The paper adapts MCTS to a cyclic “reasoning graph,” but the case for novelty over existing search‑guided generation is thin. Closest neighbors include MCTS‑style reasoning for code and RAG (e.g., RethinkMCTS for code generation; MCTS‑RAG), and iterative search over program transformations in compiler auto‑scheduling (e.g., Halide, TVM). The paper cites these lines of work but does not empirically contrast against them nor articulate a clear theoretical advantage beyond domain specifics. Add a head‑to‑head with: (i) vanilla MCTS over thought tokens (no graph), (ii) MCTS‑RAG over a CUDA corpus, and (iii) TVM/Halide‑style schedule search baselines where applicable; report compute‑normalized outcomes. A small theory section clarifying why P‑UCB on graphs with cycles has better regret or exploration properties than tree‑MCTS in this setting would also help. \n\n2. Ambiguities in the MCGS definition and missing hyperparameters.\n   Equation (1) defines P‑UCB with (N(s')) but does not disambiguate “parent” counts when nodes have multiple parents or have been reached via different paths. The rollout policy in Eq. (2) introduces λ and ϵ without stating chosen values or sensitivity. Specify the state/action visit counters precisely for graphs, list default λ, ϵ, depth/rollout caps, and include a sweep showing robustness of performance to these choices. A brief pseudo‑code block for selection/expansion/backprop on a DAG with back‑edges would make reproduction safer. See Section 3.2 and Figure 4 on page 6. \n\n3. Reward shaping risks shallow plans.\n   Rollouts treat every node as terminal and take the max reward along a trajectory (page 7), which can bias search toward early “flashy” speedups and discourage deeper combinations. Add an ablation replacing the max with: last‑state reward, discounted sum, and penalized path length; report impacts on trajectory length and speedup (particularly on medium/hard tiers in Table 2, page 9). \n\n4. Graph construction and relabeling lack quality control.\n   The relabel step in Algorithm 1 (line 7, page 5) relies on an LLM to canonicalize method names, but there is no inter‑annotator or manual audit to ensure that, say, “avoid bank conflicts” doesn’t get misfiled under “memory coalescing.” Release the taxonomy of optimization methods and the mapping rules (Appendix G.4), plus a confusion matrix from a human audit of a random sample. Include a “no‑relabel” ablation to verify that gains are not an artifact of label collapse. \n\n5. Benchmark construction may inflate speedups and invites contamination.\n   CUDAEval is built by extracting real CUDA kernels, then having an LLM generate the CPU serial counterpart and drivers before verification (Figure 5, page 7; Appendix A). That CPU baseline is unlikely to be an expert‑tuned -O3 implementation and could exaggerate speedup ratios. In addition, the source corpus (Stack v2 CUDA) likely overlaps with pretraining data of the evaluated models. Provide:\n   • CPU baselines compiled with -O3 and simple algorithmic cleanups; report speedup vs both “LLM‑CPU” and “optimized CPU.”\n   • A near‑dedup analysis (e.g., MinHash/BLEU‑SimHash) between CUDAEval and known pretraining sources for the backbones, and a leakage‑reduced split.\n   • Results on hand‑written sequential baselines for a subset of tasks to calibrate speedup realism. \n\n6. Compute‑fairness and budget parity across baselines are unclear.\n   Table 1 (page 8) reports pass@1/10 and speedup@1/10 for Standard/CoT/RAG vs ReGraphT/MCGS, but only ReGraphT variants have an explicit search budget of 200. It is not stated how the 10 samples for the other baselines are produced, nor whether compile/run evaluation time is matched. Add a table of wall‑clock and GPU hours per method for CUDAEval and ParEval, and enforce parity in generation attempts and verification calls. The overhead analysis (Figure 10, page 17) is useful but only for ReGraphT. Mirror this for the baselines. \n\n7. RAG baseline is weak.\n   Using CodeBERTScore similarity as the sole RAG baseline underrepresents retrieval quality for code (structure‑aware and repository‑level retrieval matter). Since the paper cites Repoformer and EVOR, implement stronger RAG: repository‑aware retrieval, AST‑ or CFG‑conditioned retrieval, and multi‑hop chaining of examples. Also try “ReGraph as a retriever” without MCGS to isolate where the gains come from. See Baselines on page 8. \n\n8. Measurement methodology for speedups needs tightening.\n   Reward and metrics depend on single‑run timings susceptible to GPU warm‑up, clock variability, and input‑size sensitivity. Document: number of timing repetitions, warm‑up iterations, clock locking, and variance; report medians with IQR or 95% CIs. Consider integrating compute‑sanitizer checks for race conditions and misuses that pass unit tests but are undefined at scale. This directly affects the “hierarchical reward” in Eq. (3) and strengthens the correctness claim beyond output equality. \n\n9. Difficulty labeling is model‑dependent.\n   CUDAEval tiers are defined by trajectory length from DeepSeek‑R1 (page 7), which risks encoding that model’s biases about “what counts as a reasoning step.” Cross‑validate with human ratings on a sample and with a second LLM to show the tiers are not an artifact of a single annotator. Report correlations and disagreements. \n\n10. Generalization beyond CUDA remains a claim.\n    The paper argues ReGraphT is portable, but all experiments are CUDA only. A small transfer study to Triton or OpenCL kernels, or to a CPU vectorization domain (e.g., OpenMP) using the same framework, would substantiate the portability claim. Even a pilot with 30–50 tasks would be persuasive. \n\n11. Ablations miss two levers that likely matter.\n    The paper includes useful studies on rollout counts and reward strategies (Table 4 and Figure 11, pages 18–19) and on graph size saturation (Figure 9, page 16), but omits:\n    • Edge content ablation: do edges need embedded examples, or is a method‑only graph sufficient?\n    • Graph source ablation: build ReGraph from a different LLM or a smaller sample to test sensitivity to trajectory provider quality.\n    Add both to clarify which ingredients are essential. \n\n12. Reproducibility gaps.\n    Prompts are provided (Appendix G), which is excellent, but the paper should fix random seeds, publish exact model snapshots and decoding parameters for each backbone, and release the canonical method list and the final ReGraph artifact used in Table 1. The current anonymous repo link is helpful during review but not a substitute for an archival release. \n\n13. Practical deployment knobs are not exposed.\n    Section D reports that 313 CUDAEval tasks with Qwen‑7B and budget 100 take ~6.02 hours on an A100, and ~7.53 hours on a 4090, but practitioners need guidance to trade accuracy for latency. Provide curves of pass@1 and speedup@1 vs budget, and vs rollout count, plus a “fast mode” configuration that still beats RAG on medium/hard tasks (Table 2, page 9). \n\n---\n\nBottom line. The paper’s core idea is promising, but it would benefit from clearer algorithmic specification, stronger and compute‑fair baselines, contamination‑aware benchmarking, and tighter measurement discipline. Addressing the items above would make the claims about transferring “LLM‑level reasoning” to SLMs far more convincing and easier to adopt.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WZLVDrCEww", "reviewer": "Reviewer_CAtC", "summary": "This paper addresses the challenge of generating optimized CUDA code, a task at which Large Language Models (LLMs) excel but which is difficult for smaller, more efficient Small Language Models (SLMs) due to their limited reasoning capabilities. The authors propose ReGraphT, a novel, training-free framework designed to transfer the multi-step optimization reasoning of an LLM to an SLM.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Practical Relevance and Problem Significance: The paper tackles a real and important problem. Optimizing code for parallel architectures like GPUs is a critical bottleneck in high-performance computing. Making this capability accessible via smaller, locally deployable models has significant practical implications for developer productivity, code privacy, and computational cost. The training-free nature of the framework further enhances its practicality.\n- Thorough and Comprehensive Evaluation: The experimental setup is strong. The authors not only test their method on an existing benchmark (ParEval) but also contribute a new, well-motivated benchmark (CUDAEval) that allows for a fine-grained analysis of performance across different reasoning complexities. The inclusion of multiple modern SLMs and comparisons against several strong baselines (Standard, CoT, RAG) and SOTA LLMs provide a convincing demonstration of the method's effectiveness.\n- Strong Ablation Studies: The paper includes detailed analyses that strengthen its claims. The study on the impact of ReGraph size (Figure 9, Table 3) demonstrates the convergence of the knowledge graph, while the overhead analysis (Section D) provides a transparent look at the costs involved. The ablation on MCGS traversal strategies (Section E) further validates the design choices within the framework.", "weaknesses": "- Convern on generalizability of the reasoning graph: While the paper demonstrates that the graph's structure converges, its generalizability to out-of-distribution problems is not fully explored. The graph is built from a dataset of 10K CUDA files filtered down significantly. It is unclear how well a single, pre-constructed graph would perform on CUDA tasks from entirely different domains (e.g., scientific simulation vs. deep learning kernels) that might require novel optimization patterns not seen during construction. This raises questions about the \"one-time cost\" of construction if new graphs are needed for new domains.\n- Comparison of fine-tuning based approach: The framework is presented as \"training-free,\" which is a key advantage. However, a compelling comparison would be to use the LLM-generated trajectories as a dataset to fine-tune an SLM. This would be a direct test of whether the explicit graph structure and MCGS search are more effective for imparting reasoning than standard supervised fine-tuning on the same high-quality data. While the introduction argues SFT has limited effectiveness for reasoning, an empirical comparison would make this claim more robust.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MGMdHTGcIA", "reviewer": "Reviewer_aGxM", "summary": "This paper introduces ReGraphT, a training-free framework that transfers CUDA optimization reasoning from large to small language models using a structured CUDA Reasoning Graph and Monte Carlo Graph Search. It achieves near-LLM performance on CUDA code generation benchmarks while remaining lightweight and privacy-friendly for local deployment.", "questions": "- How scalable is the LLM-driven trajectory extraction process? Does building the reasoning graph require significant manual curation or prompt engineering to ensure consistency across optimization traces?\n- What is the computational overhead of the Monte Carlo Graph Search compared to standard RAG retrieval or CoT prompting?\n- Can ReGraphT transfer beyond CUDA (e.g., OpenCL, SYCL, or Triton code)?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "- method achieves near LLM performance\n- it allows for local deployment (I wonder how important this is for CUDA code generation?). It would be more sensitive for private conversations but in this context is not really needed, why not use a bigger model?", "weaknesses": "- Code link is empty\n- The paper content leans toward systems and engineering design, rather than core theoretical or algorithmic ML contributions. Therefore it doesnt seem appropriate for ICLR.\n- The reasoning transfer mechanism is algorithmic plumbing rather than a model innovation\n- Is more of an engineering project report", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aqnTn8uj32", "reviewer": "Reviewer_2LHc", "summary": "This paper introduces ReGraphT, a retrieval-augmented generation framework to transfer the CUDA programming reasoning from large language models (LLMs) to small language models (SLM). (1) It formulates the CUDA code generation task using LLMs as a graph-based problem, where this graph encodes the CUDA optimizations and (2) incorporates Monte Carlo Graph Search (MCGS) to guide the search to improve efficiency. \nThis paper also proposes a CUDA evaluation benchmark (CUDAEval) that evaluates LLM performance in CUDA code generations based on 10K CUDA files sampled from the real-world CUDA files.\nReGaphT with MCGS outperforms the CoT or RAG method regarding the correctness and the optimization performance of the generated code.", "questions": "* The paper aims to enable SLMs to achieve large-model-level CUDA code generation performance, but the proposed ReGraphT framework still depends on a LLM to construct the reasoning graph and generate the initial optimization trajectories. How do you justify the efficiency improvement?\n* How sensitive are results to rollout count N = 10 and search budget = 200?\n* Results in Tables 1 and 2 lack variance/error bars or p-values. Improvements of 1-2 % can be stochastic generation variance. Could you provide the variance of those numbers?\n* Is the reported speedup calculated as an average across all CUDA files, or as a geometric mean?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This paper transforms the CUDA optimization problem into a structured reasoning graph. This abstraction provides a clear, interpretable framework for modeling optimization dependencies and interactions\n* Introduce of the CUDAEval benchmark, a dedicated benchmark that classifies CUDA generation tasks", "weaknesses": "* The github link for the code is empty when the review period has started (at least by Oct 16)\n* Comparisons are limited to standard prompting, CoT, and generic RAG. No comparison against distillation or reinforcement-learning-based compression (e.g., DeepSeek-R1 distilled SLMs) is given.\n* The CUDAEval benchmark defines its difficulty levels based on the length of reasoning trajectories, but that depends on how the LLM behaves rather than on the actual complexity of the CUDA code itself.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)