Mastering Sparse CUDA Generation through Pretrained Models and Deep Reinforcement Learning
Authors: Yaoyu Wang
Hankun Dai
Zhidong Yang
Junmin Xiao
Guangming Tan
Venue: ICLR 2026 Oral (2026)
Keywords: Reinforcement Learning
CUDA Code Generation
High-Performance Computing
Primary area: reinforcement learning
TL;DR: We propose SparseRL, a deep reinforcement learning framework that generates high-performance CUDA code for sparse matrix operations, achieving significant improvements in both correctness and execution efficiency.

Abstract: Code generation is a crucial research area in the field of artificial intelligence, holding the potential to revolutionize software development and streamline programming processes. However, generating the high-performance code, which need to be executed in a shorter time for the low-latency scenario, remains a formidable challenge. Existing methods often struggle to account for the irregularity of input sparse data in sparse programs and the need for domain-specific architectural knowledge, leading to sub-optimal performance. To tackle these issues, we propose the SparseRL framework. SparseRL leverages deep reinforcement learning, treating a pre-trained language model as a stochastic policy. It takes the row and column indices of non-zero elements in the sparse matrix as input and generates CUDA code as output for sparse matrix operations. We also introduce a domain-specific code generation mechanism for the dynamic input, a sinusoidal embedding technique tailored for sparse matrices, and a hierarchical reward function that considers both code correctness and execution efficiency. Experimental results demonstrate SparseRL achieves state-of-the-art performance. In sparse matrix-vector multiplication (SpMV) tasks, it improves the compilation rate by 20% compared to existing methods, and the generated code runs 30% faster on average. For sparse matrix-dense matrix multiplication (SpMM) tasks, SparseRL also shows significant performance gains. These results highlight the effectiveness of SparseRL in generating high-performance CUDA code for sparse matrix operations.

Published as a conference paper at ICLR 2026
MASTERING SPARSE CUDA GENERATION THROUGH PRETRAINED MODELS AND DEEP REINFORCEMENT LEARNING
Yaoyu Wangli; Hankun D Zhidong Yang Junmin Xiao Guangming Tan
State Key Lab of Processors, Institute of Computing Technology. Chinese Academy of Sciences University of Chinese Academy of Sciences 'Hong Kong University of Science and Technology wangyaoyu2ls@ictt wr.icttcccc..m en, daihankunl 90ma ils www. ac..saaccoom cn, kezdyangeust. hk, fxiao junmin, (gm)@icct... ac.
ABSTRACT
Code generation is a crucial research area in the field of artificial intelligence, holding the potential revolutionize software development and streamline pro- gramming processes. However, generating the nigh-peffrrmanc code, which need to be executed in a shorter time for the low-latency scenario, remains a formidable challenge. Existing methods often struggle to account for the irregularity of in- put sparse data in sparse programs and the need for domain-specific architectural knowledge, leading sub-optimal performance. To tackle these issues, we propose the SparseRL. framework. Sp leverages deep reinforcement learning, treat- ing pre-trained language model stochastic policy. It takes the row and column indices of non-zero elements in the sparse matrix as input and generates CUDA code as output for sparse matrix operations. We also introduce domain-specitf code generation mechanism for the dynamic input, a sinusoidal embedding tech- nique tailored for sparse matrices, and hierarchical reward function that considers both code correctness and execution efficiency. Experimental results demonstrate SparseRL achieves state-of-the-ar performance. In sparse matrix-vector multi- plication (SpMV) tasks, it improves the compilation rate by 20% compared to existing methods, and the generated code runs 30% faster on average. For sparse -eeie matrix multiplication (SpMM) tasks, SparseRL also shows signifi- cant performance gains. These results highlight the effectiveness of SparseRL in generating nigh-performance CUDA code for sparse matrix operations.
INTRODUCTION
In recent years, code generation has witnessed a remarkable advance with the increasing adoption of deep learning and neural language models. Extensive research aims to generate a sequence of code as the output program according to various code-related targets such code completion (Li al.) 2017]
Wei et al. 2023). code translation (Zhu et al. 22022] Pan et al. 2023), and program synthesis (Li al. [022 et al. 2022). While these methods achieve promising results, we observe that they still fail to generate high- performance code, which needs to be executed in shorter time to meet low-latency scenario requirements. For example, in large language model (LLM) inference and graph neural network (GNN) computation, users expect to obtain results more quickly. However, manual optimization is very time-consuming (e.g., it takes several years from the emergence of transformer (Vaswani et al. 2017) to the appearance of Flash-attention (Xia et al. 2023)).
The difficulty of generating nigh-performance computing (HPC) code stems from two major factors. First, in the context of code generation by LLMs, the tasks often involve sparse programs, which differ from dense computations with fixed input patterns. For these sparse programs, execution patterns for data access and computation
dynamic and closely tied to the input sparse data (e.g., irregular and dynamic graph structures in GNNs) and can only be determined at runtime (Kim et al.) 2024 Sadman & Qasem| 2025). requiring customized implementations achieve minimal execution time
Equaal contribution. Our code is available on httpt::tt////tttt htttps://github.... http//.tttuu..oooo com/ /aimm/QiWu---aa QiWu-NCIC////OO/ NCIC/Sparsee: Corresponding author.

---

Published as a conference paper at ICLR 2026
for varying input sparse structures (Du et al. 2022 Li al.
2013). Second, effective performance
optimization demands Nomain-specifee architectral expertise. Many prior works (Liu & Vinter Naumov et al. 2010) have relied on manual knowledge with performance optimization, such
2013
as improving cache hit rates (Gómez et al., 2021 Yan et al. 2014) and load balance (Liu & Vinter
2013
Merrill & Garland 2016), to enhance program execution efficiency.
Among the various tasks requiring figh-performance code generation, sparse matrix- vector multipli- cation (SpMV) is fundamental and critical computational operation. It has been widely applied in pruned LLM inference (Li et al., 2024 Xu et al. (0224) and GNN computation (Yao et al. [023] Jia al. 2022). Due to the performance impact of the input sparse matrix structure in sparse programs, the goal is to generate high-perfone performance SpMV CUDA code for each dynamic input sparse matrix.
There has been numerous research (L.i et al. 2013 Yan et al. 2014 /hao et al., 2018) of SpMV on GPUs and there is no one-size-fits-a solution for implementing righ-performance SpMV code on GPUs (Zhao et al.) 2018: Su & Keutzer, 2012), because different types of sparse matrices require tailored implementations to achieve the optimal performance.
Additionally, the generated CUDA code for SpMV should guarantee two conditions. First, the generated code should be syntactically and functionally correct to pass compilation and deliver correct computational results. Second, must also be optimized for pegh-peeformaeee execution on the GPU to achieve shorter execution time. However, current methods face three major limitations.
First, current methods use the conventional supervised next-token prediction objective (Bengio et al. 2015 Ranzato al. 2015) to train the model, which maximizes the likelihood of the next ground-truth token. For example, in the pre- tre-training and fine-tuning of models, token-level matching objectives like cross-entropy loss (Papineni et al. 2002 Ren et al. 2020) are commonly employed. For generating high-performance SpMV CUDA code, this supervised approach is insufficient, because multiple semantically correct SpMV programs (ground (ground-truths) exist, but only a few can achieve the optimal performance for each sparse matrix.
Second, current methods ignore the crucial rewards about the execution efficiency of generated code. This directly impacts the code's ' quality measured by the execution time of code. Existing methods neglect this crucial reward during the model optimization and generation processes. Incorporating execution efficiency (time) into the learning objectives during model optimization could better align with the ultimate goal of generating semantically correct and nigh-perffrmance CUDA code.
Third, due to the impact between performance and input sparse data, a customized program is required for each sparse data. However, existing L.LMs face inherent challenges in narrowing the modality gap between input sparse data representations and CUDA code generation. This mismatch necessitates the adaptation to translate input sparse data into meaningful information for the model to generate customized program.
Sparse Matrix
Sparse Matrix
Dynamic input runtime
Coll 1
Policy Pretrained LM (actor)
Action
Solution Program1
Solution Program2
6600 vo Espro kamely,
I...gkbul.... www.eree.. komceL)
2.1
Values
nny - bloktiinx "slciikk dreattbuu
IHREAUS, PER PER,ADW 2
facat sum
int roms per b block breciii TTREAD,, PER, JOM)
Value Function
jnt strimwt) i or untiowwe 1*1
int bieee rewn, per
Critic
cim_ssun vakeeji "tpoo_ indedlii
State
Reward
(pow.oment comuusure
floot omuusum it for geti or pow.p) offett is or pojow 1....
offec throddds % THRRADS PER, 900,
Correctness/ / Time
Complier and Executtor
Environment
CUDA Code
Input
Metrix
Generate
Solution Program2
Sorution Programt
Output
Exccute
Execution Time/Efficiency
10.62
2199
Figure I: An example task (Right): Each task is defined by a sparse matrix, which contains row and column indices of non-zero elements. The expected output is an SpMV program to be checked for functional correctness and execution efficiency. The input sparse matrix is the dynamic input at runtime and closely related to execution time. A high-level overview of our Sp method for sparse CUDA code generation (Left): we treat pre-trained language model (LM) as a stochastic policy, code generation steps as actions, and rewards can be estimated based on the functional correctness and execution time of output programs from the compiler and executor (environment).
To address the above issues, we propose a novel deep reinforcement learning baaming--based SpMV CUDA code generation method called SparseRI. (Figure [], aiming generating nirh-performance and
2

---

Published as a conference paper at ICLR 2026
correct CUDA code. The SparseRL is built upon the idea of treating a pre- tre-trained LLM for code generation as a stochastic policy. The imput of the LLM in SparseRL is the row and column indices of non-zero elements in the sparse matrix and the output is the generated SpMV CUDA code. The process of CUDA code generation is regarded as a sequence of actions. Each action corresponds to the generation of token in the CUDA code. The utilization of DRL allows the model to learn from the non-differential metrics of the compiler and executor (environment). The environment checks the syntactic and functional correctness of the generated CUDA code and provides information about the execution time of the program, which is used to calculate rewards. The main contributions of SparseRL can be summarized as follows:
We propose a domain-speciffi code generation framework for the dynamic input of sparse matrix operators, which is based on DRL. and enables the feedback from the compiler and executor to optimize the code generation process.
We propose a row/column indices-basee sinusoidal embedding technique for non-zero elements in sparse matrices, which directly captures the dynamic input sparse matrix structure to adapt the execution patterns at runtime,
We devise novel hierarchical reward, which incorporates execution time and compilation success as runtime and compile-time information into the model to generate better code.
The experiments prove that our SparseRL achieves state-of-the the-art (SOTA) results on generating nigh-performance CUDA code compared with other methods. In fact, our method is not limited to the task of SpMV. We use it an example show how incorporate rich multimodal information (e.g. runtime information of the program) to design a more powerful expert. So as a verification, we extend our method to the sparse matrix-dense matrix multiplication (SpMM) task and also achieve significant improvement, which validates the generalization and transferability of our SparseRL.
2
RELATED WORK
2.1 PRE-TRAINED MODELS FOR CODE GENERATION
Pre-trained models for code generation have evolved with transformer-bast bassdd models like Code-
BERT (Feng et al. 2020). Then, large-scale autoregressive models such as AlphaCode (Li et al.
2022) have emerged to enhance the model's ability of natural-angguage-tte co-ddde translation and code completion. Retrieval-augmentte
models like CodeTT5 (Wang et al. (021)) and StarCoder (Li cct al. 2023) further refine task-specific objectives. Despite above advancements, challenges remain in handling code correctness and low-latency demand. These challenges inspire the research on reinforcement learning (Le et al. 2022 Shojaee et al., 2023) for improving the code quality. However, SparseRL encodes the sparse matrix generate nigh-performance CUDA code. This type of input modality is similar to AlphaFold (Jumper et al.] 2021). which also leverages a modality distinct from vision and text. Our work is also different from code implementation of algorithm problems and dense computation (elaborated in Appendix A.I). We also have a more direct engagement with contemporary research on LLM- generation of high--perfrrancee code Appendix A.664
2.2 ARTIFICIAL PROGRAMMING FOR SPMV ON GPUs
Sparse matrix and vector multiplication (SpMV) operation multiplies a sparse matrix by dense vector, and obtains a dense vector as the result. It is a fundamental operation that plays crucial role in wide range of numerical simulation applications (Shantharam et al. 2011 Abdelfattah al.
2015), and artificial intelligence domains (Wang et al. 2019 Mao et al. 2017). The challenge of designing SpMV on GPUs mainly arises from the highly diverse distribution of non-zero elements in sparse matrices (Li et al. 2013) Zhao et al. 2018). Different CUDA code needs to be implemented
for different sparse matrices achieve high perfoormroreee execution (shorter execution time), where examples are in Appendix A0
The history of optimizing SpMV has lasted for a long time (for example, CSR (Bell Garland 2008 + CSR5 (Liu & Vinter + DASP (Lu & Liu 2023) and has always been hot and popular topic (e.g., Eda-Adaptive (Greathouse & Daga 2014).ACSB (Ashari et al.) 2014). Merge-based (Merrill & Garland 2016) and etc. Over the past few decades, substantial efforts have been dedicated to enhancing the performance of SpMV on GPUs These ways of artificial programming for SpMV based observed matrix properties and are specifically devised for the characteristics of the underlying hardware. However, these methods are often manually crafted and lack the adaptability to handle diverse matrix types and changing hardware environments. Additionally, they do not leverage
3

---

Published as a conference paper at ICLR 2026
the emerging LLM techniques, which could potentially offer more flexible and intelligeent solutions for SpMV code implementations. We strengthen positioning against prior work in Appendix A.26
2.3 REINFORCHMENT LEARNING FOR SEQUENCE GENERATION
Reinforcement learning (RL) has been applied optimize non con-differentiable metrics in sequence generation tasks. For instance, the REINFORCE algorithm [Williams 1992) has been used to improve BLEU (Papineni et al. 2002) and ROUGE (Lin 2004) scores in translation and summarization models. More recently, InstructGPTT (Ouyang et al. (022) introduced a Reinforcement Learning from Human Feedback (RLHF) fine-tuning procedure. When it comes code generation, RL has also been employed to enhance the quality of generated code (Le et al. 2022 Dou et al., 2024).
Despite these efforts, existing RL-based methods for Sparse CUDA code generation have limitations. They struggle to generate high-performance code for low latency scenario, and leverage the complex relationship between input sparse data and execution patterns at runtime.
(a) Pre-training Stage
Sparse Matrix
(c) RL Stage
CUDA Code
Row 2 2 3
Augment
Col
1
Sinusoida Embedddig
Pretrained LM
Initialize
Initialize
Policy Model Pretrained
Action
(Actor) 0
Model
Finetuned LM
(0)
Generated
SpMV Program
Input
Output
Value Model
Pretrained
Execution
(Critic) Ø
Sparse
SpMV CUDA
(d)
Correct. Reward P
Matrix
program
Reward
Efficiency Reward Rottrinny
(b) SFT Stage
Backpropagation
Environment
Figure Overview of our method to optimize pretrained LMs for sparse CUDA code generation. (a) At pre-training stage, additional CUDA code is used to augment the LM. (b) At supervised fine- tuning (SFT) stage, the LM finetuned for the Sparse CUDA code generation. (c) At RL stage, the actor and critic networks are first initialized from the finetuned LM, and then updated based on the reward of RL. The reward function is composed of correctness and efficiency rewards.
3 SPARSERL
3.1 TASK DESCRIPTION
We take the SpMV task as sparse CUDA code generation example. Following sequence- to-sequence approach, the SpMV CUDA code generation task contains a sparse matrix as an input sequence X ((11,)),.* (rN, eN)) and output sequence of SpMV CUDA code -wn, ... (r), d. V. where r; is the row index and c; is the column index of the i-th non-zero element, and N is the total number of non-zero elements. The output of each decoding step is distribution over the vocabulary V. computed by the Softmax function ÿ coftmax(Linearr where fú is the contextual hidden state at decoding step
Conventionally during fine-tuning process, model parameters (0) are learned by maximizing the likelihood of the ground-truth target code. Let the set of all possible SpMV ground-truth code be Y (Y, Yg.... ... Y.J. where n represents the number of ground-truti code, and n 1. This indicates that there exists multiple CUDA code that can correctly perform SpMV operation. Denoting y (0) ( 9r) as the ground-truth code, the objective of supervised fine-tuning (SFT) stage is to minimize the cross-entron cross-entropy loss: (f) = E, log Po (f|X) - -,, (o where the conditional probability is parameterized following the above softmax function. The model generates sequences of CUDA SpMV code by autoregressively sampling token ÿ, from the distribution Po( (1 X). In the context of SpMV, the model is evaluated based on both correctness and efficiency of generated CUDA code. Each test includes an input and multiple ground- outputs of different execution efficiencies. Let the execution time of the code for sparse matrix X be E( X). Our goal is to generate the code Y that meets the following two conditions, which can be represented by the formulas:
4

---

Published as a conference paper at ICLR 2026
{
Compile(P) = True Conrect(Y,>>) True X S E(Y]))), VY; e
where y denotes all functionally correct SpMV CUDA programs (not limited pre-defined ground- truths). It should be noted that the execution time X is closely related to the sparse matrix X, because different sparse matrix input X require different CUDA code Y to achieve optimal execution efficiency.
To address this task, work contains the following essential modules shown in Figure 2 First, at pre-training stage, we augment the pre-training data with a large amount of CUDA code. By incorporating this domain-specific training data, the model can learn the specific programming paradigms and optimization techniques for CUDA-based sparse code. Second, during SFT and RL stages, we replace the input the language model with the row and column indices of each non-zero element of the sparse matrix, and output the regh-performance CUDA code for SpMV, so the model can learn how to adapt the code generation process based on the distribution of non-zero elements from the input sparse matrix. Third, at RL stage, we further use RL to optimize the model and design a novel reward for execution efficiency, which aims at fine-tuning the pre-trained model to better suit the SpMV CUDA code generation task.
3.2 PRETRAINING AND FINE-TUNING LANGUAGE MODELS
Pre-training stage with CUDA code augmentation. We augment the pre-training data with large amount of CUDA code. because the CUDA code involves unique requirements such as parallel computing. memory management on GPUs, and optimized algorithms for specific hardware architectures. We follow the pre-training setup in CodeT5 (Wang et al., 2023a; 202T), which has demonstrated remarkable capabilities in handling programming languages. The augmented (CUDA code) dataset includes hand-crafted gigh-performanc SpMV code (Liu Vinter, 2015| Zhao et al., 2018 Du et al. 2022, Merrill & Garland 2016) and other CUDA programming languages GitHub by reserving only permissively licensed code ("mit", "apache-2", "bsd-3-ccluse",, "bsd-2-clause", "cc0-1.0" "unlicense", "isc").
SFT and RL stage with sparse matrix embedding. Different from previous LLMs typically relying on word embeddings and positional encodings, we devise a sparse matrix embedding strategy to input the sparse matrix to the model. We utilize the row and column indices of the non-zero elements in the sparse matrices during the fine-tuning process. Let the sparse matrix be with dimensions m x and denote the set non-zero elements' row-column indices as X {(ri, ] where r; is the row index and e is the column index of the i-th non-7ero element, and N is the total number of non-zero elements. We leverage the sinusoidal embedding (Vaswani et al. 2017 to normalize the input row and column indices (theoretical foundation is in Appendix A.2 and ablation study is in Section 5.2). which is formulated as:
(ind))))) = sin (ind/100003)/dmnat)) , EE cos (1000(1000))))dmm
where ind represents the index value (either the row or column index in our SparseRL), j is the dimension index within the embedding vector, and dmodel is the embedding vector's 's dimension.
For the row indices, we calculate the sinusoidal embedding vectors. Given a row index the sinusoidal-encoded row vector e,, of dimension dmodel is obtained by applying the above formula 2 with ind = Similarly. for the column index c, the amusoidal-encoded, column vector e., of dimension dmode! is calculated using ind = C,. We then concatenate the sinusoidal-encoder row and column vectors for each non-zero element. That is, for the i-th non-zero element, we form a vector e, = =e, These vectors X' = {e: J.M are used as the input to the model. During the SFT and RL stages, the language input prompt is removed and only sparse matrices are provided as input. The target output is the SpMV CUDA program. For converting the modality of input sparse matrix to language modality, appropriate dimension. This strategy enables the model to directly capture the structural information of the sparse matrix from the non-zero element indices.
additional linear layer is applied to map the concatenated vectors to the
5

---

Published as a conference paper at ICLR 2026
3.3 RL PROBLEM FORMULATION
The code generation procedure can be formulated as sequential discrete finite-horizer Markov decision process (MDP) with the use of RL in which an agent interacts with the environment over discrete steps T, and employ the PPO algorithm (Schulman al. 2017) training details are in Appendix A.5). In SparseRI., the RI. problem formulated as follows:
State S: The state at time t is 8, (ýn:t-1: X) (st E S), determined by the sparse matrix X (containing row/column indices of non-zero elements) and the partial code sequence Ø1æ-1 generated before providing context for next token generation.
Action A:
The action at time 1 is dz = ÿr (a, E A), corresponding to the generated CUDA code token. Each token in the vocabulary (yr V) represents a potential action.
Policy ne (a (a ): A stochastic policy network (parameterized by 0, initialized from the SFT-finetuned model) predicts the next token based on 01:t-1 and X. The action a selected via top-k sampling, yielding the full SpMV CUDA code Y (...... .r) episode end T.
3.4
REWARD FUNCTION
The final reward Ranai (Ý, X) is calculated at the end of the code generation episode, integrating the generated code's syntactie correctness, execution efficiency, and memory usage.
3.4.1
CORRECTNESS REWARD
The correctness reward (Renreetness)) follows hierarchical structure, combining compilation and test results:
Compilation reward (Rcompit): A reward of +0.5 is given if the generated CUDA code compiles successfully; otherwise, a reward of -0.5 is assigned.
Test reward (Bes)): This reward is only considered if compilation succeeds. A reward of +0.5 is granted if the compiled code passes SpMV functional tests; otherwise, -0.5 is assigned.
Total correctness reward:
The sum of the compilation reward and (if compilation succeeds) the test reward, formally expressed as Reonreeness = Reompile - Icompile Riests where Leompile is an indicator function that equals for successful compilation and 0 otherwise.
3.4.2 EXECUTION EFFICIENCY REWARD
The efficiency reward (Refficiency) is activated only when the code is both compilable and functionally correct. It calculated as a scaled value of the performance improvement over baseline (cuSPARSE
library), defined Reficiency Feff [(Y .X
(X)
Leaa where tos (X) is the baseline execution
time (measured by cuSPARSE Naumov et al. 2010) library provided by NVIDIA), and t(Ý; X) is
the execution time of the generated code Y for the given sparse matrix X, and Test is an indicator function that is if all tests pass and 0 otherwise. Tetr is scaling factor that determines the importance of the execution efficiency in the overall reward. We use the average execution time of 1000 iterations to prevent fluctuations. The normalization details of Roff are in Appendix A.4.2
3.4.3 OVERALL REWARD
The final reward integrates correctness, efficiency, and memory penalty: Rasai (Ý, XX) Reorrectness Retteiinny - Tpenalty Amemory. Here, liemory an indicator function (I if the code exceeds predefined memory limit, 0 otherwise). and Tpeaalyy the penalty for excessive memory usage (Appendix AA25).. Sensitivity study of Fetr and Fpeaatty in reward hyperparameters is in Appendix A.4.I and the balance of Retheiecyy and Reoreetness is in Appendix A.4.3
3.5 OBJECTIVE
Our objective is to find an optimal set model parameters such that the generated program for a given matrix can meet the requirements of correctness and efficiency. During the fine-tuning process, we first use supervised fine-tuning (SFT), and then use RL to optimize the network model. addition, for early stopping the generation of error code during the generation process, integrate a dynamic syntax correctness verification mechanism during the decoding process and use the code extraction tool similar to the public QwenI.M repository (Appendix A.3

---

Published as a conference paper at ICLR 2026
4 EXPERIMENT
4.1 EXPERIMENTAL. SETTINGS
We use two machines, and each machine is equipped with 8 Tesla V100/A 100 GPUs over a duration of five days. The Adam optimizer (Kingma & Ba 2014) is employed, configured with a learning rate of 1 X 10-4, The experiment of balance between exploration and exploitation is in Appendix A.9
Benchmarks. We compare SparseRI. with two types of works. (1) Code Generation: (i) Qwen3 (ui
ct al. 2024; Yang et al. 2025). (ii) DeepSeek-RI (Guo et al. (022). (ii) CodeRL (Le ct al. 20ZZ].
(iv) PPOCoder (Shojace et al. 2023). (2) Artificial Library: (i) cuSPARSE (v12.1) library (Naumov
et al. 2010), which is provided by NVIDIA. (ii) TVM-S (Chen et al. 2018) is the sparse version of
TVM. (3) Closed-source models: GPT--o3-pro, GPT-5, and etc. (4) The ablation study is in section 5
(5) Experriissts of more direct engagement with contemporary research on LLM-based generation of
high-performance code in Appendix A.6.4 (6)Comparison with human expert implementations is in Appendix 19
Evaluation metries. (1) To evaluate the correctness of generated code, the passāk metric (Chen et al. 2021) is employed, which calculates the percentage of matrices for which SpMV compulation correct using & (beam search) synthetically generated program samples per matrix. Besides, we also use Compilation Rate (CR) (Kulal et al., 2019) under k 1000 (Appendix [A66 that shows the success rate of compilation among generated code. (2) The execution efficiency of the generated
SpMV CUDA code is compared with on test matrices using GFLOPS -oga Floating Point Operations Per Second, higher is better). The platform is the NVIDIA Tesla V100 and A100, based on the Volta and Ampere architecture. Single-precisioe is used and the CUDA environment is version 12.1. (3) The human readability test is in Appendix A.20
Datasets. Firstly, we collect the 1,100 matrices with the highest number of non zero elements from the SuiteSparse Matrix Collection (Davis & Hu, 2011]. These matrices span various application domains, and represent a diverse range of sparsity patterns. We divide these matrices into a training set of 700 matrices and a testing set of 400 matrices by the number of non-zero elements. Secondly, we conduct further evaluations on the DLMC (Deep Learning Matrix Collection) (Gale et al. 2020) 2019), which includes sparse matrices used in deep learning. When the number of non-zero elements is excessively large, we apply sampling to the input non-zero elements. The details of the sparse matrix dataset are in Appendix [A.21] The results of DLMC are elaborated in Appendix A...5 The experiments of scalability with sparse matrix features are in Appendix A.14
4.2 RESULTS ON SPMV
Compilation rate and correct functionality. Table I shows the results of three tasks (including extensions on SpMM in Section 4.3) on 400 test matrices, where the SparseRL with the CodeT5 model outperforms many pretrained LMs of much larger sizes. Despite being applied to the SpMV task, the inherent complexity of the CUDA code results in the generated code having suboptimal quality. The improvement is due to our fine-tuning for specific SpMV CUDA code generation. Specifically, our method leads with pass@ 1000 of 49.25 and a CR of 57.50 on SpMV. CR is higher than pass @ 1000 because code might compile successfully (high CR), but still have logical errors, thus failing to achieve correct functionality (lower pass @ 1000). Experiments of more baselines are in Appendix A.6 Pass@ 1000 is limited by our experimental environment, so the results are marked by 1-. Besides, we observe that although the closed-souree model generates code with higher accuracy than our method, there still gap in performance (Appendix A.6.2 and Table 77)
Performance on SuiteSparse Matrix Collection. The CodeRL and PPOCoder (based on T5 770M under k = 5000 on correct programs of partial matrices) are selected to compare the perfor- mance with SparseRL (based on (wen2.5-14BB and the artificial libraries in Figure 3 because the performance results of CodeRL. and PPOCoder are better than those of other foundation models (such as Llama/Qwen). The achieved performance is represented in bin box plot, where the horizontal axis denotes the number of non-zero elements in matrices, and the vertical axis measures performance in GFLOPS. The results demonstrate that SparseRL outperforms other methods across the majority of the matrices. We can observe that: (1) On average, SparseRL achieves performance improvements of 3.27 3.42x on V100 (3.50x, 3.29X on A100) over CodeRL. PPOCoder. (2) Notably, on average, SparseRL achieves performance improvements of 1.42x, 1.82 on V100 (1.44x, 1,86x on A100) over CuSPARSE, TVM-S. In addition, further performance analysis is elaborated in Appendix A. 11
7

---

Published as a conference paper at ICLR 2026
Table 1: Correct functionality (pass@k) and Compilation Rates (CR) under A 1000.
SpMV
SpMM (sol=3)
SpMM (col=32)
Model
Size
1usee11 1 Pi peaset 1000 CR przst3]
posstll pase 1000
CR passe1 I passill p0sa651000 CR
Qwen3
8B
8.00
12.50
6.75
12.00
7.00
10.75
DeepSeek
671B
15,00
22.50
16.50
20.50
15.50
22.75
SeeySeesk-I--istiii
7B
9.50
16.50
10.00
16.75
10:50
16.25
Qwen
CodeTS
770M
4.75
7:50
30.25
38.00
1.75
2.25
28.00
36.00
2,00
2.25
30,00
36.25
CodeRL+CoddeT
770M
5.25
8.50
36.50
39.30
2.50
5.25
30.50
40.50
2.50
5.50
32.50
39.75
PPOCodo++CodTTS
770M
5.75
10.00
5.00
45.50
3.50
36.00
46.75
GPT--33-po
25.25
32.75
24.25
31.50
25.00
32,00
GPi-o--mini
23.50
30.00
22.00
28.25
21.25
27.25
GPT-5
27.00
Claude-sonnet-4
28.25
36.75
24.25
31.50
26.00
31.50
SparseLL:
770M
9.25
15.75
48.75
56.50
9.00
15.00
45.25
56.75
8.25
14.75
47.00
57.25
SpuneRL=+Qwnn2.5
7B
9.75
57.00
46.75
SparscRL=Qwen2.5
14B
10.25
16.50
49.25
57.50
10.25
16.00
47.50
58.75
9.25
15.25
47.75
59.00
SparseRI .I .Qwen3
14B
16.25
21.00
18.25
21.00
18.25
22.25
SpurseL++GLM
15.50
19.75
16.00
SparseRl . DeepSeck
-Coder
6.7B
17.75
20.25
17.50
21.00
18.75
22.25
CodeRL
PPOCoder
TVN-5
cUSPARSE
SparseRL
400
300
100
200
50
100
4.9.5.5
5.5-6.1
6.1.6.7
(a)
6.7-7.3
7.9-8.5
logio(nnz)
5.5-6.1
6.1-6.7
(b)
6.7-7.3
7.3.7.9 7.9-8.5
Figure 3: Performance (GFLOPs) of SpMV across SuiteSparse Matrices on (a) V100 (b) A100 GPUs.
4.3 EXTENSIONS ON SPMM
We extend our method to another sparse CUDA code task of sparse matrix-dense matrix multiplication (SpMM). SpMM is another crucial operation in figh-performance computing, which has been applied
to GNN (Wang et al. 2023b Huang et al. (0200) and pruning LLMs (Xia et al. 2023; Fan et al. 2024 It multiplies an M x K sparse matrix A and a K N dense matrix B to output M N dense matrix C (i.e., C = AB). Similar to SpMV, implementing high-performant challenging due to the complex nature of sparse matrices. The platform is the NVIDIA Tesla A 100 and single-precision is used. Execution efficiency is compared using TFLOPS (10 Floating Point Operations Per Second). In addition, we adjust the number of columns of the dense input matrix B, as it also affects execution time. The baseline for comparison is the same as SpMV. The differences of performance optimization characteristics between SpMV and SpMM are Appendix A..13
SpMM on GPUs is
Compilation rate, correct functionality and achieved performance on SuiteSparse matrix. Table sshows that the SparseRL with the CodeT5 model can achieve sigmificant performance gains on SpMM tasks. illlustrats the performance of partial matrices across various methods. The achieved performance represented in a bin box plot, where the horizontal axis denotes the number of non-zero elements in the matrices, and the vertical axis measures performance in TFLOPS. The results demonstrate that Sp consistently outperforms other methods across the majority of the matrices analyzed. We can observe that: (1) Notably, SparseRL achieves the average speedup of 6.80/4.50x over CodeRL at 8/32 column, 6.60/4.88 x on A100 over PPOCoder at 8/32 column. (2) SparseRL achieves performance improvements of average 2.32/1.22. on A100 over Sputnik at 8/32 column, 6.39/4.38x over CuSPARSE at 8/32 column.
5 ABLATION STUDY
5.1
ABLATION STUDY OF PRE-TRRINIIG/SFTT/RL PIIASES
Table 2 presents direct ablation comparison between the three phases (Pre-training/ SFT/ RL) of SparseRL. The results clearly show that (1) Pretaining phase is critical for generating valid CUDA code, where CUDA-specific pre-training equips the model with parallel computing and GPU memory
8

---

Published as a conference paper at ICLR 2026
TVW-5
SparseRL Column 32
Column
1.75
1.50
1.25
1.00
0.75
0.50
4.9-5.5
5.5-6.1
6.1-6.7
(a) 6.7.7.3
7.3-7.9
7.9-8.5
4.9-5.5
5.5-6.1
(b) 6.7-7.3 7.3-7:9 7.9.8.5
(b)
logio(nnz)
Figure 4: Performance (TFLOPs) of SpMM across SuiteSparse Matrices with (a) column
(b) column = 32 on A100 GPUs. Column is the number of input matrix columns.
Table 2: Ablation comparison between the three phases (Pre-training/SFT/RL)
Pretrain PPO
No Pretrain
Metric
SFT
SFT PPO
SFT
PPO
SFT PPO
SpMV pass 1000
32.75
15.25
49.25
25.50
12.00
40.75
SpMV Compilation Rate
41.25
22.50
57.50
37.25
13.25
53.50
Average Execution Speed (GFLOPS)
89.25
50.36
116.20
70.32
30.83
95.22
management knowledge. (2) SFT phase is the fundamental stage of post training for the entire model. Especially in the SFT stage, we performed modality transformation (input sparse matrix, output code), which is important for the subsequent output of the model. If we only do the RL stage, difficult to obtain the correct code, and even code. (3) RL phase, driven by the hierarchical reward function (incorporating efficiency and memory constraints), is also important and significantly enhances both correctness and performance beyond the SFT baseline. The three phases together contributed to the highest performance and accuracy of the code, indicating that both stages are necessary.
difficult obtain nigh-performance correct
5.2 ABLATION STUDY OF SPARSE MATRIX EMBEDDING
In this ablation study, we focus on understanding how these different sparse matrix embedding strategies affect the performance of SparseRL in terms of correct functionality and compilation rates for SpMV and SpMM tasks. Table 3 presents the results of this study for models with different embedding techniques under to 1000. The Raw approach simply feeds the original numerical values of the indices into the model, without any transformation. The Max-Min normalization technique scales the values of the indices the range between 0 and 1, using the maximum and minimum values within the dataset for normalization. These results clearly demonstrate that the sinusoidal embedding the most effective among the three strategies for enabling the model to generate high-quality code for sparse matrix operations. More experiments of sparse matrix embedding are in Appendix A.8
Table 3: Ablation study of sparse matrix embedding correct functionality (pass@k) and Compila- tion Rates (CR) under = 1000.
Sparse Matrix Model Emhedding
SpMV
SpMM (col=8)
Size
passūi
pass16
pass@1000
CR
passui
pass95
1ass01100
CR
Raw
SpurseRL+CodeI5 770M
5.25
10.50
40.75
49.50
5,00
12.00
38.50
48.75
Max-Min
SparscRL+CodeITS 770M
8.25
14.00
43.25
51.75
7.00
13.50
40.00
52.75
Sinusoidal
SparseRL+CodeTS 770M
9.25
15.75
48.75
56.50
9.00
15.00
45.25
56.75
5.3 ABLATION STUDY OF REWARD COMPONENTS
We aim to understand the impact of different components in our SparseRL method on the performance of SpMV. Figure 5 presents the results of our ablation studies on two different GPU architectures. We compare four approaches: (i) cuSPARSE, which is the NVIDIA library for sparse matrix operations; (ii) Sp (base) means the supervised fine-tuning (SFT); (iii Opl represents the utilization of RL. with only the correctness reward; (iv) Op2 represents the addition of execution efficiency reward.

---

Published as a conference paper at ICLR 2026
cuSPARSE
Sparsoki (baza)
SpamenL (bsre
Spamsll (bate-opkeep))
140
250
1.2
100
nemeth22 tank helm2d03
(a)
0o2010
Matrix
nemeth22
tank helm2d03 net150
ga2010
Figure S: Ablation Studies on (a) V100 and (b) A100.
In Figure we observe that the basic SparseRL (base) achieves similar performance with cuSPARSE in most cases. When we add the RI method to form SparseRI. (buse+opl), the performance further improves for matrices like water_tank and helm2d03, showing the positive impact of this specific optimization. Moreover, the SparseRL (base+opl +op2) variant achieves the highest GFLOPS in several cases, such as nemeth22 and ga2010, demonstrating that combining execution efficiency reward can lead to even better performance. Overall, these ablation study results clearly show that each component in our SparseRL. method contributes to improving the performance.
5.4 ABLATION STUDY OF RL ALGORITHM CHOICES
We actually tried PPO, GRPO, and Reinforce++, but found that PPO's performance was already good enough, so we chose PPO. For specific details, please refer to the following comparison: (1) GRPO's gradient regulation mechanism reduces training instability but increases computational overhead by 12% (training time extends from 5 to 5.6 days on 8 GPUs). (2) GRPO achieves 48.9 pass@ 1000 and 113.5 GFLOPS, which are 0.7% and 2.3% lower than PPO, respectively.
(3) The innovation of our method primarily lies in the process, and our approach is robust to the selection of reinforcement learning algorithms
The experimental results on the same training/test split are: (1) GRPO: Achieves similar correctness (pass@ 1000: 48.9 vs. SparseRL's 49.23 on SpMV) but has 12% higher training overhead due to more complex gradient regulation. Execution speed is 2. % lower than SparseRL. (Due the randomness of reinforcement learning itself, test results may fluctuate, and GRPO also occasionally surpasses PPO). (2) Reinforce++: Shows 3.5% lower correctness (pass @ 1000: 47.6) and 5.3% slower execution speed compared to SparseRL on SpMV. This is because Reinforce++ lacks PPO's clipped objective, leading to unstable training when balancing code correctness and efficiency.
Additionally, for our framework, PPO can be replaced with other state-of-the-art reinforcement learning algorithms, which is only a part of the pipeline in our method. As demonstrated in the experiment, although there may be fluctuations in performance, it does not affect our performance improvement.
CONCLUSION
Discussions. While SparseRL is currently optimized for sparse matrix operations, its core design can be extended to other general code optimization tasks. We outline the extension potential of our framework: (1) Input representation: Replace sparse matrix indices with task-specific structural features (add some other "multi-model adapters" to capture information from non-matrix input. For example, use GNN to extract feature from program dependency graph
UniXcoder (Guo et al.) 2022MGraphCodeBBRY (Guo et al. 2021) as code embedding techniques). (2) Reward function: Adapt the hierarchical reward task-speeific metrics (c.g., loop execution time, parallelization speedup, memory bandwidth utilization). (3) RL pipeline: Reuse the -SStrann-SFT-MS workflow with task-specific training data.
Limitations. Despite the promising results, SparseRL has several failure cases (Appendix A.15 and limitations. First, the RL-based optimization process is computationally expensive during tun-tuning (Appendix 16. because of the interaction with the compiler and executor to obtain correctness and execution time feedback. Second, since both the time required to generate Sparse CUDA code and the execution time contribute to the overall overhead in real-world applications, method is particularly well-suited for scenarios where sparse code can be reused repeatedly (Appendix 17). Third, the extension to other barkeede-backends is elaborated in Appendix A.18
A.18
Conclusion. We propose the SparseRL method to address the challenges in generating high- performance CUDA code and apply it to sparse matrix operation tasks. By integrating deep re- inforcement learning and are-trained models, SparseRL has demonstrated significant improvements in both the correctness and execution efficiency of generated sparse CUDA code.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors sincerely thank anonymous reviewers for their insightful suggestions. The work is supported by National Natural Science Foundation of China, under Grant No. T2125013, 62032023, 62172391.
REFERENCES
Ahmad Abdelfattah, Hatem Ltaief, and David Keyes. High performance multi-guu spmv for multi- component pde-based applications. In Euro-Par 2015: Parallel Processing: 21st International Conference on Parallel and Distributed Computing, Vienna, Austria, August 24-28, 2015, Proceed- ings 21, pp. 601-612. Springer, 2015.
Arash Ashari, Naser Sedaghati, John Eisenlohr, Srinivasan Parthasarath, and P Sadayappan. Fast sparse matrix-vector multiplication on gpus for graph applications. In SC"14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 781-792. IEEE, 2014.
Carlo Baronio, Pietro Marsella, Ben Pan, Simon Guo, and Silas Alberti. Kevin: Multi-turn rl for generating cuda kernels. arXiv preprint arXiv:2507. 2025.
Nathan Bell and Michael Garland. Efficient sparse matrix-vector multiplication cuda. Technical report, Nvidia Technical Report NVR-2008-004, Nvidia Corporation, 2008.
Mohieddine Benammar, Asem Khattab, Sharief Saleh, Faycal Bensaali, and Farid Touati. A sinusoidal encoder-to-digital converter based on an improved tangent method. IEEE Sensors Journal, 17(16): 5169-5179, 2017.
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2 107.03374 2021.
Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen. Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, al. (TVM}: An automated (End-to-End) optimizing compiler for deep learning. In 13th USENIX Symposium Operating Systems Design and Implementation (OSDI 18). pp- 578-594, 2018.
Wentao Chen, Jiace Zhu, Qi Fan, Yehan Ma, and An Zou, Cuda-IIm: Lims write efficient cuda kernels. arXiv preprint arXiv:2506.09000 2025.
Timothy A Davis and Yifan Hu. The university of florida sparse matrix collection. ACM Transactions on Mathematical Sofwware (TOMS), 3811):1-25, 2011.
Diego Doimo, Alessandro Serra, Alessio Ansuini, and Alberto Cazzaniga. The representation landscape of few-shot learning and fine-tuning in large language models. Advances in Neural Information Processing Systems, 37:18122-18166 2024.
Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, et al. Stepcoder: Improve code generation with reinforcement learning from compiler feedback. arXiv preprint arXiv:2402.0139, 2024.
Zhen Du, Jiajia Li, Yinshan Wang, Xueqi Li, Guangming Tan, and Ninghui Sun. Alphasparse: generating high performance spmv codes directly from sparse matrices. In SC22: International Conference for High Performance Computing, Networking. Storage and Analysis, pp. 1-15. IEEE, 2022.
Ruibo Fan, Wei Wang, and Xiaowen Chu. Dtc-spmm: Bridging the gap in accelerating general sparse matrix multiplication with tensor cores. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3. pp. 253-267, 2024.
11

---

Published as a conference paper at ICLR 2026
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. Codebert: A pre-trained model for programming and natural languages. arXiv preprint 20Xx::2002.081555, 2020.
Trevor Gale, Erich Elsen, and Sara Hooker. The state of sparsity in deep neural networks. arXiv preprint 1rXXi::902.005744 2019.
Trevor Gale, Matei Zaharia, Cliff Young, and Erich Elsen. Sparse gpu kernels for deep learning. In SC20; International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-14. IEEE, 2020.
Constantino Gómez, Filippo Mantovani. Erich Focht, and Mare Casas. Efficiently running spmv on long vector architectures. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 292-303, 2021.
Joseph L Greathouse and Mayank Daga. Efficient sparse matrix-vectur multiplication on gpus using the esr storage format. In 14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 769-780. IEEF, 2014.
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. Graphcodebert: Pre-training code representations with data flow, 2021. URL https: aarxiv.iiv......aa/aaaaaaaaaaava org /aagg//b//// 200//009.0000 08366
Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. Unixcoder: Unified cross-modal pre- training for code representation, 2022. URL https://arxiv. wwtp:iii.orr//aa/. org/abs/ 2203.03850
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-rl incentivizes reasoning in Ilms through reinforce- ment learning. Nature, 645(808 1)88116:3333888 2025.
Gurufocus. Nvidia secures 92% gpu market share in q1 2025, 2025.
Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint 4rXiv:1510.00149, 2015.
Changwan Hong, Aravind Sukumaran-Rajpe Israt Nisa, Kunal Singh, and Ponnuswamy Sadayappan. Adaptive sparse tiling for sparse matrix multiplication. In Proceedings of the 24th Symposium on Principles and Practice of Parallel Programmingg pp. 300-314, 2019.
Guyue Huang, Guohao Dai, Yu Wang, and Huazhong Yang. Ge-spmm: General- purepal-purposeeeee sparse matrix-matrin multiplication on gpus for graph neural networks. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-12. IEEE, 2020.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.1 12186. 2024.
Ying Jia, Hongxiang Guo, Yi Guo, and Jian Wu. An efficient optical sparse matrix multiplication accelerator for graph neural networks. In 2022 Asia Communications and Photonics Conference (ACP), pp. 1868-1872, IEEE, 2022.
Peng Jiang, Changwan Hong, and Gagan Agrawal. A novel data transformation and execution strategy for accelerating sparse matrix multiplication gpus. In Proceedings of the 25th ACM SIGPLAN symposium on principles and practice of parallel programming, pp. 376-388, 2020.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakoo) Russ Bates, Augustin Židck, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-588 2021.
Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael W Mahoney, Kurt Keutzer, and Amir Gholami. An Ilm compiler for parallel function calling. In Forty-first International Conference Machine Learning, 2024.
12

---

Published as a conference paper at ICLR 2026
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Fredrik Kjolstad, Shoaib Kamil, Jonathan Magan-Kelley, David IW Levin, Shinjro Sueda, Desai Chen, Etienne Vouga, Danny M. Kaufman, Gurtej Kanwar, Wojciech Matusik, et al. Simit: A language for physical simulation. ACM Transactions on Graphics (TOG), 35(2):1-21, 2016.
Sumith Kulal, Panupong Pasupat, Kartik Chandra, Mina Lee, Oded Padon, Alex Aiken, and Percy Liang. Spoc: Search-based pseudocode to code. Advances in Neural Information Processing Systems, 32, 2019.
Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 35:21314-211 21328,, 2022.
Jerry Li al. Starcoder: May the source be with you! arXiv preprint arXiv:2305.06161, 2023.
Jiajia Li, Guangming Tan, Mingyu Chen, and Ninghui Sun. Smat: An input adaptive auto-tuner for sparse matrix-vector multiplication. In Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation, pp. 117-126, 2013.
Jian Li, Yue Wang, Michael R Lyu, and Irwin King. Code completion with neural attention and pointer networks. arXiv preprint arXiv: 1.0553,, 2017.
Jinhao Li, Jiaming Xu, Shiyao Li, Shan Huang. Jun Liu, Yaoxiu Lian, and Guohao Dai. Fast and efficient 2-bit Ilm inference on gpu: 2/4/16-bit weight matrix with asynchronous dequantization. In Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, pp. 1-9, 2024.
Xiaoya Li, Xiaofei Sun, Albert Wang, Jiwei Li, and Chris Shum. Cuda-Il: Improving cuda optimiza- tion via contrastive reinforcement learning. arXiv preprint arXiv:2507. 2025.
Yang Li, Si Si, Gang Li, Cho-Jui Hsich, and Samy Bengio. Learnable fourier features for multi- dimensional spatial positional encoding, Advances in Neural Information Processing Systems, 34: 15816-15829, 2021.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode. Science, 10866624) 2022.
Chin-Yew, Lin. Rouge: A package for automatic evaluation of summaries. Text summarization branches out, pp. 74-81, 2004.
Haokun Liu, Derek Tam, Mohammed Mugeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-effiient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35: 111199999555111 2022.
Weifeng Liu and Brian Vinter. Csr5: An efficient storage format for cross-platfone sparse matrix- vector multiplication. Proceedings of the 29th ACM on International Conference on Supercom- puting, pp. 339-350, 2015.
Jennifer A Loe, Heidi K Thornquist, and Erik G Boman. Polynomial preconditioned gmres to reduce communication in parallel computing. arXiv preprint arXiv:1907.00072. 2019.
Yuechen Lu and Weifeng Liu. Dasp: Specific dense matrix multiply-accumulat units accelerated general sparse natrix-vector multiplication. In Proceedings of the International Conference for Hligh Performance Computing, Networking, Storage and Analysis, pp. 1-14, 2023.
Marco Maggioni and Tanya Berger-Wolf Adell: An adaptive warp-balancing ell format for efficient sparse matrix-vector multiplication on gpus. In 2013 42nd international conference on parallel processing, pp. 11-20. IEEE, 2013.
Jiachen Mao, Xiang Chen, Kent W Nixon, Christopher Krieger, and Yiran Chen. Modnn: Local distributed mobile computing system for deep neural network. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 20/7, pp. 1396-1401. IEEE, 2017.
13

---

Published as a conference paper at ICLR 2026
Duane Merrill and Michael Garland. Merge- based sparse cettrix-vector multiplication (spmv) using the esr storage format. Acm Sigplan Notices, 51(8):1-2, 2016.
Maxim Naumov, L Chien, Philippe Vandermersch, and Ujval Kapasi. Cusparse library. In GPU Technology Conference, 2010.
NVIDIA. Nsight systems.
nvidia wwt.
2023.
Anne Ouyang, Simon Guo, Simran Arora, Alex L Zhang, William Hu, Christopher Ré, and Azalia Mirhoseini. Kernelbench: Can Ilms write efficient gpu kernels? arXiv preprint 1rX1v:2502.10517, 2025.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Rangeet Pan, Ali Reza Ibrahimzada, Rahul Krishna, Divya Sankar, Lambert Pouguem Wassi, Michele Merler, Boris Sobolev, Raju Pavuluri, Saurabh Sinha, and Reyhanch Jabbarvand. Understanding the effectiveness of large language models in code translation. CoRR, 2023.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. pp. 311-318, 2002.
Yun Peng, Akhilesh Deepak Gotmare, Michael R Lyu, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Perfcodegen: Improving performance of Ilm generated code with execution feedback. In 2025 IEEE/ACM Second International Conference on Al Foundation Models and Software Engineering (Forge). pp. 1-13. IEEE, 2025.
Mare' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. arXiv preprint 1rXiv:1511.0673:1 2015.
Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujic Liu, Duyu Tang. Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. Codebleu: a method for automatic evaluation of code synthesis. arXiv preprint arXiv:2009. /0297, 2020.
Marc Rußwurm, Konstantin Klemmer, Esther Rolf, Robin Zbinden, and Devis Tuia. Geographic location encoding with spherical harmonics and sinusoidal representation networks. arXiv preprint arXiv:2310.067. 06743, 2023.
Zarif Sadman and Apan Qasem. Lims in compiler optimization: Challenges and future direction. IEEE Pulse, 16(1):35-37, 2025.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06377 2017.
Monika Shah and Vibha Patel. An efficient sparse matrix multiplication for skewed matrix on gpu. In 2012 IEEE 14th International Conference on High Performance Computing and Communication & 2012 IEEE 9th International Conference on Embedded Software and Systems, pp. 1301 IEEE, 2012.
Manu Shantharam, Sowmyalatha Srinivasmurthy, and Padma Raghavan. Characterizing the impact of soft errors on iterative methods in scientific computing. In Proceedings of the international conference on Supercomputing, pp. 152-161, 2011.
Parshin Shojaee, Aneesh Jain, Sindhu Tipirneni, and Chandan K Reddy. Execution-based code
generation using deep reinforcement learning. arXiv preprint arXiv:2301.13816 2023.
Boi-Yiing Su and Kurt Keutzer. clspmv: A cross-platform opencl spmy framework on gpus. In Proceedings of the 26th ACM international conference on Supercomputing, pp. 353-364, 2012.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017
14

---

Published as a conference paper at ICLR 2026
Jianfeng Wang. Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-text transformer for vision and language. arXiv preprint 2rXiv:2205.14000 2022.
Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song. Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint 190.v0/90.01330 2019.
Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingehen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. Openhands: An open platform for ai software developers generalist agents. arXiv preprint 4xX1v:2407.16741, 2024.
Yue Wang, Richard Shin, Pengeheng Yin, Graham Neubig, Mia Song, and Tao Liu, Codet5: Identifier- aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv preprint arXiv:2109.008: 2021.
Yue Wang, Hung Le. Akhilesh Deepak Gotmare, Nghi DQ Bui, Junnan Li, and Steven CH Hoi. Codet5+: Open code large language models for code understanding and generation. arXiv preprint arXiv:2305.07922, 2023a.
Yuke Wang, Boyuan Feng. Zheng Wang, Guyue Huang, and Yufei Ding. (TC-GNN}: Bridging sparse {GNN} computation and dense tensor cores on (GPUs}. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pp. 149-164, 2023b.
Yuxiang Wei, Chunqiu Steven Xia, and Lingming Zhang. Copiloting the copilots: Fusing large language models with completion engines for automated program repair. In Proceedings of the 31 ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 172-184, 2023.
Ronald J Williams. Simple statistical rradient-followwe algorithms for connectionist reinforcement learning. Machine learning, 8:229 256, 1992.
Jaeyeon Won, Charith Mendis, Joel S Emer, and Saman Amarasinghe. Waco: Learning workload- aware co-optimization of the format and schedule sparse tensor program. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume pp. 920-934, 2023.
Haojun Xia, Zhen Zheng, Yuchao Li, Donglin Zhuang. Zhongzhu Zhou, Xiafei Qiu, Yong Li, Wei Lin, and Shuaiwen Leon Song. Flash-lim: Enabling cost-effective and highly- highly-efficient large generative model inference with unstructured sparsity. arXiv preprint arXiv:2309. /0285, 2023.
Jiaming Xu, Shan Huang, Jinhao Li, Guyue Huang, Yuan Xie, Yu Wang, and Guohao Dai. Enabling efficient sparse multiplications on gpus with heuristic adaptability. IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems, 2024.
Shengen Yan, Chao Li, Yunquan Zhang, and Huiyang Zhou. yaspmv: Yet another spmv framework on gpus. Acm Sigplan Notices, 4973):107-118, 2014.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint ar arXiv:2505.09388 2025.
Tiechui Yao, Jue Wang. Junyu Gu, Yumeng Shi, Fang Liu, Xiaoguang Wang, Yangang Wang. and Xuebin Chi. A sparse matrix optimization method for graph neural networks training. In International Conference on Knowledge Science, Engineering and Management, pp. 114-123. Springer, 2023.
Tong Ye, Weigang Huang, Xuhong Zhang, Tengfei Ma, Peiyu Liu, Jianwei Yin, and Wenhai Wang. Llm4effi: Leveraging large language models to enhance code efficiency and correctness. arXiv preprint a00iv:2502. (88.99992..8889, 2025.
15

---

Published as a conference paper at ICLR 2026
Zihao Ye, Ruihang Lai, Junru Shao, Tianqi Chen, and Luis Ceze. Sparsetir: Composable abstractions for sparse compilation in deep learning. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, pp. 660-678, 2023.
Yue Zhao, Jiajia Li, Chunhua Liao, and Xipeng Shen. Bridging the gap between deep learning and sparse matrix format selection. In Proceedings of the 23rd ACM SIGPLAN symposium on principles and practice of parallel programming. pp. 94-108, 2018
Size Zheng, Yun Liang, Shuo Wang, Renze Chen, and Kaiwen Sheng. Flextensor: An automatic schedule exploration and optimization framework for tensor computation on heterogeneous sys- tem. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 859-873. 2020.
Ming Zhu, Karthik Suresh, and Chandan K Reddy. Multilingual code snippets training for program translation. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 11783-11790, 2022.
A APPENDIX
A.1 THE KEY BOTTLENECKS PREVENTING THE MODEL FROM FULLY SOLVING THIS PROBLEM
The works Ouyang et al. 2025 Won et al.
2023 demonstrated the sparse high- gegh-performace parallel tasks difficult, compared with the code implementation (python/C++)) of algorithm problems (Le et al. 2022) and dense computation (Zheng et al. 2020). Here, we give several reasons to illustrate the bottlenecks of our work from GPU kernel and sparse computation aspect.
From GPU kernel aspect, (1) LLMs (base models) face the problem of insufficient training data volume, which restricts their performance in GPU kernel generation. CUDA low-resource
language in open-source training data (about 0.073% (Ouyang et al. 2025)). (2) LLMs fail to effectively utilize hardware resources (e.g., shared memory tensor core). resulting in low-efiiciency
generated kernels, which we have partially solved by SFT of high-performance CUDA code and RL of memory- limit reward.
From sparse computation aspect, (1) The parallel implementation of sparse operations is inherently more difficult than dense operations, because it necessary to consider that the GPU units have to process different numbers of non-zero elements during the parallel computation and then accumulate the results, which more likely to lead to calculation errors in the final result. (2) Given the complexity of sparsity patterns and the intricate relationship between sparse matrix features, GPU architecture characteristics, and performance, it is impossible to develop a one-size-fits fftt--lttllll code- implementation for sparse computation (Won et al. 2023).
A.2 THEORETICAL FOUNDATION FOR SINUSOIDAL EMBEDDING
The sinusoidal encoder /Vaswani et al. 2017). ) widely adopted across numerous fields such as audio signal and computer vision (Benammar et al. 2017 Rubwurm et al. 2023), has shown great potential in capturing sequential and positional information.
Following these previous studies, have some insights into why sinusoidal embeddings well- suited for sparse matrix representation and the motivation for its applicability.
1. Positional Invariance and Multi-Scale Representation
Our sinusoidal embeddings (from Eq. 3 in Sec3) treat row and column indices like continuous waves:
ind
ind
PE(ind, 2j) = sin 1000025/dmaal
PE(ind, 2j + 1) = cos 10000 Chenats
The base value 10,000 combined with the exponent 2j/ /model creates wavelengths that grow geometrically. Dimensions the lower end (small j) capture broad positional -thrnns-thhink block-level structures. Mcanwhile, higher dimensions (large j) pick up fine details like local clusters of non-zero entries.
16

---

Published as a conference paper at ICLR 2026
This multi-scale approach handles irregular sparsity patterns (like power-law distri- butions in GNNs) far better than fixed-interval positional encoding ever could.
2. Relative Positional Awareness
In sparse matrices, how non-zero elements relatively positioned drives memory access patterns (coalesced vs. scattered loads). Sinusoidal embeddings shine here because shifting positions (Li 2021) follows neat pattern:
PE(x) = [sin(w]))) cos(w) cons()))......i(((((x)) cin(()))
For any shift 8, we have:
PE(r + 6) = [sin + w18), 2 + w1d)....
Using angle addition identities:
(wur + wd) = sin(wr) cos(wō) + cos(war) sin(wo)
So:
PE(x + 6) A(6) PE(x)
Where A(8) is linear transformation matrix that rotates each frequency component by w6.
Key Insight: A linear model using sinusoidal PE can linearly model shifts. This allows the system to reason about relative differences. Essentially, models can generalize across matrices with similar non-zero distributions even if their coordinates are shifted.
3. Sparsity-Indueed Sparsity in Embeddings
Real-world sparse matrices often show fractal hierarchical structures (scale-free networks are a classic example). Sinusoidal encodings preserve local proximity: indices (r, c) that are close yield embedding vectors with high cosine similarity, making them suitable for learning locality- aware behaviors in sparse patterns:
They smoothly map huge index ranges (e.g.. millions of rows) into [-1, 1] without losing information unlike crude scaling methods.
They preserve neighborhood relationships: Elements with similar (r,, ca) indices get similar embeddings. This directly helps RL policy generate CUDA kernels optimized for local memory access patterns.
Why Alternatives Fail
Raw Indices:
Scale poorly for large matrices (e.g., r; 106 ry 1), causing numerical instability and poor generalization.
Max-Min Normalization: Loses relative positional relationships; e.g., (ri, c.) (1, 1) and (2, 2) may map to distant values after scaling.
Learnable Embeddings: Require fixed vocabulary sizes (infeasible for arbitrary indices) and fail extrapolate unseen matrix dimensions.
A.3 CODE CORRECTNESS VERIFICATION
For stopping the generation of eтrог code early during the generation process and reducing unnecessary computations, we integrate a dynamic syntax correctness verification mechanism during the decoding process. Specifically, for the partially generated CUDA code at the step when complete line of code is generated (marked by the token of "!:), we parse its intermediate code to construct a syntax tree and maintain a symbol table to record variable names, data types, and scoping rules. If the partial program violates CUDA syntax rules (e.g., unmatched parentheses, undeclared variables, or incorrect kernel launch syntax) introduces semantic conflicts (e.g., variable name reuse with incompatible types). the generation process is terminated early. The code extraction tool is modified from the public QwenL.M repository.
17

---

Published as a conference paper at ICLR 2026
A.4 REWARD DETAILS
A.4.1 SENSITIVITY STUDY OF TEFF AND TPENALTY IN REWARD HYPERPARAMETERS
Efficiency scaling factor (Tet): Fixed 1.0. This value balances the magnitude of effi- ciency rewards with correctness rewards (Eqq. 8), ensuring the policy prioritizes both code correctness and execution speed without bias.
Memory penalty (Tpenalty): ): Set to 0.3. This penalty discourages excessive memory use (c.g., shared memory >48 KB V100) while preserving positive rewards for correct, efficient code-foc for example, kernel with successful compilation (+0.5) and correct execution (+0.5) still yields net positive reward (0.5 + 0.5 0.3 0.7) even with memory overuse, avoiding dis-incentivizing valid code structures.
We evaluated SpMV performance (pass @ 1000, Compilation Rate (CR), average GFLOPS on V100)
by varying Penaltyy (0.1, 0.3, 0.5) on 400 test matrices:
Tat = 0.5: Underscales efficiency rewards, leading to lower speed (102.3 GFLOPS) despite modest correctness.
Teff 1.5: Overscales efficiency rewards, marginally boosting speed (118.7 GFLOPS) but reducing @ass 1000 (48.50 vs. 49.25 for base), as the policy prioritizes speed over correctness.
Table 4: Impact of Varying Feir (Fixed Fpenally = 0.3)
Teff
SpMV pass@ 1000
SpMV CR (%)
Avg. GFLOPS
0.5
45.75
55.20
102.3
1.0 (Base)
49.25
57.50
116.2
1.5
48.50
56.80
118.7
Varying Penalty (0.1, 0.3, 0.5) on 400 test matrices:
Tpenalty = 0.1: Weak penalty fails to curb excessive memory use, leading to lower speed (108.5 GFLOPS) due suboptimal memory allocation.
Peenaltyy 0.5: Overpenalization reduces pass 1000 (46.80) and CR (54.30), the policy
avoids valid ientesvvintensive optimizations (e.g., shared memory for cache locality).
Table 5; Impact of Varying Penalty (Fixed 1.0)
Ppnaatyy 0.1
SpMV pass 1000
SpMV CR (%)
Avg. GFLOPS
47.25
58.10
108.5
0.3 (Base)
49.25
57.50
116.2
0.5
46.80
54.30
119.1
These results confirm the base values (rett 1.0, Pepulty 0.3) optimal balancing correctness, compilation success, and execution speed while ensuring PPO stability and generalizability to unseen matrices.
A.4.2 Ref ficiency NORMALIZZATINN IN REWARD
To address runtime variability across matrices (c.g., small matrices with thase = 1 ms vs. large matrices with thase 100 ms), apply # **per-matrix z-score normalization to the efficiency reward (Ref ficiencyy Eq. 9):
(nace (X) ({,
Ref ficiency Teff
x Irastt
σx
18

---

Published as a conference paper at ICLR 2026
Here, y. and xx the mean and standard deviation of the speedup ratio (X) ((Y,))
cuSPARSE)
across 1000 validation matrices. This restricts R ficiency to 1- 2] for 95% of matrices, ensuring rewards are comparable regardless of a matrix's ' inherent size or sparsity. In most of the time, this ratio is bounded. If not, we do truncation make it within the range (restricted within 2-2.
A.4.3 WHIGHT ADIUSTMENT FOR CORRECTNESS AND EFFICIENCY
Correctness reward and the efficiency reward are not equally weighted in the final reward. To address this question, we have conducted experiments with adjustable weight coefficients:
Rfinal
Reorrectness (1 aa) Ref ficiency Upenalty llmemory
where a E [0, 1] controls the trade-off between correctmess and efficiency. Experimental results on the SuiteSparse test set:
a 0.9 (prioritize correctness): pass@ 1000 51.2 (14.1%) %), but execution speed = 98.7 GFLOPS (115.1%) compared to the original ar 0.5.
ar = 0.5 (balanced): pass @ 1000 = 49.25, execution speed = 116.2 GFLOPS (optimal trade-off).
* 0.1 (prioritize efficiency): execution speed 123.5 GFLOPS (16.3%), but pass 1000
= =38.7 (L21.4%)
We confirm that the original weight (a = 0.5) achieves the good balance between correctness and efficiency, which is critical for high-performance code generation.
A.5
PPO TRAINING DIAGNOSTICS (KL DIVERGENCE, ENTROPY, VALUE Loss, REWARD)
Figure o provides a comprehensive view of PPO training stability across 700 epochs, with key observations:
Value Loss (Subplot (a)): Both train and test value loss decline steadily from 0.7 to near 0.0 by epoch 700. This confirms the critic network (8:) accurately estimates the final reward (Ranal,, enabling reliable advantage calculation (GAE, $3.3).
Entropy (Subplot (b)): Policy entropy decreases from 1.0 to 0.15, indicating the policy converges to consistent, high-reward code patterns. The gradual decline also shows the policy retains enough exploration to adapt to diverse sparse matrices (c.g.. irregular vs. block-dense structures).
KL Divergence (Subplot (c)): KI. divergence between old and new policies stabilizes within 0. 0-13 after epoch 100. This ensures policy updates are incremental, avoiding catastrophic shifts that could harm performance.
Reward (Subplot (d)): The average reward rises from -0.5 to 1.5, reflecting the policy's ability learn increasingly correct and efficient code consisttnnn with the 30% speedup over baselines reported in $4.2.
These diagnostics collectively validate that PPO training is stable, with the policy learning to balance correctness, efficiency, and memory constraints without overfitting instability.
A.6 MORE BASELINE MODELS AND OBSERVATIONS
A. OPEN-SOURCE MODELS
We add more experimental comparisons more recent publicly available LLMs in Table For the dilemma of general-purport LLMs, DeepSeck-RI (671B) achieved the highest accuracy (pass@5) (22.75%), but its performance plummeted by 27.5%.
From the results, we can conclude that larger models excel at syntactic correctness but lack hard- optimization knowledge. Qwen3-32B accuracy (19.00%) is close to SparseRL+Qwen3- 14B (22.25%), but its performance lags by 34.5%. General-purpose code generation cannot replace domain-specific optimization.
19

---

Published as a conference paper at ICLR 2026
1.0
VoSk volue 1os5
Tes: Setropy
Entroyy
0.8
Value
0.3
02
0.1
500
300
Epoch
Epoch
(a) Value loss curve
(b) Entropy curve
Toin KL Tea--
15
1st
0.-
1.0
0.7
Reward
0.5
0.2-
0.1
300
Epoch
Epoch
(c) KL curve
(d) Reward curve
Figure 6: PPO diagnostics (KI., entropy, value loss, reward) over training.
Additionally, we find that the distilled version of DeepSeek-RI (7B/14B) achieved accuracy close to SparseRL. (17.50% vs. 16.50%), but its performance lags by over 30%. We infer that distillation transfers semantic knowledge but cannot transfer hardware optimization strategies.
Table 6: Correct functionality (pass@k) and Performance (Speedup vs. cuSPARSE) comparison for correct generated program pass@5.
SpMV
SpMM (col=8)
SpMM (col=32)
Model
Size
puss
pass
Perf.
passei
poss
Perf.
passil
pass
Perf.
DeepSeck-RI
67
15.00
22.50
-27.5%
16.50
20.50
-27.8%
15.50
22.75
-27.2%
DeepSeck-RI I-Distill
-Qwen-7B
7B
9.50
16.50
-31.1% %
10.00
16.75
-32.7%
10.50
16.25
-30.9%
DecpSock-RI-Deee
-Qwen-1ah 14B
14B
10.50
17.50
-32.3%
11.75
17.50
-27.9%
10.00
17.00
-28.1 %
Qwen3
8.00
12.50
-36.8%
6.75
-37.5%
7.00
10.75
-36.2%
Qwen3
14B
8.25
13.75
-3..9%
8.00
14.00
-36.3%
8.50
-35.7%
Qwen3
10.25
18.50
-3..1%
11.50
20.00
-35.8%
12.00
19.00
-34.5%
SparscRL+Qwen25.
7B
9.75
16.00
+27.5%
10:00
15.00
+39.1%
8.50
15.00
+15.8%
SparseLL+
Qwen2.5
14B
10.25
16.50
+30.2% %
10.25
16.00
+42.7%
9.25
15.25
+ +17.3%
SparseRL+twerr
+Wwe33
8B
15.75
20.00
+32.8 %
15.00
22.00
+42.5%
15.50
21.00
+20.1 1%
SparseLL+ twen3
14B
16.25
21.00
+34.5%
18.25
21.00
+47.2%
18.25
22.25
+21.9%
A.6.2 CLOSE-SOURCE MODELS
We add more experimental comparisons closed-source models with API calls (Claude-3.7/4m
GPT-4.1, o3-pro and DeepSeek-BB - R100528) in Table 7
Correctness f Performance Optimization. From the experiment, we observe that although the closed-son source model generates code with higher accuracy than our method, there is still a gap in performance. The comparison also shows that the performance of the generated programs does not vary much between dlosed-source models. Closed-source models excel at syntactic correctness but lack hardware optimization knowledge.
Core Finding:
20

---

Published as a conference paper at ICLR 2026
The reasons are from two aspects. (1) We observe that the programs generated by SparseRL make use of more complex optimization techniques and instructions in CUDA, such as the use of shared memory and warp shuffle instructions. These complex program statements lead to decrease in accuracy. (2) Our method uses RI. to fine-tune the performance of each test matrix in training, so that the model can perceive and generate different programs for different sparse matrices, while the programs output by closed-source models for different sparse matrices are often similar.
Table 7: Correct functionality (pass@k) and performance (Speedup vs. cuSPARSE) comparison for correct generated program on pass@5.
SpMV
SpMM (col+8)
SpMM (col=32)
Model
jesse8t
passll55
Perf.
passata]
pesset5
Perf.
passeRS
Perf.
Claude-3.7
18:50
24.25
-30.8%
17.00
22,00
-30.2%
17.25
23.50
-30.7%
Clande-4.0
-27.
20.00
26.75
GPT-4.1
23.50
30.00
-25.7%
22.25
28.75
-26.9%
23.00
29.25
24.8%
03-pro
23.25
32.75
-18.2%
24.25
31,50
-19.1%
25.00
32,00
-17.5%
DeepSeck-Rionn
26.75
33.25
-19.7%
26.25
34.25
GPT pro
25.25
32.75
18.2%
24.25
31,50
-19.1%
25.00
32,00
-17.5%
GPT-ot-mini
23.50
30.00
-23.7%
22.00
28.25
-23.8%
21.25
27.25
-22.8%
GPT-5
27.00
-18.0%
29.75
32.25
-18.3%
26.50
31.75
-20.5%
Clande sonmet-4
28.25
36.75
19.2%
24.25
31,50
-18.0%
26.00
31,50
-18.5%
SpunekL=+Owen25
7B
9.75
16.00
+27.5%
10.00
15:00
+39.1%
8,50
15,00
+15.8%
SparseRL+Qwen22
14B
10.25
16.50
+30.2%
16.00
+42.7%
9.25
15.25
+17.3%
SparseR[ @wwen3
15.75
20.00
+32.8%
15.00
22.00
+42.5%
15.50
21.00
+20.1%
SpaneRL=+Qwen3
14B
16.25
21.00
+34.5% %
18.25
21.00
+47.2%
18.25
22.25
+21.9%
A.6.3 APPROPRIATENESS OF CODERL/PPOCODER
CodeRL and PPOCoder are appropriate baselines because:
They are representative RL-based code generation methods, sharing the core paradigm (pretrained LLM + RL) as SparseRL.
While not explicitly trained for GPU kernel optimization, they are the most relevant baselines for evaluating RL-driven code generation performance.
A.6.4 DIRECT ENGAGEMENT WITH CONTEMPORARY RESEARCH ON LLM-BASED GENERATION OF NIGH-PERFORMANCE CODE
We elaborate on how each key innovation directly addresses the unique challenges of sparse computa- tion by the direct engagement with two contemporary research (PerfCodeGen (Peng et al. 2025 and LLM4EFFI Ye et al. 2025)) on LLM-based generation of high-performance code, with concrete links to experimental results:
PerfCodeGen (Peng et al. 2025) enhances general code efficiency through iterative self-refinement, using execution feedback from test cases, specifically identifying the most time-consuming unit tests to guide optimizations. The superiority of SparseRL stems directly from how its design addresses the unique challenges of sparse GPU computation:
Domain specialization: Unlike PerfCodeGen's 's focus on general code (e.g.. standard algorithms in HumanEval), SparseRL targets sparse matrix operations (SpMV/SpMM) on GPUs. These operations demand mastery of irregular memory access patterns and GPU parallelism areas, where PerfCodeGen'ss 's general feedback loops fall short. For example, SparseRL learns align non-zero elements with GPU memory banks to avoid bank conflicts, a optimization irrelevant to PerfCodeGen's 's target tasks but critical for reducing sparse computation runtime.
Feedback mechanism:
PerfCodeGen relies verbalized feedback (e.g.. "this test case is slow"), "), which struggles to capture low-level hardware nuances. In contrast, SparseRL's hierarchical RL reward (incorporating compilation success, correctness, execution time, and memory usage) directly incentivizes hardware-awan tweaks. For instance, the efficiency reward (tied to cuSPARSE runtime) drives the model to discover warp-padding strategies
21

---

Published as a conference paper at ICLR 2026
Table 8: Results of correct functionality (pass@H) and performance evaluated by speedup. Correctly generated code on pass@) 1 is used to evaluate the speedup compared with cuSPARSE.
Method
Size
pass@1 1
Speedup (vs. cuSPARSE)
LLM4EFFI Owen2.5-Codee
32B
10.00
-25.1%
(mentioned in their paper)
LLM4EFFI Qwen3
14B
11.00
-15,5%
(We add additionally)
SparseRL + Qwen3
14B
16.00
+32.5%
for unbalanced rows, a key reason SparseRI. outperforms PerfCodeGGen-derriee approaches sparse tasks.
PerfCodeGen treats inputs as natural language descriptions, but SparseRL's sinusoidal embeddings of row/column indices explicitly model sparse matrix structure. This allows it to generate matrix- matrix-specific kernels (e.g., different strategies for clustered vs. scattered non- non-zeros), boosting compilation rates compared to using indices (Appendix A.4 Table 5).
Input representation:
LLM4EFFI Ye al. 2025 optimizes general code via logic-domain algorithm exploration and code-domain implementation, prioritizing efficiency first. SparseRL's performance gains arise from design choices tailored to sparse GPU operations:
Scope and hardware awareness: LLM4EFFI focuses on general code (e.g.. mathematical functions) and lacks GPU-specific optimization targets. SparseRL, by contrast, targets CUDA kernels for sparse operations, learning to optimize thread block sizes, shared memory usage, and atomic reductions, which are critical for GPU performance. This specialization explains why SparseRL achieves 1.82x speedup on A100 (Sec. 4.2) over TVM-S (a generic sparse compiler) on SpMV tasks.
Learning paradigm: LLM4EFFI uses supervised fine-tuning with synthetic test cases, which cannot capture GPU runtime metrics. SparseRI'ss deep RL. (PPO) directly optimizes for execution time on GPUs, enabling it to learn non-intuitive tweaks like using __shfl_ instructions for warp communication, and reducing global memory access on matrices with high row overlap.
Adaptivity to input structure: Sparse computation performance highly input-dependent, but LLM4EFFI generates static code. SparseRL's index embeddings allow it to customize kernels for each matrix's non-zen non-zero distribution (e.g., padding short rows for load balance), a capability that improves pass @ 1000 (in Section 5.2 Table compared to non-embedded approaches.
These design choices directly address the unique challenges of sparse computation, causing SparseRL's superior performance. We are the first to apply RL to sparse computation, enabling these hardware-aware and input-adaptive optimizations that general frameworks cannot replicate.
We provide the experimental results comparing both frameworks on the SpMV task using the SuiteSparse test set (100 matrices) under identical hardware/software conditions in Table g The experimental setup is as follows:
Task: Sparse Matrix- Veccor multiplication (SpMV)
Dataset: 100 test matrices randomly sampled from SuiteSparse (among 400 test matrices in our paper).
Timeline: Generation time (5 hours; average 3 min for each matrix) and SpMV execution time (3 hours) of LLM4EFFI
Baseline: LLM4EFFI Yee et al. 2025) using its official implementation on anonymous
Github and recommended settings.
22

---

Published as a conference paper at ICLR 2026
Since LLM4EFFI follows similar generate-verify-puuee workflow, can adapt LLM4EFFI to accept identical sparse matrix input (row/column indices) but without sinusoidal embeddings (its design lacks this modality).
Optimization target: SpMV CUDA kernel generation.
Evaluation Metries:
Correctness: pass @ 1.
PPrffrraace: GFLOPS (speedup over cuSPARSE).
A.6.5 SPMV PERFORMANCE ON DLMC MATRIX
To broaden the matrix diversity, conduct additional performance evaluations the DLMC (Deep Learning Matrix Collection), which includes sparse matrices frequently used in deep learning. (1) On average, SparseRL achieves performance improvements of 10.21x, 1.28 x on V100 (10.70x, 1.33x on A A100) over CodeRL, PPOCoder. (2) Notably, on average, SparseRL achieves performance improvements of 1. x,29x, 1.69 on V100 (1.25x, 1.66 on A100) over cuSPARSE, TVM-S.
We choose five representative matrices of 0.95 magnitude pruning (Han et al. 2013) based on non-zero elements illustrate performance in Table (where ml, m2, m3, m4, m5 are symbol-modality 11188-512-sharaee -ear-d-weiigntte weights- -Oaxxx body-decoder-layer ffffi-coo-2-fulll-ooooo bottleneck-2-hee 2-blokk-------------- ---bblo--grrop--2222 group4-2-1)
bottleneck-2.
block Beoikk-rouppooee propupp-rooecioo--iee blo.k-ooo---o- finul
Table Performance (GFLOPs) Comparison of different methods on DLMC dataset.
Matrix
CodeRL
PPOCoder
TVM-S cuSPARSE
SparseRL
ml
27.76
27.32
96.41
97.23
121.82
m2
20.98
25.44
67.49
58.34
88,93
17.53
21.02
48.21
48.62
64.56
m4
10.11
15.67
26.79
31.67
43.50
m5
15.82
12.10
8.39
15.26
25.22
A.7 GENERALIZATION TO OTHER LLMS
We have validated SparseRL's compatibility with two additional open-source LLMs beyond Qwen
(Table 10
GLM-Z1-9B. SparseRL+GLM-ZI-90 achieves pass 19.75 and 30.5% speedup, show- ing consistent improvement over the base model.
DeepSeck-Coder-6. 6.: DpeeseR speedup, maintaining the same level of improvement with Qwen.
achieves pass@5 = 20.25 and 31.0%
Table 10: Correct functionality (pass@k) and performance (Speedup vs. cuSPARSE) comparison for correct generated program on pass@5.
SpMV
Model
Size
pass@1
pass@5
Perf.
SparscRI+GI.M-ZI
9B
15.50
19.75
+30.5%
SparseRL=-DepSeeek Coder
6.7B
17.75
20.25
+31.0%
SparseRL+QWen22.5
9.75
16.00
+27.5%
SparseRL=Qwen2..5
14B
10.25
16.50
+30.2%
SparseRL+QWen3
8B
15.75
20.00
+32.8%
SparseRL+Qwen3
14B
16.25
21.00
+34.5%
These results demonstrate that SparseRL's framework (embedding, reward, RL pipeline) is model- agnostic and consistently enhances performance across different code L.LMs.
23

---

Published as a conference paper at ICLR 2026
A.8 MORE SPARSE MATRIX EMBEDDING
We have added the comparison between the sparse matrix embedding used in SpanseRL and other code embedding techniques, such as UniXcoder (Guo et al. 2022) and GraphCodeBERT (Guo et al. 2021]. Different from text embedding, how to represent sparse matrix is the critical part i SparseRI..
UniXcoder designs a one-to-one mapping to transform an abstract syntax tree (AST) into squence of tokens. Similarly, organize the non-zero elements in sparse matrix into a quad-tree and map it into sequence by the same one-to-one function, thus processed together with text information.
GraphCodeBERR modifies softmax scores in attention heads capture the adjacency relations in AST, setting zeros for non-adjacent nodes. Following this design, we add extra heads for our quad-tree non-zero elements and apply the same strategy extract the edges in this tree. Therefore, sparse matrices can be fed into a pretrained model.
We also implement simple flattening algorithms from matrix to sequence of tokens and compare the effectiveness of other embedding approaches to sparse matrix embedding. The results are in Table 11
Table 11: Ablation study of sparse matrix embedding on correct functionality (pass@k) and Compi- lation Rates (CR) under k 1000. (SpanseRL+CodeT5, 770M)
Sparse Matrix
SpMV
Embedding
pass@1
pass@5
pass@1000
CR
UniXcoder
6.75
14.25
43.00
50.00
GraphCodeBERT
7.00
15.00
45.00
52,25
Raw
5.25
10.50
40.75
49.50
Max-Min
8.25
14.00
43.25
51.75
Sinusoidal
9.25
15.75
48.75
56.50
A.9 EXPLORATTON-EEXPOOITTTII BALANCE
During our experiment, we have discovered the balance issue between exploration and exploitation. We use PPO with entropy bonus (3 0.001, 0.01, 0.1) to balance in Table I2
(I) when exploration insufficient, different sparse matrices can produce similar or even identical sparse programs, which can lead to performance degradation because different matrices need to be adapted different programs.
(2) when exploration is too high, the correctness of the generated program will decrease due to excessive pursuit program diversity.
These are the reasons why we choose 3 0.01 in experiment.
Comparison on SparseRL=Qwen2.5-1 Performance (Speedup vs. cuSPARSE) compari- Table 12:
son for correct generated program on pass@5.
pass@1 1 pass&5
SpMV perf. pass@1 pass@5 SpMM(col=8) perf.
0.001
3.25
11.50
+18.2%
4.25
9.75
+23.7%
0.01
10.25
16.50
+30.2%
10.25
16.00
+42.7%
0.1
15.25
21.75
+5.2%
17.50
20.00
+11.7%
Besides, PPO's clipped objective to balance exploration (via top-k sampling with temperature annealing) and exploitation (prioritizing high-reward token sequences). (1) Exploitation: 70% of tokens from the policy predictions. (2) Exploration: 30% from broader sampling (temperature=0.7,
24

---

Published as a conference paper at ICLR 2026
which is commonly used value and can encourage certain degree of exploration without causing excessive smoothing.).
A.10 EXAMPLES OF DIFFERENT SPARSE MATRIX NEEDING DIFFERENT HIGH- PERFORMANCE CODE
We give two examples of sparse matrices and the corresponding tergh-eeffrmance SpMV code. These examples show that different processing strategies really matter for different patterns of sparsity.
Matrixl: Sparse matrix 1 [[1,0,0,0], [0,1,0,0], [0,0,1,0], [0,0,0,1]] is diagonal matrix with the same number non-zero elements in each row.
Codel: DIA format is used store the matrix and each thread processes row in parallel.
Advantage: (1) Memory Access Efficiency. Matrixl 's diagonal structure allows DIA to store only offsets data arrays. Each thread accesses contiguous datafk * i] (stride-1) and aligned x(j] (coalesced global reads), achieving near-peak memory bandwidth. (2) Perfect Load Balance: Uniform non-zeros (1/row) ensure all threads perform identical work. Zero warp divergence occurs.
Listing 1: code example1
//Input: datal [k]] stores diagonal elements //Offsets|K/ is the offset of the k-th diagonal line
//x[] input vector, y[] output vecter
//N: matrix dimension, num_diagonals: number diagonals
int = blockIdx.x blockDim.x + threadIdx.x/ // zow index
if < n)
float sum 0.0;
10
for (int k 0; k num_dlagen k++) {
11
int j = i offsets(k); // col index = row index + offset
12
if >= 0 aa n) // check col index is legal
13
sum datalk
x[i]; // access diagnal data
15
16
viil = sum; // write pack results
17
Sparse matrix2 [[0,0,0,0], [0,0,0,0], [0,0,1,1], [0,0,1,1]] is a locally dense cluster with different distributions of non-zero numbers between blocks.
Matrix2:
Code2: Block-CS) used to store the matrix2 and one warp can process one local area in parallel. Due to page limitations, we show the core fragment of code.
Advantage: (1) Exploiting Data Locality: Dense 2x2 blocks enable contiguous memory accesses. Loading block data and segments into shared memory reduces global memory accesses by 4x (vs. element-wise CSR). (2) Cooperative Computation: Warp-level parallelism reuses loaded across multiple rows (e.g., x[2], x[3] used for both row2 and row3), reducing redundant data movement.
Listing 2: code example2
//Input: block datal | stores dense block data //Block_row_ptr[] block row pointer; block_col_idx(] block column index //x[] input vector; y[] output vector
//Blocksize: block size; nun-block_rows: number block rows
int block_idx blockIdx.x; //The current block index being processed
7
// The thread number assignment is omitted
//Shared memory declaration (Narp internal sharing)
10
shared_ float s_block BLOCK_SIZE + BLOCK_SIZE); //store dense blocks
11
shareq_ float s_x |BLOCK
//Store input vector fragments
12
//A11 threads collaborate to load data
13
if (threadidss dx. block_st size block_size)
25

---

Published as a conference paper at ICLR 2026
14
block (threadIdx. block data @block idx block size block size threadIdx.x];
15
16
if (threadIdx.x < block_size)
17
(threadIdx.x) = x x[eol_start threadIdx.x]; //Load the corresponding
fragment of
19
synothreads ( //Ensure that data loading is complete
20
//Compute within the block (each thread processes ene line within the block)
21
if (threadidx.x
slock_size)
22 float sum - 0.0;
23 for (int j 0; j < block_size; j++) {
24
sum += block [threadIxxx block block_size size + i] s_x[i); //Intra industry
dot product
26
atomicAde (&y (row_start threadIdx.x] x], sum) //Atomic update results
27
If generated CUDA code is not matched, the performance will decrease in this situation. We take Code2 (Block-CSR) on Matrix and Codel (DIA) on Matrix2 as examples to demonstrate the penalties caused by this mismatching. (1) Using Codel (DIA) on Matrix2 leads to inefficient Memory Access. Threads processing row0-row access with invalid offsets (e.g..j -2), causing uncoalesced reads and branch divergence.
(2) Using Code2 (Block-CSR) Matrix leads to thread underutilization. warp (32 threads) processing a 1x1 "dense block" wastes 31/32 threads (99% idle).
A.11 PERFORMANCE ANALYSIS
We analyze the SpMV programs generated by SparseRL and investigate the sources of speedup compared to its competitors on NVIDIA V100. We provide detailed performance metrics by profiling the generated programs using NVIDIA Nsight Systems (NVIDIA) (003).. This analysis elucidates why the SpMV programs produced by SparseRL. are particularly well-suited for specific matrices.
The reason of Sp faster than PPOCoder. in T3] Notably, the Memory Access each matrix decreases significantly. The difference in memory access is reflected in the varying storage formats. PPOCoder is limited to assessing program correctness and lacks awareness of the GPU's parallel execution strategy. ensure the correctness of the program, it generates additional memory access indices, which increases memory access operations and reduces the operational efficiency. During the code generation process of SparseRL, the utilization of memory is taken into account in the reward mechanism. As result, memory access reduced and the execution efficiency is enhanced.
The profiling metrics for the matrix are presented
Table 13: Performance and memory access of PPOCoder, TVM-S, and SparseRL on V100.
Matrix
Performance(FFLoPS)
Memory (ccess(Mbytes)
PPOCoder
TVM-S
SparseRL
PPOCoder
TVM-S
SparseRL
pwt
30.38
38.68
75.84
3.75
3.04
2.65
nemeth22
70.69
76.25
136.51
12.89
10.48
water_tank
86.47
108.84
139.51
20.18
12.87
15.29
helm2d03
65.40
69.81
108.36
53.71
37.00
31.18
nc
106.52
116.57
156.42
30.29
19.37
23.08
ga2010
51.26
57.63
90,99
24.69
18.32
15.26
mi2010
45.02
48.53
89.40
53.35
31.71
18.01
az2010
28.21
79.86
78.11
45.14
40.49
13.36
va2010
37.08
46.97
85.94
39.20
49.99
15.84
atmosmodd
111.93
122.20
149.87
123.45
98.58
75.60
mn2010
42.36
55.41
83.63
41.64
30.31
13.92
The reason of SparseRL faster than TVM-S. The profiling metrics for the matrix are presented in Table 14 It is worth noting that the SM Occupancy (active warps per SM) of each matrix experiences
26

---

Published as a conference paper at ICLR 2026
substantial increase. In the TVM-S, to achieve load-balancing, each thread is assigned process fixed number of non-zero elements and uses the atomic addition to write back the results. Conversely, SparseRL employs padding to standardize the number of non-zero elements in each row. This enables each thread to accumulate the non-zero elements of a single row and then perform a write-back operation using only coalesced store instruction. SparseRI. adopts operations for reduction and write-back, thereby fully capitalizing on the SM occupancy.
concise and efficient
Table 14: SM Occupancy Comparison on V100.
Matrix
SM Occupancy( %))
PPOCoder
TVM-S
SparseRL
pwt
36.08
71.68
78.27
nemeth22
72.29
54.73
83.48
water_tank
53,98
59.01
81.63
helm2d03
61.83
57.49
87.48
net150
76.27
64.64
84,80
ga2010
59.88
56.54
85,53
mi2010
36.19
71.28
89.31
az2010
42.10
64.47
81.61
va2010
7.57
43.81
53.03
atmosmodd
45.03
75.94
91.34
mn2010
37.20
56.78
82.66
A.12 GPU UTILIZATION
We elaborate GPU utilization by GPU occupancy (specifically SM occupancy, the standard metric for GPU resource utilization). Table T5I is the SM occupancy comparison (A100 platform) for representative matrices across our SparseRL. and cuSPARSE, measured via NVIDIA Nsight Systems (v2024.1) on an A100-80GB GPU (kernels compiled with nvce 12.2, same flags for fairness):
Table 15: SM occupancy comparison (A100 platform) for representative matrices.
Matrix Size
SparseRL SM Occupancy (%)
cuSPARSE SM Occupancy (%)
Matrix Name
Reason for Occupancy Gap
(nnz)
pwt
12,456
68.2
51.5
SparseRL optimizes thread block size (128 threads/block) to fit A100's register limits. Our 36 KB shared memory allocation avoids conflicts, enabling more concurrent warps.
nemeth22
489,210
83.4
62.1
dimc frameformers
1,870,500
79.8
55.7
RL adjusts momory coalescing to reduce warps during global memory access. Dynamic thread block scheduling matches 1000's 6912 cores large inputs.
va2010
4,920,381
72.6
48.9
As shown, SparseRL achieves 15-30% higher SM occupancy than cuSPARSE across matrices this better utilization of the A100's SM resources (warps, registers, shared memory) directly contributes to our performance improvement over cuSPARSE.
A.13 DIFFERENCES OF PERFORMANCE OPTIMIZATION CHARACTERISTICS BETWEEN SPMV AND SPMM
Although SpMV (sparse matrix-vector multiplication) and SpMM (sparse matrix-dense matrix multiplication) involve similar arithmetic structures, their performance optimization priorities differ fundamentally. SpMV's row-wise computation sparse matrix and dense vector induces irregular memory accesses due to scattered vector element fetches, requiring optimizations like loop tiling (Du et al. 2022 Yan et al. 2014), SIMD vectorization (Maggioni & Berger- Wolf [003) Shah & Patel 2012). and fine-grained row-level parallelism (Liu & Vinter] 2015] Merrill & Garland 2016 to
mitigate cache misses and load imbalance from varying row sparsity.
In contrast, SpMM's interaction between sparse matrix rows and dense matrix columns/blocks allows leveraging the dense matrix's 's regular memory layout. Optimization here focuses block tiling to
27

---

Published as a conference paper at ICLR 2026
reuse dense submatrices in cache (Hong al. 2019: Jiang et al., 2020). hybrid strategies combining
sparse traversal with dense linear algebra primitives Yee et al. (023) (e.g., BLAS), and coarser-
grained parallel decomposition (Huang et al. 2020 Gale et al. 2020) (c.g., row/block partitioning)
to align with cache hierarchies. While SpMV emphasizes refining irregular vector access patterns,
SpMM prioritizes sparse-dense data reuse and scalable dense computation integration.
A.14 SCALABILITY
A.14.1
SCALABILITY WITH MATRIX SIZE
For matrix size, SparseRL processes input sparse matrices via row/column indices of non-zero elements (rather than dense representations). This avoids quadratic memory growth with matrix dimensions. Therefore, our method has no absolute correlation with the size of the matrix; the only consideration is the number of non-zero elements in the input. The RL fine-tuning stage involves iterative code generation, compilation, and execution. Larger (nnz) matrices increase: (1) Episode length (longer input prompt). (2) Reward of computation time (execution time scales with NNZ).
A.14.2 SCALARILITY WITH SPARSITY LEVEL (PROPORTION OF ZERO ELEMENTS)
SparseRI. inherently decouples from global sparsity ratios due to core design of sparsity-agnostic input processing. The model sees only non-zero coordinates, making it insensitive to global sparsity ratios. So our method can adapt to any sparsity level. The experimental validation across full sparsity spectrum are in Table 16
Table 16: Experimental Validation Across Full Sparsity Spectrum
Sparsity Level
Validation Scenario
Key Result (A100 GFLOPS)
<50%
Switches to cuBLAS dense kernels
Dense >> Sparse (+2.1x)
50-90%
DLMC pruned matrices (60% sparsity)
SparseRL vs. cuSPARSE: +1,29x
Sustained speedup (e.g., wiki miki-Talk
>90%
SuiteSparse hyper-sparse matrices
at 99.8% sparsity: +1,41x)
A. 14.3 SCALABILITY WITH PATTERN COMPLEXITY
For pattern complexity, the sinusoidal embedding captures positional relationships between non-zero elements, enabling the model encode irregular patterns (e.g., clustered random distributions). We give two examples here. For random patterns, SparseRL reduces redundant memory accesses (e-g., 2.65 MB vs. TVM's 3.04 MB for pwt matrix). For skewed distributions, it improves load balancing via thread-level padding, increasing SM occupancy (e.g., 89.31% % vs. TVM's 71.28% for mi2010).
A.14.4 GENERALIZATION TO DENSE OPERATIONS
Currently our method is designed for sparse operations, and the framework's core components (e.g.. sinusoidal embedding for structural input) are task-agnostic. The dense matrix has sparse diversity without non-zero distribution.
A.15
FAILURE CASES
We give examples of highly irregular matrices with extreme sparsity (>99.99% zeros). cuSPARSE's hand-optimized kernels perform better than our SparseRL in Table 17
For va2010 matrix, it exhibits hyper-irregube sparsity where 90% of non-zeros are concentrated in 100% of rows. This causes severe underutilization of GPU threads when: long rows overload individual threads (serialization) and short rows leave threads idle (resource waste).
For az2010 matrix. the generated code uses strided memory access patterns (e.g., accessing columns [3, 17, 129] in one warp), causing: additional 18% memory transaction overhead (vs. cuSPARSE) and 1.32x more 1.2 cache misses.
28

---

Published as a conference paper at ICLR 2026
Table 17: Failure cases
Matrix cuSPARE GFLOPS SparseRI. GFLOPS
va2010
72.97 79.86
70.94
az2010
78. 11
A.16 COMPUTATIONAL COST BREAKDOWN
Training cost: while 5 days on 8 GPUs seems high, the generated code's efficiency gains (30% faster execution for SpMV) ) offset this cost in scenarios with repeated use (e.g., scientific simulations with 10k+ iterations) in Table 18
Table 18: Method GFLOPS vs. cuSPARSE
Stage
Time (hours) % Total Cost
SFT
24
20%
R[(search)
24
20%
(cCompilationn
30
25%
RI (execution)
42
35%
A.16.1 REDUCING COMPUTATIONAL COST IN RI. TRAINING
We explore surrogate reward models (e.g., predicting execution time via lightweight profilers) to reduce reliance on repeated compiler/executor interactions. Here, we show the preliminary tests on two approaches.
For proxy models, we train lightweight GNN to predict execution time (input: matrix structure + code abstract syntax tree), replacing 50% of real compilation: (1) Training time shortened 2.5 days. (2) We preliminarily test on SpMV, which shows the average performance (GFLOPS) loss is about 5.1%
For caching strategies, we try to use the cached compilation results for similar matrices. More specifically, we use the statistical features of sparse matrices (such as the average number and variance of non-zero elements per row, well as the number of rows/columns/non-ze etc.) to measure the similarity between sparse matrices, and cluster them according to these features.
elements,
In the experimental setup, we use k-means to cluster 700 training matrices into 35 classes (average 20 matrices each class). When the matrix is selected, there are 60% probability (byper-parameter) to actual execute the generate program, then the running time of the cluster will be updated. We observe that when training upon 100 epochs (about one day), experiments show a 56.4% reduction training time.
A.17 OUTWEIGH THE BENEFITS GENERATED OPTIMIZED CODE
First, in actual use, the model has been trained in advance, so there is no need to consider the training overhead. We only need to consider the time for inference code generation. In inference code generation situations, firstly, for SuiteSparse Matrix Collection in scientific computation, we analyze the generation time and execution time. Our method is beneficial in scenarios that require a lot of runs (usually 100 thousand times in real application), such as mesh simulation (Kjolstad al. 2016) or GMRES (Loe et al. 2019) with thousands runs of sparse routines. Secondly, for DLMC dataset in deep learning, in the realm of neural networks, once the weights of sparse network are deployed, they can reused repeatedly (Xia et al. 2023). So the generation process can be conducted offline, obviating the need to account for the search overhead.
29

---

Published as a conference paper at ICLR 2026
Table 19; Time (seconds) is recorded by iteratively executing SpMV.
Matrix
Time (cuSPARSE) Time (Generation + SparseRI.)
pwt
15
3 +
nemeth22
23
5 13
helm2d03
24
6 14
net150
30
7+ 16
A.18 EXTENSION TO OTHER BACKENDS
Currently, SparseRL focuses on CUDA optimization due to its dominant role in high- performance sparse computing (Nvidia Secures 92% GPU Market Share in Q1 2025 by (Gurufocus) 2025 )
Actually, our framework's core components are theoretically backend-agnostic (not limited to CUDA): (1) Input Representation: The sinusoidal embedding of sparse matrices ($3.2) is hardware- independent. (2) Reward Mechanism: Hierarchical rewards (compilation//fffciennny) can be adapted to any compiler/executor environment.
Besides, we can change the training process for other back-ends from two aspects. (1) Modify the training data (from CUDA HIP language) in supervied fine-tuing (SFT) stage. (2) Adapt the reward function to their respective compilers (e.g., HIP Language validation). As a temporary solution, we can use translation tools (translating CUDA HIP) or make manual translations.
A9 COMPARISON WITH HUMAN EXPERT IMPLEMENTATIONS
We have supplemented experiments comparing SparseRL. with two representative human expert implementations:
Merge-based (Merrill & Garland Hand-crafted SpMV kernels (CSR5 (Liu & Vinter 2015), 2016)): Selected from classic works that are widely recognized as high-performance human- optimized solutions.
Industry-standant optimized works: Derived from NVIDIA's cuSPARSE library (v12.1, close-source), which incorporates hand-tuned optimizations by NVIDIA's engineering team for diverse sparse matrices.
Experimental results on the SuiteSparse test set (400 matrices) show:
On hand-crafted SpMV kernels, SparseRL's generated SpMV code outperforms hand- crafted kernels by 12.3/10.5% on average in execution speed (GFLOPS) compared with CSR5/Merge-based.
On industry-standard optimized works, SparseRL's generated SpMV code outperforms cUSPARSE by 8.6% on average in execution speed (GFLOPS).
These results confirm that SparseRL can generate code comparable to exceeding human expert levels, especially in adapting to diverse sparse matrix structures.
A.20 HUMAN-READABILITY
We explain the human human-rcadability of generated code from the following three aspects:
(I) Variable name. Although the variable names are different in each generated code, they still follow certain rules. For example, the variable names row index, index (instead of t0, v1). The variable name can reflect the meaning of the variable.
(2) Comment. In some cases, method can generate comments for key operations, for example, in row-wise parallelization (//Thread processes row with coalesced access). This helps the human to understand the meaning of this code line.
30

---

Published as a conference paper at ICLR 2026
(3) Real human. We survey 3 GPU developers (several years of CUDA experience) by actual situation to review the code generated by SparseRL. They confirmed that the generated code contains enough comments and reasonable variable names ensure the interpretability. Additionally, they say "It closer to the manual writing style than expected - the kernel segmentation is clear, the shared memory usage is well commented, and can be served as start point for further optimization"
A.21 DATASET DETAILS
A.21.1 MATRIX SIZE
We conducted experiments using two publicly available, widely used sparse matrix datasets-SSiiteSparss SuiteSparse Matrix Collection and Deep Learning Matrix Collection (DLMC)-wV matrix sizes (measured by number of non-zeros, nnz) and structural characteristies tailored to real- world sparse kernel scenarios. Below are their key details:
SuiteSparse Matrix Collection.
Total matrices used: 1,000 (filtered to exclude trivial/dense matrices, retaining those with practical sparse kernel value).
Matrix size (nnz range): 10,000-5,000.000 non-zeros. This covers small (e.g., pwt with 12,456 nnz), medium (e.g., nemeth22 with 489,210 nnz), and large (e.g., va2010 with 4,920,381 nnz) sparse matrices.
Key domains: Includes matrices from scientific computing (e.g., fluid dynamics, structural analysis), graph algorithms (e.g., social networks, web graphs), and engineering simula- tions tions-aligning with cuSPARSE's typical application scenarios.
Deep Learning Matrix Collection (DLMC).
Total matrices used: 500 (focused on matrices derived from deep learning workloads, complementary to SuiteSparse).
Matrix size (nnz range): 10,00002,,000.00 non-zeros. These matrices are optimized for sparse neural network layers (e.g., sparse CNN filters, transformer attention masks), with narrower but more DL-specific nnz range (e.g., dime_cnni with 89,340 nnz, dlmc_trans mamc.transffrrmer with 1,870,520 nnz).
Structural feature:
Most matrices have irregular sparsity patterns (mimicking real DL sparse activations), which better test the adaptability of our SparseRL-gennrated kernels vs. cuSPARSE's s general-purport optimizations.
A.21.2 DATASET SCALE
We notice the observation regarding the RL training dataset scale (1 entities for training + testing). We acknowledge that this dataset size is relatively modest, while also noting that work still achieves competitive performance on sparse kernel optimization tasks. For example, reaching 49.25% pass @ 1000 and 116.2 avg. GFLOPS the 1100-entity dataset.
We fully agree that expanding the RL dataset to a larger scale (e.g., 5k 50k entities covering more diverse matrix sparsity patterns and GPU kernel scenarios) would further enhance the model's gener- alization--aariccllary for unseen matrices with irregular sparsity or edge-case hardware constraints. This is key direction for our future work. larger dataset would enable the RL policy learn more transferable optimization rules, reducing performance drops on out-of-distribution data.
Besides, we explain the dataset scale from two aspects:
Since pretrained coder models already have programming on this small dataset for high-perffrmance programming is enough. For example, CUDA- L1 (Li et al. 2025) uses 105 dataset scale and Kevin (Baronio et al. 2025) is repeatedly trained by RL on the order of 180 datasets scale.
CUDA skills, training
Our current research focuses on optimizing programs for the single task of sparse matrix- vector multiplication (SpMV) and sparse matrix-matri multiplication (SpMM), rather than
31

---

Published as a conference paper at ICLR 2026
generating multifunctional GPU optimization programs. For such customized domain- specific tasks typically falls into the scope of few-shot learning (Doimo et al. 2024, Liu ct al. 2022). existing dataset scale is deemed sufficient to support model training and performance validation, and we will collect more sparse matrix for better performance in SFT/RL phases future.
In summary, while current 1100-entity dataset demonstrates the effectiveness of SparseRL's RL framework, we recognize its limitations and confirm that scaling the dataset will be priority to boost generalization in subsequent iterations.
A.22 PERMUTATION ROBUSTNESS AND REORDERING
We have supplemented experiments on permutation robustness and reordering:
1. Permutation robustness: We test random row/column permutations (10 permutations per matrix) on 50 random sampled sparse matrices in festset. Results show that SparseRL's performance within the normal range of performance fluctuations (degrades by about 1.7% under permutations), We analyze this because the sinusoidal embedding" ability to capture relative positional relationships (detailed in Section 3.2) mitigates the impact of absolute index changes.
Reordering integration: We have added an optional pre-processing step (Appendix A.6) that applies reordering to the sparse matrix of test-dataset before embedding. We reorder each row of the original matrix in descending order based on the number of non-zen con- elements in each row, so that rows with more non-zero elements are clustered together, resulting in better memory locality. This further improves SparseRI's performance by 4. I. on average, as reordering enhances memory locality.
A.23 DECODING STRATEGY AND BEAM SEARCH
We clarify these hyperparameters for each method, and also supplement pass @k curves under matched decoding policy (beam size = 5 for all methods) in Figure These two 'k's are two different hyperparameters. We sorry for the misunderstanding.
The 'k' in top-k is used for sampling in the RL roll-out process and computing the gradient with these samples.
The 'k' in pass@k k is for evaluation to generate all program candidates after all training process finishes. Especially, use beam search to improve the pass rate when decoding.
SparseRL SpMV
0.6
0.5
Pass@k
0.3
0.2
0.1
SparseRL-Codel13 (770M)
SparseRL--wenn? (78)
SparseRL--Qenn2 (149)
0.0
1000
Number of samples, k
Figure 7: Pass@k curve with number of samples (k) in beam search.
32

---

Published as a conference paper at ICLR 2026
A.24 SIZE-MATCHED AND BUDGET- SEDOGET-MATCHE BASELINES
We have supplemented two sets of controlled experiments to address this concern:
Size-matched comparison: SpausRL--CodeTS--700M vs. CodeRL=+CodeII--770M vs. PPOCodder-CodeeSS-770M (all using the same CodeT5-770M backbone):
SpMV pass@ 1000: SparseRL. (48.75) > CodeRL. (36.50) > PPOCoder (35.50)
Average speedup vs. cuSPARSE: SparseRL (+27.5%) > PPOCoder (-8.2%) > CodeRL
(-10.7%)
Budget-matched comparison: All methods use k=1000 sampling budget and identical decoding time limits (3 minutes per matrix):
SpMV compilation rate: SpanseRL (56.50) PPOCoder (40.75) CodeRL (39.50)
Wall-clock search time per matrix: SparseRI. (128s) CodeRL. (132s) R PPOCoder (125s)
These results confirm SparseRL's advantages stem from its framework design (embedding, reward function) rather than model size or sampling budget.
A.25 MEMORY PENALTY DETAIL
We elaborate the meaning of "excessive memory", the threshold & measurement, and the penalty rationale:
1. Focus on Excessive Memory:
We target shared memory as the key resource for the penalty in Eq. (10), given its critical role in sparse kernel performance.
2. Threshold & Measurement:
Limit Threshold: 48 KB (V100 GPU) 75% V100's 64 KB hardware limit, aligned with the 80th percentile of human-optimized kernels (e.g., cuSPARSE).
opttass- (compiler outputs shared memory usage for generated kernels).
Measurement: Extracted via nvec
3. Penalty Rationale:
Opaque penalties risk biasing the policy to avoid beneficial shared memory use. We only penalize usage >48 KB: exceeding this hardware limit disrupts thread scheduling, lowering SM occupancy (e.g., va201 0 drops from 87.48% to 53.03%, Table 9)
and slowing performance. Moderate use (<48 KB) boosts speed without penalty.
To better illustrate the speed vs. shared memory usage trade-off for each matrix (showing how speed changes with varying shared shared memory allocation for the same matrix), we provide multi-poin ppint data 3 representative matrices and key insights for visualization:
For each matrix, speed first rises with moderate shared memory use (e.g., 'nemeth22 gains 31 % speed from 16 KB to 36 KB) due to reduced global memory access.
Beyond the 48 KB threshold, speed drops sharply (e.g., 'pwt' loses 27% speed from 48 KB to 64 KB), validating the penalty design for excessive use.
This multi-point data directly shows the non-linear trade-off, making the threshold rationale (48 KB) visually intuitive.
A.26 PRIOR ART POSITIONING AND COMPARISON CONCERNS
We strengthen positioning against prior work like AlphaSparse, SMAT, and YASpMV, and this is critical to clarifying SparseRL's novelty. Below we address the gaps and outline actionable fixes:
Clarifying SparseRL's Fundamental Differentiation from Prior Work. While AlphaSparse mmarrix-oonitioone SpMV code generation) and auto auto-tuning methods (SMAT, YASpMV, SparseTIR) target sparse kemne optimization, SparseRI. enables key capabilities they cannot:
33

---

Published as a conference paper at ICLR 2026
Trade-off Between Shared Memory Usage and SpMV Performance
remethzz
+22353
1323
Speed (OPLOPS)
412
25
20
20
80
10
8
Memory (KB)
Figure 8: Speed-Memory Trade-off for Sparse CUDA Kernels NVIDIA V100
End-to-end hardware-aware optimization via RL: Prior auto-tuning (SMAT/YASpMVV relies on heuristic search over predefined kernel templates, and AlphaSparse focuses on code generation without integrating runtime efficiency + memory constraint rewards. SparseRL's RI. loop directly optimizes for correctness, GPU runtime, and shared memory limits in single pipeline-avoicee avoiding template bias and enabling adaptive kernel designs for diverse matrix sparsity (e.g., irregular vs. block-dense).
SparseRL's ' LLM-based generative approach enables superior generalization to unseen matrices, a key edge over prior methods. Unlike SMAT/YASpMV (which require re-tuning for new matrix distributions) AlphaSparse (limited by fixed templates), SparseRL uses matrix embeddings to teach the LLM transferable optimization patterns.
Comparison Dimension. These works are compared on the two apsects (performance and search/optimization cost):
On performance (Avg. GFLOPS), we replicate AlpuasporreSMMMA//AASAA//SSSS matrix splits and report SpMV speed vs. SparseRI..
On scarch/optimizztiion cost, we measure wall-clock time: (1) Heuristic search time (SMAT/YASpMV//SH) SSMATT/ASSM////paseeTRRR) (2) AlphaSparse's code generation + tuning time; (3) SparseRL's RL fine-tuning time (excluding pre- pre-training. which is one-time cost).
Head-to-Head Comparisons. To provide "apples-to-appe validation, will add the following experiments using our existing SuiteSparse (Ik matrices) and DLMC (500 matrices) splits, tested the same NVIDIA V100 GPU. Preliminary tests subset (100 SuiteSparse matrices) show SparseRL. outperforms prior work in trade-off:
Avg. GFLOPS: SparseRL (116.2) > AlphaSparse (98.5) YASpMV (87.3) > SMAT (82.1).
Optimization Time: SparseRL (1.2h for Ik matrices, one-time RL fine-tuning) SMAT (1.5h for 1k matrices) < YaSpMV (2.2h for Ik matrices) < AlphaSparse (3.5h, per-matrix search).
A.27 GLOSED-SOURGE LLM COMPARISON CONCERNS
The closed-soures models lack hardware-optimized RL loop and matrix embeddings, and their "higher correctness but poor runtime" reflects under-adaptation to GPU kernels, not superior RL designs. To isolate methodology impact, add two controlled experiments:
34

---

Published as a conference paper at ICLR 2026
1. SparseRL on Closed-Source Outputs. Feeding CeClaudd--ooneete kernels into SparseRL's RL loop (reward: efficiency + correctness by off polippl GRPO) improves runtime by 28-35% on 100 test matrices:
Model
Base Runtime (ms)
After SparseRL RL (ms)
Speedup +35%
Claude-Sonnet-4
87.6
56.9
GPT-5
92.3
66.5
+28%
Controlled Comparison. On LLaMA 70B (matching closed-sonee source scale):
Pipeline
Correctness (%)
Avg. Runtime (ms)
SFT-only (Baseline)
72.1
105.4
SparseRL (Full Method)
71.8
68.2
Sp' RL embedding design drives a 35% runtime improvement (vs. SFT-only) on the same model scale, confirming its methodology (not just model specialization) explains performance gains. Revised supplements include full results.
A.28 PRE-TRAINING SCALING CONCERNS
We notice the observation potential scaling challenges of pre-training with CUDA code augmenta- tion for larger open LLMs (beyond Qwen3-14B) and this a reasonable consideration. We address with clarity our pre-training paradigm and mitigation strategies below:
First, we acknowledge that pre- pre-training (even for domain adaptation) ineurs higher costs than fine- tuning, which is a common consideration for scaling to larger models. However, key distinction that we do not train a large LLM from scratch. Instead, we perform incremental domain-specinee pre-training: we take pre-trained open LLMs (e.g., Qwen3-14BB and augment their knowledge with CUDA code for sparse kernel optimization. This avoids the prohibitive cost training a large model from the ground up, significantly reducing the baseline scaling burden.
Second, regarding practical scaling to larger models (e.g., 70B-parameter open LLMs), we leverage the distributed parallelism strategies to control time and resource costs: Using data parallelism (across 8-16 V100/A 100 GPUs) and tensor parallelism (for larger model parameters), we can linearly reduce pre-training time-for for example, our Qwen3 14B pre-training takes 2-3 days 8 V100 GPUs, and scaling to a 70B model on 16 A100 GPUs would extend this to 4-5 days (not exponentially). which remains computationally feasible.
In summary, while scaling pre-training to larger LLMs requires more resources, our incremental pre-training paradigm (not scratch training) and parallel optimization strategies keep this feasible (with typical pre-training cycles of 2-3 days for 14B models), and manageable extensions to larger models via distributed computing.
A.29 PROMPT DESIGN/DETAIL AND RATIONALE FOR REMOVING LANGUAGE INPUT
We elaborate the prompt design:
In the pretraining stage, we use prompt from other systems (in our method, we use the public QwenLM repository), and use both their system and user prompt in pre-training.
In the SFT stage, use a progressive input prompt strategy, initially using a mixture of prompt text content and sparse matrix input, gradually removing the prompt text content.
Then in the RL stage, only sparse matrices inputted, like the modality conversion similar to that of converting an image to text (Wang et al. 2022).
Prompt detail: The selection of prompt text in SFT stage very diverse, and we use the prompt text from CUDA-LLM (Chen et al. 2025 LLM4EFFI Ye et al. 2025 OpenHands (Wang al. 2024 articles. The input consists two parts: text prompt, embedding of sparse matrix. The example is that:
35

---

Published as a conference paper at ICLR 2026
Listing 3: Initial prompt
You are an AI assistant tasked with writing optimized CUDA code to implement specified compatational operations.
Your code will undergo three critical validation stages: first, compilation verification using an external CUDA compiler to ensure syntactic correctness; second, functional validation with real-world test cases to confi rm accurate computation resulta; third, performance benchmarks rking to evaluate execution efficiency. These validation outcomes will drive an iterative reinforcement learning loop to continuously refine your code quality over time.
4
Therefore, your code must pricritize
core objectives:
1. Strict adherence to CJDA syntax rules and quaranteed functional correctness (matching expected computation results for all test sccnarios)
2. Maximized exccution efficicncy (leveraging CUDA parallel computing architecture, optimizing menory access patterns, thread block configurations, and minimizing kernel latency)
Below is a common CUDA code example for reference:
10
12
[A CUDA program example]
13
14
15
Please generate efficient CUDA code that meets the above requirements for the requested computational task.
16
17
[Frompt of SpMV task]
18
The output should
content whole oou file containing ONE kernel function, completing the reference code
20
21
below:
22
[Code|
Do not modity the test part.
You Listing 4: Prompt of SpMV task (markdown is readable for LLMs)
assistant tasked with writing optimized CUDA code specifically for the x*Sparse Matrix-Vector Multiplication (SpMV) task. Below is a detailed definition, implementation requirements, and guidelines to ensure high-quality, production-grade code:
### 1. SpMV Task Definition
SpMV refers to the computational operation of multiplying a sparse natrix (a matrix with high proportion of zero-valued elements) by dense vector, resulting in dense output vector. Formally, it is defined
Given sparse matrix \ Vin \mathbb/R) (m \times nt and a dense vector \ x \in (nathbb/A)"" V, compute the output vector \ y Vin \ mathbb(R) "m. A) where \( - A Nedot x \).
The core challenge SpMV lies in efficiently leveraging parallelism while mitigating the irregular memory access patterns inherent to sparse data lsttucturescritica for achieving high performance on CUDA-enabled GPUs.
### 2. Mandatory Implementation Phases
10 When designing the SpMV CUDA kernel, you +*must structure the code into three sequential phases** (aligning with standard high-performance SpMV design practices):
11 **Phase Format Adjustment**
36

---

Published as a conference paper at ICLR 2026
12 Prepare the input sparse matrix for GPU acceleration (e.g., convert/ compatibilize with GPU-optimized formats like CSR, ELL, coo, or hybrid formats such as CSR-ELL), Ensure format consistency with the Input data and minimize overhead for subsequent parallel computation.
13
Key considerations: Avoid redundant data copies, optimize host-to- device (H2D) memory transfers, and ensure format alignment. with thread block/grid contigurations.
14
15
**Fhase 2: Parallel Compute**
16
Distribute the matrix-vector multiplication workload
GPU threads Each thread/thread block is assigned to compute a subset of the output vector elements (e.g., one thread per row of the sparse matrix, or vectorized threads for dense row segments).
17
Key considerations: Maximize thread occupancy, minimize thread divergence, and align memory access with GPU memory hierarchies register, shared menory, global momory).
18
19
**hase Reduction (as necded)**
20
For formats workloads requiring partial aggregation (e.g., when multiple threads conpute contributions to the same output element), implement efficient parallel reduction.
21
Key considerations: Use shared menory to minimize global memory access, avoid warp divergence, and optimize reduction tree depth.
UFF 3. Ferformance Optimization Guidelines
24
To achieve state-of-the-art efficiency, your code must prioritize the following optimizations:
25
Load Balancing*4: Distribute workloads evenly
threads /blocks to avoid idle resources (critical for sparse matrices with highly variable row lengths).
26
*iigh Concurrency*:: Maximize CPU occupancy via optimal thread block
size (e.g., 128/256/512 threads per block), grid dimension tuning, efficient use of streaming multiprocessors (SMs). Memory Access Rfficiency++:
28
Minimize global memory latency by leveraging coalesces access (align
memory requests to GPU memory transaction in boundaries).
29
Resolve shared memory bank cenflicts (e.g., via padding, data indexing).
zeordering, or bank-conflict-fee
30
Use registers for frequently accessed variables and shared memory for data reused across threads.
31
Minimiize Overhead**: Reduce kernel launch overhead, avoid unnecessary
data transfers between host and device, and eliminate redundant computations.
32
33
### 4. Reference Example
34
Below is representative example of optimized CUDA SpMV kernel (C3R format) for reference:
35
36
IA CUDA SpMV example]
37
38
#*# 5. Output Requi rements Constraints
39
Your code **uus be provided as single ',cu' file+ (e.g.,
spmv_ www.opptiiizee.cu..
40
Do not modify any existing files in the file system, including test cases, input data loaders, validation scripts.
Ensure coipatibility with standard CUDA toolchains (CUDA 11.0+ recommended) for seaniess compilation with external compilers. 42 Frioritize **syntactic correctncss** (no compilation errers), functional accuracy.. (pass all real-world SpMV test cases), and peak performancer* (meet the optimization guidelines above).
44 Your code will undergo compilation verification, functional validation witn diverse sparse natrix test cases (e.g., unstructured, banded, symmetric), and performance benchmarking. Results will feed into
37

---

Published as a conference paper at ICLR 2026
reinforcement learning Loop refine future literationsso prioritize both correctness and efficiency.
45
F llasee generate the optimized CUDA SpMV code adhering to all the above 46
specifications.
47
sparse matrix input embedding
Rationale for removing language input:
We give the reasons for removing language input from
lwo aspects:
Modality alignment:
Sparse matrix structure is inherently non-linguistic. Using only index embeddings eliminates the modality gap between natural language prompts and structural input, enabling the model to focus matrix-specific patterns. like the modality conversion similar to that of converting an image to text (Wang et al.) (022).
Experiment: Ablation experiments show that removing language prompts increases pass 1000 by 7.3% and execution speed by 9.2% compared to using prompts like "Generate pigh-peeformance CUDA code for SpMV with the given sparse matrix".
Table 20: "Text Prompt" means that text prompt always exists in the SFT stage. "Text Prompt & No Text-prompt" means that initially using a mixture of prompt text content and sparse matrix input, gradually removing the prompt text content.
Metric
Text Prompt
NexText prompt
Text Prompt & No-prompt
SpMV pass @ 1000
45.75
18.25
49.25
SpMV Compilation Rate
53,00
26.50
57.50
Average Execution Speed (GFL.OPS)
106.01
60.36
116.20
38


Reviews:
[{"review_id": "3dGsKY95EC", "reviewer": "Reviewer_PAdt", "summary": "The paper proposes SparseRL, a framework to generate high-performance sparse CUDA code for SpMV and SpMM tasks. The authors employ deep reinforcement learning to fine-tune code models with a sinusoidal embedding for sparse matrix row/column indices. Experiments on the SuiteSparse and DLMC datasets demonstrate that SparseRL significantly outperforms strong baselines such as CodeRL, PPOCoder, and cuSPARSE in both correctness and efficiency.", "questions": "1. Have you compared SparseRL’s generated CUDA code against human expert implementations? Demonstrating superiority over human-optimized kernels would make the results more compelling.\n2. It would be better to try some other RL algorithms (e.g., GRPO, Reinforce++) to test robustness and potential further gains. Additionally, including a comparison with SFT baseline would help demonstrate the contribution of the RL phase.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. SparseRL provides a complete process of the training pipeline from pretraining to SFT and RL optimization. The idea of encoding sparse matrix structures using sinusoidal embeddings is elegant and theoretically well-motivated.\n2. The experiments are very comprehensive and convincing. It is particularly interesting to observe clear improvements in code efficiency, which is highly relevant in CUDA generation scenarios. The ablation studies are thorough too and leave few gaps.", "weaknesses": "1. My most concern is the paper’s organization. Many insightful analyses are leaved to the appendix. Reducing some descriptive parts (like section 3.1 and 3.3) and moving more analytical discussion into the main body would make the core contributions more clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hAFp1YIGie", "reviewer": "Reviewer_o7fk", "summary": "The paper introduces SparseRL, a domain‑specialized framework that treats a pretrained code LLM as a stochastic policy and fine‑tunes it with PPO to generate high‑performance CUDA kernels for sparse operators, focusing on SpMV and extending to SpMM. The model ingests the sparse matrix itself via a row/column sinusoidal index embedding, then emits CUDA; a compiler/executor environment supplies rewards for compilation success, functional correctness, and runtime, with a memory‑usage penalty. The RL loop (Figure 1, p.2) and the pretrain→SFT→RL pipeline (Figure 2, p.4) anchor the method. On 400 SuiteSparse test matrices (700 for training) plus DLMC, they report higher compilation and correctness rates and sizable speedups vs CodeRL, PPOCoder, cuSPARSE, and TVM‑S; e.g., for SpMV they claim roughly +20% compilation rate and ~30% faster runtime on average, with pass@1000 up to 49.25 and CR 57.50 (Table 1, p.8). Figures 3–4 visualize GFLOPS/TFLOPS gains on V100/A100. \n\nKey contributions\n\n* A DRL‑based, domain‑specific code generation framework that conditions on dynamic sparse inputs and uses compiler/executor feedback to optimize generated CUDA for per‑matrix performance (Figures 1–2). \n* A row/column sinusoidal embedding tailored to sparse matrices that maps non‑zero coordinates into the LLM’s input space to better capture runtime execution patterns. \n* A hierarchical reward that combines compilation pass, unit‑test correctness, runtime efficiency, and a memory‑overuse penalty, enabling direct optimization for latency. \n* Empirical SOTA on SpMV with strong generalization to SpMM: improved pass@k and CR, and consistent performance speedups vs strong baselines on V100/A100, backed by ablations (embedding choice, reward components) and profiling analyses (occupancy, memory traffic).", "questions": "1) Reward scaling and stability\n\nQuestion. What exact values do you use for the efficiency scaling factor `r_eff` and the memory penalty `r_penalty` in Eqs. (9)–(10), and how are rewards normalized across matrices with very different runtimes? Provide PPO diagnostics (KL, entropy, value loss) over training.\nWhy it matters. Reward scale drives PPO stability and whether speedups generalize beyond the training matrices.\nWhat would help. Precise hyperparameters, per‑epoch reward traces, and a short sensitivity study varying `r_eff` and `r_penalty`. See §3.4 and Eqs. (9)–(10), p. 6–7. \n\n---\n\n 2) Definition and computation of pass@k\n\nQuestion. In §4.1 you say pass@k uses “k (beam search) synthetically generated program samples,” but §3.3 says RL sampling uses top‑k with temperature. Which decoding is used for pass@k for each method and setting?\nWhy it matters. Beam vs sampling materially changes pass@k and comparability to prior work.\nWhat would help. A one‑line table mapping each model to its decoding strategy for pass@k, plus pass@k curves under a matched decoding policy. See §3.3 (top‑k sampling) and §4.1 (beam), p. 6 and p. 8. \n\n---\n\n 3) Budget and model‑size parity for baselines\n\nQuestion. Table 1 reports pass@1000 for some models but not others due to “environment limits,” and performance plots compare SparseRL with Qwen‑14B against CodeRL/PPOCoder using CodeT5‑770M “under k = 5000 on correct programs of partial matrices.” Can you provide size‑matched and budget‑matched comparisons?\nWhy it matters. Current results confound method quality with model capacity and sampling budget.\nWhat would help. Add a row (or supplementary figure) with SparseRL+CodeT5‑770M and CodeRL/PPOCoder+CodeT5‑770M under identical k and decoding; also report wall‑clock search time per matrix. See Table 1 and Figure 3, p. 8–9. \n\n---\n\n 4) Formal task definition\n\nQuestion. Eq. (2) requires the generated Ŷ to be in a set of “ground‑truth code” Y, yet your system synthesizes new correct programs. Will you revise the formalization or justify this assumption?\nWhy it matters. The mismatch makes the objective look constrained when the method is not.\nWhat would help. A corrected statement aligning with “synthesize any program that compiles, passes tests, and minimizes runtime.” See §3.1, p. 5. \n\n---\n\n 5) Matrix representation: permutation robustness and reordering\n\nQuestion. How robust is performance to row/column permutations and standard reorderings (e.g., RCM, AMD)? Did you test permutation‑invariance or include reordering as a pre/post step?\nWhy it matters. Absolute index sinusoidal embeddings are sensitive to ordering; SpMV performance is often improved via reorderings.\nWhat would help. A permutation/reordering robustness table and, if possible, a variant that augments sinusoidal embeddings with permutation‑invariant features. See §3.2 and the embedding ablation in Table 6, p. 5 and p. 20. \n\n---\n\n 6) Memory penalty details\n\nQuestion. What exactly constitutes “excessive memory” for the penalty in Eq. (10): registers, shared memory, global memory, or allocated buffers? How is the limit chosen and measured?\nWhy it matters. Shared memory can increase speed while lowering occupancy; an opaque penalty could bias the policy.\nWhat would help. Thresholds per resource, measurement method, and a small plot showing speed vs memory usage trade‑offs on a few matrices. See §3.4.3, p. 7 and SM‑occupancy analysis in Table 9, p. 23.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality\n\n* Frames sparse GPU kernel generation as an RL problem where a pretrained code LM is the policy and the compiler/executor acts as the environment. The hierarchical reward mixes compilation success, unit‑test correctness, runtime, and a memory‑overuse penalty, directly aligning learning with latency and resource goals. This closes the loop between code synthesis and low‑level performance feedback in a way that is rare for sparse kernels. See the pipeline in Figure 2 (p.4) and reward in Eq. 10 (p.7). \n* Introduces an input representation that conditions the LM on the actual sparse matrix: sinusoidal embeddings of row and column indices for nonzeros. That idea bridges the modality gap between sparse structure and code and enables per‑matrix specialization, rather than “one kernel fits all.” The mechanism and motivation are detailed in §3.2 and reinforced by the embedding ablation in Table 6 (p.20). \n* Combines pretraining on CUDA, supervised fine‑tuning for SpMV, and PPO fine‑tuning with on‑hardware feedback. The pretrain→SFT→RL stack is common in language tasks, but focusing it on sparse CUDA with dynamic inputs is a creative recombination that removes assumptions typical in dense‑kernel work. Figure 2 (p.4) and §3 lay out the stages cleanly. \n* Adds a pragmatic dynamic syntax checker during decoding (Appendix A.3) to terminate hopeless generations early, a small but thoughtful systems detail that improves training efficiency. \n\nQuality\n\n* Sound problem setup and thorough evaluation. Correctness is measured with pass@k and compilation rate; performance is measured as GFLOPS/TFLOPS on V100 and A100 with CUDA 12.1, using 1000 iterations to stabilize timing. Metrics and hardware details appear in §4.1 and Table 1 (p.8). \n* Strong baselines: compares against CodeRL and PPOCoder as learning methods and against cuSPARSE and TVM‑S as engineered libraries. On SpMV, SparseRL improves pass@1000 to 49.25 and CR to 57.50, and shows consistent speedups over both learned and library baselines; results are summarized in Table 1 (p.8) and Figure 3 for SuiteSparse on both GPUs (p.9). \n* Generalization beyond SpMV. The same machinery transfers to SpMM with competitive accuracy and clear TFLOPS gains for different dense‑matrix widths, shown in Table 1 and Figure 4 (p.9). \n* Careful ablations and diagnostics. The paper isolates the impact of the RL reward components, the sparse‑matrix embedding choice, and exploration settings, with clear trends in Figure 5 (p.19), Table 6 (p.20), and Table 7 (p.21). Profiling ties wins to reduced memory traffic and higher SM occupancy vs. baselines (Tables 8–9, p.23). This strengthens the causal story behind the performance improvements. \n* Transparency about limits and cost. The authors document failure cases on extreme matrices, compute overheads of RL fine‑tuning, and scenarios where benefit dominates cost, with concrete numbers (Appendix A.10–A.12). This raises confidence in the claims rather than overpromising. \n\nClarity\n\n* The problem and method are explained with clean, layered visuals. Figure 1 (p.2) gives a task‑level view with the environment loop; Figure 2 (p.4) shows training stages and reward flow. The mathematical setup in §3 is precise without being dense, and equations for PPO and rewards are easy to track (Eqs. 4–5, 9–10). \n* Experimental presentation is readable and decision‑relevant. Table 1 concentrates correctness and compilation outcomes across models, while Figures 3–4 summarize performance across matrix sizes. Appendices include small code snippets to illustrate why different sparsity patterns demand different kernels (Appendix A.6), which makes the motivation for per‑matrix specialization concrete. \n* Limitations and scope are clearly demarcated, including notes on backend portability and when offline search costs are amortized (Appendix A.12–A.13). \n\nSignificance\n\n* For the code‑generation community, the work shifts the target from “compiles and is correct” to “compiles, is correct, and runs fast for my data,” with measured improvements over vendor and compiler baselines on widely used sparse collections. That is a meaningful step toward data‑conditioned, performance‑aware program synthesis. See Figure 3 (SpMV speedups) and the stated averages vs. cuSPARSE/TVM‑S (p.8–9). \n* For practitioners in GNNs and pruned LLM inference, per‑matrix kernel synthesis that beats cuSPARSE on average can translate into tangible latency wins without hand‑engineering every corner case. The extensions to SpMM suggest the approach is not limited to a single operator (p.9). \n* The technique is likely to influence future research on learning‑to‑optimize kernels: the reward design is modular, the input representation is lightweight, and the analysis ties learned choices to classic GPU optimization principles. The profiling evidence in Tables 8–9 (p.23) makes the case that the system is learning the right low‑level instincts, not just gaming the benchmark. \n\nConclusion: A well‑executed, domain‑specific RL framework that conditions on sparse inputs and optimizes for runtime delivers real, measurable wins over both learned and hand‑engineered baselines, with clear analysis and honest limitations. The combination of ideas feels fresh for sparse GPU codegen and practically relevant to a growing slice of ML and HPC workloads.", "weaknesses": "1. Positioning vs. prior art is underdeveloped\n   The paper cites AlphaSparse (Du et al., 2022) and classic auto‑tuning work (e.g., SMAT, YASpMV), but there’s no direct, apples‑to‑apples comparison or a clear articulation of what SparseRL fundamentally enables that these matrix‑conditioned search/auto‑tuning systems cannot. Since AlphaSparse also generates SpMV code from the matrix, the novelty claim would be stronger with a head‑to‑head on the same matrices and GPUs, plus a discussion of search cost vs. RL fine‑tuning cost. Actionable fix: add baselines from AlphaSparse/SMAT/YASpMV/SparseTIR or replicate their search pipelines on your SuiteSparse/DLMC splits; report both performance and wall‑clock search/optimization time. \n\n2. Reward design is plausible but under‑specified and potentially unstable\n   The efficiency reward is a scaled time ratio against cuSPARSE (Eq. 9), added after ±0.5 compile/test terms and a memory penalty (Eq. 10). The paper does not specify the actual values of the scaling factor `r_eff` or the memory penalty, nor how reward magnitudes are normalized across matrices with very different runtimes. This matters because reward scale drives PPO stability, exploration pressure, and convergence. Actionable fix: report exact hyperparameters and show sensitivity curves for `r_eff` and the penalty; consider per‑matrix normalization (e.g., log speedup, z‑scored runtime) and reweighting between correctness and efficiency to demonstrate PPO stability across nnz regimes. Include training‑time reward traces and KL/entropy diagnostics. Figures 1–2 set the stage for this analysis but don’t provide it. \n\n3. Matrix representation ignores permutation issues and reordering\n   Sinusoidal row/column embeddings make the model sensitive to absolute indices. SpMV performance, however, is heavily affected by row/column order; classical systems often reorder to improve locality and load balance. The paper neither evaluates robustness to permutations nor includes reordering in the action space. Actionable fix: (a) report robustness under random row/column permutations and standard reorderings; (b) add an optional pre‑processing/reordering step as part of generation, or augment the embedding with permutation‑invariant features (row nnz histograms, bandwidth, blockiness) or learnable Fourier features, and quantify the gains (extend Table 6 beyond Raw/Max‑Min/Sinusoidal). \n\n4. Evaluation budgets are not fully comparable across methods\n   Table 1 reports pass@k with k up to 1000 for some models, but many baselines are “–” at k=1000 due to “environment limits,” and performance plots use Qwen‑14B for SparseRL while CodeRL/PPOCoder are shown on CodeT5‑770M (and performance is measured “under k=5000 on correct programs of partial matrices” for those baselines). This mixes model capacity and sampling budgets with method quality. Actionable fix: standardize sample counts, temperatures, and decoding schemes across all learned methods; provide pass@k curves and wall‑clock search time per matrix; repeat GFLOPS comparisons with SIZE‑MATCHED models (e.g., SparseRL+CodeT5‑770M vs. CodeRL/PPOCoder‑CodeT5‑770M) and again with SIZE‑MATCHED larger backbones. Clarify whether pass@k uses sampling or beam for each model. \n\n5. Closed‑source LLM comparisons conflate domain specialization with methodology\n   Tables 2–3 show better correctness for large closed‑source models but poor runtime, then attribute the gap to “lack of hardware optimization knowledge.” Those models were not adapted with your RL loop or matrix embeddings, so this primarily shows domain specialization matters, not that the RL formulation is superior. Actionable fix: run your RL framework on one closed‑source model’s outputs through the same compiler/executor loop where licensing permits, or at least run the identical SFT+RL recipe on an open counterpart with comparable size to isolate the effect of the RL+reward vs. model scale. \n\nThe work is promising, the claims would be more robust with (i) stricter budget matching and size‑matched baselines, (ii) reward/optimization transparency and stability evidence, (iii) permutation/reordering awareness in the representation or pipeline, (iv) broader hardware/backends, and (v) a concrete cost‑benefit curve for real deployments. Most of these are addressable within the current framework and would materially strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t5kdckVvmS", "reviewer": "Reviewer_mzXd", "summary": "This paper targets the problem of code generation for sparse matrix-vector multiplication (SpMV) and sparse matrix-dense matrix multiplication (SpMM) tasks. Their tool, SparseRL, solved the challenges of sparse programs and the shortage of domain-specific architectures by improving large language models to make them a stochastic policy. This policy generated different solutions before being judged in terms of functional correctness and execution efficiency. For code candidate generation, the policy model was pre-trained and fine-tuned to do this task. Next, at the Reinforcement learning stage, the proposed model will be optimized by the authors’ proposed reward function for execution efficiency, besides the correctness reward. The experiment was done on two tasks and compared with a set of well-known closed-source Large Language Models (LLMs). The authors have collected 1100 matrices with the largest amount of non-zero elements from the Suite Sparse Matrix Collection, with 700 for training and 400 for testing. The evaluation was performed using Correct functionality (pass@k) and compilation rates (CR) scores, with k as the number of generated programs as candidates per matrix. The accuracy results show that SparseRL can be built on various open LLMs, and SparseRL models can outperform other famous LLM-based approaches in terms of the pass@k score and the compilation rate score.", "questions": "- In Formula 8, do correctness and efficiency always contribute the same weight to the final score? It is interesting to see the ability of the model to generate more correct code or more efficient code based on the reward function. Authors can provide a coefficient score (from 0 to 1) and modify it to see the variety of quality in the generated code. For example, if R_final=0.9*R_Correctness+0.1*R_efficiency, a good RL model will tend to generate more correct code but be inefficient in terms of running time.\n- Will the approach be transparent and getting consistent accuracy improvement with other LLMs besides Qwen?\n- While this approach is good for matrix operations, will it be applicable to other general types of code optimization? I suggest authors should describe the potential of extending the scale of this application in camera ready version.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well written. Every figure and table is well described.\n- The novelty of the rewarding mechanism is the core contribution of this paper. The proposed reward score ensures the functional correctness and execution efficiency of the code.  \n- The idea of integrating LLM with RL makes sense to me since there has been trendy research on applying RL for AI. \n- Experiments were conducted with a number of good baselines.", "weaknesses": "- This work required a pre-training stage with CUDA code augmentation. Since the pre-training stage is an expensive process compared to fine-tuning, this work might have a scaling issue when pre-trained on bigger open LLMs than Qwen3 14B.\n- Lack of comparison between the sparse matrix embedding used in SparseRL and other code embedding techniques, such as UniXcoder [1] and GraphCodeBERT [2]. Although these models are not originally built for embedding a matrix, authors can design a simple flattening algorithm from a matrix to a sequence of tokens and compare the effectiveness of other embedding approaches to sparse matrix embedding.\n- The RL training dataset is on a small scale with 1100 entities for both training and testing.\n\nReferences\n1.UniXcoder: Unified Cross-Modal Pre-training for Code Representation. Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, Jian Yin.\n2.GraphCodeBERT: Pre-training Code Representations with Data Flow. Daya Guo et al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q9YZUCOvOr", "reviewer": "Reviewer_C533", "summary": "This paper introduces SparseRL, a training framework that leverages pre-trained language models to generate high-performance sparse CUDA kernels for sparse matrix–vector (SpMV) and sparse matrix–dense matrix (SpMM) operations on GPU. This method includes (1) a sinusoidal embedding of sparse matrix structures, (2) domain-specific pretraining on CUDA code, and (3) a hierarchical reward function design, and leverages the PPO algorithm for training. A compilation success gain and performance speedup gain is achieved compared with prior work.", "questions": "1. What's the prompt? The paper says, \"During the SFT and RL stages, the language input prompt is removed and only sparse matrices are provided as input.\" Why is this a good approach? In KernelBench, the Python code that demonstrates the functionality is provided. I think a better prompt should include a baseline GPU kernel and ask the model to optimize an existing implementation.\n2. The performance improvement over cuSPARSE is quite surprising to me. How large are those matrices? What's the GPU utilization?\n3. The ablation study of each individual part is unclear to me. Is pre-training necessary? Is SFT necessary? Is sinusoidal embedding necessary? Why aren't these results in the main paper?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Using LLMs for GPU kernel optimization is an important topic that is worth more research efforts\n2. Sparse matrix multiplication is a novel subset of problems in kernel optimization.", "weaknesses": "1. PPO is an old algorithm. Why isn't GRPO used?\n2. Important baselines, such as directly prompting gpt-5, gpt-o4-mini, claude-4-opus, are missing. The compared ones are GPT-3, Llama3.1, but these baselines are not very meaningful now.\n3. Is PPOCoder the right baseline to compare against? Is PPOCoder / CodeRL trained for GPU kernel optimization? It seems an unfair comparison.\n5. Writing suggestion: Subsection 3.4 is too long with little information. However, the appendix with more experimental results is not even discussed in the main paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)