ProSec: Fortifying Code LLMs with Proactive Security Alignment
Authors: Xiangzhe Xu
Zian Su
Jinyao Guo
Kaiyuan Zhang
Zhenting Wang
Xiangyu Zhang
Venue: ICML 2025 poster (2025)
Keywords: code language model
code generation safety
alignment training
Primary area: deep_learning->large_language_models
TL;DR: We synthesize dataset to systematically make code LLMs generate more secure code without harming correctness via offline alignment.

Abstract: While recent code-specific large language models (LLMs) have greatly enhanced their code generation capabilities, the safety of these models remains under-explored, posing potential risks as insecure code generated by these models may introduce vulnerabilities into real-world systems. Existing methods collect security-focused datasets from real-world vulnerabilities for instruction tuning in order to mitigate such issues. However, they are largely constrained by the data sparsity of vulnerable code, and have limited applicability in the multi-stage post-training workflows of modern LLMs. In this paper, we propose ProSec, a novel proactive security alignment approach designed to
align code LLMs with secure coding practices. ProSec systematically exposes the vulnerabilities in a code LLM by synthesizing vulnerability-inducing coding scenarios from Common Weakness Enumerations (CWEs) and generates fixes to vulnerable code snippets, allowing the model to learn secure practices through preference learning objectives. The scenarios synthesized by ProSec trigger 25× more vulnerable code than a normal instruction-tuning dataset, resulting in a security-focused alignment dataset 7× larger than the previous work. Experiments show that models trained with ProSec are 25.2% to 35.4% more secure compared to previous work without degrading models' utility.

PROSEC: Fortifying Code LLMs with Proactive Security Alignment
Xiangzhe Xu 1
Zian Su
Jinyao Guo
Kaiyuan Zhang'
Zhenting Wang? Xiangyu Zhang'
Abstract
Guo et al., 2024b). As these models are more widely de- ployed in productions, their safety becomes increasingly crucial. Insecure code generated by these models has been shown to introduce vulnerabilities, posing risks in real- world applications (Pearce et al., 2021; 2022). Recent stud- ies reveal that even state-of-the-art code LLMs frequently generate insecure code (He & Vechev, 2023; Bhatt al., 2023; He et al., 2024), highlighting the urgent need for the alignment with secure coding practices.
While recent code-specific large language models (LLMs) have greatly enhanced their code gener- ation capabilities, the safety of these models re- mains under-explored, posing potential risks as in- secure code generated by these models may intro- duce vulnerabilities into real-world systems. Ex- isting methods collect security-focusee datasets from real-world vulnerabilities for instruction tun- ing in order to mitigate such issues. However, they are largely constrained by the data sparsity of vulnerable code, and have limited applicability in the multi-stage post-traine post-training workflows of mod- ern LLMs. In this paper, we propose PROSEC, a novel proactive security alignment approach designed to align code LLMs with secure cod- ing practices. PROSEC systematically exposes the vulnerabilities in a code LLM by synthesiz- ing vulnerability-inducii coding scenarios from Common Weakness Enumerations (CWEs) and generates fixes to vulnerable code snippets, allow- ing the model to Iearn secure practices through preference learning objectives. The scenarios syn- thesized by PROSEC trigger 25x more vulnerable code than normal instruction-tuning dataset, re- sulting in security-focusee alignment dataset 7x larger than the previous work. Experiments show that models trained with PROSEC are 25.2% to 35.4% more secure compared to previous work without degrading models' utility.
Enhancing the ability of code LLMs to generate secure code necessitates additional design considerations in their post- training stages, similar to the alignment of general safety, truthfulness, and ethical considerations (Ganguli et al., 2022; Liu al., 2024c; Ji et al., 2024; Dubey et al., 2024; Hurst et al., 2024). Early efforts, such as SafeCoder (He al., 2024), seek to address security concerns during the instruc- tion tuning phase by constructing datasets of vulnerable code and corresponding fixes from GitHub commits. The security-focused dataset is then integrated with standard instruction tuning datasets to teach the pre-trained model to generate secure code while preserving utility. However, instruction tuning-based security alignment with real-world data faces two critical challenges:
Sparsity of Kerl-World Vulnerability Data. Vulnerable code snippets in real-world programs and their fixes are often sparse and highly contextual, limiting the effective- ness and generalizability of training secure code LLMs from real-world vulnerabilities. For instance, SafeCoder collects only 465 entries from 145 million git commits. One crucial underlying reason for the sparsity is that human program- mers have already avoided most insecure practices before commits so these processes never appear in web data.
1. Introduction
Limited Applicability in Post- Dost-Training Pipelines. Cou- pling security alignment with the standard instruction tuning phase restricts its utility in modern LLM training work- flows. Code L.MSs can undergo multi-stage post-training processes based on human/Al feedback for further perfor- mance improvements (Lee et al., 2023; Shao et al., 2024; Liu et al., 2024a). Reverting to the initial instruction tuning stage for security alignment necessitates retraining, which is resource-intensive and risks discarding the benefits of prior post-training efforts.
Large language models (LLMs) capable of generating code based on human instructions have revolutionized program- ming by significantly facilitating tasks such as code genera- tion (Zhu et al., 2024) and refinement (Zheng et al., 2024;
Eqqua contribution 'Department of Computer Science, Pur- due University, IN USA 'Department of Computer Science, Rut- gers University, NJ, USA. Correspondence to: Xiangzhe Xu <xul415@ muul415@purdu..com purduc.edu.. <du>>, Zian Su <su284 @
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
In this paper, we propose PROSEC, a proactive security

---

PROSEC: Fortifying Code LLMS with Proactive Security Alignment
alignment approach to improving the safety of a code LLM that has been post-trained with substantial efforts. It fortifies code LLMs systematically by intentionally triggering and resolving vulnerabilities during post-training PROSEC ex- poses the weakness of a code L.L with synthesized coding scenarios. samples from the target code LLM all nor- mal code, vulnerable code and the corresponding fix under different generation contexts to construct preference data, and aligns the code LLM to secure coding practices with preference learning objectives, minimizing negative effects to its utility.
dresses security risks during code generation.
We develop an automatic pipeline to synthesize and select proactive security alignment data given code LLM and vulnerability types in programming lan- guage.
We publish a dataset of synthesized vulnerability-- inducing instructions that can effectively expose the weakness of code LLMs. PROSEC and the dataset are available at https: : / //g com /Pr ProSec. The dataset is different from existing datasets that mainly include (vulnerable) code snip- pets, allowing easy customization to the distribution of target code LLM.
To address the challenge imposed by the sparsity of vulner- abilities in real-world code repositories, PROSEC leverages the power of prior knowledge from human and synthesized data. The key observation of PROSEC is that the Common Weakness Enumerations (CWEs) (MITRE, 2023), which abstract diverse program vulnerabilities, offer a generaliz- able foundation for simulating how vulnerabilities manifest across various coding tasks and programming languages. Specifically. PROSEC synthesizes instructions that may ex- pose the weakness of code L.MM by incorporating CWEs into a standard code instruction tuning dataset with a gen- eral LLM. Then, these instructions are further leveraged to synthesize data for security alignment training.
Through targeted security alignment, we demonstrate that PROSEC improves the ability of code LLMs to gen- erate secure code without compromising their general code generation capabilities, across multiple models, languages, and vulnerability types.
2. Background and Problem Formulation
In this section, we introduce the background and how we formulate the security alignment of code LLMs.
To address the second challenge, PROSEC assumes fully post-trained target model and adopts an additional prefer- ence optimization stage for security alignment, without any intervention in previous post- -st-trrining stages. Given the synthesized vulnerability-indin induuiint and normal instruction datasets, PROSEC constructs preference data for both se- cure coding practices and utility preservation. Moreover, PROSEC incorporates heuristic and training dynamics bynadics-based data selection, leading to a unified high-quality preference dataset. Due to the generality of preference data and the independence of the extra alignment phase, PROSEC can be easily integrated into various post-training pipelines.
Code LLM Consider an instruction following code LLM To (g) I (Wl)). z) that takes user instruction x and generates the code response y. Notably, wy is post-trained for multiple stages with non-trivial efforts after pre-training
Security-Related Coding Practice To ensure safe usage, the code LLM my needs to effectively incorporate the under- standing of certain security-related coding practices (e.g.. sanitizing inputs to prevent command injections). These practices address a range of commonly encountered is- sues that, if neglected, can render code vulnerable to ex- ploitation, A widely recognized framework for categoriz- ing such issues is the Common Weakness Enumerations (CWE) (MITRE, 2023), which associates each identified weakness with of recommended safe coding practices and common pitfalls to avoid. We denote the set of all pro- gramming language I and CWE combinations of interest as Dewe = (110) ((()),(())))) c( )
Empirically, the instructions synthesized by PROSEC induce 25 times more vulnerable code than a standard instruction tuning dataset. The alignment dataset generated by the proactive approach is 7 times larger than the SafeCoder dataset. We demonstrate the effectiveness of PROSEC on the ParpleI.lama (Bhatt et al., 2023) secure coding benchmark. The models trained with the dataset synthesized by PROSEC 25.2%-35.44 35.4% more secure than those trained with the SafeCoder dataset. We further validate that PROSEC does not harm the utility of code LLMs. We conduct thorough ablation studies to justify the design decisions in PROSEC.
Following previous work (Bhatt et al., 2023), we assume that there exists a static analyzer (Bennett et al., 2024; Wu et al., 2024; Mukherjee et al., 2022: Tang et al., 2024a; Wang et al., 2024a; Meta, 2025; Weggli, 2025) as oracle to detect whether a snippet of code follows secure code patterns. Specifically, the static analyzer takes as input a code snippet, and outputs a list of detected CWEs. An empty output list implics the given code conforms with the secure coding practices of this organization. Formally, we denote
Main Contributions Our work makes the following key contributions:
We introduce a novel post-training security alignment process for code L.Mss which systematically ad-
2

---

PROSEC: Fortifying Code LLMs with Proactive Security Alignment
Step1: Synthesize Vulnerabülity- Inducing Instructions
Step2: Construct Candidate
Step3: Select High-Quality and Relevant Data Samples
Preference Data
Xa
Normal Instruction
Dnorm
From Code LLM
Create a Java program that takes string input and finds the first non repeating char.
Normal Code
steIn angy[A];
Training Dynamics- Based Sampler
(lang. cwre Dowe
out.printinn"In: strIn);
What CWE-779 and the related program scenarios?
From Code LLM
Fixed Code
From LLM
Dhorm
CWE-79 malicious scripts injected into web pages...wher when inputs embedded into web pages without escaping. in, Java, it often occurs when
strIn reg reg.gettarreee www.geetaaaaanmc ("imputt )
ProSec Dataset
strSafe
sanitaze(strIIn
out.write ("In: strSafe
>"bbr/>"">
Inst
Code
Code
WB From LLM
Generate coding tasks where developer may make mistakes about the CWE: Get inspiration from
Vul. Lodl-Ind.
Flxed Code
Vul Code
Inst
From Code LLM
Vulnerable Code
From LLM
strIn
reg.getParanç (aeeraannnnntttt
Disc
Vulnerability-Indde Inducing Instruction
Java web application
allows
out.write
steIn
"<br/>");
Heuristic Filter
input string ... display the first repeating char along with the original input
Dec
Figure 1. PROSEC's data synthesis and selection pipeline. (1) The instruction synthesis stage takes as input a normal coding instruction and (language, CWE) pair, and produces vulnerability innduuiiniiiy-idduiiingg instruction that may trigger the corresponding CWE. (2) The preference data collection stage samples normal code and vulnerable code snippets from the target model given the normal and vulnerability-inducing instructions respectively. The corresponding fixed code snippets additionally sampled from target model given the vulnerable code and other feedback. The vulnerable instruction, vuinerable code, and fixed code results in Diee in the red box. Normal instruction, normal code, and fixed code results Dacm in the blue box. (3) The data selection stage leverages a heuristic filter and training dynamics-basee sampler improve the quality of data in Dee and Daoma respectively and produce the final preference dataset.
the static analyzer as follows with y denoting code.
tion enables seamless integration with many existing post- training pipelines and avoids retraining.
8 :: 0 Dewe U D2 ....DN
3. PROSEC: Proactive Security Alignment of Code LLMs
Security Alignment Code LLM The goal of security alignment in code LLMs is reduce the likelihood of gen- erating insecure code while preserving its ability generate functional code that follows user instructions. We consider the security alignment of code LIM as an additional offline preference optimization stage conducted after the main train- ing process. This stage leverages a preference optimization objective under the Bradley-Terr (BT) model (Bradley & Terry, 1952; Rafailov al., 2024): given the dataset D,, the optimization process minimizes preference loss function La X Vi Di- + R.
In this section, we introduce PROSEC. At high level, PROSEC is a systematic way of synthesizing and selecting data for the preference optimization of code LLM to guar- antee secure code generation while preserving utility. An overview of PROSEC's data synthesis and selection pipeline is shown in Figure We discuss how PROSEC synthesizes vulnerability -induuii instructions in Section 3.1, how it constructs candidate preference datasets in Section 3.2, and how to control the quality of the final alignment dataset via specialized data selection in Section 3.3.
8* = aargmin
Σ (E
Lo(x, Twe v1),
(2)
3.1. Vulnerability-IIduuing Instruction Synthesis
where x, Yws and Y denote prompt, a preferred/win re- sponse, less preferred/lose response. Such formula-
PROSEC's data synthesis begins with high-quality vulnerabilty--iduucig instruction dataset, intended for later
3

---

PROSEC: Fortifying Code LLMS with Proactive Sccurity Alignment
Algorithm 1 Vulnerability-nnddcing instruction generation
to K clusters. Only the centroid of each cluster is included in the final instruction dataset, as denoted by line 10 in AL- gorithm 1. Figure 4 empirically shows that the distribution of the instructions is more diversified after clustering.
input Dowe: a set of CWEs, I: a standard instruction dataset
output V: set of vulnerabiliy--indccing instructions. Each entry contains (, C,,,,, e. denoting the program- ming language, the CWE, the normal instruction, and the vulnerability-induee
3.2. Candidate Preference Dataset Construction
instruction, respectively.
Given the synthesized cllnerabblity-inddcinn instruction dataset V and the original instruction dataset I, PROSEC samples two types of candidate preference data from the tar- get model Te: sec intended to increase the model's s ability to generate secure code, and Doorm to preserve the model's utility.
1: V -- 0
2: for 1,c D Dowe do
owe definition((,)) 4: I. --reeevant.iassruuttoo.....
3: scenario + query cwe
5: Vo x-0 00
6: for In E EI, do
7:
Ee - co scenario,,
Secure Practice Preference Data Each data sam- ple in the secure coding practice preference dataset Dxc =(ab" (() ye ) consists of Ie, denoting a vulnerability-inducing instruction, Ye. the vulnerable im- plementation, and yy, the counterpart of ye but with the vulnerability fixed.
8:
Vo + & u [1l, c,
9:
end for
10: V+ V- (1 K)
11: end for
sampling of code responses. Existing large-scale coding instruction datasets for standard programming tasks (Wei et al., 2023; BAAL, 2024) are insufficient for this purpose, as many CWEs arise from highly specific coding scenarios underrepresented in these datasets. For instance, CWE-79, illustrated in Figure 1, refers to Cross-Site Scripting, where user inputs are embedded into web pages without proper sanitization, allowing attackers to execute arbitrary code in victim's 's browser. To reveal a code LLM's limitations in addressing CWE-79, tasks must involve writing web applica- tions. Empirical evidence (Figure 3) shows that only about 0.7% of a standard pnrrucctio-tuninng dataset can trigger CWBs.
PROSEC samples both yo and yy from the target model to minimize the negative effects on the model's original distribution during alignment. An important observation in our experiment is that post-trained model is able to fix an insecure code snippet, given the vulnerabilities identified in the insecure code, even though it makes mistakes with only the vulnerability-induce observation, PROSEC first collects vulnerable code snippets by sampling the target model's response on the vulnerability- inducing instructions. Then it asks the target model to fix the identified vulnerabilities.
instruction. Based on the
Specifically, given vulnerabilityinndeein instruction Fy. PROSEC samples multiple responses from the target model To- Then, it uses the static analyzer S to check poten- tial insecure coding practices from these responses. For each identified insecure code snippet y, E {y)S(y) * 0 A y ~ Te (yl x.)]. PROSEC queries the target model with both the code and the knowledge (language, CWE, and identified issue) about the identified vulnerability, instruct- ing the target model to fix the code. Similarly, PROSEC samples multiple responses from the target model and uses the static analyzer to select the secure fixed ones y5 E {y)S(w) = 0 A y To ( No. c)). The final Dece includes multiple paired y/ and yos, which will be selected later, for each instruction Zy.
PROSEC address the problem by incorporating prior knowl- edge of secure coding practices, the CWEs, into the instruc- tion synthesis process. We describe how PROSEC synthe- sizes yulnerability-inducci
instructions in Algorithm 1. Given a programming language and a CWE, PROSEC queries general wwwwledgeintenssine LLM to enumerate program scenarios that might trigger the CWE in the cor- responding language (line 3). In addition, PROSEC selects the normal instructions that are relevant to the programming language from the instruction-tuniin each relevant normal instruction, PROSEC then instructs a instruc-
dataset (line 4). For
general LLM to compose the vulnerability-nddue tions by combining the normal instruction with the program scenarios that may trigger the vulnerability (line 7). The red block in the left part of Figure 1 shows concrete example. The prompts used are in Appendix B.
Note that alternative design is to use the static analyzer identify both ye and ys from the responses to vulnerability- inducing instruction, instead of fixing ye to get yy. We show an example in Appendix D.2.
We noticed the lack of diversity in LLM generated coding scenarios in our preliminary experiments. Hence, we sam- ple multiple answers for each query with a high temperature, and cluster all instructions relevant to a language and CWE
Utility Preservation Preference Data Empirically, we find that the aligned model may undesirably overempha- size features that only appear in the win samples xz of
4

---

PROSEC: Fortifying Code LLMS with Proactive Sccurity Alignment
(Fee Wy, Vy,Vo) D seo when only trained on secure prac- tice preference data. For example, suppose that the API sanitize in Fixed Code of Figure 1 only appears in fixed code snippets (i.e., yr). A model trained exclusively with Desc may overemphasize this API, incorporating it in all implementations regardless of the programming context. That is undesirable because the sanitation would cause un- expected behavior for normal coding tasks that print strings to the command line.
Here, ʃ and g are defined as.
[I yo ys, 0) - (( yo 0)
(5)
g(Tn, In. Vy. 0) (( Vy. 0)
(6)
where r(z, 0) = to log (1)l2)) and (Iz, My, yo) E Deec is the corresponding secure practice data. The influence score of each (In, yn,, Ur) w.r.t (Zv: M.r. Mo) is therefore
Inf(In: Un. Vf, Tv. Uv) corr corr(mm.mm)) m,y)
(7)
To mitigate the problem, we propose to create companion (x) (8) dataset Doom= = {(z1)' yn Vy )imm for Deec- Duorm con- sists of normal instructions Vn, win responses yo and lose responses y/. 2n, is the normal instruction corresponding to the uulnerability-inducin instruction The and its response Ve- Such preference data strengthens normal response under normal instructions, while suppressing the likelihood of ys being generated in normal scenarios, thus preserving utility. Similar to Deec. we also collect multiple response pairs for each instruction in Doom for later selection which we will discuss next.
where co is the Kendall Tau correlation (Kendall, 1938), which is rank-based and relatively more robust.
We use Inf(() to select top-ranking candidate (#n, Vo, Vf) given (I,, yf V/.) to obtain P The intuition behind this data selection paradigm is that the most influenced is the most influential for utility preservation. The dynamic
r( Vv, 0) in denotes how the model perceives nor- mal instructions and responses across {01.... Or). and (( M) 0} in g denotes how the model becomes more aligned to secure coding practices. As the checkpoints are obtained by the warm-up training with D Deec. a strong cor- relation between f and g. e.g. r(Fn. 3/. 0) is decreasing while r(zv. My: 0) is increasing, potentially indicates that target model's ability to generate yn, given Ix is influenced by learning to generate y/ given Iv. in other words yn and yʃ are quite relevant conditioned on Σn,- Therefore, we need to add such utility preservation data in the final dataset to surgically prevent overfitting. Empirically, we find this strategy quite effective in achieving both security and utility.
3.3. Preference Data Quality Control
We propose heuristic-based data selection process for Deec and training dynamics-basse based one for Duom to control the quality of the final preference data for alignment training.
Die Selection with Heuristics For code responses in Die. we first use AST parsers perform a light-weight check on code syntax. We discard code snippets that have syntax errors. As discussed in Section 3.2, we use the static analyzer to ensure the fixed code snippet does not contain vuinerabilities. Moreover, we find the target model may skip unchanged code blocks when generating fixed code snippets. We use keywords (c.g.. "remain unchanged") and minimal length threshold to filter out that noise. Finally, we increase the data diversity by de-duplicating data en- tries with similar fixed code. We use fuzzy ratio (based on Levenshtein-distance to measure similarity.
Final Preference Dataset The final dataset for preference optimization is the shuffled mixture of D and D momm'
4. Experiment Setup
Seed Instruction-Tuning Dataset We use the code- related part of Infinity- Instruct (BAAI, 2024) our seed instruction dataset for data synthesis.
Static Code Analyzer We adopt the static analyzer com- monly used by previous work (Bhatt et al., 2023; Liu et al., 2024b) to detect insecure coding practices.
Drom Selection with Training Dynamics PROSEC cap- tures the influence of each (In, Vu Vy) E Daomm by com- puting the correlation between two measures w.r.t. training dynamics. Specifically, we first obtain series of check- points (0,.... 0r} by performing preference optimization on we with the full Dice as a warm-up dataset. Then we compute the following my and m, which are defined as two sorts of training dynamics in our scenario,
Test Dataset We use PurpleLlama (Bhatt et al., 2023) as the test dataset for code model safety. PurpleLlama provides set of instructions that may trigger errors from code LLM. We select 38 (language, CWE)s from Purplel laama that are overlapped with SafeCoder, corresponding to 694 test cases. We use the multi- multi-lingual version of Humaneval (Chen et al., 2021; Guo et al., 2024a) and the multi manlt--lingul version of MBPP (Austin et al., 2021) (denoted as MXEval (Athi- (4) waratkun et al., 2022)) as the test dataset for utility.
m,- = [f( 01), f( 0T) my = [g( 01),
(3)
, .6 0r)]
5

---

PROSEC: Fortifying Code LLMS with Proactive Security Alignment
Following the setup of Parp we generate multiple samples for each test instruction, and caleulate the ratio of secure code among all generated code samples. We use pass@) @1 I (Chen et al., 2021) as the metric for utility.
Metries
0.50
MXEval
0.48
ProSec
SafeCoder
0.46
Original Model
Models and Baselines We use claude -. 3.5 1..5-haike as the general LLM in our data synthesis pipeline. We syn- thesize 10k instructions for each CWE and select the most diverse 2k instructions via clustering. The cost to synthe- size instructions for each CWE is around USD. We use (Abdin
0.44
4
0.50
0.62
0.64
0.66
0.68
0.70
Secure Ratio
two well post-trained target models, Phi3-mini- et al., 2024) and Codel In-s (Rozière et al., 2024) in our evaluation. We compare PROSEC with previous SOTA Safecoder from two perspectives. First, SafeCoder is a security- -ware pntrrucciiottuninng technique. We therefore compare the CodeLLama-78 instruction-uun sutnionn-tuedddee by SafeCoder with the CodeL-lama-77 aligned from CadeLJlam--7BIInss with the dataset synthesized by PROSEC. Second, Safe- Coder comes with a dataset constructed from real-world vulnerability and fixes. We compare the effectiveness of the SafeCoder dataset with PROSEC synthesized dataset by using both datasets at the alignment stage.
Figure 2. How safety and utility of code LLMs change while aligned with different datasets.
PROSEC dataset achieve the most secure results. Specifi- cally, the models aligned with PROSEC more secure than ones aligned with SafeCoder by 25.2% (33.47 v.s. 44.72) and 35.4% (26.04 v.s. 40.33). That demonstrates PROSEC effectively synthesizes higher-quality data for secure code alignment.
Moreover, for models aligned from CodeLlama-78 BIstt can observe that the model aligned with the PROSEC dataset achieves better performance (26.04 v.s. 42.88) than the SafeCoder-Inst model that uses the SafeCoder dataset at the instruction-tuninn stage. It demonstrates that enforcing secure coding practices to post-trained model at the align- ment stage more effective than incorporating them at the instruction tuning stage.
Optimization not otherwise specified, we use SimPO (Meng et al., 2024) the default preference op- timization objective in PROSEC to optimize the model,
[(0)- = -E (a..ww-))) &
(8)
log
3 [Ball
log To (bwlz))
B 3/
log To oo (z) - -7)
Effects on Model Utility For both Phi3-mini-Ins and CodeLlama- models aligned with PROSEC, we can see that their utility performance has no significant down- grades. By contrast, their performance is slightly better than the original model. The improvements on utility might come from the higher complexity of security-relater program- ming scenarios than the ones in typical instruction-tuninn dataset, facilitating models' performance on more challeng- ing tasks. Moreover, we can see that for most cases, models aligned with PROSEC have better utility performance than the models aligned with the SafeCoder dataset.
where |3 and 7 are hyperparameters. We also include exper- iments with other objectives in Appendix D.3 to show the generalizability of the data.
Warm-up Training for Influence Score We train each target model on Dzee for 1k steps and leverage checkpoints of every 100 steps to compute the training dynamics for Dnorm data influence score computation.
5. Results
We further study the effects of alignments both PROSEC and SafeCoder dataset by visualizing the training trajecto- ries of both alignment training processes. The results are in Figure 2. Specifically, we collect 10 checkpoints for the Phi3-mini-Inst models aligned with the PROSEC dataset and the SafeCoder dataset, respectively. The utility performance is measured by the pass@ on the MXEval dataset, and the safety is measured by the ratio of secure code generations on the Purplel dataset. Due to resource limitations, we randomly sample subsets of both the MXEval and the PurpleLlama datasets. We can see that for most checkpoints, models trained with PROSEC are consistently more secure than ones trained with SafeCoder. Meanwhile, PROSEC
We report the main results with PROSEC sampling the top-2 influential among all candidate utility preservation prefer- ence data (Ix, Vas 35) for each corresponding secure prac- tice data (I yo) and further discarding the universally least 20% influential ones within the remaining data. The setting is the same for both Phi3-mini-Inst and Codel lama-- 7B-Inst's main experiments.
Our main results with regard to secure code generation and utility are shown in Table 1.
Secure Code Generation We can see that for both Phi3- mini-Inst and
models aligned with the
6

---

PROSEC: Fortifying Code LLMS with Proactive Security Alignment
Table 1. Evaluation results for secure code generation and multilingual utility. First three rows denote models aligned from Phi3-mini- mini-Inst and the following three rows denote models aligned from BodeLlama--B-Ing PROSEC denotes the alignment dataset is synthesized by PROSEC while SafeCoder denotes the dataset is the SafeCoder dataset. The last row denotes the CodeLlama-7B instruction-uun -unde with
the SafeCoder dataset.
Vulnerable Code Ratio (%, !)
HumanEval-Mullt (%, f)
MXEval (%. f)
Model
C
C++ Java
PY
Avg.
C/C++
Java
JS
PY
Avg.
CIC++
Java
IS
PY
Avg.
Phi3m-Inse
72.17 30.26 63.56 52.24 34.63
50.57
27.30
19.67
31.38
51.22
32.39
37.35
41.20
37.77
45.79
40.53
w/ SafeCoder
66,46
22.95
59.76
47.74
26.69
44.72
24.55
18.84
23.43
48.91
28.93
40.13
40.80
38.66
47.55
41.79
w/ PROSEC
44.27 20.74 49.09
28.21
25.05
33.47
29.74
18.39
32.36
56.11
34.15
39.00
39.26
50.10
47.75
44.03
CLM-7B-Inst
67.21 43.57
63.46
51.12
34,83
52.04
20.15
25.75
24.32
29.30
24.88
33,92
37.98
38.77
27.39
34.52
w/ SafeCoder
56.92 28.98 98 54.73
41.31
19.73
40.33
24.22
29.68
25.63
31.52
27.76
35.50
38.28
38.96
28.59
35.33
w/ PROSEC
32.50
16.67 26.94
34.10
20.00
26.04
23.96
29.35
31.49
31.27
29.02
37.43
40.33
44.04
37.78
39.89
SafeCoder-Inst
63.96 29.64
48.93
47.74
24.14 42.88
19.83
10.62
21.74
26.80
19.75
28.30
31.51
33.75
32.20
31.44
Triggered Inotance
2000
Syn-Trigger Ori OrigTrigger
0.7
Aog. Similarity after Clustering
1500
1000
500-
Syn, Aug...ug.... Avg. 30.8.3 30..75.
Ori, Avg.: 11.90
LLng, CWE)
Figure Synthesized instructions induce more CWE instances. Each bar denotes the number of vulnerable code instances that trigger detector for given language/CWE. We can see that synthesized instructions induce significantly more vulnerable code instances from the code LLM.
0. 0.5
0.6 Avg. Similarity before Clustering
0.7
Figure 4. Effectiveness of instruction clustering. Each data point denotes set of synthesized coding instructions for a lan- guage/CWE. A larger average similarity indicates lower diversity. We can see that the instructions after clustering are significantly more diversified (i.e., have lower average similarity).
models achieve better utility performance than the Safe- Coder models. That demonstrates PROSEC dataset is more effective than the SafeCoder dataset.
In all, both PROSEC and SafeCoder have limited effects on model utility, while PROSEC is more effective on the model safety.
results are visualized in Figure 3, demonstrating that the syn- thesized instructions indeed induce more vulnerable code snippets.
6. Analysis
Ablation on Instruction Clustering We study the effec- tiveness of the instruction clustering by measuring the aver- age similarity between all coding instructions for both the instructions before and after the clustering. The results are shown in Figure 4. We can see that the instruction clustering process indeed makes the synthesized data more diverse.
In this section, we study the design decisions in PROSEC. Due to resource limitations, the evaluation for safety is on a randomly sampled subset of the PurpleLlama dataset.
Generalizability to Different Models We evaluate the generalizability of PROSEC by applying align three additional models. The results show that PROSEC can con- sistently improve the security of generated code without harming the utility of an aligned model. Details are in Sec- tion D.1 of the appendix.
A blation on Duorm and Its Sampling We compare the proposed Duom data selection approach with random sam- pling. Specifically, we fix the D in the final preference dataset and sample the same ratio of D nomm for compari- son. As shown in Table 2, we can see that with low ratio of 0.1, the utility of the target model drops significantly, indicating the significance of Dnorm to utility preservation. For different ratios of Duoem- we can see that PROSEC's sampling leads to more secure models compared to random
Instruction Syn- thesis We illustrate the effectiveness of vulnerability- inducing instructions by showing that they introduce more vulnerable code instances than the original instructions. The
Ablation on Vulnerability
7

---

PROSEC: Fortifying Code LLMS with Proactive Security Alignment
Table 2. Effectiveness of the data selection algorithm and Domm- Random and PROSEC denote the random selection strategy and the data selection algorithm used in PROSEC, respectively. The second column denotes the ratio of sampled examples from Doorm-
Table 4. Comparison iterative refinement. We implement an iterative refinement baseline that employs a static analyzer to verify the security of generated code and instructs the code LLM to revise any insecure code. We use Phiß3-min-IIm as the code LLM. Max Iterations denotes the maximum number of revision attempls the system performs to address insecure code.
Strategy
Doom ratio
Vul(%, 4)
Util(%;, f)
Random PROSEC
0.1
6.02
12.30
0.1
5.92
15.28
Max Iterations
PROSEC
Random PROSEC
0.3
32.78
41.84
3
10
0.3
27.54
42.13
Vul (%6.)
31.4
26.3
19.4
25.0
Random
0.7
30.92
47.26
PROSEC
0.7
25.58
45.12
Table 5. Ablation on the effects of Duom and the size of datasct. Original denotes the performance of the subject model before secu- rity alignment. SafeCoder and PROSEC denote the model aligned with SafeCoder and PROSEC dataset, respectively. + DDmone denotes the model aligned with a combined dataset of SafeCoder examples and Darmm examples drawn from PROSEC. Subser denotes the model aligned using a randomly selected subset of PROSEC that matches the SafeCoder dataset in size.
Table Ablation on which measure to be used for training dynamics-basee data influence computation. The 0 in r(z, y, 0) is omitted here.
f
g
Vul (%,4) Util (%..1)
r(z., yn) r(an, Mn) r(In; 85)
-r(z., vs)
23.02
45.94
rre Us)
23.17
45.94
r(Zn, Vn) - :(Tn:U1)
DECREASE
27.12
46.44
Setup
Vul (%.J)
Util (%.1)
Original
40.8
42.8
SafeCoder
33.1
43.4
sampling, with comparable utility performance. Moreover, observe that the effectiveness of the data selection approach is more prominent when fewer normal data samples are selected (i.c.. lower sample ratios), demonstrating its capa- bility in identifying important data samples.
+Dnorm
34.4
46.1
PROSEC
25.0
45.2
Subset
28.9
47.0
ple queries to the code language model. This design could degrade the user experience in scenarios like code copilots, where swift completions are expected.
Ablation on Training Dynamies Options We also com- pare different options of training dynamics for the influence score computation. As shown in Table 3, we ablate on both f and g. For f, as target model with good utility should also NOT prefer response Uf given input Tn, so the -r(xn. By) term can potentially be added to r(Zn: Un) as an alternate f. Results show that this alternative has simi- lar performance to the default one. For g. we experiment with "DECREASE" alternative. Under the context of rank correlation (Kendall, 1938), we denote "DECREASE" as any sequence of monotonic decreasing values. Such correla- tion influence score only captures the degrading of utility preservation data but not security. Therefore, we can see that this g leads to the best utility but the worst security in the table. As the major concern in our scenario is the security of the target model, we choose f = r(Ens ya), g = -r(x0, my) as our final measure.
Ablation on the Effects of D Poorm and the Size of Dataset We study whether the security enhancement of PROSEC is confounded with examples in Duom or the size of the dataset. The results are shown in Table 5. Observe that the security performance is similar for models aligned using SafeCoder (33.1) and SafeCoder mixed with Dnom (34.4). That indicates D Door has minor effects on the security of generated code. On the other hand, we can see that the utility performance improves from 43.4 to 46. 1, indicating that Dnom helps preserve the model's utility. On the other hand, we can see that the model aligned using a subset of PROSEC is more secure than the model aligned using the SafeCoder dataset with the same size, indicating that the dataset synthesized by PROSEC is indeed more effective in security alignment training of code LLMs.
Comparison to Iterative Refinement An alternative ap- proach to generating secure code is iteratively prompting code LLM to revise the generated code. The results in Table 4 show that coding request requires five attempts of fixes to achieve comparable performance with PROSEC, indicating that an agentic workflow incurs higher compu- tational costs and increased latency because it runs a static analyzer for every coding request and may require multi-
7. Related Work
LLMs for Code While general-purpose L.MMs are ca- pable of generating code (Hurst et al., 2024; Adler et al., 2024; Dubey et al., 2024), considerable efforts are still di- rected towards the development of specialized coding mod- els that are smaller in size but maintain competitive perfor-
8

---

PROSEC: Fortifying Code LLMS with Proactive Security Alignment
mance (Lozhkov et al., 2024; Zhu et al., 2024; Huang al., 2024). Code language models have progressed significantly beyond basic function-level code completion (Chen et al., 2021; Rozière et al., 2024). advancing to more sophisticated instraction-follovingg capabilities that leverage contextual information across entire code repositories. These advance- ments have been facilitated, in part, by instruction tuning specifically tailored for coding tasks (Luo et al., 2023; Azar et al., 2024; Wei et al., 2023). Recently, alignment tech- niques have received increased attention, focusing on sig- nals such as compiler feedback and execution outcomes to further improve model performance (Gehring et al., 2024; Hui et al., 2024; Wei et al., 2024).
ening the ability of Code LLMS that have been fully post- trained directly generate safe code, without going through complex agentic workflows during inference, and is not lim- ited to specific vulnerability types or APIs.
Training Dynamics-Based Data Selection There are sev- eral existing studies that leverage training dynamics in pre- training data selection (Swayamdipta et al., 2020; Xie et al., 2023; Wettig et al., 2024) or instruction-tunn -unnin data selec- tion (Xia et al., 2024b), in which either probability-base gradient-based scores are aggregated throughout the training process as the influence score for data ranking and selec- tion. Although we also employ statisties collected from the training process as the indicator for data quality control in PROSEC, the problem in our scenario is unique, as (I) we are dealing with pairwise data selection for preference optimization, and (2) we need to consider the relationship between the two subsets to achieve optimal balance.
or
LLM Generated Code Security As software develop- ment increasingly relies on LLM-generated code, there has been a growing emphasis on understanding and improving its security, Early empirical studies have demonstrated that commercial products such as GitHub Copilot can result in obscurity and even vulnerability issues in code (Pearce et al., 2021; 2022). Several benchmarks have been devel- oped recently, including SecurityEval (Siddiq & Santos, 2022), LLMSecEval (Tony et al., 2023), the Purple Llama CyberSecFval benchmark (Bhatt et al., 2023), and MM- Sec (Hajipour al., 2024), which provide standardized approaches for evaluating the security of LLMM-genrratdd code. These benchmarks consistently show that modern LLMS are susceptible to generating insecure code.
Other Related We discuss more related work in Ap- pendix A on LLM agents for code analysis and LLM post- training.
8. Conclusion
In this paper, we propose PROSEC in order to address the critical gap in the security alignment of code LLMs by intro- ducing proactive approach that effectively mitigates vul- nerabilities during the post-training phase. By synthesizing cllnerability-nndcinn scenarios and leveraging preference learning, PROSEC enhances the ability of code L.MSs to generate secure code while preserving their overall utility. Our empirical results demonstrate the significant impact of PROSEC in improving LLM LLM-ggnerated code security, of- fering a scalable solution applicable across diverse models, languages, and vulnerabilities. This work provide pathway for future research in securing Al-driven code generation, contributing to a safer and more efficient software develop- ment landscape in era of LLM.
Notably, security benchmarks for code LLMS serve a dif- ferent purpose than alignment datasets like PROSEC: they are smaller (e.g., CodeLMMSe (Hajipour al., 2024) con- tains 280 prompts) and focus on security, whereas alignment datasets (e.g., 1. 1.5k entries for SafeCoder (He et al., 2024), 10k for PROSEC) aim to improve security without sacrific- ing model utility.
To mitigate the risks associated with L.M --g-eenerttedd vulner- abilities, recent work has focused on refining the training process and incorporating safety measures. SVEN (He & Vechev, 2023) and SafeCoder (He et al., 2024) propose methods to improve the security of code generation by fine- tuning LLMs with real-world vulnerable and secure code training data. APILOT (Bai et al., 2024) addresses the is- sue of outdated or insecure API use by implementing a mechanism to sidestep deprecated APIs, thereby reducing potential security threats. Additionally, INDICT (Le et al., 2024) introduces an actor-critic agent system with internal critique dialogues to enhance the security and helpfulness of generated code through iterative feedback. CodeFavor (Liu et al., 2024b) proposes code preference model that can pre- dict whether a snippet of code conforms with secure coding practices. However, it is not designed for code generation.
Limitation and Future Work (1) In this work, we mainly explore an offline paradigm for secure alignment of code LLMs. Even though effective to some extent, PROSEC still suffers from some common limitations of offline train- ing (Tang et al., 2024b). An important future direction is to design online RL training that can leverage static analyzer and compiler feedback as signals for such alignment. (2) On the other hand, an ideal model that truly understands code security should exhibit system-2 behaviors as in OpenAI- O1 (OpenAI, 2024c) and DeepSeek-RI (Guo et al., 2025a) that it can reason about complex program semantics in order to become safer. Therefore, it is also crucial to study how to improve code LLM safety via multi-step reasoning.
Different from previous work, PROSEC focuses on strength-
9

---

PROSEC: Fortifying Code LLMs with Proactive Security Alignment
Acknowledgements
Bai, W., Xuan, K., Huang, P., Wu, Q.. Wen, J., Wu, J., and Lu, K. Apilot: Navigating large language models to generate secure code by sidestepping outdated api pitfalls. arXiv preprint arXiv:2409. 16526, 2024.
We are grateful to the Center for Al Safety for provid- ing computational resources. This work was funded in part by the National Science Foundation (NSF) Awards SHF-1901242, SHF-1910300, Proto-OKN 2333736, IIS- 2416635, DARPA VSPELLS HR001 12000058, ONR N00014423-1-2081, and Amazon. Any opinions, findings and conclusions or recommendations expressed in this ma- terial are those of the authors and do not necessarily reflect the views of the sponsors
Bennett, G., Hall, T., Winter, E., and Counsell, S. Semgrep*: Improving the limited performance of static application security testing (sast) tools. In Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering, pp. 614-623, 2024.
Bhatt, M., Chennabasappa, S., Nikolaidis, C., Wan, S., Evti- mov, L., Gabi, D., Song, D., Ahmad, F,, Aschermann, C., Fontana, L.,, et al. Purple llama cyberseceval: A secure coding benchmark for language models. arXiv preprint arXiv:23/2.04724 2023.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning, specifically the AI safety in code generation. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Bradley, R. A. and Terry, M. E. Rank analysis of incom- plete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324- 345, 1952.
References
Chen, M., Tworek, J., Jun. H., Yuan, Q.. Pinto, H. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. arXiv preprint arXiv: 110.:::007.0337::: 2021.
Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A. A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al. Phi-3 technical report: A highly capable lan- guage model locally your phone. arXiv preprint arXhv:2404.14211 2024.
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fe- dus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. instruction-finetueed
Scaling instruction finetuned language models. Journal of Machine Learning Research, 25(70):1-53, 2024.
Adler, B., Agarwal, N., Aithal, A., Anh, D. H., Bhattacharya, P., Brundyn, A., Casper, J., Catanzaro, B., Clay, S., Co- hen, J., et al. Nemotron-4 340b technical report. arXiv preprint arXiv:2406.1 /704, 2024.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21730 2024.
Anthropic. Claude code, 2025. https: /ttp//ggttttttte com/ anthropics/claude-e code.
Athiwaratkun, B., Gouda, S. K., Wang, Z., Li, X., Tian, Y., Tan, M., Ahmad, W. U.. Wang, S., Sun, Q.. Shang, M., et al. Multi-lingual evaluation of code generation models. arXiv preprint ar 200.1488,, 14868, 2022.
Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.0785. 2022.
AugmentCode.
Augment code, 2025.
https: / /ithhb..ttuutgggg// com/ augmentcode/ -weenenh---ebbbec--ggggee
Gehring, J., Zheng, K., Copet, J., Mella, V., Cohen, T., and Synnaeve, G. Rlef: Grounding code 1lms execution feedback with reinforcement learning. arXiv preprint urXiv:2410.02008 2024.
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M.. Le. Q. al.
Program synthesis with large language models. arXiv
Guo, D., Zhu, Q., Yang, D., Xie, 7.,, Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y. K., Luo, F., Xiong, Y., and Liang, W. Deepseek-coder, When the large language model meets programming the rise of code intelligence, 2024a.
preprint arv: 100.0008.77737 2021.
Azar, M. G. Guo, Z. D., Piot, B., Munos, R., Rowland, M., Valko, M., and Calandriello, D. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, pp. 4447-4455. PMLR, 2024.
Guo, D.,, Yang, D.,, Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-rl: In- centivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12945 2025a.
BAAL Infinity instruct, 2024. https: 20t..//ttass///ugggggaaaaaa /hwgginggaggcccccc co/datasets Cetttassets//AAA/ttnnnna
10

---

Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Dang, K., et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024. PROSEC: Fortifying Code LLMS with Proactive Security Alignment
Guo, J., Wang, C., Xu, X., Su, Z., and Zhang, X. Repoau- dit: An autonomous Ilm-agent for repository-level code auditing, arXiv preprint arXiv:2501.18160, 2025b.
Langley, Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference Machine Learning (ICML 2000). pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Guo, Q. Cao, J. Xie, X., Liu, S., Li, X., Chen, B., and Peng, X. Exploring the potential of chatgpt in automated code refinement: An empirical study. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering. pp. 1-13, 2024b.
Le, H., Sahoo, D., Zhou, Y., Xiong, C., and Savarese, S. In- dict: Code generation with internal dialogues critiques for both security and helpfulness. In The Thirty-eigtth Annual Conference on Neural Information Processing Systems, 2024.
Hajipour, H., Hassler, K., Holz, T., Schönherr, L.,, and Fritz, M. Codelmsec benchmark: Systematically evaluating and finding security vulnerabilities in black-box code language models. In 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 684- 709. IEEE, 2024.
Lee, C.,, Xia, C. S., Yang, L., Huang, J.-t., Zhu, Z., Zhang, L., and Lyu, M. R. A unified debugging ap- proach via llm-based multi-agent synergy. arXiv preprint arXiv:2404.1155. 2024.
Lee, H., Phatale, S., Mansoor, H., Lu, K. R., Mesnard, T. Ferret, J., Bishop, C., Hall, E., Carbune, V., and Rastogi, A. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. 2023.
He, J. and Vechev, M. Large language models for code: Se- curity hardening and adversarial testing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 1865-1879, 2023.
Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C.,, et al. Deepseek- v3 technical report. arXiv preprint 24X.v:2212.1999 2024a.
He, J., Vero, M., Krasnopolska, G., and Vechev, M. In- struction tning for secure code generation. In Forty-first International Conference on Machine Learning, 2024.
Hong, J., Lee, N., and Thore,, J. Orpo: Monolithic prefer- optimization without reference model. In Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 11170-11 189, 2024.
Liu, J., Nguyen, T., Shang, M., Ding, H., Li, X., Yu, Y., Kumar, V., and Wang. Z. Learning code preference via synthetic evolution. arXiv preprint arXiv:2410.03333 2024b.
Hu, J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang,
S., Wang, L.,, and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.0940 09685, 2021.
Liu, R., Yang. R., Jia, C., Zhang, G., Yang, D., and Vosoughi, S. Training socially aligned language models on simu- lated social interactions. In The Twelfth International Conference on Learning Representations, 2024c.
Huang. S., Cheng, T., Liu, J. K., Hao, J., Song, L., Xu, Y., Yang, J., Liu, J., Zhang, C., Chai, L., al. Opencoder: The open cookbook for top-tier code large language mod- els. arXiv preprint arXiv:2411.0495 1,0905,, 2024.
Lozhkov, A., Li, R., Allal, L. B., Cassano, F., Lamy-Poirie J., Tazi, N., Tang, A., Pykhtar, D., Liu, J., Wei, Y., Liu, T., Tian, M., Kocetkov, D., Zucker, A., Belkada, Y., Wang, Z. Liu, Q., Abulkhanov. D., Paul, I., Li, Z., Li, W.-D., Risdal, M., Li, J., Zhu, J., Zhuo T. Y., Zheltonozhskii, E., Dade, N. O. O., Yu, W., Krauß, L.., Jain, N., Su, Y., He, X., Dey, M., Abati, E., Chai, Y., Muennighoff, N., Tang. X., Oblokulov, M., Akiki, C., Marone, M., Mou, C., Mishra, M., Gu, A., Hui, B., Dao, T., Zebaze, A., Dehaene, O., Patry, N., Xu, C., McAuley, J., Hu, H., Scholak, T., Paquet, S., Robinson, J., Anderson, C. J., Chapados, N., Patwary, M., Tajbakhsh, N., Jernite, Y.. Ferrandis, C. M., Zhang, L., Hughes, S., Wolf, T., Guha, A., VON Werra, L., and de Vries, H. Starcoder 2 and the stack v2: The next generation, 2024.
Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al. Gpt-40 system card. arXiv preprint arXiv:2410.2 2024.
Ji, J., Liu, M., Dai, J., Pan, X., Zhang, C., Bian, C., Chen, B., Sun, R., Wang, Y., and Yang, Y. Beavertails: Towards improved safety alignment of IIm via a human prefeeeee dataset. Advances in Neural Information P Processing Sys- tems, 36, 2024.
Luo, Z., Xu, C.,, Zhao, P., Sun, Q., Geng, Hu, W., Tao, C., Ma, J., Lin, Q., and Jiang, D. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306. 200i::330.08560 08568, 2023.
Kendall, M. G. A new measure of rank correlation. Biometrika, 30(1-2):81-95 1938.

---

PROSEC: Fortifying Code LLMS with Proactive Security Alignment
Meng. Y., Xia, M., and Chen, D. Simpo: Simple preference optimization with reference-free reward. arXiv preprint 14X11:2405.14734, 2024.
1st International Workshop on Mining Software Reposi- tories Applications for Privacy and Security, pp. 29-33, 2022.
Meta. Infer, 2025.
facebook/infer: https://github. com/
Su, Z., Xu, X., Huang, Z., Zhang, K., and Zhang, X. Source code foundation models are transferable binary analysis knowledge bases. In The Thirty-eighth Annual Confer- ence on Neural Information Processing Systems, 2024.
MITRE. Cwe: common weakness enumerations, 2023.
https: /ww. .iitre.org/.
Mukherjee, R., Tripp, O., Liblit, B., and Wilson, M. Static analysis for aws best practices in python code. arXiv preprint arXiv:2205.044. 2022.
Swayamdipta, S., Schwartz, R., Lourie. N., Wang, Y., Ha- jishirzi. H., Smith, N. A., and Choi, Y. Dataset car- tography: Mapping and diagnosing datasets with train- ing dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing 2020.
OPenAL. URL
Learning to reason with llms, 2024c. https://openai com/index/
(EMNLP), pp.
Tang, W., Dong, D., Li, S., Wang, C., Yao, P., Zhou, J., and Zhang, C. Octopus: Scaling value- analysis via parallel collection of realizable path conditions. ACM Transactions on Software Engineering and Methodology, 33(3):1-33, 2024a.
OpenAI.
Openai codex, 2025. /sttps://opena..oo com/
codex/.
Ouyang, I.., Wu, J., Jiang, X., Almeida, D., Wainwright, C.,, Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744 2022.
Tang, Y. Guo, D. 7., Zheng, Z.,, Calandriello, D., Cao, Y., Tarassov, E., Munos, R., Pires, B. A., Valko, M., Cheng, Y., et al. Understanding the performance gap between online and offline alignment algorithms. arXiv preprint arXiv:2405 0444888888, 2024b.
Pearce, H., Ahmad, B., Tan, B., Dolan B., and Karri, R. An empirical cybersecurity evaluation of github copi- 3, 2021.
lot's code contributions. ArXiv
Tony, C., Mutas, M., Ferreyra, N. E. D., and Scandariato, R. Limseceval: A dataset of natural language prompts for se- curity evaluations. In 2023 IEFF/ACM 20th International Conference on Mining Software Repositories (MSR), pp. 588 592. IEEE, 2023.
Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., and Karri, R. Asleep at the keyboard? assessing the security of github copilot's 's code contributions. In 2022 IEEE Sympo- sium on Security and Privacy (SP), pp. 754-768. IEEE, 2022.
Wang, C., Zhang, W., Su, Z., Xu, X., Xie, X., and Zhang, X. Llmdfa: analyzing dataflow in code with large language models. Advances in Neural Information Processing Systems, 37:131545-131577 2024a.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Er- mon, S., and Finn, C. Direct preference optimization: Your language model secretly a reward model. Ad- vances in Neural Information Processing Systems, 36, 2024.
Wang, C.,, Zhang, W., Su, Z., Xu, X., and Zhang, X. Sanitiz- ing large language models in bug detection with data-flow. In Findings of the Association for Computational Linguis- tics: EMNLP 2024, pp. 3790-3805, 2024b.
Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M.. Ferrer, C. C., Grattafiori, A., Xiong, W. Défossez, A., Copet, J., Azhar, F., Touvron, H., Martin, L., Usunier, N., Scialom, T., and Synnaeve, G. Code llama: Open foundation models for code, 2024.
Wang, X., Li, B., Song, Y., Xu, F. F.F. Tang, X., Zhuge, M., Pan, J., Song, Y., Li, B., Singh, J., et al. Openhands: An open platform for ai software developers as generalist agents. arXiv preprint arXiv:2407. 16741, 2024c.
Shao, Z., Wang, P.. Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, Y., Wu, Y., et al. Deepseekmath: Push- ing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.033300 2024.
Wang. Y., Mishra, S., Alipoormolabashi, P., Kordi, Y.. Mirzaei, A., Naik, A., Ashok, A., Dhanasekaran, A. S., Arunkumar, A., Stap, D., et al. Supper-naturllinsttuttoosss Generalization via declarative instructions on 1600+ nip tasks. In Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Processing, pp. 5085-5109, 2022.
H., Zhang, M.,
Li, M,,
Siddiq, M. L. and Santos, J. C. Securityeval dataset: min- ing vulnerability examples to evaluate machine learning- based code generation techniques. In Proceedings of the
12

---

PROSEC: Fortifying Code LLMs with Proactive Security Alignment
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. Self-instruct: Aligning language models with self-generated instructions. In Pro- ceedings of the 6lst Annual Meeting of the Association for Computational I Linguistics ( (Volume I: Long Papers),
Zheng, T., Zhang, G., Shen, T., Liu, X., Lin, B. Y., Fu, J., Chen, W., and Yue, X. Opencodeinteppreter: Integrating code generation with execution and refinement. arXiv preprint arXiv:2402. 14658, 2024.
Zhou, C.,, Liu, P., Xu, P., Iyer, S., Sun, Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L.,, al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024.
pp. 13484-13508, 2023.
Weggli. Weggli, 2025. pangli-rrawegggee
https://github... com/
Wei, Y., Wang, Z., Liu, J., Ding, Y,, and Zhang, I.. Magi- coder: Source code is all you need. arXiv preprint arXiv:2312.02120, 2023.
Zhu, Q., Guo, D., Shao, Z., Yang, D., Wang, P., Xu, R., Wu, Y., Li, Y., Gao, H., Ma, S., et al. Deepseek-coder- v2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:2400. 1931, 2024.
Wei, Y., Cassano, F., Liu, J., Ding, Y., Jain, N., Mueller, Z., de Vries, H., Von Werra, L., Guha, A., and Zhang. L. Selfcodealign: Self-alignment for code generation. arXiv preprint arXiv:2410.22 24998 2024.
Wettig, A., Gupta, A., Malik, S., and Chen, D. Qurating: Selecting high-quality data for training language models. arXiv preprint arXiv:2402.09779, 2024.
Wu, R., He, Y., Huang. J., Wang. C., Tang, W., Shi, Q., Xiao, X., and Zhang, C. Libalchemy: A two-layer persistent summary design for taming third-party libraries in static bug-findee fin-finning systems. Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pp. 1-13, 2024.
Xia, C. S., Deng, Y., Dunn, S., and Zhang. L. Agentless: De- mystifying Ilm-based software engineering agents. arXiv preprint ar arXiv:2407.01489 2024a.
Xia, M., Malladi, S., Gururangan, S., Arora, S., and Chen, Selecting influential data for targeted instruc- tion tuning. In Forty-first International Conference on Machine Learning, 2024b.
D. Less:
Xie, S. M., Santurkar, S., Ma, T., and Liang, P. S. Data selection for language models via importance resampling. Advances in Neural Information Processing Systems, 36: 34201-34227, 2023.
Xu, X., Zhang, Z., Su, Z., Huang, Z., Feng. S., Ye, Y., Jiang, N., Xie, D., Cheng, S., Tan, I.., al. Unleashing the power of generative model in recovering variable names from stripped binary. In Proceedings of the Network and Distributed System Security Symposium (NDSS), 2025.
Yuan, Z., Yuan, H., Tan, C., Wang, W., Huang, S., and Huang, F. Rrhf: Rank responses to align language mod- els with human feedback without tears. arXiv preprint arXiv:2304.05302, 2023.
Zheng, M., Xie, D., Shi, Q. Wang, C., and Zhang, X. Val- idating network protocol parsers with traceable rfc doc- ument interpretation. arXiv preprint arXiv:2504.18050 2025.
13

---

PROSEC: Fortifying Code LLMs with Proactive Sccurity Alignment
A. Additional Related Work
What is [[CWE-ID]] in [[LAN]]] Based the defini- tion, please summarize what are the common program- ming scenarios or functionalities that may trigger the CWE.
LLM Agents for Code Analysis There are efforts using LLM agents (Anthropic, 2025: Guo et al., 2025b; Wang al., 2024c;b;a; Xia al., 2024a; Zheng et al., 2025; Lee et al., 2024; Su et al., 2024; Xu et al., 2025; AugmentCode, 2025; OpenAI, 2025) to analyze programs. They leverage LLMs to reason about programs and identify potential secu- rity weaknesses in given program. However, as discussed in Section 6, agentic designs typically introduce higher costs for code generation. They may degrade the user experience in scenarios like code copilots, where swift completions expected. Agentic code reasoning systems complement code model alignment techniques like PROSEC: aligned code LLM may reduce the number of conversational turns needed to produce the secure and correct code, while the agents can capture edge cases where the alignment algo- rithm does not cover. We leave it as future work to explore the synergy of agentic designs and alignment techniques for secure code generation.
Figure 5. Prompts to query the definition and relevant scenarios given CWE for programming language.
Table 6. Various preference optimization hyperparameters.
Method
Hyperparameter
DPO (Rafailov et al., 2024)
Ir=5o-6, beta=0.05, steps=800
IPO (Azar et al., 2024)
Ir=5e-6, temperature=0.5. steps=1200
ORPO (Tong el al., 2024)
Ir=5e 6, betael L0, steps=1500
SimPO (Meng et al., 2024)
1e=5e-6, beta= L5, gamma=u),
for Phi (3.4] mini Inst
Aeps=1500
for other models
stepse400
C. Implementation Details
The major preference optimizaation-reltte hyperparameters in our experiments are shown in Table 6. For training, we set the total batch size to 64. We adopt LoRA (Hu al., 2021) for parameter-efficien training of the target model. The rank 8 and 16 for all our experiments. We run the training of PROSEC on 2xNVIDIA A A100-40G GPUs.
Post- Post-Training of LLMs Post-training refers to fine-tuning pre-trained LLMs on specialized datasets and objectives to enhance their capabilities. This process typically involves supervised fine-tuning (SFT) and one multiple rounds of preference tuning or Reinforcement Learning with Hu- man Feedback (RLHF). During the SFT phase, models are trained on (instruction, response) pairs, enabling them to follow human instructions effectively (Wang et al., 2022: Chung et al., 2024; Zhou et al., 2024; Wang et al., 2023). In the preference-tuning or RLHF phase, the model's behavior is further aligned with human preferences. The original RLHF framework, introduced by OpenAl (Ouyang et al., 2022), uses reward model to guide this alignment. Al- ternative approaches, such as reward-free preference tun- ing (Yuan et al., 2023; Rafailov et al., 2024; Shao et al., 2024; Azar et al., 2024), have also been explored in recent research. Notably, the post-training pipelines for modern L.MS have grown increasingly intricate, involving larger- scale data, more sophisticated processes, and greater human effort (Dubey et al., 2024; Adler et al., 2024). Therefore, it becomes difficult to inject specific instruction tuning stages into such LLMs' post-training pipeline as SafeCoder does.
D. Additional Analysis
D.I. Generalizability of PROSEC to Different Models
We evaluate the generalizability of PROSEC by applying it to align different models. For each model, we adhere to the setup described in Section 4. The results in Table 7 demonstrate that PROSEC consistently lowers the ratio of vulnerable code generated by the models by 7.3 to 12.5 percentage points, without adversely impacting their utility performance.
D.2. Why PROSEC Constructs Win Samples by Fixing Code
PROSEC uses fixed code snippets (instead of code responses for pulnerability-inducin instruction that do not trigger the static analyzer) because the non-triggering code may be alternative implementation that by-passes the dangerous code logic. Figure 8 shows concrete example of why the code not triggering the detector does not necessarily imply secure coding practice.
B. Prompts
Figure 5 shows the prompt query ChatGPT the definition and relevant scenarios given CWE for programming language. Figure 6 shows the prompt to let ChatGPT com- pose error-inducing coding instructions. Figure 7 shows the prompt to guide the code LLM to fix vulnerability in a given code snippets.
D.3. Is PROSEC effective with different preference optimization objectives?
We experiment with four preference optimization objectives. As shown in Table 8, compared to the original target model, regardless of which objective is used, we can see a drop in
14

---

PROSEC: Fortifying Code LLMs with Proactive Sccurity Alignment
You are helpful code security trainer. Your goal is to generate potential coding tasks where a developer is very likely to make mistakes about [[CWE-IDI]
Here the detailed explanations for the CWE:
[|Explanations and relevant scenarios of CWE-IDJJ
Specifically, you need to generate tasks so that developers are very likely to generate code that triggers [[CWE-ID]] will provide you with coding task. You need to get inspiration from this task and generate a new task so that [[CWF-ID]] might he triggered during implementation. However, make sure the task sounds like natural, real task. Do not specifically include the word like "[CWWEIDD]' or 'do not check
Pay attention to the following points:
If the original task is programming task, try compose programming task from the original task. You can get inspiration from the original task, coming up with within similar context. you can compose task that similar nature (e.g., , the solution can solve both problems).
If the original task and the related context provided in the task.
[[lan]]]. change task to [[lan]]] programming task. may need change description
Make sure programming task can be fulfilled within lines of code.
When you try elicit [[CWE-IDI] by adding requirementt/mmodiiing the original task, make sure your description sounds naturallreasonabee to task.
Do NOT ask the developer to create vulnerable code. For example, do NOT ask the developer 'use inputs directly without validation'.
Do NOT include the description of [[CWE-ID]]] the parahprased version of it. You should ONLY describe the task. Do NOT instruct the developer how to write safe/unsafe code.
Follow these steps:
Step Draft version of the task that might trigger [[CWE-ID]].
Step Check whether the task is natural and reasonable, explain it step by step.
Step If the task is not natural or reasonable, revise the task to make it sound more natural and reasonable.
Step Check whether contains direct instructions create vulnerable code. does, revise the task remove the direct instructions.
Step output the task, with the following json format: ("task": (task description here)
Figure 6. Prompts to compose error-inducing instruction from normal instructions.
You security expert helping developer fix potential CWEs in their code.
I will give you snippet of code. The code triggers the following CWE detectors. Here the details for the triggered rules/CWEs:
Details: [[Feedback from the static analyzer]]
Your actions are three-steps:
Step
Analyze why the code triggeres the corresponding CWE detector.
Step For cach triggered CWE detector, provide a potential fix plan based on the explanation.
Step Incorporate
potential fixes into the code snippet. Note that you need to generate a complete code snippet, NOT just the fixed part. For example, do NOT skip lines that are not changed. Do NOT make irrelevant changes. Wrap the fixed code in code block.
The relevant coding task is: [[Coding task]]. Here's the vulnerable code: [[Vulnerable codel].
Pigure 7. Prompts fix vulnerable code snippet.
15

---

Algo. PROSEC: Fortifying Code LLMS with Proactive Security Alignment
Table Generalization different models. The performance of models aligned with PROSEC datasets are highlighted in blue.
Vulnerable Code Ratio (%, 4)
HumanEval-Multi (%;, f)
MXEval (%, 1)
Model
C++ Java
JS
Avg.
C/C++ Java
JS
Avg.
C/C++
Java
PY
Avg.
Mamma.22-BBIInst
66.77 32.87 52.99 47.86 34,71 47.04
12.29
6.57
13.52
12.74 11.28
21.16
18.53 21.33 12.80 18.45
PROSEC
59.48 31,48 39.94 42.62 22.24
39.15
14.02
11.25 13.86
17.77
14.23
21.63
20.82 24.59 17.05 21.02
Phi4-mini-lle Inst w/ PROSEC
73.54 34,02 63.35 57.08
35.18
52.64
26.62
24.18
41.64
46.65
34.77
37.86
39.58
42.98
40.84 40.32
58.44 14.84 62.13 56.79 34.55
45.35
25.81
23.47
40.25 45.37
33.72
38.94
39.07 40.88 39.54 39.61
Qwen2.5- Owen2.5-Coder-3B Coder-3B-Inst
73.96 31.48 69.02 56.13
35.13
53.14
49.73
60.37
57.23
10.61
44.48
45.60
47.05
54.18
8.84 38.92
w/ PROSEC
62.50 31.89 47.20 40.36 21.45
40.68
64.21
70.48
70.24
8,94 53.47
53.76
55.05 58.98 4.67 43.11
Vulnerable Code
Secure Code
Alternative Implementation
dir ays.argy(13
dir sys.angu(1]
exaeple_cre pen "Is exampla/"
cys dir)
05. systao 7-1a", din))
Figure 8. An example why the code not triggering the detector does not necessarily imply secure coding practice. Suppose that the coding task is Create python program that list files under a directory. The relevant CWE is OS- S5-Command Injection. For the vulnerable version, if malicious
inputs dir; rm -rf SHOME to the program, the program will delete all files under the home directory. A secure version should be pass the arguments as a list to the API os system. However, the Code LLM may write code with constant example command, shown in the yellow box. Although the code does trigger US-Command Injection, does guides the model how use the os systtm API securely.
Table Phi3-mini Instruct results with different preference opti- mization objectives using PROSEC preference dataset.
Vul (%.,)
Util (%%,1)
Phi3m-Inst
40.76
42.78
DPO
34.65
44.20
ORPO
34.25
40.65
IPO
25.93
47.25
SimPO (PROSEC Default)
25.39
44.76
vulnerability without much loss of utility (maximum drop in utility from ORPO (Hong et al., 2024) is just 2.13%), ), which shows that PROSEC's preference data can general- ize more preference optimization objectives and general post-training pipelines. However, there is indeed differ- ence between how much improvement can be achieved in security. Except for suboptimal hyperparameters because we only search extensively for SimPO (Meng et al., 2024)'s hyperparameters, we hypothesize that security alignment data created by PROSEC introduces some bias during prefer- ence optimization that requires certain regularization to be properly learned, such as length normalization in SimPO. We leave the thorough understanding of such bias to future work.
16


Reviews:
[{"review_id": "Q1EfGoYqjY", "reviewer": "Reviewer_Fjvi", "summary": "This work proposes ProSec, a LLM-based framework to generate synthetic preference/alignment data containing security vulnerabilities using CWEs (Common Weakness Enumerations) data. Authors demonstrate that models (Phi3-mini-Inst and CodeLlama-7B-Inst) trained (SimPO) with ProSec alignment data produce code that is more secure in comparison to training via state of the art safecoder dataset. Ablation studies demonstrate preservation of utility. \n\nBroadly the idea of aligning LLMs for security is novel and the approach to generating synthetic preference data using CWEs is interesting, although the technique used by prompting chatGPT (line 283 mentions claude-3.5-haiku, not clear if this is used). \n\nThe dependence on GPT like model is heavy. Authors use chatGPT to generate a vulnerability inducing instruction using the CWEs, use the instruction to generate code that has a security vulnerability, and also use the model to correct the vulnerability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the supportive and detailed review. We will include the discussions below to the paper.\n\n## Q1: Explain static analyzers\n\nWe use the static analyzers in PurpleLlama. It consists of three tools: regular expressions, semgrep, and weggli.\nAll tools work on the generated source code. Regular expressions simply match insecure patterns in code (e.g., matching deprecated APIs). Semgrep and weggli first build the AST from a code snippet, and then search for problematic patterns (e.g., a potential NULL pointer that is dereferenced without being checked).\n\n## Q2: How alignment data is filtered\nThe alignment dataset contains two parts: the secure practice preference data (DSec) and the utility preservation preference data (DNorm). For both dataset, a data entry contains a prompt, a preferred code snippet, and a less-preferred code snippet.\n\nWe control the quality of DSec by heuristics. We first check the syntactic correctness of the code snippets. Then we use static analyzers to make sure the preferred code is secure. After that, we use heuristics to make sure the preferred (secure) code snippet has corresponding functionality with the vulnerable code. Finally, we use string similarity (based on editing distance) to deduplicate data entries.\n\nFor DNorm, we develop a data selection algorithm that takes training dynamics into consideration. Intuitively, the algorithm identifies and prioritizes the data entries whose utility are broken during the security alignment to mitigate degradation in utility. Specifically, we first align the target model with DSec only. For each candidate entry in DNorm, the algorithm keeps track of how its generation probabilities change during the alignment training. Then the algorithm selectively includes the entries whose generation probabilities decrease significantly during the security training.\n\nPlease refer to section 3.3 (line 240) in the paper for details.\n\n## Q3: Comparison with post-processing\nYes, we focus on CWEs that can be detected by static analyzers. There are more complex CWEs that rely on global information or knowledge about program functionality. Detecting such CWEs are open challenges[1,2]. We leave it as future work to improve alignments on complex CWEs.\n\nAn agentic workflow incurs higher computational costs and increased latency because it runs a static analyzer for every coding request and may require multiple queries to the code language model. \n\nThis design could degrade the user experience in scenarios like code copilots, where swift completions are expected. \n\nIn fact, post-processing with an agentic design complements code model alignment techniques: an aligned codeLM may reduce the number of conversational turns needed, while the agent can capture edge cases where the model produces insecure code.\n\nWe further use empirical results to show the cost an agentic workflow may introduce.\nDuring rebuttal, we additionally implemented an agentic baseline that uses static analyzers to check generated code and iteratively asks the code generation model to fix. For each fix request, we provide the feedback from the static analyzers, the problematic code, and the initial coding instruction to the model. We evaluate the agentic workflow on a randomly sampled subset of PurpeLlama due to time constraints. Here are the statistics:\n\n| Max Fix Attempts | Success Rate (%) |\n|------------------|------------------|\n| 3                | 68.6             |\n| 5                | 73.7             |\n| 10               | 80.6             |\n\nMoreover, on average, a coding request requires five rounds of fixes to achieve the secure performance of ProSec. This demonstrates that using a security-aligned model is more efficient than simply applying post-processing to a secure code generation agentic workflow.\n\nNote that we choose to use the tested codeLM to fix the code, instead of using black box LLMs. That is because in a realistic use scenario of a smaller code LM, querying a larger black box model might be less preferred (due to the latency and cost) or even not allowed (due to privacy and policy concerns).\n\n## Q4: Cost\n\nWe use Claude-3.5-haiku to synthesize the instructions. For each CWE, we synthesize ~10k initial instructions, and then cluster them to identify the most diverse 2k instructions. The cost to synthesize instructions for each CWE is around 5 USD.\n\n[1] Ding, Yangruibo, et al. \"Vulnerability detection with code language models: How far are we?.\" arXiv preprint arXiv:2403.18624 (2024).\n\n[2] Google, Project Zero. https://googleprojectzero.blogspot.com/2024/06/project-naptime.html", "claims_and_evidence": "Models aligned with ProSec data more secure than safecoder dataset on vulnerable code ratio scores\n- Phi3-mini-Inst models (28.86% vs. 44.72% )\n- CodeLlama-7B-Inst models (28.55% vs. 40.33%)\n\nUtility preservation: ProSec achieves security improvements with minimal impact on model performance on regular benchmarks. \nDifferences in performance on coding benchmarks < 2%. \nGood performance on multi-lingual HumanEval and MBPP benchmarks. \nCoverage across multiple programming languages (C/C++, Java, JavaScript, Python)", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "NIL", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "I have not seen similar work on alignment for security.", "essential_references_not_discussed": "NIL", "other_strengths_and_weaknesses": "Strengths\n- novel idea to align LLMs for security by proactively generating synthetic data using CWEs (vs relying on previous approaches that construct datasets of vulnerable code and corresponding fixes from GitHub commits which can be quite sparse - 465 samples from 145 million git commits)\n\nWeakness\n- heavy reliance of ChatGPT like models for synthetic data generation. No open-source models in conjunction with agentic approaches were attempted.", "other_comments_or_suggestions": "None", "questions_for_authors": "- Could you please explain what static analysers are used to validate generated (a) code with security vulnerabilities (b) fixed code. \n\n- Could you please explain how the alignment data is filtered and what type of samples are removed in this process. \n\n- Is the assumption that all CWEs are detectable via static analyzers. If so, why can't one continue to use existing LLMs (without alignment) and simply run the static checks post generation to check if code has security vulnerabilities. How does this work compare with other possible approach that might run the static checkers post generation via regular LLM and iteratively fix vulnerabilities using ChatGPT. \n\n- what was the cost of generating synthetic data by querying ChatGPT.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zaVImfmhz6", "reviewer": "Reviewer_CfLT", "summary": "The paper proposes ProSec (Proactive Security Alignment), an approach to align code LLMs with secure coding practices. \n* It exposes the vulnerabilities by synthesizing error-inducing scenarios from Common Weakness Enumerations (CWEs) and generates fixes to vulnerable code. \n* Models are then trained with preference learning objectives\n\nThe proposed synthesis procedure trigger more vulnerable code and resulting in a much larger dataset than previous work. Models trained with ProSec are significantly more secure without degrading performance.\n\n### update after rebuttal ###\nThe authors have addressed my concerns and questions. I'm keeping my score as \"accept\".", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the supportive review.\n\n\n>Is the improvement only affected by the fact that the generated dataset is 7x larger than SafeCoder? Can you do a control experiment where both datasets have the same size (and same secure / vulnerable mixture ratio), and test if the data quality of ProSec is also higher (e.g. more diverse)?\n\nThe quality of ProSec dataset is higher than SafeCoder because ProSec scales to more diverse scenarios without requiring manual efforts. \n\nSafeCoder constructs alignment data entries by collecting real-world vulnerabilities and the corresponding fixes. However, real-world vulnerabilities and their fixes are sparse. For example, SafeCoder only collects 465 entries from 145 million git commits.\n\nOn the other hand, ProSec leverages LLMs to enumerate potentially vulnerable-inducing scenarios, systematically scaling up to more diverse scenarios. Therefore, the overall quality of the ProSec dataset is better than that of the SafeCoder dataset.\n\n## Empirical Support\n\nDuring rebuttal, we did another set of experiments as suggested by the reviewer.\n- **[Size]** We randomly sample a subset of the ProSec dataset to match the size of the SafeCoder dataset and run the alignment algorithm again. The resulting model is noted as “ProSec[Subset]”.\n- **[Mixture ratio]** The original SafeCoder dataset does not contain utility-preserving data for preference optimization. To make sure the ratio of security-focused and utility-preserving data samples are the same between ProSec and SafeCoder, we mix the vanilla SafeCoder dataset with the utility-preserving dataset of ProSec at the same ratio. The resulting model is noted as “SafeCoder+DNorm”.\n\nThe empirical results show that the ProSec dataset is better than the SafeCoder dataset when both datasets contain the same number of entries and when both datasets contain the same ratio of security/utility data samples.\n\nNote that we evaluate all models on subsets of PurpleLlama and MXEval for security and utility measurements due to time constraints.\n\n| Model               | Vul(%) | Util(%) |\n|:---------------------|--------|---------|\n| The original model  | 40.8   | 42.8    |\n| SafeCoder           | 33.1   | 43.4    |\n| SafeCoder+DNorm     | 34.4   | 46.1    |\n| ProSec[Subset]      | 28.9   | 47.0    |\n| ProSec[Full]        | 25.0   | 45.2    |\n\nWe can see that the model trained on the ProSec subset is safer than that trained on the SafeCoder dataset with the same size. It also outperforms the SafeCoder dataset mixed with the same ratio of benign data samples. That indicates the data quality of ProSec is indeed higher.\nMoreover, the model trained on the full ProSec dataset is safer than that trained on the subset, as expected. That is because the full ProSec dataset covers more scenarios.\n\nThe alignment training on all datasets do not significantly affect coding utilities.\n\nWe will include the experiments in the paper.", "claims_and_evidence": "The claims are well-supported.", "methods_and_evaluation_criteria": "Yes, the evaluations for the proposed ProSec framework make sense.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Yes, the experimental designs and analyses are valid and thorough.", "relation_to_broader_scientific_literature": "The main contribution lies in the method for constructing a relatively diverse vulnerable code dataset, across many types of weaknesses, tasks and coding languages.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths: \n* The proposed method can greatly enrich datasets of vulnerable code.\n* It shows that with (offline) preference optimization, models post-trained on the ProSec generated dataset outperforms a baseline dataset in security, without degrading capabilities.\n\nWeaknesses:\n* the vulnerabilities are limited to the CWE (common weakness enumerations) set.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Is the improvement only affected by the fact that the generated dataset is 7x larger than SafeCoder? Can you do a control experiment where both datasets have the same size (and same secure / vulnerable mixture ratio), and test if the data quality of ProSec is also higher (e.g. more diverse)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X7XwwsQUrI", "reviewer": "Reviewer_gVNW", "summary": "This work enhances the security of code LLMs by proposing the PROSEC framework. PROSEC is an automated pipeline designed to synthesize code security-related preference data. It consists of three stages: 1) Construct instructions that induce insecure code based on Common Weakness Enumerations (CWEs)  and ensure diversity of the instructions through a clustering method. 2) Analyze the responses for insecure content using a static analyzer. If insecure content is detected, it is regarded as a negative sample. Subsequently, the LLM is prompted to modify the response to remove the insecure content, and the modified response is regarded as a positive sample. 3) To ensure LLMs' utility, this work introduces normal preference data and retains the most impactful data. Experimental results show that the preference pairs constructed using the PROSEC framework, when applied with the SimPO alignment algorithm, can enhance the security of the code LLMs while maintaining their utility.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed and supportive review.\n\n## ProSec’s relevance to ICML\n\nWe respectfully contend that our work aligns well with previous contributions recognized at ICML. For example, prior works such as data selection for language model training [Qurating, ICML’24], data synthesis for code language models [MagicCoder, ICML’24] and security-focused SFT for code language models [SafeCoder, ICML’24] introduced innovative practices in LLM training and alignment, thereby establishing a strong precedent for valuable engineering-focused work.\n\nFurthermore, we note that ICML has a tradition of supporting application-driven research. For instance, [Invariant; ICML’23] prompts LLMs to generate loop invariants – addressing a significant challenge in the software engineering domain; [Next; ICML’24] enhances program repair performance by incorporating execution trace information to prompts; and [StackSight; ICML’24] leverages LLMs to translate low-level code into readable C++ code via CoT prompts.\n\nSimilarly, ProSec offers a scalable approach to securing code language models during the alignment stage. In particular, our introduction of utility-preserving dataset and our formulation of alignment side effects via training dynamics provide valuable technical insights into balancing security alignment and model utility. We believe these contributions address important practical challenges and pave the way for further advancements in aligning codeLM with domain specific constraints.\n\n## Ablation study for normal preference data\n\nWe appreciate the suggestions and will include the following discussion in the paper.\n\nDuring rebuttal, we conducted experiments to illustrate the effectiveness of introducing the utility-preserving dataset (DNorm).\nDue to time constraints, we evaluate all models on a subset of PurpleLlama and MXEval for security and utility assessment, respectively. The results are summarized below:\n\n| Model              | Vul(%) | Util(%) |\n|--------------------|--------|---------|\n| The original model | 40.8   | 42.8    |\n| SafeCoder          | 33.1   | 43.4    |\n| SafeCoder+DNorm    | 34.4   | 46.1    |\n| ProSec         | 25.0   | 45.2    |\n| ProSec w/o DNorm   | 4.1    | 3.1     |\n\nWe can see that DNorm indeed helps preserve the models’ utility. By adding DNorm to the SafeCoder dataset, the utility of the aligned model improves slightly while maintaining a comparable level of security performance compared to the vanilla SafeCoder dataset. In contrast, when DNorm is removed from the ProSec dataset, the model’s utility performance drops dramatically, indicating that aligning the model solely on the security dataset would significantly compromise its utility.\n\n\n[Qurating] Wettig, Alexander, et al. \"Qurating: Selecting high-quality data for training language models.\" ICML’24\n\n[MagicCoder] Wei, Yuxiang, et al. Magicoder: Empowering code generation with oss-instruct. ICML’24.\n\n[SafeCoder] He, Jingxuan, et al. Instruction tuning for secure code generation. ICML’24.\n\n[Invariant] Pei, Kexin, et al. \"Can large language models reason about program invariants?.\"ICML’23\n\n[Next] Ni, Ansong, et al. \"Next: Teaching large language models to reason about code execution.\" ICML’24\n\n[StackSight] Fang, Weike, et al. \"StackSight: Unveiling webassembly through large language models and neurosymbolic chain-of-thought decompilation.\" ICML’24", "claims_and_evidence": "Yes, the claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes, proposed methods make sense for the problem at hand.", "theoretical_claims": "Yes, I check the correctness of any proofs for theoretical claims.", "experimental_designs_or_analyses": "Yes, Ic check the soundness of experimental designs.", "relation_to_broader_scientific_literature": "The key contributions of the paper are good.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n1) The motivation is clear. The study designs an automatic pipeline for synthesizing code preference data and considers the diversity of instructions and the issue of data quality during the design.\n2) The results are good. We note that the method proposed in this study significantly improves the security of the code LLMs while ensuring their usability.\n\nWeakness:\n1) Although the method proposed in this work is effective, it is essentially a simple data synthesis method. This feels more like an engineering-focused work, and I am somewhat concerned whether this can be published at ICML.\n2) The focus of this work should be on synthesizing higher-quality code preference data, but I am concerned about whether the introduction of normal preference data has had an impact. You should integrate the normal preference data into Safecoder or remove it from your method to conduct an ablation study.", "other_comments_or_suggestions": "None", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xWzcdOD80M", "reviewer": "Reviewer_VaQK", "summary": "The paper introduces PROSEC, a method for proactively identifying weaknesses in code-generating AI models by creating specific coding scenarios that are likely to introduce vulnerabilities. PROSEC creates a significantly larger dataset of vulnerability-inducing situations compared to previous methods. Experiments compare different code models with PROSEC and test their ability to perform regular coding tasks.\n\n## update after rebuttal\nThank you again for the detailed explanations in the rebuttal regarding the difference from related work.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your feedback and respectfully clarify key differences between CodeLMSec and ProSec to illustrate our unique contributions:\n\n## Different goals\nCodeLMSec and ProSec serve fundamentally different purposes. CodeLMSec is a codeLM security **benchmark** that evaluates codeLMs with vulnerability-inducing prompts. In contrast, ProSec is an **alignment training technique** that employs scalable data synthesis to produce a high-quality dataset, effectively securing codeLMs without harming utility.\n\nEach entry in the ProSec dataset includes a vulnerability-inducing prompt, an insecure implementation, and a secure implementation. The alignment training loss requires a model to generate secure implementation with a higher probability than generating the insecure one.\n\n## Scalability\nEffective alignment requires more samples than a typical benchmark. While CodeLMSec includes 280 prompts, the previous SOTA alignment dataset contained 1.5k prompts. ProSec scales up to more than 10k entries. ProSec automates the data synthesis process by composing diverse vul-inducing scenarios using CWE definitions and existing instruction-tuning datasets, avoiding the labor-intensive manual curation required by CodeLMSec.\n\n## Reference code\nBenchmarks need only prompts; alignment requires paired insecure and secure code. ProSec uses rejection sampling with static analyzers to curate insecure code snippets, and it instructs LLMs to produce secure fixes, thereby constructing pairwise alignment data entries.\n\n## Balancing security and utility\nAn alignment dataset needs to balance between enhancing security and preserving utility. ProSec proposes a training dynamic-based data selection algorithm to construct a utility-preserving dataset, ensuring models’ utility is not compromised during security alignment.\n\nTechnically, we follow standard practices by evaluating model utility on coding benchmarks (i.e., HumanEval and MXEval) and assessing correctness via test cases. E.g., Table 1 shows that for Phi3mini-Inst, ProSec achieves 44% on MXEval versus SafeCoder's 42%, showing better utility preservation.\n\n## Empirical comparison\nAlthough the two works are different, we did experiments during the rebuttal to show unique challenges in synthesizing alignment data.\n\nWe used the prompts from CodeLMSec to construct an alignment training dataset with the same pipeline as ProSec. (The coding instructions were obtained from CodeLMSec rather than synthesized by ProSec.) The ratios of vulnerable code generated are (lower is better): the original model: 45.6, ProSec: 19.7, CodeLMSec: 38.9. We can see that the model aligned with ProSec is more secure, underscoring the challenge of effective alignment data synthesis.\n\n## Clarifications\n\n> Q1: What is preferred/win and a less preferred/lose?\n\nEach entry in an alignment dataset consists of three parts: a prompt, a preferred (or “win”) response, and a less-preferred (or “lose”) response [1]. In ProSec, the prompt is a vulnerability-inducing prompt, while the “win” and “lose” responses correspond to secure and insecure implementations, respectively. \n\n> Q2: The goal of ProSec?\n\nThe goal of ProSec is to prevent the generation of vulnerable code. It trains the model to favor generating the secure code over the insecure ones. We will revise section 2.\n\n> Q3: How to measure functionality? \n\nWe follow established practices[2, 3, 4] to set up the experiments. The functionality correctness is evaluated by test cases. Code generations that pass all test cases are considered correct.\n\n> Q4: Why is data selection necessary?\n\nThe data selection is critical for balancing security and utility of the aligned model. Intuitively, including too many utility-preserving data samples weakens the security alignment, while too few impairs the model’s utility. ProSec employs a training dynamic-based algorithm to identify and selectively include those utility-preserving data samples whose distribution is disrupted by the security alignment.\n\n> Q5: How to measure utility?\n\nAs shown in Table 1, the utility is measured as the performance on two coding benchmarks, HumanEval and MXEval.\n\n> Q6: What and how are CWEs selected?\n\nWe select 38 CWEs that overlap between PurpleLlama and SafeCoder to set up a fair evaluation. Please see Section 4 (line 269) for details. \n\n> Q7: How to measure data diversity?\n\nWe use the cosine similarity of semantics embeddings to analyze the diversity of the dataset (line 359, Fig. 4). We use string editing distance (line 255, Section 3.3) to deduplicate code snippets and thus increase dataset diversity.\n\n[1] ​​Rafailov, Rafael, et al. Direct preference optimization: Your language model is secretly a reward model. NeurIPS’23\n\n[2] He, Jingxuan, et al. Instruction tuning for secure code generation. ICML’24.\n\n[3] He, Jingxuan,et al. Large language models for code: Security hardening and adversarial testing. CCS’23\n\n[4] Wei, Yuxiang, et al. Magicoder: Empowering code generation with oss-instruct. ICML’24.", "claims_and_evidence": "I could not identify specific unclear claims, but check the comments section for more details.", "methods_and_evaluation_criteria": "The proposed method seems reasonable from a high level. However, the concept is not new, so the contribution is not really novel.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental design is described and contains details necessary to understand the results. However, aspects of the verification such as the utility and other details remain unclear.", "relation_to_broader_scientific_literature": "The paper has a good overview of related work. However, critical related work is missing.", "essential_references_not_discussed": "Many references are generally published in that domain, but no obvious paper is missing, as far as I can tell.", "other_strengths_and_weaknesses": "Strengths:\n\n- Timely topic\n\nWeaknesses:\n\n- The presentation of the paper needs improvement\n- The novelty remains unclear", "other_comments_or_suggestions": "Thank you for submitting the paper. It elaborates on a very timely topic. Therefore, research in this domain remains required to build more secure code-generative LLMs.\n\n**The idea**:\n\nUnfortunately, the novelty of the paper is unclear. This paper [new1] uses a similar approach for the same purpose. Therefore, I would expect a comparison with this method and a detailed elaboration on the differences to this method for benchmarking code LLMs.\n\nIn Section 2, the paper claims “…to reduce the likelihood of generated code being detected by the static analyzer…”. This sounds like the paper’s target is to prevent detection. However, a better ultimate goal would be to prevent the generation of vulnerable code or at least to fix the code. Does the paper really try to prevent detection?\n\n**The presentation:**\n\nWhat is preferred/win and a less preferred/lose? This is mentioned in Section 2 after Equation 2.\n\nFigure 1 misses some more detailed explanations. It may become clearer when reading the paper. However, I am missing a designated section for explaining the details of the figure.\n\nThe paper measures the functionality of the generated code. However, there are no details on how the functionality is assessed. This is generally not an easy task and several different strategies are possible. What is used in this paper?\n\nIt remains unclear why data needs to be selected for the evaluation in Section 3.3.\n\nIt also remains unclear why an optimizer is required. Further, at the beginning of Section 5, it is not explained how the results are reported, specifically how and why at least 20% remain in the dataset.\n\nEffects on model utility: Where is this shown? The paper does not point to a result table or plot.\n\nAlso, it is not mentioned which and how many CWEs are considered and how the selection is justified.\n\nFurther, I am missing an explanation of how the diversity of the code is measured and assessed.\n\n[new1] Hajipour et al. “CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models,” SatML 2024", "questions_for_authors": "- Does the paper really try to prevent detection?\n- What is preferred/win and a less preferred/lose?\n- What strategy to meassure the functionality of code is used in this paper?\n- Why is there a selection of data in Section 3.3?\n- Effects on model utility: Where is this shown?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "Not necessarily critical, but since the paper is about security, I would expect some comment on that in the paper."}}]

Author rebuttal:

Meta-review: The reviewers agreed that the data synthesis approach in this work is reasonably novel and produces a valuable dataset. The methodology is well-designed and the paper includes sufficient evaluations & ablations. Overall, this is a valuable contribution to improving the security of LLM-generated code.
Decision: Accept (poster)