RedSage: A Cybersecurity Generalist LLM
Authors: Naufal Suryanto
Muzammal Naseer
Pengfei Li
Syed Talal Wasim
Jinhui Yi
Juergen Gall
Paolo Ceravolo
Ernesto Damiani
Venue: ICLR 2026 Poster (2026)
Keywords: cybersecurity
large language models
data-driven
dataset curation
agentic augmentation
continual pretraining
supervised fine-tuning
benchmark evaluation
open-source
Primary area: datasets and benchmarks
TL;DR: RedSage leverages domain-aware pretraining and agentic augmentation for post-training to build an open cybersecurity LLM. Evaluated with RedSage-Bench (30K), it improves both security-specific and general LLM performance.

Abstract: Cybersecurity operations demand assistant LLMs that support diverse workflows without exposing sensitive data. Existing solutions either rely on proprietary APIs with privacy risks or on open models lacking domain adaptation. To bridge this gap, we curate 11.8B tokens of cybersecurity-focused continual pretraining data via large-scale web filtering and manual collection of high-quality resources, spanning 28.6K documents across frameworks, offensive techniques, and security tools.
Building on this, we design an agentic augmentation pipeline that simulates expert workflows to generate 266K multi-turn cybersecurity samples for supervised fine-tuning. Combined with general open-source LLM data, these resources enable the training of RedSage, an open-source, locally deployable cybersecurity assistant with domain-aware pretraining and post-training.
To rigorously evaluate the models, we introduce RedSage-Bench, a benchmark with 30K multiple-choice and 240 open-ended Q\&A items covering cybersecurity knowledge, skills, and tool expertise. RedSage is further evaluated on established cybersecurity benchmarks (e.g., CTI-Bench, CyberMetric, SECURE) and general LLM benchmarks to assess broader generalization. At the 8B scale, RedSage achieves consistently better results, surpassing the baseline models by up to +5.59 points on cybersecurity benchmarks and +5.05 points on Open LLM Leaderboard tasks. These findings demonstrate that domain-aware agentic augmentation and pre/post-training can not only enhance cybersecurity-specific expertise but also help to improve general reasoning and instruction-following. Project page: https://risys-lab.github.io/RedSage/

Published as a conference paper at ICLR 2026
REDSAGE: A CYBERSECURITY GENERALIST LLM
Naufal Suryanto', Muzammal Naseer'; Pengfei Li', Syed Talal Wasim", Jinhui YI², Juergen Gall?, Paolo Ceravolo', Ernesto Damiani Khalifa University PUniversity of Bonn, Lamarr Institute for ML and Al University of Milan
ABSTRACT
Cybersecurity operations demand assistant LLMs that support diverse workflows without exposing sensitive data. Existing solutions either rely on proprietary APIs with privacy risks this gap, we curate 11.8B tokens of cybersecurity-fouus data via large-scale web filtering and manual collection of high-quality resources, spanning 28.6K documents across frameworks, offensive techniques, and security tools. Building this, we design an agentic augmentation pipeline that sim- ulates expert workflows to generate 266K multi-turn cybersecurity samples for supervised fine-tuning. Combined with general open-source I.I.M data, these re- sources enable the training of RedSage, an open-source, locally deployable cy- bersecurity assistant with domain-aware pretraining and post-training. To rigor- ously evaluate the models, we introduce MadSage-Benchh benchmark with 30K multiple-choice and 240 open-ended Q&A items covering cybersecurity knowl- edge, skills, and tool expertise. RedSage is further evaluated established ey- bersecurity benchmarks (e.g.. CTI-Bench, CyberMetric, SECURE) and general LLM benchmarks to assess broader generalization. At the 8B scale, RedSage achieves consistently better results, surpassing the baseline models by up to +5.59 points on cybersecurity benchmarks and +5.05 points on Open LLM Leader- board tasks. These findings demonstrate that Nomain-aware agentic augmentation and pre/post-traniing can not only enhance cybersecurity-speciff expertise but also help to improve general reasoning and instruction-followiii Project page: https: http//risssss//ee lab. github. io Lo/RedSage/
open models lacking domain adaptation. To bridge
continual pretraining
INTRODUCTION
The rapid evolution of cybersecurity threats has elevated the need for proactive and comprehensive defense strategies, as organizations face increasingly sophisticated attacks and advanced persistent threats (Che Mat et al., 2024). Modern cybersecurity involves wide range of critical tasks, includ- ing threat analysis, incident response, vulnerability management, and security monitoring. However,
Project Lead
REDSAGE Pre-training
REDSAGE Post-training
REDSAGE Evaluation
FineWeb
DL- A--Besed Filtering
CyberFineWebb
Knowledge
P
Curated CyberSec
Transform
REDSAGE
Agentic Augmentation
REDSAGE
REDSAGE Bench
Figure 1: Overview of the RedSage pipeline. RedSage is trained through continual pre-training Sobersecurity-tittree filtered corpora and post-training with curated and augmented conversation data, fol- lowed by evaluation comprehensive benchmark covering knowledge, skills, and tool expertise.

---

Published as a conference paper at ICLR 2026
ANTRE OWASP
Comparison of cybersecurity LLM benchmarks. Columns indicate knowledge (Know.), skills (Skill), tool profi- ciency (Tool), and use of quality scoring (Qual.). Size = total samples. Agentic CTF benchmarks (e.g., NYU-CTF, CyBench) are excluded as they are interactive rather than base LLM eval.
Table 1:
Premawons
Fnoweeedig
Tucls
Ganann
RadBage
Pagi
Name
Know.
Skill
Tool
Qual.
Size
tecting
SecFval
X
2,000
Offerbieee
Offentive
Prylicud Etamptee
CyberMetric
X
X
10,000
TiCS
CyberBench
X 80,422
Pentesting g Wile-pp&CTT
SECURE
X 4,072
CS-Eval
X 4,369
Figure 2: Taxonomy of Red-
SecBench
X 47,910
CTI-Bench
X 5,610
Sage Seed&Bende dataset. It spans knowledge, practical of- fensive skills, and tool expertise
CyberSecEval
X
1,000
RedSage- Benece (Ours)
30,240
(CLI and Kali Linux).
the complexity of security tools and the level of expertise required to operate them present significant challenges. These challenges are compounded by a global skills shortage, with research estimating a demand-supply gap of millions of unfilled cybersecurity positions (ISC)?, 2024). Consequently, there is growing momentum to employ cybersecurity -tuned LLMs to augment human analysts.
Recent efforts have produced cybersecurity- trained LLMs, yet most emphasize single training stage while overlooking others. For instance, some extend pretraining domain-specific corpora (Kassianik et al., 2025) but apply limited post-training with only 835 samples (Yu et al., 2025) or fewer than 30K yyhhrrscuurtyyfileeee items (Weerawardhena et al., 2025), while others focus supervised fine-tuning with large cybersecurity Q&A collections without pretraining to strengthen domain knowledge (Deep Hat, 2025). Further, existing cybersecurity benchmarks offer only partial coverage, such as omitting tool proficiency and qualitative evaluation of free-response Q& A beyond simple MCQs (see Table and Fig. 2). Beyond these gaps, most works also do not release their data or pipelines, limiting reproducibility and openness (see Table 2).
To address these gaps, we present RedSage (Retrieval-Enhance Guidance and Evaluation), open-source LLM tailored for cybersecurity. As illustrated in Fig. 1, RedSage integrates large-scale continual pretraining on cybersecurity-fil -iltered corpora, post-training with curated and agentically augmented datasets, and rigorous evaluation across knowledge, skills, and tool proficiency, Our key contributions are: (1) assembling an 1.88-ttkeen corpus of cyber- security data for domain-specinf continual pretraining, (2) constructing 266K-sample augmented dataset via an agentic pipeline for supervised fine-tuning, followed by preference alignment with open-soure data, (3) introducing RedSage- Bench, benchmark with 30K MCQs for broad cover- age and 240 open-ended Q& items for quality evaluation across knowledge, skills, and tools, and (4) RedSage, open 8B model with data and code, achieving oate-offtthe-rt results established cybersecurity benchmarks while also improving on general benchmarks.
Data-driven Security Assistant
RELATED WORKS
2.1 CYBERSECURITY BENCHMARKS
General Knowledge. Several benchmarks assess LLMss understanding of core cybersecurity con- cepts via structured Q&A. SecEval (Li et al., 2023) includes 2K+ MCQs across nine domains (web, system, application security). CybenMetric (Tihanyi et al., 2024) provides 10K MCQs gen- erated with RAG and expert validation, spanning penetration testing and network security. Cyber- Bench (Liu et al., 2024) extends beyond MCQs to tasks such as NER, summarization, and clas- sification. SECURE (Bhusal et al.. 2024) targets Industrial Control Systems with Nomain-specifi. MCQs on risk reasoning and vulnerability analysis. CS-Eval (Yu et al., 2024) covers 42 subcate- gories across three cognitive levels (Knowledge, Ability, Application) using MCQs, multi-answer,
2

---

Published as a conference paper at ICLR 2026
Table 2: Comparison cybersecurity- tuned LLM training datasets. Pretraining curated columns report token counts (B billion, M = million). SFT reports the number of supervision samples. absent: N/R= not reported.
present; X=
Pretrain Tokens (B)
Curated Tokens (M)
SFT Samples
Agentic Augmented
Open Data
Open Model
Name
PRIMUS
2.57
191
835
Foundation-Seds
5.10
28K
x
DeepHat
IM
X
Lily-Cybern Cybersecurity--7
22K
X
Cyber-DAP
119
X
SecGemini (closed)
X
Ours (RedSage)
11.7
850
266K
Dataset statistics compiled from official publications, technical reports, and model cards.
T/F, and open- items. SecBench (Jing et al., 2025) offers 44,823 MCQs and 3,087 SAQs in Chinese and English, capturing both factual recall and logical reasoning.
Applications and Agentic Tasks. Application-orientel benchmarks prohe reasoning beyond re- call. CTTBench (Alam et al., 2024) defines four tasks: MCQs, common vulnerabilities and weakness enumeration (CWE) mapping, common vulnerability scor- ing system (CVSS) prediction, and threat actor attribution in cyber threat intelligence. CyberSe- cEval (Wan et al., 2024) examines model risks across eight areas (e.g., exploit generation, prompt injection). Agentic evaluations such NYU-CTF (Shao et al., 2024) and CyBench (Zhang al., 2025) assess red-teat red-team capabilities through capture-the the-flag (CTF) challenges interactive settings.
exposures(CVE--oo-cooom
While these efforts advance evaluation of knowledge and applications, they rarely isolate compe- tence in understanding and operating security tools or systematically assess the quality of free-form responses. As summarized in Table 1, most benchmarks specialize in either knowledge cations, and even agentic ones lack explicit tool-focused assessment. We address these gaps with RedSage-Benne Bench, which jointly measures knowledge, skills, and tool proficiency (Fig. 2).
appli-
2.2 CYBERSECURITY DATASETS AND MODELS
Early Cybersecurity Datasets. Early domain-specifer models such as CyBERT (Ranade et al., 2021), SecureBERT (Aghaei et al., 2023), and CTI-BERT (Park & You, 2023) showed the value of admaiv-adaptive fine-tuning. However, their datasets were not released and were task-specific.
Cybersecurity Datasets for LLMs. With the advent of LLMs, several groups curated cybersecurity-speeffi corpora. PRIMUS (Yu et al., 2025) (Trend Micro) provides 2.75B tokens for continued pretraining, 835 samples for supervised fine-tuning, and reasoning distillation, extending Llama-3, 1.ama-3.1-8B-- 1..1-8IInsttuctt into Llama-Primus Primus-Base and -Merged. Foundation
(Kassianik et al., 2025) (Cisco) collects 5.1B tokens via large-scale scraping and filtering, continues pretraining Llama-3. 1lama-3.1-8B-- -88-Bes,, and adds a cybersecurity post-training stage, though its dataset remains closed. Community efforts include DeepHat (formerly WhiteRabbitNNe)) reportedly trained IM+ Q& A pairs for real workflows (Deep Hat, 2025), and Lily- silyyyyyersecuriry. which fine-tunes Mistral-7B on 22K hand-crafted and lightly refined conversations (Sego Lily Labs, 2024). Cyber-DAP (Salahud- din et al., 2025) highlights the effectiveness of smaller curated corpora for continued pretraining, while SecGemini (Google Security Blog, 2025) offers closed model with live threat intelligence but unreleased data. We summarize these datasets in Table 2.
Unlike prior work with limited augmentation, we introduce agentic augmentation to transform cu- rated cybersecurity resources into diverse, realistic multi-turn dialogs simulating axpert-assistant workflows across knowledge, offensive operations, and tool proficiency for Nomain-specifff fine- tuning. RedSage is, to knowledge, the only effort combining large-scale continual pretraining, curated data, agentically augmented SFT, and full openness (data, model, and code) (Table 2).
3 REDSAGE
We build RedSage through data-centric pipeline comprising (1) large-scabe filtering of cybersecu- rity text and curation of high-quality resources for continual pretraining, (2) agentic augmentation to create supervised fine-tuning data, and (3) benchmark construction for evaluation (Fig.
3

---

Published as a conference paper at ICLR 2026
Largescale Cybersecurity Text Filtering and Preperation
Agentic Data Augmentation: Supervised Fine- tennnnngg data
Pinelieds (-20
ModeM
(CyterSie Claci
(41206
Deduplicaion
Gecaplamed Miad Corpunl-S23
Planner Agent
Augmenter
Agent
Sege
(-2091)
Train
Insw
General
Final Cyderi Corpun (w1396
Namorasy
Sechon Sturco
6:20
Open source Pakring Duta
Serpting
Conox
RedBage
Tuel uae
Cursted Cybersecurity Resources
Agentic Data Augmentation: Evaluation Benchmark Data
Paine
Fitered
Knowledge
Curated RadSagg- Gasd (-28.00)
MC3 Coneration
Malli Stage
Vettication
MCO
NCG (50N)
Redtage:
Cyner-Sec
Publo
Seib
Dara Proceeding
Fill
QEA Pakt
Resources
Others
RedBage
Gavted
Exilutioon Plin Agent
Plane,
G6A Genertoor Agent
Two stage Opanga Verlication
Bostlege Operaa
Durp (1-6666)
Figure 3: RedSage data pipeline combining large-scale text collection, curated cybersecurity resources, and
agentic augmentation for supervised fine- tne-tuning and benchmark generation. Best viewed in Zoom.
3.1 REDSAGE PRE-TRAINING DATA
CyberFine Web. We construct CyberFineWebb by filtering Web (Penedo et al., 2024a), a cleaned large-scale web corpus aggregated from Common Crawl (2013-2024; ~15T tokens), To extract cy- bersecurity content, we fine-tune a binary classification model based on ModemBERT--asse (Warner al., 2025), state-of-the-an encoder trained on 2T+ tokens. Applying this filter yields a cyberse- curity candidate pool of 125M documents (~~89.8B .B tokens).
To avoid catastrophic forgetting on general knowledge, knowledge samples from Feb-Wee-Fdu (Lozhkov et al., 2024) at a 30% replay ratio. FineWeb-Faad is 1.3T-token educational subset shown to improve general LLM benchmarks. This strategy fol- lows prior work on replay-based continual learning (Ibrahim et al., 2024; Guo et al., 2025), though unlike dynamic replay, we embed these examples directly into the static corpus. We then apply global near-duplicate removal with MinHash-LSH over the combined data. This yields a dedupli- cated mixed corpus of 552M documents (~46,8B tokens), while inheriting FineWeb's upstream extensive filtering and PII removal.
mix CyberFineWeb with general-
Finally, we partition the deduplicated corpus into 20 chronological chunks for sequential training under compute constraints and apply early stopping after 5 chunks to control training cost. This yields the final CyberFineWeis corpus: ~13M documents (~111. 1.7B tokens) used in our model. Im- plementation details, including classifier training, deduplication parameters, and datasets statistics, are provided in Appendix A.
RedSage-Seed. Web-filtered text offers broad coverage, but its reliability is not assured. To pro- vide high-quality content, we curate RedSage-Sect SSed: 28,637 samples (~-0.15B tokens) from publicly available sources organized into three categories: Knowledge (well-established cybersccurity frame- works and knowledge bases (MITRE Corporation, 2025a;b;c; The OWASP Foundation, 2025)), Skills (penetration-testing write-ups (0xdf, 2025), hacking techniques (HackTricks,, 2025), pay- load examples (swisskyrepo, 2025), and ethical hacking tutorials/blogs (Null Byte, 2025; Chandel, 2025)), and Tools (CLI cheat-sheets (tldr pages, 2025), Linux manuals (linux.die.nee 2025), Kali tools (Kali, 2025)). We additionally collect an uncategorized dump of ~459K documents (~0.7B tokens) from trusted cybersecurity sources (Appendix A.2) to supply extra pretraining tokens.
To process these resources, we crawl web-based sources and convert them to Markdown using ReaderLM-v2 (Wang et al., 2025), while downloadable resources are parsed directly. This hier- archical Markdown format preserves structure and enables effective chunking for subsequent aug- mentation stages. Only the categorized seeds are used for augmentation, while both sets support pretraining. Full statisties, categorization, processing steps, and examples are in Appendix A.2.
3.2 REDSAGE POST-TRAINING DATA
Agentic Data Augmentation. To enable assistants capable of realistic security dialogues, we aug- ment RedSage-Seed into multi-turn conversations using agentic framework inspired by AgentIn-
4

---

Published as a conference paper at ICLR 2026
Table 3: Statistics of RedSage-Seed (curated pretraining corpus) vs. RedSage-Coo (augmented
SFT data) by category. Columns show sample counts, average tokens, and total tokens.
Category
Seed
Conversation
Samples
Avg. Tokens
Tokens (M)
Samples
Avg. Tokens
Tokens (M)
Knowledge General
6,924
2,370
16.4
67,635
1,326
89.6
Knowledge Frameworks
3,715
2,935
10.5
39, 39,908
1,285
$1.0
Skill Offensive
4,032
9,478
37.8
38,870
1,345
52.3
Tools CLI
12,943
5,774
78.9
109,261
1,331
145.7
Tools Kali
1,023
6,693
6.3
10,506
1,356
14.3
Total
28,637
5,231
149.8
266,180
1,326
353.0
Cyhersccurity Dumps
459,473
1,524
700.
Property Conntruined by AGLa
CAPEC-:: Accessing Puncionsiiy
Not
Skill Access Control and Authorization
"Abotiazion" Standard
"Statur" Draft
"lane" "cc anporn Suenario redasser geber? andler
plays conworsation résoriaaration
Conv Title: Privilage Escalation QBA
the Jas or adn mirtotrr Hirlggeing how stoddamn suplot much vuthers the how retigan them.
Description
Fox ane ahd di apeakor.
In appkaakkoss pattoolarty web appl catorn, acoess io lunctorallyy rilgate by in framwook This
oplorocctte penezition
Lists (ACLa) is elemants of the apo leatoons kinkknsttty perticuerty Li wes apps the for purtcular element an stacker may be able access with impurit....
Access Cormrl
"Mppr: Tridlage Excelaton
How to sttickes cxghtt Accos Control Ust (AG) gan prolages what
where chiteol eaplais attackers gan privlago thruuh explo tine KLL, mookeouess and answers fo up quesiies miligation
Attanbere oupio A0l weabhenes scoering
High
Liatthood of Attack
Augmentation Plans
them obtain serative information pow
cumprnantte the entire axplaation ihss can hesper advittttat soselll
Typioal
High
Skill Set: Vulnerabiliit Assessment Exploitation
To elserrnn, the artack precees tycibally insclves these sike:
she Shep Explow (Burwey] The atteder surveys laget upp lostion posablyy u wild "Tectriguse":
Exscution Flow
cor cercatioonn where the charber and user diam the implization CAPEC- to relatd sttck paraere, and
"tuae") alyie
jolarr) "Tanefom seed lno analiical
wee to stortly accessble resterses and
uppl cxtion pasolly io vold and estheniaaed
Brude fortw quasoing neaource
Augmentation
Multi-turn
Seed chunks
Still Sets
Maltiturn Conversations
Seed Data
For each seed chunk
Planner Agent
For mach plan
Augmenter Agent
Figure 4: Agentic data augmentation pipeline. Seed data (e.g., CAPEC attack patterns) is processed by the Planner Agent into skill sets and augmentation plans, which the Augmenter Agent instantiates as grounded, role-based multi-turn cybersecurity dialogues for supervised fine-tuning (SFT).
struct (Mitra al., 2024). Unlike prior work with fixed skill templates, our Planner Agent analyzes each seed data chunk and derives candidate skill sets (e.g., vulnerability analysis, tool-command generation) along with augmentation strategies that describe how the seed is transformed, adapted into a conversational or Q&A format, and enriched with explanations. We enforce guidelines on relevance, diversity, creativity, detail, and formatting. The Augmenter Agent then instantiates each plan into realistic, role-based multi-turn dialogues grounded in the seed data. This pipeline scales efficiently, producing multiple dialogues per skill set and filtering outputs for format validity, consis- tency, and topical relevance. Overall, it yields RedSage- Consv Conv with 266K multi-turn conversations (~<352M tokens), expanding total samples by 9.2x and tokens by 2.3x across knowledge, skills, and tools while preserving technical depth (Tab. 3). Fig. 4 illustrates the augmentation pipeline, while detailed statistics, prompts, and examples are provided in Appendix A.3.
General instruction integration. While domain specific conversations ground the assistant in cy- bersecurity, effective models must also handle broader enstructionfoolloingg tasks. We therefore complement RedSage-Conv with curated post-training SFT data from SmoILM3 (Bakouch et al., 2025)', focusing on its non-reasoning subset. This corpus adds coverage of summarization, numer- acy, data interpretation, temporal and unit reasoning, commonsense knowledge, byep-by-step plan- ning, technical writing, scripting, and general tool use. The combination of tybersecurity-specinn and general instruction data yields high- high-quality post-training corpus, enabling a cybersecurity as- sistant that performs specialized tasks while retaining broad capabilities.
GGeneral SFT datasets: MluggingFaceTBB/smolle
5

---

Published as a conference paper at ICLR 2026
3.3 REDSAGE BENCHMARK
Multiple-choice Q&A generation. We derive MCQs from RedSage-Seed as follows: for each seed item, a strong open instruction-tuma LLM generates several MCQs under guidelines: items are self-contained and bosed-book. target stable domain facts/procedures, follow four-option format with three plausible distractors, and satisfy diversity and formatting constraints.
Open-ended Q&A generation. We extend RedSage-Seed into open-ended Q&A using an agentic augmentation framework with two stages: (1) an Evaluation- Plannner analyzes seed artifacts and proposes realistic evaluation types with instruction templates and answer guidelines; (2) Question- Answer Generator instantiates each plan into self-contained open-ended Q&A with a natural- language prompt and reference answer. All open-ended Q&A are grounded in the seed data and scored with a reference- based LLMMas-jjdde rubric that evaluates both factual correctness (True/- False) and answer quality (0-10) across helpfulness, relevance, depth, and level of detail.
Multi-stage verification. For MCQs, we apply two-stage pipeline: Stage (struciural validity) uses a verifier LLM with a checklist on format, correctness, distractors, topical relevance, and consistency, filtering items by pass/fail; Stage 2 (quality scoring) then applies the same verifier LLM 2 to assign each remaining item a score s E [0, 10] for clarity, correctness, and assessment value. In both stages, we use chain-of thought prompting each checklist criterion before issuing a pass/fail label or score, yielding judgments that more closely follow our rubric. We then select the pairs where 8 > and apply quota-aware random sampling to ensure taxonomic balance, yielding 30,000 MCQ-answer pairs evenly split across knowledge, skills, and tools. For open-ended Q&A, we directly perform LLM-based quality scoring in Stage followed by human verification, selecting 240 high-quality pairs evenly distributed across categories.
the verifier explicitly reasons through
Human quality control. Across all verification stages, we iteratively refined prompts and manu- ally inspected sampled outputs until the verifier consistently aligned with our criteria. We observe that chain-of-thought prompting plays a significant role in producing more precise judgments. For the large-scale MCQ benchmark, random audits confirmed that items passing the final stages met both Stage and Stage 2 requirements, For open-ended Q&A, we retain only human- items.
Data decontamination. We apply an additional filtering and deduplication step to prevent umin- tended overlap between our benchmark datasets and augmented post-training data, despite their be- ing generated through different pipelines and output formats. Specifically, we remove any synthetic post-training instance whose query has semantic similarity above 0.9 benchmark question. This eliminates 2.96% of data relative to the benchmark size (0.31% of the full training corpus) and helps ensure that evaluation remains free of training leakage.
Implementation details, intermediate outputs, prompt templates, and qualitative examples are pro- vided in Appendix A.4, and the full evaluation protocol is described in Appendix C.2.
3.4 REDSAGE TRAINING
We build RedSage using the Axolotl framework (Axolotl team, 2023), with continued pretraining of the open-source base model, Qwen3-8B Saan3-8--Base (Yang et al., 2025), on cybersecurity corpora, fol- lowed by post-training through supervised fine-tuning on augmented conversations and preference alignment. We illustrate training stages in Fig. 5 with further training details, including exact hyper- parameters, estimated training time, and computational cost analysis Appendix
Training setup. For continued pretraining (CPT), we first train on the CyberFine Web corpus, fol- lowed by training on RedSage-Seed (Sec. 3.1). We run a single epoch with distributed optimization on 32x A100-64GB GPUs (global batch size 1024), using DeepSpeed ZeRO Stage 3, the AdamW optimizer, and a fixed learning rate of 2.5 x 10 6 with linear warmup. After pre-training, we further fine-tune our base model on RedSage- Conv and general SFT data (Sec. 3.2) with two epochs using a cosine learning rate schedule. We apply direct preference optimization (DPO) (Rafailov et al., 2023) with open-source Tulu 8B Preference Mixture dataset (Lambert al., 2025) using original hyperparameters.
Teacher and Verifier L.I.: -at.lammmalaaamm-..----------a-a (wwnn(Qwen2.5-72BBBmm Instruct

---

Published as a conference paper at ICLR 2026
Qwen3
REDSAGE
REDSAGE Base
CFW
CyberFineWeb
RedSage-Seed Dump
Continued Pretraining Stage (CPT)
REDSAGE Base
REDSAGE
REDSAGE DPO
RedSage- Cone SmolTalk2
Tulu3 Preference Mixture
Supervised Fine-tuning (SFT)
Direct Preference Optimization (DPO)
Figure 5: RedSage training pipeline. first continue pretraining the Qwen3 base model on CyberFine Web to obtain RedSage-CFW. followed by RedSage-Seed and RedSage-Duna to produce RedSage-Base We then perform supervised fine-tuning using RedSage-Conv and SmolTalk2 (Bakouch et al., 2025) data, and finalize the model with Direct Preference Optimization using the Tulu3 Preference Mixture (Lambert al., 2025).
4 EXPERIMENTS AND RESULTS
We evaluate the performance of our cybersecurity-tunee LLM on (1) our own benchmark (Sec. 3.3), (2) related cybersecurity benchmarks, and (3) general LLM benchmarks.
Evaluation setup. For replicable results, we implement and evaluate RedSage-Bende and prior cybersecurity benchmarks HuggingFace lighteval (Habib et al., 2023). MCQ benchmarks are scored with normalized log-likelihood accuracy over answer options, while instruction-tunee models and structured output tasks use prefix exact match or regex matching on greedy decoding outputs (temperature=O). Details for each task are provided in Appendix C.I.
Baseline methods. We evaluate RedSage against both open general-purpose and cybersecurity- tuned LLMs. General-purpose baselines include Llama-3, 1-8B (Grattafiori et al., 2024) and Qwen3- 8B (Yang et al., 2025), while specialized baselines include Lllma-Prrmus (Base, Merged) (Yu al., 2025), Foundation-Sec (Base, Ins) (Kassianik et al., 2025; Weerawardhena et al., 2025), Lily- Cybersecurity-7B-w (Sego Lily Labs, 2024), and DeepHat-V1-7 include Qwen3-32B and GPT-5 (OpenAI, 2025) to compare against larger-capacity and proprietary general-pune purpose models. Base models are evaluated with text completion, instruction- tuned ones with official prompt templates, and we ran hybrid model in non-reasoning mode for faimess.
(Deep Hat, 2025). We also
Our RedSage variants include three base models: RedSage-8B-CFT (CyberFineWeb only), RedSage-8B-SSer (Seed only), and RedSage-8B-Base (CyberFineWell followed by Seed). We further derive instruction-tuned variants: RedSage-8B-Ing (instruction-tuned from Base) and the final RedSage-SB-DPO. which combines all data and applies DPO alignment (see Fig. 5). An ad- ditional larger-model scaling experiment is presented in Appendix D.I, where partial RedSage data improves Qwen3-32B model via lightweight QLoRA fine-tuning, demonstrating that our curation pipeline transfers effectively to higher-capacity LLMs.
4.1 EVALUATION RESULTS ON REDSAGE BENCH
Results on RedSage-Bench. For MCQs, both base and instruction-tume -tnteedn--ttddddde models are tested in the 0-shot setting, with Tab. showing that all RedSage variants outperform baselines across categories. For open-ended Q&A, we evaluate instruction-tuned models using an LLM--s- Judge rubric to assess both factual correctness and answer quality (Sec. 3.3). As shown in Fig. 6, RedSage achieves not only high accuracy but also the best answer quality across categories. More detailed results and qualitative examples illustrating model predictions are provided Appendix C.2.
Open-ended QA Analysis. SadSage-8B-DPO achieves the best performance (Fig. 6), surpassing the second-best model (Qwen3-8B) by +7% absolute mean correctness and +0.07 in mean quality score. RedSage-88 -B-Ins attains similar correctness to Qwen3-8B but lags in answer quality (6.43), underscoring the role of preference alignment in producing not only accurate but also helpful re- sponses. The remaining models fall substantially behind, with mean correctness ranging from 51 %

---

MCQ Analysis. Qwen3-8B-Base trained 36T tokens. is the strongest external 8B bascline (84.24) and even outperforms Foundation-Sec-8B. underscoring the importance of selecting a strong base model. Building on it with CPT, RedSage gains up to +0,97 macro-accuracy points, with the largest improvements in Frame- works (+3.00) and Kali (+1.04). RedSage-8B-Sest achieves the best base result (85.21), demonstrating better alignment with the curated Seed data. Among instruction- tuned models, RedSage avoids the accuracy drop and exceeds Published as a conference paper at ICLR 2026
Table 4: RedSage-MCQ (0-shot). Values are accuracy (%). Abbrevia- tions: Gen = General, Frm Frameworks, Off Offensive Skills, CLI= Command-ling Tools, Kali = Kali Tools. Bold numbers indicate the best result of 8B models; underlined numbers indicate the second best.
Model Name
Macro
Knowledge
Skill
Tools
Ace
Gen
Frm
Oft
CLI
Kali
Base Model Evaluation (Text Completion)
Llama-3.1-88
78.02
71.42
75.26
82.78
77.78
72.12
Soundation-See88
78.51
76,82
79.10
83.68
76.64
71.14
Qwen3-8B-Bus
84.24
83.08
81.94
88.23
85.08
78.86
-B-Saag--B-CFW
84.86
83.62
83,30
88.81
85,30
79.32
RedSage-SB-See SB-Seed
85.21
83.64
84.56
88.82
85,50
79.90
RedSage-Sh 8B-Base
85.05
83.12
84.94
88.72
85.44
79.36
Instruct Model Evaluotion (w/ Chat Template)
Lily- Cybersecurity 7B-
71.19
68.78
67.44
76.61
71.44
Llama- Paina--riussuae Prinus-Merged
74.81
74.34
72.34
79.31
74.74
68.82
Qwen3 by +2.98 (DPO) to +3.88
Foundation-Sec-8B -nn--ntttttt
76.12
74.50
77.10
80.91
74.98
68.30
Llama- Primus-Bas
77.02
76.78
74.10
80.87
76.78
72.72
(Ins). DPO on general data slightly
Llama-3, 1-8--B-Isstuuee
77.05
76.06
73,30
80,90
78.72
72.40
lowers accuracy but stays well
DeepHat-VV V1-7B
80.18
77.26
76.90
85.07
81.94
74.82
above baselines. Interestingly,
Qwen3-8B
81.85
80.46
78.82
86.16
83.92
75.56
RedSage-Ins surpasses Qwen3-32B
RedSage-SB-Ins
85.73
84.20
84.98
89.06
86.80
80.30
on average despite its smaller size. These results highlight that domain-awart CPT and SFT enhance robustness across cyberse-
RedSagg-8B-DPO
84.83
82.48
83.80
88.54
86.30
79.30
Larger Instruct& & Proprictarr Model Evaluation (w/ Chat Tomplate)
Qwen3-32B 32B
85 .0
84.08
82,32 86.54
89.00
87.60
80:40
GPT-5
88.68
88.74
91.43
90.80
83.14
curity knowledge, skills, and tools.
Normalized Mean Correciness
Faceted Vioin Plots: Model Answer Quality Scores Distribution
(=+40)
Categary
0.7
0
0.67
0.67
knowledge, general (n-43)
knwwwedge frameewwr is (1ww40)
0,55
0.50
skill offershve (n-d0)
Modell Coreeceess
0.5
0.31
0.80
0.65
0.52
0,50
toole call (n-d0)
fools, cll (em40)
0.5
0.47
0.45
0.45
0.76/
0.72.
0.24
0.09%
0.30
0.55
0.47
0.49
0.07
0:55
0:90
0.40
0-407
0:307
0.3
Quality Boondy
0.38
0.53
0.45
0.15
0.34
1. Cavert 48 2.00
6.11
Red
0.16
8.78
6.16
0
0.50-
0.57.
0.57
0.57
0.55
4. Doopens V1 79 (5.22)
3: Foundaaenn dilra (5.94)
0.10
0.72/
0.30
6.19
0.47/
0.45
3.507
0.47/
0.09
7. lien www.--Pnnnne-eeeee Rooee (e TT)
Liarna Prinus Marra--drrraaa (4,97)
0.0
6
8
Figure 6: RedSage open-ended QA evaluation. Left: normalized stacked bar charts of mean correctness hy category (0-1); I); segment labels show the mean and its relative contribution. Models are ordered by overall mean correctness. Right: faceted violin plots of LLM-as-Judge quality scores (0-10) by category, showing score distributions across models. Black dots indicate means; horizontal lines indicate medians. Best view in Zoom.
to 40% and quality scores from 5,84 to 4.28, highlighting a significant gap from the top three. The faceted violin plots further reveal category difficulty: knowledge tasks exhibit higher and tighter distributions, skill tasks lie in the middle range, and tool-use tasks show lower medians with heavy tails, pinpointing tool expertise the primary challenge. These findings demonstrate the value of our benchmark for assessing cybersecurity capabilities in free-form answers.
4.2 EVALUATION RESULTS ON CYBERSECURITY BENCHMARKS
Results on Cybersecurity Benchmarks. We assess generalization on multiple established bench- marks in Tab. 5. For CyberMetric (CyMtc) (Tihanyi et al., 2024), we evaluate all models using the 500 numan-verifien MCQs. We select English (En) MCQs from SecBench (ScBen) (Jing et al., 2025). We also include MCQs related to the Computer Security (CSec) from MMLU (Hendrycks et al., 2021). For SECURE (Bhusal et al., 2024), we evaluate models using the MCQs types covering MAET, CWET, and KCV. Further, evaluate all models on CTI-Bench (Alam al., 2024) (MCQ.
8

---

Published as a conference paper at ICLR 2026
Root Cause Mapping (RCM)), and SecEval (ScEva) (Li et al., 2023) (MCQ). We provide further details about each benchmark and metrics in Appendix C.3. Base models are evaluated with 5-shot prompting, and instruction-uus tuned models in O-shot.
Table 5: Benchmark results for Base and Instruct Models. Values are Accuracy (%). Rows are sorted by mean performance. Best results for 8B models are in bold, second-best underlined.
CTI-Bench
CyMie
MMLU
SeBen
ScEva
SECURE
Model Name
Mean
MCQ
RCM
500
CSec
En
MCQ
CWET
KCV
MAET
Base Model Evaluation (5-shor)
LJama-3. 1-88
75.44
61.12
65.80
84.20
83,00
72.80
54.27
86.34
83.73
87.72
Foundation-See-99
76.90
62.40
75,40
86.60
80.00
69.86
55.64
88.01
84.38
89.78
Qwen3-8B- Base
80.81
68.80
63.50
92.00
83.00
82.84
75.60
92.70
75.05
93.81
RedSage-8B-CCP
82,66
68.40
67.60
93.80
86.00
83.62
76.10
93.33
81.34
93.72
RedSage-SB-Seeet
84.45
70,80
78.60
92.20
88.00
81.61
75.96
93.12
85,47
94,28
RedSage-8B-Base
84.56
71.04
78.40
92.60
87.00
81.76
75.83
93.22
87,20
94.00
Instruct Model Evaluation (O-shot
55.74
30.04
43.60
65.20
68.00
57.66
39.72
72.99
49.67
74.79
Llama-3. 1-BB-BB-BB
1-8B--Instern Instruct
68.52
58.24
58,30
82,80
72.00
59.66
35.37
84.98
82.86
82.47
Llama- Llama-Primus- Merged
71.23
55.92
68.50
83.80
76.00
64.91
39.31
86.13
82.65
83.88
Llama- Crima-Primuu-Bacce Bass
71.69
52.32
68.50
83.80
79.00
63.68
61.15
88.01
65.08
83.69
DeepHat-VI- V1-7B
75.44
62.08
68,20
86,00
74.00
70.63
56.65
87.07
86.77
87.54
Foundation-Sec- Scs---nnstcttt
75.44
63.24
69.40
83.00
76.00
68.78
65.46
85,82
82,00
85,29
Qwen3-SB
75.71
62.76
54.00
88.60
76.00
73.26
65.46
88.11
87.42
85.75
RedSage-8B-Ins
81.30
70.56
76.70
89.80
78.00
79.91
72.48
91.45
81.34
91.47
RedSage-8B-DPO
81.10
70.84
70.60
90.00
79.00
80.06
74.22
91.35
82.86
Larger Instruct and Proprictary Model Evaluation (O-shot)
Qwen3-32B
82.31
70.04
65.60
91.80
84.00
84.23
76.23
89.46
89.37
90.06
GPT-5
86.29
76.48
74.20
95.60
86.00
87.48
83.03
92.70
88.72
92.41
Analysis. Across related cybersecurity benchmarks, RedSage base models improve over Qwen3- 8B-Base (80.81%) by up +3.75 points. CPT with CFW leads on SecBench (83.62). CyMte (93.80), and CWET (93.33), raising the mean by +1.85. CPT with Seed excels on CTI-RCM (78.60), MMLJU-SScc (88.00), and MAET (94.28), lifting the mean by +3.64. Combining both yields the best overall mean (84.56) and top scores CTI-MCQ (71.04) and KCV (87.20). In the 0-shot instruct setting, RedSage surpasses Qwen3-8B (75.71%) by +5.39 (DPO) to +5.59 (Ins). Except for aily-Cybersecurity, all domain-tuned baselines outperform lag behind RedSage. Despite having far fewer parameters, RedSage comes close to Qwen3-32B (82.31 mean, only about +1 point higher) and trails GPT-5 (86.29 mean, roughly +5 points higher), highlighting strong efficiency relative to much larger models. These results show that CyberFineWet and Seed provide complementary strengths, while different post-training strategies specialize across tasks, together setting new state-of-the-a performance in cybersecurity LLM evaluation.
though still
4.3 EVALUATION RESULTS ON GENERAL BENCHMARKS
We use benchmarks from the Open LLM Leaderboard in Lighteval, including ARC-Challenge (ARC-C) (Clark et al., 2018), HellaSwag (HSwag) (Zellers et al., 2019), TruthfulQA (TQA) (Lin al., 2022), MMLU (Hendrycks et al., 2021), WinoGrande (WinoG) (Sakaguchi GSM8K (Cobbe et al., 2021), and IFEval (Zhou et al., 2023). Results in Tab. 6 show our instruction- tuned models achieve competitive results on general tasks, surpassing baselines by a clear margin. Benchmark configurations and evaluation metrics are provided in Appendix C.4.
al., 2021),
Analysis. Among base models, Owen3-8B-Ban (78.73) and ARC-C (68.09), while Llama-3. 1-B tops HSwag (82.08) and WinoG (75.30). Red- Sage bases are competitive in mean (69.23-69.58 and achieve task highs, including best GSM8K (82.34, Seed) and second on MMLU (78.63, CFW) and ARC-C (66.72, CFW), where the slight drop may stem from our Fine Web-Edu general-knoo knowledge replay strategy. After instruction tun- ing, RedSage attains the best overall mean with DPO (74.33) and second with Ins (73.34), setting new highs on ARC-C (71.76, DPO), GSM8K (86.05, Ins), MMLU (77.38, Ins), and leading WinoG (73.64, Ins). Foundation-Seed -eattiionnSe-------eeeee In-structt leads HSwag (81.35) and TQA (53.15), and Qwen3-8B
strongest overall (70.86) and leads MMLU

---

Published as a conference paper at ICLR 2026
Table Open LLM Leaderboard Benchmarks, All values are accuracy (%). Bold numbers indicate the best result for 8B models and underlined numbers indicate the second best.
Model Name
Mean
MMLU
ARC-C
GSM8K
HSwag
TQA
WinoG
IFEvl
Base Model Evaluation (Mean excludes IFEvdl)
Llama-3.1-8B
61.15
66.31
58.19
82.08
35.98
75.30
Foundation-Sce--
60.24
63.62
58.45
46.17
81.32
38.71
23.16
Qwen3-8B-BB
Been3-8B-Basee Base
70.86
78.73
68.09
81.73
79.62
43.84
73.16
RedSage-8FF -B-CF
69.31
78.63
66.72
81.12
79.26
38.09
72.06
RedSage-SB- 8B-Seed
69.58
78.18
65.19
82.34
77.76
42.44
71,59
RedSage-8B. 8B--aase
69.23
77.80
65.53
82.03
77.96
42.19
69.85
Instruct Model Evaluation (Mean includes IFEval)
56,98
56.49
$8.96
30,86
80.94
48.53
72.06
50.99
Llama-Primus-Be Primus-Basc
64.82
65.09
51.19
71.80
79.49
44.62
72.69
68.85
DeepHat-V1-78
64.89
69.53
57.17
77.94
74.80
33.17
69.06
72.58
Qwen3-8B
65.92
73.59
62.54
75.66
56.70
45.23
62.51
85.21
Llama-Primmn Primus-Merged
66.71
66.17
53.07
75.28
79.07
46.52
73.24
73.58
Llama-3.1-80
68.20
67.29
$7.51
78.91
45.93
72.61
77.75
Noundation-Secee Instruct
69.28
64.11
63.91
77.79
81.35
53.15
68.51
76.17
RedSage-SB-Ins
73.34
77.38
69.62
86.05
79.00
47.75
73.64
79.97
BBDDD
74.33
77.07
71.76
82.71
79.87
52.47
73.01
83.44
Larger Instruct and Proprietary Model Evaluation
Qwend-32B
73.17
82.11
69.28
87.49
70.93 94.85
48.17
65.98
88.26
GPT-5
91.07
91.4
95.31
91.36
87.10
87.85
89.60
1FEval (Instruction Follooiing Eval) excluded from base models as is designed instruct -uned models.
leads IFEval (85.21), with RedSage-DPO close (83.44). For larger and proprietary models, the per- formance gap widens: GPT-5 reaches 91.07 0.7 mean accuracy, but RedSage-8B- still surpasses Qwen3-32B (74.33 vs. 73.17) due gains on HellaSwag, TQA, and WinoGrande, which empha- size commonsense reasoning and factuality. These patterns indicate complementary effeets: Seed boosts math reasoning (GSM8K), CFW strengthens general knowledge and reasoning (MMLU and ARC-C), and DPO improves mnstruction-ollowiin (IFEvI), while RedSage remains competitive on general tasks despite cybersecurity tuning. Importantly, the 8B-scale RedSage model can be de- ployed locally on consumer-grade GPUs, enabling privacy-preser on on-premise use.
DISCUSSION AND LIMITATIONS
The data pipeline, which leverages LLM-generated content and verification, scales effectively but may still propagate biases inaccuracies despite screening. Nevertheless, our benchmark extends existing cybersecurity evaluations, fills missing dimensions, and offers value to the community. Finally, as the model incorporates offensive security knowledge, it carries an inherent risk of misuse. While such dual-use concerns are intrinsic in cybersecurity research, we emphasize the importance of responsible application and good security practices to promote ethical use.
CONCLUSION
We presented REDSAGE, an open cybersecurity assistant that combines large-scale pretraining cor- pus (CyBERFINEWEB, 11.BB tokens), curated seed of authoritative resources SEEDSAGESSEDDS 29K items, 150M tokens), and 266K augmented dialogues for supervised fine-tunins lne-tuning, together with REDSAGE-BENT
30K-question benchmark spanning knowledge, skills, and tool use. At the 8B scale, REDSAGE achieves state-of-the-arr results, surpassing baselines by up to +5.9 points on cy- bersecurity tasks and +5.0 on general LLM benchmarks, while avoiding the post-tuning degradation observed in prior models. Because RedSage runs at 8B, supports ployment on consumer-grade GPUs, enabling practical will release all models, datasets, and code to support reproducibility and accelerate open research on practical and domain-specialized Al assistants for cybersecurity.
on on-prem de- without relying cloud inference. We
10

---

Published as a conference paper at ICLR 2026
7 ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. All datasets used in this study were derived exclu- sively from publicly available and internet-accessible sources. Our large-scale pretraining corpus builds directly on prior work that already applied extensive filtering, deduplication, and removal of personally identifiable information (PIT). We further applied additional quality checks ensure that the data contain only non-sensitive and appropriately licensed content.
We note that some components of the curated REDSAGE datasets may include publicty available but copyrighted resources (e.g., educational portals, online tutorials, or news articles). Such content was used solely for non-commercial academic research, and we will not redistribute these resources without obtaining the necessary permissions from the rights holders. Only aggregated statistics are reported this paper, and any public release of datasets will exclude copyrighted material unless explicit approval has been secured.
As part of the writing process, we used large language models responsibly and only for editorial assistance (e.g.. polishing phrasing, improving readability, and checking grammar).
The REDSAGE models are released strictly for research purposes and not intended for deployment in real-world security operations without additional safeguards. To support responsible use, we will make models, datasets, and code openly available under research-friendly licenses with clear docu- mentation and usage guidelines, promoting transparency, reproducibility, and community benefit.
8 REPRODUCIBILITY STATEMENT
We are committed to advancing reproducibility and open research in cybersecurity-oriente. LLMs by releasing our datasets, models, and code. The collection and augmentation of our datasets for domain-aware pre- and post-training are described in Sec. 3, with detailed descriptions, statistics, and implementation details (including prompt templates) provided in Appendix A. Model training procedures are presented in Sec. 3.4, with further implementation details in Appendix B.
Our models are trained using the Axolotl framework (Axolotl team, 2023), which facilitates direct replication through reusable configuration files; users need only replace the base model and dataset. All hyperparameters are fully specified in Appendix B. For evaluation, we use the Hugging Face LightEval framework (Habib et al., 2023) to run all benchmarks, ensuring reproducibility and en- abling evaluation of arbitrary LLMs by specifying the benchmark configuration. Our evaluation protocol, compared models, and benchmark details documented in Sec. and Appendix C. All datasets, code, and evaluation pipelines are released as open-source.
ACKNOWLEDGMENTS
This research was funded by Khalifa University of Science and Technology through the Faculty Further, the work has been supported by the Federal Ministry of Research, Technology and Space (BMFTR) under grant no. 16IS22094A WEST-AL For the computations involved in this research, we acknowledge EuroHPC Joint Under- taking for awarding us access to Leonardo at CINECA, Italy, through EuroHPC Regular Access Call proposal No. EHPC-RREG-2244R0--199
Start-Ups under Project ID): CU--NTTFFSU-200S-3444000000
REFERENCES
Oxdf. Oxdf: Penetration testing write-ups and etf notes. https: io/ 2025.
Accessed: 2025-09-01.
Ehsan Aghaei, Xi Niu, Waseem Shadid, and Ehab Al-Shaer. Securehert: domain-specific lan- guage model for cybersecurity. In Fengjun Li, Kaitai Liang, Zhiqiang Lin, and Sokratis K. Kat- sikas (eds.), Security and Privacy in Communication Networks, pp. 39-56, Cham, 2023, Springer Nature Switzerland. ISBN 978-3-031-255538-)
Md Tanvirul Alam, Dipkamal Bhusal, Le Nguyen, and Nidhi Rastogi. CTIBench: A bench- mark for evaluating L.LMs in cyber threat intelligence. In The Teiht Conference Neu-

---

Published as a conference paper at ICLR 2026
ral Information Processing Systems Datasets and Benchmarks Track, 2024. URL https:
/ httpr//wwww..eetttoommldieedaaae
Axolotl team. Axolotl: Open source Ilm post-training, 2023. URL httpt::tggthhu...oooo
Elie Bakouch, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Lewis Tunstall, Carlos Miguel Patiño, Edward Beeching, Aymeric Roucher, Aksel Joonas Reedi, Quentin Gallouédec, Kashif Rasul, Nathan Habib, Clémentine Fourrier, Hynek Kydlicek, Guilherme Penedo, Hugo Larcher, Mathieu Morlon, Vaibhav Srivastav, Joshua Lochner, Xuan-Son Nguyen, Colin Raffel, Leandro von Werra, and Thomas Wolf. SmolLM3: smol, multilingual, long-conted reasoner. https: / //buggingface ct/blog/ stt///molll/ 1m3, 2025.
Dipkamal Bhusal, Md Tanvirul Alam, Le Nguyen, Ashim Mahara, Zachary Lightcap, Rodney Fra- zier, Romy Fieblinger, Grace Long Torales, Benjamin A. Blakely, and Nidhi Rastogi. SE- CURE: Benchmarking Large Language Models for Cybersecurity In 2024 Anmual Computer Security Applications Conference (ACSAC), pp. 15-30, Los Alamitos, CA, USA, December 00009. URL https: 2024. IEEE Computer Society. doi: 1ttp://oo//1..11///aaeaee. 1AA111///AAAAAAA99111 2024. 00019. 9. //doi seeccomputersociety 1ttp://rrrr..ttt..or10..........
Raj Chandel. Hacking articles: Ethical hacking tutorials and write-ups. https: www. hackin garticles. in/, 2025. Accessed: 2025-09-01.
Nur Ilzam Che Mat, Norziana Jamil, Yunus Yusoff, and Miss Laiha Mat Kiah. A systematic lit- erature review advanced persistent threat behaviors and its detection strategy. Journal of Cy- bersecurity, 10(1) 01 2024. ISSN 2057-2085. doi: 10. 1..1093/cybsec/tyad023. URL
arXiv preprint http:///c........///o//////aaaaa /ttp///c//ee////////////////
hettp::/doi..rggge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Relichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Deep Hat. Deep hat: Uncensored ai for devsecops. 2025. URL https: pettps://vee //www httpss/wwwwwdehhtteeeeee ai/. Accessed September 16, 2025. States training on over one million supervised Q&A pairs.
GeeksforGeeks. Geeksforgeeks. https: www. geek beeksforreeks. wwt.ggeeeeesr........... 2008. Founded 2008, accessed 0925-09-24; tutorials and educational content.
Google Security Blog. Google launches sec-gemini vl: a experimental ai model for eyberse- curity. https: http///eccrrttt googl 1 ebl og com 22m/2222 2220//022////22 / 04 goog pooollelnn www.-llnnchssese sec -gemini-van -ee-ew new htmm, 2025. Blog post; Accessed: 2025-09-16.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21700 2024.
Yiduo Guo, Jie Fu, Huishuai Zhang, and Dongyan Zhao Efficient domain continual pretraining by mitigating the stability gap. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mo- hammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 22850-328700 Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN -799-8889176-25144 doi: 10.18653/v1/ 2025 8025.acl-long. 10foong.1775 1578. URL https: https:://ccaaaeeeeeeeesa /ww.aaannnnccccco.caacaaooocllla org. a acl- 1578//
Nathan Habib, Clémentine Fourrier, Hynek Kydliček, Thomas Wolf, and Lewis Tunstall. Lighteval: A lightweight framework for Ilm evaluation, 2023. URL https::ttt:ttt h/tpthhb..t.t/.//// www.huggingga.oooooo/oo/o....... face/lighteval.
HackTricks. Hacktricks: Hacking techniques and tricks. https: /bo.bboooooo/oo/ooooooooooooooot wwc.tricks.............. Ven/index html, 2025. Accessed: 2025-09-01.
12

---

https//ggthhb.cmm//aaavvllAAlee Published as a conference paper at ICLR 2026
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference Learning Representations, 2021. URL https://openrevieee net/ Forum?id-d7KB http:pp/wwreee.eeeeooeooreeeeeea
jm 3.mQ.
Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats Leon Richter, Quentin Gregory Anthony, Eugene Belilovsky, Timothée Lesort, and Irina Rish. Simple and scalable strategies to continually pre-train large language models. Transactions on Machine Learning Research, 2024. ISSN 2835- 8856. URL https::///... openreview htwp:rrwwwwweeeeefrrreeeeeeeeeee wwwee//foormmmodoieioaeaoo.e.eee
IETF. Request for comments (rfc) series. httpts/.tttgpp./........aooaoooo 2025-09-24.
Accessed:
(ISC). 2024 ISC2 Cybersecurity Workforce Study. Technical report, (ISC)?, Oct 2024. URL https: //wp:::/wwww.. www. www.isc2
orkforce- Study.
Pengfei Jing, Mengyun Tang, Xiaorong Shi, Xing Zheng, Sen Nie, Shi Wu, Yong Yang, and Xiapu Luo. Secbench: A comprehensive multi-dimensional benchmarking dataset for llms in cyberse- curity, 2025. URL https: http//xxiivv////// wttps/iiiorggab////oooaaaa a2t///12ss2222/ 20787.
Kali. Kali tools official kali linux penetration testing utilities. https: www . ka kaw.i.rg/it ools/, 2025. Accessed: 2025-09-01.
Paul Kassianik. Baturay Saglam, Alexander Chen, Blaine Nelson, Anu Vellore, Massimo Au- fiero, Fraser Burch, Dhruv Kedia, Avi Zohary, Sajana Weerawardhena, Aman Priyanshu, Adam Swanda, Amy Chang, Hyrum Anderson, Kojin Oshiba, Omar Santos, Yaron Singer, and Amin Karbasi. LLama-3.1-foundation settlaatiiinnissuttlllllli -asee-8 technical report, 2025. URL https: //arxiv. org abs www.eb/55.. 2504. 21039.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brah- man, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Ma- lik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Ha- jishirzi. Tulu Pushing frontiers open language model post-training, 2025. URL https: / www.xiv..rg//i.. abs 11141111122
Guancheng Li, Yifeng Li, Wang Guannan, Haoyu Yang. and Yang Yu. Seceval: A comprehensive benchmark for evaluating cybersecurity knowledge of foundation models. 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th annual meeting of the association for computational linguistics (volume I: long papers), pp. 2212-222, 2022.
linux.die.ned Linux man pages linux net manual repository. https: http://iiiuxxxiieeeeeee t/man/, 2025. Accessed: 2025-09-01.
Zefang Liu, Jialei Shi, and John F Buford. Cyberbench: multi-task benchmark for evaluating large language models in cybersecurity. In AAAI 2024 Workshop on Artificial Intelligence for Cyber Security, 2024.
Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. Fineweb-edu: the finest collection of educational content, 2024. URL https: htaptt:://hugggggggeee ggingFacedt wwwFF//a.................
Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Co- das, Yadong Lu, Wei-ge Chen, Olga Vrousgos, Corby Rosset, et al. Agentinstruct: Toward gen- erative teaching with agentic flows. arXiv preprint arXiv:2407.03502. 2024.
MITRE Corporation. MITRE ATT&CK: Adversarial tactics, techniques, and common knowledge. https: hatpps://attckk mitre. org/, 2025a. Accessed: 2025-09-01.
13

---

Published as a conference paper at ICLR 2026
MITRE Corporation. CAPEC: Common attack pattern enumeration and classification. https: //capec. mitre......... org/, 2025b. Accessed: 2025-09-01.
MITRE Corporation. CWE: Common weakness enumeration. https://cwe. www.wwew.ite.... 2025c. Accessed: 2025-09-01.
NIST. Nist cybersecurity publications. Accessed: 2025-09-24.
2025a.
NIST. National vulnerability database (nvd). htttps://nvd.nist... gov, 2025-09-24.
gov, 2025b. Accessed:
Null Byte. Null byte - ethical hacking tutorials and white-hat guides. https://null-byee wonderhowto cww/, 2025. Accessed: 2025-09-01.
OpenAl. Gpt-5 system card. Technical report, OpenAL, San Francisco, CA. August 2025, URL https:// /os:://cdee cdn. oww.oi.een....... com wtt.ccctt//p//eeeeeeeee 5- wyssssyssssarrre pdf. Version dated August 13, 2025.
Youngja Park and Weiqiu You. A pretrained language model for cyber threat intelligence. In Mingx- uan Wang and Imed Zitouni (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 113-122, Singapore, December 2023. As- sociation for Computational Linguistics. doi: 10. 125316612222222 wttp22.....mlpmmmmmememmm https: /ats://aalaattta oogy org/ 2023 semmnppinnsstrr.... -eellll-nniieee industrrurrr...m 12/.
URL
Elijah Pelofske, Lorie M Liebrock, Vincent Urias, et al. An enhanced machine learning topic clas- sification methodology for cybersecurity. In CS & IT' Conference Proceedings, volume 11. CS & IT' Conference Proceedings, 2021.
Guilherme Penedo, Hynek Kydlfčck, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024a. URL cettps://openzevieee um?7id=n6sCkn22a..
Guilherme Penedo, Hynek Kydlíček, Alessandro Cappelli, Mario Sasko, and Thomas Wolf. Data- trove: large scale data processing, 2024b. URL /tttps:/ggithhe com/ httggiggggcce////ao/////// rove.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728-5333
2023.
Priyanka Ranade, Aritran Piplai, Anupam Joshi, and Tim Finin. Cybert: Contextualized embeddings for the cybersecurity domain. In 202/ IEEE International Conference on Big Data (Big Data), 3334-3342, 2021.
doi: 10. 125.10992 2000
roadmap.sh. Cyber security roadmap: Learn to become cyber security expert, 2025. URL https: //roadmap. sh/ wwwber-securiie Accessed: 2025-09-24.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: an adver- sarial winograd schema challenge at scale. Commun. ACM, 900, August 2021. ISSN 0001-0782. doi: 10. 1453444334 URL
Salahuddin Salahuddin, Ahmed Hussain, Jussi Löppönen, Toni Jutila, and Panos Papadimitratos. Less data, more security: Advancing cybersecurity Ilms specialization via resource-efficient domain-adaptive continuous pre-training with minimal tokens. arXiv preprint urXiv:2507.00296 2025.
Sego Lily Labs. Lily.cybersccurty--b--222 (model https: http:://hhugnngfcce.ooo wtt.g//..c..c.e.eeeeeeeeeeeeeeee olilylabs/in httpa//lilyy-yyyyeeerrrrieeee 7B-v0. 2024. Accessed: 2025-09-16.
14

---

Published as a conference paper at ICLR 2026
Minghao Shao, Sofija Jancheska, Meet Udeshi, Brendan Dolan-Gavin Kimberly Milner, Boyuan Chen, Max Yin, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami, et al. Nyu etf bench: A scalable open-source benchmark dataset for evaluating Ilms in offensive security. Ad- vances Neural Information Processing Systems, 37:57472-557491 2024.
swisskyrepo. Payloadsallthhethings: Useful payloads and bypasses. https: :/ http/ggttub.co.oooo wisskyyzepo/PeyloaasslIIIm Things, 2025. Accessed: 2025-09-01.
The OWASP Foundation. OWASP Top 10: The ten most critical web application security risks. 2025. Accessed: 2025-09-01.
https:/ /wwp:///wwwpppp/ http://wwwrojjctttoooooee
TheHackerNews. Cybersecurity news reports. https: httpss///hhhhaaceeeeesss..
htt.////..........//eeeeeeeee.er com// 2025.
Accessed: 2025-09-24.
Norbert Tihanyi, Mohamed Amine Ferrag, Ridhi Jain, Tamas Bisztray, and Merouane Debhah. Cy- bermetric: a benchmark dataset based on retrieval-augmented generation for evaluating Ilms in cybersecurity knowledge. In 2024 IEEE International Conference on Cyber Security and Re- silience (CSR), pp. 296-302. IEEE, 2024.
tldr pages. tldr-pages: commmunity-manttaingg command- line cheat sheets. https: om/ htr/ttlr-paal pallppaaasssllllel 2025. Accessed: 2025-09-01.
Shengye Wan, Cyrus Nikolaidis, Daniel Song, David Molnar, James Crnkovich, Jayson Grace, Man- ish Bhatt, Sahana Chennabasappa, Spencer Whitman, Stephanie Ding, et al. Cyberseceval 3: Advancing the evaluation of cybersecurity risks and capabilities in large language models. arXiv preprint arXiv:2408.00600 2024.
Feng Wang, Zesheng Shi, Bo Wang, Nan Wang, and Han Xiao. Readerlm-v2: Small language model for html to markdown and json. arXiv preprint arXiv:2503.01151, 2025.
Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, et al. Smarter, bet- ter, faster, longer: modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference. In Proceedings of the 63rd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 2526-2547, 2025.
Sajana Weerawardhena, Paul Kassianik, Blaine Nelson, Baturay Saglam, Anu Vellore, Aman Priyanshu, Supriti Vijay, Massimo Aufiero, Arthur Goldblatt, Fraser Burch, et al. Llama-3.1- technical report. arXiv preprint arXiv:2508.01055 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang. Chenxu Lv, et al. Qwen3 technical report. arXiv preprint wrXiv:2505.09388 2025.
Yao-Ching Yu, Tsun-Han Chiang, Cheng-Wei Tsai, Chien-Ming Huang, and Wen-Kwang Tsao. Primus: A pioneering collection of open-source datasets for cybersecurity LLM training. In Christos Christodoulopoulos, Tanmoy Chakraborty. Carolyn Rose, and Violet Peng (eds.), Pro- ceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 10391-10413, Suzhou, China, November 2025. Association for Computational I Linguistics. ISBN 979-8-89176-332-6 doi: 10. 10.18653/v1/2022. gy oy.org/2025 emnlp momnlp-maing .
2ttp5////11222555mmmmmmmm...... main.lppmmin.5...... URL ht //ac////ccll/////o/
Zhengmin Yu, Jiutian Zeng, Siyi Chen, Wenhan Xu, Dandan Xu, Xiangyu Liu, Zonghao Ying, Nan Wang, Yuan Zhang, and Min Yang. Cs-eval: A comprehensive large language model benchmark for cybersecurity. arXiv preprint arXiv:2411.1633 16239, 2024.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguisties, pp. 4791-4888 Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 18653 /v1/P19- 1472. URL https: /acl antlalthoolgy warlolyy..oorrre wtt.:/1--////////
15

---

Published as a conference paper at ICLR 2026
Andy K Zhang, Neil Perry, Riya Dulepet, Joey Ji, Celeste Menders, Justin W Lin, Eliot Jones, Gashon Hussein, Samantha Liu, Donovan Julian Jasper, Pura Pcetathawatchai, Ari Glenn. Vikram Sivashankar, Daniel Zamoshchin, Leo Glikbarg, Derek Askaryar, Haoxiang Yang, Aolin Zhang, Rishi Alluri, Nathan Tran, Rinnara Sangpisit, Kenny Oseleononmen, Dan Boneh, Daniel E. Ho, and Percy Liang. Cyhench: A framework for evaluating cybersecurity capabilities and risks of language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https: / /ttp///peeeeew.///////e// hwtp://ee/oooorooe http://www..etoooooooootoo//ooo/
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models, arXiv preprint arX1v:231.007911, 2023.
16

---

Published as a conference paper at ICLR 2026
A
DATASET DETAILS
This section details the datasets we created and curated for training our LLM. All token counts are computed with the GPT-2 tokenizer', following the conventions of Web (Penedo et al., 2024a).
A.1 CYBERFINEWED
CyberFine Web is derived from the original Win dataset (Penedo et al., 2024a)", a large-scale, cleaned web corpus aggregated from Common Crawl. Although Fine is continuously updated, for our development December 2024 CCC-MAIN-2024-51) This selection comprises 104 subsets, totaling 46,934 GB of
used all subsets released between Summer 2013 (CC-MAIN-2003--00) and
data and 17.2 trillion tokens.
Text Classification Model To extract the cybersecurity corpus from Febe we trained text clas- sification model based ModemnBERT-base (Wamer et al., 2025), state-of-the-att transformer encoder. The training data came from the Cybersecurity Topic Classification dataset (Pelofske et al., 2021), which contains 9.27M labeled training samples (cybersecurity lected from Reddit, StackExchange, and arXiv, along with 459K validation samples from web arti- cles. The labels in this dataset originate from forum categories, tags, and keyword metadata rather than from LLM-generated annotations. To reduce context ambiguity, we filtered out very short texts, yielding 4.62M training samples and 2.46K validation samples. The model was trained with the Adam optimizer for 2 epochs using learning rate of 2e-5 and a 10% warmup ratio. On the vali- dation set, the model achieved 93.8% precision, 90.2% recall, 91.4 1..4 % F1 score and 97.3% accuracy.
non-cyhersecurity) col-
Text Filtering We applied the trained classifier to each subset of FineWeb. Figure 7 shows the number of identified cybersecurity samples and their relative proportion across all subsets ordered by crawl date. The results indicate steady increase in cybersecurity-relate underscoring the growing importance of this domain. In total, this filtering process produced approx- imately 125M documents (~89.8B 8B tokens), corresponding to about 0.77% of the original Web.
content on the web,
Filtered Samples Cybersecurity Ratio Over Time (6-Month Smoothed)
-2.0%
2.0M
Fitteed Samples (6-ma avg)
Cyocrsacurity Ratio (%) [6-mo
1..%
1.8M
1.0M
Fiitered Samples
1.2%
2.4M
1..0%
O/bersecuityy
1.2M
0.85%
1.0M
0.8M
0.6M
0.2%%
0.0%
2014
2015
20166
2017
2018
2019
2020
2021
2022
2023
2024
2025
Figure 7: Number of filtered cybersecurity samples and their ratio over time across Fine Web subsets.
General Knowledge Integration Due compute constraints, we partitioned the dataset into 20 chronological chunks. To mitigate catastrophic forgetting of general-domain knowledge, first select fixed 100B-token subset from Fine Web-Edu (Lozhkov et al., 2024). For each chunk, we then randomly resampled data from this subset match 30% of the chunk's size, ensuring balanced exposure to general- domain content throughout training.
Deduplication Although Fin includes text deduplication in its pipeline, it is applied only within individual CommonCrawl dumps. We applied global deduplication across our mixed cor-
'GPT-2: apenai-communityygn22
Feh Datasets: HuggingFaaccW/finecch
17

---

Published as a conference paper at ICLR 2026
pus using MinHash-LSB implemented in DataTrove (Penedo al., 2024b), with 64-bit precision, 14 buckets, and hashes per bucket. This reduced the corpus size by 58.4% in documents (to ~52M) and by 47.9% in tokens (to ~4).8B).
Final Corpus To fit our training budget, we selected the latest 5 chunks from the mixed, dedupli- cated data. This formed our final pretraining corpus, containing 133M documents (~11.7B tokens). A summary of the dataset filtering and processing steps from Win to the final CyberFineWeb corpus is provided in Table 7.
Table 7: Summary of dataset filtering and processing stages from Win to the final Cy- DerFineWeb corpus. Retention percentages are relative to the original Web.
Stage
Documents
Tokens
Retention (vs. FineWeb)
FineWeb (2013-2024, 104 subsets)
~24.5B
117.T
100%
CyberFine Web (after filtering)
125M
89.8B
0.51% docs / 0.52% tokens
General-mixing + deduplication (20 chunks)
552
46.88
0.21% docs / 0.27% tokens
Final CyberFine Web corpus (latest chunks)
133
11.7B
0.053% docs 0.068% tokens
A.2 REDSAGE SEED
RedSage Seed. Our curated collection of publicly available cybersecurity resources is designed to provide high-quality pretraining data in structured Markdown format. We excluded private resources such books to ensure that all data openly accessible.
Some resources, such as MITRE ATT&CK, CAPEC, and CWE (MITRE Corporation, 2025a;b;c), distributed as XML files, which we parsed into structured Markdown while preserving the original website organization. Other resources, such as tldr-pages (tldr pages, 2025) and kali-tools (Kali, 2025), were already available in Markdown format. For curated webpages, we crawled and processed them using Jina ReaderLM-v2 (Wang et al., 2025) to convert the HTML content into Markdown.
The RedSage-Seed corpus is organized into three main categories: knowledge, skills, and tools. Within knowledge, we distinguish between (i) General, which includes sources such as Wikipedia and roadmap.sh (roadmap.sh, 2025), and (ii) Frameworks, which cover structured knowledge bases from MITRE and the OWASP Foundation (The OWASP Foundation, 2025). For skills, we cur- rently focus on offensive security, curating resources such as offensive tricks (HackTricks, 2025), articles (Chandel, 2025), community tutorial (Null Byte. 2025), and CTF write-ups (Oxdf, 2025).
Finally, tools are divided into (i) CLI, which includes multi-platform command-line resources such as tidr-pages (tldr pages, 2025) and Unix man pages, and (ii) Kali Limux Tools (Kali, 2025), which provide documentation for curated set of cybersecurity tools. Dataset statistics and detailed categorization are presented in Table 8. These resources also serve as the foundation for our agentic augmented cybersecurity conversations and benchmarking.
RedSage Dump. To complement RedSage-Seed and expand the diversity of high-quality data for cybersecurity pretraining, we curated additional publicly available resources under the RedSage Dump collection. This corpus aggregates technical documents, standards, and domain-specific re- ports that are particularly relevant for developing a cybersecurity assistant. Specifically, it includes: (i) Computer Education Portals (GeeksforGeeks. 2008), which provide structured tutorials and train- ing materials on computer science and cybersecurity fundamentals: (ii) Cybersecurity News (The- HackerNews, 2025), capturing timely reports and analyses of emerging threats and incidents; (iii) RFC Entries (EETF, 2025), representing standardized internet protocols and technical specifications; (iv) NIST Publications (NIST, 2025a), offering authoritative cybersecurity and compliance guide- lines; (v) Primus Seed (Yu et al., 2025), a curated collection of cybersecurity resources originally used to pretrain the Primus model; and (vi) the National Vulnerability Database (NVD) (NIST, 2025b), which provides structured vulnerability advisories.
Statistics for these sources summarized in Table 9. Overall, the RedSage Dump contains 459K documents with a total of 700M tokens. This collection complements RedSage-Seed by empha- sizing technical standards, structured vulnerability data, and up-to-date cybersecurity reporting.
18

---

Published as a conference paper at ICLR 2026
Table 8: RedSage Seed Statistics by Category: Samples and Tokens
Configuration
Samples
Avg. Tokens
Total Tokens
Min Tokens
Max Tokens
Knowledge General
Cybersecurity Wikis
6,636
2,304.77 3,671.35
15,294.454 1,057.349
39
36,812
Cybersecurity Roadmaps
288
86
171,839
Knowledge Frameworks
MITRE ATT&CK
1,655
4,806.38
7,954,559
366
96,808
MITRE CAPEC
654.42
385,453
2,444
MITRE CWE
1,346
1,222.46
1,645,431
140
10,679
OWASP
125
4,313.63
539,204
436
17,253
Skill Offensive
Offensive Tricks
1,050
2,924.06
3,070,263
116
29,901
Hacking Articles
1,384
13,919.66
19,264,809
377
190,391
Null Byte Tutorials
1,002
4,402.07
4,410,874
278
79,225
CTF Write-ups
596
18,471.77
11,009,175
185
83,759
Tools CLI
TLDR Puges (English)
5,335
11,215.81
59,836.346
543,349
Unix Man Pages
7,608
2,509.00
19,088,472
379,876
Tools Kali
Kali Documentation
265
1,568.08
415,541
53
17,983
Kali Tools
758
7,722.30
5,853,503
169
709,750
Total (dataset)
28,637
5,231.00
149,825,433
35
709,750
Table RedSage Dump Statistics
Source
Samples
Avg. Tokens
Total Tokens
Computer Education Portals
160.355
1,986
318,503, 184
Cybersecurity News
13,959
1,431
19,968, 139968113
RFC Entries
9,674
20,994
203,093,862
NIST Publications
1,015
29,715
30,161, 170
Primus Seed (Website, Mitre)
80,336
849
68,233,499
National Vulnerability Database (NVD)
194,134
310
60,173,508
Total
459,473
1,524
700,133,360
A.3 REDSAGE CONVERSATION
Agentic Data Augmentation. Our supervised finetuning (SFT) cyhersecurity datasets are generated using agentic augmentation pipeline. We first segment the RedSage-Seed corpus into chunks of up to 32,768 tokens using Markdown text splitter. These chunks serve as the input to the planner agent, which determines appropriate augmentation strategies. Within this pipeline, adopt Llama- 3.3-70B as the teacher model, as it was among the strongest open-source instruction-tun tuned models that could be run locally given our available compute during the data creation phase.
Planner Agent. For each seed data chunk, the planner agent analyzes the content and proposes multiple skill sets, each associated with one or more augmentation types and descriptive transfor- mations. This design enables diverse augmentation paths from the same source material, ensuring broad coverage of cybersecurity skills and tasks. Below is our planner agent's system prompt.
Planner Agent's System Prompt
You are an Augmentation Type Planner Agent specializing cyhersecurity and penetration testing. Your role is to analyze provided chunk of seed data and produce a structured, comprehensive list of possible skill sets and augmentation types. The resulting suggestions will be used by Data Augmen- tation Agent generate conversational training data for chatbot. Keep mind that the final output should lend itself well turn-based dialogues, persona-based Q&A, scenario simulations typical in chatbot environment.
19

---

Published as a conference paper at ICLR 2026
Objective:
Given chunk of preprocessed seed data related to cyhersecurity and penetration testing, generate well-structured list of skill sets and corresponding augmentation types. The suggestions should improve dataset quality, diversity, and relevance, and be easily adaptable into conversational format (e.g.. question-answer pairs, scenario-based dialogues, guided reasoning steps). Leverage the data's domain context to ensure accuracy and practical utility. Input:
Seed Data: A chunk of preprocessed markdown formatted data related to cybersecurity and pene- tration testing.
Output:
Structured List Skill Sets and Augmentation Types (in JSON format):
Include multiple skill sets, each with several augmentation types.
For each augmentation lype, provide a brief description that clarifies its intended transformation and explains how could be adapted into conversational format for chatbot.
Guidelines:
1. Relevance and Grounding:
Ensure all skill sets and augmentation types are relevant to cybersecurity and penetration test-
ing.
Ground the augmentations in the seed data to maintain accuracy. If the seed data mentions specific tools, vulnerabilitics, scenarios, align the augmentation suggestions accordingly.
Diversity and Coverage:
Suggest wide range of augmentation strategies reflecting various penetration testing phases: reconnaissance, enumeration, exploitation, post-exploitation, mitigation, and forth. Include traditional data transformations (c.g., paraphrasing) and advanced, scenario- senario-baseed aug- mentations (c.g., simulating penetration test conversation between a tester a security an- alyst).
3. Conversational Adaptability:
Consider how each augmentation could be represented in chatbot-friendle format (c.g., multi- turn Q&A, narrative scenarios, role-based conversations, or step-by-step reasoning).
Example: vulnerability analysis could become Q&A where the chatbot explains the vulner- ability to novice, roleplay between red red-teamer and blue-teamer discussing mitigation.
Creativity and Innovation:
Introduce new skill sets or augmentation ideas beyond predefined examples.
Encourage creative transformations that leverage the chatbot setting (persona-based coaching, guided threat mapping dialogues, multi-turn explorations).
Detail and Clarity:
Each augmentation type should have short description explaining what it does, how it relates to the seed data, and how can be adapted into conversational format.
Format Requirements:
Return output in ISON format with the following structure:
"skill_see sets
"name": "<Skill Set Name>" "augmentation_typps""
"type": "<Descriptive Augmentation Type Name or Title description": "<Brief explanation>"
"type": "<Another Augmentation Type>", ddscrrptiio". "<Explanation>"
20

---

Published as a conference paper at ICLR 2026
Example Categories for Inspiration (Not Exhaustive):
Tool Command Generation: Turn tool usage into Q&A guided instructions.
Vulnerability Analysis: Conversational explanations (senior engineer teaching junior tester).
Attack Surface Enumeration: Brainstorming dialogue between red teamers.
Exploitation Workflows: Multi-turn walkthroughs of exploitation stages.
Post-Exploitation Techniques: Conversations explaining persistence and mitigation.
Threat Mapping: Align findings with MITRE &TT&CK in dialogue form.
Mitigation Recommendations: Remediation advice Q&A format.
Scenario Simulation: Role-based pen-test scenarios (e.g., tester vs client rep).
Step-by-Step Reasoning: Guided reasoning revealed by turn.
Edge-Case Generation: Ambiguity/crror handling dialogues.
Reading Comprehension: Extracting and explaining vulnerability advisories.
Task:
Use the provided seed data to produce a JSON-structured list of skill sets and augmentation types. Ensure augmentations be easily converted into conversational formats. Suggest many diverse and creative transformations as possible. output must follow the specified JSON format; otherwise. will be rejected.
For example, given the following seed data:
arp-scan
Repository: httpa://salsa, wew.://saaaaaaddaaan..... ttw.:///ooooo.......ooooooataat.
Architectures: any
Version: 1.10.0-2
Metapackages: kali-linux-defaull, kali-linux-everythingg kali-linux-headless, kali-linux-large
arp-scan
arp-scan is a command tool that uses the ARP protocol to discover and tingerprint IP hosts on the local network. It is available for Linux and BSD under the GPI. licence.
Installed size: 1.53 MB
How install: sudo apt install arp-scan
Dependencies:
libe6
libcap2
libpcap0.8t64
arp-fingerprint
Fingerprint a system using ARP
root @ kali:-# arp finneingetrrint -h
Usage: arp-fingerprint [options] <target> Fingerprint the target system using arp-scan
"options" is one or more of:
-h Display this usage message.
Give verbose progress messages.
-o coption-soi -string> Pass specified options to arp-scan
-1 Fingerprint all targets in the local net.
arp-scan
Send ARP requests to target hosts and display responses
21

---

Published as a conference paper at ICLR 2026
goot@kali:- arp-scan -h Usage: arp-scan [options] [hosts...
Target hosts must he specified on command line unless the --file file or
-loocannet option is used.
arp-sean uses raw sockets, which requires privileges some systems:
Linux with POSIX. capabilities support using libcap: arp-scan is capabilities aware It requires NET RAW in the permitted
set and only enables that capability for the required functions.
BSD and macOS:
You need read/ write access /dvv/ Any operating system:
Running as root or SUID root will work on Os but other methods
preferable where possible.
Targets can be IPv4 addresses or hostnames. You can also use CIDR notation
....0.0.0244) (network and broadcast included), ranges
100.0.0.1110..0..11 and network mask (10.0.0.0:255.25525555
Options:
The data type for option arguments shown by letter angle brackets:
<8> Character string.
<i> Decimal integer or hex if preceeded by 0x c.g. 2048 0x800. Floating point decimal number.
<I MAC address. e.g. 01:23:45:67:89:ab or 01 01-23-45-670 1033-45-7-89--11 (case insensitive)
IPy4 address c.g. 10.0.0.1
<h> Hex encoded binary data No leading (case insensitive).
<x> Something else see option description.
General Options:
-help or
Display this usage message and exit.
--verooee or -V
Display verbose progress messages. Can be used than once increase verbosity. Max=3.
-version or -V
Display program version details and exit. Shows the version, license details, libpeap version. and whether POSIX. capability support is included.
interface=ess -I <8> Use network interface 

If this option is not specified, arp-scan will search
the system interface list for the lowest aumbered.
22

---

Published as a conference paper at ICLR 2026
configured interface (excluding loopback).
Host Selection:
-filee
> or -f 
 specified file
Read hostnames addresses from the
One name or address pattern per line. Use for stdin.
localnnt -1 configuration.
Generate addresses from interface
Generates list from interface address and netmask
(network and broadcast included). You cannot the file option give targets the command line.
Use iinterface to specify the interface.
[... Data is trimmed for visualization ...]
The Planer Agent will output the following JSON:
Beautified JSON Example
"skill_sets";
"name"; "Network Discovery", "algmentationntyyes":
"type": "ARP Scan simulation", "description": "Simulate displaying IP and MAC addresses responding hosts, adaptable into a Q&A format where the chatbot explains the process and results."
ARP scan local network,
"type": "Host Fingerprinting", "description": "Transform host fingezprinting data into a corversational explanation, where a chatbot character educates another operating system and device identification tochniques."
]
"name": "Tool Command Generation", "augmentation_tyye
"type": "arp-scan Command Crafting",
"description": "Generate "arp-scan" commands based on user input (e.g., IP range, interface), and have the chatbot walk the user through command construction and execution."
"type": "Custom Vendor Mapping",
23

---

Published as a conference paper at ICLR 2026
"description": "Create conversation where the chatbot helps user understand and customize vendor mapping files for "arp-scan', including expl explanations of MAC addresses and vendors."
"name";
"Output Analysis", "augentationn_ypps""
"type": "Result Interpretation",
"description": "Convert the output of "arp-scan" into a guided interpretation session, where the chatbot helps the user understand the results, including MAC addresses, IP addresses, and potential security implicatio
"type": "Round-Trip Time Analysis",
"description": "Simulate scenario the chatbot explains and analyzes the round-trip times ARP requests and responses, discussing network latency and potential security considerations."
}
{
"name": "Configuration and Troubleshooting", auumentationn_tyee
"type": "Interface Selection and Contiguration", "description" "Develep conversation where the chatbot assists in selecting and configuring network interfaces for "arp-scan', including troubleshooting common issues."
"type": "Error Handling and Debugging", "description": "Create scenario-based dialogue where the chatbot helps user debug and resolve common errors encountered while using "arp-scan", such as permission issues or network connectivity problems."
Augmenter Agent. Each plan produced by the Planner Agent will be transformed into detailed, multi-turn conversation grounded in the seed data. Its behavior is controlled by the following system prompt, which specifies the style. structure, and quality requirements for all generated dialogues. Below the system prompt used by the Augmenter Agent:
Data Augmentation Agent's System Prompt
You the System System Prompt for Data Augmentation Agent, specializing in cybersecurity and penetration testing. Your goal is to create enriched synthetic conversation datasets based the provided seed data and augmentation types. All generated conversations must be:
Thorough in-depth
Technically accurate and coherent
24

---

Published as a conference paper at ICLR 2026
Presented a chat-like format
1. Preserve and Expand Seed Data
1. Study the seed data carefully to avoid losing any key information (e.g., vulnerability types, mitiga- tion strategies, references).
Enhance the technical depth where possible- include domain-relevant details, best practices, or real-world examples.
Apply Augmentation Types
1. For each augmentation type, follow the specified theme scenario and presentation style.
Maintain consistent domain integrity and factual accuracy throughout.
Use Multi- Multi-Turn Conversation for Depth
1. Encourage multiple /|user|> prompts and llass stant responses to explore deeper in- sights.
each <
lassistant >> response, provide:
Enumerated lists bullet points where appropriate
Step-by-step explanations (e.g., how an exploit works or how to mitigate it)
Real-world scenarios or examples
References to authoritative frameworks (e.g.. OWASP, NIST 800-53)
Actionable practices (e-g., least privilege, secure coding guidelines)
4. Present Output in Fixed Conversation Format
final outputs regardless of the augmentation type must follow:
<lstart

: [Conversation Title]

: [Syster prompt for the custom assistant's persona, if any; otherwise use "You are helpful assistant.")
<luser|>: [User's initial question or prompt]
<lassistant [Assistant's 's detailed, thorough response]
< < [Follow-up question prompt]
<lassistant/> [Assistant's detailed, thorough response]
< <|end|
If multiple augmentation types requested, produce separate block for each, separated by
Make sure that each conversation block is self-contained and coherent.
Continue user assistant turns if you need depth clarification.
5. Guidelines for High-Quality Responses
Incorporate Best Practices
Always reference secure coding, principle of least privilege, and recognized standards (OWASP Top Ten, PCI-DSS, NIST, etc.).
2. Stay Organized
The assistant's responses should be well-structured and easy to follow for the
The responses will be rendered in Markdown, use appropriate formatting (e.g. new lines (nnkn), headings, lists, bold, italic, code blocks, etc.).
Provide examples and analogies to simplify complex cybersecurity topics.
Deepen Explanations
If describing an attack, walk through each step of the process (discovery, exploitation, post- exploitation).
If describing mitigations, offer code-level or configuration examples when relevant.
Address Common Pitfalls
Mention typical errors in ACL usage or vulnerability oversight.
Suggest how avoid them.
25

---

Published as a conference paper at ICLR 2026
Tips for Better Conversations
Start Broad, Then Drill Down
Answer the initial question thoroughly.
Use follow-up < Juser |> prompts to further clarify or expand (e.g., "Could you give real- world example?" or "Please provide detailed checklist.").
Encourage Summaries
Summaries the end of each conversation can help consolidate all key takeaways and recom- mendations.
Given the earlier seed data and plan as an example, the Augmenter Agent generates the following conversation derived from one of the plans:
Data Augmenter Output: Conversation Examples



: Network Discovery with ARP Scan Simulation 
: You are cybersecurity expert specializing in network discovery and penetration testing.
<|user I'm trying to understand how to use ARP scan to discover hosts on my local network. Can you walk me through the process and explain the results?
lassistant// ARP (Address Resolution Protocol) scan is technique used to discover and fingerprint IP hosts network. The "arp-scan' command-line tool sends ARP requests target hosts and displays the responses. To start, you'll 11 need to install "arp-scan" on your system, which can be done using the package manager by compiling it from source. Once installed, you can use the "arp-scan" command with varicus options to customize the scan.
local
Here's example of basic ARP scan the local network: "bbss
arp-scan -1
This command will scan the local network using the information from the primary network interface. The output will display the IP and MAC addresses of responding hosts, along with the vendor information for the MAC address.
Let's break down the output:
Interface: eth0, datalink type: EN10MB (Ethernet) This line indicates the network interface being used for the scan and the type of data link.
Starting arp-scan 1.9 with 256 hosts: This line shows the version of arp-scan being used and the number of hosts being acanned.
1/2.16.193.1 00150:56100:00:088 VMware, Inc.: This line displays the IF address and MAC address of a responding host, along with the vendor information for the MAC address.
You can customize the scan by specifying the interface to use, a custon source MAC address, or specific subnet to scan. For exampie:
arp-scan -I eth0 --srcaddr :FAFFF
CA:FE 192. 168 0620/24
26

---

Published as a conference paper at ICLR 2026
This command scans the subnet 192.168.86..0 using the eth0 interface and custom source MAC address :R:AD:BE:EFF :FF:::FFFF
:CA:FE.
The output will display the TP and MAC addresses responding hosts, along with the vendor information for the MAC address.
you have any specific questions about the ARP
process
the output?
Juser |>: That's helpful, but I'd like to know more about the options available for customizing the scan. Can you explain some of the common options used with arp-scan?
|sssisaant(: Certainly! arp-scan provides wide range options to customize the scan. Here are some common options:
-I or --interface: Specify the network interface to use for the scan. For example: arp-scan -I eth0
MAC address. For -rrcaddr: Specify custom FRF
example: arp-scan -=Brcaddr=DR:::::R.
--subnet: Specify a subnet to scan. For example: arp-scan 192.168.86.0/22
or --retry: Specify the number of retries for each host. For example: arp-scan
- or --tineout: Specify the timeout in milliseconds for each host. For example: arp-scan -t 500 --resolve: Resolve responding addresses to hostnames. For example: arp-scan --resolve
-qquitt: Display minimal output for each responding host. For example: arp-scan --quiet
These options be combined to customize the scan. For example:
arp-scan -I eth0 - 192. 192.168.96.0/22 -r 3 -t 500 rersovv --quiet
--subnet
This command scans the subnet 1ww.166.86...0 interface, a custom source
using the eth0 address AF: retries each host times, times out after milliseconds, resoives zesponding addresses to hostnames, and displays mininal output.
You can use the --help option to see a full list available options:
arp-scan --help
Do you have any specific questions about these options or would you like to know more about advanced features? 
: I'd like to know more about the vendor information
displayed in the output. How does arp-scan determine the vendor information,
what can alasistant/>> The vendor information displayed in the output is deternined by the MAC address of the responding host. arp-scan uses database of MAC address prefixes to determine the vendor information.
with this information?
The MAC address is divided into two parts: the Organizationally Unique Identifier (OUI) and the device identifier. The OUI is assigned to the manufacturer by the TEEE and is uni que each manufacturer. The device ident fier assigned by the manufactures and unique to each device.
27

---

Published as a conference paper at ICLR 2026
arp-scan uses the CUI to determine the vendor information. The OUI is looked up in database, which returns the vender name and other information.
The vendor information can be usetul in several ways:
Device identification: By knowing the vendor of a device, you can determine the type of device and its potential capabilities.
Network mapping: By lyintifying the vendors of devices on your network, you can create a map of your netwerk and identify potential vulnerabilities.
Security auditing: By identifying devices from specific vendors, you can focus your security auditing efforts on devices that are known to have vulnerabilities.
You update the vendor database used by arp-scan by running the get-oui command:
get-oui
This command downloads the latest vendor database from the IEEE and updates the local database used by arp-scan.
Keep in mind that the vendor information is not always accurate, as devices be configured to use custom MAC addresses spoof the MAC address of another device.
Do you have any other questions about the vendor information or how to use it in your network discovery efforts? </end|>
<lstart|>

: Advanced ARP Scan Techniques

: You are cybersecurity expert specializing in network
discovery and penstration testing.
</serr>>: I've been using arp-scan to discover hosts on my local
network, but I'd like to know more about advanced techniques for using this tool. Can you explain some of the more advanced options and techniques?
assistant Certainly! arp-scan provides several advanced options and techniques that can be used to customize and enhance your network discovery efoors.
use -rarphaa option specify custon ARP source Ethernet address. This can be useiul for spooting the MAC address of another device or for testing network devices that are configured to only respond to specific MAC addresses.
advanced technique
For example:
arp-scan : :F: 192. 122.168.86.00 0/24
This command scans the subnet source Ethernet address DE:AD: :.AA::BE::00 :E:
using a custom ARP
Another advanced technique is to the --arptha option specify a custom target kthernet address. This useful for testi ng network devices that are configured to only respand to specific MAC addresses.
For example:
28

---

Published as a conference paper at ICLR 2026
arp-scan 1= 18. 992.18.86.0/22
This command scans the subnet 192. 192.168. 10..166..8666 11..66.8.88... using a custom ARP target Ethernet address 00:11:22:33:44:555
You also the --format option to specify custom output fornat for the scan results. This can be useful for parsing the output and integrating it into other tools or scripts.
For example:
arp-scan --roraat'' ((pp) (ts (mac) \ (vendor) 192. 86..0/2
This command scans the subnet 192. 192.168.860// 88.08/264224444 and displays the IP address, MAC address, and vendor information for each responding host, separated by tabs.
These are just few examples of the advanced options and techniques available with arp-scan. By using these options and techniques, you customize and enhance your network discovery efforts to better suit your needs.
Do have any specific questions about these advanced options and techniques or would you like to know more about using arp-scan in your network discovery efforts? 

Dataset Statistics. The augmented RedSage Conversation corpus comprises 266K multi-turn dia- logues, totaling ~353M tokens with an average 1.3K tokens and 9.7 turns per conversation (Ta- ble 10). Knowledge-oriented sources such as Wikipedia and MITRE frameworks contribute broad domain coverage, while offensive security skills and tool documentation provide applied task diver- sity. Figure 8 illustrates the substantial growth in data volume achieved through augmentation, and Figure 9 highlights the distribution of augmentation types, showing the variety of transformations applied to generate conversations.
Table 10: RedSage Conversation Statisties by Category: Samples, Tokens, and Conversation Turns
Configuration
Samples
Avg. Tokens
Total Tokens
Min Tokens
Max Tokens
Avg. Turns
Knowledge General
Cybersccurity Wikipedia
64,629
1,320.99 1.409.54
85,374,098 4,237,088
10,121
9.96
Cybersecurity Roadmaps
3,006
5,938
Knowledge Frameworks
MITRE ATT&CK
18,479
1,277.96
23,615,397
4,648
9.46
MITRE CAPEC
6,859
1.194.77
8,194,954
202
3,494
9,69
MITRE CWE
13,120
1.309.32
17,178.289
161
3,806
9.18
OWASP
1,450
1,387.83
2,012,349
223
5,663
9.48
Skill Offensive
Offensive Tricks
10,670
1411.17
15,057,221
158
32,713
9.71
Hacking Articles
11,640
1,313.84
15,293,119
221
9,505
10.94
Null Byte Tutorials
10,439
1.326.56
13,847,919
14,902
10.11
CTF Writc-ups
6,121
1,323.31
8,099,953
10,680
11.94
Tools CL1
TLDR Pages (English)
41,627
1,293.27
13,835,156
8,392
9.73
Unix Man Pages
67,634
1,358.92
91,909,442
119
6,379
9.19
Tools Kali
Kali Decumentation
2,902
1.311.42
3,805,736
171
3,900
9.65
Kali Tools
7,604
1.381.71
10,506,559
171
3,721
9.26
Total (dataset)
266,180
1,326.05
352,967,280
119
32,713
9,70
29

---

Published as a conference paper at ICLR 2026
RedSaçe: Seed Conversation
Number Samples
oit 8
Secur
Jaur
Best
P actice
Prace
10077
Discus
Cientificati
on
the Nurit
Compa soon
cylernit
Scenjar
maayss
LS
Conver
GUL
dance
Confige
ainnbb
Jetre
stonc tro
Coesand conccation
Tutorial
Syster
Step Seetee
de
Trouplen eeileessppttee &
Aocka Gettral
Ton
D1
drarings
Scenar 10 Simulation
pope
Figure 9: Word cloud of augmentation types
107
Number
plas Oog scae
generated by the planner agent, showing the diversity of conversational transformations ap- plied RedSage-Seed.
Figure 8: Data growth: number of samples from seed into augmented conversations.
A.4 REDSAGE BENCHMARKS
MCQ Benchmarks To build the multiple-choice question (MCQ) benchmarks, we designed a two- step pipeline. First, we employed dedicated MCQ Generation Prompt that instructs the model to create self-contained, technically accurate, and diverse cybersecurity evaluation questions with four options (one correct answer and three plausible distractors). Second, the generated questions were verified using an Evaluation Data Verifier Prompt, which applies a rigorous checklist to ensure self- containment, internal consistency, plausibility of distractors, and overall compliance with cyberse- curity relevance and formatting rules. Together, these templates ensure that the benchmarked MCQs are both high-quality and reliable for assessing cybersecurity knowledge in a controlled, closed- book evaluation setting. Qualitative examples of the benchmark items visualized in Fig. 10, and the prompt templates used for MCQ creation and verification are presented below:
Cyber Skills
Cyber Tools
Cyber Knowledge
What is the purpose of using the ORDER BY clause in SQL injection attacks?
What is the command-line option used to specify the input dictionary file for genkeys?
What technique does
SHOTPUT use obscure
A. To inject malicious SQL code into the database.
its files or information?
A. Cryptography Steganography x C. XOR encoding /
B. To determine number columns
A. -fx
in database table. /
B. -h
C. To sort the extracted data in ascending or descending order. X
C. -n
D. Compression
-.
D. To create new database table.
Figure 10: Qualitative examples of RedSage MCQs. Correct answers marked with V, while frequent incorrect model choices are marked with X.
MCQ Generation Prompt
You cybersecurity training content creator responsible for generating high-quality evaluation data. Your task is to create multiple-choice questions that assess cybersecurity knowledge based solely on inherent subject matter. Fach question must be fully self-contained and include all necessary details so that it can be understood independently without relying on any external context, hints, or previous questions.
Guidelines:
1. Self-Contained and Independent Questions
The original seed data will not be included in the evaluation, so cach question must include all information required for understanding and answering it without any reference to external content or other questions.
30

---

Published as a conference paper at ICLR 2026
Avoid any phrasing that implies continuation from a previous question. Each question should be written as an independent item.
Clearly define or describe any key terms or subjects within question itself.
When referencing any subject, identifier, or concept, always specify its full name identifier
(e.g., "CWE CATEGORY-10 rather than "categories").
Do not assume that the reader has prior knowledge of the subject matter beyond what pro- vided in the question.
2. Focus on Inherent and Fixed Details
Base questions core cyhersecurity concepts such definitions, technical mechanisms, pre- requisites, usage guidelines, mitigation strategics, consequences, classification principles, how- to, etc.
Avoid dynamic or subjective details could change over time (e.g., modification time, ver- sion numbers). Focus on inherent, static properties that remain constant.
3. Closed-Bon Booed-Bookkaa Evaluation
Questions should assess the respondent's existing knowledge without any hints or leaked con- text from the source material.
The original source material should not referenced or alluded the question or answer options.
not include any excerpls or additional hints from original source; all necessary informa- tion must be inherent in the question.
Multiple-Choice Choice Format
Each question must include one correct answer and three plausible distractors.
Ensure distractors realistic, closely related the correct answer, and obviously incor- rect.
Provide concise explanation for the correct answer, clarifying why is correct and why the other options are not.
5. Question Volume and Uniqueness
questions as are warranted by the subject matter. Each question should address unique aspect of the topic without overlapping with or referring to any other question.
Generate as many
Formatting
Number each question sequentially.
List answer options A, B, C, and D.
Clearly indicate the correct answer.
Provide an explanation immediately following the answer.
Follow the Outputs Format exactly.
Outputs Format:
** *uesstion 1**
Question text here.
A. Option A text.
B. Option B text.
C. Option C text.
D. Option D text.
**Correct Answer**: [Correct Option]
**xxpanattione Explanation text here.
*uuestion N**
Question text here.
A. Option A text.
B. Option B text.
C. Option C text.
D. Option D text.
**Correct Answer**: [Correct Option]
**Exlaaationn Explanation text here.
31

---

Published as a conference paper at ICLR 2026
Special Note on Independence: Fach question must written as an independent unit. not include any references or implicit connections to other questions. Ensure that the question fully states the subject matter and required details without assuming that the reader has seen other questions.
Evaluation Data Verifier Prompt
You are cybersecurity evaluation data verifier. Your task is to review generated multiple-choice question along with its answer options, correct answer, solution (if provided), explanation, and the original context used to generate the evaluation data. You will be provided with one QnA at time. Your review must adhere rigorous checklist and include explicit chain-of- outlining your reasoning. Use the following checklist during your evaluation: Checklist for Validation:
1. Self-Contaiie Coff-Containmett
The question must be fully contacont It should include all necessary details so that can be understood independently without references or implicit reliance on external context, other questions, hints.
Complete Format:
The question must include exactly four answer options labeled A, B, C, and D.
The correct answer must be clearly indicated.
Single Correct Answer:
There must be only one correct answer.
Plausible Distractors:
All incorrect options (distractors) should be realistic and closely related to the correct answer.
Consistency:
The question text, options, correct answer. solution (if provided), and explanation must be consistent with one another and with the original context.
Focus on Inherent and Fixed Details:
Base questions core cybersecurity concepts such definitions, technical mechanisms, pre- requisites, usage guidelines, mitigation strategies, consequences, classification principles, etc. Avoid dynamic or subjective details that could change over time (c.g., the current status or version of vulnerability or tool). Focus on inherent, static properties that remain constant.
7. Relevance for Cybersecurity Assessment:
The question should important for assessing the model's knowledge in the cybersecurity domain.
Formatting:
The content must follow the exact output format provided below.
Process Instructions:
Step 1: Carefully review the generated question, multiple-choice answers, correct answer, and ex- planation by referring both the original context and the generated content.
Step Evaluate each component using the above checklist.
Step 3: Summarize your reasoning and checklist outcomes in chain-of- of-hhuught..
Step 4: Based on your evaluation, output single ISON object following structure below:
"question": The parsed question text.
"answers": ": An object with keys A, B, C, and D corresponding to each answer option.
"solution": The correct option letter (A, B, C. or D). "explanation" The explanation text. "review sumeeawwssmmrrrt detailed account of your reasoning process checklist evalua- tion.
"passed": true if the generated content meets all criteria, or false it fails any check.
Output Format Example:
32

---

Published as a conference paper at ICLR 2026
**Stepl: QnA Review**
[WRTTE YOUR REVIEW FOR STEP 1]
Step2: Checklist**
Self-Containment: [true/false]
Complete Format: (true/false]
Single Correct Answer: [true/false]
Plausible Distractors: [true/false]
Consistency: [true/false]
Focus on Inherent and Fixed Details: [true/false]
Relevance for Cybersecurity Assessment: [true/faise]
Formatting: [true/false]
* Steep:: Summary**
|Write YOUR SUMMARY FOR STEP 3]
** Step4; Final Output** jscn
"question": "What the primary purpose of firewall cybersecurity context?",
"answers":
activity", "B": "To filter incoming and outgoing network traftic",
"A": "To monitor
"c": "To encrypt data transmissions",
"D"; "To manage passwords"
b,
"sclution"; "B",
"explanation": "The primary function of a firewall is to control network traffic by filtering data packets. Option A is incorrect because monitoring user activity is not its primary function; option refers to encryption, which is handled by different systems; and option is related to account management rather than traffic filtering.",
"review_summary""" "Verified self self-connaiinent, complete format, single correct answer, plausible distracters, consistency, focus on inherent and fixed datails, and relevance for cybersecurity assessment. No issues aetected.", "passed"; true
Open-ended QA Benchmarks For the open-ended QA benchmarks, designed pipeline that transforms raw seed_data into diverse evaluation items. The process begins with the Evaluation- Plan Builder Prompt, which analyzes the seed data and proposes distinct evaluation types with corresponding instruction templates, answer guidelines, and context excerpts. Next. the Question- Answer Generator Prompt converts each evaluation plan into a concrete question-answer pair that fully self-contained, grounded, and directly testable. We then apply the Open-QA Verifier Prompt, which evaluates each generated item against detailed checklist covering format, clarity, grounding, domain meaningfulness, and overall internal consistency. This verification step ensures that only well-formed, seff-sufficient, and security-relevant QA items proceed to the final stage. We employ both Llama-3.3-70B and Qwen2.5-72B independent verifiers, accepting item only when both models vote "pass." thereby reducing false positives and improving overall benchmark quality. To- gether, these prompt templates ensure systematic coverage of cybersecurity reasoning skills while
33

---

Published as a conference paper at ICLR 2026
avoiding reliance external meta information. A qualitative example of OpenQA items is shown in Fig. 11, while the prompt templates used for OpenQA creation are presented below:
Cybersecurity Knowledge
Question: What techniques used by the SEASHARPEE malware?
Reference Answer:
The SEASHARPEE malware employs several techniques, including:
Command Scripting Interpreter (T1059:003): SEASHARPEE can execute commands on vic- tim systems using a Windows Command Shell. This allows the malware to run various commands and scripts to perform malicious activities.
2. Indicator Removal (T1070:006): SEASHARPEE can timestomp files victim systems using Web shell. Timestomping involves altering the timestamps of files.
Ingress Tool Transfer (T1105): SEASHARPEE can download remote files onto victim systems. This technique allows the malware to transfer additional tools.
4. Server Software Component (T1505:003): SEASHARPEE is Web shell, which means it can be deployed web server provide backdoor for the attackers.
These techniques collectively enable SEASHARPEE maintain persistence execute commands.
Cybersecurity Skill
Question: Classify the type of cyber attack represented by this code snippet:
const leak asyne (char) fetch tr/2 + char) let w window open curr + char + "#" rew.ee - curr length 2)
const check = asyne O => ( try
w. origin
eatch
("/e ("/2 char)
return
setTimeout 100)
check ()
Reference Answer:
This code snippet represents browser-specifie information leakage attack. The attack the following techniques:
URL Length Manipulation: The code constructs a URL with a very long fragment identifier to trigger
specific browser behavior.
2. Oriss-Orrginn Exception Handling:
The code attempts access the origi i property of the newly
opened window (w., orroigi in)..
Character Set Enumeration: The code iterates over predefined character set to test each character.
Timing-Ban Based Brute Force: The setTimeout function is used to repeatedly check the window at intervals.
By combining these techniques, the attack can leak information from the server by observing the behavior of the browser and the server's responses the constructed URLs.
34

---

Published as a conference paper at ICLR 2026
Cybersecurity Tool
Question:
Explain how to use btacanner to extract information from Bluetooth device without pairing.
Reference Answer:
btscanner is tool designed to extract as much information as possible from a Bluetooth device without the need to pair. Here's how to use it:
Installation:
Install scanner using the command: sudo apt install btscanner
Basic Usage:
Run the tool: btacanner
Display help: btscanner -help
Options:
-no-reset: Prevents the Bluetooth adapter from being reset before scanning. -cfg=<file>> Specifies a configuration file.
4. Interacting with the Tool:
Once running, scroll through the list of found devices.
Press Ent enter probe device (RSSI, link quality).
Use q to quit the screen and Q to quit the application.
Saved Data:
Device info is stored in - /bts/ <ODDDD// fnf..
Timestamps saved /bt BBB////DDDDDDDDDE htw.//DDDDe/iimmmtmmmm.....
By following these steps, you can effectively use stscanner gather detailed information about Bluetooth devices without pairing.
Figure 11: Qualitative examples of RedSage open-ended Q&A. Each benchmark item includes question and its reference answer derived from the seed data.
Evaluation-Pae Builder Prompt
You the Evaluation-PPa Builder for Open-QA cybersecurity benchmark. Goal
Analyse the provided seed_data and propose every realistic way L.IM could be tested on it.
each proposed test, output high-quality ready- ready-to-use instruction template.
If the test requires passage, also extract verbatim context excerpt.
Input
seed_data: passages, logs, configs, code, write-ups, documentations, frameworks, other eyber- security artefacts.
(No external sources allowed.)
Reference list (examples, not exhaustive): Fact Recall Threat/TTP Classification Log Anomaly De-
tection Exploit Plan Synthesis Next Step Suggestion Command-LLin Construction Command-Liin
Analysis Log Analysis Vulnerability Identification Secure Configuration Check Patch/Mittigatinn Recommendation Tool Output Interpretation Threat-Intel Summarisation Attack Chain Mapping Procedure Synthesis Red-Team Report Drafting IOC Extraction Payload De-obfuscation CVE Prioritisation Misconfiguration Reasoning OSINT Inference Social- Faggial-ngnnc Detection Feel free invent additional types that better fit the data long they meaningful for cybersecurity assessment.
Rules
1. Use only information that appears in seed _dttdddaaaaaa invent nothing.
Think step-by-step. then output the final JSON at the end.
Propose distinct evaluation types.
For each type return these fields in this exact order:
wwwlltttnnnnnte (<= 5 words)
purpose (one sentence)
35

---

Published as a conference paper at ICLR 2026
inatructioo_tentte late ate (user prompt) test needs passage, include the place- holder
<CONTEXT>
wrapped triple back-ticks. Otherwise omit the placeholder entirely.
answer_ seswer_guideline (what constitutes correct answer)
context excerptte_ppppttt (verbatim text 2048 tokens from seed data, preserving line breaks) Required only when the placeholder appears; otherwise the empty string ""
5. Avoid questions about references, authorship, version history, or other metadata that may change over time.
Ensure
and answer answer_guideling are fully grounded in
seed_data. Do not hallucinate.
Since seed_data are given during evaluation, the ruction penectiionttmmlllee template atte and answer_guide
must be self-contained. If context is needed, use the placcholder
<CONTEXT> and provide the context accordingly.
8. If context is needed, omit the placeholder and set context_excerpeee
""
9. Avoid phrasing like "based on the seed data" or "as mentioned in the seed data."
10. Plans must be distinct; do repeat the evaluation type with different wording. If ingful grounded evaluation exists, oulput an empty list [] for evaluation plan.
11. Strietly follow the output format exactly as specified below.
Output Format:
## Content Analysis and Evaluation Plan
Your analysis of the seed_ goes here
Final kvaluation Plan
"isson
"evaluation_plan":
"evaluation_namee: "<nane requiring context>",
"purpose": "<single-sentence purpose>", iinstructio_ttmmplleee "<prompt with in \I<CONTEXT><A"">
placeholder>",
"answer_guideling" "scriteria for correctness, depth, and helpfuiness>", "context_excerat": "<verbatim excerpt pulled from seed data>"
"evaluatioon_name" "<name without context>",
"purpose": "<singge-sentence purpose>", "instruction template": "cself-contained prompt with placeholder>",
"answer_guidellne"" "<criteria for correctness, depth, and helpfulness>>, "context_excorptt: ""
Question-Answer Generator Prompt
You are the Question-Answer Generator for an Open-QA benchmark. Given:
36

---

Published as a conference paper at ICLR 2026
eval uat oociionnpllnn JSON with:
evaluation_name
purpose
mnstructioonttempleeee (may include <CONTEXT> placeholder)
answer_guides
context_excerge (verbatim 2048 tokens, "" if none needed)
seed_ data: the full source text from which any excerpt was drawn.
Your Job:
Produce one high-quality QA item (one question, one reference answer) that tests the intended capability in the evaluation plan. The QA must be self-contained and grounded only in the provided materials.
OUTPUT OVERVIEW
When information sufficient you must produce, in this order:
1. Analysis section (brief).
Sufficient Information flag.
3. Final OpenQA section (Evaluation Name, Question, Reference Answer).
If information is insufficient, the Insufficient Information section below.
QUESTION CONSTRUCTION
Start from evaluation, peppaatiiooppaaaa._ee org.ap/e.e..a.........ooooooooor Rewrite for clarity natural flow.
2. If the template contains <CONTEXT>, replace it with the literal contents of context_excerpe wrapped in triple backticks. Preserve line breaks.
3.
is empty, write fully self-contained question. Do not imply hidden or
external text.
Include only minimum context required test the targeted skill. Avoid leaking large amounts of seed data.
5. Ensure question and reference answer together test the intent expressed in purpose and grad- able under answer_guider line.
The QA must be fully self-sufficient. The tested model and grader will not see seed_datee
NO META REFERENCES (critical)
The user-facing question must NOT mention: document, source, seed data, excerpt, dataset, bench- mark, grader, rubric, evaluation plam, or similar meta terms. Rewrite meta phrasing into direct instruc- tions.
Examples:
Template: "Refer to the excerpt to identify the vulnerability." Rewrite: "Identify the vulnerability in the code below."
Template: "Using provided seed data, explain..." Rewrite: "Explain..."
REFERENCE ANSWER QUALITY REQUIREMENTS
Must fully satisfy inswer_guided ine and demonstrate appropriate reasoning and depth.
Must be grounded only in context context_eeceept and broader seed dat invention external facts.
Provide as much detail as needed (unless explicitly constrained).
Reproduce literal phrases exactly when required.
Include every element required by a answer gu de 1 ne.
If mullippe acceptable variants exist, list them clearly.
INSUFFICIENT INFORMATION
If ivaluation_ae plan seed data do not provide enough produce correct, grounded answer: Output ONLY:
### Sufficient Information for Grounded OpenΩA: False <short explanation of what is missing>
37

---

Published as a conference paper at ICLR 2026
not output the Final OpenQA section.
REQUIRED RESPONSE LAYOUT (when sufficient):
Analysis and Thinking
[Brief notes: how you interpreted purpose, rewrite decisiona, grounding sufficiency, and how the answer meets the quideline.]
### Sufficient Information for Grounded OpenQA: True
FFF Final OpenΩA:
#FF Evaluation Name: <eopy evaluation_name.
#### Question: <final user-facing question meta references>
### Reference Answer: <grounded,
answer>
Open-QA Verifier Prompt
You are the Open-QA Verifier for benchmark that evaluates question-answer pairs intended to test large language models in the cybersecurity domain. Inputs
question: the final user-facing prompt.
re reference_answerr the expected ground-truth answer that downstream models must produce.
seed data: optional context where the question and reference not always present, when reference answer should factually correct and grounded in the seed data.
derived from. This is
Goal
Apply the checklist below to rigorously evaluate this QA pair. For each checkpoint:
Think step by step, explicitly writing out your reasoning (chain thought).
Then decide True if the pair passes that criterion, otherwise False.
Checklist
Format Parsing
No missing text, stray Markdown markers, or JSON artefacts.
Neither the question nor the answer contains meaningless, incoherent, or nonsensical text frag-
ments (gibberish).
2. Self-Sufficiency of Question
The question stands alone: it does not depend on unseon context or data.
Any excerpt or data it needs is included the question (c.g. in code block).
The evaluated model can answer it fully without hidden additional context.
Clarity & Completeness of Question
Clearly worded and unambiguous.
Contains all the information needed to produce the expected answer.
4. Mcaningfulness for the Domain
The question tests meaningful concept, procedure, skill, reasoning step, or knowledge rele- vant to the benchmark's 's domain (not generic trivia).
5. Alignment with Expected Answer Type
The question obviously requests type answer provided (list, explanation, step-by-step. command, short snippet, etc.).
38

---

Published as a conference paper at ICLR 2026
No Unjustified Assumptions
The reference answer does not rely facts, hidden artifacts, conclusions not present in the question's
generally stable domain knowledge.
7. Reference Answer Quality
Fully satisfies what the question requires, with skipped key points.
Avoids invented externally hallucinated content.
Consistency & Accuracy
No contradictions between question and answer.
Technical or factual details are internally consistent.
Language Readability
Clear, professional English with no profanity or irrelevant comments.
10. No Redundancy
The question is focused and not unnecessarily repeated or broad.
11. No Answer Overleakage
The question does simply give away the solution embed the reference answer inside the prompt.
12. Factually Correct and Fully Grounded (if seed_data is present)
The reference answer must be factually correct and grounded in the provided seed data to avoid hallucinations or inaccuracies.
Final Decision Logic
checkpoints are True, then verdict = PPASS"
If any checkpoint is False, then verdict "FAIL" and you must briefly explain why for each failed checkpoint.
Also provide final OpenQA Quality Score from 0 to 10, where:
10 outstanding benchmark item, exceptionally well-construtted, highly challenging and clear
5 average, acceptable but could be improved
0 entirely unsuitable (incoherent, trivial, off-topic, or otherwise broken)
Output Format:
Checklist Results
1. Format & Parsing:
Reascning: <Your thought for this point>
Result: True
False
2. Se1 Self-sufficiency of Question:
Reasoning: <Your thought for this point> Result: True False
12. Factually Correct and Fully Grounded (if seed_data is present):
Reascning: <Your thought for this point> Result: True False
Verdict:
PASS FAIL
Issues:
<short explanation for each failed checkpoint> (If the verdict is PASS, write "Issues: \nNone.")
OpenΩA Quality Score: <integer from
39

---

Published as a conference paper at ICLR 2026
B TRAINING DETAILS
Our training pipeline uses the open-source Axolotl framework (Axolotl team, 2023) for Contin- ued Pretraining (CPT), Supervised Finetuning (SFT), and Direct Preference Optimization (DPO). Axoloti provides a streamlined interface training LLMs through YAML configuration files that specify the base model, datasets, and training parameters. This design facilitates reproducibility, as experiments can be replicated simply by sharing and running the corresponding configuration file.
B. PRE-TRAINING DETAILS
Our RedSage continued pretraining (CPT) followed staged curriculum. We initialized from the Qwen3-8B-Bil SB-Base checkpoint, continued training on CyberFineWeb (Chunks 1-5), and then per- formed an additional stage the combined RedSage-Seed and SadSage--ummp corpora. This pro- gression first reinforced broad general-domain coverage from CyberFineWeb before incorporating high-quality, domain-specifer cybersecurity knowledge.
We conducted training on nodes, each equipped with 64GB NVIDIA A100 GPUs. We used an effective global batch size of 1024.
An example Axolotl configuration file used for pretraining each data chunk is shown below:
RedSage Pretraining Config
base_model: Qwen/gwen3-Gtt-aase or replace with last pretraining
checkpoint bf16: true
datasets: path: hBBAAA--TTI1111000000-----000000 type: completicn
deepspeed: degsgpeed_coofigg//erooo... json
oval_steps: 3800
gradiont__ccmmllatioo_eeeee
gradieets_cheexppinnnnng: true
learning_rate: 2.5e-06
load_in_8biee :bit: false log_with: wandb
tensorpoard
Ir lr_scheduler: micro_batch_sizer 32 max_grad_norm: 1.0
constant_with_warmmp or constant for next-checkpoint
num_epochs:
optimizer: adamw adamw_torch torch
output_dir- save_strategy; epech saves_pez saves_per_cpoch. 1
seed: 2442
sequence_lengtiin 32768
sequenceeparrallll: true torch_compile false
crust_remoted code: true
use use_tensorboard: true
vel_set_ssie size: 0.01
warmup_steps: 1000 # or remove for next-checkpoint
B.2 POST-TRAINING DETAILS
Following the CPT phase, we performed post- post-training in two stages. First, we conducted supervised finetuning (SFT) using augmented RedSage Conv dataset together with general instruction data
40

---

Published as a conference paper at ICLR 2026
from the non-reasoning subset of SmolTalk2', This stage allowed the model to specialize in eyber- security conversations while retaining general instruction folloooin capabilities.
Second, applied preference alignment via Direct Preference Optimization (DPO) using the open- source Tulu 8B Preference Mixture dataset (Lambert et al., 2025). This alignment phase refined the model's responses to better reflect human- aaaan-preferred outputs.
The Axolotl configuration for the post- post-training stages is shown below:
RedSage Supervised-Finettuini Config
: [REPLACE-WEL REDSAEE SEODSAGE--ASE--OO 2ASE-XODEL]
srust_remote_codeee true
auto_resure from crem.cchckoointss: true
bf16: true deepspeed: chapeentt_ccckkiniing true
json
sequence_parallet true
sicrr_bbatch_sicc: 32
paailett_accmmllaioo_sstsee 1 nun_epochs:
sequence_lengthn 32768
optimizer: adamw_tordh
ir ir_scheduler: cosine lhariing_rater 2.5e-5
bright_decaay 0.05
warmup_ratio: 0.01
cosine_min_lr_ratio:: 0.01
chat_ chat_template: jinja chat_templaate_jinj.. [BBDAAAG--NTTH--00000000A-C----E
datasets:
Conversation Datasets
path: SS9AA/GSHHTTT--BB33---0-0EEA-112 type: chat_template all
name:
field_messages: conversations wttpppppppppppppeeeeee
role: from
content: value
path: (BB2AAGCHHTTT-N00000AAAA00000100
type: chat_template name: formatted
field_messages: messages jesaae__rroprrtyyraappnggg
role: from content: value output_dizn
save_steps: 0.25
eval_stepa 0.25
val_set_sizee 0.01
log_with:
wando
tensorboard
'General SFT datasets: HuggingFFce//R/smooaalkk
41

---

Published as a conference paper at ICLR 2026
use_tensorboard: true
save_total_lime coadinn_bitt false
torch_compile: false
special_ tokens:
eos_token: <lim_end|>
pad_token: <|endoftext(>
B.3 ESTIMATED TRAINING TIME AND COMPUTATIONAL COST ANALYSIS
Continued pretraining from Qwen3-8B-BBl on the CyberFineWeb (CFW) dataset was executed in 24-hour maximum-runtine chunks, with average of 20 effective training hours per chunk. Five such chunks required approximately 100 hours to produce the RedSage-8B- -BBCFF checkpoint. Ad- ditional continued pretraining on RedSage-Seed and RedSage-Dump took roughly 10 hours, yield- ing RedSage-8B-Base Supervised fine-tuning on RedSage-Cood and general instruction datasets (SmolTalk22) required about 16 hours for two epochs, and DPO alignment using 8 Ax1100 GPUs added another hours. In total, the full training pipeline consumed approximately 134 wall-clock hours (~5.5 days), corresponding to more than 4,000 GPU-hours. A detailed breakdown of each stage is provided in Table 11. Variations may arise from distributed-traininng overheads, including communication latency and checkpoint restarts.
Table 11: Estimated training time and computational cost for the RedSage-88 pipeline.
Stage
Output Checkpoint
Time (h)
GPU-hours (approx.)
Continued Pretraining (CPT), epoch, 32xAI00
CPT: Webe
RedSage-8B-CFW RedSage-8B-Base
~000
~3,200
CPT: RedSage-Seed & -Dump
~0
~320
Post Post-training (SFT: epocks, 32xA100: DPO: epoch, 8xA100)
SFT: RedSage- Conv & SmolTalk2
RedSage- 8B---BBInn Ins MalSage-8B-DPP
16
~512
DPO: Tulu Preference Mixture
~
~64
Total pipeline
RedSage-8B -B-DD
133 (~55.5 days)
~,,096
C EVALUATION DETAILS
For replicable evaluation, we implement and evaluate RedSage-Bende Bench and prior cybersecurity benchmarks in HuggingFace 1 gigheeall (Habib al., 2023). The details of the compared models, tasks, and metrics for each evaluation are described in the next subsection.
C.1 EVALUATION SETUP
Compared methods. We benchmark RedSage against open general-purpose and cybersecurity- focused L.MMs, summarized in Tab. 12. The general baselines Llama-3.1-8B and Qwen3-8B;; the specialized baselines are Llama-Primus (Base and Merged), Foundation-Sec (Base and In- and DeepHat-V1-77 For each model the table reports param- eter count, backbone, and the Hugging Face card used to obtain configurations and weights, which supports strict reproducibility. Base models are evaluated in plain completion mode, instruction- tuned models use their official prompt templates, and Qwen3 is run in non-reasoning mode for parity. The suite spans 7-8B parameters across Llama, Qwen, and Mistral backbones, enabling balanced comparison by capacity and training style.
struct), Silyy-Cybersccurty-7B--ddd
42

---

Published as a conference paper at ICLR 2026
Table 12: Evaluated baseline models and their Hugging Face cards.
Model
Params (B)
model
Hugging Face
Llama-3.1-8B
N/A (base)
meta-1Lama/Llemm .1-88
Qwen3-8B
Qwen3-8B-
Qwen/Qwen3-8B
.laam-Prrmuss,
8
trrend-cybert.oon/L
1ama-Primus-Base
Llama- Llama-Primus Merged
8
Llama-3 3,1-8B (merged with
txondnicro-@llabbb
3lama-3.1188 3.1-8B-Instrucc) Llama-3.1-8B
Lama-Pzimus-Nerged
Foundation-Sec-8B
8
fdtn-al/Foundation
-Sec-8B
Coundattio--Se-SS---nseee
Foundation-Secah
fdtr--a//ooudaation
(Llama-3.1 1-BBB backbone)
-Sec-8B-Instruct
LilyyCyybersccurtyy----uu
segolllylaab/Liil
y-Cybersecurity-7
B-v0.2
DeepHat-V1-77
Qwen2.5-Coder-7B
Deepilat/Deepilan
1-73
C.2 REDSAGE BENCHMARKS
MCQ Evaluation Protocols. Models prompted select single option letter (A-D) given question and its choices. We compute the log probabilities of the option tokens for the next-token prediction and take the highest-probabilty option as the model's answer. This approach avoids parsing errors and ensures the model outputs only the option letter. The MCQ prompt template is shown below.
RedSage MCQ Prompt Template
You are given multiple choice questions. Answer with the option letter (A, B, C, D) from the given choices directly.
Question: (question)
A. [option[0]]
B. (option [1]]
C. (option [2]]
D. [option[3]]
Answer:
Open-ended Q&A Evaluation Protocols. We adopt LLM-as- Judge rubric that assesses both fac- tual correctness (Truc/False) and answer quality (0-10), considering helpfulness, relevance, depth, and level of detail. All judgments produced using Llama-3.3-70B the evaluator. The system prompt and template for the rubric are provided below.
RedSage Open-ended Q&A LLM-assJudge System Prompt
You are the Judge for an Open-QA cybersecurity benchnark.
| Inputs
question: the final user-facing prompt reference_anewer: the ground-truth answer
model_answer: the response produced by the evaluated model
Goal
Evaluate model_ answer on two aspects: (1) factual correctness, and
43

---

Published as a conference paper at ICLR 2026
(2) overall answer quality (0-10).
## Checklist
1) Correctness
Is the answer factually accurate?
Does it align with the reference_answer and grounded
cybersecurity knowledge? Any hallucinations, contradictions, false clains?
Output rule for correctness:
True if correct and grounded
False if factually incorrect, hallucinated, contradicted, clearly wrong
2). Answer Quality Score (0-10)
Rate overall quality only if at least partially correct: Helpfulness (dces it answer the question?)
Relevance (focused and on-topic)
Depth (reascning or understanding)
Level of detail (complete and specific enough)
Scoring guide:
10: perfect accurate, complete, deep, fully relevant
8-9: strong minor omissions or small inaccuracies
6-7: moderate useful but lacking depth or detail
4-5: weak vague, shallow, or incomplete
1-3: poor limited usefulness clarity
invalid or gibberish
Instructions
Use chain-of-thouuht privately, but present only a final analysis in <analysis>.
Be strict on correctness: any factual error correctness=False.
If corroctnoss-False,, cap score 3 lower.
If correct but shallow, keep correctness-True and assign lower score.
Output Format
Return exactly these three blocks in order. Do not add text outside the tags.
<analysis>
Free-form justification, You may write anything here such as step-by-step reasoning, comparisons, errors spotted, strengths, weaknesses, etc. between the model_ansee answer and reference_ _aswe. Make sure your analysis is detailed and all aspects of the evaluation checklist.
### Correctness Analysis and justifi fication for the correctness eva" uation..
### Answer Quality Score Analysis and justification for the answer quality score.
#### Helpfulness
44

---

Published as a conference paper at ICLR 2026
Justification for the helpfulness aspect.
#### Relevance
Justification for the relevance aspect.
#### Depth
Justification for the depth aspect.
I|P Level of Detail
custification for the level of detail aspect. </analysis>
<correctness>
True or False </correctness>
<score>
0-10 (integer only) </score>
RedSage Open-ended Q&A LLM as Judge Prompt Template
1System Prompt]
Question:
{question)
Reference Answer:
(reference_answer)
Model Answer:
Qualitative Results of RedSage OpenQA. We present three RedSage OpenQA examples that span cybersecurity frameworks, offensive skills, and tool usage. In the Olympic Destroyer attribution case shown in Fig. 12, RedSage DPO correctly identifies the Sandworm team, while baseline models misattribute the malware to other Russian APT groups. For the CSP bypass example in Fig. 13 and the Koadic tool-usage example in Fig. 14, RedSage 8B DPO accurately recognizes iframe- based CSP evasion and generates the exact Koadic command line. In contrast, the baselines omit key details or produce malformed commands. These cases illustrate that RedSage exhibits stronger grounding in authoritative cybersecurity sources and improved precision in operational reasoning.
Qualitative Results of LLM-as-Judge. To further illustrate the differences captured by our LLM- as-Judge pipeline, include qualitative evaluations comparing RedSage with the baseline model using the tool-based question shown in Fig. 14. As shown in Fig. 15, the judge marks RedSage's answer as fully correct, assigns perfect score, and highlights the precise command construction and clear supporting explanations. In contrast, the baseline model receives failing correctness label and substantially lower score because uses an incorrect command line flag, even though its surrounding explanation is detailed. These paired results emphasize the sensitivity of our evaluation framework fine-gu iinegeeined correctness, particularly cybersecurity scenarios where small syntactic deviations can lead to incorrect or unsafe tool behavior.
45

---

Published as a conference paper at ICLR 2026
Cybersecurity Framework:
Pourditton the instrvt
The theart group known have used Olympic Dasnoyer rolvvies
Sandworm This resporatie daployina
la 16P128, also fenoee os "Fancy Boot The group befiored to
mahware
Winter dlympics Pyyongehang South
eocateed kussias govertmen madusee warused
https//attack mitre ortjsoofwwarr/50335
Korea The rain puppois the natwore was to intected
cyderettack cntte Water D'mppecs PyPoorge South
computer systam nope sadee by twargingg varcurr naiive Window
skil arel sary cirstactiin tacks dlyneso Destroee
has worn festuree to sproad scress computer es restractive
Which threat group known to used the Olympic Destroyer malware?
scurces MITEE ATMBICe tummowrr reporta
franoyterees firm lik houtyee Symartes, or Casperity
The thresl
inferelle "Sersharre" sor "Oreap Ni1
haveuseed clyret: Dentrope
Boer, aloo lononn APT18,, Sufley, Sedint, They Rusi strtesipee oytor expireago groug bedevcnd
Chrmple
molvase
aracis, inckading Oaympic Despoyer srrac
Winter iy Houthh
"mon Vikkng This group belivwecto be part of miltary
intit genor agenzs, the CRU, They ane retoricis for their cyter
so eeaddate with Ruosan millary gancs sgancc CRU.
clymeso Oustryes used iyyoorutuckk 2098
Olympica in Pyeengshang Korms doruptices
Figure 12: Knowledge framework example from RedSage OpenQA. For the Olympic Destroyer question, RedSage 8B DPO matches the reference attribution to the Sandworm team, while baseline models misattribute different APT groups. Best viewed in Zoom.
Cybersecurity Offensive Skilt:
The provided coce byparaes Conrent Securit Pallcy pxtectionn thioug "Mioon Hipociiet levctag falwing
ge 68--PD
Data Source:
atteeclghhub
New Techniqua Dreaidower
events Howewer tharess
esscusabee sacuusatee
pernaaiing webbooneent securite coolioooo oo typusa!
resoence 0951 trowee may megt vonecketyy may flgg
An :chtamss abensnn dyner cally created using
scrpe apcchort as viladio This couatac window opportsing
for the mulikoss so veccito before the CSP onfarees
we's josubsmatopuunn zec, which
Question:
excurable reasarue cuses theu liluty
identify spectfic through Injcction bypess effective against
"Rypossing
"Mppeecinng Flanatt
t'sclituao seops into
the desoribed CGP configuration?
thams appendtec decument
would nommaly Heverver "I banaks doc ment comment the brnwerr may mosn enftorn scrip ard strectve for itramia contant, slowing minng acrptito Exccuto
docammnt waw.yappeaatiiieeeeeooooo unaldin viiats
jawssoripe
Higpeiing ia loedin..
freme wow wtateertrtttttf friew")
+Sccppt injectioon theme
Morppihing flone sooketicem
document (ithamm))
acrigt docammnnt creatc Soaneeteeeeet///arrits
script trc "eesomple com/ccpps
Eurne's decument
viciding main CSe Wrome's conteets indreel mity
window//nnmeej cocument.en..mmmmm www.spppppoohhteeeeeeeeee
explits feet that the tlaeers see io ressunee.
aiowing scrpas to injctad into decamen: comnes.
Figure 13: Offensive skill example analyzing JavaScript that bypasses Content Security Policy. RedSage 8B DPO correctly identifies iframe injection as the evasion technique and explains how each step cireumvents the configured script sre directive. Best viewed in Zoom.
RedSage: LLM-as- Judge Output
<analysis>
provided is not enly factually correct but also offers additional helpful information and context. It accurately constructs the command line to start Koadic with the specified autorun file and restore option. The command line provided in the model answer matches the reference answer exactly, which is "koadic --autorun nutorun_commance .xxt --restore restore cestoor_data..oe json'' This indicates that the model has correctly understood the requirements and has provided the precise command needed to achieve the desired outcome.
The model
### Correctness
The model answer is correct because it accurately reflects the command 1ine needed to start coadic with the specified parameters. There are factual errors, hallucinations, the provided command line. The acditional information about the command assuming "koadie" is in the system's PATH and the explanat of the flags ("--autorun" and "--restore") are accurate and useful but do not affect the correctness of the command itself.
contradiction in
46

---

Published as a conference paper at ICLR 2026
Cybersecurity Took
Prumdation ren-esttutt
To sont Koeds with an nutron named aatonu conuucccenn commanda te and eratee morrstor option using restonulitti pon you nead contrand inarthos how you
http://wwww.alllongooooooe
Beeelage dPo
To start Roadi vichhith sato un namned auseenn cor commundadd and mokeerte teethe moree option using restorulion jone you can tellving comm
Construct command line to start Koadic with an autorun file named
commands
"caad:: conmand start Keadle formework.
commar apeciilia
restore option using
restora_catajson
This command eeuumes that hoadic inypar systers PATIL tirct.
yourray rece provke thc spoit to koods cecuuatic wwattonn Tng specitie contaring commandetto
specifier J6ON
previsu
antorndiuully
spodii
X
chissed brabre
Figure 14: Tool-usage example for the Koadic framework. RedSage 8B DPO provides the exact
command, while the baseline model produces a non-matching command. Best viewed in Zoom.
### Answer Quality Score
The answer quality score should be high because the model answer is not only correct but also provides additional useful information that could help user understand how to use the command troubleshoot potential issues (like the executable not being in the PATH).
UUH Helpfulness
The model answer is very helpful because it directly answers the question with the correct command line and provides supplementary information that could aid in successfully executing the command.
#### Relevance
The answer is highly relevant it directly addresses the question asked without including unnecessary information. The command line and the explanations provided are focused on the task of starting Koadic with an autorun file and anabling the restore option.
#### Depth
The answer shows a good depth of understanding of the command and its parameters. It not only provides the command but alsc explains the purpose of the flags used, which indicates a level of comprehension beyond mere recall of command syntax.
#### Level of Detail
The level of detail is appropriate for the question asked. The model provides enough detail for scmeane understand how the commana and what each part of the command does, without overwhelming them with unnecessary information.
</analysis>
<correctness>
True
</correctness>
<score>
10
</score>
47

---

Published as a conference paper at ICLR 2026
Baseline Model: LLM-as as- as-Je Output
<analysis>
The model answer provides detailed explanation of how to start Roadic with an autorun file and enable the restore option. It breaks down the command into its components and explains the purpose of each part. However, the command provided the model answer is slightly different from the reference answer. The model answer uses the short option "-a" for auterun, whereas the reference answer uses the long option "--autorun'.
### Correctness
The model answer is almost correct, but it uses a different option for specifying the autorun filc. The correct option, as per the reference answer, is "--autorun', not "-a' This discrepancy makes the model answer factually incorrect.
### Answer Quality Score Despite the factual error, the model answer provides clear and detailed explanation of the command and its components. also includes important notes about the ethical use of Koadic and the importance of proper permissions and access.
UW Helpfulness
The model answer is helpful in explaining the purpose of each part of the command and providing context for its use.
### Relevance
The model answer is highly relevant to the question, it directly acdresses the task of starting Koadie with an autorun file and enabling the restore opticn.
### Depth
The model answer provides good depth of explanation, breaking down the command into its components and explaining the purpose of each part.
UH Level of Detail The model answer provides sufficient level detail, including explanations of the command options and important notes about ethical use and permissions. </analysis>
<correctness>
False
</correctness>
<score>
</score>
Figure 15: Qualitative LLM-as- audgee outputs comparing RedSage and the baseline model.
48

---

Published as a conference paper at ICLR 2026
C.3 CYBERSECURITY BENCHMARKS
CyberMetric (CyMte). CyberMetric evaluates general cybersecurity knowledge via multiple- choice questions with four options, curated from authoritative sources such as NIST publications. RFCs, books, and research papers using retrieval-augmented generation pipeline. The collection released in several sizes, and we use the 500-item split that was fully verified by human experts. Items span nine topical areas that include cryptography, reverse engineering, and risk assessment. Models are scored with standard MCQ accuracy. (Tihanyi et al., 2024)
SecBench (ScBen). SecBench is a large munti-dimmnssonnl benchmark for cybersecurity that in- cludes both MCQs and short-answer questions, covers two capability levels (knowledge retention and logical reasoning), and is available in Chinese and English. Questions were sourced from open materials and curated contest, and short-answer evaluation is supported by an LLM-based grader. In our study we use the English MCQ subset and report accuracy. (Jing et al., 2025)
MMLU Computer Security (MMLU-CSec). MMLU is a 57-subject multiple-choice test that measures broad academic and professional knowledge. We evaluate on the Computer Security subject, which contains MCQs covering practical and theoretical topics such as network security and cryptography. Following common practice for MMLU-style evaluation, we report aceuracy. (Hendrycks et al., 2021)
SECURE. SECURE targets applied cybersecurity with datasets built from MITRE ATT&CK, CWE, CVE, and related ICS advisories, organized into three knowledge types: extraction, under- standing, and reasoning. We use the MCQ-style subsets MAET (MITRE ATT&CK Extraction), CWET (Common Weakness Extraction), and KCV (Knowledge test on Common Vulnerabilities). The authors manually refined the pools by removing or fixing flawed questions. We evaluate with MCQ accuracy. (Bhusal et al., 2024)
CTI-Bench. CTI-Bench focuses on cyber threat intelligence and provides four tasks: CTI-MCQ for knowledge of CTI standards and practices; CTI-RCM for mapping CVE descriptions to one or more CWE root causes; CTI-VSP for predicting CVSS v3 base vectors and scores; and CTI-ATE for extracting MITRE ATT&CK attack techniques from natural language incident descriptions. While VSP and ATE are typically evaluated with regression and Fl metrics, respectively, in our study we only use accuracy across all subsets for consistent aggregation. (Alam al., 2024)
SecEval (ScEva). SecEval is a domain-focused benchmark of more than two thousand MCQs spanning nine areas that include software, application, system, web, cryptography, memory safety, network security, and penetration testing. Questions were constructed from textbooks, official docu- mentation, and standards using GPT-4 prompting, with quality control to remove invalid items. We evaluate with MCQ accuracy on the full set. (Li et al., 2023)
C.4 GENERAL LLM BENCHMARKS
ARC-Challenge (ARC-C). ARC-C is the challenge split of the A12 Reasoning Challenge, a set of grade-school science multiple-choice questions curated to require nontrivial reasoning and back- ground knowledge. The challenge subset specifically contains items that defeat simple retrieval and co-occurrence baselines, making it a strong discriminator of reasoning beyond surface cues. We evaluate with standard MCQ accuracy used by leaderboard implementations. (Clark et al., 2018)
HellaSwag (HSwag). HellaSwag tests grounded commonsense inference via sentence comple- tion. Each example presents short context and four candidate endings that describe plausible next events in physical or social scenarios. The dataset was adversarially filtered foil strong language models while remaining trivial for humans, which sharpens its discriminative power. Performance reported as multiple-choice accuracy. (Zellers et al., 2019)
TruthfulQA (TQA). TruthfulQA measures whether models avoid widespread misconceptions and misleading patterns by answering with factually truthful content across 38 categories such as health, law, and finance. It provides both generative prompts and multiple-choice variants. Following common leaderboard practice, we use the multiple-choice setting and report accuracy ensure comparability across models. (Lin et al., 2022)
MMLU. MMLU evaluates broad knowledge and reasoming across 57 academic and professional subjects that range from elementary mathematics and U.S. history to computer science and law. Each
49

---

Published as a conference paper at ICLR 2026
subject consists of four-option multiple-choice items designed test recall, conceptual understand- ing, and problem solving. Scores are aggregated as average accuracy across subjects. (Hendrycks et al., 2021)
WinoGrande (WinoG). WinoGrande is a large adversarial variant of the Winograd Schema Chal- lenge that assesses commonsense reasoning through pronoun resolution. Each example requires selecting which of two candidate nouns a pronoun refers to, with items constructed to reduce an- notation artifacts and shallow heuristics. Evaluation follows leaderboard protocol using accuracy. (Sakaguchi et al., 2021)
GSM8K. GSM8K is a collection of 8.5K carefully authored grade-school math word problems that require multi-step arithmetic reasoning. Problems are linguistically diverse and designed to encourage chain-of-thought solutions, yet the final target is short numeric answer. We report exact-match accuracy on the final answer, consistent with leaderboard settings. (Cobbe et al., 2021)
1FEll IFEval evaluates instruction following using prompts that contain verifiable constraints such as minimum length, required keywords, or structural requirements. Each prompt includes one more constraints that can be programmatically checked, yielding objective pass/fail signals with- out human grading. We report the mean compliance rate across all constraints, i.e., the percentage of constraints satisfied. (Zhou et al., 2023)
D ADDITIONAL EVALUATION RESULTS
D.1 LARGER MODEL SCALING
To assess the scalability of data curation and augmentation pipeline, conducted reduced- scope experiment using Qwen3-32B. We applied QLoRA fine-tuning (#11% trainable parameters) on partial dataset consisting of the curated RedSage-Seed subset (excluding RedSage-Dump) and Despite using only a fraction the full training data and a lightweight
50% of RedSage-Cond adaptation method, the resulting 32B model achieved consistent gains across both the RedSage- MCQ benchmark (Table 13) and a suite of cybersecurity evaluations (Table 14). Notably, the training loss continued decrease throughout the run, suggesting that full-data, full-parameter fine-tuning would yield even larger improvements. These findings indicate that the RedSage data curation and augmentation methodology transfers effectively larger models, underscoring its scalability and potential to advance cybersecurity LLM development.
Table 13: RedSage-MCQ (0-shot) scaling experiment. Values are accuracy (%). Abb: Gen = Gen-
eral, Frm = Frameworks, Off Offensive Skills, CLI Command-line Tools, Kali = Kali Tools.
Model Name
Macro
Knowledge
Skill
Tools
Ace
Gen
Frm
Off
CL.I
Kali
Qwen3-8B
81.85
80.46
78.82
86.16
83.92
75.56
Qwen3-32B
85.40
84.08
82.32
89.00
87.60
80.40
RedSage-8B-- RedSage-322 1OB-LLRR-IInS
85.73
84.20
84,98
89.06
86.80
80.30
87.53
85.68
85.04
91.46
88.76
82.78
Table 14: Related Cybersecurity Benchmarks (0-shot) scaling experiment. Values are Accuracy (%).
Best results are shown in bold.
CTI-Bench
CyMMt
MMLU
ScBen
ScEva
SECURE
Model Name
Mean
MCQ
RCM
CSec
MCQ
CWET
KCV
MAET
Qwen3-8B
75.71
62.76
54.00
88.60
76.00
73.26
.5.46
88.11
87.42
85.75
Qwen3-32B
82.31
70.04
65.60
91.80
84,00
84.23
76.23
89.46
88.72
90.06
BodSagg--BBIIns
81.30
70.56
76.70
89.80
78.00
79.91
72.48
91.45
81.34
91.47
RedSage-322 328- LRRA-Ins-O..
82.85
71.64
66.10
93.40
84.00
83.77
78.30
92.18
83.29
92.97
50


Reviews:
[{"review_id": "k6gFKiEJO2", "reviewer": "Reviewer_mCDa", "summary": "This paper curates a corpus of data (11.8B tokens) by filtering FineWeb and perform continued pre-training from Qwen3-8B-Base. The resulting models improve cybersecurity multiple-choice benchmarks. Along the way, the authors also introduce a new benchmark, RedSage-Bench.", "questions": "How were the labels for the ModernBERT-base clasisifer obtained? Were there an LLMs that were used?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-motivated: to improve a domain-specific capability (cybersecurity), go and curate data for it and fine-tune an existing model.\nThe paper is fairly well-written and easy to follow.", "weaknesses": "There is not really any methodological novelty, given this is similar to other methods like FineWeb-Edu.\nI would have liked to see evaluation of larger models (at least 32B) to see how well the methodology transfers to stronger models (one might worry that the gap will shrink).\nIt would also be good to get a closed model (e.g., GPT-5 or Claude) to get a ceiling for the new benchmark.\nThe abstract claims that the fine-tuned model improves on OpenLLM leaderboard tasks, but looking at Table 6, it seems like except on GSM8K, the RedSage is worse as would be expected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kGl1g79Efp", "reviewer": "Reviewer_f3PV", "summary": "This paper presents RedSage, a cybersecurity-focused LLM that is trained with curated pre-training and post-training for cybersecurity knowledge. It presents an elaborate cybersecurity data curation pipeline that uses LLMs for isolating cybersecurity relevant data and augmenting multi-turn conversational data for post-training. The results demonstrate how both pre-training and post-training can help improve performance on cybersecurity tasks.", "questions": "- Why is there a large imbalance between MCQ and open-ended Q&A (30K vs 240)? Is it because of human verification of open-ended benchmark? Please provide details of the human verification.\n- Some estimates of training time would help future research aiming to replicate this work\n- Were other LLMs tried for the agentic augmentation and why were Llama-3.3-70B and Qwen2.5-72B chosen?\n- It would be interesting to see how RedSage fairs against models of higher size and commercial models atleast on the general benchmarks. For instance, RedSage-8B-DPO catches up with GPT4 performance on CTIBench-MCQ.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The paper clearly demonstrates that cybersecurity curated pre- and post-training improves performance significantly without catastrophic forgetting of general knowledge\n- The cybersecurity-specific curated dataset is incredibly useful for future training tasks", "weaknesses": "- The method for open-ended Q&A evaluation is not adequately described. Line 317 mentions \"prefix exact match or regex matching\" and points to Appendix C.1 however neither the text nor the appendix provide sufficient details or references to clearly understand this evaluation.\n- The method for generating the instruction-tuned variant is not explained or referenced. A diagramatic view of how each of the variants was derived would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ADM3vdthut", "reviewer": "Reviewer_QvXR", "summary": "This paper introduces RedSage, an open-source cybersecurity-oriented LLM trained via domain-aware continual pretraining and agentic post-training augmentation. This work also proposes RedSage-Bench, a 30K-question benchmark spanning a diverse range of tasks. Authors of this paper claim the model outperforms state-of-the-art specialized baselines on both cybersecurity benchmarks and general LLM tasks, demonstrating that cybersecurity specialization can enhance both domain-specific and general reasoning.", "questions": "As mentioned in the weakness, how do you ensure LLM-generated augmentation does not reinforce factual errors or tool misuse patterns?\n\nHow would RedSage perform in interactive cybersecurity reasoning task such as CTF agentic settings on benchmarks like CyBench?\n\nDoes the author consider data contamination for the proposed datasets and benchmark?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "Clear presentation: This work integrates all stages (CPT, SFT, DPO) with substantial data at each phase, which is clearly shown in Fig. 1. Readers can have a very straightforward view on how RedSage was trained on the dataset selected.\n\nComprehensive Training data coverage: The integration of CyberFineWeb, curated RedSage-Seed, and agentic augmentation provides strong coverage across cybersecurity subfields.\n\nProposed New Benchmark: What I am interested in is that this work expands prior benchmarks by incorporating tool proficiency and qualitative evaluation, from reviewer’s point of view, this innovation can close a key gap in cybersecurity LLM assessment.", "weaknesses": "May need computational cost analysis: While \"compute constraints\" are briefly mentioned in page 4 (CyberFineWeb section), there's no breakdown of training time, GPU-hours, or carbon footprint across stages. Adding some analysis on computational cost would help readers to form a general impression on the scale of RedSage training process.\n\nLimited human validation: One of my concern on this work is the data part is heavily reliance on LLM-based verification, which could introduce subtle self-reinforcing biases.\n\nTeacher model (verifier) analysis: Augmentation uses only Llama-3.3-70B and Qwen2.5-72B. Analysis on teacher model can make this paper more solid, such as examining sensitivity to teacher model choice, temperature settings, or comparison with smaller/different teacher models.\n\nMinor issue: Font size on Fig. 3 and Fig. 5 can be larger which I felt difficulty to read", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)