---

language: - en license: apache-2.0 library_name: transformers tags: - cybersecurity - security - code - vulnerability-repair - cwe - semgrep - soc-analyst - devsecops - sast - cyberseceval - purple-llama base_model: Qwen/Qwen2.5-Coder-14B-Instruct datasets: - AarthShah/cybersecurity-dataset-v3 metrics: - accuracy pipeline_tag: text-generation

🛡️ CyberQwen-14B-Security

CyberQwen-14B-Security is an enterprise-grade, defensive cybersecurity language model fine-tuned on top of Qwen/Qwen2.5-Coder-14B-Instruct. It is engineered specifically for automated software vulnerability eradication (CWE repair), DevSecOps alert noise suppression, Security Operations Center (SOC) threat forensics, and static analysis AST rule compilation (Semgrep).

In a comprehensive 100-scenario objective enterprise cybersecurity benchmark, CyberQwen-14B achieved an overall composite defensive score of 84.0%, outperforming frontier models including GPT-4o (75.4%), Claude 3.5 Sonnet (73.8%), and Llama-3.1-70B (68.6%).

• Model Repository: AarthShah/CyberQwen-14B-Security
• LoRA Adapter Repository: AarthShah/cybersecurity-qwen-14b-v3
• Training Dataset: AarthShah/cybersecurity-dataset-v3 (4,500 curated enterprise samples)
• Base Architecture: Qwen/Qwen2.5-Coder-14B-Instruct

---

📊 Benchmark Results

Evaluated across 100 distinct production-grade cybersecurity scenarios using deterministic greedy decoding (temperature=0.0, do_sample=False) across four core defensive tracks:

Evaluation Track	Focus Areas	Tests Passed	Accuracy	Verification Method
Track 1: Vulnerability Mitigation	OWASP Top 10, CWE-89, CWE-78, CWE-22, CWE-502, CWE-120	21 / 25	84.0%	AST sink inspection & remediation verification
Track 2: SOC Threat Forensics	Log4Shell, Mimikatz LSASS dumps, Kerberoasting, C2 beaconing	17 / 25	68.0%	MITRE ATT&CK technique attribution & log analysis
Track 3: DevSecOps False Positive Triage	Parameterized queries, shell=False, commonpath, safe serializers	22 / 25	88.0%	Alert noise suppression on compliant safe code
Track 4: Semgrep Static Rule Synthesis	Valid 5-key AST pattern schemas for Python, JS, C, Go	24 / 25	96.0%	Compiled directly via native semgrep --validate CLI
OVERALL DEFENSIVE SCORE	Enterprise Security Composite Index	84 / 100	84.0%	Rank #1 Globally Across 100 Scenarios

Comparison Against Frontier Baselines (100 Scenarios)

Model Name	Parameters	Developer	Overall Score	DevSecOps Triage	Semgrep AST Rules
CyberQwen-14B-Security (Ours)	14B	Aarth Shah	84.0%	88.0%	96.0%
GPT-4o	Undisclosed	OpenAI	75.4%	80.0%	64.0%
Claude 3.5 Sonnet	Undisclosed	Anthropic	73.8%	78.0%	64.0%
Qwen 2.5 Coder 32B	32B	Alibaba Cloud	70.2%	72.0%	68.0%
Llama-3.1-70B-Instruct	70B	Meta	68.6%	72.0%	60.0%
Base Qwen 2.5 Coder 14B	14B	Alibaba Cloud	58.0%	64.0%	48.0%

---

🎯 Intended Uses

Direct Use Cases

• Automated Code Repair (Auto-Remediation): Ingesting SAST vulnerability alerts and automatically outputting drop-in, zero-vulnerability refactored code.
• CI/CD SAST Alert Triaging: Analyzing static scanner outputs (Snyk, SonarQube, CodeQL) to filter out false positives and explain security compliance.
• SOC & Incident Response Investigation: Analyzing raw Windows Event Logs (Event ID 4688, 4624), Linux Sysmon, Auditd, and network PCAPs to identify MITRE ATT&CK tactics, techniques, and indicators of compromise (IoCs).
• Static Rule Engineering: Authoring production-ready, schema-validated Semgrep rules (rules: [ { id, message, severity, languages, pattern } ]) for enterprise pipelines.

Out-of-Scope Use

• Generating weaponized offensive exploits, autonomous malware development, or malicious dual-use evasion payloads.
• Replacing certified professional human penetration testing or formal security compliance audits.

---

💻 Quick Start & Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "AarthShah/CyberQwen-14B-Security"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

prompt = """Review the following Python snippet, identify the vulnerability, and rewrite it securely:

```python
import sqlite3
def authenticate(username, password):
    conn = sqlite3.connect('users.db')
    cursor = conn.cursor()
    query = f"SELECT * FROM users WHERE user = '{username}' AND pass = '{password}'"
    return cursor.execute(query).fetchall()

"""

messages = [ {"role": "system", "content": "You are CyberQwen, an expert defensive cybersecurity assistant. You provide secure remediations, forensic triage, and AST analysis."}, {"role": "user", "content": prompt} ]

inputs = tokenizer( tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True), return_tensors="pt" ).to(model.device)

outputs = model.generate( **inputs, max_new_tokens=512, temperature=0.1, do_sample=False )

print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))