Aaqisher667/tinyllama-cyber-finetuned

TinyLlama-1.1B Cybersecurity Fine-Tuned

A fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 trained on cybersecurity instruction-response data using LoRA (Low-Rank Adaptation).

Model Details

Property	Value
Base model	TinyLlama/TinyLlama-1.1B-Chat-v1.0
Model type	Causal Language Model
Parameters	1.1B
Fine-tuning method	LoRA (PEFT)
Training framework	Unsloth + TRL
Language	English
License	Apache-2.0
Domain	Cybersecurity

Training Details

Setting	Value
Dataset	AlicanKiraz0/Cybersecurity-Dataset-v1
Training examples	~2,500
Epochs	3
LoRA rank (r)	16
LoRA alpha	16
Learning rate	2e-4
Batch size	2
Gradient accumulation	4
Max sequence length	512
Quantization	4-bit (NF4)
Hardware	Tesla T4 GPU

How to Load

With Unsloth (recommended):

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = "Aaqisher667/tinyllama-cyber-finetuned",
    max_seq_length = 512,
    dtype          = None,
    load_in_4bit   = True,
)
FastLanguageModel.for_inference(model)

Example usage:

messages = [
    {"role": "system", "content": "You are a cybersecurity expert assistant."},
    {"role": "user",   "content": "What is a SQL injection attack?"},
]
chat = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
inputs = tokenizer(chat, return_tensors="pt").to(model.device)
import torch
with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens = 256,
        temperature    = 0.7,
        do_sample      = True,
    )
response = tokenizer.decode(
    out[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)
print(response)

Intended Use

This model is intended for:

• Cybersecurity question answering
• Security concept explanation
• Vulnerability analysis assistance
• On-device / edge deployment scenarios requiring a lightweight cybersecurity AI assistant

Limitations

• Small model size (1.1B) — responses may lack depth compared to larger models
• Fine-tuned on a limited dataset (~2,500 examples)
• Not suitable for generating exploit code or offensive security tooling
• Always verify cybersecurity advice from authoritative sources

Hardware Requirements

Setting	Minimum
VRAM (4-bit)	~2 GB
VRAM (full)	~5 GB
RAM	8 GB

Runs comfortably on a free Google Colab T4 GPU.

Training Data

This model was fine-tuned on AlicanKiraz0/Cybersecurity-Dataset-v1.

Property	Detail
Dataset	AlicanKiraz0/Cybersecurity-Dataset-v1
Size	~2,500 instruction-response pairs
Domain	Cybersecurity — defensive concepts, vulnerability analysis, security best practices
Format	System / User / Assistant instruction tuning format
Source	Publicly available on HuggingFace