⚡ sunithaAI-small (Fine-Tuned Qwen2.5 0.5B)

sunithaAI-small is an ultra-fast, lightweight 0.5B instruction model fine-tuned on Qwen/Qwen2.5-0.5B-Instruct by developer Riya.

• Developer / Creator: Riya
• Base Model: Qwen/Qwen2.5-0.5B-Instruct (0.49B parameters)
• Quantization: 4-bit NF4 double-quantization
• VRAM Footprint: ~0.4 GB (Runs on any free GPU or even CPU / Edge / Mobile)
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Adapter Type: PEFT LoRA (rank=16, alpha=32)
• License: Apache-2.0

---

🚀 Quickstart & Inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-0.5B-Instruct"
adapter_id = "shiva123782/sunithaAI-small"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

prompt = "<|im_start|>user\nWho created you and what is your purpose?<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda" if torch.cuda.is_available() else "cpu")
with torch.no_grad():
    tokens = model.generate(**inputs, max_new_tokens=80)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))

---

📊 Evaluation & Metrics

Metric	Base Model (Qwen2.5 0.5B)	Fine-Tuned (sunithaAI-small by Riya)
Developer	Alibaba Cloud	Riya
Model Size	494M params	494M + 4.2M LoRA params
VRAM on T4	0.4 GB	0.4 GB + 30 MB LoRA
Inference Speed	~48 tokens/sec	~47 tokens/sec
Persona Adherence	General assistant	sunithaAI Persona

---

📜 Citation

@misc{sunithaAI-small-2026,
  author = {Riya},
  title = {sunithaAI-small: Fine-Tuned Qwen2.5 0.5B Instruction Model},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/shiva123782/sunithaAI-small}}
}