Qwen3-0.6B -- Dataset A SFT (Trial T2)

LoRA supervised fine-tuning of Qwen/Qwen3-0.6B on open-r1/OpenR1-Math-220k, merged into a standalone full model (16-bit). Part of an IBA NLP Assignment 04 LoRA ablation study (trials T1-T5).

Trial T2: Data efficiency -- 50% of 500 (250 rows)

Hyperparameter Value
LoRA rank 16
Data fraction 50% of 500 rows
Learning rate 0.0002
Epochs 1
Target modules q_proj, k_proj, v_proj, o_proj
Max seq length 2048

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetA-T2")
tok   = AutoTokenizer.from_pretrained("COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetA-T2")
msgs  = [{"role": "user", "content": "What is 2+2?"}]
ids   = tok.apply_chat_template(msgs, return_tensors="pt",
                                add_generation_prompt=True, enable_thinking=True)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0]))
Downloads last month
4
Safetensors
Model size
0.6B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetA-T2

Finetuned
Qwen/Qwen3-0.6B
Adapter
(515)
this model

Dataset used to train COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetA-T2