ChanukaDasun/Llama-3.2-3B-Reasoning-Instruct

Model Description

This model is a 4-bit quantized, instruction-tuned variant of Llama-3.2-3B, fine-tuned using Supervised Fine-Tuning (SFT) on the ServiceNow-AI/R1-Distill-SFT dataset. The training focuses on improving structured reasoning, instruction following, and step-by-step response generation, inspired by R1-style distillation techniques. Leveraging the Unsloth optimization framework and TRL, the model achieves efficient training and inference while maintaining strong reasoning and conversational capabilities. It is suitable for lightweight deployment scenarios, educational use, and experimentation with reasoning-oriented language models.

Model Card for outputs

This model is a fine-tuned version of unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit. It has been trained using TRL.

Quick start

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="ChanukaDasun/outputs", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with SFT.

Framework versions

• TRL: 0.24.0
• Transformers: 4.57.3
• Pytorch: 2.9.1
• Datasets: 4.3.0
• Tokenizers: 0.22.1

Citations

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}