NotAiLOL/Qwen2-0.5B-Math

Coding model comming soon!

Uploaded model

• Developed by: NotAiLOL
• License: apache-2.0
• Finetuned from model : unsloth/Qwen2-0.5B-Instruct-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Details

This model was trained on microsoft/orca-math-word-problems-200k for 3 epochs with rsLoRA + QLoRA.

Training Loss Graph

The model follows the Alpaca format:

<|im_start|>system
You are a professional mathematician.|im_end|>

<|im_start|>user
{}<|im_end|>

<|im_start|>assistant
{}