You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Qwen3-0.6B-SafeMath

Fine-tuned Qwen3 causal language model checkpoint for SafeMath experiments.

Model Details

  • Architecture: Qwen3ForCausalLM
  • Model type: qwen3
  • Precision: bfloat16
  • Context length: 40,960 tokens
  • Hidden layers: 28
  • Hidden size: 1,024
  • Attention heads: 16

Training Summary

  • Epochs: 1.0
  • Batch size: 8
  • Gradient accumulation: 2
  • Max sequence length: 1,024
  • Learning rate: 1.5e-5
  • Weight decay: 0.1
  • Warmup ratio: 0.05
  • Global steps: 2,023
  • Validation loss: 0.8385
  • Validation perplexity: 2.3129

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zzzzk77/Qwen3-0.6B-SafeMath"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

Files

This repository contains a safetensors model checkpoint, tokenizer files, generation config, training config, and training metrics.

Downloads last month
3
Safetensors
Model size
0.6B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for zzzzk77/Qwen3-0.6B-SafeMath

Finetuned
Qwen/Qwen3-0.6B
Finetuned
(1363)
this model