Model Card for qwen3-chat-sum-5

This model is a fine-tuned version of unsloth/Qwen3-1.7B. It has been trained using TRL.

Quick start

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="hongduc05/qwen3-chat-sum-5", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with SFT.

Framework versions

TRL: 0.24.0
Transformers: 4.57.6
Pytorch: 2.11.0
Datasets: 4.3.0
Tokenizers: 0.22.2

Citations

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}

Evaluation — `checkpoint-363`

This checkpoint was evaluated on the held-out Vietnamese chat-summarization test split using Transformers + PEFT, without vLLM. The base model Qwen/Qwen3-1.7B was loaded with BitsAndBytes 4-bit NF4 quantization and the LoRA adapter from hongduc05/qwen3-chat-sum-5/checkpoint-363 was attached with PeftModel.from_pretrained.

Inference setup

Base model: Qwen/Qwen3-1.7B
Adapter checkpoint: hongduc05/qwen3-chat-sum-5/checkpoint-363
Adapter config recorded base: unsloth/Qwen3-1.7B
Backend: Transformers + PEFT + BitsAndBytes
Quantization: 4-bit NF4, double quantization, compute dtype torch.bfloat16
Thinking: disabled with enable_thinking=False
Sampling: temperature=0.7, top_p=0.8, top_k=20, min_p=0.0
Max sequence length: 2048
Max new tokens: 256
Data split: 80/10/10, seed 42; rows after cleaning: 4810; test rows: 481
Vietnamese metric tokenization: underthesea
METEOR implementation: exact Vietnamese token matching, without English stemming/WordNet

Test results

Metric	Value
Mean BLEU	0.173951
Mean ROUGE-1 F1	0.510965
Mean ROUGE-2 F1	0.237379
Mean ROUGE-L F1	0.425935
Mean METEOR	0.432727
Latency mean (sec/sample)	4.187413
Latency p50 (sec/sample)	4.101579
Latency p95 (sec/sample)	5.519641
Mean prompt tokens	432.10
Mean generated tokens	42.94