Model Card for Qwen-1.2B-Instruct

⚠️ EXPERIMENTAL MODEL DISCLAIMER

Please do not use this model for real-world tasks. This model was trained on a tiny, sliced subset of data strictly as a rapid test to evaluate a custom language translation fine-tuning pipeline. Because it is not fully optimized, there is a 98% chance the model will hallucinate or output inaccurate text.

You are welcome to explore this repository to see how the tokenization, data structures, and Hugging Face SFTTrainer training pipeline were constructed for English-to-French translation tasks.

This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct. It has been trained using TRL.

Quick start

from transformers import pipeline

# Note: The model is trained specifically to translate English text to French
prompt = "Translate the following sentence to French:\nHello, how are you today?"

generator = pipeline("text-generation", model="pateltakshm/Qwen-1.2B-Instruct", device="cuda")
output = generator([{"role": "user", "content": prompt}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with Supervised Fine-Tuning (SFT) using Low-Rank Adaptation (LoRA) on an English-to-French parallel dataset text split.

Framework versions

• TRL: 1.6.0
• Transformers: 5.10.2
• Pytorch: 2.11.0+cu128
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citations

Cite TRL as:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}