Hugging Face model: himanshunakrani9/qwen2.5-3b-ultrafeedback-dpo
Author: himanshunakrani9
Created: 2026-05-30 17:25:38
Last modified: 2026-05-30 18:11:48
Library: peft
Likes: 0 | Downloads: 3
Tags: peft, safetensors, dpo, preference-optimization, qlora, lora, trl, qwen, assistant, en, dataset:argilla/ultrafeedback-binarized-preferences-cleaned, arxiv:2305.18290, license:apache-2.0, region:us
Card metadata:
license: apache-2.0
base_model: Qwen/Qwen3-4B-Instruct
datasets: argilla/ultrafeedback-binarized-preferences-cleaned
tags: dpo, preference-optimization, qlora, lora, peft, trl, qwen, assistant
language: en