Hugging Face model: kshitij-hf/brickgpt-dpo-2048
Author: kshitij-hf
Created: 2025-12-07 17:24:07
Last modified: 2025-12-08 00:10:05
Pipeline: text-generation
Library: peft
Likes: 0 | Downloads: 1
Tags: peft, safetensors, dpo, reinforcement-learning, robotics, lego, manufacturing, physics-aware, text-generation, low-rank-adaptation, conversational, en, dataset:dpo_dataset.parquet, base_model:AvaLovelace/BrickGPT, base_model:adapter:AvaLovelace/BrickGPT, license:llama3.2, model-index, region:us
Base model relation: adapter
Base models: id: AvaLovelace/BrickGPT
Card metadata:
base_model: AvaLovelace/BrickGPT
library_name: peft
license: llama3.2
language: en
datasets: dpo_dataset.parquet
tags: dpo, reinforcement-learning, robotics, lego, manufacturing, physics-aware, text-generation, low-rank-adaptation
pipeline_tag: text-generation
training_args: method: Direct Preference Optimization (DPO)
precision: bfloat16 (QLoRA/NF4)
optimizer: paged_adamw_8bit
learning_rate: 5e-05
warmup_ratio: 0.1
batch_size: 1 (Effective 16 via Gradient Accumulation)
beta: 0.1
lora_config: r=32, alpha=16, Dropout=0.05, Targets: q_proj, v_proj