LordAhkam/chef-llm-dpo-v2

Chef.LLM DPO v2 — Turkish Recipe Assistant (Gemma-4-12B + DPO LoRA)

Model Description

This is a LoRA adapter for unsloth/gemma-4-12b-it, fine-tuned with Direct Preference Optimization (DPO) to improve its behavior on Turkish-language cooking/recipe questions. Rather than re-teaching the model's entire recipe-writing behavior from scratch (as earlier SFT attempts in this project did), this adapter uses a small, targeted set of preference pairs to nudge the model away from a few specific known failure modes while staying close to the strong base model everywhere else.

This is the second (and, for now, final) DPO iteration in this project — see "Training History" below.

Intended Use

Turkish-language recipe generation, cooking Q&A, and food-science explanations. Research/hobby use.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/gemma-4-12b-it", torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "LordAhkam/chef-llm-dpo-v2")
tokenizer = AutoTokenizer.from_pretrained("LordAhkam/chef-llm-dpo-v2")

messages = [{"role": "user", "content": "Mercimek çorbası tarifi verir misin?"}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_tensors="pt", return_dict=True
).to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=0.7, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training Details

• Method: DPO (Direct Preference Optimization), via trl's DPOTrainer
• LoRA config: r=8, alpha=16, target modules: q_proj, k_proj, v_proj, o_proj (attention-only), dropout=0.05
• Hyperparameters: beta=0.1, 3 epochs, learning_rate=5e-6, per_device_batch_size=1, gradient_accumulation_steps=8
• Training data: 47 (prompt, chosen, rejected) preference pairs, hand-curated for this project — a mix of (a) pairs contrasting this project's own earlier failed fine-tune attempts (degenerate repetition-loop outputs) against coherent base-model answers, and (b) pairs correcting specific factual/structural defects found during evaluation (a leaked internal token, a misidentified regional dish, an incorrect food-chemistry claim).
• Hardware: 1x NVIDIA L40S (48GB), AWS g6e.xlarge

Training History

This project ran two DPO rounds on the same base model:

• Round 1 (42 pairs): fixed a degenerate repetition-loop defect completely, fixed 1 of 5 targeted factual defects. Eval score: 90.0%.
• Round 2 (this model, 47 pairs — round 1's 42 pairs + 5 "hard negative" pairs mined from round 1's own remaining failures): fixed 2 of 5 targeted factual defects (including round 1's fix), but introduced a few new minor issues. Eval score: 89.4%.

Round 2 was chosen for release because it permanently fixed a more structurally significant defect (an internal reasoning-trace token leaking into the final output) even though the aggregate score is statistically unchanged from round 1.

Evaluation

Evaluated on a 78-prompt Turkish recipe test set (basic recipes, dietary-constraint recipes, regional/technical/measurement questions, professional gastronomy terminology, and length/format instruction-following), scored on a 4-criterion rubric (format, fluency, factual consistency, detail; 1-5 each, hunter-mode scoring — actively checking for hallucinations rather than rewarding fluency alone).

Model	Score
Untuned base (unsloth/gemma-4-12b-it)	85.1%
This model (DPO round 2)	89.4%
DPO round 1 (not released)	90.0%
Earlier SFT attempts (v1/v2/v3)	68-78%

Known Limitations

This model is not fully corrected on all known defects. Confirmed still-present issues as of the round-2 evaluation:

• Regional dish hallucination: asked about "Kitel Raha" (a Mardin-region dish), the model still misidentifies it as a sweet dessert — it is actually a savory bulgur-and-meat dish (a type of içli köfte).
• Regional dish hallucination: asked about "kadayıf dolması" attributed to Bitlis, the model still fabricates a "Bitlis usulü" origin and technique — the dish is more commonly associated with Erzurum and made by a different (fried, rolled) method.
• Food science misconception: the model still claims acrylamide forms from overheating/smoking oil. Acrylamide is primarily a Maillard-reaction product formed in starchy foods, not an oil-degradation product.
• Minor issues introduced during round-2 training: one output had stray non-Latin characters injected mid-word; one flour volume-to-weight conversion was off by roughly 25%; one sodium nitrite curing-dose figure was stated about 10x too high.

Given these, this model should be treated as a research/hobby artifact for general Turkish recipe writing and formatting, not as an authoritative source for regional-dish provenance, food chemistry, or food-safety-critical dosing information (e.g. meat curing).

License

Inherits the Gemma Terms of Use from the base model (unsloth/gemma-4-12b-it).