timothydillan/gemma4-e2b-balinese-assistant-v5

Gemma 4 E2B Balinese Assistant v5 LoRA

Experimental Balinese assistant LoRA adapter for timothydillan/gemma4-e2b-balinese-cpt.

This is a research checkpoint for Open Indonesia Models. It is intended to test whether a smaller, conversation-first v5 data mix improves over the v4 adapter, which showed severe greedy decoding loops and sampled off-topic drift.

Training

• Base model: timothydillan/gemma4-e2b-balinese-cpt
• Training data: sft_train_assistant_v5.jsonl
• Rows kept after response-label masking: 13,336
• Sequence length: 1,536
• Epochs: 1
• LoRA rank / alpha: 16 / 16
• Learning rate: 5e-5
• Runtime: Kaggle T4
• Final train loss: 0.4692

The corrected v5 training run completed with finite loss. No NaN loss or non-finite training guard failure was observed.

Status

This adapter is not yet release-quality. It must pass smoke evaluation and native-speaker review before it should replace the current v4 assistant adapter.

Known risks:

• Low-resource Balinese generation may still be grammatically or culturally wrong.
• The training mix includes synthetic and translated data.
• The model may still answer off-topic or repeat phrases.
• Do not use for medical, legal, financial, safety-critical, or authoritative cultural guidance.

Loading

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "timothydillan/gemma4-e2b-balinese-assistant-v5"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(model_id, device_map="auto")

Evaluation

Primary next gate: compare against v4 with scripts/llm_assistant_smoke_eval.py, focusing on:

• phrase-loop rate;
• direct answer quality;
• Balinese marker use;
• prompt-intent hits;
• native-speaker review.