timothydillan/gemma4-e2b-balinese-assistant

Gemma-4-E2B Balinese Assistant — EARLY CHECKPOINT (CPT->SFT v1)

⚠️ EARLY RESEARCH CHECKPOINT — NOT a finished assistant. This model speaks fluent Balinese (the CPT fluency stage works) but does not yet reliably follow instructions — it tends to answer off-topic and degenerate into repetition. Published for transparency and reproducibility, not for production use. A retrained version with grounded, instruction-balanced Balinese data is in progress.

Part of Open Indonesia Models.

Pipeline: google/gemma-4-E2B-it -> CPT (Balinese fluency, ~25M tokens) -> SFT (instruction LoRA). Base is the CPT model timothydillan/gemma4-e2b-balinese-cpt.

What works / what doesn't (honest)

• ✅ Fluency: grammatical, high-register (alus) Balinese.
• ❌ Instruction-following: ignores the question, loops on phrases.
• Why: the SFT mix was ~67% translation/story (teaches text generation, not answering) and the instruction portion was machine-translated. Fix = grounded, instruction-heavy, cleaned Balinese data (next round).

Serve (base + adapter)

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model = AutoPeftModelForCausalLM.from_pretrained("timothydillan/gemma4-e2b-balinese-assistant")   # pulls CPT base + this adapter
tok = AutoTokenizer.from_pretrained("timothydillan/gemma4-e2b-balinese-assistant")
# Gemma 4 is multimodal: message content must be a list of typed parts.
msgs = [{"role": "user", "content": [{"type": "text", "text": "Om Swastiastu!"}]}]

LoRA r=16, 4516 steps. Target: a small on-device Balinese assistant. Research checkpoint.