TheCoderScientist/GarudaCoder-27B-ID-lora

GarudaCoder-27B 🦅 — Indonesian Coding Assistant

QLoRA adapter (r=8, alpha=8) on top of Qwen3.8-27B (unsloth 4-bit build) — successor of GarudaCoder-7B-Coder-ID-lora. Fine-tuned on a curated high-quality Indonesian coding dataset. Focus: explaining the reasoning before writing code, honesty about uncertainty (anti-hallucination), and Indonesian developer context. All model output is in Indonesian.

Trained capabilities

• Chain-of-thought: root cause → alternatives + trade-offs → solution
• Self-correction: draft → mental test → find its own bug → fix
• Multi-turn debugging, asking clarification on ambiguous requests, refusing misguided requests
• Grounded answers: architecture, debugging workflows, and Indonesian edge cases

Usage

IMPORTANT — avoid stuck downloads: the HF Xet backend is known to stall at the Reconstructing (incomplete total...) stage (xet-core#850). Disable Xet before any imports:

import os
os.environ["HF_HUB_DISABLE_XET"] = "1"  # MUST be the very first cell/line

Via Unsloth (recommended)

from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    "TheCoderScientist/GarudaCoder-27B-ID-lora",
    max_seq_length=1024, load_in_4bit=True,
)
FastModel.for_inference(model)

pesan = [
    {"role": "system", "content": "<copy the system prompt from the section below — use verbatim>"},
    {"role": "user", "content": "Kenapa [[0]*3]*3 di Python bikin semua baris ikut berubah?"},
]
inputs = tokenizer.apply_chat_template(pesan, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
out = model.generate(input_ids=inputs, max_new_tokens=1024, temperature=0.3, do_sample=True)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Via transformers + PEFT (without Unsloth)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.float16)
base = AutoModelForCausalLM.from_pretrained(
    "unsloth/Qwen3.8-27B-unsloth-bnb-4bit", quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, "TheCoderScientist/GarudaCoder-27B-ID-lora")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen3.8-27B-unsloth-bnb-4bit")

System prompt

Use verbatim — the model was trained with it; without it, reasoning quality degrades.

Kamu adalah GarudaCoder, asisten coding berbahasa Indonesia yang teliti. Utamakan kebenaran teknis dan kode yang dapat diuji. Jelaskan ringkasan alasan yang dapat diaudit sebelum solusi, nyatakan asumsi dan edge case yang relevan, dan jangan mengarang fakta, API, versi, atau angka. Jika informasi tidak cukup, ajukan pertanyaan klarifikasi. Untuk informasi yang berubah atau bersifat lokal, minta atau gunakan sumber yang dapat diverifikasi. Tolak praktik berbahaya dan tawarkan alternatif yang aman. Jangan tampilkan chain-of-thought privat.

Important notes

• Sampling config used during eval:

{
  "max_new_tokens": 1024,
  "do_sample": true,
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 20,
  "repetition_penalty": 1.0
}

• VRAM: 4-bit weights ≈ 20.5 GiB total → needs 2× T4 (16 GB) or a single 24 GB+ GPU. Does not fit on one T4.
• This is a LoRA adapter (PEFT), not a full model — the Qwen3.8-27B 4-bit base is downloaded automatically on load.
• On T4 (no bfloat16) this architecture trains in float32 via Unsloth; 4-bit inference works fine.
• Full provenance (per-file dataset SHA256, seeds, revisions) is committed in this repo at training/manifest_qwen38.json.

Troubleshooting

Symptom	Cause	Fix
Download stuck at Reconstructing (incomplete total...)	hf_xet stall bug	HF_HUB_DISABLE_XET=1 before imports, or pip uninstall hf-xet
CUDA out of memory on a single T4	27B in 4-bit needs ~2×16 GB	use 2 GPUs or a 24 GB+ GPU
English / shallow answers	system prompt not used	use the system prompt above verbatim
KeyError / adapter won't attach	wrong base model	use unsloth/Qwen3.8-27B-unsloth-bnb-4bit (pinned revision in manifest)

Training

• QLoRA 4-bit, r=8, alpha=8, max_seq 1024, 1 epoch, 229 examples, Kaggle T4×2
• SFT with train_on_responses_only (instruction/response masking)
• Base pinned: unsloth/Qwen3.8-27B-unsloth-bnb-4bit @ 8aa5f05d26b7205477066e1449e0af13f762a299 (upstream Qwen/Qwen3.8-27B @ 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0)
• Seed 42 · environment: kaggle-t4x2 · Python 3.12.13 · torch 2.10.0+cu128 · CUDA 12.8 · VRAM 29.12 GiB
• Holdout: 80 examples kept out of training entirely
• Anti-leakage: holdout/benchmark files gated (FILE_DILARANG_UNTUK_TRAINING)

Lineage

• Predecessor: GarudaCoder-7B-Coder-ID-lora (Qwen2.5-Coder-7B)
• This qwen3_5 model was trained 2x faster with Unsloth