Gmartinezr/Shiba-Coder-27B-Max

Shiba Coder 27B Max

LoRA adapter for prism-ml/Bonsai-27B-unpacked — code generation + reasoning.

Trained on a mix of reasoning, code, and real software-engineering trajectories. The adapter is not merged — it is loaded on top of the base model at inference time.

Base model	prism-ml/Bonsai-27B-unpacked
Architecture	Qwen3_5ForConditionalGeneration (27 B)
Method	LoRA (SFT) via Unsloth + TRL SFTTrainer
Chat template	qwen3-thinking
Context window	8 192 tokens
Trainable params	159,383,552 / 27,516,112,112 (0.58 %)
Code	github.com/GinesMr/Shiba-Coder-27B-Max
License	Apache 2.0

---

Quick start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "prism-ml/Bonsai-27B-unpacked", torch_dtype="auto"
)
tok = AutoTokenizer.from_pretrained("prism-ml/Bonsai-27B-unpacked")
model = PeftModel.from_pretrained(base, "Gmartinezr/Shiba-Coder-27B-Max")

messages = [{"role": "user", "content": "Write a FastAPI endpoint for user registration."}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**inputs, max_new_tokens=512)[0], skip_special_tokens=True))

llama.cpp (no merge)

The repo also ships a pre-converted F16 GGUF (Shiba-Coder-LoRA-F16.gguf, 305 MB) that can be attached at inference time:

llama-cli -m Bonsai-27B-Q1_0.gguf \
  --lora Shiba-Coder-LoRA-F16.gguf \
  -ngl 99 --seed 42 --temp 0.2 -n 1024 \
  -p "your prompt"

Converting the LoRA yourself requires a transformers build that understands qwen3_5 (≥ 5.x from source at the time of writing): pip install git+https://github.com/huggingface/transformers.git

---

Training at a glance

Duration	27.30 h
Steps	930 (1 epoch)
Effective batch	16 (1 × 16 grad-accum)
Optimizer	adamw_8bit (LR 1e-4, linear, warmup 3 %)
Precision	bf16 over a 4-bit base
Final train loss	0.2722
Best eval loss	0.2888 (step 800)

See results/train_summary.md for the full hyperparameter table and loss progression.

---

Dataset

Source	Examples	Purpose
r0b0tlab/qwen3.8-max-distillation-50k	5 000	Qwen3.8-Max distillation — reasoning + code trajectories
guell00/qwen-3.8-code	10 000	Code-focused SFT
TIGER-Lab/SWE-Next-SFT-Trajectories	2 000	Real SWE bugfix / feature trajectories (tool calls)

≈ 17 000 examples total, filtered to ≤ 8 192 tokens, split 98 / 2 train / eval. Full rationale in data/DATASETS.md.

---

Files in this repo

File	Description
adapter_model.safetensors	LoRA weights (609 MB, F16)
adapter_config.json	PEFT adapter config
tokenizer.json / tokenizer_config.json	Qwen3.5 tokenizer
chat_template.jinja	qwen3-thinking chat template
Shiba-Coder-LoRA-F16.gguf	Pre-converted F16 GGUF for llama.cpp (305 MB)

---

Limitations

• Specialized for code + reasoning; general chat quality is inherited from the base model and may differ.
• LoRA r=32 / α=64 is intentionally low-rank — it adapts style and behaviour, not the full knowledge of the base.
• No public benchmark numbers (HumanEval / MBPP / SWE-bench) yet.
• Inherits the biases and failure modes of the base model and the training data.

---

License

Apache 2.0. Training datasets and the base model carry their own licenses; verify compatibility before commercial use.