build-small-hackathon/gemma-4-cal-gguf

gemma-cal E4B — a calendar-native edge LLM (GGUF)

gemma-cal-e4b-Q4_K_M.gguf is a QLoRA fine-tune of Gemma-4 E4B (~4B effective parameters, ~5.3 GB at Q4_K_M) built for exactly one job: reading a messy human conversation — or a photo of a flyer/invite — and emitting a single validated ActionPlan: events with exact ISO datetimes, calendar conflicts, proposed alternatives, a drafted reply, and a clarifying question when the plan is too vague to schedule.

It is the production model of OffGridSchedula serving on a 16 GB T4 via llama.cpp, fully local, no cloud AI APIs. Vision (screenshots/flyers) works by pairing it with the base E4B's projector: unsloth/gemma-4-E4B-it-GGUF / mmproj-F16.gguf.

Why an edge fine-tune

• Edge-sized by design: runs on a ~$0.40/hr T4, a gaming GPU, or an Apple-silicon laptop — local-first as a parameter count, not a tagline.
• Schema-bulletproof: 100% schema validity on the project eval even with no system prompt, with stronger no-event discipline (doesn't invent events from "thanks!") and a higher rate of asking when a date is TBD.
• Convention-trained: learns the product's date semantics ("next Tuesday" = next week's Tuesday; weekday-anchored relative dates) instead of generic internet priors.
• Eval-gated: every retrain must clear a 60-example task eval (start-exact datetime matching, F1, validity, clarification) before it can be published — the pipeline has rejected eight regressed models to date. Full scorecard: the project's docs/eval-roadmap.md.

How to run

# text + vision via llama.cpp server (OpenAI-compatible API on :8080/v1)
MODEL=$(python -c "from huggingface_hub import hf_hub_download as d; print(d('ParetoOptimal/gemma-4-cal-gguf','gemma-cal-e4b-Q4_K_M.gguf'))")
MMPROJ=$(python -c "from huggingface_hub import hf_hub_download as d; print(d('unsloth/gemma-4-E4B-it-GGUF','mmproj-F16.gguf'))")
llama-server -m "$MODEL" --mmproj "$MMPROJ" -ngl 999 -c 8192 --jinja --port 8080

Use the explicit filename rather than the -hf repo:Q4_K_M shorthand — this repo also stores legacy training artifacts at the same quant.

The model is trained to answer with only an ActionPlan JSON object. Typical user turn:

Current datetime: Monday, 2026-09-14T09:00:00
Existing calendar: (none provided)

Conversation:
Room parent: Picture day is Thursday — photos at 9am, wear the green shirt!
Me: thanks!

Return the ActionPlan JSON now.

→

{
  "reasoning": "School picture day Thursday Sep 17 at 9am; wear green shirt.",
  "events": [{"title": "School picture day", "start": "2026-09-17T09:00:00",
              "end": null, "location": "School", "attendees": [],
              "reminder_minutes": 720, "notes": "Wear green class shirt"}],
  "conflicts": [], "proposed_times": [],
  "reply_draft": "Got it — green shirt Thursday!", "needs_clarification": null
}

Honest evaluation

Scored on the project's 60-example held-out eval (50 gold events; start-exact datetime matching; temp 0; same constrained-JSON call production uses):

with system prompt	stock E4B	gemma-cal E4B
schema validity	1.0	1.0
event F1	0.97	0.97
start-exact recall	0.96	0.96
clarification recall	1.0	1.0

bare (no system prompt)	stock E4B	gemma-cal E4B
schema validity	0.967	1.0
no-event accuracy	0.70	0.80
clarification recall	0.50	0.625
event F1	0.682	0.644

i.e. parity with stock under the engineered prompt (identical error counts) and better schema validity and discipline with no prompt at all. Published after six eval-gated training iterations; the publish-at-parity call was an explicit owner decision (the auto-gate requires strict dominance).

Training

• Base: google/gemma-4-E4B-it
• Method: QLoRA (4-bit) with Unsloth; LoRA r=16, alpha=16, targets q/k/v/o/gate/up/down; merged to 16-bit before GGUF conversion.
• Recipe details that mattered: trained on Gemma-4's native chat template (<|turn>role … <turn|> — the same template embedded in the GGUF and served by llama-server --jinja), loss masked to the assistant turn only, LR 5e-5, 1 epoch.
• Data: 139 hand-authored thread-style examples (4× upsampled) + 2,000 examples converted from SMCalFlow (CC BY-SA 4.0 — Semantic Machines et al., "Task-Oriented Dialogue as Dataflow Synthesis," TACL 2020), with LISP date/time programs resolved against per-example reference datetimes and convention-conflicting rows filtered out.
• Hardware: single A100-80GB on Modal; convert_hf_to_gguf.py + llama-quantize for export.

Reproduce / retrain (eval-gated): training/gated_retrain.py in the project repo.

Files in this repo

File	Size	Status
gemma-cal-e4b-Q4_K_M.gguf	~5.3 GB	The model. Production edge fine-tune (this card).
gemma-cal-Q4_K_M.gguf	~18.7 GB	Legacy 31B training artifact from earlier iterations; not served.
mmproj-F16.gguf	~1.2 GB	Legacy projector for the 31B artifact. For the E4B, use unsloth/gemma-4-E4B-it-GGUF / mmproj-F16.gguf.

Limitations & responsible use

• Specialized: scheduling extraction only — not general chat, Q&A, or code.
• English only; expects the reference datetime (with weekday) supplied in the prompt.
• Q4_K_M quantization; verify extracted dates before trusting blindly (the app surfaces everything for review before saving).
• Derivative of Gemma-4 — use is subject to Google's Gemma Terms of Use.

Base model: Google Gemma-4 E4B. Tooling: Unsloth (training), llama.cpp (conversion + serving). Training data includes SMCalFlow (CC BY-SA 4.0, Semantic Machines et al.).