build-small-hackathon/elysium-MiniCPM-V-4.6-F16-GGUF

🌿 Elysium — MiniCPM-V-4.6 F16 GGUF

A single-file F16 (full-precision) GGUF of Elysium, a QLoRA fine-tune of openbmb/MiniCPM-V-4.6 trained to always emit valid ElysiumResponse JSON (schema v1.0.0).

Submission to the Build Small Hackathon.

📦 File

File	Size	Quantization	Quality
elysium-f16.gguf	1.89 GB	F16 (full precision)	★★★★★ — zero quantization loss

Only F16 is published in this repo. Integer quantizations (Q4_K_M / Q6_K / Q8_0) were intentionally skipped — F16 preserves 100 % of the fine-tuned weights so the JSON-schema compliance guarantees from training are not eroded by quantization noise.

🚀 Quick start — llama.cpp CLI

./llama-cli \
    -m elysium-f16.gguf \
    -n 2048 \
    --temp 0 \
    -p "What is the capital of Tamil Nadu?"

🚀 Quick start — Python (llama-cpp-python)

from llama_cpp import Llama
import json

llm = Llama(
    model_path="elysium-f16.gguf",
    n_ctx=4096,
    n_gpu_layers=-1,
    verbose=False,
)

# Use the SAME system prompt the model was fine-tuned with
SYSTEM = open("system_prompt.txt").read()
prompt = f"<|im_start|>system\n{SYSTEM}<|im_end|>\n<|im_start|>user\nWhat is the capital of Tamil Nadu?<|im_end|>\n<|im_start|>assistant\n"

out = llm(prompt, max_tokens=2048, temperature=0.0, stop=["<|im_end|>"])
parsed = json.loads(out["choices"][0]["text"])
print(parsed["direct_answer"])
# → "Chennai is the capital of Tamil Nadu, India."

🎓 Training recipe (summary)

Technique	Setting
Base model	openbmb/MiniCPM-V-4.6 (1.3 B params — SigLIP2-400M + Qwen3.5-0.8B)
Adapter	QLoRA, r=64, α=128, dropout=0.05
Quantization (training)	4-bit NF4, BF16 compute
Target modules	q/k/v/o/gate/up/down_proj + embed_tokens + lm_head
Loss	Response-only masking + 3× JSON structural-token upweight
Regularisation	NEFTune α=5.0, weight_decay=0.01, max_grad_norm=0.5
Optimiser	paged_adamw_8bit, lr=1e-4, cosine + warm restarts
Curriculum	Examples sorted shortest-first
Schema validation	Every 25 steps against ElysiumResponse v1.0.0

📐 ElysiumResponse Schema v1.0.0

The model is fine-tuned to always emit a single JSON object containing: schema_version, session_id, timestamp_utc, interaction_type, direct_answer, multimodal_perception, hypergraph_delta, council_deliberation, tool_calls, daily_action_field, probabilistic_forecasts, strain_metadata, ui_directives, metadata.

metadata.schema_validation_passed is always true.

🛠 Notes for hackathon judges

• The Qwen3.5 MTP (Multi-Token Prediction) head is stripped during conversion (--no-mtp flag) — standard autoregressive decode works perfectly without it.
• Vision tower (vpm + resampler) is NOT included in this GGUF. llama.cpp text inference is fully functional. For multimodal use the BF16 transformers checkpoint instead.

🙏 Credits

• Base model: OpenBMB MiniCPM-V-4.6
• Quantization: llama.cpp
• Hackathon: Build Small Hackathon