MonumentalSystems/Holo-3.1-0.8B-lora-demo

Holo-3.1-0.8B — Atlas LoRA demo adapter

A tiny, deliberately-overfit demo LoRA for Hcompany/Holo-3.1-0.8B, trained to prove out the LoRA runtime in the Atlas pure-Rust CUDA inference engine end-to-end (train → PEFT export → serve → applied delta), with exact parity between Atlas and peft/transformers.

It teaches a fixed persona + a made-up "codeword" the base model cannot know, so the adapter's effect is unambiguous at batch size 1.

Prompt	Base Holo-3.1-0.8B	With this adapter (Atlas or peft)
"What is the Atlas launch codeword?"	"...codeword is ""ATLAS""..."	"The Atlas launch codeword is STARFALL-4728."
"Who are you?"	"I am Qwen3.5, ...developed by Tongyi Lab..."	"I am Sparky, the Atlas demo assistant running on a DGX GB10."

Atlas serving constraints (why the config looks the way it does)

Atlas LoRA v0 applies the BF16 delta at attention k/v/o on the full-attention layers only. This adapter is shaped to match:

• target_modules = ["k_proj", "v_proj", "o_proj"] — q_proj is gated on Holo and rejected; dense-FFN delta is not yet wired.
• layers_to_transform = [3, 7, 11, 15, 19, 23] — the 6 full-attention layers (the other 18 are Gated-DeltaNet/linear; Atlas hard-rejects a LoRA tensor on them).
• r = 32, lora_alpha = 64 (scale = α/r = 2.0), use_rslora = false, use_dora = false, bias = "none".

Rank 32 (rather than a minimal 8) is used so the exact codeword digits survive the BF16-delta-on-NVFP4-base quantization mismatch when served by Atlas.

Usage

PEFT:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Hcompany/Holo-3.1-0.8B", trust_remote_code=True)
model = PeftModel.from_pretrained(base, "MonumentalSystems/Holo-3.1-0.8B-lora-demo")

Atlas (requires the F32-adapter loader fix — adapter.rs F32→BF16 conversion):

spark serve Hcompany/Holo-3.1-0.8B \
  --lora-adapter demo=MonumentalSystems/Holo-3.1-0.8B-lora-demo \
  --max-lora-rank 64

Training

peft + transformers Trainer, ~10 epochs on 6 overfit Q→A pairs, r=32, lora_alpha=64, lr 2e-4, bf16, ~4 min on a single NVIDIA GB10 (Grace-Blackwell, aarch64, CUDA 13). This is a demo artifact, not a general-purpose fine-tune.