hyperNix-3.1-mini_c

Chat-tune of Compactbot/hypernix-3.1-mini with HyperNix on HuggingFaceTB/smol-smoltalk (440871 examples, assistant-only loss). Best val_loss 2.6645 (untuned 6.9494). A ~49M model: expect short, simple answers.

thank you compact ai for letting me even do the continued pretraining run run on your compute I really aprechate it

Layers	8
d_model / heads / kv heads	512 / 8 / 2
Max seq len	512
RoPE theta	100000
Vocab	32000

Prompt format: User: <text>\nAssistant: <text> (EOS ends the turn).

from hypernix.training.brewer import BrewerConfig, BrewerModel
from safetensors.torch import load_file
from huggingface_hub import snapshot_download
import json, pathlib
d = pathlib.Path(snapshot_download("<this repo>"))
model = BrewerModel(BrewerConfig.from_dict(json.loads((d / "config.json").read_text())))
model.load_state_dict(load_file(d / "model.safetensors"), strict=False)  # lm_head is tied, not stored
model.eval()