itsalxgg/Nushy-5

Nushy5 🐕

Nushy5 is a 321,696-parameter decoder-only transformer. It reads a text prompt, picks one of ten dog moods, and generates a short bark in that mood. Output is a mood tag followed by bark tokens such as woof, *tail wag*, and *low growl*. It does not answer questions.

Nushy5 uses standard transformer components in a custom overall architecture, so it loads through transformers with trust_remote_code=True. transformers requires the bundled tokenizer (tokenizer_nushy5.py) and mood router (router.py) to load and run the weights.

Run it

Download the repo so the tokenizer and router are importable, then install the dependencies:

hf download itsalxgg/nushy-5 --local-dir nushy5 && cd nushy5
pip install torch transformers numpy safetensors

import torch
from transformers import AutoModelForCausalLM
from tokenizer_nushy5 import NushyTokenizer, END
from router import SemanticRouter

model = AutoModelForCausalLM.from_pretrained(".", trust_remote_code=True).eval()
tok, router = NushyTokenizer(), SemanticRouter()

prompt = "who's the best boy?"
mood = router.route(prompt)                        # "affectionate"
head = f"<|system|>\nNushy\n<|user|>\n{prompt}\n<|assistant|>\n"
ids  = torch.tensor([tok.encode(head) + [tok.stoi[f'<|mood:{mood}|>']]])
out  = model.generate(ids, max_new_tokens=24, do_sample=True, top_k=12,
                      temperature=0.8, eos_token_id=tok.stoi[END], pad_token_id=0)
print(tok.decode(out[0].tolist()[len(tok.encode(head)):]))
# <|mood:affectionate|> *nose boop* *wag wag* *tail wag* woof!

onnx/nushy5.int8.onnx runs the same weights under ONNX Runtime, using tokenizer_nushy5.py and router.py for encoding and mood selection.

Architecture

Field	Value
Layers	3
Attention	grouped-query, 4 query heads, 2 KV heads
Embedding width	96
Positional	rotary (RoPE), theta 10000, scalable via PI, NTK, YaRN
Context	128 tokens trained
Vocab	154, expandable (novel *action tags* cost 0 new ids)

Measured

Metric	Value	Source
Prompt to mood routing	100%, 64 of 64 held-out gold	eval_routing_hard.py
Mood legibility (probe)	81% against 10% chance	eval_expressiveness.py
KV-cache vs full forward	3.3e-7 max abs diff	model.py
torch vs ONNX Runtime	1e-5 max abs diff	export_onnx.py
int8 ONNX size	0.36 MB	on disk

Mood selection lives in a separate numpy router. The generator classifies prompt intent at about 40%, so the router picks the mood and the generator maps that mood to a bark.

Limits

• No GGUF build. ollama and llama.cpp support a fixed set of architectures and reject this custom one. Run Nushy5 through transformers or ONNX Runtime.
• The router misses a small tail of idioms such as "hit the hay".
• Greedy decoding repeats barks. Keep do_sample=True.

License

MIT.