Instructions to use itsalxgg/nushy-5-thinking with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use itsalxgg/nushy-5-thinking with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="itsalxgg/nushy-5-thinking", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("itsalxgg/nushy-5-thinking", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use itsalxgg/nushy-5-thinking with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "itsalxgg/nushy-5-thinking"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "itsalxgg/nushy-5-thinking",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/itsalxgg/nushy-5-thinking

SGLang

How to use itsalxgg/nushy-5-thinking with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "itsalxgg/nushy-5-thinking" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "itsalxgg/nushy-5-thinking",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "itsalxgg/nushy-5-thinking" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "itsalxgg/nushy-5-thinking",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use itsalxgg/nushy-5-thinking with Docker Model Runner:
```
docker model run hf.co/itsalxgg/nushy-5-thinking
```

Nushy5-thinking 🐕💭

Nushy5-thinking is a reasoning model. Before it answers, it thinks: at length, visibly, inside a <think> block, weighing its options with the full deliberative might of 321,888 parameters. Then it barks.

The reasoning does not improve the answer. That is the breakthrough. Nushy5-thinking takes the industry's most expensive idea, scaling inference-time compute, and points it at a dog. You can make it think for four tokens or four hundred. The bark at the end is equally unhelpful either way.

<think> *nose boop* *stare* arf? *paw* hmmm *stare* *sit* ... grrr? no *low growl*
woof?! ... aha *low growl* ... mrrf hmm?! ... aha wait *sit* wait *sniff* hm hmmm
huff... </think>
<|mood:affectionate|> awoo! *lean* awoo... awoo!

The breakthrough

Nushy5-thinking adds one capability to Nushy5: Extended Canine Deliberation. Given any prompt, the model emits a running internal monologue (sniffing, head tilts, muttered "hmm", "no", "wait", the occasional imaginary squirrel), commits to a mood, and barks. The --min-think knob sets the minimum length of the deliberation, so you can dial the agony from "brief" up to "two full minutes of a dog second-guessing itself".

Honest version: the <think> content is theatre. The committed mood comes from the router, not from the reasoning. Longer thinking makes the dog think longer. It does not make the dog smarter. No dog here can be made smarter.

Run it (recommended)

The bundled sampler forces the reasoning to any length and commits the correct mood.

hf download itsalxgg/nushy-5-thinking --local-dir nushy5-thinking && cd nushy5-thinking
pip install torch transformers numpy safetensors
python sample_think.py --prompt "who's the best boy?" --min-think 200

--min-think N forbids the </think> token until N reasoning tokens have passed.

Raw transformers

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("itsalxgg/nushy-5-thinking", trust_remote_code=True)

A plain generate() produces the ramble and a bark, but the model picks its own mood (about 40% correct) and does not force a minimum reasoning length. The sampler suppresses in-think mood tokens and forces the router's mood after </think>, which is why its punchlines land. Use the sampler for the real behavior.

How it was trained

Warm-started from the shipped Nushy5 r2 weights (all 38 tensors transfer; the two new token rows start fresh), then fine-tuned on 9,000 examples of <think> ...ramble... </think> <|mood:X|> bark. Same architecture as Nushy5 (3 layer, 96 embedding, grouped-query attention), vocab grown by two tokens (<think>, </think>), final loss 2.07. No reasoning capability was created at any point in this process.

What the breakthrough is not

It is not smarter. The reasoning does not inform the mood or the bark.
It is not coherent. A 321k model rambling for 200 tokens is nonsense, by design.
It still does not answer the question.

License

MIT.

Downloads last month: 7

Safetensors

Model size

337k params

Tensor type

F32