Instructions to use hungrynovalabs/nova-pup-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hungrynovalabs/nova-pup-4b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="hungrynovalabs/nova-pup-4b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("hungrynovalabs/nova-pup-4b")
model = AutoModelForMultimodalLM.from_pretrained("hungrynovalabs/nova-pup-4b", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use hungrynovalabs/nova-pup-4b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hungrynovalabs/nova-pup-4b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hungrynovalabs/nova-pup-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/hungrynovalabs/nova-pup-4b

SGLang

How to use hungrynovalabs/nova-pup-4b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hungrynovalabs/nova-pup-4b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hungrynovalabs/nova-pup-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hungrynovalabs/nova-pup-4b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hungrynovalabs/nova-pup-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use hungrynovalabs/nova-pup-4b with Docker Model Runner:
```
docker model run hf.co/hungrynovalabs/nova-pup-4b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Nova Pup 4B

By Matthew Salinas Hernandez — Hungry Nova Labs LLC

A 4B-parameter Linux systems specialist and multi-agent problem solver, trained end-to-end on a single RTX 5090.

Also available on Ollama: hungrynovalabs/nova-pup (ollama run hungrynovalabs/nova-pup:4b)

Highlights

3x its base model on a sealed closed-book Linux diagnostics exam (25% → 75%)
MMLU 73.7% (vs 74.0% base) — specialization without catastrophic forgetting
Built for agent swarms: an 8-agent pack runs in 15.5 GB VRAM at 1,539 tok/s aggregate on one RTX 5090 (Q8_0 GGUF via Ollama), with zero per-agent slowdown
Playful, rigorous "toon pup" persona — step-by-step reasoning with machine-checkable final answers

Why this model exists

I originally built Nova Pup as a local Linux study and troubleshooting assistant. I had collected technical books, manuals, man pages, and documentation over years of study, but I needed a more practical way to connect and apply that material while working.

The purpose was not to reproduce or redistribute those sources. It was to build a small assistant that could help explain concepts, troubleshoot systems, and apply Linux knowledge to new problems.

After testing it for my own use, I decided to release the trained model so other Linux users could experiment with it as well. The original books, PDFs, and training corpus are not included with the model.

Training

Base: InternScience Agents-A1-4B (Qwen3.5 hybrid linear-attention family)
Phase 1 — Continued pretraining: BF16 LoRA, 3,000 steps, 4,096-token packed sequences over a curated, deduplicated, secret-scrubbed Linux corpus
Phase 2 — Solver SFT: 1,200 steps on a programmatically generated, machine-verified puzzle curriculum: logic-grid deduction, bash-pipeline reconstruction, answer verification, and task decomposition — with a consistent persona layer
Adapters merged into the base weights; this repo contains the merged BF16 safetensors

Benchmarks

Measured with EleutherAI lm-evaluation-harness (0-shot loglikelihood except GSM8K 5-shot/300 samples), BF16, RTX 5090. Base model evaluated identically.

Benchmark	Base Agents-A1 4B	Nova Pup 4B
Linux closed-book exam (sealed, 16q)	25%	75%
MMLU (57 subjects)	74.0%	73.7%
ARC-Challenge (acc_norm)	54.4%	56.6%
HellaSwag (acc_norm)	71.7%	73.9%
Winogrande	69.0%	69.8%
TruthfulQA MC2	49.6%	45.9%
GSM8K (strict)	77.7%	71.0%

Known limitations (read before using)

Math regressed: GSM8K dropped 6.7 points vs base — math was not in the training mix. Pair with a calculator/code tool for arithmetic-heavy work.
TruthfulQA dipped 3.7 points; calibrate confidence accordingly.
4B parameters: built to be small, fast, and replicated in swarms — not to out-reason frontier models.
8k context in the published configuration; long-context training planned.
Distinct persona: a playful cartoon pup that verifies its work. Override the system prompt for a neutral assistant.

Usage (transformers)

from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "hungrynovalabs/nova-pup-4b", dtype=torch.bfloat16, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("hungrynovalabs/nova-pup-4b")

messages = [{"role": "user", "content": "My apt is broken after a full disk. First steps?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
print(tokenizer.decode(model.generate(**inputs, max_new_tokens=400)[0]))

Note: the model class is Qwen3_5ForConditionalGeneration (vision-language base); use AutoModelForImageTextToText, not AutoModelForCausalLM.

Training data & licensing

Nova Pup was trained using lawfully acquired technical materials and publicly available Linux documentation. Openly licensed materials retain their respective licenses. Copyrighted materials were used only for intermediate model training and are not distributed with the model.

Base model: InternScience Agents-A1-4B (Apache-2.0). Based on Qwen3.5-4B (Apache 2.0) by the Qwen Team.

Downloads last month: 61

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for hungrynovalabs/nova-pup-4b

Base model

InternScience/Agents-A1-4B

Adapter

(1)

this model

Adapters

1 model

Quantizations

1 model