Nova-1-0.8B-GGUF

GGUF quantizations of Nova-1-0.8B — the compact, function-calling assistant from HyperAI — ready for llama.cpp, llama-server, llama-cpp-python, and Ollama. See HyperAiCorp/Nova-1-0.8B for the full BF16 model, benchmarks, and training details.

Files

File	Quant	Size	Notes
Nova-1-0.8B-f16.gguf	f16	1.9 GB	Highest fidelity
Nova-1-0.8B-q8_0.gguf	Q8_0	1.0 GB	Recommended
Nova-1-0.8B-q4_k_m.gguf	Q4_K_M	0.6 GB	Bad

All files are ready to run with llama.cpp (llama-cli, llama-server, llama-cpp-python).

Usage (llama.cpp)

llama-cli -m Nova-1-0.8B-q4_k_m.gguf \
  -p "You are Nova, a virtual AI assistant developed by HyperAI. User: Who are you? Assistant:" \
  -n 128

or via llama-server:

llama-server -m Nova-1-0.8B-q4_k_m.gguf --port 8080
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"system","content":"You are Nova, a virtual AI assistant developed by HyperAI."},{"role":"user","content":"Who are you?"}]}'

Nova supports multiple tool-calling protocols (XML <tool_call>, JSON function calling, plain-text TOOL name | param=value) declared in the system prompt.

Disabling Thinking Mode

Important: Nova is built on the Qwen3.5 reasoning backbone. In some runtimes it can emit a visible chain-of-thought block (Thinking Process: / thinking tags) before answering. That monologue slows responses and breaks tool-calling — always disable thinking mode when serving Nova.

llama.cpp / llama-server

llama-server -m Nova-1-0.8B-q4_k_m.gguf --port 8080 --reasoning off

Ollama

Create a local model with a template that pre-closes the thinking block (so the model answers immediately):

FROM Nova-1-0.8B-q4_k_m.gguf

PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER stop "<|im_end|>"

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
 thinking

 response

"""

SYSTEM """You are Nova, a virtual AI assistant developed by HyperAI. Be helpful, polite and concise, answer in the user's language. Your name is Nova; when asked who you are, always answer "I am Nova"."""

ollama create Nova -f Modelfile
ollama run Nova "Who are you?"

Note: the template must end with the response marker after thinking, exactly as above. That is what disables the visible chain-of-thought (the model sees thinking already closed and answers directly). Older Ollama versions rewrite templates stored in the GGUF, so always pass TEMPLATE explicitly in the Modelfile.

License

Apache-2.0, consistent with the base Qwen3.5-0.8B and the full Nova-1-0.8B model. Built by HyperAI.