Nebium-Small-GGUF

Quantized and FP16 GGUF format binaries for Nebium-Small (117M parameters).

Designed for low-latency CPU and GPU execution with llama.cpp and Ollama.

• PyTorch Base Model: nabin2004/nebium-small
• GGUF Repository: nabin2004/nebium-small-gguf
• Source Repository: github.com/nabin2004/nebium

---

Artifacts

Filename	Precision	Description
nebium-small.gguf	FP16	Full-precision baseline export
tokenizer.json	Tokenizer	BPE vocabulary and merge definitions

---

Inference with llama.cpp

# Clone and compile llama.cpp
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make

# Download GGUF binary
huggingface-cli download nabin2004/nebium-small-gguf nebium-small.gguf --local-dir .

# Run prompt continuation
./llama-cli -m nebium-small.gguf -p "e2e4 e7e5 g1f3" -n 25 --temp 0.7

---

Inference with Ollama

# Modelfile
FROM ./nebium-small.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|eos|>"
SYSTEM You are an autoregressive chess next-move prediction model using UCI move notation.

ollama create nebium-small -f Modelfile
ollama run nebium-small "e2e4 e7e5"

License

MIT License.