Nebium-Large-GGUF

Quantized and FP16 GGUF format binaries for Nebium-Large (762M parameters).

Designed for high-efficiency CPU and GPU local inference via llama.cpp, Ollama, and Python bindings (llama-cpp-python).

• Base PyTorch Weights: nabin2004/nebium-large
• Framework Source: github.com/nabin2004/nebium

---

Available GGUF Binaries

Filename	Precision	Memory Footprint	Recommended Use Case
nebium-large.gguf	FP16	~1.5GB	Full precision baseline for local evaluation
nebium-large-q8_0.gguf	Q8_0	~800MB	Recommended precision with negligible perplexity loss
nebium-large-q4_k_m.gguf	Q4_K_M	Minimal	Resource-constrained edge and CPU environments

---

Execution with llama.cpp

CLI Inference

# Clone and build llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Download the model
huggingface-cli download nabin2004/nebium-large-gguf nebium-large.gguf --local-dir .

# Run prompt continuation
./llama-cli -m nebium-large.gguf \
  -p "e2e4 e7e5 g1f3" \
  -n 30 \
  --temp 0.7 \
  --top-p 0.95

---

Execution with Ollama

Modelfile

Create a file named Modelfile:

FROM ./nebium-large.gguf

PARAMETER temperature 0.7
PARAMETER top_p 0.95
PARAMETER stop "<|eos|>"

SYSTEM You are an autoregressive chess move continuation engine predicting next moves in UCI notation.

Build and Run

ollama create nebium-large -f Modelfile
ollama run nebium-large "e2e4 e7e5"

---

Python llama-cpp-python Usage

from llama_cpp import Llama

llm = Llama(
    model_path="nebium-large.gguf",
    n_ctx=1024,
    n_threads=4,
)

prompt = "e2e4 e7e5 g1f3 b8c6"
output = llm(
    prompt,
    max_tokens=20,
    temperature=0.7,
    stop=["<|eos|>", "<|pad|>"],
)

print("Generated moves:", output["choices"][0]["text"])

---

Architecture Specifications

• Base Architecture: Causal Transformer with Rotary Position Embeddings (RoPE)
• Hidden Dimension ($d_{model}$): 1280
• Attention Heads: 20
• Layers: 36
• Context Length: 1024 tokens
• Vocabulary: 5000 UCI BPE tokens

---

License

MIT License.