sizzlebop/HybridIntelligence-0.5B-GGUF

Hybrid Intelligence 0.5B GGUF

This repository provides GGUF quantizations for Merlin-Research/HybridIntelligence-0.5B.

Hybrid Intelligence 0.5B is built on the Falcon-H1 hybrid architecture, combining Mamba2 state-space blocks (SSM) with standard multi-head attention layers across 36 hidden layers. It couples causal language generation with neuromorphic uncertainty signals, optimized through Direct Preference Optimization (DPO) and biological neural network (BNN) calibration.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, followed by standard k-quant quantization.

---

Available Files and Quantizations

File	Quant Type	Size	Description / Recommendation
HybridIntelligence-0.5B-BF16.gguf	BF16	996.36 MB	Native bfloat16 base conversion. Highest fidelity reference weights.
HybridIntelligence-0.5B-Q8_0.gguf	Q8_0	530.54 MB	Near-lossless 8-bit quantization. Recommended for best quality evaluation.
HybridIntelligence-0.5B-Q6_K.gguf	Q6_K	410.21 MB	High quality retention with minimal degradation. Excellent accuracy/memory balance.
HybridIntelligence-0.5B-Q5_K_M.gguf	Q5_K_M	353.55 MB	Good balance between memory consumption and text generation fidelity.
HybridIntelligence-0.5B-Q4_K_M.gguf	Q4_K_M	300.22 MB	Fast, lightweight 4-bit quant. Recommended default for edge devices and mobile setups.
HybridIntelligence-0.5B-Q3_K_M.gguf	Q3_K_M	241.70 MB	Compact footprint when memory headroom is strictly limited.
HybridIntelligence-0.5B-Q2_K.gguf	Q2_K	191.60 MB	Maximum compression for ultra-constrained low-RAM environments.

---

Model Architecture Details

• Base Architecture: FalconH1ForCausalLM (falcon_h1)
• Layers: 36 (hybrid Mamba2 SSM + Multi-Head Attention)
• Hidden Size: 1024
• Attention Heads: 8 query heads / 2 KV heads, head dimension 64
• Mamba Heads: 24 heads, state dimension 128, conv kernel 4
• Intermediate Size: 2048
• Context Length: 16,384 tokens
• Vocabulary Size: 32,784
• Native Dtype: bfloat16

---

Prompt Format and Usage

Falcon-H1 uses standard autoregressive completion format:

The key difference between biological and artificial neural networks is

---

Quickstart Guide

1. llama.cpp

Run generation with llama-cli:

llama-cli -m ./HybridIntelligence-0.5B-Q4_K_M.gguf \
    -p "The emergence of intelligence in complex adaptive systems can be described as" \
    -n 256 \
    --temp 0.7 \
    --top-p 0.9 \
    --repeat-penalty 1.1

Launch a local inference server with llama-server:

llama-server -m ./HybridIntelligence-0.5B-Q4_K_M.gguf \
    --port 8080 \
    -c 4096

2. Ollama

Create a Modelfile:

FROM ./HybridIntelligence-0.5B-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9

Build and run:

ollama create hybrid-intelligence -f Modelfile
ollama run hybrid-intelligence "Explain state-space models in simple terms."

3. LM Studio

1. Copy the .gguf file to your LM Studio models folder.
2. Select HybridIntelligence-0.5B-Q4_K_M.gguf from the model menu.
3. Use raw text completion or standard system/user templates.