Instructions to use unsloth/flint-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use unsloth/flint-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf unsloth/flint-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf unsloth/flint-GGUF:UD-Q4_K_XL

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf unsloth/flint-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf unsloth/flint-GGUF:UD-Q4_K_XL

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf unsloth/flint-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
./llama-cli -hf unsloth/flint-GGUF:UD-Q4_K_XL

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf unsloth/flint-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf unsloth/flint-GGUF:UD-Q4_K_XL

Use Docker

docker model run hf.co/unsloth/flint-GGUF:UD-Q4_K_XL

LM Studio
Jan

vLLM

How to use unsloth/flint-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "unsloth/flint-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "unsloth/flint-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/unsloth/flint-GGUF:UD-Q4_K_XL

Ollama
How to use unsloth/flint-GGUF with Ollama:
```
ollama run hf.co/unsloth/flint-GGUF:UD-Q4_K_XL
```
Unsloth Desktop

How to use unsloth/flint-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/flint-GGUF:UD-Q4_K_XL

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "unsloth/flint-GGUF:UD-Q4_K_XL"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use unsloth/flint-GGUF with Docker Model Runner:
```
docker model run hf.co/unsloth/flint-GGUF:UD-Q4_K_XL
```

Lemonade

How to use unsloth/flint-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull unsloth/flint-GGUF:UD-Q4_K_XL

Run and chat with the model

lemonade run user.flint-GGUF-UD-Q4_K_XL

List all available models

lemonade list

Hermes Agent

How to use unsloth/flint-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/flint-GGUF:UD-Q4_K_XL

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default unsloth/flint-GGUF:UD-Q4_K_XL

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use unsloth/flint-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/flint-GGUF:UD-Q4_K_XL

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "unsloth/flint-GGUF:UD-Q4_K_XL" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Read our How to Run Flint Guide!

See Unsloth Dynamic 3.0 GGUFs for our quantization benchmarks.

See further below for quant benchmarks.

See below for Flint running in Unsloth Desktop with thinking controls:

Flint Unsloth GGUF Benchmarks

Flint Unsloth GGUF top-1 accuracy vs size

Recursive Flint

Blog | Hugging Face | Local engine | Documentation

Flint is Recursive’s mixture-of-experts language model for coding agents, with 39.8 billion total parameters and approximately 2.5 billion active per token. It is designed to investigate codebases, fix bugs, implement features, and write tests.

Flint supports text input and output, reasoning, and multi-turn tool use. In our release evaluations, it scores 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2, and 74.2% on SWE-bench Verified.

This repository contains the BF16 checkpoint, model configuration, tokenizer, chat template, and an SGLang serving kit. Quantized variants are prepared separately for NVIDIA and Apple Silicon runtimes.

Evaluation

Flint results use temperature 1.0, top-p 0.95, and top-k 20. Terminal-Bench uses Terminus 2; SWE-bench uses mini-swe-agent. Agentic scores report average pass@1 over at least five attempts. † marks scores measured by Recursive; other comparison scores are developer-reported.

Evaluation setups vary across sources. Flint’s Terminal-Bench score counts solutions obtained through web retrieval as unsuccessful; developer-reported scores may not apply the same adjustment.

Model specifications

Property	Flint
Developer	Recursive
Architecture	Decoder-only mixture-of-experts Transformer
Total parameters	39.8B
Active parameters per token	Approximately 2.5B, including the output head¹
Layers	49: 2 dense layers followed by 47 MoE layers
Attention	35 sliding-window layers and 14 full-attention layers
Sliding window	513 tokens: the current token and 512 preceding tokens
Attention heads	32 query heads and 4 key/value heads; head dimension 128
Attention normalization	Per-head QK RMSNorm and learned per-head attention sinks
Position encoding	RoPE on sliding-window layers; no positional encoding on full-attention layers
Experts per MoE layer	256 routed experts, with 6 selected per token, plus an always-active shared expert
Documented context configuration	262,144 tokens, including prompt and generated output
Vocabulary	150,144 tokens
Input / output	Text / text
Stated knowledge cutoff	June 2026

¹ The active-parameter convention includes the output head and excludes the full input embedding matrix. The active body alone is approximately 2.2B parameters. “40B-A2.5B” is the rounded model-size designation.

Recommended generation settings

Setting	Value
Temperature	1.0
Top-p / top-k	0.95 / 20
Output budget for hard problems	At least 32768 tokens, including reasoning

The server applies these sampling settings when a request omits them. Flint supports OpenAI-compatible tool calls. Use the checkpoint’s chat template and preserve returned reasoning_content in conversation history. Prompt and output share the context window.

Training

Flint was trained from scratch through pretraining, mid-training, supervised fine-tuning, and reinforcement learning. Training includes text and code, instruction following, and agentic tasks.

Limitations and responsible use

Flint’s supervised fine-tuning included safety-oriented examples, but the model can still produce incorrect or harmful content, faulty code, or inappropriate tool calls. Test generated code and review consequential changes before applying them. Tool-using applications should enforce their own permissions and execution boundaries; structured tool-call grammar does not prevent the model from choosing an unsafe or incorrect action.

Long reasoning can increase latency and exhaust the output budget before an answer is complete. The context configuration is a capacity limit, not a guarantee of uniform accuracy across all sequence lengths or tasks. Results can vary with checkpoint precision and inference runtime; evaluate the specific checkpoint and runtime you intend to deploy.

License and reporting

Flint is released under the Apache 2.0 license. Hosted API terms and data practices are separate from the weight license.

For model and serving issues, contact support@recursive.com and include the checkpoint revision, runtime version, hardware, launch command, and a minimal reproduction. Report safety concerns or suspected misuse to safety@recursive.com, and security vulnerabilities to security@recursive.com.

Downloads last month: 7,401

GGUF

Model size

40B params

Architecture

flint

Hardware compatibility

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Model tree for unsloth/flint-GGUF

Base model

RecursiveSI/flint

Quantized

(23)

this model

Space using unsloth/flint-GGUF 1

Collection including unsloth/flint-GGUF

Unsloth Dynamic 3.0 Quants

Collection

Introducing Dynamic V3.0 quants, our new SOTA quantization methodology. • 7 items • Updated about 11 hours ago • 90