Instructions to use 42ailab/DeepSeek-OCR-2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 42ailab/DeepSeek-OCR-2-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="42ailab/DeepSeek-OCR-2-GGUF",
	filename="deepseek-ocr-2-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use 42ailab/DeepSeek-OCR-2-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M

Use Docker

docker model run hf.co/42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use 42ailab/DeepSeek-OCR-2-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "42ailab/DeepSeek-OCR-2-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "42ailab/DeepSeek-OCR-2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M

Ollama
How to use 42ailab/DeepSeek-OCR-2-GGUF with Ollama:
```
ollama run hf.co/42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M
```

Unsloth Studio

How to use 42ailab/DeepSeek-OCR-2-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for 42ailab/DeepSeek-OCR-2-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for 42ailab/DeepSeek-OCR-2-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for 42ailab/DeepSeek-OCR-2-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use 42ailab/DeepSeek-OCR-2-GGUF with Docker Model Runner:
```
docker model run hf.co/42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M
```

Lemonade

How to use 42ailab/DeepSeek-OCR-2-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull 42ailab/DeepSeek-OCR-2-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.DeepSeek-OCR-2-GGUF-Q4_K_M

List all available models

lemonade list

DeepSeek-OCR-2 · Fully Local Document Recognition (Lightweight Tier)

English | 简体中文 · Also on ModelScope · 42ailab.com

Give it a document photo or scan and this model reads the whole page into editable text and tables — entirely on your own computer. No cloud, free, private. The two files total about 2.5 GB, well within reach of an ordinary laptop.

The model was built by DeepSeek and released under Apache-2.0 (DeepSeek-OCR 2: Visual Causal Flow). The GGUF quantization was produced by community contributor sabafallah (Apache-2.0 permits redistribution). This repo is neither a new model nor our own quantization — what we did was mirror that community quantization byte for byte and wire it into the 42model engine so it works with one click on the desktop.

1. The problem it solves

Turning documents — contracts, invoices, papers, books, scans, screenshots — into text that is editable, searchable and feedable to an AI is one of the highest-frequency steps in knowledge work:

Convert to editable documents: turn the contents of images and scans into text and tables you can edit and search directly;
Build retrieval and Q&A: digitize paper and image material, then feed it to an AI for summarization and questions;
Grab text on the fly: text inside a screenshot or a photographed page, without retyping it.

Getting decent document recognition used to mean uploading your files to someone else's server — with the privacy and compliance concerns, and possibly the cost, that entails. This model keeps that step entirely local. Its other virtue is being small: about 2.5 GB total, runnable on an 8 GB machine, which makes it the lightweight OCR tier in our curated library.

2. How it works

The recognition capability comes from DeepSeek's DeepSeek-OCR 2. Its published design points:

Reads the whole page rather than slicing it up: the full page image is first compressed into a small number of "vision tokens" (roughly 256–1120 per page), then body text and layout are read out in one pass — so structures spanning columns and lines are less likely to get scrambled. This idea of compressing long documents into vision tokens comes from its predecessor DeepSeek-OCR (Contexts Optical Compression);
Adaptive resolution: the image is sampled dynamically between 768×768 and 1024×1024 depending on page complexity — less compute on simple pages, finer detail on complex ones;
Text and layout together: it can return plain text only, or text with layout positions.

What we did (no model training, no quantization):

Mirrored the community GGUF quantization byte for byte, with each file's sha256 identical to the source;
Wired it into the engine: layout annotations in the recognition output are stripped automatically by the engine, so what you get is clean body text. On Apple GPUs (Metal) the engine automatically runs the vision component on the CPU — this generation of the DeepSeek-OCR architecture has a known precision issue on Metal, and once worked around you never have to think about it;
Kept the same Apache-2.0 license as upstream.

3. How well it performs

For a full evaluation of the model itself, refer to the upstream paper (arXiv:2601.20552) — we do not repeat scores we have not verified. Here we list only the two things we can vouch for ourselves:

What we verified	Result
Mirror fidelity (our files vs the community source, byte for byte)	Both files match exactly in sha256 and size, with no re-compression
Runnability on the engine (the inference engine version we bundle)	DeepSeek-OCR / OCR-2 is a natively supported architecture — no custom build or branch switch needed

We did not re-run public document-parsing benchmarks, so we give no "points lost to quantization" figures.

4. Limitations and what's next

Positioned as the lightweight tier: about 2.5 GB and easy on memory, its virtues are size and quick installation; for documents with very complex layouts, the larger OCR tiers in our curated library are more reliable.
Decorative vertical text (such as vertically set Japanese headings) and other rare layouts can still be misread — a shared difficulty of this generation of document OCR.
Apple GPU (Metal): the vision component has a known precision issue on Metal, which the engine works around automatically (vision on CPU, text still GPU-accelerated); ordinary users need not think about it.
Next: track upstream model updates; evaluate our own quantizations to shrink the footprint further.

5. How to download and use it

This model is packaged for 42model, which is the recommended way to get it:

Desktop app

Open Model Library → OCR, find DeepSeek-OCR-v2 and download it;
Click Start to set it as the current OCR model.

After that, use the OCR feature to turn images and scans into text.

Files and license

File	Role	Size
`deepseek-ocr-2-Q4_K_M.gguf`	Text decoder (Q4_K_M quantization)	~1.95 GB
`mmproj-deepseek-ocr-2-q8_0.gguf`	Vision encoder (Q8_0, near-lossless)	~512 MB

Per-file sha256 values are listed on the Files page and can be verified independently.

License: the model itself is DeepSeek-OCR 2, © DeepSeek, Apache-2.0 (official sources: GitHub · Hugging Face · ModelScope · paper). The GGUF quantization was produced by community contributor sabafallah; this repo mirrors that quantization and is likewise governed by Apache-2.0. By using it you agree to the upstream license terms.

Citation

Please cite upstream DeepSeek (DeepSeek-OCR 2 and its predecessor):

@article{wei2026deepseek,
  title={DeepSeek-OCR 2: Visual Causal Flow},
  author={Wei, Haoran and Sun, Yaofeng and Li, Yukun},
  journal={arXiv preprint arXiv:2601.20552},
  year={2026}
}

@article{wei2025deepseek,
  title={DeepSeek-OCR: Contexts Optical Compression},
  author={Wei, Haoran and Sun, Yaofeng and Li, Yukun},
  journal={arXiv preprint arXiv:2510.18234},
  year={2025}
}

This repo only mirrors and adapts for local use; we claim no credit for the model or the quantization.

About us

42ailab — an AI research lab exploring the boundaries of intelligence. Grounded in cognitive science, we work toward a deep integration of AI and human intelligence — to truly understand and augment intelligence, carbon-based and silicon-based alike.

42model — a high-performance local inference engine from 42ailab that runs translation, transcription, recognition, chat and coding on your own machine, free and private; with optional cloud compute for fine-tuning your own models and bringing them back to run locally.

Downloads last month: 267

GGUF

Model size

3B params

Architecture

deepseek2-ocr

Hardware compatibility

4-bit

Model tree for 42ailab/DeepSeek-OCR-2-GGUF

Base model

deepseek-ai/DeepSeek-OCR-2

Quantized

(7)

this model

Papers for 42ailab/DeepSeek-OCR-2-GGUF

DeepSeek-OCR 2: Visual Causal Flow

Paper • 2601.20552 • Published Jan 28 • 73

DeepSeek-OCR: Contexts Optical Compression

Paper • 2510.18234 • Published Oct 21, 2025 • 95