nandraj/NaviDC-OCR-GGUF

NaviDC-OCR GGUF

GGUF conversions of the original StarDoc-AI/NaviDC-OCR vision-language model for use with llama.cpp and other GGUF-compatible inference runtimes.

Important: This repository contains converted GGUF files of the original StarDoc-AI/NaviDC-OCR model. It is not the original model repository, and the GGUF files have not been trained or fine-tuned by the maintainer of this repository.

---

⚠️ Requires a patched llama.cpp build — stock llama.cpp fails to load this model with a check_tensor_dims shape error; see PATCHES.md for the required source changes before building.

---

Original Model

Original model: StarDoc-AI/NaviDC-OCR

Original model organization: StarDoc-AI

Original model repository: StarDoc-AI/NaviDC-OCR

NaviDC-OCR is a lightweight (~1.2B parameter) open-source Vision-Language Model designed specifically for document parsing.

The original model is designed to handle both:

• Digital documents
• Camera-captured documents

The model supports document understanding tasks including:

• OCR / text extraction
• Document parsing
• Table recognition
• Formula recognition
• Layout understanding
• Reading order
• Distorted document analysis
• Scientific document understanding

Please refer to the original StarDoc-AI/NaviDC-OCR repository for the official model documentation, training details, evaluation results, architecture information, examples, and original model files.

---

About This Repository

This repository provides GGUF versions converted from the original StarDoc-AI/NaviDC-OCR model.

The conversion was performed using the GGUF conversion and quantization tools provided by the llama.cpp project.

The main purpose of this repository is to make NaviDC-OCR available in GGUF format for users who want to experiment with local inference, CPU inference, GPU inference, or llama.cpp-compatible applications.

---

GGUF Files

The repository may contain the following files:

File	Description
NaviDC-OCR-f16.gguf	F16 GGUF version of the main model
NaviDC-OCR-mmproj-f16.gguf	F16 multimodal projector / vision component
NaviDC-OCR-Q2_K.gguf	Q2_K quantized main model
NaviDC-OCR-Q3_K_S.gguf	Q3_K_S quantized main model
NaviDC-OCR-Q3_K_M.gguf	Q3_K_M quantized main model
NaviDC-OCR-Q3_K_L.gguf	Q3_K_L quantized main model
NaviDC-OCR-Q4_K_S.gguf	Q4_K_S quantized main model
NaviDC-OCR-Q4_K_M.gguf	Q4_K_M quantized main model
NaviDC-OCR-Q5_K_S.gguf	Q5_K_S quantized main model
NaviDC-OCR-Q5_K_M.gguf	Q5_K_M quantized main model
NaviDC-OCR-Q6_K.gguf	Q6_K quantized main model
NaviDC-OCR-Q8_0.gguf	Q8_0 quantized main model

Not all quantization variants may be available at all times.

---

What is the MMProj file?

For multimodal models, llama.cpp uses a separate GGUF file containing the multimodal components.

The file:

NaviDC-OCR-mmproj-f16.gguf

is the multimodal projector / vision-side GGUF component.

Depending on the llama.cpp version and model architecture, the main GGUF model and the MMProj GGUF may both be required for image-based inference.

The MMProj file is intentionally kept separate from the main quantized LLM weights.

For multimodal models, llama.cpp documentation recommends keeping multimodal components in a high-quality format such as BF16 or Q8_0 when possible because these components directly process the input image and their quality can affect the final generation quality.

---

Quantization

The original model was first converted to GGUF in F16 format and then quantized from the F16 GGUF model.

---

Citation

If you use NaviDC-OCR or the GGUF models in this repository, please cite the original NaviDC-OCR paper:

BibTeX

@article{cai2026navidcocr,
  title={NaviDC-OCR: Navigating Document Parsing Across Digital and Camera-Captured Documents},
  author={Cai, Peng and Zou, Zhaofan and Liu, Shifa and Wang, Yikun and Tang, Jiawei and Yang, Kaicheng and Tong, Meng and He, Zhongjiang and Sun, Hao},
  journal={arXiv preprint arXiv:2608.12898},
  year={2026}
}