OpenOCRv2

Text detection and recognition for scene/document images, deployable with plain torch + transformers. Three models live in one repository, in three subfolders — download only what you need with subfolder=.

Subfolder	Model	Architecture	Params	Weights
det/	OpenOCRv2_Det	HGNetv2-b4 + DCNv2 + polygon regression head	9.36 M	36 MB
rec_ctc/	OpenOCRv2_Rec_CTC	ConvNeXt + 2D-RoPE text encoder + linear CTC head	21.97 M	85 MB
rec_ar/	OpenOCRv2_Rec_AR	Same encoder + autoregressive decoder (RoPE, KV cache)	36.66 M	141 MB

Requirements

For Windows CPU inference, install these versions in a fresh Python 3.11 virtual environment:

pip install torch==2.14.0 transformers==5.22.2 safetensors==0.8.0 numpy==2.2.6 pillow==12.3.0 opencv-python-headless==5.0.0.93

For NVIDIA GPU inference, install a CUDA-enabled build of torch using the PyTorch installation guide. Use the same versions of the other packages.

opencv-python-headless is only needed for the crop step (det.ocr(...)). Polygon detection (det.detect(...)) and recognition (rec.recognize(...) on already-cropped images) work without it.

If you see Run pip install cv2 — that is transformers' fallback hint and the package name does not exist. Install opencv-python-headless instead.

OpenOCRv2 does not require torchvision. If importing the model raises operator torchvision::nms does not exist, use a clean environment with the packages above, or install a torchvision build that matches your torch.

Usage

• CTC (rec_ctc) is the faster recognizer. Use it when throughput or latency matters most.
• AR (rec_ar) is the accuracy-focused recognizer. It decodes one token at a time, so it is slower.

Each example runs on CUDA when torch.cuda.is_available() is true and uses CPU otherwise. Pass NumPy images as shown. The models move their input tensors to the selected device, while OpenCV cropping stays on the CPU.

det.detect() and rec.recognize() already disable gradient recording for the model forward passes. det.ocr() uses both methods.

Full-page OCR

Replace page.jpg with a page or scene image. Each example prints the text in reading order and each region's recognition score, text, and quadrilateral.

CTC (speed-focused)

import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

with Image.open("page.jpg") as source:
    image = np.asarray(source.convert("RGB"))

det = AutoModel.from_pretrained(REPO, subfolder="det", trust_remote_code=True).to(device).eval()

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ctc", trust_remote_code=True).to(device).eval()
result = det.ocr(image, rec)

print("CTC — text in reading order")
print("\n".join(result["text"]))
print("Score, text, quadrilateral")
for region in result["regions"]:
    print(f'{region["rec_score"]:.3f}', region["text"], region["quad"].tolist())

AR (accuracy-focused)

import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

with Image.open("page.jpg") as source:
    image = np.asarray(source.convert("RGB"))

det = AutoModel.from_pretrained(REPO, subfolder="det", trust_remote_code=True).to(device).eval()

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ar", trust_remote_code=True).to(device).eval()
result = det.ocr(image, rec)

print("AR — text in reading order")
print("\n".join(result["text"]))
print("Score, text, quadrilateral")
for region in result["regions"]:
    print(f'{region["rec_score"]:.3f}', region["text"], region["quad"].tolist())

Detection only

Pass the full RGB image to the detector. It returns polygons in original image pixel coordinates and a detection score for each polygon.

import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

with Image.open("page.jpg") as source:
    image = np.asarray(source.convert("RGB"))

det = AutoModel.from_pretrained(REPO, subfolder="det", trust_remote_code=True).to(device).eval()
page = det.detect([image])[0]

for polygon, score in zip(page["polygons"], page["scores"]):
    print(f"{score:.3f}", polygon.tolist())

Recognition only

Use one upright text line per crop. The recognizers take a list of BGR uint8 arrays and return (text, confidence) pairs in input order.

CTC (speed-focused)

import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

crop_paths = ["line_1.jpg", "line_2.jpg"]
crops = []
for path in crop_paths:
    with Image.open(path) as source:
        rgb = np.asarray(source.convert("RGB"))
    crops.append(rgb[..., ::-1].copy())  # RGB -> BGR

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ctc", trust_remote_code=True).to(device).eval()
print("CTC — text and confidence")
for path, (text, confidence) in zip(crop_paths, rec.recognize(crops)):
    print(f"{path}: {text} ({confidence:.3f})")

AR (accuracy-focused)

import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

crop_paths = ["line_1.jpg", "line_2.jpg"]
crops = []
for path in crop_paths:
    with Image.open(path) as source:
        rgb = np.asarray(source.convert("RGB"))
    crops.append(rgb[..., ::-1].copy())  # RGB -> BGR

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ar", trust_remote_code=True).to(device).eval()
print("AR — text and confidence")
for path, (text, confidence) in zip(crop_paths, rec.recognize(crops)):
    print(f"{path}: {text} ({confidence:.3f})")

If you crop yourself, keep these two details — they are the difference between correct output and silently degraded output:

• Quad order is TL, TR, BR, BL after cv2.minAreaRect + sort-by-x, not sort-by-angle.
• Crops taller than 1.5× their width must be rotated 90° before recognition, or vertical text comes back wrong.

det.ocr already does both. modeling_openocrv2_det exposes the helpers as polygon_to_quad and get_rotate_crop_image if you want to reuse them.

Batching

rec.recognize(crops, batch_size=32, micro_batch_size=8):

• batch_size is the logical batch — it decides the padding size, and the attention length changes with it, so predictions depend on it.
• micro_batch_size is how many run per forward pass (memory only, does not affect output).

Accuracy figures were produced with batch_size=32. Changing it will shift results slightly.

Preprocessing contracts

These must match the pipeline the weights were frozen with, or output changes:

• Detection: RGB, letterbox to 1600×1600 with zero padding, ImageNet normalisation (mean [123.675, 116.28, 103.53], std [58.395, 57.12, 57.375]).
• Recognition: aspect-preserving resize onto the ladder [[64,64], [96,48], [112,40], [128,32]] (width, height), then (x/255 - 0.5) / 0.5, zero-padded to the batch maximum.

Defaults live in each config.json (preprocess, image_size, thresholds) and are applied automatically.

Files

modeling_openocrv2_det.py
modeling_openocrv2_rec_ctc.py
modeling_openocrv2_rec_ar.py
det/
├── config.json
└── model.safetensors           # 447 tensors, incl. the 14x28 polygon basis
rec_ctc/  rec_ar/
├── config.json
├── model.safetensors
└── characters.json             # 19685 / 19687-entry alphabet, bundled

The auto_map entries resolve modeling modules from the repository root. The subfolder argument selects the config and weights. To load a local download, pass the path to its root in place of REPO and keep subfolder=.

The character table ships inside each recogniser repo, so decoding needs nothing beyond from_pretrained.