dkovalen/markone-ocr-weights

Mark One OCR Weights

This repository contains the custom inference weights for Mark One OCR, the second-place solution in the final of the Handwritten to Data challenge. The models detect, classify, and transcribe regions in Ukrainian handwritten documents. They were trained or fine-tuned on the open RUKOPYS dataset.

The complete, supported inference pipeline—including detector ensembling, geometry correction, TrOCR, optional Qwen page-context correction, final post-processing, CLI, Docker, and the web app—is available at Mark One OCR.

Repository contents

Path	Purpose
detector_old.pt	First fine-tuned Ultralytics RT-DETR-L checkpoint
detector_new.pt	Complementary fine-tuned RT-DETR-L checkpoint
recognizer/	Fine-tuned TrOCR VisionEncoderDecoderModel inference files
recognizer_processor/	Matching TrOCR image processor and tokenizer
warp_selector.pkl	LightGBM gate used by the conservative UVDoc geometry stage
SHA256SUMS	Checksums for every inference artifact

The detector class order is:

0 handwritten, 1 printed, 2 formula, 3 table,
4 annotation, 5 image, 6 graph

The two detector checkpoints are an ensemble. Running either checkpoint alone is useful for inspection, but does not reproduce the challenge pipeline. This is a multi-component weight bundle, so the repository root is not intended to be loaded by a single from_pretrained() call or Hugging Face inference widget. Load a component as shown below or use the complete pipeline.

Use the complete pipeline

git clone https://github.com/Isterikus/markone-ocr
cd markone-ocr
python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[all]'
python scripts/download_models.py

# Base detector + geometry + greedy TrOCR inference
markone-ocr infer /path/to/pages --output outputs/base

# Optional page-context correction through a running Ollama service
markone-ocr infer /path/to/pages \
  --enhance --backend ollama --model qwen3.6:latest \
  --output outputs/enhanced

See the GitHub README for Ollama, vLLM, Docker, web app, batch, and low-VRAM profiles. Qwen and UVDoc weights are external dependencies and are not included in this repository.

Load the recognizer by itself

The recognizer expects a cropped text region, not a full document page. Greedy decoding (num_beams=1) is the supported configuration.

from pathlib import Path

import torch
from huggingface_hub import snapshot_download
from PIL import Image
from transformers import TrOCRProcessor, VisionEncoderDecoderModel

repo_id = "dkovalen/markone-ocr-weights"
root = Path(
    snapshot_download(
        repo_id,
        allow_patterns=["recognizer/*", "recognizer_processor/*"],
    )
)

processor = TrOCRProcessor.from_pretrained(root / "recognizer_processor")
model = VisionEncoderDecoderModel.from_pretrained(root / "recognizer").to("cuda").eval()

crop = Image.open("text-crop.png").convert("RGB")
pixel_values = processor(images=crop, return_tensors="pt").pixel_values.to("cuda")
with torch.inference_mode():
    token_ids = model.generate(pixel_values, num_beams=1, max_new_tokens=128)

print(processor.batch_decode(token_ids, skip_special_tokens=True)[0])

Load a detector checkpoint by itself

from huggingface_hub import hf_hub_download
from ultralytics import RTDETR

repo_id = "dkovalen/markone-ocr-weights"
checkpoint = hf_hub_download(repo_id, filename="detector_old.pt")
detector = RTDETR(checkpoint)
results = detector.predict("page.jpg", imgsz=1536, device=0)

for result in results:
    print(result.boxes.xyxy, result.boxes.cls, result.boxes.conf)

This example is raw single-checkpoint inference. Use the full Mark One pipeline for its two-checkpoint, full-page plus sliding-window ensemble, class-aware filters, geometry safeguards, coordinate mapping, and final post-processing.

Evaluation

The competition metric was:

score = 0.15 × Detection F1
      + 0.05 × Classification Accuracy
      + 0.30 × (1 − Region CER)
      + 0.50 × (1 − Page CER)

Full pipeline configuration	Public test composite
RT-DETR ensemble + geometry + greedy TrOCR	0.862
+ Qwen3.5 9B	0.896
+ Qwen3.6 35B-A3B	0.905
+ Qwen3.5 122B-A10B	0.914

These are results from the challenge test distribution. They are not guaranteed for unrelated document collections. The Qwen models in the enhanced rows are served separately and are not part of this weight bundle.

Intended use and limitations

The release is intended for research, evaluation, and OCR applications involving Ukrainian handwritten documents. It is especially suited to pages structurally similar to RUKOPYS.

• The recognizer is specialized for Ukrainian Cyrillic handwriting and may perform poorly on other languages, scripts, layouts, or modern camera photos.
• Large tables are a weak point for the base TrOCR-only profile.
• Handwriting is ambiguous. Human review is required for legal, medical, identity, archival, or other high-impact uses.
• A Qwen enhancer may replace an incorrect transcription with plausible but still incorrect text.
• warp_selector.pkl is a Python pickle. Load it only from this trusted, checksum-verified repository; pickle files can execute code when deserialized.

Training data and upstream models

• Dataset: UkrainianCatholicUniversity/rukopys, licensed CC-BY-4.0.
• Recognizer base: Kansallisarkisto/cyrillic-htr-model, licensed Apache-2.0.
• Detector architecture/runtime: Ultralytics RT-DETR-L.

The compact training path and recorded profiles for both released RT-DETR checkpoints are available in training/detectors. Data-curation experiments are kept separate from the model trainer. See THIRD_PARTY_NOTICES.md for dependency attribution.

License

The Mark One code and custom weights are released under GNU AGPL-3.0-only because the detector pipeline uses Ultralytics RT-DETR under AGPL-3.0. Third-party components retain their respective licenses; see LICENSE and THIRD_PARTY_NOTICES.md for details.

Citation

If you use Mark One OCR in research, you can cite the software as:

@misc{markoneocr2026,
  title        = {Mark One OCR},
  author       = {Mark One},
  year         = {2026},
  url          = {https://github.com/Isterikus/markone-ocr}
}