AXDocLayout-M v1.0

AX-DocLayout detects the layout regions of a single document page. Supply an already-decoded RGB page raster to obtain labelled bounding boxes and confidence scores in that page's original pixel coordinates. The model identifies 11 region types, including text, titles, tables, pictures, captions, and formulas.

Property	Value
Model id	`AutomatosX/AXDocLayout-M-v1.0`
Public weight revision	`1f7412f07780b58e7d98cdb3dd7b9d054b317353`
Task / pipeline tag	`object-detection`
Architecture	RT-DETRv2-M, `rtdetrv2_r50vd_m`
Backbone	ResNet-50-vd
Page preprocessing	`rgb-letterbox-1024/1`
Weight formats	FP32 safetensors and FP32 ONNX, opset 16
Output contract	`ax-doclayout.detection/1`
License	Apache-2.0

Purpose and practical value

Document processing often needs to distinguish a title from body text, locate a table, or separate a picture from its caption before another component processes those regions. AX-DocLayout provides the boxes and region labels for that step.

These detections can help a caller route text regions to an OCR system, identify regions for a separate table-processing component, or present layout regions for human review. Original-raster coordinates let those components work against the same page image. A versioned label map, preprocessing contract, and JSON schema make integration explicit.

The detector returns regions; downstream components supply OCR, table structure, document jobs, translation, and other processing. Any claim about accuracy gains or processing-time savings requires evaluation in the caller's own workflow.

Architecture origin and model design

The detector architecture comes from the official RT-DETR repository, by Wenyu Lv and the RT-DETR contributors. AXDocLayout-M v1.0 uses the RT-DETRv2-M variant rtdetrv2_r50vd_m; its architecture reference is pinned to upstream commit 068dfde65f2667ad6555883c69d73de886518cad.

The design combines a ResNet-50-vd backbone, a hybrid feature encoder, and a Transformer detection decoder. The backbone provides features at multiple scales; the encoder combines those features; the decoder produces class scores and bounding boxes. The released inference route uses 300 object queries and the first three decoder layers. An optional routed profile for the PyTorch backend reads up to the fifth decoder layer. Its label space has 12 slots: reserved id 0 and 11 public region classes.

The architecture attribution does not identify these weights as an upstream pretrained checkpoint. This is the AXDocLayout-M v1.0 weight bundle with its own document-layout labels and inference contract. Upstream benchmark scores and speed measurements do not describe this bundle.

Architecture reference: Wenyu Lv, Yian Zhao, Qinyao Chang, Kui Huang, Guanzhong Wang, and Yi Liu (2024), RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer.

Release and installation

The FP32 weights and inference-only Python source are public on this model repository. The source under inference/ contains no training helpers or training data. Python 3.12 or 3.13 is required. Previously published source wheels were withdrawn; no replacement wheel is published.

Download only the inference source at its immutable source revision:

python -m pip install "huggingface-hub>=0.36,<2"
python - <<'PY_SOURCE'
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="AutomatosX/AXDocLayout-M-v1.0",
    revision="f721a0ed8a32001e27778ceef4c093dd505c0a87",
    allow_patterns=["inference/**"],
    local_dir="ax-doclayout-source",
    token=False,
)
PY_SOURCE

# ONNX Runtime on CPU
python -m pip install -e "ax-doclayout-source/inference[onnx]"

# Alternatively, PyTorch on CPU
python -m pip install -e "ax-doclayout-source/inference[torch]"

Source revision: f721a0ed8a32001e27778ceef4c093dd505c0a87. The source revision above and the weight revision used by from_pretrained serve different purposes: source installation versus immutable model bytes. Access to a private GitHub repository is not required.

library_name: ax-doclayout identifies the Python package, whose import is ax_doclayout. This bundle is not a Transformers model and does not use AutoModel or transformers.pipeline. The object-detection task tag describes the model's task; it does not promise a hosted inference service.

Python usage

The caller decodes and orients the raster before passing it to the model. This example reads a raster image, not a PDF:

import json

import numpy as np
from PIL import Image

from ax_doclayout import AXDocLayout, PageImage

model = AXDocLayout.from_pretrained(
    "AutomatosX/AXDocLayout-M-v1.0",
    revision="1f7412f07780b58e7d98cdb3dd7b9d054b317353",
    backend="onnx",  # use "pytorch" with the torch extra installed
)

with Image.open("page.png") as image:
    rgb = np.asarray(image.convert("RGB"), dtype=np.uint8)

result = model.predict(PageImage(rgb=rgb, page_index=0))
print(json.dumps(result.to_json(), indent=2, allow_nan=False))

The downloaded source contains inference/examples/predict_page.py, a complete command-line example with PDF rejection, pixel limits, and content-free error messages:

python ax-doclayout-source/inference/examples/predict_page.py page.png --backend onnx
python ax-doclayout-source/inference/examples/predict_page.py page.png --backend pytorch --page-index 0

The first load downloads and caches the nine model-bundle files from the specified commit. Both formats are downloaded because the trusted receipt binds the complete bundle. Each load verifies all file SHA-256 digests, the artifact manifest, the content root, and FP32 weight metadata before creating the selected backend. The loaded backend is reused for subsequent pages. No credentials or page images are sent to a remote inference service.

Supply the exact revision and backend. Floating main, latest, tags, untrusted revisions, unsupported backends, and mismatched files are rejected. Missing backend dependencies and download failures raise domain errors. There is no silent fallback between ONNX and PyTorch.

Optional routed exit profile

The default profile reads every class from the third decoder layer. The safetensors file contains all six decoder layers, so the PyTorch backend can also read each class from a decoder layer chosen for that class, in one forward pass:

model = AXDocLayout.from_pretrained(
    "AutomatosX/AXDocLayout-M-v1.0",
    revision="1f7412f07780b58e7d98cdb3dd7b9d054b317353",
    backend="pytorch",
    exit_profile="routed",
)

python ax-doclayout-source/inference/examples/predict_page.py page.png \
  --backend pytorch --exit-profile routed

The routed profile is optional and the default behavior is unchanged. It is available only with the PyTorch backend: requesting it with ONNX raises an error, because the ONNX file always implements the default profile. The weights, the nine bundle files, and the result schema are unchanged, and a result does not record which profile produced it, so record the profile yourself when you store results. Routed inference runs two more decoder layers and extra head read-outs, so it takes more CPU time per page. This card publishes no benchmark for it: the profile was assessed internally only, and its effect differs by class. Evaluate it on representative pages before adopting it. The default profile remains the documented, qualified behavior.

Input and preprocessing

PageImage takes a non-empty H x W x 3 NumPy uint8 RGB array and a non-negative, zero-based page_index. Pages above 50,000,000 pixels are rejected. Page orientation belongs to the caller.

Preprocessing follows rgb-letterbox-1024/1:

Resize the longer side to 1024 using bilinear interpolation and preserve the aspect ratio.
Pad symmetrically to 1024 x 1024 with white RGB (255, 255, 255). Put an odd padding remainder on the bottom and right.
Convert to float32 NCHW and divide by 255, yielding values in [0, 1].
Run the detector, undo padding and scaling, and clip boxes to the original raster. Drop non-finite or degenerate boxes.

For direct ONNX integration, images is float32 [N, 3, 1024, 1024] and orig_target_sizes is int64 [N, 2]. Despite the input name, the exported graph expects the letterbox canvas size, [1024, 1024] for each page. Its outputs are labels, boxes, and scores. The raw boxes are in letterbox-space xyxy coordinates and must be inverse-mapped before use on the original raster. The Python API handles these steps for one page at a time.

Output and label map

The result's to_json() returns the ax-doclayout.detection/1 contract, including the model id, immutable revision, artifact digest, page dimensions, and detected elements. Each element contains id, class_id, class_name, confidence, and bbox with x_min, y_min, x_max, and y_max.

Boxes use half-open pixel edges in raster-pixels-top-left, with x increasing rightward and y increasing downward. They refer to the original raster, not PDF points or the 1024 canvas. Element ids follow {page_index}:{ordinal} after filtering. They do not encode reading order or hierarchy.

ID	Class name
0	reserved, never returned
1	`caption`
2	`footnote`
3	`formula`
4	`list_item`
5	`page_footer`
6	`page_header`
7	`picture`
8	`section_header`
9	`table`
10	`text`
11	`title`

The taxonomy id is ax-doclayout-doclaynet-11/1. caption and footnote are generic classes. The public API uses score floor 0.30 under threshold profile default/1; reserved and unknown class ids and non-finite scores are dropped. Confidence is a detector score, not a guarantee of correctness.

Evaluation and limitations

This card does not publish benchmark results, latency, or throughput. Validate the pinned model on representative pages and the intended backend before relying on it in a workflow. There is no accuracy guarantee for any language, document domain, small region size, scan quality, or unusual layout. The fixed input resolution can reduce detail on large pages.

The model does not open PDFs, perform OCR, return extracted text, predict reading order or hierarchy, emit masks, infer orientation, or reconstruct table cells. A formula box does not contain a formula transcription, and a table box does not contain a table structure. The inference route documented here is CPU FP32. GPU, TensorRT, reduced precision, and automatic compatibility with other detectors are not qualified by this example. The routed profile is likewise CPU FP32 and not qualified.

Training information disclosure

The training dataset and training method are not disclosed in this release. The bundle publishes the model weights, architecture identity, label map, and inference configuration. It does not include the training corpus, training procedure, or an experiment reproduction recipe.

License and weight integrity

The weights and bundle are Apache License 2.0; see LICENSE and NOTICE. The upstream RT-DETR repository also uses Apache-2.0.

File	SHA-256	Bytes
`model.safetensors`	`4d9ee36db616f3f4ef293588e4258b5f6f42d7a84097a5afb520779076045bf2`	146,759,800
`model.onnx`	`d9f25943af5c12fe7ae13e7723608d1e3297004eb2cbabd6853054ab246267da`	133,276,848