Mike0021/pulpie-orange-small-onnx

Pulpie Orange Small ONNX

Optimized ONNX export of feyninc/pulpie-orange-small, a 210M-parameter EuroBERT token-classification model for extracting main content from HTML.

The model takes tokenized simplified HTML blocks packed as:

[BOS] block_0 [<|sep|>] block_1 [<|sep|>] ... [EOS]

It returns logits shaped [batch_size, sequence_length, 2]. Pulpie reads predictions at the <|sep|> token positions; class 1 is main, class 0 is other.

Files

File	Description
model.onnx	Validated FP32 ONNX Runtime model, dynamic batch and sequence axes
tokenizer.json, tokenizer_config.json, special_tokens_map.json	Source tokenizer files
config.json, configuration_eurobert.py, modeling_eurobert.py	Source config/code for reference and tokenizer/model metadata
export_report.json	Export settings and ONNX graph summary
verification_report.json	Checker, parity, extraction, benchmark, and memory results
quantized_report.json	Int8 dynamic-quantization experiment results

Install

pip install onnxruntime transformers "pulpie[markdown]" huggingface_hub numpy

Raw ONNX Runtime Inference

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

repo_id = "Mike0021/pulpie-orange-small-onnx"
model_path = hf_hub_download(repo_id, "model.onnx")

tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
sep_id = tokenizer.convert_tokens_to_ids("<|sep|>")

block = '<p _item_id="0">This is the main article text.</p>'
input_ids = [tokenizer.bos_token_id]
input_ids += tokenizer.encode(block, add_special_tokens=False)
input_ids += [sep_id, tokenizer.eos_token_id]

inputs = {
    "input_ids": np.asarray([input_ids], dtype=np.int64),
    "attention_mask": np.ones((1, len(input_ids)), dtype=np.int64),
}

session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
logits = session.run(["logits"], inputs)[0]
label = int(logits[0, input_ids.index(sep_id)].argmax())
print("main" if label == 1 else "other")

Pulpie-Style Extraction

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

from pulpie.chunker import SEP_TOKEN, extract_blocks, pack_chunks, tokenize_blocks
from pulpie.markdown import to_markdown
from pulpie.model_utils import extract_item_ids, predictions_to_labels
from pulpie.reconstruct import extract_main_html
from pulpie.simplify import simplify

repo_id = "Mike0021/pulpie-orange-small-onnx"
model_path = hf_hub_download(repo_id, "model.onnx")
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])

def extract_markdown(html: str) -> str:
    simplified, map_html = simplify(html)
    blocks = extract_blocks(simplified)
    item_ids = extract_item_ids(blocks)
    sep_id = tokenizer.convert_tokens_to_ids(SEP_TOKEN)
    chunks = pack_chunks(
        tokenize_blocks(blocks, tokenizer),
        max_tokens=8192,
        sep_token_id=sep_id,
        bos_token_id=tokenizer.bos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

    predictions = [0] * len(blocks)
    for chunk_ids, block_indices in chunks:
        input_ids = np.asarray([chunk_ids], dtype=np.int64)
        attention_mask = np.ones_like(input_ids, dtype=np.int64)
        logits = session.run(["logits"], {"input_ids": input_ids, "attention_mask": attention_mask})[0][0]
        sep_positions = np.nonzero(input_ids[0] == sep_id)[0]
        preds = logits[sep_positions].argmax(axis=-1).tolist()
        for i, block_idx in enumerate(block_indices):
            if i < len(preds):
                predictions[block_idx] = int(preds[i])

    labels = predictions_to_labels(item_ids, predictions)
    return to_markdown(extract_main_html(map_html, labels))

Verification

Environment: CPU-only, PyTorch 2.12.1+cu130, Transformers 4.57.6, ONNX 1.22.0, ONNX Runtime 1.27.0.

Check	Result
onnx.checker.check_model(model.onnx)	Pass
onnxruntime.InferenceSession	Pass
Nonstandard ONNX node domains	None
Main ONNX opset	18
Dynamic axes	batch_size, sequence_length
Graph nodes	671
End-to-end ONNX extraction	Non-empty Markdown, 149 chars on article_basic

Numerical parity against PyTorch FP32/eager:

Input	Shape	Max abs diff	Mean abs diff
article_basic	[1, 86, 2]	6.50e-6	1.73e-6
docs_page	[1, 131, 2]	8.58e-6	1.53e-6
news_article	[1, 140, 2]	1.00e-5	1.55e-6

All three inputs passed np.allclose(..., atol=1e-4, rtol=1e-3).

Benchmarks

CPU forward benchmark on a 131-token pulpie chunk, 5 measured reps:

Runtime	Mean latency	Min	Max
PyTorch FP32 eager	181.29 ms	143.41 ms	233.41 ms
ONNX Runtime FP32	110.98 ms	107.31 ms	119.91 ms

ONNX Runtime was 1.63x faster on this CPU benchmark. No CUDA device was available in the export environment, so GPU latency is not reported.

Approximate process RSS after model load and one inference:

Runtime	RSS	Delta after tokenizer/input prep
PyTorch FP32 eager	1723 MB	751 MB
ONNX Runtime FP32	1916 MB	945 MB

File sizes:

Artifact	Size
Source model.safetensors bf16	404 MB
Validated model.onnx FP32	808 MB

Quantization

Dynamic int8 quantization was evaluated locally. It reduced the ONNX file from 808 MB to 203 MB and loaded in ONNX Runtime, but it did not preserve extraction behavior: on the three verification examples it predicted no main blocks and produced empty Markdown. Because this did not maintain accuracy, the int8 model is not the validated artifact here. See quantized_report.json for the measured failure details.

Reproduction

The accepted model was exported with torch.onnx.export, opset 17 requested and opset 18 emitted by the current PyTorch exporter. ONNX Runtime extended optimization was tested but introduced com.microsoft fused nodes, so the uploaded model.onnx uses ORT basic optimization to keep the graph standard-domain and checker-clean.

python scripts/export_onnx.py --source-model artifacts/source_model --output-dir artifacts/onnx_model
python scripts/verify_onnx.py --source-model artifacts/source_model --onnx-model artifacts/onnx_model/model.onnx

Model weights retain the source model's CC BY-NC 4.0 license.