sizzlebop/pulpie-orange-small-onnx

Pulpie Orange Small ONNX & Quantized

Optimized ONNX Runtime releases of feyninc/pulpie-orange-small, a 210M-parameter EuroBERT token-classification model developed by Feyn for extracting primary content and removing boilerplate (navigation links, ads, cookie disclaimers, headers, and footers) from HTML pages.

This repository provides three high-fidelity ONNX model variants: the reference FP32 export, an optimized FP16 conversion, and a Q4 MatMul block-quantized variant. Both quantized variants maintain 100% classification accuracy and boilerplate separation across test suites.

Model Files

Filename	Precision / Format	Size	Accuracy Parity vs FP32	Recommended Use Case
model.onnx	FP32 (Full Precision)	807.9 MB	Reference (100%)	Reference testing, CPU/GPU validation
model_fp16.onnx	Float16 (convert_float_to_float16)	404.0 MB	100% exact label parity (max logit diff: 0.0027)	GPU inference (CUDA, DirectML, WebGPU)
model_q4.onnx	4-bit MatMul (MatMulNBitsQuantizer)	433.3 MB	100% exact label parity	Compact CPU/GPU deployment

Note on Q4 Model Size: EuroBERT includes a 128,257-token vocabulary embedding matrix ([128257, 768]), which occupies 375.8 MB on its own. The Q4 variant quantizes all transformer projection and MLP weights down to 4-bit while preserving full embedding fidelity, keeping total file size at 433.3 MB without classification loss.

How Pulpie Works

Pulpie structures HTML blocks into sequence chunks:

[BOS] block_0 [<|sep|>] block_1 [<|sep|>] ... [EOS]

The model outputs logits with shape [batch_size, sequence_length, 2]. Logits evaluated at each <|sep|> token index indicate the class for that block:

• Class 1: Main content
• Class 0: Boilerplate / other

Installation

Install ONNX Runtime and helper dependencies:

pip install onnxruntime transformers numpy

For GPU acceleration (CUDA 12.x):

pip install onnxruntime-gpu transformers numpy

Quickstart: Python Inference with ONNX Runtime

You can run any of the three models directly with ONNX Runtime:

from pathlib import Path
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

# Load tokenizer from the original base model
tokenizer = AutoTokenizer.from_pretrained("feyninc/pulpie-orange-small", trust_remote_code=True)
sep_id = tokenizer.convert_tokens_to_ids("<|sep|>")

# Choose model variant: "model.onnx", "model_fp16.onnx", or "model_q4.onnx"
model_path = "model_fp16.onnx"
providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
session = ort.InferenceSession(model_path, providers=providers)

# Example HTML block
block_text = '<p _item_id="0">This is the main article content.</p>'
token_ids = tokenizer.encode(block_text, add_special_tokens=False)
input_ids = [tokenizer.bos_token_id] + token_ids + [sep_id, tokenizer.eos_token_id]

inputs = {
    "input_ids": np.asarray([input_ids], dtype=np.int64),
    "attention_mask": np.ones((1, len(input_ids)), dtype=np.int64),
}

# Run inference
logits = session.run(["logits"], inputs)[0]
sep_index = input_ids.index(sep_id)
predicted_class = int(logits[0, sep_index].argmax(axis=-1))

print(f"Block label: {'main' if predicted_class == 1 else 'boilerplate'}")

End-to-End HTML Extraction with Pulpie Library

If you have the pulpie package installed:

pip install "pulpie[markdown]"

import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer
from pulpie.chunker import SEP_TOKEN, extract_blocks, pack_chunks, tokenize_blocks
from pulpie.markdown import to_markdown
from pulpie.model_utils import extract_item_ids, predictions_to_labels
from pulpie.reconstruct import extract_main_html
from pulpie.simplify import simplify

tokenizer = AutoTokenizer.from_pretrained("feyninc/pulpie-orange-small", trust_remote_code=True)
sep_id = tokenizer.convert_tokens_to_ids(SEP_TOKEN)
session = ort.InferenceSession("model_q4.onnx", providers=["CPUExecutionProvider"])

def extract_markdown_from_html(raw_html: str) -> str:
    simplified, map_html = simplify(raw_html)
    blocks = extract_blocks(simplified)
    block_tokens = tokenize_blocks(blocks, tokenizer)
    chunks = pack_chunks(
        block_tokens,
        max_tokens=8192,
        sep_token_id=sep_id,
        bos_token_id=tokenizer.bos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

    predictions = [0] * len(blocks)
    for chunk_ids, block_indices in chunks:
        input_ids = np.asarray([chunk_ids], dtype=np.int64)
        attention_mask = np.ones_like(input_ids, dtype=np.int64)
        logits = session.run(["logits"], {"input_ids": input_ids, "attention_mask": attention_mask})[0][0]
        sep_positions = np.nonzero(input_ids[0] == sep_id)[0]
        preds = logits[sep_positions].argmax(axis=-1).tolist()
        for idx, block_idx in enumerate(block_indices):
            if idx < len(preds):
                predictions[block_idx] = int(preds[idx])

    labels = predictions_to_labels(extract_item_ids(blocks), predictions)
    main_html = extract_main_html(map_html, labels)
    return to_markdown(main_html)

Verification & Parity Results

All three models were verified against the benchmark suite on three distinct document types (article_basic, docs_page, and news_article).

Document	Total Blocks	FP32 Predictions	FP16 Predictions	Q4 Predictions	Extraction Status
article_basic	4	[1, 1, 1, 0]	[1, 1, 1, 0]	[1, 1, 1, 0]	100% Match (149 chars)
docs_page	6	[0, 1, 1, 1, 1, 0]	[0, 1, 1, 1, 1, 0]	[0, 1, 1, 1, 1, 0]	100% Match (207 chars)
news_article	6	[0, 1, 1, 1, 1, 0]	[0, 1, 1, 1, 1, 0]	[0, 1, 1, 1, 1, 0]	100% Match (264 chars)

License & Attribution

• Model Weights License: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0), as licensed by the original creator Feyn.
• Original Model Checkpoint: feyninc/pulpie-orange-small