opsis-v1-nano-g3

opsis-v1-nano-g3 is a small local RAG visual-ingestion co-processor. It runs on one image or document crop and returns either concise discovery text or a GitHub-flavored Markdown table for downstream indexing and retrieval.

It is not a general-purpose assistant, not a replacement for native PDF text extraction, and not an authoritative OCR system. It sits before indexing, or beside a PDF routing pipeline, so visual content can become searchable without a hosted VLM API or inference GPU.

Compared with opsis-v1-nano-g2, g3 keeps the same v1 capability and output contract while moving to a 100,000-example curriculum and adapting only the vision encoder and vision-to-text connector. On the 600-case reviewed benchmark, g3 improves table structure and CPU latency while remaining close to g2 on semantic discovery. It is a balanced research checkpoint, not a universal quality upgrade over g2.

Native V1 Output Modes

Mode	Native output	Intended use
description	<description>concise factual text</description>	Ordinary images, charts, and diagrams that need searchable RAG metadata.
table	<table>GitHub-flavored Markdown table</table>	Table images that need recoverable rows and cells rather than prose.

Charts stay in description mode and should include visible labels, values, and trends. Diagrams stay in description mode and should include visible nodes and relationships.

Output Contract

The raw Hugging Face model output is one tagged string. It is not unrestricted assistant prose. Decode it by wrapper:

Wrapper	Parsed kind	Decoding
<description>...</description>	description	Strip the wrapper and retain concise factual text.
<table>...</table>	table	Strip the wrapper and retain the complete Markdown table.

The companion parser normalizes either result into one object:

{
  "kind": "description",
  "text": "A line chart shows quarterly revenue rising from $12M in Q1 to $18M in Q4.",
  "latency_seconds": 3.64,
  "model_id": "yafitzdev/opsis-v1-nano-g3",
  "prompt_version": "rag-image-v3"
}

The model does not return bounding boxes, OCR confidence, source coordinates, PDF text blocks, or verified numeric facts. Preserve the source image or page crop beside generated text when exact visual evidence matters.

Intended Use

Use this model when a RAG or retrieval system needs local visual signals for:

• indexing ordinary images with concise discovery descriptions,
• converting compact table images into Markdown,
• recording visible chart labels, values, and trends,
• recording labeled diagram nodes and relationships,
• enriching visual regions routed out of otherwise native-text PDFs,
• keeping document ingestion local on CPU-only systems.

This model is not intended to replace reliable native table extraction, parse full scanned pages as a layout engine, verify high-stakes measurements, or replace source review when exact cells and labels matter.

Input Format

The model accepts one raster image per inference. For direct Transformers use, pair the image with the v1 parser prompt:

Parse this image for RAG search. If it is a table, output only the complete
table as Markdown. Otherwise output only a concise factual description. Include
meaningful visible text. For a diagram, name its labels and relationships. For
a chart, mention its labels, values, and trend. Do not speculate or add a heading.

For PDFs, extract reliable native text normally and send only table, image, chart, or diagram regions to Opsis. Crop quality and readable resolution have a direct effect on output quality.

Quick Start

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

MODEL_ID = "yafitzdev/opsis-v1-nano-g3"
PROMPT = """Parse this image for RAG search. If it is a table, output only the complete
table as Markdown. Otherwise output only a concise factual description. Include meaningful
visible text. For a diagram, name its labels and relationships. For a chart, mention its labels,
values, and trend. Do not speculate or add a heading."""

processor = AutoProcessor.from_pretrained(
    MODEL_ID,
    size={"longest_edge": 1024},
)
model = AutoModelForImageTextToText.from_pretrained(
    MODEL_ID,
    dtype=torch.float32,
).eval()
image = Image.open("image.png").convert("RGB")

messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": PROMPT},
    ],
}]
prompt_text = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
)
inputs = processor(text=prompt_text, images=[image], return_tensors="pt")

with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        do_sample=False,
        max_new_tokens=768,
        repetition_penalty=1.05,
    )

generated = output_ids[:, inputs["input_ids"].shape[-1]:]
text = processor.batch_decode(generated, skip_special_tokens=True)[0]
print(text)

CPU ONNX

The repository includes the quality-preserving split FP32 ONNX runtime:

• onnx/vision_encoder.onnx
• onnx/embed_tokens.onnx
• onnx/decoder_model_merged.onnx

With the rag-image-parser project installed, run:

uv run rag-image-parse parse .\image.png `
  --model yafitzdev/opsis-v1-nano-g3 `
  --backend onnx `
  --onnx-variant fp32 `
  --threads 8 `
  --image-longest-edge 1024

Mixed INT8 reduced routing and Markdown validity in evaluation and is not included in this repository.

Evaluation

The permanent Opsis gold benchmark contains 600 visually reviewed, source- unique cases with zero training overlap: tables, charts, diagrams, ordinary images, PDF-derived crops, and hard cases. Both checkpoints used greedy FP32 ONNX inference at a 1024-pixel longest edge on the same CPU environment.

Metric	G2	G3
kind accuracy	0.9967	0.9950
valid Markdown tables	0.9939	0.9879
exact tables	0.0364	0.0303
correct table shape	0.3939	0.4242
mean table cell precision	0.4806	0.4886
mean table cell recall	0.4419	0.4406
mean table cell F1	0.4468	0.4486
required-term recall	0.6644	0.6601
directed-relation recall	0.5463	0.5481
median CPU latency	3.807 s	3.635 s
p95 CPU latency	12.104 s	11.531 s
peak process RSS	2,394 MiB	2,402 MiB

Bucket-level checkpoint comparison:

Benchmark slice	G2	G3
table cell F1	0.4468	0.4486
table shape accuracy	0.3939	0.4242
chart required-term recall	0.9225	0.9183
diagram required-term recall	0.7113	0.7008
diagram directed-relation recall	0.5463	0.5481
ordinary-image required-term recall	0.3595	0.3612

G3 is not uniformly better. It is stronger on table structure, table precision, ordinary-image terms, directed relationships, and latency. G2 remains slightly stronger on image-kind routing, valid-table rate, exact tables, chart terms, and diagram terms. These small deltas should not be generalized beyond this benchmark.

Training Data

Training bucket	Examples	Validation examples	Role
clean tables	25,000	250	Compact table image to Markdown.
clean charts	12,500	125	Chart labels, values, and trends.
clean diagrams	12,500	125	Diagram nodes and relationships.
clean ordinary images	25,000	250	Ordinary-image discovery descriptions.
PDF-style visual crops	25,000	250	Document-domain versions balanced across all four content types.
Total	100,000	1,000	Source-held-out balanced visual parsing.

After distributing the PDF bucket, the effective mix is 31,250 tables, 18,750 charts, 18,750 diagrams, and 31,250 ordinary images. The corrected chart curriculum contains 12,500 real ChartQA examples and 6,250 synthetic examples. The manifest has 96,875 unique image paths; 3,125 real-chart examples are deterministically rehearsed to preserve the exact balanced 100,000-example run.

Sources include PubTabNet, ChartQA, AI2D, DOCCI, and locally generated charts and diagrams. Training used LoRA rank 8 with alpha 16 for one epoch at a 1024-pixel longest edge and selected checkpoint 6,000 by validation loss. It updated 692,736 of 257,177,664 parameters (0.269%), restricted to vision attention and the vision-to-text connector; the text decoder remained frozen.

Artifacts

This repository contains:

• model.safetensors: standalone merged Transformers checkpoint,
• onnx/vision_encoder.onnx: FP32 vision encoder and connector,
• onnx/embed_tokens.onnx: FP32 token embedding graph,
• onnx/decoder_model_merged.onnx: FP32 autoregressive decoder,
• tokenizer, processor, generation, and model configuration files,
• rag_image_parser_config.json: Opsis release and training metadata,
• SHA256SUMS: checksums for the packaged model and ONNX graphs.

Limitations

1. Exact table recovery remains low. Better shape accuracy does not mean reliable exact cell transcription; exact table accuracy is 3.0%.
2. G3 is a mixed trade-off, not a strict upgrade. Several measured gains are small, while chart and diagram term recall are slightly below g2.
3. Generated labels and values can be wrong. Descriptions and cells are RAG discovery metadata, not verified evidence.
4. English-centric mixed-source training. Multilingual behavior is not established, and some source pools use deterministic augmentation.
5. Crop quality matters. Tiny text, dense pages, poor scans, and incorrect visual-region routing reduce performance.
6. CPU latency varies by image complexity. Dense tables and long outputs can be substantially slower than the reported median.

License

Mixed-source research preview. The training mixture includes research-restricted AI2D data and sources with separate attribution, redistribution, or underlying- image terms. This package must not be represented as commercially cleared. Review and satisfy every source obligation before commercial use.