majorpurple/utilvision-onnx

UtilVision ONNX Models: Two-Stage Digital Meter Vision Pipeline

This repository hosts the optimized, edge-ready ONNX models for UtilVision, an in-browser computer vision pipeline for automated digital utility meter reading (AMR). These models are designed to execute entirely client-side inside web browser runtimes via ONNX Runtime Web (WebGPU and WebAssembly SIMD multi-threading) with zero server-side inference overhead.

• Interactive Production Application: https://utilvision.vercel.app/
• System Architecture and Benchmarks: https://utilvision.vercel.app/about.html

---

Base Models and License

Both models in this repository are specialized task-specific fine-tunes, not trained from scratch:

• Stage 2 LCD Detector: Fine-tuned from Ultralytics YOLO11n (Ultralytics/YOLO11), licensed under the GNU Affero General Public License v3.0 (AGPL-3.0).
• Stage 3 Digit Recognizer: Fine-tuned from PaddlePaddle PP-OCRv6 (PaddlePaddle/PP-OCRv6_medium_rec), licensed under the permissive Apache License 2.0.

Repository License Governance

Because the Stage 2 detector incorporates fine-tuned weights derived from Ultralytics YOLO11n under the GNU Affero General Public License v3.0 (AGPL-3.0), the copyleft provisions of the AGPL-3.0 govern the combined distribution of these model weights. Downstream developers incorporating the Stage 2 detector into network-accessible services or distributed software must comply with AGPL-3.0 source disclosure obligations, or obtain a commercial Ultralytics Enterprise license.

---

Pipeline Architecture

The pipeline decouples display localization from numeric sequence transcription to eliminate character hallucinations and isolate the active reading from high-clutter faceplates:

[Raw Field Capture (Photo / Video Frame)]
                 |
                 v
[Stage 2 Detector: YOLO11n Anchor-Free FPN] (640x640)
                 |
                 v (Isolates primary LCD bounding box)
[Canonical 96px Aspect-Preserving Normalization]
                 |
                 v (Canonical height 96px, dynamic width)
[Stage 3 Recognizer: PP-OCRv6 SVTR-CTC] (11-Token Head)
                 |
                 v (Pruned CTC Sequence Decoding)
[Verified Numeric kWh Reading] (e.g., "007894.21")

---

Pipeline Models and Parameter Summary

Stage	Model File	Base Architecture	Parameter Count	Precision	File Size	Task
Stage 2	stage2.onnx	YOLO11n Anchor-Free FPN	2.62M (2,624,389)	FP32	10.6 MB	LCD Counter Bounding Box Detection
Stage 3	stage3_fp16.onnx	PP-OCRv6 SVTR-CTC (11-Token)	15.51M (15,506,550)	FP16	31.4 MB	Constrained Digit Sequence Recognition
Stage 3	stage3_fp32.onnx	PP-OCRv6 SVTR-CTC (11-Token)	15.51M (15,506,550)	FP32	62.3 MB	High-Precision Sequence Recognition
End-to-End	UtilVision	Decoupled Cascade	18.13M (18,130,939)	FP16 Default	42.0 MB	Full In-Browser AMR Pipeline

---

Model Specifications and Tensor I/O

1. Stage 2 Detector (stage2.onnx)

• Task: LCD digital counter bounding box localization.
• Architecture: Ultralytics YOLO11n anchor-free feature pyramid network.
• Parameter Count: 2,624,389 parameters (2.62M).
• File Size: 10.6 MB.
• Input:
  • Tensor Name: images
  • Shape: [1, 3, 640, 640]
  • Data Type: float32
  • Format: RGB planar channel layout
  • Letterbox Padding: Grey #727272 (RGB 114, 114, 114)
  • Normalization Formula: f32 = pixel / 255.0 (maps [0, 255] to [0.0, 1.0])
• Output:
  • Tensor Name: output0
  • Shape: [1, 5, 8400]
  • Data Type: float32
  • Format: 8400 candidate anchor predictions. The 5 rows represent [center_x, center_y, width, height, confidence_score].

2. Stage 3 Recognizer (stage3_fp16.onnx and stage3_fp32.onnx)

• Task: Constrained digital counter transcription with CTC decoding.
• Architecture: Single Visual Token Representation (SVTR) backbone with 11-token pruned CTC classification head (tokens: digits 0-9 plus decimal point).
• Parameter Count: 15,506,550 parameters (15.51M).
• Input:
  • Tensor Name: x
  • Shape: [1, 3, 96, dynamic_width] (canonical height of 96 pixels; width scaled dynamically to preserve original LCD aspect ratio)
  • Data Type: float32
  • Format: RGB planar channel layout
  • Normalization Formula: f32 = (pixel / 255.0 - 0.5) / 0.5 (maps [0, 255] to [-1.0, 1.0])
• Output:
  • Tensor Name: save_infer_model/scale_0.tmp_0
  • Shape: [1, timesteps, 12]
  • Data Type: float32
  • Format: CTC probability logits across 12 tokens:
    • Index 0: CTC blank token (<blank>)
    • Index 1 to 10: Digits '0' through '9'
    • Index 11: Decimal point '.'

Precision Variants

• stage3_fp16.onnx (31.4 MB) - Default / Recommended: Half-precision floating point weights. Reduces download size and runtime memory bandwidth by 50% with zero measurable loss in exact-match transcription accuracy on edge hardware.
• stage3_fp32.onnx (62.3 MB) - Optional High-Precision: Full 32-bit floating point weights available on-demand via the application precision toggle.

---

Inference Hyperparameters and Runtime Execution

• Stage 2 Localization Parameters:
  • Minimum Detection Confidence Threshold: 0.25
  • Letterbox Dimensions: 640x640 with #727272 grey padding (RGB 114, 114, 114)
  • Conditional Test-Time Augmentation (TTA): 75% center zoom crop triggered automatically when primary detection confidence is below 0.60
  • Camera Stream Amortization: DETECT_EVERY = 5 (executes detection every 5th frame during continuous camera stream, amortizing bounding box computation)
• Stage 3 Recognition Parameters:
  • Canonical Display Height: 96 pixels
  • Dynamic Width Calculation: Scaled proportionally with aspect ratio preservation (typical range: 160px to 480px)
  • Decoding Scheme: Best-path greedy CTC decoding with contiguous duplicate suppression and blank token omission
  • Output Grammar: Verified strictly against utility ledger regex ^\d+(\.\d+)?$ (preserves verbatim leading zeros)

---

Benchmark Performance Across 10 Generations

All metrics evaluated on a master held-out test split under a Strict Physical Meter-Grouped Stratification Protocol (150 unique physical meters, zero meter ID overlap between training and test sets):

Gen	Architecture and Training Strategy	Exact Match	Digit Acc	Industrial (Set C)
1	Off-the-shelf CRNN baseline	24.1%	61.4%	8.2%
2	Custom 7-segment digit dictionary	52.8%	81.3%	19.5%
3	ResNet-18 visual feature extractor	68.3%	88.9%	31.2%
4	SVTR visual token interaction neck	74.1%	91.2%	38.6%
5	Targeted photometric and blur augmentations	81.4%	94.1%	46.2%
6	Dual-domain domestic and commercial balancing	85.9%	95.8%	49.8%
7	Industrial high-voltage domain integration	89.4%	97.1%	50.4%
8	High-density annotation boundary audit	91.8%	97.8%	51.7%
9	Canonical 96px aspect-preserving crop normalization	94.1%	98.6%	84.6%
10	Champion Stage 3 Recognizer (synthetic glare and segment cutout)	96.5%	99.4%	96.2%

Figures are from internal benchmark evaluation and are under continued revalidation as held-out testing expands.

---

Edge Runtime Latency

Benchmarked across 50 warm iterations using high-resolution client timing (performance.now()):

Hardware / Platform	Runtime Provider	Precision	End-to-End Latency
Apple M2 / NVIDIA RTX 4060	WebGPU (WGSL Shaders)	FP16	78.7 ms (Stage 2: 31.2 ms, Stage 3: 47.5 ms)
Intel Core i7-13700H	WASM SIMD (4 Threads)	FP16	214.3 ms (Stage 2: 74 ms, Stage 3: 140 ms)
OnePlus 13R (Snapdragon 8 Gen 3)	WASM SIMD (4 Threads)	FP16	Sub-1s (650 ms to 950 ms)
OnePlus 11R (Snapdragon 8+ Gen 1)	WASM SIMD (4 Threads)	FP16	1.0s to 1.5s
Cloud Vision APIs (Broadband)	Remote HTTPS Endpoint	N/A	1,100 ms to 2,200 ms (Network roundtrip)
Cloud Vision APIs (Cellular)	Remote HTTPS Endpoint	N/A	2,500 ms to 5,000+ ms (Network roundtrip)

Cold-Start and Runtime Warmup Notice: Initial inference passes (the first 1 to 2 reads upon launching the application) exhibit higher latency due to client-side runtime initialization:

• WebGPU Shader Compilation: Browser GPU drivers compile WebGPU Shading Language (WGSL) compute kernels upon the initial forward pass.
• WASM JIT Tiering and Thread Pool Initialization: WebAssembly engines allocate linear heap buffers, spawn Web Worker threads for SharedArrayBuffer parallelism, and transition execution through optimizing JIT compilers.
• Model Binary Deserialization: ONNX graphs are decompressed from local browser CacheStorage into active runtime memory. Subsequent inference cycles immediately operate at the steady-state latencies reported in the benchmark table above.

---

Cross-Origin Direct Fetch Usage

The model files in this repository can be fetched directly from web clients without intermediary proxy servers:

// Stage 2 Detector (10.6 MB)
const s2Url = 'https://huggingface.co/majorpurple/utilvision-onnx/resolve/main/stage2.onnx';

// Stage 3 Recognizer FP16 (31.4 MB - Recommended)
const s3Fp16Url = 'https://huggingface.co/majorpurple/utilvision-onnx/resolve/main/stage3_fp16.onnx';

// Stage 3 Recognizer FP32 (62.3 MB - On-Demand)
const s3Fp32Url = 'https://huggingface.co/majorpurple/utilvision-onnx/resolve/main/stage3_fp32.onnx';

Hugging Face Hub resolve/main endpoints serve valid Access-Control-Allow-Origin: * and Content-Length headers, allowing progress-tracked browser downloads and persistent offline caching via the CacheStorage API.