TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
Paper • 2109.10282 • Published • 13
How to use inYourOwnBrowser/trocr-base-handwritten-ONNX with Transformers.js:
// npm i @huggingface/transformers
import { pipeline } from '@huggingface/transformers';
// Allocate pipeline
const pipe = await pipeline('image-text-to-text', 'inYourOwnBrowser/trocr-base-handwritten-ONNX');This is a mirror of onnx-community/trocr-base-handwritten-ONNX, which is itself an ONNX conversion of microsoft/trocr-base-handwritten for use with Transformers.js.
Hosted here as a pinned, self-controlled dependency for a browser-based, offline-first PDF tool. Pinning the app to a specific commit on this repo means the handwriting recognition model can't change out from under the app without an explicit update.
Only the files needed are included (not the full set of quantization variants from the source repo):
onnx/encoder_model.onnx, onnx/encoder_model_fp16.onnxonnx/decoder_model_merged.onnx, onnx/decoder_model_merged_fp16.onnximport { pipeline } from '@huggingface/transformers';
const captioner = await pipeline(
'image-to-text',
'inYourOwnBrowser/trocr-base-handwritten-ONNX',
{ dtype: 'fp32' } // or 'fp16'
);
const output = await captioner(imageDataUrl);
No changes were made to the model weights — this repo only re-hosts a subset of the original ONNX files.
Base model
microsoft/trocr-base-handwritten