Skill: topdu/opencr-skill (version 0.1.4)

--- name: openocr-skills description: Extract text from images, documents and scanned PDFs using OpenOCR - supports text detection, recognition, universal VLM recognition, and document parsing with layout analysis author: openocr version: "0.1.4" tags: [ocr, text-detection, text-recognition, document-parsing, vlm, unirec, layout-analysis, formula, table] tools: [computer, code_execution, file_operations] library: name: OpenOCR url: https://github.com/Topdu/OpenOCR stars: 1k+ --- # OpenOCR Skill ## Overview This skill enables intelligent text extraction, document parsing, and universal recognition using **OpenOCR** - an accurate and efficient general OCR system. It provides a unified interface for text detection, text recognition, end-to-end OCR, VLM-based universal recognition (text/formulas/tables), and document parsing with layout analysis. Supports Chinese, English, and more. ## How to Use 1. Provide the image, scanned document, or PDF 2. Optionally specify the task type (det/rec/ocr/unirec/doc) 3. I'll extract text, formulas, tables, or full document structure **Example prompts:** - "Extract all text from this image" - "Detect text regions in this photo" - "Recognize the formula in this screenshot" - "Parse this PDF document with layout analysis" - "Convert this scanned page to Markdown" ## Domain Knowledge ### OpenOCR Fundamentals ```python from openocr import OpenOCR # Initialize with a specific task engine = OpenOCR(task='ocr') # Run OCR on an image (callable interface) results, time_dicts = engine(image_path='image.jpg') # Results contain detected boxes with recognized text for result in results: for line in result: box = line[0] # Bounding box coordinates text = line[1][0] # Recognized text conf = line[1][1] # Confidence score print(f"{text} ({conf:.2f})") ``` ### Supported Tasks ```python # Available task types tasks = { 'det': 'Text Detection - detect text regions with bounding boxes', 'rec': 'Text Recognition - recognize text from cropped images', 'ocr': 'End-to-End OCR - detection + recognition pipeline', 'unirec': 'Universal Recognition - VLM-based text/formula/table recognition (0.1B params)', 'doc': 'Document Parsing - layout analysis + universal recognition (0.1B params)', } # Task selection via parameter det_engine = OpenOCR(task='det') rec_engine = OpenOCR(task='rec') ocr_engine = OpenOCR(task='ocr') unirec_engine = OpenOCR(task='unirec') doc_engine = OpenOCR(task='doc') ``` ### Configuration Options ```python from openocr import OpenOCR # === Text Detection === detector = OpenOCR( task='det', backend='onnx', # 'onnx' (default) or 'torch' onnx_det_model_path=None, # Custom detection model (auto-downloads if None) use_gpu='auto', # 'auto', 'true', or 'false' ) # === Text Recognition === recognizer = OpenOCR( task='rec', mode='mobile', # 'mobile' (fast) or 'server' (accurate) backend='onnx', # 'onnx' (default) or 'torch' onnx_rec_model_path=None, # Custom recognition model use_gpu='auto', ) # === End-to-End OCR === ocr = OpenOCR( task='ocr', mode='mobile', # 'mobile' or 'server' backend='onnx', # 'onnx' or 'torch' onnx_det_model_path=None, # Custom detection model onnx_rec_model_path=None, # Custom recognition model drop_score=0.5, # Confidence threshold for filtering det_box_type='quad', # 'quad' or 'poly' (for curved text) use_gpu='auto', ) # === Universal Recognition (UniRec) === unirec = OpenOCR( task='unirec', unirec_encoder_path=None, # Custom encoder ONNX model unirec_decoder_path=None, # Custom decoder ONNX model tokenizer_mapping_path=None, # Custom tokenizer mapping JSON max_length=2048, # Max generation length auto_download=True, # Auto-download missing models use_gpu='auto', ) # === Document Parsing (OpenDoc) === doc = OpenOCR( task='doc', layout_model_path=None, # Custom layout detection model (PP-DocLayoutV2) unirec_encoder_path=None, # Custom UniRec encoder unirec_decoder_path=None, # Custom UniRec decoder tokenizer_mapping_path=None, # Custom tokenizer mapping layout_threshold=0.5, # Layout detection threshold use_layout_detection=True, # Enable layout analysis max_parallel_blocks=4, # Max parallel VLM blocks auto_download=True, # Auto-download missing models use_gpu='auto', ) ``` ### Task-Specific Usage #### Text Detection ```python from openocr import OpenOCR detector = OpenOCR(task='det', backend='onnx') # Detect text regions results = detector(image_path='image.jpg') boxes = results[0]['boxes'] # np.ndarray of bounding boxes elapse = results[0]['elapse'] # Processing time in seconds print(f"Found {len(boxes)} text regions in {elapse:.3f}s") for box in boxes: print(f" Box: {box.tolist()}") ``` #### Text Recognition ```python from openocr import OpenOCR # Mobile mode (fast, ONNX) recognizer = OpenOCR(task='rec', mode='mobile', backend='onnx') # Server mode (accurate, requires torch) # recognizer = OpenOCR(task='rec', mode='server', backend='torch') results = recognizer(image_path='word.jpg', batch_num=1) text = results[0]['text'] # Recognized text string score = results[0]['score'] # Confidence score elapse = results[0]['elapse'] # Processing time print(f"Text: {text}, Score: {score:.3f}, Time: {elapse:.3f}s") ``` #### End-to-End OCR ```python from openocr import OpenOCR ocr = OpenOCR(task='ocr', mode='mobile', backend='onnx') # Run OCR with visualization results, time_dicts = ocr( image_path='image.jpg', save_dir='./output', is_visualize=True, rec_batch_num=6, ) # Process results for result in results: for line in result: box, (text, confidence) = line[0], line[1] print(f"{text} ({confidence:.2f})") ``` #### Universal Recognition (UniRec) ```python from openocr import OpenOCR unirec = OpenOCR(task='unirec') # Image input result_text, generated_ids = unirec(image_path='formula.jpg', max_length=2048) print(f"Result: {result_text}") # PDF input (returns list of tuples, one per page) results = unirec(image_path='document.pdf', max_length=2048) for page_text, page_ids in results: print(f"Page: {page_text[:100]}...") ``` #### Document Parsing (OpenDoc) ```python from openocr import OpenOCR doc = OpenOCR(task='doc', use_layout_detection=True) # Parse a document image result = doc(image_path='document.jpg') # Save outputs in multiple formats doc.save_to_markdown(result, './output') doc.save_to_json(result, './output') doc.save_visualization(result, './output') # Parse a PDF (returns list of dicts, one per page) results = doc(image_path='document.pdf') for page_result in results: doc.save_to_markdown(page_result, './output') ``` ### Command-Line Interface ```bash # Text Detection openocr --task det --input_path image.jpg --is_vis # Text Recognition openocr --task rec --input_path word.jpg --mode server --backend torch # End-to-End OCR openocr --task ocr --input_path image.jpg --is_vis --output_path ./results # Universal Recognition openocr --task unirec --input_path formula.jpg --max_length 2048 # Document Parsing openocr --task doc --input_path document.pdf \ --use_layout_detection --save_vis --save_json --save_markdown # Launch Gradio Demos openocr --task launch_openocr_demo --share --server_port 7860 openocr --task launch_unirec_demo --share --server_port 7861 openocr --task launch_opendoc_demo --share --server_port 7862 ``` ### Processing Different Sources #### Image Files ```python from openocr import OpenOCR ocr = OpenOCR(task='ocr') # Single image results, _ = ocr(image_path='image.jpg') # Directory of images results, _ = ocr(image_path='./images/', save_dir='./output', is_visualize=True) ``` #### PDF Files ```python from openocr import OpenOCR # UniRec handles PDFs natively unirec = OpenOCR(task='unirec') results = unirec(image_path='document.pdf', max_length=2048) # OpenDoc handles PDFs natively with layout analysis doc = OpenOCR(task='doc', use_layout_detection=True) results = doc(image_path='document.pdf') # Save each page for page_result in results: doc.save_to_markdown(page_result, './output') doc.save_to_json(page_result, './output') ``` #### Numpy Array Input ```python import cv2 from openocr import OpenOCR ocr = OpenOCR(task='ocr') # Read image as numpy array img = cv2.imread('image.jpg') # Pass numpy array directly results, _ = ocr(img_numpy=img) ``` ### Result Formats ```python # Detection result format det_result = [{'boxes': np.ndarray, 'elapse': float}] # Recognition result format rec_result = [{'text': str, 'score': float, 'elapse': float}] # OCR result format (detection + recognition) ocr_result = (results_list, time_dicts) # results_list: [[[box, (text, confidence)], ...], ...] # UniRec result format # Image: (text: str, generated_ids: list) # PDF: [(text: str, generated_ids: list), ...] # one per page # Doc result format # Image: dict with layout blocks and recognized content # PDF: [dict, ...] # one per page ``` ## Best Practices 1. **Choose the Right Task**: Use `ocr` for general text, `unirec` for formulas/tables, `doc` for full documents 2. **Use Mobile Mode for Speed**: `mode='mobile'` is much faster; use `mode='server'` only when accuracy is critical 3. **Use ONNX Backend**: Default ONNX backend works on CPU without extra dependencies 4. **Set Appropriate Thresholds**: Adjust `drop_score` (OCR) and `layout_threshold` (Doc) for your use case 5. **Enable Layout Detection**: For documents with mixed content (text + formulas + tables), always enable `use_layout_detection` 6. **Batch Processing**: Use `rec_batch_num` to control recognition batch size for throughput optimization 7. **GPU Acceleration**: Install `onnxruntime-gpu` or PyTorch with CUDA for significant speedup ## Common Patterns ### Full Document Processing Pipeline ```python from openocr import OpenOCR import os def process_documents(input_dir, output_dir): """Process all documents in a directory.""" doc = OpenOCR(task='doc', use_layout_detection=True) os.makedirs(output_dir, exist_ok=True) for filename in os.listdir(input_dir): if filename.lower().endswith(('.jpg', '.png', '.pdf', '.bmp')): filepath = os.path.join(input_dir, filename) print(f"Processing: {filename}") result = doc(image_path=filepath) # Handle PDF (list) vs image (dict) if isinstance(result, list): for page_result in result: doc.save_to_markdown(page_result, output_dir) doc.save_to_json(page_result, output_dir) else: doc.save_to_markdown(result, output_dir) doc.save_to_json(result, output_dir) print(f"All results saved to {output_dir}") process_documents('./docs', './output') ``` ### OCR with Custom Post-Processing ```python from openocr import OpenOCR import re def extract_structured_text(image_path, drop_score=0.5): """Extract and structure text from an image.""" ocr = OpenOCR(task='ocr', drop_score=drop_score) results, _ = ocr(image_path=image_path) lines = [] for result in results: for line in result: box = line[0] text = line[1][0] confidence = line[1][1] # Calculate bounding box center y_center = sum(p[1] for p in box) / 4 lines.append({ 'text': text, 'confidence': confidence, 'y_center': y_center, 'box': box, }) # Sort by vertical position (top to bottom) lines.sort(key=lambda x: x['y_center']) return lines result = extract_structured_text('page.jpg') for line in result: print(f"{line['text']} ({line['confidence']:.2f})") ``` ### Formula Recognition ```python from openocr import OpenOCR def recognize_formula(image_path): """Recognize mathematical formula from image.""" unirec = OpenOCR(task='unirec') text, ids = unirec(image_path=image_path, max_length=2048) # UniRec outputs LaTeX for formulas print(f"LaTeX: {text}") return text latex = recognize_formula('formula.png') # Output: \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} ``` ### Table Extraction ```python from openocr import OpenOCR def extract_table(image_path): """Extract table content from image.""" unirec = OpenOCR(task='unirec') text, ids = unirec(image_path=image_path, max_length=2048) # UniRec outputs LaTeX table format print(f"Table: {text}") return text table_latex = extract_table('table.png') ``` ## Examples ### Example 1: Batch OCR with Progress ```python from openocr import OpenOCR import os def batch_ocr(image_dir, output_dir='./ocr_results'): """OCR all images in a directory.""" ocr = OpenOCR(task='ocr', mode='mobile') os.makedirs(output_dir, exist_ok=True) image_files = [ f for f in os.listdir(image_dir) if f.lower().endswith(('.jpg', '.jpeg', '.png', '.bmp', '.tiff')) ] all_results = {} for i, filename in enumerate(image_files): filepath = os.path.join(image_dir, filename) print(f"[{i+1}/{len(image_files)}] Processing: {filename}") results, time_dicts = ocr( image_path=filepath, save_dir=output_dir, is_visualize=True, ) texts = [] for result in results: for line in result: texts.append(line[1][0]) all_results[filename] = texts print(f" Found {len(texts)} text lines") # Save all text with open(os.path.join(output_dir, 'all_text.txt'), 'w') as f: for filename, texts in all_results.items(): f.write(f"--- {filename} ---\n") f.write('\n'.join(texts)) f.write('\n\n') return all_results results = batch_ocr('./images') ``` ### Example 2: Document to Markdown Converter ```python from openocr import OpenOCR import os def doc_to_markdown(input_path, output_dir='./markdown_output'): """Convert document images or PDFs to Markdown.""" doc = OpenOCR( task='doc', use_layout_detection=True, use_chart_recognition=True, ) os.makedirs(output_dir, exist_ok=True) result = doc(image_path=input_path) if isinstance(result, list): # PDF: multiple pages for page_result in result: doc.save_to_markdown(page_result, output_dir) print(f"Converted {len(result)} pages to Markdown") else: # Single image doc.save_to_markdown(result, output_dir) print("Converted image to Markdown") print(f"Output saved to: {output_dir}") # Convert a scanned PDF doc_to_markdown('paper.pdf') # Convert a document image doc_to_markdown('page.jpg') ``` ### Example 3: Multi-Task Comparison ```python from openocr import OpenOCR def compare_tasks(image_path): """Compare results from different OpenOCR tasks.""" # 1. Detection only det = OpenOCR(task='det') det_result = det(image_path=image_path) num_boxes = len(det_result[0]['boxes']) print(f"Detection: Found {num_boxes} text regions") # 2. End-to-End OCR ocr = OpenOCR(task='ocr') ocr_results, _ = ocr(image_path=image_path) ocr_texts = [line[1][0] for result in ocr_results for line in result] print(f"OCR: Extracted {len(ocr_texts)} text lines") for t in ocr_texts[:5]: print(f" - {t}") # 3. Universal Recognition unirec = OpenOCR(task='unirec') text, _ = unirec(image_path=image_path) print(f"UniRec: {text[:200]}...") return { 'det_boxes': num_boxes, 'ocr_texts': ocr_texts, 'unirec_text': text, } compare_tasks('document.jpg') ``` ### Example 4: Gradio Demo Launch ```python from openocr import launch_openocr_demo, launch_unirec_demo, launch_opendoc_demo # Launch OCR demo launch_openocr_demo(share=True, server_port=7860, server_name='0.0.0.0') # Launch UniRec demo launch_unirec_demo(share=True, server_port=7861) # Launch OpenDoc demo launch_opendoc_demo(share=True, server_port=7862) ``` ## Limitations - Text recognition accuracy depends on image quality - Very small or heavily rotated text may reduce accuracy - `server` mode requires PyTorch and is slower than `mobile` mode - UniRec and Doc tasks use 0.1B parameter VLM, larger models may yield better results - PDF processing converts pages to images internally, very large PDFs may use significant memory - Complex handwritten text accuracy varies - GPU recommended for best performance, especially for Doc and UniRec tasks ## Installation ```bash # Basic installation (CPU, ONNX backend) pip install openocr-python # GPU-accelerated ONNX inference pip install openocr-python[onnx-gpu] # PyTorch backend (for server mode) pip install openocr-python[pytorch] # Gradio demos pip install openocr-python[gradio] # All optional dependencies pip install openocr-python[all] # From source git clone https://github.com/Topdu/OpenOCR.git cd OpenOCR python build_package.py pip install ./build/dist/openocr_python-*.whl ``` ## Resources - [OpenOCR GitHub](https://github.com/Topdu/OpenOCR) - [PyPI Package](https://pypi.org/project/openocr-python/) - [UniRec Paper](https://github.com/Topdu/OpenOCR#unirec) - [OpenDoc Documentation](https://github.com/Topdu/OpenOCR#opendoc) - [Model Zoo & Configs](https://github.com/Topdu/OpenOCR/tree/main/configs)


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)