Package: python-doctr
Optical Character Recognition made seamless & accessible to anyone, powered by TensorFlow 2 & PyTorchWhat you can expect from this repository:efficient ways to parse textual information (localize and identify each word) from your documentsguidance on how to integrate this in your current architectureQuick TourGetting your pretrained modelEnd-to-End OCR is achieved in docTR using a two-stage approach: text detection (localizing words), then text recognition (identify all characters in the word).
As such, you can select the architecture used fortext detection, and the one fortext recognitionfrom the list of available implementations.fromdoctr.modelsimportocr_predictormodel=ocr_predictor(det_arch='db_resnet50',reco_arch='crnn_vgg16_bn',pretrained=True)Reading filesDocuments can be interpreted from PDF or images:fromdoctr.ioimportDocumentFile# PDFpdf_doc=DocumentFile.from_pdf("path/to/your/doc.pdf")# Imagesingle_img_doc=DocumentFile.from_images("path/to/your/img.jpg")# Webpagewebpage_doc=DocumentFile.from_url("https://www.yoursite.com")# Multiple page imagesmulti_img_doc=DocumentFile.from_images(["path/to/page1.jpg","path/to/page2.jpg"])Putting it togetherLet's use the default pretrained model for an example:fromdoctr.ioimportDocumentFilefromdoctr.modelsimportocr_predictormodel=ocr_predictor(pretrained=True)# PDFdoc=DocumentFile.from_pdf("path/to/your/doc.pdf")# Analyzeresult=model(doc)Dealing with rotated documentsShould you use docTR on documents that include rotated pages, or pages with multiple box orientations,
you have multiple options to handle it:If you only use straight document pages with straight words (horizontal, same reading direction),
consider passingassume_straight_boxes=Trueto the ocr_predictor. It will directly fit straight boxes
on your page and return straight boxes, which makes it the fastest option.If you want the predictor to output straight boxes (no matter the orientation of your pages, the final localizations
will be converted to straight boxes), you need to passexport_as_straight_boxes=Truein the predictor. Otherwise, ifassume_straight_pages=False, it will return rotated bounding boxes (potentially with an angle of 0°).If both options are set to False, the predictor will always fit and return rotated boxes.To interpret your model's predictions, you can visualize them interactively as follows:result.show()Or even rebuild the original document from its predictions:importmatplotlib.pyplotaspltsynthetic_pages=result.synthesize()plt.imshow(synthetic_pages[0]);plt.axis('off');plt.show()Theocr_predictorreturns aDocumentobject with a nested structure (withPage,Block,Line,Word,Artefact).
To get a better understanding of our document model, check ourdocumentation:You can also export them as a nested dict, more appropriate for JSON format:json_output=result.export()Use the KIE predictorThe KIE predictor is a more flexible predictor compared to OCR as your detection model can detect multiple classes in a document. For example, you can have a detection model to detect just dates and addresses in a document.The KIE predictor makes it possible to use detector with multiple classes with a recognition model and to have the whole pipeline already setup for you.fromdoctr.ioimportDocumentFilefromdoctr.modelsimportkie_predictor# Modelmodel=kie_predictor(det_arch='db_resnet50',reco_arch='crnn_vgg16_bn',pretrained=True)# PDFdoc=DocumentFile.from_pdf("path/to/your/doc.pdf")# Analyzeresult=model(doc)predictions=result.pages[0].predictionsforclass_nameinpredictions.keys():list_predictions=predictions[class_name]forpredictioninlist_predictions:print(f"Prediction for{class_name}:{prediction}")The KIE predictor results per page are in a dictionary format with each key representing a class name and it's value are the predictions for that class.If you are looking for support from the Mindee teamInstallationPrerequisitesPython 3.8 (or higher) andpipare required to install docTR.Since we useweasyprint, you will need extra dependencies if you are not running Linux.For MacOS users, you can install them as follows:brewinstallcairopangogdk-pixbuflibffiFor Windows users, those dependencies are included in GTK. You can find the latest installer overhere.Latest releaseYou can then install the latest release of the package usingpypias follows:pipinstallpython-doctr:warning: Please note that the basic installation is not standalone, as it does not provide a deep learning framework, which is required for the package to run.We try to keep framework-specific dependencies to a minimum. You can install framework-specific builds as follows:# for TensorFlowpipinstall"python-doctr[tf]"# for PyTorchpipinstall"python-doctr[torch]"For MacBooks with M1 chip, you will need some additional packages or specific versions:TensorFlow 2:metal pluginPyTorch:version >= 1.12.0Developer modeAlternatively, you can install it from source, which will require you to installGit.
First clone the project repository:gitclonehttps://github.com/mindee/doctr.git
pipinstall-edoctr/.Again, if you prefer to avoid the risk of missing dependencies, you can install the TensorFlow or the PyTorch build:# for TensorFlowpipinstall-edoctr/.[tf]# for PyTorchpipinstall-edoctr/.[torch]Models architecturesCredits where it's due: this repository is implementing, among others, architectures from published research papers.Text DetectionDBNet:Real-time Scene Text Detection with Differentiable Binarization.LinkNet:LinkNet: Exploiting Encoder Representations for Efficient Semantic SegmentationText RecognitionCRNN:An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition.SAR:Show, Attend and Read:A Simple and Strong Baseline for Irregular Text Recognition.MASTER:MASTER: Multi-Aspect Non-local Network for Scene Text Recognition.ViTSTR:Vision Transformer for Fast and Efficient Scene Text Recognition.PARSeq:Scene Text Recognition with Permuted Autoregressive Sequence Models.More goodiesDocumentationThe full package documentation is availableherefor detailed specifications.Demo appA minimal demo app is provided for you to play with our end-to-end OCR models!Live demoCourtesy of :hugs:Hugging Face:hugs:, docTR has now a fully deployed version available onSpaces!
Check it outRunning it locallyIf you prefer to use it locally, there is an extra dependency (Streamlit) that is required.Tensorflow versionpipinstall-rdemo/tf-requirements.txtThen run your app in your default browser with:USE_TF=1streamlitrundemo/app.pyPyTorch versionpipinstall-rdemo/pt-requirements.txtThen run your app in your default browser with:USE_TORCH=1streamlitrundemo/app.pyTensorFlow.jsInstead of having your demo actually running Python, you would prefer to run everything in your web browser?
Check out ourTensorFlow.js demoto get started!Docker containerWe offers Docker container support for easy testing and deployment.Using GPU with docTR Docker ImagesThe docTR Docker images are GPU-ready and based on CUDA11.8.
However, to use GPU support with these Docker images, please ensure that Docker is configured to use your GPU.To verify and configure GPU support for Docker, please follow the instructions provided in theNVIDIA Container Toolkit Installation Guide.Once Docker is configured to use GPUs, you can run docTR Docker containers with GPU support:dockerrun-it--gpusallghcr.io/mindee/doctr:tf-py3.8.18-gpu-2023-09bashAvailable TagsThe Docker images for docTR follow a specific tag nomenclature:<framework>-py<python_version>-<system>-<doctr_version|YYYY-MM>. Here's a breakdown of the tag structure:<framework>:tf(TensorFlow) ortorch(PyTorch).<python_version>:3.8.18,3.9.18, or3.10.13.<system>:cpuorgpu<doctr_version>: a tag >=v0.7.1<YYYY-MM>: e.g.2023-09Here are examples of different image tags:TagDescriptiontf-py3.8.18-cpu-v0.7.1TensorFlow version3.8.18with docTRv0.7.1.torch-py3.9.18-gpu-2023-09PyTorch version3.9.18with GPU support and a monthly build from2023-09.Building Docker Images LocallyYou can also build docTR Docker images locally on your computer.dockerbuild-tdoctr.You can specify custom Python versions and docTR versions using build arguments. For example, to build a docTR image with TensorFlow, Python version3.9.10, and docTR versionv0.7.0, run the following command:dockerbuild-tdoctr--build-argFRAMEWORK=tf--build-argPYTHON_VERSION=3.9.10--build-argDOCTR_VERSION=v0.7.0.Example scriptAn example script is provided for a simple documentation analysis of a PDF or image file:pythonscripts/analyze.pypath/to/your/doc.pdfAll script arguments can be checked usingpython scripts/analyze.py --helpMinimal API integrationLooking to integrate docTR into your API? Here is a template to get you started with a fully working API using the wonderfulFastAPIframework.Deploy your API locallySpecific dependencies are required to run the API template, which you can install as follows:cdapi/
pipinstallpoetry
makelock
pipinstall-rrequirements.txtYou can now run your API locally:uvicorn--reload--workers1--host0.0.0.0--port=8002--app-dirapi/app.main:appAlternatively, you can run the same server on a docker container if you prefer using:PORT=8002docker-composeup-d--buildWhat you have deployedYour API should now be running locally on your port 8002. Access your automatically-built documentation athttp://localhost:8002/redocand enjoy your three functional routes ("/detection", "/recognition", "/ocr", "/kie"). Here is an example with Python to send a request to the OCR route:importrequestswithopen('/path/to/your/doc.jpg','rb')asf:data=f.read()response=requests.post("http://localhost:8002/ocr",files={'file':data}).json()Example notebooksLooking for more illustrations of docTR features? You might want to check theJupyter notebooksdesigned to give you a broader overview.CitationIf you wish to cite this project, feel free to use thisBibTeXreference:@misc{doctr2021,title={docTR: Document Text Recognition},author={Mindee},year={2021},publisher={GitHub},howpublished={\url{https://github.com/mindee/doctr}}}ContributingIf you scrolled down to this section, you most likely appreciate open source. Do you feel like extending the range of our supported characters? Or perhaps submitting a paper implementation? Or contributing in any other way?You're in luck, we compiled a short guide (cf.CONTRIBUTING) for you to easily do so!LicenseDistributed under the Apache 2.0 License. SeeLICENSEfor more information.