Package: deepdoctection
A Document AI Packagedeepdoctection is a Python library that orchestrates document extraction and document layout analysis tasks using deep learning models. It does
not implement models but enables you to build pipelines using highly acknowledged libraries for object detection, OCR
and selected NLP tasks and provides an integrated framework for fine-tuning, evaluating and running models. For more
specific text processing tasks use one of the many other great NLP libraries.deepdoctection focuses on applications and is made for those who want to solve real world problems related to
document extraction from PDFs or scans in various image formats.Check the demo of a document layout analysis pipeline with OCR on
:hugs:Hugging Face spaces.Overviewdeepdoctection provides model wrappers of supported libraries for various tasks to be integrated into
pipelines. Its core function does not depend on any specific deep learning library. Selected models for the following
tasks are currently supported:Document layout analysis including table recognition in Tensorflow withTensorpack,
or PyTorch withDetectron2,OCR with support ofTesseract,DocTr(Tensorflow and PyTorch implementations available) and a wrapper to an API for a commercial solution,Text mining for native PDFs withpdfplumber,Language detection withfastText,Deskewing and rotating images withjdeskew.Document and token classification with all LayoutLM models provided by the Transformer library.
(Yes, you can use any LayoutLM-model with any of the provided OCR-or pdfplumber tools straight away!).Table detection and table structure recognition withtable-transformer.There is a small dataset for token classificationavailableand a lot of newtutorialsto show, how to train and evaluate this dataset using LayoutLMv1, LayoutLMv2, LayoutXLM and LayoutLMv3.Comprehensive configuration ofanalyzerlike choosing different models, output parsing, OCR selection.
Check thisnotebookor thedocsfor more infos.Document layout analysis and table recognition now runs with Torchscript (CPU) as well and Detectron2 is
not required anymore for basic inference.deepdoctection provides on top of that methods for pre-processing inputs to models like cropping or resizing and to
post-process results, like validating duplicate outputs, relating words to detected layout segments or ordering words
into contiguous text. You will get an output in JSON format that you can customize even further by yourself.Have a look at theintroduction notebookin thenotebook repofor an easy start.Check therelease notesfor recent updates.Modelsdeepdoctection or its support libraries provide pre-trained models that are in most of the cases available at theHugging Face Model Hubor that will be automatically downloaded once
requested. For instance, you can find pre-trained object detection models from the Tensorpack or Detectron2 framework
for coarse layout analysis, table cell detection and table recognition.Datasets and training scriptsTraining is a substantial part to get pipelines ready on some specific domain, let it be document layout analysis,
document classification or NER.deepdoctection provides training scripts for models that are based on trainers
developed from the library that hosts the model code. Moreover,deepdoctection hosts code to some well established
datasets likePublaynetthat makes it easy to experiment. It also contains mappings from widely used data
formats like COCO and it has a dataset framework (akin todatasetsso that
setting up training on a custom dataset becomes very easy.This notebookshows you how to do this.Evaluationdeepdoctection comes equipped with a framework that allows you to evaluate predictions of a single or multiple
models in a pipeline against some ground truth. Check againherehow it is
done.InferenceHaving set up a pipeline it takes you a few lines of code to instantiate the pipeline and after a for loop all pages will
be processed through the pipeline.importdeepdoctectionasddfromIPython.core.displayimportHTMLfrommatplotlibimportpyplotaspltanalyzer=dd.get_dd_analyzer()# instantiate the built-in analyzer similar to the Hugging Face space demodf=analyzer.analyze(path="/path/to/your/doc.pdf")# setting up pipelinedf.reset_state()# Trigger some initializationdoc=iter(df)page=next(doc)image=page.viz()plt.figure(figsize=(25,17))plt.axis('off')plt.imshow(image)HTML(page.tables[0].html)print(page.text)DocumentationThere is an extensivedocumentationavailable
containing tutorials, design concepts and the API. We want to present things as comprehensively and understandably
as possible. However, we are aware that there are still many areas where significant improvements can be made in terms
of clarity, grammar and correctness. We look forward to every hint and comment that increases the quality of the
documentation.RequirementsEverything in the overview listed below thedeepdoctection layer are necessary requirements and have to be installed
separately.Linux or macOS. (Windows is not supported but there is aDockerfileavailable)Python >= 3.81.12 <= PyTorch < 2.0orTensorflow >= 2.9 and CUDA. If you want to run the models provided by Tensorpack a GPU is
required. You can run on PyTorch with a CPU only.deepdoctection uses Python wrappers forPopplerto convert PDF documents into
images.With respect to the Deep Learning framework, you must decide betweenTensorflowandPyTorch.TesseractOCR engine will be used through a Python wrapper. The core
engine has to be installed separately.The following overview shows the availability of the models in conjunction with the DL framework.TaskPyTorchTorchscriptTensorflowLayout detection via Detectron2/Tensorpack✅✅ (CPU only)✅ (GPU only)Table recognition via Detectron2/Tensorpack✅✅ (CPU only)✅ (GPU only)Table transformer via Transformers✅❌❌DocTr✅❌✅LayoutLM (v1, v2, v3, XLM) via Transformers✅❌❌InstallationWe recommend using a virtual environment. You can install the package via pip or from source.Install with pip from PyPiMinimal installationIf you want to get started with a minimal setting (e.g. running thedeepdoctection analyzer with
default configuration or trying the 'Get started notebook'), installdeepdoctection withpip install deepdoctectionIf you want to use the Tensorflow framework, please install Tensorpack separately. Detectron2 will not be installed
and layout models/ table recognition models will run with Torchscript on a CPU.Full installationThe following installation will give you ALL models available within the Deep Learning framework as well as all models
that are independent of Tensorflow/PyTorch. Please note, that the dependencies are very complex. We try hard to keep
the requirements up to date though.ForTensorflow, runpip install deepdoctection[tf]ForPyTorch,first installDetectron2separately as it is not distributed via PyPi. Check the instructionhere. Then runpip install deepdoctection[pt]This will installdeepdoctection with all dependencies listed above thedeepdoctection layer. Use this setting,
if you want to get started or want to explore all features.If you want to have more control with your installation and are looking for fewer dependencies then
installdeepdoctection with the basic setup only.pip install deepdoctectionThis will ignore all model libraries (layers above thedeepdoctection layer in the diagram) and you
will be responsible to install them by yourself. Note, that you will not be able to run any pipeline with this setup.For further information, please consult thefull installation instructions.Installation from sourceDownload the repository or clone viagit clone https://github.com/deepdoctection/deepdoctection.gitTo get started withTensorflow, run:cd deepdoctection
pip install ".[tf]"Installing the fullPyTorchsetup from source will also installDetectron2for you:cd deepdoctection
pip install ".[source-pt]"Running a Docker container from Docker hubStarting from releasev.0.27.0, pre-existing Docker images can be downloaded from theDocker hub.docker pull deepdoctection/deepdoctection:<release_tag>To start the container, you can use the Docker compose file./docker/pytorch-gpu/docker-compose.yaml.
In the.envfile provided, specify the host directory wheredeepdoctection's cache should be stored.
This directory will be mounted. Additionally, specify a working directory to mount files to be processed into the
container.docker compose up -dwill start the container.CreditsWe thank all libraries that provide high quality code and pre-trained models. Without, it would have been impossible
to develop this framework.ProblemsWe try hard to eliminate bugs. We also know that the code is not free of issues. We welcome all issues relevant to this
repo and try to address them as quickly as possible. Bug fixes or enhancements will be deployed in a new release every 4
to 6 weeks.If you likedeepdoctection ......you can easily support the project by making it more visible. Leaving a star or a recommendation will help.LicenseDistributed under the Apache 2.0 License. CheckLICENSEfor additional information.