Package: genalog
Genalog - Synthetic Data GeneratorGenalog is an open source, cross-platform python package forgenerating document images with synthetic noise that mimics scanned analogdocuments (thus the namegenalog). You can also add various text degradations to these images. The purpose of this tool is to provide a fast and efficient way to generate synthetic documents from text data by leveraging layout from templates that you create in simple HTML format.OverviewGenalog has various capabilities:Flexible format Image GenerationCustom image degradationExtract Text from Images using Cognitive Search PipelineGet OCR Performance MetricsThe aim of this project is to provide a complete solution for generating synthetic images from any text data rich in natural language and to imitate most of OCR noises founded in scanned text documents.Please refer to ourGenalog documentationfor more tutorials.InstallationSee theGenalog install guidefor more details.To install the latest release:pip install genalogExtra Installation Steps in MacOs and WindowsWe have a dependency onWeasyprint, which in turn has non-python dependencies includingPango,cairoandGDK-PixBufthat need to be installed separately.So far,Pango,cairoandGDK-PixBuflibraries are available inUbuntu-18.04and later by default.If you are running on Windows, MacOS, or other Linux distributions, please seeinstallation instructions from WeasyPrint.NOTE: If you encounter the errors likeno library called "libcairo-2" was found, this is probably due to the three extra dependencies missing.Getting StartedThe following is a summary of the common applications scenarios of Genalog. Please refer theJupyter notebook examplesthat make use of the core code base of Genalog and repository utilities.TLDRIf you are interested in a full document generation and degration pipeline, please see the following notebook:DescriptionIndepth Jupyter Notebook Examples1Analog Document Generation PipelineDemo NotebookElse we have in-depth walkthroughs of each of the module in Genalog.StepsIndepth Jupyter Notebook ExamplesQuick Start Guides1Create Template for Image GenerationDemo NotebookHere is our guide to Document Generation2Degrade Prebuilt ImagesDemo NotebookHere is our guide to Image Degradation3Get Text From Images Using OCRDemo NotebookHere is our guide to Extracting Text4Align Text Produced from OCR with Ground Truth TextDemo NotebookHere is our guide to Text Alignment5NER Label Propagation from Ground Truth to OCR TokensDemo NotebookHere is our guide to Label PropagationWe also provide notebooks for the complete end-to-end scenario of generating a synthetic dataset connecting all the components of genalog:ScenarioIndepth Jupyter Notebook1Synthetic Dataset Generation with LABELED NER DatasetDemo NotebookOther Requirements:If you want to use the OCR Capabilities of Azure to Extract Text from the Images You'll require the following resources:Azure Cognitive Search ServiceQuickstart Guide HereAzure Blob StorageQuickstart Guide HereSeeAzure Docsfor more information on Azure Cognitive Search.Package ReleasePlease seeRELEASE.mdfor more details on the release process.Repo Structuregenalog
├────genalog
│       ├─── generation                      # generate text images
│       ├──── degradation                    # methods for image degradation
│       ├──── ocr                            # running the Azure Search Pipeline
│       └──── text                           # methods to Align OCR Output Text with 
├────devops                                  # CI/CD pipelines
├────docs                                    # containing online documentaions
├────examples                                # example Jupyter Notebooks for Various 
├────tests                                   # tests
├────tox.ini                                 # CI orchestration and configurations
├────README.md
└────LICENSETrademark NoticeThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft’s Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.Microsoft Open Source Code of ConductThis project has adopted theMicrosoft Open Source Code of Conduct. For more information see theCode of Conduct FAQor contactopencode@microsoft.comwith any additional questions or comments.Contribution GuidelinesThis project welcomes contributions and suggestions. Most contributions require you to
agree to a Contributor License Agreement (CLA) declaring that you have the right to,
and actually do, grant us the rights to use your contribution. For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether you need
to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the
instructions provided by the bot. You will only need to do this once across all repositories using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct.
For more information see theCode of Conduct FAQor contactopencode@microsoft.comwith any additional questions or comments.CollaboratorsGenalog was originally developed by theMAIDAP team at Microsoft Cambridge NERDin association with the Text Analytics Team in Redmond.