Package: ninvoice2data
Data extractor for PDF invoices - ninvoice2data==============================================|Circle CI|This project has been selected for `GSoC2018 <https://developers.google.com/open-source/gsoc/>`__. Read more`here <https://wiki.debian.org/SummerOfCode2018/Projects/ExtractingDataFromPDFInvoicesAndBillsDetails>`__.A modular Python library to support your accounting process. Tested onPython 2.7 and 3.4+. Main steps:1. extracts text from PDF files using different techniques, like``pdftotext``, ``pdfminer`` or OCR – ``tesseract``, ``tesseract4`` or``gvision`` (Google Cloud Vision).2. searches for regex in the result using a YAML-based template system3. saves results as CSV, JSON or XML or renames PDF files to match thecontent.With the flexible template system you can:-  precisely match content PDF files-  plugins available to match line items and tables-  define static fields that are the same for every invoice-  define custom fields needed in your organisation or process-  have multiple regex per field (if layout or wording changes)-  define currency-  extract invoice-items using the ``lines``-plugin developed by `HolgerBrunn <https://github.com/hbrunn>`__Go from PDF files to this:::{'date': (2014, 5, 7), 'invoice_number': '30064443', 'amount': 34.73, 'desc': 'Invoice 30064443 from QualityHosting', 'lines': [{'price': 42.0, 'desc': u'Small Business StandardExchange 2010\nGrundgeb\xfchr pro Einheit\nDienst: OUDJQ_office\n01.05.14-31.05.14\n', 'pos': u'7', 'qty': 1.0}]}{'date': (2014, 6, 4), 'invoice_number': 'EUVINS1-OF5-DE-120725895', 'amount': 35.24, 'desc': 'Invoice EUVINS1-OF5-DE-120725895 from Amazon EU'}{'date': (2014, 8, 3), 'invoice_number': '42183017', 'amount': 4.11, 'desc': 'Invoice 42183017 from Amazon Web Services'}{'date': (2015, 1, 28), 'invoice_number': '12429647', 'amount': 101.0, 'desc': 'Invoice 12429647 from Envato'}Installation------------1. Install pdftotextIf possible get the latest`xpdf/poppler-utils <https://poppler.freedesktop.org/>`__ version. It’sincluded with macOS Homebrew, Debian and Ubuntu. Without it,``pdftotext`` won’t parse tables in PDF correctly.2. Install ``ninvoice2data`` using pip::pip install ninvoice2dataUsage-----Basic usage. Process PDF files and write result to CSV.- ``ninvoice2data invoice.pdf``- ``ninvoice2data *.pdf``Choose any of the following input readers:- pdftotext ``ninvoice2data --input-reader pdftotext invoice.pdf``- tesseract ``ninvoice2data --input-reader tesseract invoice.pdf``- pdf miner ``ninvoice2data --input-reader pdfminer invoice.pdf``- tesseract4 ``ninvoice2data --input-reader tesseract4 invoice.pdf``- gvision ``ninvoice2data --input-reader gvision invoice.pdf`` (needs ``GOOGLE_APPLICATION_CREDENTIALS`` env var)Choose any of the following output formats:- csv ``ninvoice2data --output-format csv invoice.pdf``- json ``ninvoice2data --output-format json invoice.pdf``- xml ``ninvoice2data --output-format xml invoice.pdf``Save output file with custom name or a specific folder``ninvoice2data --output-format csv --output-name myinvoices/invoices.csv invoice.pdf``**Note:** You must specify the ``output-format`` in order to create``output-name``Specify folder with yml templates. (e.g. your suppliers)``ninvoice2data --template-folder ACME-templates invoice.pdf``Only use your own templates and exclude built-ins``ninvoice2data --exclude-built-in-templates --template-folder ACME-templates invoice.pdf``Processes a folder of invoices and copies renamed invoices to newfolder. ``ninvoice2data --copy new_folder folder_with_invoices/*.pdf``Processes a single file and dumps whole file for debugging (useful whenadding new templates in templates.py)``ninvoice2data --debug my_invoice.pdf``Recognize test invoices:``ninvoice2data ninvoice2data/test/pdfs/* --debug``If you want to use it as a lib just do::from ninvoice2data import extract_dataresult = extract_data('path/to/my/file.pdf')Template system---------------See ``ninvoice2data/extract/templates`` for existing templates. Just extend thelist to add your own. If deployed by a bigger organisation, there shouldbe an interface to edit templates for new suppliers. 80-20 rule. For ashort tutorial on how to add new templates, see`TUTORIAL.rst <TUTORIAL.rst>`__.Templates are based on Yaml. They define one or more keywords to findthe right template and regexp for fields to be extracted. They couldalso be a static value, like the full company name.Template files are tried in alphabetical order.We may extend them to feature options to be used during invoiceprocessing.Example:::issuer: Amazon Web Services, Inc.keywords:- Amazon Web Servicesfields:amount: TOTAL AMOUNT DUE ON.*\$(\d+\.\d+)amount_untaxed: TOTAL AMOUNT DUE ON.*\$(\d+\.\d+)date: Invoice Date:\s+([a-zA-Z]+ \d+ , \d+)invoice_number: Invoice Number:\s+(\d+)partner_name: (Amazon Web Services, Inc\.)options:remove_whitespace: falsecurrency: HKDdate_formats:- '%d/%m/%Y'lines:start: Detailend: \* May include estimated US sales taxfirst_line: ^    (?P<description>\w+.*)\$(?P<price_unit>\d+\.\d+)line: (.*)\$(\d+\.\d+)last_line: VAT \*\*Development-----------If you are interested in improving this project, have a look at our`developer guide <DEVELOP.rst>`__ to get you started quickly.Roadmap and open tasks-----------------------  integrate with online OCR?-  try to ‘guess’ parameters for new invoice formats.-  can apply machine learning to guess new parameters?Maintainers------------  `Manuel Riel <https://github.com/m3nu>`__-  `Alexis de Lattre <https://github.com/alexis-via>`__Contributors-------------  `Harshit Joshi <https://github.com/duskybomb>`__: As Google Summer ofCode student.-  `Holger Brunn <https://github.com/hbrunn>`__: Add support for parsinginvoice items.Related Projects-----------------  `OCR-Invoice <https://github.com/robela/OCR-Invoice>`__ (FOSS \| C#)-  `Docparser <https://docparser.com/>`__ (Commercial \| Web Service)-  `A-PDF <http://www.a-pdf.com/data-extractor/index.htm>`__(Commercial)-  `PDFdeconstruct <http://www.glyphandcog.com/PDFdeconstruct.html?g6>`__(Commercial)-  `CVision <http://www.cvisiontech.com/library/document-automation/forms-processing/extract-data-from-invoice.html>`__(Commercial).. |Circle CI| image:: https://circleci.com/gh/invoice-x/ninvoice2data.svg?style=svg:target: https://circleci.com/gh/invoice-x/ninvoice2data