Package: payment-text-parser
Payment text parserDescriptionInputs a text string and parses entity/address free text input to:Flag entity fields ('ORG' for companies, 'PER' for individuals, 'PER_ORG' for uncertain decision) <= Based onSpacyFlag address components of the address ('house number', 'town', 'country', ...) <= Based onLibpostalFlag other fields (i.e. not entity nor address) with POS tags ('NE', 'ADJ', 'NN', ..) <= Based onCoreNLPMore generally, the package includes following features:A data generator for entity/address fields and free text fields (based on open data)A model distinguishing entity/address field from free text (using Keras/TensorFlow and CoreNLP) to apply dedicated heuristicsA series of cleaning_postprocessing steps including true case recognition (CoreNLP)A parser of entity/address/other fields as described above using re-trained vanilla Spacy model (based on labeled open data)Simple heuristics and metrics applied after the parsing to improve accuracyPart-of-speech (POS) tagging of the remaining flags (Spacy and/or CoreNLP) for downstream processingThis package is specifically intended to be used together with the upstream Swiftflow pipeline that parses
all fields from the SWIFT MT messages, including the entity/address and free text fields, which are decisive for inter-banking
transactional communication.InstallationThe package uses essentally Libpostal and Spacy.
Also, it uses Keras on Tensorflow to recognize if the text input is a free text or an entity/address text.Pre-requiste: LibpostalRefer toLibpostal installation.
Once Libpostal is installed, the Python binderpostalwill be installed as part
of the package with pip (see below)Payment_text_parserThe other dependencies, includingSpacy, will be installed viapipon the present package:Create environmentOne recommends to use Python 3.7.Native Python:/usr/local/bin/python3 -m venv <my_env>
source <my_env>/bin/activate`Conda:conda create --name <my_env> python=3.7` conda activate <my_env>From pippip install payment-text-parser --use-feature=2020-resolver
python -m spacy download de_core_news_smFrom gitpip install git+https://gitlab.com/alpina-analytics/payment_text_parser.git
python -m spacy download de_core_news_smFrom requirements.txtgit clone https://gitlab.com/alpina-analytics/payment_text_parser.git
cd payment_text_parser
pip install -r requirements.txt
python -m spacy download de_core_news_sm
export PYTHONPATH=$(pwd)UsageScriptfrom payment_text_parser.entity_extractor.entity_extractor import ExtractorClass
e = ExtractorClass(text)
d_res = e.d_resWebserver# Launch
python main.py

# Test
curl -H "Content-type: application/json" -X POST http://127.0.0.1:5000/parse -d '{"text":"John Deere Les Abues 2 75000 Paris"}'Optional : start Stanford NLP serverRequired if :Field type detection enabled byExtractorClass(text,check_field_type=True)POS-tagging of rest fields enabled byExtractorClass(text,create_nlp_tags_rest_text=True)If not started, an warning message will be prompted, however full processing can still take place.CoreNLP server can be started as follow:cd ./core_nlp/stanford-corenlp-full-2018-10-05
java -Xmx4g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer \
-serverProperties StanfordCoreNLP-german.properties \
-preload tokenize,ssplit,pos,parse \
-status_port 9000  -port 9000 -timeout 15000ReferencesSpacyhttps://spacy.io/Libpostalhttps://github.com/openvenues/libpostalCoreNLPhttps://stackoverflow.com/questions/33259191/installing-libicu-dev-on-machttps://stackoverflow.com/questions/50217214/import-error-for-icu-in-mac-and-ubuntu-although-pyicu-is-installed-correctly/50364835#50364835https://www.khalidalnajjar.com/setup-use-stanford-corenlp-server-python/