Package: deepparse
Here is Deepparse.Deepparse is a state-of-the-art library for parsing multinational street addresses using deep learning.Use deepparse toparse multinational address using one of our pretrained models with or without attention mechanism,parse addresses directly from the command line without code to write,parse addresses with our out-of-the-box FastAPI parser,retrain our pretrained models on new data to improve parsing on specific country address patterns,retrain our pretrained models with new prediction tags easily,retrain our pretrained models with or without freezing some layers,train a new Seq2Seq addresses parsing models easily using a new model configuration.Read the documentation atdeepparse.org.Deepparse is compatible with thelatest version of PyTorchandPython >= 3.8.Countries and ResultsWe evaluate our models on two forms of address dataclean datawhich refers to addresses containing elements from four categories, namely a street name, a
municipality, a province and a postal code,incomplete datawhich is made up of addresses missing at least one category amongst the aforementioned ones.You can get our datasethere.Clean DataThe following table presents the accuracy (using clean data) on the 20 countries we used during training for both our
models. Attention mechanisms improve performance by around 0.5% for all countries.CountryFastText (%)BPEmb (%)CountryFastText (%)BPEmb (%)Norway99.0698.3Austria99.2197.82Italy99.6598.93Mexico99.4998.9United Kingdom99.5897.62Switzerland98.998.38Germany99.7299.4Denmark99.7199.55France99.698.18Brazil99.3197.69Netherlands99.4799.54Australia99.6898.44Poland99.6499.52Czechia99.4899.03United States99.5697.69Canada99.7699.03South Korea99.9799.99Russia98.996.97Spain99.7399.4Finland99.7799.76We have also made a zero-shot evaluation of our models using clean data from 41 other countries; the results are shown
in the next table.CountryFastText (%)BPEmb (%)CountryFastText (%)BPEmb (%)Latvia89.2968.31Faroe Islands71.2264.74Colombia85.9668.09Singapore86.0367.19Réunion84.378.65Indonesia62.3863.04Japan36.2634.97Portugal93.0972.01Algeria86.3270.59Belgium93.1486.06Malaysia83.1489.64Ukraine93.3489.42Estonia87.6270.08Bangladesh72.2865.63Slovenia89.0183.96Hungary51.5237.87Bermuda83.1959.16Romania90.0482.9Philippines63.9157.36Belarus93.2578.59Bosnia88.5467.46Moldova89.2257.48Lithuania93.2869.97Paraguay96.0287.07Croatia95.881.76Argentina81.6871.2Ireland80.1654.44Kazakhstan89.0476.13Greece87.0838.95Bulgaria91.1665.76Serbia92.8776.79New Caledonia94.4594.46Sweden73.1386.85Venezuela79.2370.88New Zealand91.2575.57Iceland83.777.09India70.363.68Uzbekistan85.8570.1Cyprus89.6489.47Slovakia78.3468.96South Africa95.6874.82Moreover, we also tested the performance when using attention mechanism to further improve zero-shot performance on
those countries; the result are shown in the next table.CountryFastText (%)FastTextAtt (%)BPEmb (%)BPEmbAtt (%)CountryFastText (%)FastTextAtt (%)BPEmb (%)BPEmbAtt (%)Ireland80.1689.1154.4481.84Serbia92.8795.8876.7991.4Uzbekistan85.8587.2470.176.71Ukraine93.3494.5889.4292.65South Africa95.6897.2574.8297.95Paraguay96.0297.0887.0797.36Greece87.0886.0438.9558.79Algeria86.3287.370.5984.56Belarus93.2597.478.5997.49Sweden73.1389.2486.8593.53Portugal93.0994.9272.0193.76Hungary51.5251.0837.8724.48Iceland83.796.5477.0996.63Colombia85.9690.0868.0988.52Latvia89.2993.1468.3173.79Malaysia83.1474.6289.6491.14Bosnia88.5487.2767.4689.02India70.375.3163.6880.56Réunion84.397.7478.6594.27Croatia95.895.3281.7685.99Estonia87.6288.270.0877.32New Caledonia94.4599.6194.4699.77Japan36.2646.9134.9749.48New Zealand91.259775.5795.7Singapore86.0389.9267.1988.17Romania90.0495.3882.993.41Bangladesh72.2878.2165.6377.09Slovakia78.3482.2968.9696Argentina81.6888.5971.286.8Kazakhstan89.0492.3776.1396.08Venezuela79.2395.4770.8896.38Indonesia62.3866.8763.0471.17Bulgaria91.1691.7365.7693.28Cyprus89.6497.4489.4798.01Bermuda83.1993.2559.1693.8Moldova89.2292.0757.4889.08Slovenia89.0195.0883.9696.73Lithuania93.2887.7469.9778.67Philippines63.9181.9457.3683.42Belgium93.1490.7286.0689.85Faroe Islands71.2273.2364.7485.39Incomplete DataThe following table presents the accuracy on the 20 countries we used during training for both our models but for
incomplete data. We didn't test on the other 41 countries since we did not train on them and therefore do not expect to
achieve an interesting performance. Attention mechanisms improve performance by around 0.5% for all countries.CountryFastText (%)BPEmb (%)CountryFastText (%)BPEmb (%)Norway99.5299.75Austria99.5598.94Italy99.1698.88Mexico97.2495.93United Kingdom97.8595.2Switzerland99.299.47Germany99.4199.38Denmark97.8697.9France99.5198.49Brazil98.9697.12Netherlands98.7499.46Australia99.3498.7Poland99.4399.41Czechia98.7898.88United States98.4996.5Canada98.9696.98South Korea91.199.89Russia97.1896.01Spain99.0798.35Finland99.0499.52Getting Started:fromdeepparse.parserimportAddressParserfromdeepparse.dataset_containerimportCSVDatasetContaineraddress_parser=AddressParser(model_type="bpemb",device=0)# you can parse one addressparsed_address=address_parser("350 rue des Lilas Ouest Québec Québec G1L 1B6")# or multiple addressesparsed_address=address_parser(["350 rue des Lilas Ouest Québec Québec G1L 1B6","350 rue des Lilas Ouest Québec Québec G1L 1B6",])# or multinational addresses# Canada, US, Germany, UK and South Koreaparsed_address=address_parser(["350 rue des Lilas Ouest Québec Québec G1L 1B6","777 Brockton Avenue, Abington MA 2351","Ansgarstr. 4, Wallenhorst, 49134","221 B Baker Street","서울특별시 종로구 사직로3길 23",])# you can also get the probability of the predicted tagsparsed_address=address_parser("350 rue des Lilas Ouest Québec Québec G1L 1B6",with_prob=True)# Print the parsed addressprint(parsed_address)# or using one of our dataset containeraddresses_to_parse=CSVDatasetContainer("./a_path.csv",column_names=["address_column_name"],is_training_container=False)address_parser(addresses_to_parse)The default predictions tags are the following"StreetNumber": for the street number,"StreetName": for the name of the street,"Unit": for the unit (such as apartment),"Municipality": for the municipality,"Province": for the province or local region,"PostalCode": for the postal code,"Orientation": for the street orientation (e.g. west, east),"GeneralDelivery": for other delivery information.Parse Addresses From the Command LineYou can also use our cli to parse addresses using:parse<parsing_model><dataset_path><export_file_name>Parse Addresses Using Your Own Retrained ModelSeeherefor a complete
example.address_parser=AddressParser(model_type="bpemb",device=0,path_to_retrained_model="path/to/retrained/bpemb/model.p",)address_parser("350 rue des Lilas Ouest Québec Québec G1L 1B6")Parse Address With Our Out-Of-The-Box FastAPI Parse ModelYou can use Out-Of-The-Box RESTAPI to parse addresses:Installation:First, ensure that you have Docker Engine and Docker Compose installed on your machine.
if not, you can install them using the following documentations in the following order:Docker EngineDocker ComposeOnce you have Docker Engine and Docker Compose installed, you can run the following command to start the FastAPI application:dockercomposeupappSentry:Also, you can monitor your application usage withSentryby setting the environment variableSENTRY_DSNto your Sentry's project
DSN. There is an example of the .env file in the project's root named.env_example. You can copy it using the following command:cp.env_example.env#### Request Examples:Oncetheapplicationisupandrunningandtheport8000isexportedonyourlocalhost,youcansendrequestwithone
ofthefollowingmethods:##### cURL POST request:```shell
curl-XPOST--location"http://127.0.0.1:8000/parse/bpemb-attention"--http1.1\-H"Host: 127.0.0.1:8000"\-H"Content-Type: application/json"\-d"[{\"raw\": \"350 rue des Lilas Ouest Quebec city Quebec G1L 1B6\"},{\"raw\": \"2325 Rue de l'Université, Québec, QC G1V 0A6\"}]"Python POST request:importrequestsurl='http://localhost:8000/parse/bpemb'addresses=[{"raw":"350 rue des Lilas Ouest Quebec city Quebec G1L 1B6"},{"raw":"2325 Rue de l'Université, Québec, QC G1V 0A6"}]response=requests.post(url,json=addresses)parsed_addresses=response.json()print(parsed_addresses)Retrain a ModelSeeherefor a complete example
using Pickle
andherefor a complete example using CSV.# We will retrain the fasttext version of our pretrained model.address_parser=AddressParser(model_type="fasttext",device=0)address_parser.retrain(training_container,train_ratio=0.8,epochs=5,batch_size=8)One can also freeze some layers to speed up the training using thelayers_to_freezeparameter.address_parser.retrain(training_container,train_ratio=0.8,epochs=5,batch_size=8,layers_to_freeze="seq2seq",)Or you can also give a specific name to the retrained model. This name will be use as the model name (for print and
class name) when reloading it.address_parser.retrain(training_container,train_ratio=0.8,epochs=5,batch_size=8,name_of_the_retrain_parser="MyNewParser",)Retrain a Model With an Attention MechanismSeeherefor a complete
example.# We will retrain the fasttext version of our pretrained model.address_parser=AddressParser(model_type="fasttext",device=0,attention_mechanism=True)address_parser.retrain(training_container,train_ratio=0.8,epochs=5,batch_size=8)Retrain a Model With New TagsSeeherefor a
complete example.address_components={"ATag":0,"AnotherTag":1,"EOS":2}address_parser.retrain(training_container,train_ratio=0.8,epochs=1,batch_size=128,prediction_tags=address_components,)Retrain a Seq2Seq Model From ScratchSeeherefor
a complete example.seq2seq_params={"encoder_hidden_size":512,"decoder_hidden_size":512}address_parser.retrain(training_container,train_ratio=0.8,epochs=1,batch_size=128,seq2seq_params=seq2seq_params,)Download Our ModelsHere are the URLs to download our pretrained models directlyFastText,FastTextAttention,BPEmb,BPEmbAttention,FastText Light(
usingMagnitude Light).Or you can use our cli to download our pretrained models directly using:download_model<model_name>Starting at version 0.9.8, we will also release the weights with the GitHub release note availablehere.InstallationBefore installing deepparse, you must have the latest version ofPyTorchin your environment.Install the stable version of deepparse:pipinstalldeepparseInstall the stable version of deepparse with the app extra dependencies:pipinstalldeepparse[app]# for bash terminalpipinstall'deepparse[app]'# for ZSH terminalInstall the latest development version of deepparse:pipinstall-Ugit+https://github.com/GRAAL-Research/deepparse.git@devCiteUse the following for the article;@misc{yassine2020leveraging,
    title={{Leveraging Subword Embeddings for Multinational Address Parsing}},
    author={Marouane Yassine and David Beauchemin and François Laviolette and Luc Lamontagne},
    year={2020},
    eprint={2006.16152},
    archivePrefix={arXiv}
}and this one for the package;@misc{deepparse,
    author = {Marouane Yassine and David Beauchemin},
    title  = {{Deepparse: A State-Of-The-Art Deep Learning Multinational Addresses Parser}},
    year   = {2020},
    note   = {\url{https://deepparse.org}}
}Contributing to DeepparseWe welcome user input, whether it is regarding bugs found in the library or feature propositions ! Make sure to have a
look at ourcontributing guidelinesfor more details on this matter.LicenseDeepparse is LGPLv3 licensed, as found in
theLICENSE file.