Fighting crime with Transformers: Empirical analysis of address parsing methods in payment data (2024)

Abstract: In the financial industry, identifying the location of parties involved in payments is a major challenge in the context of Anti-Money Laundering transaction monitoring. For this purpose address parsing entails extracting fields such as street, postal code, or country from free text message attributes. While payment processing platforms are updating their standards with more structured formats such as SWIFT with ISO 20022, address parsing remains essential for a considerable volume of messages. With the emergence of Transformers and Generative Large Language Models (LLM), we explore the performance of state-of-the-art solutions given the constraint of processing a vast amount of daily data. This paper also aims to show the need for training robust models capable of dealing with real-world noisy transactional data. Our results suggest that a well fine-tuned Transformer model using early-stopping significantly outperforms other approaches. Nevertheless, generative LLMs demonstrate strong zero_shot performance and warrant further investigations.

## introduction to ensure adherence with regulatory requirements, it is essential for financial institutions to understand precisely where the money is originating and where it is flowing. the new standard of international payment messages iso 20022 for swift has the potential to simplify the task of locating payment parties by enabling the beneficiary and originator address to be delivered in a structured format. however, a considerable amount of messages are still delivered with an address in free text form. this problem is further exacerbated by the use of legacy payment processing platforms. thus address parsing is required to extract address fields such as street, postal code, city, or country. our work has three main contributions. firstly, it offers an open-sourced, augmented dataset, addressing the limitations of bench-marking on clean datasets and enabling research on noisy real-world payment data. secondly, by empirically analyzing and comparing various techniques, this paper uncovers an effective approach for multinational address parsing on distorted data. lastly, we opensource the fine-tuned state-of-the-art model, aiding future research and application in a multinational setup written in latin alphabet and transliterated in ascii format. ## related work looking through prior art, we identified various solutions to the problem at hand. early rule-based attempts @xcite were shown to be incapable of dealing with the complexities of noise in real-world data. generative approaches based on hidden markov models @xcite and probabilistic conditional random fields (crf) @xcite give the first promising results on realworld data. a notable and popular off-the-shelf solution based on the crf approach is libpostal, a c library that enables multilingual address parsing and normalization foot_1 . as it was trained on millions of multilingual, international addresses, this model shows great potential for robustness when faced with unknown addresses from different countries. therefore we consider it as the baseline solution used for our benchmarks. in recent years, advancements in natural language processing have been driven by deep learning techniques @xcite . this is evidenced by solutions like deepparse @xcite , which leverages a seq2seq bidirectional-lstm neural network architecture, and transformer-based approaches such as the one proposed by @xcite . the efficacy of transformer models has also been explored in domain-specific scenarios, as demonstrated in studies performed by @xcite and @xcite . while these existing frameworks provide valuable insights most of them have either a domain, country, or robustness limitation. our work seeks to benchmark some of these techniques on payment data and build on them to develop a solution tailored to our task while also exploring more innovative approaches. ## data the basis of our data comes from the training data of deepparse foot_2 @xcite , which is a set of postal addresses across multiple countries, with the exact number of samples per country being reported in their work. this data has been generated using data from the lipbostal project, which in itself was trained on real-world addresses from openstreetmap. each sample is provided as a tuple: the address (string) and a list of tags, one for each word in the address. a tag can be either streetname, streetnumber, unit, postalcode, municipality, or province. the dataset is comprised of two types of data: clean that have all the aforementioned tags in every sample, except for unit and postalcode being optional, and incomplete missing at least one of the other tags. we only sample from the clean data, from which we present an example in from the clean data of the original dataset, we extract at most 100k samples from each country's training and testing set and we union the extracted samples to get a total of 3.4m addresses. during sampling, we remove all addresses that are not written in the latin alphabet as this is the standard in swift messages. out of these addresses, we resample 100k rows for testing and leave the rest for training as we need a large volume of data to train the large models. then from a set of addresses from countries that were not used in training and testing sets, we create a zero-shot dataset by extracting at most 100k samples from each country. from this point onward, we will refer to these datasets as synthetic data, from which we create three versions v 0 , v 1 and v 2 as described in section 3.1. ## proposed approaches given a free text as string, the task is to parse the address fields. the possible fields should be chosen among "name", "streetname", "streetnumber", "unit", "municipality", "postalcode", "province" and "country". note that not all words need to be parsed. input: "jakob-sturm-w. 35 80995 munich bavaria" output: {"streetname": "jakob-sturm-w.", "streetnumber": "35", "postalcode": "80995", "municipality": "munich", "province": "bavaria"} ## conclusion in summary in this paper, we introduced a new open-sourced dataset reflecting the limitations and noise of real-world payment data. this will enable better benchmarking of expected production performance and further research on this problem. our experimental results highlight the importance of training robust models that are capable of dealing with noise. we achieve state-of-the-art performance on the synthetic zero-shot data and our production data with a fine-tuned xlm-roberta-large model. a derivative of this model is currently deployed in our production systems. overall even though the generative llms were not able to match the performance of the encoder transformer models, we believe there is a strong potential that warrants further inquiry. possible directions for further improvement would be the use of context-free grammars such as the lmql library @xcite to force a more structured output, and the introduction of geography knowledge with a llamaindex. in the final version of this paper, we intend to opensource all training and evaluation code, along with the fine-tuned models. by doing so, we aim to offer a valuable resource to other companies and researchers confronting analogous challenges in address parsing, applicable across various real-world applications beyond the financial industry. ↩→ the following keys: "name", "streetnumber", "streetname", "municipality", ↩→ "postalcode", "unit", "country", "countrycode". each word is separated by a space and should be classified without any ↩→ modification. each word in the input has a prefix with the index i as '[i]-' and it should be ↩→ ignored for the classification but it should remain as-is in the output. ↩→ sub sequence of words should be classified as follow: 'name': words corresponding to an indiviual name or institution name. 'streetnumber': words corresponding to a street number. 'streetname': words corresponding to a street name. 'municipality': words corresponding to a municipality or city. 'postalcode': words corresponding to a postal code. 'unit': words corresponding to a unit number. 'country': words corresponding to a full country name. 'countrycode': words corresponding to a country iso2 code. output indicator: 2. usually a name comes before the address. 3. "@xmath0 [5]-tv [6]-sd [7]-9 ↩→ [8]-japaratinga [9]-57950 [10]-000 [11]-br" ### output: {{"name": "[0]-thomassen [1]-gulbrandsen [2]-og [3]-gundersen", "streetname": ↩→ "[5]-tv [6]-sd [7]-9", "municipality": "[8]-japaratinga", "postalcode": ↩→ "[9]-57950 [10]-000", "countrycode": "[11]-br"}} ### input: {address} [/inst] ### output: """