# Validation algorithm for aligning postal addresses available on the Internet

Mariya Evtimova

<figure><img src="image_1.png" /></figure>

## To cite this version:

Mariya Evtimova. Validation algorithm for aligning postal addresses available on the Internet. ICAMCS, Aug 2023, Paris, France. pp.75-80, 10.1109/ICAMCS59110.2023.00019.1. hal-04542517

HAL Id: hal-04542517

https://inria.hal.science/hal-04542517

Submitted on 22 Apr 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Validation algorithm for aligning postal addresses available on the Internet*

*Note: Sub-titles are not captured in Xplore and should not be used

1st Mariya Evtimova

University Paris 1 Panthéon-Sorbonne

CRI, University Paris 1 Panthéon-Sorbonne

Paris, France

0000-0002-1110-8004

**Abstract**—The internet often serves as a vast pool of information that can be utilized to retrieve personal information and locate individuals. Typically, governmental institutions in higher education provide addresses to people associated with them. However, the information available on the internet relating to personal data is not always accurate and can lead to misunderstandings, especially concerning the urgency of finding the right person at the right time.

One potential solution is the development of an algorithm for validating and verifying addresses. This algorithm could help identify outdated or incorrect address information within personal data available on the internet. This issue could be addressed by implementing an algorithm that incorporates a verification mechanism for addresses, ensuring users receive accurate information.

In this paper, it is proposed a novel algorithm for address verification utilizing ROBERTA. The paper outlines an address verification algorithm that leverages the Hugging Face Transformers Library to implement the ROBERTA model proposed in the study. This algorithm is applied to the personal address data available on the internet. The algorithm's performance is evaluated, yielding notably positive results in address validation.

**Index Terms**—ROBERTA, NLP, machine learning, address validation, postal address, internet personal information.

## I. INTRODUCTION

The proliferation of digital information on the internet is experiencing exponential growth, with numerous websites, particularly those affiliated with public institutions, disseminating personal data. Nonetheless, it's important to note that the data obtained from diverse internet sources is not always correct, [1]. An increasing number of individuals are turning to online resources to access pertinent address-related information, as the internet's unhindered accessibility facilitates swift communication, [2]. However, this accessibility carries the inherent risk of encountering outdated information, potentially leading to misinterpretations and severed connections with intended recipients, [3]. Thus, the accurate acquisition of address information stands as pivotal for effective communication.

It's crucial to emphasize that the composition of postal address entries displays greater diversity compared to conven-

tional descriptors, [4], [5]. Conversely, not all components of an address are essential for practical use. For instance, the provided address above notably excludes the "Postal code" entry. Similarly, instances exist where suite details are absent. A significant challenge in verifying internet-sourced addresses pertains to the intricacies of vocabulary usage. Language nuances can breed semantic confusion due to the presence of synonyms (words with identical meanings) and polysemy (a single word with divergent meanings).

As an example, the term "Avenue" might be abbreviated variably as "Av.," "av," or "Ave." Further exemplifying polysemy, the street name "rue de Tivoli" is present in both Marseilles and Paris, illustrating this phenomenon, [6].

It's pertinent to acknowledge that the order of address components differs among various institutions. Notably, certain establishments prioritize the inclusion of street numbers before street names, while others position house addresses after the street name.

Complications arising from erroneous address databases, characterized by duplicate entries and inconsistent variants, give rise to a complex landscape where accurate data retrieval becomes both arduous and unreliable. The data classification encompassing names and addresses is underscored by idiosyncratic attributes that contribute to distinct complexities in their management. This data domain is particularly susceptible to volatility due to the dynamic nature of institutional affiliations, address changes, and name modifications. Moreover, data input for names and addresses frequently exhibits cluttered tendencies, as front-end interfaces allow free-form entry, often incorporating comments and additional data without validation.

Further complicating matters is the subjective nature inherent in the representation of names and addresses. Individuals possess the liberty to express identical entities in varied ways, despite referring to the same entity. Unfortunately, a universally accepted standard or framework that could encapsulate name and address data while simultaneously evaluating its quality remains conspicuously absent. This challenge is compounded by the intricate interplay of cultural contexts within France, which significantly influence the interpretation
# Validation algorithm for aligning postal addresses available on the Internet*

*Note: Sub-titles are not captured in Xplore and should not be used

1st Mariya Evtimova

University Paris 1 Panthéon-Sorbonne

CRI, University Paris 1 Panthéon-Sorbonne

Paris, France

0000-0002-1110-8004

**Abstract**—The internet often serves as a vast pool of information that can be utilized to retrieve personal information and locate individuals. Typically, governmental institutions in higher education provide addresses to people associated with them. However, the information available on the internet relating to personal data is not always accurate and can lead to misunderstandings, especially concerning the urgency of finding the right person at the right time.

One potential solution is the development of an algorithm for validating and verifying addresses. This algorithm could help identify outdated or incorrect address information within personal data available on the internet. This issue could be addressed by implementing an algorithm that incorporates a verification mechanism for addresses, ensuring users receive accurate information.

In this paper, it is proposed a novel algorithm for address verification utilizing ROBERTA. The paper outlines an address verification algorithm that leverages the Hugging Face Transformers Library to implement the ROBERTA model proposed in the study. This algorithm is applied to the personal address data available on the internet. The algorithm's performance is evaluated, yielding notably positive results in address validation.

**Index Terms**—ROBERTA, NLP, machine learning, address validation, postal address, internet personal information.

## I. INTRODUCTION

The proliferation of digital information on the internet is experiencing exponential growth, with numerous websites, particularly those affiliated with public institutions, disseminating personal data. Nonetheless, it's important to note that the data obtained from diverse internet sources is not always correct, [1]. An increasing number of individuals are turning to online resources to access pertinent address-related information, as the internet's unhindered accessibility facilitates swift communication, [2]. However, this accessibility carries the inherent risk of encountering outdated information, potentially leading to misinterpretations and severed connections with intended recipients, [3]. Thus, the accurate acquisition of address information stands as pivotal for effective communication.

It's crucial to emphasize that the composition of postal address entries displays greater diversity compared to conven-

tional descriptors, [4], [5]. Conversely, not all components of an address are essential for practical use. For instance, the provided address above notably excludes the "Postal code" entry. Similarly, instances exist where suite details are absent. A significant challenge in verifying internet-sourced addresses pertains to the intricacies of vocabulary usage. Language nuances can breed semantic confusion due to the presence of synonyms (words with identical meanings) and polysemy (a single word with divergent meanings).

As an example, the term "Avenue" might be abbreviated variably as "Av.," "av," or "Ave." Further exemplifying polysemy, the street name "rue de Tivoli" is present in both Marseilles and Paris, illustrating this phenomenon, [6].

It's pertinent to acknowledge that the order of address components differs among various institutions. Notably, certain establishments prioritize the inclusion of street numbers before street names, while others position house addresses after the street name.

Complications arising from erroneous address databases, characterized by duplicate entries and inconsistent variants, give rise to a complex landscape where accurate data retrieval becomes both arduous and unreliable. The data classification encompassing names and addresses is underscored by idiosyncratic attributes that contribute to distinct complexities in their management. This data domain is particularly susceptible to volatility due to the dynamic nature of institutional affiliations, address changes, and name modifications. Moreover, data input for names and addresses frequently exhibits cluttered tendencies, as front-end interfaces allow free-form entry, often incorporating comments and additional data without validation.

Further complicating matters is the subjective nature inherent in the representation of names and addresses. Individuals possess the liberty to express identical entities in varied ways, despite referring to the same entity. Unfortunately, a universally accepted standard or framework that could encapsulate name and address data while simultaneously evaluating its quality remains conspicuously absent. This challenge is compounded by the intricate interplay of cultural contexts within France, which significantly influence the interpretation
and management of name and address data.

## II. RELATED WORK

### A. In the section are presented recent relative research of ROBERTA language model applied in address verification.

In the study conducted by, [7], a deep learning methodology is introduced to enhance the quality of global address data utilized for imported food-safety management. The proposed approach involves the classification of user-input addresses into administrative divisions specific to the respective country. By transforming the addresses into standardized formats, the quality of the address data is assessed and enhanced. Within the context of research, [8], a filtering approach named Distill-AER has been put forth. This technique is designed to facilitate the transformation of knowledge extracted from a well-populated labeled dataset of standard addresses in the realm of Big Data. The objective is to adapt this knowledge for the targeted task of recognizing entities within addresses that hold a specific significance. To facilitate this transfer, a labeled spoken dialogue dataset containing address entities is constructed through the utilization of the data augmentation paradigm. [9] introduce a two-stage address validation methodology that incorporates standardization and classification steps, both leveraging the ROBERTA pre-trained language model. The proposed approach is evaluated through experiments conducted on real datasets, showcasing its effectiveness and reliability.

### B. Other machine learning techniques applied for address validation

[10] present HyPASS, a software detailed in their study, which encompasses a hybrid approach combining Software-Defined Networking (SDN) principles with host discovery and address validation techniques. The primary objective of HyPASS is to mitigate source spoofing attacks by enhancing network security measures. The paper by, [11] introduces an automated probabilistic method, based on a hidden Markov model (HMM), that utilizes national address guidelines and an extensive national address database. This approach aims to process raw input addresses by performing cleaning, standardization, and verification tasks. [12] introduce a novel robust architecture called DeepParse in their paper, which is specifically designed for postal address parsing. This architecture implements address parsing and also reflect Named Entity Recognition (NER) problems. DeepParse treats input data at various levels such as characters, trigram characters, and words, to extract features and carry out address validation. The model was trained using a synthetically generated dataset and subsequently tested with real addresses. In the research carried out by, [13], an application of one-dimensional transformation in CNN (Convolutional Neural Networks) is utilized for address parsing. The results obtained from the evaluation demonstrate a high accuracy for a labeled dataset comprising nearly 20,000 samples. Notably, the proposed network architecture possesses a scalable nature, eliminating the need for any pre- or post-processing stages. Erickson et al., [14] introduce a tool

designed for the annotation of electronic health records. The research focuses on training random forests to identify patients who are experiencing homelessness. To validate the efficacy of each model, a 10-fold cross-validation technique is employed.

## III. STANDARDS FOR POSTAL ADDRESS IN FRANCE

### A. French standard overview

In France, postal addresses follow a specific format. Here is the standard structure for a postal address in France:

* Recipient's Name: Full name of the person or organization the mail is addressed to.

* Building Number and Street Name: The building number comes before the street name. For example, "12 Rue de la Paix" (12, Peace Street).

* Postal Code and City: The five-digit postal code comes before the name of the city. For example, "75001 Paris" (75001 being the postal code for the 1st arrondissement of Paris).

* Cedex (Optional): If the recipient's address includes a Cedex (Courrier d'Entreprise à Distribution Exceptionnelle) number, it should be included on a separate line below the postal code and city. Cedex is used for mail addressed to specific companies, organizations, or government agencies that have their own distribution system. For example, "Cedex 2" or "CEDEX 2".

* Country (Not required for domestic mail) If the mail is being sent from outside of France, the country name should be included in uppercase letters at the last line of the address. For domestic mail within France, the country name is not necessary.

Here's an example of a postal address in France:

Monsieur Jean Dupont
12 Rue de la Paix
75001 Paris
France

It's important to note that the exact format may vary slightly depending on the region or specific requirements of the local postal service, [15]. It is prudent to consistently cross-validate prevailing norms and directives set forth by La Poste, the national postal service of France. Alternatively, one may opt to engage in dialogue with the intended recipient to ascertain any supplementary details that could potentially be deemed requisite.

### B. Proposal of address model

Figure 1 introduces an address denoted as $Adr = \{adr_1, ..., adr_n\}$, wherein adr signifies a collection of addresses adrᵢ, and 'i' denotes a word, while 'n' signifies the address's length. The primary objective of parsing Adr is to allocate a label 'i' to each word adrᵢ within Adr from the corresponding set of address tags denoted as T; $T = \{P, C, CO, SR, BN, CE, SN, PB\}$.

These tags find their definitions in the address model depicted
and management of name and address data.

## II. RELATED WORK

### A. In the section are presented recent relative research of ROBERTA language model applied in address verification.

In the study conducted by, [7], a deep learning methodology is introduced to enhance the quality of global address data utilized for imported food-safety management. The proposed approach involves the classification of user-input addresses into administrative divisions specific to the respective country. By transforming the addresses into standardized formats, the quality of the address data is assessed and enhanced. Within the context of research, [8], a filtering approach named Distill-AER has been put forth. This technique is designed to facilitate the transformation of knowledge extracted from a well-populated labeled dataset of standard addresses in the realm of Big Data. The objective is to adapt this knowledge for the targeted task of recognizing entities within addresses that hold a specific significance. To facilitate this transfer, a labeled spoken dialogue dataset containing address entities is constructed through the utilization of the data augmentation paradigm. [9] introduce a two-stage address validation methodology that incorporates standardization and classification steps, both leveraging the ROBERTA pre-trained language model. The proposed approach is evaluated through experiments conducted on real datasets, showcasing its effectiveness and reliability.

### B. Other machine learning techniques applied for address validation

[10] present HyPASS, a software detailed in their study, which encompasses a hybrid approach combining Software-Defined Networking (SDN) principles with host discovery and address validation techniques. The primary objective of HyPASS is to mitigate source spoofing attacks by enhancing network security measures. The paper by, [11] introduces an automated probabilistic method, based on a hidden Markov model (HMM), that utilizes national address guidelines and an extensive national address database. This approach aims to process raw input addresses by performing cleaning, standardization, and verification tasks. [12] introduce a novel robust architecture called DeepParse in their paper, which is specifically designed for postal address parsing. This architecture implements address parsing and also reflect Named Entity Recognition (NER) problems. DeepParse treats input data at various levels such as characters, trigram characters, and words, to extract features and carry out address validation. The model was trained using a synthetically generated dataset and subsequently tested with real addresses. In the research carried out by, [13], an application of one-dimensional transformation in CNN (Convolutional Neural Networks) is utilized for address parsing. The results obtained from the evaluation demonstrate a high accuracy for a labeled dataset comprising nearly 20,000 samples. Notably, the proposed network architecture possesses a scalable nature, eliminating the need for any pre- or post-processing stages. Erickson et al., [14] introduce a tool

designed for the annotation of electronic health records. The research focuses on training random forests to identify patients who are experiencing homelessness. To validate the efficacy of each model, a 10-fold cross-validation technique is employed.

## III. STANDARDS FOR POSTAL ADDRESS IN FRANCE

### A. French standard overview

In France, postal addresses follow a specific format. Here is the standard structure for a postal address in France:

* Recipient's Name: Full name of the person or organization the mail is addressed to.

* Building Number and Street Name: The building number comes before the street name. For example, "12 Rue de la Paix" (12, Peace Street).

* Postal Code and City: The five-digit postal code comes before the name of the city. For example, "75001 Paris" (75001 being the postal code for the 1st arrondissement of Paris).

* Cedex (Optional): If the recipient's address includes a Cedex (Courrier d'Entreprise à Distribution Exceptionnelle) number, it should be included on a separate line below the postal code and city. Cedex is used for mail addressed to specific companies, organizations, or government agencies that have their own distribution system. For example, "Cedex 2" or "CEDEX 2".

* Country (Not required for domestic mail) If the mail is being sent from outside of France, the country name should be included in uppercase letters at the last line of the address. For domestic mail within France, the country name is not necessary.

Here's an example of a postal address in France:

Monsieur Jean Dupont
12 Rue de la Paix
75001 Paris
France

It's important to note that the exact format may vary slightly depending on the region or specific requirements of the local postal service, [15]. It is prudent to consistently cross-validate prevailing norms and directives set forth by La Poste, the national postal service of France. Alternatively, one may opt to engage in dialogue with the intended recipient to ascertain any supplementary details that could potentially be deemed requisite.

### B. Proposal of address model

Figure 1 introduces an address denoted as $Adr = \{adr_1, ..., adr_n\}$, wherein adr signifies a collection of addresses adr$_i$, and 'i' denotes a word, while 'n' signifies the address's length. The primary objective of parsing Adr is to allocate a label 'i' to each word adr$_i$ within Adr from the corresponding set of address tags denoted as T; $T = \{P, C, CO, SR, BN, CE, SN, PB\}$.

These tags find their definitions in the address model depicted
in Figure 1, whereby the address tags symbolize composite
attributes.

<figure><img src="image_1.png" /><figcaption>Fig. 1. Address model</figcaption></figure>

IV. APPLICATION/SERVICES THAT COULD BE INTEGRATED
FOR VERIFICATION OF POSTAL ADDRESS

There are several applications and services that can be
integrated to verify postal addresses in France. A list with
the most well-known address services:

* La Poste API: La Poste, the French postal service, provides an API called "La Poste Data" that allows you to validate and standardize postal addresses. This API can help verify the accuracy of addresses, validate postal codes, and retrieve additional information about addresses.

* Google Maps Geocoding API: Google Maps Geocoding API can be used to geocode and verify addresses in France. It provides geocoding services that can convert addresses into geographic coordinates and verify the existence of a particular address.

* SmartyStreets: SmartyStreets is a widely used address verification service that offers international coverage, including France. It provides address validation, standardization, and geocoding features. Their API can be integrated into applications to ensure accurate and up-to-date address verification.

* Experian Data Quality: Experian Data Quality offers address verification solutions that can be integrated into applications. Their address validation services include features like address standardization, postal code validation, and geocoding.

* Loqate: Loqate is a global address verification and geocoding service that supports address validation in France. It offers features such as address autocomplete, standardization, and geocoding to ensure accurate address verification.

These are just a few examples of services that can be integrated to verify postal addresses in France. And each address service provided above have own pricing structure and specific features, so it's advisable to explore their documentation before further processing with integration and use.

V. DESCRIPTION OF THE ALGORITHM FOR AUTOMATIC DETECTION OF POSTAL ADDRESSES USING PRE- TRAINED MODEL

A. Process flow for address validation

In Fig.2 is defined a process flow for address validation and
verification. The approach need pre- trained model specifically
designed for address validation tasks. Address verification

<figure><img src="image_2.png" /><figcaption>Fig. 2. Process flow for address validation using ROBERTA</figcaption></figure>

stages using ROBERTA model.

* Dataset Preparation: Prepare a dataset of labeled addresses for training and evaluation. Each address should have a corresponding label indicating whether it is a valid or invalid address.

* Model Training: The ROBERTA model can undergo training using a pre-established dataset. Subsequently, the pretrained ROBERTA model can be further improved through techniques such as transfer learning when tasked with address validation.

In the pre-training phase, a specific proportion of input tokens is subjected to random masking, and the model is trained to predict these masked tokens based on their contextual context. This particular objective incentivizes the model to acquire bidirectional representations and adeptly capture contextual dependencies.

* Data Preprocessing: Prepare the input address data for compatibility with the ROBERTA model by subjecting it to a preprocessing stage that involves tokenization into numerical representations. This process encompasses tasks such as transforming text into subword tokens, incorporating specialized tokens like [CLS] and [SEP], and adjusting sequence lengths through padding or truncation to reach a predefined size.

The objective of this pre- processing step is two fold: to
standardize entities, [16] and cleanse addresses by elim-
inating special characters and rectifying various spelling
errors. To accomplish this, a dictionary-driven approach
is employed, which offers keywords for defining address
components (such as city, road, etc.) as well as commonly
used abbreviations for these terms. Additionally, a spell
checker named "pyap" <fnref n="1" /> and "pyspellcheck" <fnref n="2" /> is utilized
to further enhance address accuracy.

* Model Inference: Employ the trained ROBERTA model to assess the validity of an address. Direct the preprocessed

<footnote n="1">¹https://pypi.org/project/pyap/</footnote>

<footnote n="2">²https://pypi.org/project/pyspellchecker/</footnote>
in Figure 1, whereby the address tags symbolize composite
attributes.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Address model</figcaption></figure>

IV. APPLICATION/SERVICES THAT COULD BE INTEGRATED
FOR VERIFICATION OF POSTAL ADDRESS

There are several applications and services that can be
integrated to verify postal addresses in France. A list with
the most well-known address services:

* La Poste API: La Poste, the French postal service, provides an API called "La Poste Data" that allows you to validate and standardize postal addresses. This API can help verify the accuracy of addresses, validate postal codes, and retrieve additional information about addresses.

* Google Maps Geocoding API: Google Maps Geocoding API can be used to geocode and verify addresses in France. It provides geocoding services that can convert addresses into geographic coordinates and verify the existence of a particular address.

* SmartyStreets: SmartyStreets is a widely used address verification service that offers international coverage, including France. It provides address validation, standardization, and geocoding features. Their API can be integrated into applications to ensure accurate and up-to-date address verification.

* Experian Data Quality: Experian Data Quality offers address verification solutions that can be integrated into applications. Their address validation services include features like address standardization, postal code validation, and geocoding.

* Loqate: Loqate is a global address verification and geocoding service that supports address validation in France. It offers features such as address autocomplete, standardization, and geocoding to ensure accurate address verification.

These are just a few examples of services that can be integrated to verify postal addresses in France. And each address service provided above have own pricing structure and specific features, so it's advisable to explore their documentation before further processing with integration and use.

V. DESCRIPTION OF THE ALGORITHM FOR AUTOMATIC DETECTION OF POSTAL ADDRESSES USING PRE- TRAINED MODEL

A. Process flow for address validation

In Fig.2 is defined a process flow for address validation and
verification. The approach need pre- trained model specifically
designed for address validation tasks. Address verification

<figure><img src="image_4.png" /><figcaption>Fig. 2. Process flow for address validation using ROBERTA</figcaption></figure>

stages using ROBERTA model.

* Dataset Preparation: Prepare a dataset of labeled addresses for training and evaluation. Each address should have a corresponding label indicating whether it is a valid or invalid address.

* Model Training: The ROBERTA model can undergo training using a pre-established dataset. Subsequently, the pretrained ROBERTA model can be further improved through techniques such as transfer learning when tasked with address validation.

In the pre-training phase, a specific proportion of input tokens is subjected to random masking, and the model is trained to predict these masked tokens based on their contextual context. This particular objective incentivizes the model to acquire bidirectional representations and adeptly capture contextual dependencies.

* Data Preprocessing: Prepare the input address data for compatibility with the ROBERTA model by subjecting it to a preprocessing stage that involves tokenization into numerical representations. This process encompasses tasks such as transforming text into subword tokens, incorporating specialized tokens like [CLS] and [SEP], and adjusting sequence lengths through padding or truncation to reach a predefined size.

The objective of this pre- processing step is two fold: to
standardize entities, [16] and cleanse addresses by elim-
inating special characters and rectifying various spelling
errors. To accomplish this, a dictionary-driven approach
is employed, which offers keywords for defining address
components (such as city, road, etc.) as well as commonly
used abbreviations for these terms. Additionally, a spell
checker named "pyap" <fnref n="1" /> and "pyspellcheck" <fnref n="2" /> is utilized
to further enhance address accuracy.

* Model Inference: Employ the trained ROBERTA model to assess the validity of an address. Direct the preprocessed

<footnote n="1">¹https://pypi.org/project/pyap/</footnote>

<footnote n="2">²https://pypi.org/project/pyspellchecker/</footnote>
address data through the model and record the model's prediction.

I propose a parsing technique (described in Fig. 2) that involves two primary stages in generating a contextual representation of an address, denoted as Adr, using the pre-trained ROBERTA model on a collection of addresses. These stages are as follows:

1) ROBERTA computes the input representations for Adr by summing token, position, and segment embeddings. Token embeddings for each token are generated using a byte-level BPE (Byte-Pair Encoding) tokenizer. Position embeddings encompass positional information for each token within the address. Segment embeddings assign the same label to tokens within the same address.

2) The input address representation is subsequently processed by 12 transformer encoders, which capture contextual information for each token through self-attention mechanisms. This process yields a sequence of contextual embeddings, denoted as S.

*   **Post-processing:** Apply post-processing methods to interpret the model's prediction and translate it into a format comprehensible to humans, such as assigning a "valid" or "invalid" label.

Subsequently, the resulting representation is directed to a tagging layer to acquire address tags, utilizing the IOB (Inside, Outside, Beginning) tagging scheme. A linear layer is employed, taking the last hidden state of the sequence, denoted as $H = \{h_1, ..., h_n\}$, as input and providing the tag predictions, noted as T.

I propose a classification method based on ROBERTA (as described in Fig. 3) for categorizing addresses as either "Valid" or "Invalid." This method comprises two key steps to fusion the two vector representations using concatenation function:

1) Contextual vector representation of the address: Contextual address embedding: Retrieve the contextual vector representations (H) of the address (Adr), generated during the pre-trained ROBERTA model's address parsing step.

2) Vector representation of the address tags: Address tags embeddings: The address parsing step yields n tags, denoted as $T = \{t_1, ..., t_n\}$. Since these tags are word-level, their length aligns with the address Adr's length (n). A lookup table is used to map these tags to $\{id_1, ..., id_n\}$, followed by a linear layer (fully connected layer) to obtain the tag embeddings, represented as $W = \{w_1, w_2, ..., w_n\}$.

Then a classification of addresses according to resulted vectors is applied.

Address classification is executed via a linear layer (fully connected layer). Initially, this layer takes the embedding fusion vector as input, generating class logits (probabilities) as output. It's essential to note that the training objective function is CrossEntropy. Subsequently,

the Argmax function is applied to these probabilities to determine the predicted class.

*   **Evaluation and Fine-tuning:** Evaluate the performance of the model using evaluation metrics such as accuracy, precision, recall, and F1 score. Based on the evaluation results, the model can be fine-tuned by iterating on the training and evaluation steps to improve its performance. Additionally, fine-tuning the model on a labeled address dataset specific to your validation task can further improve the accuracy and performance. Fine-tune ROBERTA for NER (Named Entity Recognition) tasks by adding a token classification layer on top of the base model. The token classification layer assigns labels to individual tokens in the input text, identifying named entities such as street number, name of the street and others.

It's important to note that implementing and training a model like ROBERTA for address validation can be a complex task that requires domain-specific data, computational resources, and expertise in natural language processing and machine learning. Additionally, it's crucial to have a diverse and representative dataset that covers different address formats and variations.

In the proposed research model for the pre-trained address validation models that is integrated in the application it is used the french national datasets <fnref n="3" /> <fnref n="4" /> that use ROBERTA for address validation.

<figure><img src="image_3.png" /><figcaption>Fig. 3. ROBERTA model for address validation</figcaption></figure>

## VI. EXPERIMENTS AND RESULTS

The last phase of the model defined in Fig.2 is described in this section. Currently, the BAN contains 25 million addresses across France. The parsing and classification of the dataset were conducted using two real-world datasets. The first dataset is the BAN, which comprises millions of structured addresses

<footnote n="3">³https://adresse.data.gouv.fr/donnees-nationalesandorspecific libraries</footnote>

<footnote n="4">⁴https://shorturl.at/agjyB</footnote>
address data through the model and record the model's prediction.

I propose a parsing technique (described in Fig. 2) that involves two primary stages in generating a contextual representation of an address, denoted as Adr, using the pre-trained ROBERTA model on a collection of addresses. These stages are as follows:

1) ROBERTA computes the input representations for Adr by summing token, position, and segment embeddings. Token embeddings for each token are generated using a byte-level BPE (Byte-Pair Encoding) tokenizer. Position embeddings encompass positional information for each token within the address. Segment embeddings assign the same label to tokens within the same address.

2) The input address representation is subsequently processed by 12 transformer encoders, which capture contextual information for each token through self-attention mechanisms. This process yields a sequence of contextual embeddings, denoted as S.

*   **Post-processing:** Apply post-processing methods to interpret the model's prediction and translate it into a format comprehensible to humans, such as assigning a "valid" or "invalid" label.

Subsequently, the resulting representation is directed to a tagging layer to acquire address tags, utilizing the IOB (Inside, Outside, Beginning) tagging scheme. A linear layer is employed, taking the last hidden state of the sequence, denoted as $H = \{h_1, ..., h_n\}$, as input and providing the tag predictions, noted as T.

I propose a classification method based on ROBERTA (as described in Fig. 3) for categorizing addresses as either "Valid" or "Invalid." This method comprises two key steps to fusion the two vector representations using concatenation function:

1) Contextual vector representation of the address: Contextual address embedding: Retrieve the contextual vector representations (H) of the address (Adr), generated during the pre-trained ROBERTA model's address parsing step.

2) Vector representation of the address tags: Address tags embeddings: The address parsing step yields n tags, denoted as $T = \{t_1, ..., t_n\}$. Since these tags are word-level, their length aligns with the address Adr's length (n). A lookup table is used to map these tags to $\{id_1, ..., id_n\}$, followed by a linear layer (fully connected layer) to obtain the tag embeddings, represented as $W = \{w_1, w_2, ..., w_n\}$.

Then a classification of addresses according to resulted vectors is applied.

Address classification is executed via a linear layer (fully connected layer). Initially, this layer takes the embedding fusion vector as input, generating class logits (probabilities) as output. It's essential to note that the training objective function is CrossEntropy. Subsequently,

the Argmax function is applied to these probabilities to determine the predicted class.

*   **Evaluation and Fine-tuning:** Evaluate the performance of the model using evaluation metrics such as accuracy, precision, recall, and F1 score. Based on the evaluation results, the model can be fine-tuned by iterating on the training and evaluation steps to improve its performance. Additionally, fine-tuning the model on a labeled address dataset specific to your validation task can further improve the accuracy and performance. Fine-tune ROBERTA for NER (Named Entity Recognition) tasks by adding a token classification layer on top of the base model. The token classification layer assigns labels to individual tokens in the input text, identifying named entities such as street number, name of the street and others.

It's important to note that implementing and training a model like ROBERTA for address validation can be a complex task that requires domain-specific data, computational resources, and expertise in natural language processing and machine learning. Additionally, it's crucial to have a diverse and representative dataset that covers different address formats and variations.

In the proposed research model for the pre-trained address validation models that is integrated in the application it is used the french national datasets <fnref n="3" /> <fnref n="4" /> that use ROBERTA for address validation.

<figure><img src="image_5.png" /><figcaption>Fig. 3. ROBERTA model for address validation</figcaption></figure>

## VI. EXPERIMENTS AND RESULTS

The last phase of the model defined in Fig.2 is described in this section. Currently, the BAN contains 25 million addresses across France. The parsing and classification of the dataset were conducted using two real-world datasets. The first dataset is the BAN, which comprises millions of structured addresses

<footnote n="3">³https://adresse.data.gouv.fr/donnees-nationalesandorspecific libraries</footnote>

<footnote n="4">⁴https://shorturl.at/agjyB</footnote>
extracted from the French database. The second dataset is a
collection of 3,683 structured addresses from a French higher
education database.

For the evaluation setup, ROBERTA utilizes GELU activa-
tion and the ADAM Optimizer. A dropout rate of 0.2 and a
learning rate of 0.001 are set to maximize accuracy. To prevent
overfitting, L1 regularization technique and early stopping are
employed.

The pretraining of ROBERTA is conducted within the PyTorch framework. Two pretrained ROBERTA models are created, with each model being associated with one of the following datasets:

1) The French BAN corpus, comprising 25 million ad-
dresses extracted from the database.

2) The French higher education dataset, which encom-
passes a collection of 3,683 addresses.

<figure><img src="image_4.png" /><figcaption>Fig. 4. Evaluation of the datasets</figcaption></figure>

From the data provided from the evaluation represented in Fig.4 is able to conclude that the results when using the Dataset 2 is better. In Fig.5 is presented an evaluation with different number of Transformers when using Dataset 2. In

<figure><img src="image_5.png" /><figcaption>Fig. 5. Evaluation of the Dataset 2 with different number of Transformers</figcaption></figure>

Fig.5 the results show that 8 Transformers is enough for
Dataset 2. In Tab.1 are presented different research approaches
and their level of accuracy.

<table><caption>TABLE I
COMPARING THE APPROACH WITH ANOTHER APPROACHES</caption>
  <thead>
    <tr>
      <th rowspan="2">N</th>
      <th colspan="2">Table comparing approaches</th>
    </tr>
    <tr>
      <th>Approach</th>
      <th>Accuracy</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>1</td>
      <td>Proposed approach</td>
      <td>98,2%</td>
    </tr>
    <tr>
      <td>2</td>
      <td>NER [9]</td>
      <td>90,44%</td>
    </tr>
    <tr>
      <td>3</td>
      <td>CNN [10]</td>
      <td>96%</td>
    </tr>
  </tbody>
</table>

The increasing number of internet sites containing personal
information and addresses is constantly growing on the inter-
net, and this data needs to be synchronized and updated, [17].
The websites providing personal information are typically
public institutions' websites. This information is generally
used by everyone to find addresses (for writing letters or
locating office coordinates).

These sites change their content frequently, and the information needs to be updated daily to ensure the online information is accurate, [18], [19]. This has led to the development of an algorithm for address verification using ROBERTA, which can be employed to align data from various internet sources. The paper describes an algorithm for address verification using the Hugging Face Transformers Library for the implementation of the proposed ROBERTA model, [20], [21], [22]. The proposed algorithm achieves a 98.2% accuracy rate and a relatively high F-measure in comparison to the other algorithms applied in address validation, [23]. Further evaluation needs to be conducted using a different number of Transformers on the initial dataset.

ACKNOWLEDGMENT

I want to express special thanks to J. D. for this article. I
want to thanks also to the support from M. organization.

REFERENCES

[1] Cai, Wentao, Shengrui Wang, and Qingshan Jiang. "Address extraction: Extraction of location-based information from the web." Web Technologies Research and Development-APWeb 2005: 7th Asia-Pacific Web Conference, Shanghai, China, March 29-April 1, 2005. Proceedings 7. Springer Berlin Heidelberg, 2005.

[2] Fedushko, Solomia, and Yuriy Syerov. "Design of registration and validation algorithm of member's personal data." International Journal of Informatics and Communication Technology 2.2 (2013): 93-98.

[3] Dakrory, Sara, et al. "Extracting geographic addresses from social media using deep recurrent neural networks." 2021 9th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC). IEEE, 2021.

[4] Beverly, Robert, et al. "Understanding the efficacy of deployed internet source address validation filtering." Proceedings of the 9th ACM SIGCOMM conference on Internet measurement. 2009.

[5] Nagabhushan, P., Shanmukhappa A. Angadi, and Basavaraj S. Anami. "Symbolic data structure for postal address representation and address validation through symbolic knowledge base." Pattern Recognition and Machine Intelligence: First International Conference, PReMI 2005, Kolkata, India, December 20-22, 2005. Proceedings 1. Springer Berlin Heidelberg, 2005.

[6] U.S. POSTAL SERVICE FACILITIES: Improvements in Data Would Strengthen Maintenance and Alignment of Access to Retail Services. GAO Reports. December 2007:i-61. Accessed August 29, 2023.

[7] Soeng, Saravit, et al. "Deep Learning Based Improvement in Overseas Manufacturer Address Quality Using Administrative District Data." Applied Sciences 12.21 (2022): 11129.
extracted from the French database. The second dataset is a
collection of 3,683 structured addresses from a French higher
education database.

For the evaluation setup, ROBERTA utilizes GELU activa-
tion and the ADAM Optimizer. A dropout rate of 0.2 and a
learning rate of 0.001 are set to maximize accuracy. To prevent
overfitting, L1 regularization technique and early stopping are
employed.

The pretraining of ROBERTA is conducted within the PyTorch framework. Two pretrained ROBERTA models are created, with each model being associated with one of the following datasets:

1) The French BAN corpus, comprising 25 million ad-
dresses extracted from the database.

2) The French higher education dataset, which encom-
passes a collection of 3,683 addresses.

<figure><img src="image_6.png" /><figcaption>Fig. 4. Evaluation of the datasets</figcaption></figure>

From the data provided from the evaluation represented in Fig.4 is able to conclude that the results when using the Dataset 2 is better. In Fig.5 is presented an evaluation with different number of Transformers when using Dataset 2. In

<figure><img src="image_7.png" /><figcaption>Fig. 5. Evaluation of the Dataset 2 with different number of Transformers</figcaption></figure>

Fig.5 the results show that 8 Transformers is enough for
Dataset 2. In Tab.1 are presented different research approaches
and their level of accuracy.

<table><caption>TABLE I
COMPARING THE APPROACH WITH ANOTHER APPROACHES</caption>
  <thead>
    <tr>
      <th rowspan="2">N</th>
      <th colspan="2">Table comparing approaches</th>
    </tr>
    <tr>
      <th>Approach</th>
      <th>Accuracy</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>1</td>
      <td>Proposed approach</td>
      <td>98,2%</td>
    </tr>
    <tr>
      <td>2</td>
      <td>NER [9]</td>
      <td>90,44%</td>
    </tr>
    <tr>
      <td>3</td>
      <td>CNN [10]</td>
      <td>96%</td>
    </tr>
  </tbody>
</table>

The increasing number of internet sites containing personal
information and addresses is constantly growing on the inter-
net, and this data needs to be synchronized and updated, [17].
The websites providing personal information are typically
public institutions' websites. This information is generally
used by everyone to find addresses (for writing letters or
locating office coordinates).

These sites change their content frequently, and the information needs to be updated daily to ensure the online information is accurate, [18], [19]. This has led to the development of an algorithm for address verification using ROBERTA, which can be employed to align data from various internet sources. The paper describes an algorithm for address verification using the Hugging Face Transformers Library for the implementation of the proposed ROBERTA model, [20], [21], [22]. The proposed algorithm achieves a 98.2% accuracy rate and a relatively high F-measure in comparison to the other algorithms applied in address validation, [23]. Further evaluation needs to be conducted using a different number of Transformers on the initial dataset.

ACKNOWLEDGMENT

I want to express special thanks to J. D. for this article. I
want to thanks also to the support from M. organization.

REFERENCES

[1] Cai, Wentao, Shengrui Wang, and Qingshan Jiang. "Address extraction: Extraction of location-based information from the web." Web Technologies Research and Development-APWeb 2005: 7th Asia-Pacific Web Conference, Shanghai, China, March 29-April 1, 2005. Proceedings 7. Springer Berlin Heidelberg, 2005.

[2] Fedushko, Solomia, and Yuriy Syerov. "Design of registration and validation algorithm of member's personal data." International Journal of Informatics and Communication Technology 2.2 (2013): 93-98.

[3] Dakrory, Sara, et al. "Extracting geographic addresses from social media using deep recurrent neural networks." 2021 9th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC). IEEE, 2021.

[4] Beverly, Robert, et al. "Understanding the efficacy of deployed internet source address validation filtering." Proceedings of the 9th ACM SIGCOMM conference on Internet measurement. 2009.

[5] Nagabhushan, P., Shanmukhappa A. Angadi, and Basavaraj S. Anami. "Symbolic data structure for postal address representation and address validation through symbolic knowledge base." Pattern Recognition and Machine Intelligence: First International Conference, PReMI 2005, Kolkata, India, December 20-22, 2005. Proceedings 1. Springer Berlin Heidelberg, 2005.

[6] U.S. POSTAL SERVICE FACILITIES: Improvements in Data Would Strengthen Maintenance and Alignment of Access to Retail Services. GAO Reports. December 2007:i-61. Accessed August 29, 2023.

[7] Soeng, Saravit, et al. "Deep Learning Based Improvement in Overseas Manufacturer Address Quality Using Administrative District Data." Applied Sciences 12.21 (2022): 11129.
[8] Wang, Yitong, et al. "Distill-AER: Fine-Grained Address Entity Recognition from Spoken Dialogue via Knowledge Distillation." Natural Language Processing and Chinese Computing: 11th CCF International Conference, NLPCC 2022, Guilin, China, September 24-25, 2022, Proceedings, Part I. Cham: Springer International Publishing, 2022.

[9] Guermazi, Yassine, Sana Sellami, and Omar Boucelma. "A RoBERTa Based Approach for Address Validation." New Trends in Database and Information Systems: ADBIS 2022 Short Papers, Doctoral Consortium and Workshops: DOING, K-GALS, MADEISD, MegaData, SWODCH, Turin, Italy, September 5-8, 2022, Proceedings. Cham: Springer International Publishing, 2022.

[10] Meena, Ramesh Chand, et al. "HyPASS: Design of hybrid-SDN prevention of attacks of source spoofing with host discovery and address validation." Physical Communication 55 (2022): 101902.

[11] Christen, Peter, and Daniel Belacic. "Automated probabilistic address standardisation and verification." Australasian Data Mining Conference. 2005.

[12] Abid, Nosheen, Adnan ul Hasan, and Faisal Shafait. "DeepParse: A trainable postal address parser." 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE, 2018.

[13] Delil, Selman, et al. "Parsing Address Texts with Deep Learning Method." 2020 28th Signal Processing and Communications Applications Conference (SIU). IEEE, 2020.

[14] Erickson, Jennifer, Kenneth Abbott, and Lucinda Susienka. "Automatic address validation and health record review to identify homeless Social Security disability applicants." Journal of Biomedical Informatics 82 (2018): 41-46.

[15] GUERMAZI, Yassine, SELLAMI, Sana, et BOUCELMA, Omar. GeoRoBERTa: A Transformer-based Approach for Semantic Address Matching. 2023.

[16] Booth, Darryl. "Building Capacity by Normalizing Addresses." Journal of Environmental Health 84.8 (2022): 32-34.

[17] Basu, Subhadip, et al. "A novel framework for automatic sorting of postal documents with multi-script address blocks." Pattern Recognition 43.10 (2010): 3507-3521.

[18] Lewis, Taylor, Joseph McMichael, and Charlotte Looby. "Evaluating Substitution as a Strategy for Handling US Postal Service Drop Points in Self-Administered Address-Based Sampling Frame Surveys." Sociological Methodology 53.1 (2023): 158-175.

[19] De, Shankkha, and Dipti Verma. "Deep Convolutional Transfer Learning approach for Bengali handwritten character recognition from document image." Science and Culture (2023)

[20] Wolf, Thomas, et al. "Huggingface's transformers: State-of-the-art natural language processing." arXiv preprint arXiv:1910.03771 (2019).

[21] Jain, Shashank Mohan. "Tasks Using the Hugging Face Library." Introduction to Transformers for NLP: With the Hugging Face Library and Models to Solve Problems. Berkeley, CA: Apress, 2022. 69-136.

[22] Ushio, Asahi, and Jose Camacho-Collados. "T-NER: an all-round python library for transformer-based named entity recognition." arXiv preprint arXiv:2209.12616 (2022).

[23] Kayed, Mohammed, Sara Dakrory, and Abdelmaged Amin Ali. "Postal address extraction from the web: a comprehensive survey." Artificial Intelligence Review 55.2 (2022): 1085-1120.
[8] Wang, Yitong, et al. "Distill-AER: Fine-Grained Address Entity Recognition from Spoken Dialogue via Knowledge Distillation." Natural Language Processing and Chinese Computing: 11th CCF International Conference, NLPCC 2022, Guilin, China, September 24-25, 2022, Proceedings, Part I. Cham: Springer International Publishing, 2022.

[9] Guermazi, Yassine, Sana Sellami, and Omar Boucelma. "A RoBERTa Based Approach for Address Validation." New Trends in Database and Information Systems: ADBIS 2022 Short Papers, Doctoral Consortium and Workshops: DOING, K-GALS, MADEISD, MegaData, SWODCH, Turin, Italy, September 5-8, 2022, Proceedings. Cham: Springer International Publishing, 2022.

[10] Meena, Ramesh Chand, et al. "HyPASS: Design of hybrid-SDN prevention of attacks of source spoofing with host discovery and address validation." Physical Communication 55 (2022): 101902.

[11] Christen, Peter, and Daniel Belacic. "Automated probabilistic address standardisation and verification." Australasian Data Mining Conference. 2005.

[12] Abid, Nosheen, Adnan ul Hasan, and Faisal Shafait. "DeepParse: A trainable postal address parser." 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE, 2018.

[13] Delil, Selman, et al. "Parsing Address Texts with Deep Learning Method." 2020 28th Signal Processing and Communications Applications Conference (SIU). IEEE, 2020.

[14] Erickson, Jennifer, Kenneth Abbott, and Lucinda Susienka. "Automatic address validation and health record review to identify homeless Social Security disability applicants." Journal of Biomedical Informatics 82 (2018): 41-46.

[15] GUERMAZI, Yassine, SELLAMI, Sana, et BOUCELMA, Omar. GeoRoBERTa: A Transformer-based Approach for Semantic Address Matching. 2023.

[16] Booth, Darryl. "Building Capacity by Normalizing Addresses." Journal of Environmental Health 84.8 (2022): 32-34.

[17] Basu, Subhadip, et al. "A novel framework for automatic sorting of postal documents with multi-script address blocks." Pattern Recognition 43.10 (2010): 3507-3521.

[18] Lewis, Taylor, Joseph McMichael, and Charlotte Looby. "Evaluating Substitution as a Strategy for Handling US Postal Service Drop Points in Self-Administered Address-Based Sampling Frame Surveys." Sociological Methodology 53.1 (2023): 158-175.

[19] De, Shankkha, and Dipti Verma. "Deep Convolutional Transfer Learning approach for Bengali handwritten character recognition from document image." Science and Culture (2023)

[20] Wolf, Thomas, et al. "Huggingface's transformers: State-of-the-art natural language processing." arXiv preprint arXiv:1910.03771 (2019).

[21] Jain, Shashank Mohan. "Tasks Using the Hugging Face Library." Introduction to Transformers for NLP: With the Hugging Face Library and Models to Solve Problems. Berkeley, CA: Apress, 2022. 69-136.

[22] Ushio, Asahi, and Jose Camacho-Collados. "T-NER: an all-round python library for transformer-based named entity recognition." arXiv preprint arXiv:2209.12616 (2022).

[23] Kayed, Mohammed, Sara Dakrory, and Abdelmaged Amin Ali. "Postal address extraction from the web: a comprehensive survey." Artificial Intelligence Review 55.2 (2022): 1085-1120.