# Improving a Street-Based Geocoding Algorithm Using Machine Learning Techniques
## 4 Experimental Evaluation
### Experimental Case 3: Address Matching Using Machine Learning
We generated and tested three kinds of training data consisting of 20% of matching pairs and 80% of non-matching pairs (Training data 1), 50% of matching pairs and 50% of non-matching pairs (Training data 2), or 80% of matching pairs and 20% of non-matching pairs (Training data 3). The training data have a column 'Matching' used as a label for training, which provides information on whether the records are the matching pairs (Matching: 1) or non-matching pairs (Matching: 0). In the address matching, to choose the best machine learning model, the performance of three machine learning models was compared with the matching without any similarity metrics and 17 similarity metrics using training data through the model evaluation and selection. Without any similarity metrics, matching shows 85% of accuracy using Training data 1 and 37% of accuracy using Training data 3 (Table 5). As the ratio of matching pairs is becoming more substantial, the matching accuracy without similarity metrics becomes smaller too because the matching only identifies the same word pairs, and this method cannot regard similar words as identical. Among different similarity metrics, Overlap shows the highest accuracy using Training data 1 (95%), whereas Jaro-Winkler Sorted, Tversky, and Jaccard show the best accuracy using Training data 3 (96%). Most of similarity metrics show the highest accuracy in Training data 3 when compared to Training data 1 and 2, which is opposite to the matching without similarity metrics, possibly because the similarity metrics can match similar words, and the Training data 3 has the highest percentage of matching pairs with aliases. However, when the training data are 50% + 50% pairs and are not biased to the matching or non-matching pairs, many similarity metrics have the lowest accuracy. It indicates that it is difficult for similarity metrics to successfully identify both matching and non-matching pairs well when the matching and non-matching pairs are of equal parts in the training data. Three machine learning models have higher accuracy than all the 17 similarity metrics in the three kinds of training data with different pair combinations. All the three methods are tuned with optimal hyper-parameters to achieve the best performance. We performed hyper-parameter tuning \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Training Data 1 1 ** & **Training Data 3 1 ** \\ \hline Matching without similarity metrics & 85 & 61 & 37 \\ [PERSON] & 89 & 92 & 95 \\ [PERSON] & 89 & 85 & 92 \\ [PERSON] Reversed & 92 & 90 & 95 \\ [PERSON] Sorted & 91 & 91 & 96 \\ [PERSON] & 85 & 67 & 80 \\ [PERSON] & 58 & 59 & 80 \\ [PERSON] & 89 & 87 & 94 \\ [PERSON]\_[PERSON] & 85 & 68 & 84 \\ [PERSON] & 85 & 81 & 90 \\ [PERSON]\_[PERSON] & 85 & 76 & 87 \\ [PERSON] & 89 & 89 & 94 \\ [PERSON] & 90 & 91 & 96 \\ [PERSON] & 95 & 89 & 92 \\ [PERSON] & 85 & 80 & 90 \\ [PERSON] & 90 & 91 & 96 \\ [PERSON]\_[PERSON] & 89 & 89 & 94 \\ [PERSON] & 87 & 63 & 80 \\ [PERSON] & 100 & 99 & 97 \\ RF & 100 & 99 & 98 \\ XGB & 100 & 100 & 98 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performance of address matching without any similarity metrics, 17 similarity metrics, and three machine learning methods. using a grid search. This method sets a list of parameters, and the range of values for each parameter. For each classifier, the algorithm attempts every combination of parameters. Then, the best set of hyper-parameter values was chosen based on accuracy. SVM, RF, and XGB have the highest accuracy using Training data 1 (100%), and it gets a bit lower as the ratio of matching pairs increases. SVM and RF show 99% of accuracy using Training data 2, while XGB achieves 100%. Using Training data 3, SVM has 97% of accuracy, whereas RF and XGB show 98%. XGB shows the highest accuracy (100%) in Training data 2, tuned with 0.05 for learning rate, 400 gradient boosted trees, and 0.7 for subsample ratio of columns.