# Record Linkage of Chinese Patent Inventors and Authors of Scientific Articles
## 3 Experiments and Discussion
### Evaluation of Record Linkage
#### 3.2.2 Evaluation of Matching Quality  
We measure the quality of record linkage using metrics utilized in the evaluation of binary classification: precision, recall and F-measure, because the final part of record linkage is the classification of previously generated candidate record pairs into matches and non-matches. We do not provide accuracy, specificity, and the false positive rate, due to high class imbalance, which is present among record pair candidates. Three different types of classifiers, described in Section 2.2, in multiple configurations, were evaluated. Four baselines based on TC were defined. These baselines refer to the most simple methods of linking records where names of patent inventors and authors of articles are the only attributes considered and a record pair is classified as a _true-match_ if both records agree on Chinese names (Baseline 1. ), English names (Baseline 2. ), names in either language (Baseline 3. ), or finally names in both languages (Baseline 4.). Other models are trained and evaluated using different subsets of features, as depicted in Table 5.  
During experiments 16 different feature subsets (FSs), marked with symbols X1, X2, X3 and the letters A through M, were tested. Subsets X1, X2 and X3 were used only by the baselines. FSs marked A to C used an exact match of Chinese names and different fuzzy similarities of English names. Evaluation on these FSs aimed to test the impact of fuzzy name matching on record linkage results. FS marked D extends base features with Feature 6, thus testing whether comparison of patents and papers by their content improves the results. Consequently, FSs marked E and F test how adding ASJC similarities \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Indexing Method** & **True Links** & **Candidates** & **RR** & **PC** & **PQ** \\ \hline Cartesian product & 4540 & 71,584,786 & - & - & - \\ \hline B-EN + SN-CN & 4510 & 26,617 & 1.000 & 0.993 & 0.169 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Statistics for baseline and best variant of indexing measured on a testing subset. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**FS** & **Feat. ** & **1. ** & **2. ** & **3. ** & **4. ** & **5. ** & **6. ** & **7. ** & **8. ** \\ \hline X1 & \(\bullet\) & & & & & & \\ \hline X2 & & \(\bullet\) & & & & & \\ \hline X3 & & \(\bullet\) & \(\bullet\) & & & & \\ \hline A & & \(\bullet\) & & \(\bullet\) & & & \\ \hline B & & \(\bullet\) & & & \(\bullet\) & & \\ \hline C & & \(\bullet\) & & & \(\bullet\) & & \\ \hline D & & \(\bullet\) & \(\bullet\) & & & \(\bullet\) & \\ \hline E & & \(\bullet\) & \(\bullet\) & & & & \(\bullet\) \\ \hline F & & \(\bullet\) & \(\bullet\) & & & & \(\bullet\) \\ \hline G & & \(\bullet\) & & \(\bullet\) & & \(\bullet\) & \\ \hline H & & \(\bullet\) & & \(\bullet\) & & \(\bullet\) & \\ \hline I & & \(\bullet\) & & \(\bullet\) & & & \(\bullet\) \\ \hline J & & \(\bullet\) & & \(\bullet\) & & \(\bullet\) & \\ \hline K & & \(\bullet\) & & \(\bullet\) & & \(\bullet\) & \(\bullet\) \\ \hline L & & \(\bullet\) & \(\bullet\) &changes the results. FSs marked G to L contain other configurations of features that were interesting for evaluation. Finally, set M contains all the created features. Linear correlations between features in training subsets are depicted in Figure 5. Features 1, 2, and 6 to 8 were weakly or moderately correlated with each other, with maximum Pearson's Correlation Coefficient (PCC) of 0.4 between features 1 and 6. As expected, Features 2-5 were highly correlated to each other, because they all represent similarities between the same attributes--English names of authors. Feature 4, containing a SoftTFIDF score, was arbitrarily selected out of Features 3-5 and used in the described FSs G to L. TC and LR were evaluated on each FS marked A to M and MLP was evaluated only on M, as this model was expected to have worse performance on other FSs and required separate hyperparameter tuning on each FS.