# Name Disambiguation Scheme Based on Heterogeneous Academic Sites
## 4 Performance Evaluation
### Comparative Performance Evaluation of Name Disambiguation Schemes  
A performance comparison with existing name disambiguation schemes was conducted to demonstrate the superiority of the proposed method. In this study, two schemes were compared: (1) the Protasiewicz method [7], an existing rule-based name disambiguation scheme, that uses the attributes of papers to create rules and runs HAC with the weights of these rules to disambiguate names. (2) The Chen Ya method [13], an existing deep learning-based name disambiguation scheme, that learns paper attributes using a GCN and then runs HAC with the resulting weights to disambiguate names. The HAC of the proposed scheme was set with a termination criterion of 0.2 and used the complete linkage method. The dataset used for performance evaluation is summarized in Table 6.  
Figure 10: Precision and F1-measure according to the HAC linkage method.  
Figure 9: Precision and F1-measure according to the HAC termination criterion.  
The superiority of the proposed method was demonstrated through a comparative performance evaluation of precision, recall, and F1-measure for name disambiguation accuracy between the proposed and existing methods.  
Figure 11 shows the results of the comparative performance evaluation of precision based on the name disambiguation schemes. The precision of the proposed scheme exhibited a very high performance, with scores >0.99 for all keywords. However, the existing rule-based scheme, the Protasiewicz method, showed a decent average performance but underperformed for certain keywords. Additionally, the deep learning-based scheme demonstrated the poorest average performance. The data used for the performance evaluation were collected from actual academic search services. This indicated that existing studies need more detailed preprocessing and analysis when performing name disambiguation analysis based on real data. Furthermore, it implies that these characteristics must be reflected, since not all academic search services provide the same metadata.  
Figure 12 displays the results of the comparative performance evaluation of recall based on the name disambiguation schemes. The recall of the proposed scheme demonstrated a very high performance, with scores >0.97 for all keywords. The existing deep learning-based Chen Ya method showed excellent performance for keywords \"cloud computing\", \"AI verification\", and \"steering control\"; however, the proposed method outperformed for all keywords.  
Figure 13 displays the results of the comparative performance evaluation of F1-measure based on the name disambiguation schemes. The F1-measure of the proposed scheme demonstrated a very high performance, with scores >0.98 for all keywords. Compared with the existing rule-based Protasiewicz scheme and deep learning-based Chen Ya scheme, the proposed scheme exhibited higher performance across all keywords, thereby proving its superiority. A deep learning-based scheme is useful for diversifying keywords or feature dimensions. However, unlike the rule-based scheme, it is difficult to determine the similarities of affiliation or co-author name. For this reason, the deep learning-based scheme shows lower performance than the proposed rule-based scheme.  
Figure 11: Precision according to the name disambiguation schemes.  
Figure 12: Recall based on the name disambiguation schemes.  
Figure 13: F1-measure based on the name disambiguation schemes.