## 4 Empirical Results
### Project Recommendation Results  
In this section, we showed the only top 1, 5, and 10 hit scores of the most significant metrics. Most of the ratio-based fusion metrics and optimized single metrics were very weak in all cases. Therefore, we removed them from the score tables below. The results of all metrics according to all \(n\) values are provided in the Appendix A. We used two similarity metrics to calculate project's similarity scores. To evaluate the accuracy of developer metrics, we used from two approaches. Thus, we give four results for the recommendation system with the combinations of similarity methods and evaluation approaches (Figure 7). After we generated these single, fusion, and binary fusion metrics, we applied all developer metrics to the model. We presented the hit scores percentiles that were obtained according to the two evaluation approaches (detailed in the Section 3.4 ) in Tables 9 and 10. In these tables, the columns represent the top-\(n\) hit scores in percentiles, and the green (1 st), blue (2 nd), and red (3 rd) cells show the leading metrics in each top-\(n\) scores. In addition, we styled the most successful five metrics according to overall scores in **bold**. We used the mean reciprocal rank (MRR) (It is commonly used in question-answering systems. Here, we used number of models instead of number of queries.) evaluation method (Equation (19)) to calculate this overall score where \(n\) represents the number of models created _(n:6)_.  
\[score_{MRR}=\frac{1}{n}\sum_{i=0}^{n}\frac{1}{rank_{i}} \tag{19}\]  
For example, as shown in Table 9, the MRR value of _comments_ is calculated as in Equation (20). The metric ranks for all metrics were used in each of the six models. \[score_{MRR}^{comments}=\frac{1}{6}*(\frac{1}{2}+\frac{1}{1}+\frac{1}{2}+\frac {1}{15}+\frac{1}{16}+\frac{1}{18})=0.36 \tag{20}\]  
In addition to Tables 9 and 10, we presented all top-\(n\) hit scores charts of the most successful five metrics (+1 Sun_metric) for each methods in Figures 8 and 9. Table 9 shows the results of hit scores using the context-free and context-based similarity methods as evaluated using the community relation approach. When the results were analyzed, the _pr_opened_ metric was the most successful according to MRR scores. As PRs indicate projects to which a developer contributed directly, we believe that modeling based on PR creation in a given project increases the success of the project recommendation system. In addition, the fusion metrics _comment_, _binary_pr_related_, _binary_issue_related_, and _binary_comment_ also attracted our attention, as these metrics all related to commenting activity. The results therefore indicate the importance of discussion on the collaboration platforms. Figure 7: The hit scores calculations with 4 different parameters combinations. Table 10 shows the results of hit scores according to the context-free and context-based similarity methods as evaluated using the language experience approach. This approach elicited higher hit scores than the first approach as we expected. (In the community relation approach, the model recommends the top-\(n\) projects of the approximately 40,000 projects. On the contrary, the language experience approach makes only recommendation from 94 distinct language projects). However, it is important that evaluate the success rate of the two approaches independently. The binary fusion metrics clearly stand out and most fusion metrics gave better hit scores than single metrics. In this context, because a fusion  
Figure 8: The most successful 5 developer metrics all hit scores (community).