## 5 Empirical Studies
### Results and Discussion
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Datasets**} & \multirow{2}{*}{**Granularity**} & \multirow{2}{*}{**Metrics**} & \multicolumn{4}{c}{**Trial I**} \\ \cline{4-6}  & & & **A** & **B** & **C** & **D** \\ \hline \multirow{4}{*}{Vuze} & File level & HR@10 & 14.35 \(\pm\) 0.69 & 16.25 \(\pm\) 0.42 & 16.31 \(\pm\) 0.26 & **16.70**\(\pm\) 0.41 \\  & & NDCG@10 & 0.081 \(\pm\) 0.002 & 0.095 \(\pm\) 0.001 & 0.098 \(\pm\) 0.002 & 0.10 \(\pm\) 0.001 \\  & \multirow{2}{*}{Package level} & HR@10 & 14.76 \(\pm\) 0.341 & 15.69 \(\pm\) 0.484 & 16.09 \(\pm\) 0.379 & **16.35**\(\pm\) **0.301** \\  & & NDCG@10 & 0.088 \(\pm\) 0.001 & 0.096 \(\pm\) 0.001 & 0.103 \(\pm\) 0.001 & 0.10 \(\pm\) 0.001 \\ \hline \multirow{4}{*}{Spring Framework} & File level & HR@10 & 11.49 \(\pm\) 0.379 & 11.80 \(\pm\) 0.543 & **12.56**\(\pm\) **0.253** & 12.54 \(\pm\) 0.221 \\  & \multirow{2}{*}{Package level} & NDCG@10 & 0.079 \(\pm\) 0.002 & 0.08 \(\pm\) 0.003 & 0.087 \(\pm\) 0.002 & 0.08 \(\pm\) 0.001 \\  & & HR@10 & 22.50 \(\pm\) 0.731 & 22.59 \(\pm\) 0.723 & **23.13**\(\pm\) **1.096** & 22.27 \(\pm\) 0.829 \\  & & NDCG@10 & 0.16 \(\pm\) 0.004 & 0.17 \(\pm\) 0.006 & 0.177 \(\pm\) 0.007 & 0.173 \(\pm\)0.004 \\ \hline \multirow{4}{*}{Elasticsearch} & \multirow{2}{*}{File level} & HR@10 & 15.41 \(\pm\) 0.257 & **16.51**\(\pm\) **0.186** & 16.27 \(\pm\) 0.251 & 15.97 \(\pm\) 0.098 \\  & & NDCG@10 & 0.101 \(\pm\) 0.001 & 0.112 \(\pm\) 0.001 & 0.11 \(\pm\) 0.001 & 0.107 \(\pm\) 0.001 \\  & \multirow{2}{*}{Package level} & HR@10 & 18.29 \(\pm\) 0.259 & **18.476**\(\pm\) **0.276** & 18.405 \(\pm\) 0.195 & 17.82 \(\pm\) 0.149 \\  & & NDCG@10 & 0.130 \(\pm\) 0.001 & 0.130 \(\pm\) 0.001 & 0.131 \(\pm\) 0.000 & 0.127 \(\pm\) 0.000 \\ \hline \multicolumn{6}{c}{**Trial II**} \\ \hline \multirow{4}{*}{Vuze} & File level & HR@10 & 37.80 \(\pm\) 1.06 & **38.61**\(\pm\) **0.72** & 37.00 \(\pm\) 0.55 & 37.51 \(\pm\) 0.21 \\  & \multirow{2}{*}{Package level} & NDCG@10 & 0.21 \(\pm\) 0.005 & 0.22 \(\pm\) 0.003 & 0.21 \(\pm\) 0.002 & 0.58 \(\pm\) 0.003 \\  & & HR@10 & **57.76**\(\pm\) **0.227** & 55.20 \(\pm\) 0.455 & 53.89 \(\pm\) 0.326 & 20.67 \(\pm\) 0.0 \\  & & NDCG@10 & 0.357 \(\pm\) 0.001 & 0.325 \(\pm\) 0.001 & 0.334 \(\pm\) 0.001 & 0.12 \(\pm\) 0.0 \\ \hline \multirow{4}{*}{Spring Framework} & File level & HR@10 & 26.62 \(\pm\) 0.325 & 27.19 \(\pm\) 0.424 & **29.84**\(\pm\) **0.455** & 28.31 \(\pm\) 0.267 \\  & \multirow{2}{*}{Package level} & NDCG@10 & 0.171 \(\pm\) 0.001 & 0.172 \(\pm\) 0.002 & 0.189 \(\pm\) 0.001 & 0.180 \(\pm\) 0.002 \\  & & HR@10 & 51.76 \(\pm\) 0.451 & 52.23 \(\pm\) 0.231 & **53.45**\(\pm\) **0.202** & 53.01 \(\pm\) 0.000 \\  & & NDCG@10 & 0.362 \(\pm\) 0.002 & 0.359 \(\pm\) 0.002 & 0.363 \(\pm\) 0.000 & 0.363 \(\pm\) 0.000 \\ \hline \multirow{4}{*}{Elasticsearch} & File level & HR@10 & 33.86 \(\pm\) 0.344 & **34.56**\(\pm\) **0.312** & 34.47 \(\pm\) 0.249 & 34.55 \(\pm\) 0.181 \\  & \multirow{2}{*}{Package level} & NDCG@10 & 0.213 \(\pm\) 0.001 & 0.220 \(\pm\) 0.001 & 0.220 \(\pm\) 0.001 & 0.220 \(\pm\) 0.001 \\ \cline{1-1}  & \multirow{2}{*}{Package level} & HR@10 & 35.395 \(\pm\) 0.488 & **35.72**\(\pm\) **0.332** & 35.32 \(\pm\) 0.406 & 35.348 \(\pm\) 0.208 \\ \cline{1-1}  & & NDCG@10 & 0.225 \(\pm\) 0.002 & 0.224 \(\pm\) 0.001 & 0.222 \(\pm\) 0.002 & 0.222 \(\pm\) 0.001 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Evaluation performance using normalized discount cumulative gain (NDCG) and hit ratio (HR) of the FCP2 Vec model in multiple trial scenarios. Trial I results from the default parameter values for an estimator and Trial II results from Bayesian optimization over hyperparameters using the following split ratios: A = 90:5.5, B = 85:7.5.7.5, C = 80:10:10, and D = 70:15:15.  
we used t-SNE to reduce the vector size from thirty-seven dimensions to two dimensions. This means that the data were originally represented by thirty-seven different features, but were reduced to just two dimensions, which makes them easier to plot and visualize.