## 4 Evaluation
### One-to-Many Basic Block Matching  
To evaluate the performance of BlockMatch and other baselines in the last-mile problem of the BCSD tasks regarding matching among numerous basic blocks, we conducted ex  
Figure 8: Results of basic block similarity detection across compilers.  
Figure 7: Results of basic block similarity detection across optimizations.  
periments to retrieve the most similar basic block from the large pool-size set of basic blocks. We performed BlockMatch and baselines in both cross-compiler and cross-optimization scenarios to provide a comprehensive evaluation of their performance. Moreover, we employed multiple pool sizes to gauge BlockMatch's capability under varying levels of difficulty and to discuss the effect of pool size on BCSD approaches. In order to determine the range of pool size values, we conduct a statistical analysis on the distribution of the number of basic blocks in functions across all projects in the dataset. As shown in Figure 9, functions with more than 10 basic blocks account for 49.3% of the total, while functions with more than 1000 basic blocks only constitute 0.01%. Thus, for our experiment, we select a pool size within or less than 1000.  
Tables 1 and 2 report the recall@1 and MRR results for each approach across different compilers and optimization options in multiple pool sizes (10 and 100). BlockMatch significantly surpasses all baselines in terms of recall@1 and MRR. For example, in the cross-optimization (XO) experiment with a pool size of 10, BlockMatch yields a recall@1 score of 0.912 and a MRR score of 0.943, increasing the recall@1 by 14.3% and 12.6% as compared to SAFE and PalmTree. In the cross-compile (XC) experiment (pool size = 100), BlockMatch outperforms its nearest baseline competitor, PalmTree [23], by 0.407 for recall@1 and by over 40% for the MRR.  
The experimental results from Tables 1 and 2 reveal a decline in recall@1 and MRR for BCSD approaches as the pool size increases. To delve deeper into the impact of pool size on the performance of BCSD approaches, we conduct experiments with pool sizes ranging between 1 and 1000. Figure 10 displays the result curves of these experiments  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Recall@1**} & \multicolumn{3}{c}{**MRR**} \\ \hline
**Models** & **XO** & **XC** & **Average** & **XO** & **XC** & **Average** \\ \hline SAFE & 0.798 & 0.573 & 0.685 & 0.861 & 0.695 & 0.778 \\ PalmTree & 0.810 & 0.587 & 0.699 & 0.862 & 0.693 & 0.778 \\ BlockMatch & 0.912 & 0.790 & 0.851 & 0.943 & 0.860 & 0.901 \\ \hline \hline \end{tabular}
\end{table}
Table 1: BCSD results for multiple scenarios at the basic block level (pool size = 10).  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Recall@1**} & \multicolumn{3}{c}{**MRR**} \\ \hline
**Models** & **XO** & **XC** & **Average** & **XO** & **XC** & **Average** \\ \hline SAFE & 0.654 & 0.364 & 0.509 & 0.712 & 0.451 & 0.581 \\ PalmTree & 0.720 & 0.407 & 0.564 & 0.757 & 0.480 & 0.619 \\ BlockMatch & 0.798 & 0.588 & 0.693 & 0.847 & 0.673 & 0.760 \\ \hline \hline \end{tabular}
\end{table}
Table 2: BCSD results for multiple scenarios at the basic block level (pool size = 100).  
Figure 9: The distribution of the number of basic blocks in functions, where the shaded area represents functions with more than 10 basic blocks, accounts for 49.3% of the total.  
according to variations in pool size in cross-optimization and cross-compiler experiment configurations. For ease of observation, we apply a logarithmic x-axis in Figure 10. Across a range of pool sizes from 1 to 1000, BlockMatch consistently outperforms PalmTree and SAFE in terms of Recall@1 and MRR. Furthermore, we observe that BlockMatch exhibits an more significant advantage over the baselines in cross-compiler experiments. This is attributable to BlockMatch using a contrastive training framework to generate a substantial amount of high-quality samples for training, thereby enabling our model to effectively handle hard samples.