# A Survey on Bug Deduplication and Triage Methods from Multiple Points of View
## 6 Evaluation Methods and Results
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Work** & **Methods** & **Effects** & **Dataset** \\ \hline \hline \multicolumn{1}{l}{Deduplication through local references} & \multicolumn{1}{l}{Reducing search space based on} & \multicolumn{1}{l}{Up to 53\% recall rate} & \multicolumn{1}{l}{74 k from FireFox} \\ \hline Time-based deduplication & BM25 Fext & 45 k from eclipse & 77\% RR20 \\ \hline FactorI\(\_\)CS & Enhancing LCS using size matching within group weight & \(\geq\)70\% recall rate & 97 k\(\_\) from FireFox and 41 k\(\_\) from Eclipse \\ \hline Fusion approach For deduplication & MULAN-based multilabel classification model & 72\% recall rate and up to 40\% performance improvement & 111 k from FireFox \\ \hline \multirow{2}{*}{Triaging for very large bug repositories} & \multicolumn{1}{l}{Tet cosine similarity, time window, and document factors} & \multicolumn{1}{l}{29\%; original recall and low duplicate recall} & \multicolumn{1}{l}{246 k from Eclipse, Firefox, and Open Office} \\  & \multicolumn{1}{l}{document factors} & \multicolumn{1}{l}{triggering guide} & \multicolumn{1}{l}{} \\ \hline \multirow{2}{*}{Deduplication using correlations} & \multicolumn{1}{l}{Stack trace signature, temporal locality, and crash comment textual similarity} & \multicolumn{1}{l}{50\% and 47\%; Recall Rate and 58\% and 35\% precision for the FireFox and Eclipse} & \multicolumn{1}{l}{1 k \% types from FireFox and MSR and 20 k+ respectively} \\  & \multicolumn{1}{l}{} & \multicolumn{1}{l}{} & \multicolumn{1}{l}{} \\ \hline LIVE & LDA and word embedding & 0.558 RR202\% & 76k \& from Mozilla \\ \hline Refined feature-based deduplication & Resolution field extraction & Not mentioned & 10–22\% recall rate improvement and 7\(\sim\)18\% precision improvement & \\ \hline Duplication based on multiple factors & Reasonable parameter selection & 80\% TP and 0.01\%FP for deduplication & 3 M from sybrot \\ \hline Stack trace similarly aggregation & Aggregate computing & 15\% RR01 improvement & 40 k from Netheans and 210 k from JeftBraints \\ \hline \multirow{2}{*}{CrasBearch} & \multicolumn{1}{l}{Locality-sensitive hashing} & \multicolumn{1}{l}{11\% F-score improvement compared with minor hashing} & \multicolumn{1}{l}{1 k from eight real-world programs} \\ \hline K-detector & AST comparison & 0.986 AUC on SAP HANA & 10 k dump from SAP HANA \\ \hline Reformulating queries & Three different queries & 42 k from 20 open-source projects & 56\(\_\)6\(\sim\)78\% duplication detection \\ \hline CosTriage & Reduce cost of assigning bugs & 30\% cost reduction & 13 k from Apache, 152 k from Eclipse, 5 k from Linux kernel, and 162 k from Mozilla \\ \hline Duplicate based on contextual approach & Multiple context feature comparison & 11.5\% accuracy improvement, 41\% Kappa & 37 k from Android, 43 k from Eclipse, 71 k improvement, and 16.8\% AUC improvement & \\ \hline \multirow{2}{*}{Triage based on developer analysis} & \multicolumn{1}{l}{Unigram model and Kullback-Leibler (KL) divergence} & \multicolumn{1}{l}{\(\sim\)75\% precision and \(\sim\)40\% and \(\sim\)52\% F1 score} & 8 k from Eclipse and 10 k from Mozilla \\ \hline \end{tabular}
\end{table} 
Table 4: Effect comparison of works using the IR method based on bug reports.