## 5 Case Study and Bug Classification
### Manual Bug Classification
The formula for K is given as follows:  
\[\text{K}=(\text{Pr(a)}-\text{Pr(e)})/(1-\text{Pr(e)}) \tag{6}\]  
Pr(a) is the relative observed agreement among raters in the above equation, and Pr(e) is the hypothetical probability of chance agreement. The calculated K value for ESE1 and ESE2 based on the results of Table 4 is 0.63. To interpret this coefficient, we have used the evaluation table of [PERSON] [18] given in Table 5. This evaluation table concluded that the classifications performed by ESE1 and ESE2 are defined as \"agreement with significant importance\". Table 6 present the accuracy and F1 measure. As observed from Table 6, the accuracy and F1 measure are very close in each iteration, implying that multiple iterations are necessary to classify the bugs correctly. In turn, this will lead to an increased time for correcting bugs and the increased cost of the project. To sum up, even though expert software engineers classify the bugs, there seem to be differences in classifications. It should be noted that, as stated before, the classifications are based on bug reports that were derived from black-box testing. The source code was thus not visible. The differences in classifications and the low F1 measures in the above table could be explained from this perspective. Investigation of bug labels and explanations \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Accuracy (\%)** & **Weighted F1 Measure (\%)** \\ \hline
**ESE1** & 60.32 & 58.48 \\
**ESE2** & 56.75 & 55.36 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Accuracy and weighted F1 measure for manual classification by two ESE. \begin{table}
\begin{tabular}{l c c} \hline \hline \multirow{2}{*}{**Bug Category**} & **Ground Truth** & **Classification of** & **Classification of** \\  & **Classification** & **ESE1** & **ESE2** \\ \hline Assignment– & 54 & 19 & 20 \\ initialization & 245 & 244 & 200 \\ External interface & 67 & 43 & 16 \\ Other & 138 & 126 & 268 \\ \hline Total & 504 & 504 & 504 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Manual classification by two expert software engineers (ESE). \begin{table}
\begin{tabular}{c c} \hline \hline \multicolumn{1}{c}{**K Measure**} & **Interpretation** \\ \hline K \textless{} 0 & No agreement \\
0.00–0.20 & Agreement with less importance \\
0.20–0.40 & Agreement with medium importance \\
0.41–0.60 & Agreement with general importance \\
0.61–0.80 & Agreement with significant importance \\
0.81–1.00 & Agreement with excellent importance \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Interpretation of K measure [18]. showed that it is hard to remove the discrepancy. Labels and explanations are sometimes written shortly, and it might be hard to understand the details of the bugs even for the software developers. Furthermore, bug labels and explanations may also include errors. Altogether we could conclude that the classification is subjective and, to some extent, error-prone and requires further improvement to reduce the time to classify the bugs and herewith the correction of the bugs. Some sample bug definitions and possible root causes are given in Table 7. As observed from this table, it is impossible to determine the error's root cause in these cases by just reading the bug definitions. Therefore, software engineers use their past experience to guess the type of bug for these definitions.