# Investigation on Self-Admitted Technical Debt in Open-Source Blockchain Projects
## 3 SATD Detection in Blockchain Software Projects
### Choice of Detection Technique and Configuration
In particular, as shown in Table 6, the optimized classifier detects a number of comments with a SATD 85% lower than the classifier of the reference configuration. Additionally, if we manually examine comments that were classified as SATD using the reference setup but not the optimized configuration, we discover that these are not necessarily false positives. For example, comments that have keywords like TODO, FIXME, and XXX are labeled \"WITHOUT CLASSIFICATION\" by the classifier with the optimized configuration. The same comments are detected as SATD by the classifier with the reference configuration. Some of these comments and their classification labels are shown in Table 7.  
As a further consideration, we must mention that this study aims to compare the presence of SATD in blockchain projects with that in other open-source projects. Having chosen the study in [11] as a reference, the same configuration must be maintained. Both considerations lead us to decide to use the reference configuration in [11] for SATD detection in blockchain projects. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Class** & **TP** & **FN** & **FP** & **TN** & **Acc** & **P** & **R** & **F1** \\ \hline DESIGN & 209 & 73 & 48 & 5438 & 0.979 & 0.813 & 0.741 & 0.776 \\ REQUIREMENT & 42 & 30 & 16 & 5680 & 0.992 & 0.724 & 0.583 & 0.646 \\ WITHOUT CLASSIFICATION & 5380 & 34 & 73 & 281 & 0.981 & 0.987 & 0.994 & 0.990 \\ \hline Micro-averaged accuracy/F1: & 0.97625 & & & & & & \\ Macro-average F1: & 0.80394 & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of classification results with the two configuration props, namely the reference configuration and the optimized configuration. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Comment** & **Classification—Reference** & **Classification—Optimized Prop** \\ \hline TODO actually disable crypto verification & REQUIREMENT & WITHOUT CLASSIFICATION \\ TODO Shutdown VM if an & REQUIREMENT & WITHOUT CLASSIFICATION \\ error occurs & & DESIGN & WITHOUT CLASSIFICATION \\ Probably because signature & & DESIGN & WITHOUT CLASSIFICATION \\ wrong & & & & \\ If p node Dispatch panics then & & & \\ we should log the panic and & & & \\ then re raise the panic This is & & DESIGN & WITHOUT CLASSIFICATION \\ why the above defer is broken & & & \\ into two parts & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Samples of comments and the comparison of the classification obtained with the models created via the reference prop file and via the optimized prop file. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Class** & **TP** & **FN** & **FP** & **TN** & **Acc** & **P** & **R** & **F1** \\ \hline DESIGN & 189 & 96 & 303 & 5183 & 0.913 & 0.380 & 0.660 & 0.482 \\ REQUIREMENT & 42 & 30 & 39 & 5657 & 0.988 & 0.519 & 0.583 & 0.549 \\ WITHOUT CLASSIFICATION & 5100 & 314 & 98 & 256 & 0.929 & 0.981 & 0.942 & 0.961 \\ \hline Micro-averaged accuracy/F1: & 0.92372 & & & & & & \\ Macro-average F1: & 0.66423 & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Classification Result with the configuration in [11] of 5768 samples in test set. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Project** & **Number of Comments** & \multicolumn{2}{c}{**SATD Design.**} & \multicolumn{2}{c}{**SATD Requirement.**} \\  & & **Reference** & **I Optimized** & **Reference** & **I Optimized** \\ \hline Avalanche & 12,071 & 909131 & 179129 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Classification result with the optimized configuration prop of 5768 samples in test set.