## 5 Results and Discussion
\begin{table}
\begin{tabular}{c c} \hline \hline
**Error Combination** & **Mean \(\pm\) Std** \\ \hline A1 & 0.71 \(\pm\) 0.01 \\ \hline A2 & 0.65 \(\pm\) 0.02 \\ \hline A3 & 0.71 \(\pm\) 0.02 \\ \hline A1A2 & 0.77 \(\pm\) 0.01 \\ \hline A1A3 & 0.78 \(\pm\) 0.01 \\ \hline A2A3 & 0.78 \(\pm\) 0.01 \\ \hline A1A2A3 & 0.83 \(\pm\) 0.01 \\ \hline B1 & 0.91 \(\pm\) 0.01 \\ \hline B2 & 0.92 \(\pm\) 0.01 \\ \hline B1B2 & 0.92 \(\pm\) 0.01 \\ \hline A1B1 & 0.77 \(\pm\) 0.01 \\ \hline A2B1 & 0.74 \(\pm\) 0.01 \\ \hline A3B1 & 0.77 \(\pm\) 0.01 \\ \hline A1A2B1 & 0.81 \(\pm\) 0.01 \\ \hline A1A3B1 & 0.83 \(\pm\) 0.01 \\ \hline A2A3B1 & 0.81 \(\pm\) 0.01 \\ \hline A1B2 & 0.77 \(\pm\) 0.02 \\ \hline A2B2 & 0.74 \(\pm\) 0.01 \\ \hline A3B2 & 0.79 \(\pm\) 0.01 \\ \hline A1A2B2 & 0.83 \(\pm\) 0.01 \\ \hline A1A3B2 & 0.83 \(\pm\) 0.01 \\ \hline A2A3B2 & 0.87 \(\pm\) 0.01 \\ \hline A1B1B2 & 0.82 \(\pm\) 0.01 \\ \hline A2B1B2 & 0.8 \(\pm\) 0.01 \\ \hline A3B1B2 & 0.83 \(\pm\) 0.01 \\ \hline A1A2B1B2 & 0.86 \(\pm\) 0.01 \\ \hline A2A3B1B2 & 0.87 \(\pm\) 0.01 \\ \hline A1A2A3B1B2 & 0.88 \(\pm\) 0.01 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Mean \(\pm\) standard deviation for accuracy per error combination. The results indicate that training a reasonably effective neural network for label quality evaluation is achievable using a _small set_. Once trained, this neural network can be applied to a _large set_, automatically assessing whether the labels are of good quality or not. To recapitulate label quality assurance from a requirements engineering perspective, let us review Figure 4 for the \"Production-like Image Data Acquisition\" activity. In this activity, relevant data are collected. The subsequent activity, \"Data Quality Assurance\", involves manual visual inspection of a random subset for a sanity check, along with statistics about widths, heights, pixel histograms and semantic similarity analysis. The \"Data Labeling\" activity follows, which includes careful and high-quality labeling of a _small set_ of collected data by the internal team. A rule of thumb for the size of the subset is approximately 5% of the full dataset. This work uses 4.2% of all relevant data for training the label quality assurance model. Once the label quality assurance model is complete, the remaining unlabeled data from the _large set_ can be distributed among multiple annotators, and the model will immediately alert annotators if a label is suspicious. The quantitative parameters that demonstrate the efficacy of this approach include an accuracy of 82%, precision of 82% and recall of 82%, as shown in the test classification report (Table 2). For qualitative parameters, the paper presents various examples of true positives, true negatives, false positives and false negatives (Figures 20-23). These examples help illustrate the model's ability to correctly identify good and bad labels as well as highlight cases where the model makes incorrect predictions, providing a visual assessment of the label quality assurance performance. In summary, our findings demonstrate the effectiveness of the machine learning-based label quality assurance model by differentiating between good and bad labels with an accuracy of 82%. The model performs particularly well in detecting B2 and B1B2 error combinations, while it faces challenges in detecting the A2 error subtype. The approach we proposed enables efficient and accurate label quality assurance in object detection tasks, allowing for rapid evaluation of labels in large datasets. This method can greatly reduce the time and effort spent on manual review, ultimately improving the overall quality and reliability of the labeled data used for training computer vision models. The proposed approach has several advantages. First, it automates the label quality assurance process, saving significant time and effort that would have been spent on manual review. Second, it allows for the rapid assessment of label quality in large datasets, which can be crucial for developing high-performance computer vision models.