# Tire Defect Detection via 3D Laser Scanning Technology
## 3 Experiments and Discussion
### Ablation Studies  
In this section, we conducted comprehensive experiments on the rendered image dataset to justify the effectiveness of our method. All the results are shown in Table 4.  
We first conducted experiments to justify the effectiveness of DA-FPN. In model A, we adopted vanilla FPN [22], while we used DA-FPN in model B. We can clearly find that model B outperformed model A in all metrics. In particular, model B achieved 0.357 AP\({}_{S}\), which was 5% better than that of model A, indicating that DA-FPN showed more sensitivity to tiny defects. Then, we compared different query generation strategies. Model C adopted random selection by randomly selecting query vectors and corresponding coordinates in the feature map. However, this approach caused the performance to drop a lot (AP drops 44% from 0.691 to 0.480), which justified the effectiveness of our proposed top-k selection. Lastly, we tuned the hyperparameter \(L\), which is the number of DA-decoding blocks, and we can find that setting more blocks results in better performance. Considering the computational complexity, we set \(L=4\) in practice.  
As illustrated in Figure 10, some failure cases exist. One category is missing detection. When one defect is located at the center or edge of a tire pattern, capturing distinctive geometric features is hard, leading to missing detection. Another category is wrong detection. In this case, our method is confused by some normal areas whose patterns are similar to those of tire defects. Although our detector failed to make correct predictions  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Methods** & **mAP** & **AP\({}_{50}\)** & **AP\({}_{75}\)** & **AP\({}_{S}\)** & **AP\({}_{M}\)** & **AP\({}_{L}\)** \\ \hline A: FPN [22] & 68.7 & 89.7 & 73.9 & 34.0 & **59.2** & 89.4 \\ B: DA-FPN & **69.1** & **89.8** & **74.1** & **35.7** & **59.2** & **89.6** \\ \hline C: Random selection & 48.0 & 86.0 & 50.0 & 7.1 & 48.0 & 59.0 \\ D: Top-k selection & **69.1** & **89.8** & **74.1** & **35.7** & **59.2** & **89.6** \\ \hline E: \(L=1\) & 24.9 & 59.0 & 14.7 & 0.0 & 15.6 & 31.3 \\ E: \(L=1\) & 49.3 & 86.0 & 50.1 & 7.2 & 47.3 & 60.0 \\ F: \(L=3\) & 58.4 & 87.0 & 73.2 & 30.1 & 58.1 & 85.7 \\ G: \(L=4\) & **69.1** & **89.8** & **74.1** & **35.7** & **59.2** & **89.6** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Ablation study of several components of our proposed DA-Detector.  
Figure 9: Bar charts of mAP on three datasets.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Methods** & **mAP** & **AP\({}_{50}\)** & **AP\({}_{75}\)** & **AP\({}_{S}\)** & **AP\({}_{M}\)** & **AP\({}_{L}\)** \\ \hline Ours & **40.9** & **72.0** & **38.6** & **8.2** & **30.1** & **62.2** \\ \hline \hline \end{tabular}
\end{table}
Table 3: _Cont._in the former case (i.e., missing detection), the geometric features of tire defects can be distinguished by human eyes, as shown in Figure 10. The performance of our model was constrained by the small dataset due to the lack of original data. Therefore, we aim to collect more defect data to promote the performance of our model. To tackle the latter failure case (i.e., wrong detection), we intend to devise a more robust rendering approach to extract more distinguishing geometric features.