# Automated Car Damage Assessment Using Computer Vision: Insurance Company Use Case
## 3 Methodology
### Bounding Box Generation  
We defined a prediction box merging process with the objective of unifying the predictions generated by each model. The general box-merging algorithm is detailed in Algorithm 1. The first step consists of filtering out the predictions (list \(P\)) that do not exceed the confidence score established for the model that generated it. Subsequently, the predictions are ordered based on their confidence score. This is based on the premise that predictions with higher confidence scores tend to have a higher probability of being a correct prediction. After obtaining the list of predictions ordered by their confidence score values, \(Q\), we proceeded to iterate it and call the current one as the candidate prediction. Then, we obtained the predictions that intersect with the candidate prediction based on the value of Intersection over Union (IoU), which we call _IouThreshold_ and define as 0.5. Figure 2: YOLOv 5 generates variability in predictions according to the depth with which a model has been trained. Thus, by combining the predictions, it is possible to identify new areas of possible damage and to reinforce predictions that are repeated across several models. ```
1:Inputs:
2:\(P=[p_{1}...p_{n}]\)\(\triangleright\)\(n\) prediction boxes
3:\(S=[s_{1}...s_{m}]\)\(\triangleright\)\(m\) score thresholds, one per model
4:IouThreshold\(\triangleright\) IoU threshold
5:\(NumVotes\)\(\triangleright\) Minimum number of votes
6:Output:\(F\)\(\triangleright\) Final prediction boxes
7:\(P^{\prime}\leftarrow[]\)
8:for\(i\gets 1\ldots n\)do\(\triangleright\) Filter predictions by scores
9:\(j\gets model(p_{i})\)\(\triangleright\) Get model from prediction
10:if\(score(p_{i})\geq s_{i}\)then\(\triangleright\) Score is greater than model's threshold
11:\(P^{\prime}\gets concat(P^{\prime},p_{i})\)\(\triangleright\) Add box for next step
12:endif
13:endfor
14:\(Q\gets sort(P^{\prime},score)\)\(\triangleright\) Sort predictions by scores
15:while\(size(Q)\
eq 0\)do\(\triangleright\) Merge boxes
16:\(votes\gets 0\)
17:\(candidate\gets Q[1]\)\(\triangleright\) Get first element of Q
18:\(B\leftarrow[candidate]\)\(\triangleright\) For contributing boxes
19:\(Q^{\prime}\leftarrow[]\)\(\triangleright\) For remaining boxes
20:for\(i\gets 2\ldots size(Q)\)do\(\triangleright\) Get intersections
21:if\(IOU(candidate,Q[i])>\)IouThresholdthen
22:\(B\gets concat(B,Q[i])\)\(\triangleright\) Accumulate votes
23:\(votes\gets votes+1\)
24:else
25:\(Q^{\prime}\gets concat(Q^{\prime},Q[i])\)\(\triangleright\) Keep no intersects
26:endif
27:endfor
28:\(Q\gets Q^{\prime}\)\(\triangleright\) Intersections were deleted
29:if\(votes\geq NumVotes\)then
30:\(boxMaxArea\gets getBoxWithMaxArea(B)\)
31:\(F\gets concat(F,boxMaxArea)\)\(\triangleright\) Add box to \(F\)
32:endif
33:endwhile
34:return\(F\)
```  
**Algorithm 1** Box fusion pseudocode  
Each prediction intersecting with the candidate prediction (also known as the cluster) will contribute with a vote. If the number of votes surpasses the defined variable \(NumVotes\), then we will proceed to compute the best prediction box that binds all of them together. This is \(boxMaxArea\), and it has the largest area including all the boxes. Finally, we add it to our final list of predictions \(F\) and remove all predictions that intercepted with the candidate prediction by updating the list \(Q\). To identify the optimal minimum number of votes, we performed several iterations with the PSO algorithm, allowing us to set \(NumVotes\) to 9, achieving a balance between correct predictions and false positives. To address the scenario where different types of damage overlap, we define each prediction as a general damage and classify them using a previously trained classifier to identify the type of damage. Figure 3 exemplifies the results of applying prediction fusion to different scenarios. We use as an example four vehicles with different damages.