# Ten Years of Active Learning Techniques and Object Detection: A Systematic Review
## 4 Active Learning and Object Detection: A Literature-Driven Multi-Perspective Analysis
### (RQ4) Query Strategies Taxonomy
#### Black box-based methods  
Classical query strategies adapted for OD, using the softmax layer output as a confidence metric, are considered black-box methods. In this case, the confidence from all objects belonging to an image is computed at the box level. Considering that the confidence of the prediction \(i\) that belongs to the class \(j\) is \(p_{i}^{j}\) and the number of classes is \(c\), the Least Confident (LC) method [60] selects the detected object instances whose prediction is the least confident by the model for each class (see Equation (3)). Alternatively, the Margin of Confidence (MC) method [60] considers the top two confidence rated objects of the same class \(j\) with the smallest margin (see Equation (4)). Entropy-based (E) method [46] selects the instance with the highest Uncertainty of belonging to a target class. This method is more exhaustive than LC and MC for the OD task, since it considers the uncertainty of all objects belonging to the same class (see Equation (5)). Another alternative for computing the uncertainty can be explored in [62].  
\[U_{j}=\arg\max p_{ij\in[1,\ldots,c]}^{j} \tag{3}\]  
\[U_{j}=\arg\max p_{ij\in[1,\ldots,c]}^{j}---\arg\max p_{kj\in[1,\ldots,c]/i}^{j} \tag{4}\]  
\[U_{j}=\sum-p_{i}^{j}\log p_{ij\in[1,\ldots,c]}^{j}. \tag{5}\]  
Regarding the design of sampling strategies, performing them for OD has a higher complexity than for classification tasks, since the impact of the background that affects the former constitutes an imbalance factor. In practice, this aspect reflects in the way that uncertainty is calculated using OD algorithms. Firstly, multiple boxes and their confidences are computed after an OD-based image processing, followed by the application of an aggregation method (see Equations (6)-(9)) combining the individual scores of each box-label pair to calculate the score of the entire image, in compliance with the recommendations of [45].  
\[U(I)=\sum_{i}^{c}U_{j} \tag{6}\]  
\[U(I)=\sum_{1}^{c}U_{j}/c \tag{7}\]  
\[U(I)=\min_{c}U_{j} \tag{8}\]  
\[U(I)=\max_{c}U_{j}. \tag{9}\]  
In [34], the authors evaluate LC sampling techniques using the YOLOv5 model according to the Federated Learning approach, more specifically, using unlabelled data available from its devices, considering sum, average, and maximum as aggregation methods. While the sum method depends on the detection count per image, the average method allows the comparison between images, and the maximum method deals with noisy detection [34; 62].  
Another black-box strategy is the query by committee, in which the disagreement of the predictions made from a consortium of models [37] is used to select unlabelled instances and, therefore, to try to reduce the generalisation error in further training sessions. Still on the same topic, three questions were posed in [79]:  
* How to build the committee?
* How to quantify the disagreement?
* How to combine the committee member responses to obtain a robust classification?  
In [62], a combination between a method of AL for OD and an incremental learning scheme were proposed, to enable continuous exploration of unlabelled datasets. The authors proposed a set of uncertainty-based AL \"metrics\" suitable for OD algorithms,which are computed from a DL model trained incrementally and an approach to leveraging class imbalances during sample selection over the PASCAL VOC 2012 dataset. To select unlabelled images from a pool, they compute, for each image, a detection score based on the confidence margin, considering the whole set of detections, which are aggregated as a sum into a unique image-score (Figure 9).