Original: Box Prediction Rebalancing for Training Single-Stage Object Detectors with Partially Labeled Data
Revised: Box Prediction Rebalancing for Training Single-Stage Object Detectors with Partially Labeled Data

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Partial labeling schemes, in which annotators may label some instances of classesof interest and not label other instances, can significantly reduce annotation budgetsand enable machine learning algorithms that might otherwise be impossible.
Revised: Partial labeling schemes, in which annotators may label some instances of classes of interest and not label other instances, can significantly reduce annotation budgets and enable machine learning algorithms that might otherwise be impossible.

Original: However, these schemes introduce noise that makes training machine learning modelsdifficult.
Revised: However, these schemes introduce noise that makes training machine learning models difficult.

Original: The Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA)uses a partial labeling scheme for its training set, which consists of thousands ofpartially labeled video frames.
Revised: The Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA) uses a partial labeling scheme for its training set, which consists of thousands of partially labeled video frames.

Original: To combat the challenge of training on partiallylabeled data, we propose Box Prediction Rebalancing for single-stage object detectors and test our method on YOLOv5, a state-of-the-art single-stage detector.
Revised: To combat the challenge of training on partially labeled data, we propose Box Prediction Rebalancing for single-stage object detectors and test our method on YOLOv5, a state-of-the-art single-stage detector.

Original: We rebalance the percentage of positive and negative detections included in theloss computation of the end-to-end model, improving our model’s performanceand generalizability.
Revised: We rebalance the percentage of positive and negative detections included in the loss computation of the end-to-end model, improving our model’s performance and generalizability.

Original:  Introduction
Revised:  Introduction

Original: In large-scale, real-world object detection datasets, partially labeled annotations are important as theyallow annotators to be more efficient.
Revised: In large-scale, real-world object detection datasets, partially labeled annotations are important as they allow annotators to be more efficient.

Original: This is prevalent in DUSIA, a public dataset containing videosof marine invertebrate species and habitats, with the goal of detecting, classifying, and countingthese species.
Revised: This is prevalent in DUSIA, a public dataset containing videos of marine invertebrate species and habitats, with the goal of detecting, classifying, and counting these species.

Original: By applying machine learning methods and increasing performances on DUSIA, wecan further the effort for advancement in computationally analyzing videos for efficiency from amarine scientist setting.
Revised: By applying machine learning methods and increasing performances on DUSIA, we can further the effort for advancement in computationally analyzing videos for efficiency from a marine scientist setting.

Original: Among the frames from DUSIA’s training set that are labeled with boundingboxes, it is not guaranteed all individuals of each species of interest have bounding box labels, but allexisting bounding box labels are accurately labeled.
Revised: Among the frames from DUSIA’s training set that are labeled with bounding boxes, it is not guaranteed all individuals of each species of interest have bounding box labels, but all existing bounding box labels are accurately labeled.

Original: However, the validation and testing frames arefully annotated, labeling all species of interest, enabling effective evaluation for object detectors.
Revised: However, the validation and testing frames are fully annotated, labeling all species of interest, enabling effective evaluation for object detectors.

Original: McEver et al.
Revised: McEver et al.

Original: [2] propose their novel Context Driven Detector, a derivative of Faster R-CNN, todetect the marine invertebrates.
Revised: [2] propose their novel Context Driven Detector, a derivative of Faster R-CNN, to detect the marine invertebrates.

Original: However, we propose the use of YOLOv5 [1] due to its strongerability to detect small objects while also scanning images at a much higher frame rate.
Revised: However, we propose the use of YOLOv5 [1] due to its stronger ability to detect small objects while also scanning images at a much higher frame rate.

Original: To enhance training on a partially annotated set, McEver et al.
Revised: To enhance training on a partially annotated set, McEver et al.

Original: [2] introduce a novel negative regiondropping method, in which proposals from Faster R-CNN’s [4] Region Proposal Network (RPN)that do not cover an object with ground truth are dropped from the loss calculation of the RPN partof the two-stage detector.
Revised: [2] introduce a novel negative region dropping method, in which proposals from Faster R-CNN’s [4] Region Proposal Network (RPN) that do not cover an object with ground truth are dropped from the loss calculation of the RPN part of the two-stage detector.

Original: Because the model may be detecting objects in training frames that areactually species, but have no ground truth, by dropping negative proposals, the model learns to paymore attention to true positive labels.
Revised: Because the model may be detecting objects in training frames that are actually species, but have no ground truth, by dropping negative proposals, the model learns to pay more attention to true positive labels.

Original: Building off the ideas of negative region dropping, we propose Box Prediction Rebalancing, a novelmethod for single-stage object detectors that takes advantage of the multi-step loss function of YOLOoriginally introduced by [3] but adapted by Ultralytics [1].
Revised: Building off the ideas of negative region dropping, we propose Box Prediction Rebalancing, a novel method for single-stage object detectors that takes advantage of the multi-step loss function of YOLO originally introduced by [3] but adapted by Ultralytics [1].

Original: Our method is applied to both YOLOv5’sbox regression and objectness loss and showed to boost the model’s performance and generalizabilityon DUSIA.
Revised: Our method is applied to both YOLOv5’s box regression and objectness loss and showed to boost the model’s performance and generalizability on DUSIA.

Original: [Figure]
Revised: [Figure]

Original:  Methods
Revised:  Methods

Original: YOLO differs from Faster R-CNN in that the model does not contain an RPN.
Revised: YOLO differs from Faster R-CNN in that the model does not contain an RPN.

Original: YOLO models aretherefore single-stage, end-to-end networks.
Revised: YOLO models are therefore single-stage, end-to-end networks.

Original: YOLOv5’s loss function consists of three parts: boxregression, objectness, and classification loss.
Revised: YOLOv5’s loss function consists of three parts: box regression, objectness, and classification loss.

Original: The box regression and objectness loss update basedon a tensor of the Intersection over Union (IoU) from the final predictions with the ground truth labelsfor a given training batch fed into the model.
Revised: The box regression and objectness loss update based on a tensor of the Intersection over Union (IoU) from the final predictions with the ground truth labels for a given training batch fed into the model.

Original: In Faster R-CNN, Ren et al.
Revised: In Faster R-CNN, Ren et al.

Original: [4] define a negative proposal as a proposal box with less than 0.3 IoUwith some ground truth labels.
Revised: [4] define a negative proposal as a proposal box with less than 0.3 IoU with some ground truth labels.

Original: We adapt this definition from region proposals directly to YOLO’sdetection during training, and randomly drop percentages of negative boxes from the tensor of IoUs,which is then used to update the loss.
Revised: We adapt this definition from region proposals directly to YOLO’s detection during training, and randomly drop percentages of negative boxes from the tensor of IoUs, which is then used to update the loss.

Original: YOLOv5 calculates the box regression loss based on the IoUs of the final predictions.
Revised: YOLOv5 calculates the box regression loss based on the IoUs of the final predictions.

Original: By randomlyremoving a percentage of boxes that have low IoU with ground truth boxes (i.e. negative detections)and rebalancing the box predictions, we incline the model to update the box regression loss less fromnegative predictions, as they may, in fact, be organisms that simply do not have a ground truth label.
Revised: By randomly removing a percentage of boxes that have low IoU with ground truth boxes (i.e. negative detections) and rebalancing the box predictions, we incline the model to update the box regression loss less from negative predictions, as they may, in fact, be organisms that simply do not have a ground truth label.

Original: YOLOv5 uses a Binary Cross Entropy Objectness loss, which also uses the IoU values from the finalpredictions.
Revised: YOLOv5 uses a Binary Cross Entropy Objectness loss, which also uses the IoU values from the final predictions.

Original: Objectness is extremely important to our problem, as it is the probability the model thinksan object is there.
Revised: Objectness is extremely important to our problem, as it is the probability the model thinks an object is there.

Original: So if there are species without labels, the model will penalize itself unnecessarily.
Revised: So if there are species without labels, the model will penalize itself unnecessarily.

Original: We use the same process of randomly removing a percentage of negative predictions from the tensorof IoUs for predictions in a batch, and by modifying the inputs for objectness loss, we see the greatestimprovement in Mean Average Precision (mAP).
Revised: We use the same process of randomly removing a percentage of negative predictions from the tensor of IoUs for predictions in a batch, and by modifying the inputs for objectness loss, we see the greatest improvement in Mean Average Precision (mAP).

Original:  Experiments
Revised:  Experiments

Original: We run two separate experiments, where we test rebalancing the box predictions as outlined before.
Revised: We run two separate experiments, where we test rebalancing the box predictions as outlined before.

Original: One experiment demonstrates the impact of removing negative detections from the box regressionloss computation, and the other shows the impact of removing negative detections from the objectnessloss.
Revised: One experiment demonstrates the impact of removing negative detections from the box regression loss computation, and the other shows the impact of removing negative detections from the objectness loss.

Original: As an indicator of performance for our experiments, we use mAP with an IoU threshold of 0.5.
Revised: As an indicator of performance for our experiments, we use mAP with an IoU threshold of 0.5.

Original: We start by tuning the hyperparameters of the YOLOv5 model on DUSIA.
Revised: We start by tuning the hyperparameters of the YOLOv5 model on DUSIA.

Original: We find the optimalhyperparameters without box rebalancing and use these same hyperparameters for all experiments.
Revised: We find the optimal hyperparameters without box rebalancing and use these same hyperparameters for all experiments.

Original: The default YOLOv5 model outperformed the original out-of-the-box Faster R-CNN tested by McEveret al.
Revised: The default YOLOv5 model outperformed the original out-of-the-box Faster R-CNN tested by McEver et al.

Original:  We then experiment with various rebalancing percentages for both the box regressionand objectness loss, as described by Table 1 and Table 2.
Revised: [2] as shown in Table 1. We then experiment with various rebalancing percentages for both the box regression and objectness loss, as described by Table 2 and Table 3.

Original: By randomly removing percentages ofpredictions from only the box regression loss, the validation mAP did not improve, however, the testmAP did increase by 0.7% at 25 percent removed.
Revised: By randomly removing percentages of predictions from only the box regression loss, the validation mAP did not improve, however, the test mAP did increase by 0.7% at 25 percent removed.

Original: In contrast, removing percentages of negativepredictions from the objectness loss increased the test mAP at all percentages tested except for 25percent, which neither increased nor decreased the test mAP.
Revised: In contrast, removing percentages of negative predictions from the objectness loss increased the test mAP at all percentages tested except for 25 percent, which neither increased nor decreased the test mAP.

Original: By dropping 75% of negative predictions,the val mAP increased by 0.6% and the test mAP increased by 1.5%.
Revised: By dropping 75% of negative predictions, the val mAP increased by 0.6% and the test mAP increased by 1.5%.

Original:  [Table]
Revised: [Equation] [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Overall, box prediction rebalancing for our single-stage detector gave the model stronger generalizability, as it showed to increase test mAP from both the box and objectness loss experiments.
Revised: Overall, box prediction rebalancing for our single-stage detector gave the model stronger generalizability, as it showed to increase test mAP from both the box and objectness loss experiments.

Original:  Conclusion
Revised:  Conclusion

Original: We propose Box Prediction Rebalancing for single-stage object detectors to combat the challenge oflearning from partially annotated datasets.
Revised: We propose Box Prediction Rebalancing for single-stage object detectors to combat the challenge of learning from partially annotated datasets.

Original: By randomly removing percentages of negative predictionsfrom YOLOv5’s box regression and objectness loss, our model performs better as it will learn lessfrom negative boxes which may be true species of interest without ground truth labels.
Revised: By randomly removing percentages of negative predictions from YOLOv5’s box regression and objectness loss, our model performs better as it will learn less from negative boxes which may be true species of interest without ground truth labels.

Original: In the future,we hope to experiment with the effects of rebalancing predictions from both the box and objectnessloss at the same time and also experiment with selectively targeting negative predictions to remove,instead of randomly removing percentages.
Revised: In the future, we hope to experiment with the effects of rebalancing predictions from both the box and objectness loss at the same time and also experiment with selectively targeting negative predictions to remove, instead of randomly removing percentages.