HatEnthusiast/cv_assignment_3

Videos with detections and applied Kalman filter: Video 1: https://youtu.be/CYbvjG3W8mQ Video 2: https://youtu.be/ogdO6Lvd0Ok

Dataset choice: "Drone Computer Vision Dataset" https://universe.roboflow.com/drone-rwsrk/drone-cmxwz

Detector configuration: The pre-trained Ultralytics YOLO26n model served as the pre-trained backbone, with the dataset above serving as the fine-tuning. Most of the configuration leveraged the default settings to benefit from the included, robust data augmentation. However, a minimum confidence level of .4 was applied to reduce the false positives caused by birds in the distance. Additionally, the batch size was increased to 128 images per batch to take advantage of the larger GPU used during training.

Kalman filter state design and noise parameters: The Kalman filter was designed to track the coordinates and velocity of the drone across two dimensions. The noise parameters, described below in more detail, account for the implicit inaccuracies in the detection mechanism (i.e., the model), the process noise (i.e., the erratic movement of the drone), and how reliably the Kalman filter predictions should be viewed before several sequential detections. All other Kalman filter settings and values were default (i.e., they followed exactly the basic implementation described in the docs here: https://colab.research.google.com/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/08-Designing-Kalman-Filters.ipynb#scrollTo=DT1IW1smWpd5).

• R (measurement noise) = 9 pixels to account for the inconsistency in detection boundary box size, which would result in an inconsistent center point even when the drone was relatively stationary.
• Q (process noise) = 25 pixels to account for the quick, sometimes erratic movement of the drone. Both Q and R could have been increased further, but these values produced an acceptable result.
• P (beginning state variance) = 500 pixels to account for the very inaccurate starting estimate.

Failure cases and how the tracker handles missed detections: I implemented three systems to handle missed or multiple detections:

1. In instances of 10 or fewer consecutive missed detections, the Kalman filter uses its own previous prediction (aka "posterior") to predict the next frame. After 10 consecutive missed detections, the filter drops out to avoid diverting too far from the actual path of the drone and to prevent predicting the path of a drone that may not be in frame. Note that, for the sake of clarity, the detections parquet files include these frames in which the drone was not detected but the Kalman filter made a prediction but did include a Kalman filter prediction. These frames containing only a Kalman filter prediction were still excluded from the output videos per the instructions.
2. When the predicted coordinates are outside of the frame (i.e., the drone flew out of view of the camera), the velocity is divided by a constant integer, in this case 10. This prevents predicting positions that are so far out of frame as to be unreliable guesses, given the drone could and very likely would change course while out of frame.
3. In the instance of multiple detections within a single frame (caused in the example videos by birds), the detection with the higher confidence would be considered the latest sensor reading.

@software{yolo26_ultralytics, author = {Glenn Jocher and Jing Qiu}, title = {Ultralytics YOLO26}, version = {26.0.0}, year = {2026}, url = {https://github.com/ultralytics/ultralytics}, orcid = {0000-0001-5950-6979, 0000-0003-3783-7069}, license = {AGPL-3.0} }