# Point of Interest Recognition and Tracking in Aerial Video during Live Cycling Broadcasts
###### Abstract  
Road cycling races, such as the Tour de France, captivate millions of viewers globally, combining competitive sportsmanship with the promotion of regional landmarks. Traditionally, points of interest (POIs) are highlighted during broadcasts using manually created static overlays, a process that is both outdated and labor-intensive. This paper presents a novel, fully automated methodology for detecting and tracking POIs in live helicopter video streams, aiming to streamline the visualization workflow and enhance viewer engagement. Our approach integrates a saliency and Segment Anything-based technique to propose potential POI regions, which are then recognized using a keypoint matching method that requires only a few reference images. This system supports both automatic and semi-automatic operations, allowing video editors to intervene when necessary, thereby balancing automation with manual control. The proposed pipeline demonstrated high effectiveness, achieving over 75% precision and recall in POI detection, and offers two tracking solutions: a traditional MedianFlow tracker and an advanced SAM 2 tracker. While the former provides speed and simplicity, the latter delivers superior segmentation tracking, albeit with higher computational demands. Our findings suggest that this methodology significantly reduces manual workload and opens new possibilities for interactive visualizations, enhancing the live viewing experience of cycling races.  
landmark recognition; object tracking; computer vision; live processing; aerial video +
Footnote †: journal: Medicalapproach allows for the easy addition of new POIs in different races without requiring a retraining of the model, making it practical for real-world usage. Additionally, video editors still need the freedom to override the automatic pipeline. Hence, the system offers a semi-automatic workflow with manual input that can be used in case the automatic pipeline fails or when an unknown POI appears. This combination of both automatic and semi-automatic operation results in a good balance of reducing the manual workload while still providing the control and confidence to handle unexpected situations that happen during live broadcasts. Furthermore, the tracking of POIs across the video stream should be accurate and smooth to benefit the viewing experience. Finally, the live broadcast must remain uninterrupted by the processing pipeline. Therefore, the methodology is tailored for live usage, ensuring that every component works in real time or in a non-blocking manner. The contributions of this work can be summarized as follows:  
* A saliency and Segment Anything-based methodology to propose potential POI region masks;
* A keypoint matching-based POI recognition only requiring a few reference images per POI;
* A comparison of traditional and deep learning approaches for the tracking of POIs;
* A complete POI recognition and tracking system to process live helicopter video streams.  
The remainder of this paper is organized as follows: First, the related literature regarding landmark recognition and object tracking is discussed in Section 2. Next, Section 3 elaborates our methodology for point of interest recognition and tracking during live cycling broadcasts. Then, the results of the proposed approach are presented and discussed in Section 4, providing both quantitative and qualitative evaluations. Before concluding in Section 6, Section 5 explores additional applications of the methodology beyond the live usage scenario, highlighting its broader potential and versatility.