Title: GOT-Edit: Geometry-Aware Generic Object Tracking via Online Model Editing
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Generic Object Tracking, Model Editing, Model Prediction, Visual Geometry, Null-Space

Abstract: Human perception for effective object tracking in a 2D video stream arises from the implicit use of prior 3D knowledge combined with semantic reasoning.
In contrast, most generic object tracking (GOT) methods primarily rely on 2D features of the target and its surroundings while neglecting the 3D geometric cues, making them susceptible to occlusion, distractors, and large variations in geometry and appearance. 
To address this, we present GOT-Edit, a novel approach that integrates visual geometry knowledge into a generic object tracker through online model editing. 
Our approach first leverages the features from the Visual Geometry Grounded Transformer (VGGT), which is trained on large-scale 3D-annotated data and can provide reliable geometric cues directly from a set of 2D images.
To tackle the challenge of seamlessly combining geometry and semantics, we develop an online model editing strategy with a null-space constraint that adaptively fuses geometric information while preserving the tracker's learned semantic knowledge, yielding consistently better performance across diverse scenarios than a naive fusion of both cues.
Extensive experiments on multiple GOT benchmarks demonstrate that GOT-Edit achieves superior robustness and accuracy, particularly under occlusion and clutter, establishing a new paradigm for combining 2D semantics with 3D geometric reasoning for generic object tracking.

TL;DR: We develop an online model-editing strategy for a generic object tracker that adaptively fuses geometric information while preserving the semantic knowledge learned by the tracker.