Layered tracker switching for visual surveillance
Authors: James W. Davis
Venue:  (2008)

Surveillance and monitoring are two important applications of computer vision research. Many computer vision algorithms have been proposed to monitor both indoor and outdoor spaces using one or more visual sensors. Finding the location of objects (detection) in complex real-world scenes and following their position over time (tracking) are two principal tasks for these systems. In this thesis, we develop a computational framework for object tracking that layers multiple sources of information (such as position, velocity, and appearance) by automatically selecting the most appropriate tracking algorithm based on the given scene context. The proposed framework is independent of the choice of tracking algorithms used and is capable of automatically evaluating the spatial context based on object interactions in the scene. Our approach is to employ multiple trackers that have complimentary modes of success and failures. Also, these tracking algorithms range from simple to complex in terms of computation. We do not always need to deploy the most expensive tracker in all cases.Furthermore, we also propose a set of novel computer vision algorithms for tracking objects in both 2D and 3D spaces. These algorithms address various shortcomings of the existing tracking approaches such as occlusion reasoning, information fusion, model update strategies, execution speeds, etc. The proposed algorithms are deployed in the aforementioned layered tracker switching framework, resulting in robust and efficient paradigm for tracking objects in complex urban environments.First, we present an online, recursive filtering technique to model linear dynamical systems that operate on the state space of symmetric positive definite (SPD) matrices that lie on a Riemannian manifold . This filtering approach is applied to the problem of object tracking by recursively estimating and updating the SPD covariance feature matrices representing objects in the scene. The online filtering process on the Riemannian manifold is used to recursively obtain the smoothed object location and the updated appearance model simultaneously. In addition, we also present a set of efficient optimizations to enhance covariance tracking algorithms that employ covariance matrices of image features to represent object appearance. The proposed optimization techniques improve the tracking accuracy while reducing execution times.Next we describe a 3D Kernel-based appearance tracking algorithm that combines information from multiple camera views to track an object directly in the 3D space. Its novelty lies in the proposed unified approach to 3D kernel tracking, that amounts to fusing the appearance features from all available camera sensors, as opposed to tracking the object appearance in the individual 2D views and fusing the results. The elegance of the method resides in its inherent ability to handle problems encountered by various 2D trackers, including scale selection, occlusion, view-dependence, and correspondence across different views.Finally, we demonstrate the applicability of the proposed tracker switching framework by developing practical systems for 2D and 3D object tracking. The layered tracking framework benefits by dynamically switching between simpler position and dynamics based algorithms and the advanced appearance based algorithms developed in this thesis. The net result is an improvement in tracking performance and efficiency.