OpenRaiser/Intern-Atlas huggingface dataset:
A Unified Framework for Tracking through Occlusions and across Sensor Gaps
Authors: Robert Kaucic, A. G. Amitha Perera, Glen Brooksby, John P. Kaufhold, Anthony Hoogs
Venue: CVPR
Year: 2005

A common difficulty encountered in tracking applications is how to track an object that becomes totally occluded, possibly for a significant period of time. Another problem is how to associate objects, or tracklets, across non-overlapping cameras, or between observations of a moving sensor that switches fields of regard. A third problem is how to update appearance models for tracked objects over time. As opposed to using a comprehensive multi-object tracker that must simultaneously deal with these tracking challenges, we present a novel, modular framework that handles each of these problems in a unified manner by the initialization, tracking, and linking of high-confidence tracklets. In this track/suspend/match paradigm, we first analyze the scene to identify areas where tracked objects are likely to become occluded. Tracking is then suspended on occluded objects and re-initiated when they emerge from behind the occlusion. We then associate, or match, suspended tracklets with the new tracklets using full kinematic models for object motion and Gibbsian distributions for object appearance in order to complete the track through the occlusion. Sensor gaps are handled in a similar manner, where tracking is suspended when the sensor looks away and then re-initiated when the sensor returns. Changes in object appearance and orientation during tracking are also seamlessly handled in this framework. Tracklets with low lock scores are terminated. Tracking then resumes on untracked movers with corresponding updated appearance models. These new tracklets are then linked back to the terminated ones as appropriate. Fully automatic tracking results from a moving sensor are presented.
Computer Vision for Music Identification: Video Demonstration
Authors: Yan Ke, Derek Hoiem, Rahul Sukthankar
Venue: CVPR
Year: 2005

This paper describes a demonstration video for our music identification system. The goal of music identification is to reliably recognize a song from a small sample of noisy audio. This problem is challenging because the recording is often corrupted by noise and because the audio sample will only match a small portion of the target song. Additionally, a practical music identification system should scale (in both accuracy and speed) to databases containing hundreds of thousands of songs. Recently, the music identification problem has attracted considerable attention. However, the task remains unsolved, particularly for noisy real-world queries. We cast music identification into an equivalent sub-image retrieval framework: identify the portion of a spectrogram image from the database that best matches a given query snippet. Our approach treats the spectrogram of each music clip as a 2D image and transforms music identification into a corrupted sub-image retrieval problem.
Full Body Tracking from Multiple Views Using Stochastic Sampling
Authors: Roland Kehl, Matthieu Bray, Luc Van Gool
Venue: CVPR
Year: 2005

We present a novel approach for full body pose tracking using stochastic sampling. A volumetric reconstruction of a person is extracted from silhouettes in multiple video images. Then, an articulated body model is fitted to the data with stochastic meta descent (SMD) optimization. By comparing even a simplified version of SMD to the commonly used Levenberg-Marquardt method, we demonstrate the power of stochastic compared to deterministic sampling, especially in cases of noisy and incomplete data. Moreover, color information is added to improve the speed and robustness of the tracking. Results are shown for several challenging sequences, with tracking of 24 degrees of freedom in less than 1 second per frame.
Algebraically Accurate Volume Registration Using Euler's Theorem and the 3-D Pseudo-Polar FFT
Authors: Yosi Keller, Amir Averbuch, Yoel Shkolnisky
Venue: CVPR
Year: 2005

We present an algorithm for the registration of rotated and translated volumes, which operates in the frequency domain. The Fourier domain allows to compute the rotation and translation parameters separately, thus reducing a problem with six degrees of freedom to two problems of three degrees of freedom each. We propose a three-step procedure. The first step estimates the rotation axis. The second computes the planar rotation relative to the rotation axis, and the third recovers the translational displacement by using the phase correlation technique. The rotation estimation is based on Euler's theorem, which allows to represent a rotation using only three parameters. Two parameters represent the rotation axis and one parameter represents the planar rotation perpendicular to the axis. By using the 3D pseudo-polar FFT, the estimation of the rotation axis is shown to be algebraically accurate. A variant of the angular difference function registration algorithm is derived for the estimation of the planar rotation around the axis. The experimental results show that the algorithm is accurate and robust to noise.
Indexing with Unknown Illumination and Pose
Authors: Ira Kemelmacher, Ronen Basri
Venue: CVPR
Year: 2005

The task of identifying 3D objects in 2D images is difficult due to variation in objects' appearance with changes in pose and lighting. The task is further complicated by the presence of occlusion and clutter. Shape indexing is a method for rapid association between features identified in an image and their corresponding 3D features stored in a database. Previous indexing methods ignored variations due to lighting, restricting the approach to polyhedral objects. In this paper, we further develop these methods to handle variations in both pose and lighting. We focus on rigid objects undergoing a scaled-orthographic projection and use spherical harmonics to represent lighting. The resulting integrated algorithm can recognize 3D objects from a single input image; furthermore, it recovers the pose and lighting of each familiar object in the given image. The algorithm has been tested on a database of real objects, demonstrating its performance on cluttered scenes under a variety of poses and illumination conditions.
An Axiomatic Approach to Corner Detection
Authors: Charles S. Kenney, Marco Zuliani, B. S. Manjunath
Venue: CVPR
Year: 2005

This paper presents an axiomatic approach to corner detection. In the first part of the paper we review five currently used corner detection methods (Harris-Stephens, Forstner, Shi-Tomasi, Rohr, and Kenney et al ) for graylevel images. This is followed by a discussion of extending these corner detectors to images with different pixel dimensions such as signals (pixel dimension one) and tomographic medical images (pixel dimension three) as well as different intensity dimensions such as color or LADAR images (intensity dimension three). These extensions are motivated by analyzing a particular example of optical flow in pixel and intensity space with arbitrary dimensions. Placing corner detection in a general setting enables us to state four axioms that any corner detector might reasonably be required to satisfy. Our main result is that only the Shi-Tomasi (and equivalently the Kenney et al. 2-norm detector) satisfy all four of the axioms.
Multitarget Tracking with Split and Merged Measurements
Authors: Zia Khan, Tucker R. Balch, Frank Dellaert
Venue: CVPR
Year: 2005

In many multitarget tracking applications in computer vision, a detection algorithm provides locations of potential targets. Subsequently, the measurements are associated with previously estimated target trajectories in a data association step. The output of the detector is often imperfect and the detection data may include multiple, split measurements from a single target or a single merged measurement from several targets. To address this problem, we introduce a multiple hypothesis tracker for interacting targets that generate split and merged measurements. The tracker is based on an efficient Markov chain Monte Carlo (MCMC) based auxiliary variable particle filter. The particle filter is Rao-Blackwellized such that the continuous target state parameters are estimated analytically, and an MCMC sampler generates samples from the large discrete space of data associations. In addition, we include experimental results in a scenario where we track several interacting targets that generate these split and merged measurements.
Pixels that Sound
Authors: Einat Kidron, Yoav Y. Schechner, Michael Elad
Venue: CVPR
Year: 2005

People and animals fuse auditory and visual information to obtain robust perception. A particular benefit of such cross-modal analysis is the ability to localize visual events associated with sound sources. We aim to achieve this using computer-vision aided by a single microphone. Past efforts encountered problems stemming from the huge gap between the dimensions involved and the available data. This has led to solutions suffering from low spatio-temporal resolutions. We present a rigorous analysis of the fundamental problems associated with this task. Then, we present a stable and robust algorithm which overcomes past deficiencies. It grasps dynamic audio-visual events with high spatial resolution, and derives a unique solution. The algorithm effectively detects pixels that are associated with the sound, while filtering out other dynamic pixels. It is based on canonical correlation analysis (CCA), where we remove inherent ill-posedness by exploiting the typical spatial sparsity of audio-visual events. The algorithm is simple and efficient thanks to its reliance on linear programming and is free of user-defined parameters. To quantitatively assess the performance, we devise a localization criterion. The algorithm capabilities were demonstrated in experiments, where it overcame substantial visual distractions and audio noise.
A Dense Stereo Matching Using Two-Pass Dynamic Programming with Generalized Ground Control Points
Authors: Jae-Chul Kim, Kyoung Mu Lee, Byoung-Tae Choi, Sang Uk Lee
Venue: CVPR
Year: 2005

A method for solving dense stereo matching problem is presented in this paper. First, a new generalized ground control points (GGCPs) scheme is introduced, where one or more disparity candidates for the true disparity of each pixel are assigned by local matching using the oriented spatial filters. By allowing "all" pixels to have multiple candidates for their true disparities, GGCPs not only guarantee to provide a sufficient number of starting pixels needed for guiding the subsequent matching process, but also remarkably reduce the risk of false match, improving the previous GCP-based approaches where the number of the selected control points tends to be inversely proportional to the reliability. Second, by employing a two-pass dynamic programming technique that performs optimization both along and across the scanlines, we solve the typical inter-scanline inconsistency problem. Moreover, combined with the GGCPs, the stability and efficiency of the optimization are improved significantly. Experimental results for the standard data sets show that the proposed algorithm achieves comparable results to the state-of-the-arts with much less computational cost.
Skeletal Parameter Estimation from Optical Motion Capture Data
Authors: Adam G. Kirk, James F. O&apos, Brien, David A. Forsyth
Venue: CVPR
Year: 2005

In this paper we present an algorithm for automatically estimating a subject's skeletal structure from optical motion capture data. Our algorithm consists of a series of steps that cluster markers into segment groups, determine the topological connectivity between these groups, and locate the positions of their connecting joints. Our problem formulation makes use of fundamental distance constraints that must hold for markers attached to an articulated structure, and we solve the resulting systems using a combination of spectral clustering and nonlinear optimization. We have tested our algorithms using data from both passive and active optical motion capture devices. Our results show that the system works reliably even with as few as one or two markers on each segment. For data recorded from human subjects, the system determines the correct topology and qualitatively accurate structure. Tests with a mechanical calibration linkage demonstrate errors for inferred segment lengths on average of only two percent. We discuss applications of our methods for commercial human figure animation, and for identifying human or animal subjects based on their motion independent of marker placement or feature selection.
Skeletal Parameter Estimation from Optical Motion Capture Data
Authors: Adam G. Kirk, James F. O&apos, Brien, David A. Forsyth
Venue: CVPR
Year: 2005

In this paper, we present results of our algorithm for automatically estimating a subject's skeletal structure from optical motion capture data. Our algorithm consists of a series of steps that cluster markers into groups representing body segments, determine their topological connectivity, and locate the positions of the connecting joints. Our results show that the system works reliably even when only one or two markers are attached to each segment. We tested an implementation of this algorithm with both passive and active motion capture data and found it to work well. Its computed skeletal estimates closely match measured values, and the algorithm behaves robustly in the presence of noise, marker occlusion, and other errors typical of motion capture data.
Hand Tracking with Flocks of Features
Authors: Mathias Kölsch, Matthew Turk 0001
Venue: CVPR
Year: 2005

Tracking hands in live video is a challenging task: the hand appearance can change too rapidly for appearance-based trackers to work, and color-based trackers (that do not rely on geometry) have to make limiting assumptions about the background color. This article shows the results of hand tracking with "Flocks of Features", a tracking method that combines motion cues and a learned foreground color distribution to achieve fast and robust 2D tracking of highly articulated objects. Many independent image artifacts are tracked from one frame to the next, adhering only to local constraints. This concept is borrowed from nature since these tracks mimic the flight of flocking birds - exhibiting local individualism and variability while maintaining a clustered entirety. Hand tracking has important applications for interaction with wearable computers, for intuitive manipulation of virtual objects, for detection of activity signatures, and much more. Tracking with Flocks of Features is not limited to hands - any articulated or appearance-changing object can benefit from this multi-cue tracking method.
Scene-Adapted Structured Light
Authors: Thomas P. Koninckx, Pieter Peers, Philip Dutré, Luc Van Gool
Venue: CVPR
Year: 2005

In order to overcome several limitations of structured light 3D acquisition methods, the colors, intensities, and shapes of the projected patterns are adapted to the scene. Based on a crude estimate of the scene geometry and reflectance characteristics, the local intensity ranges in the projected patterns are adapted, in order to avoid over- and under-exposure in the image. This avoids the infamous specularity problems and generally increases accuracy. The estimated geometry also helps to limit the effect of aliasing caused by the sampling of foreshortened patterns. Furthermore, the approach also accounts for the adverse effects that small motions during scanning would normally have. Moreover, the approach yields a confidence measure at every pixel of the range image. Last but not least, the scanner consists of consumer products only, and therefore is cheap.
Speckle-Constrained Filtering of Ultrasound Images
Authors: Karl Krissian, Ron Kikinis, Carl-Fredrik Westin, Kirby G. Vosburgh
Venue: CVPR
Year: 2005

Ultrasound images provide the clinician with non-invasive, low cost, and real-time images that can help them in diagnosis, planning and therapy. However, although the human eye is able to derive the meaningful information from these images, automatic processing is very difficult because of the noise and artefacts present in the image. In this work, we propose to extend the current anisotropic diffusion technique to deal with the speckle noise present in the Ultrasound images. To this end, we use a previously derived model of the noise, and we write the restoration scheme as a energy minimization constrained by the noise model and parameters. This approach leads to a new data attachment term whose optimal weight can be automatically estimated.
OBJ CUT
Authors: M. Pawan Kumar, Philip H. S. Torr, Andrew Zisserman
Venue: CVPR
Year: 2005

In this paper, we present a principled Bayesian method for detecting and segmenting instances of a particular object category within an image, providing a coherent methodology for combining top down and bottom up cues. The work draws together two powerful formulations: pictorial structures (PS) and Markov random fields (MRFs) both of which have efficient algorithms for their solution. The resulting combination, which we call the object category specific MRF, suggests a solution to the problem that has long dogged MRFs namely that they provide a poor prior for specific shapes. In contrast, our model provides a prior that is global across the image plane using the PS. We develop an efficient method, OBJ CUT, to obtain segmentations using this model. Novel aspects of this method include an efficient algorithm for sampling the PS model, and the observation that the expected log likelihood of the model can be increased by a single graph cut. Results are presented on two object categories, cows and horses. We compare our methods to the state of the art in object category specific image segmentation and demonstrate significant improvements.
Optimal Sub-Shape Models by Minimum Description Length
Authors: Georg Langs, Philipp Peloschek, Horst Bischof
Venue: CVPR
Year: 2005

Active shape models are powerful and widely used tool to interpret complex image data. By building models of shape variation they enable search algorithms to use a priori knowledge in an efficient and gainful way. However, due to the linearity of PCA, non-linearities like rotations or independently moving sub-parts in the data can deteriorate the resulting model considerably. Although non-linear extensions of active shape models have been proposed and application specific solutions have been used, they still need a certain amount of user interaction during model building. In this paper the task of building/choosing optimal models is tackled in a more generic information theoretic fashion. In particular, we propose an algorithm based on the minimum description length principle to find an optimal subdivision of the data into sub-parts, each adequate for linear modeling. This results in an overall more compact model configuration. Which in turn leads to a better model in terms of modes of variations. The proposed method is evaluated on synthetic data, medical images and hand contours.
Hybrid Joint-Separable Multibody Tracking
Authors: Oswald Lanz, Roberto Manduchi
Venue: CVPR
Year: 2005

Statistical models for tracking different moving bodies must be able to reason about occlusions in order to be effective. Representing the joint statistics across different bodies is computationally hard, since the size of the representation grows exponentially with the number of bodies being tracked. Separable tracking, with one tracker per body, cannot deal with occlusions effectively. We propose a new model, dubbed Hybrid Joint-Separable (HJS), that uses a representation size that grows linearly with the number of bodies, and a computational complexity that grows quadratically. This model can reason explicitly about occlusions. We describe a particle filter implementation of this model, and present promising experimental results.
Online Learning of Probabilistic Appearance Manifolds for Video-Based Recognition and Tracking
Authors: Kuang-Chih Lee, David J. Kriegman
Venue: CVPR
Year: 2005

This paper presents an online learning algorithm to construct from video sequences an image-based representation that is useful for recognition and tracking. For a class of objects (e.g., human faces), a generic representation of the appearances of the class is learned off-line. From video of an instance of this class (e.g., a particular person), an appearance model is incrementally learned on-line using the prior generic model and successive frames from the video. More specifically, both the generic and individual appearances are represented as an appearance manifold that is approximated by a collection of sub-manifolds (named pose manifolds) and the connectivity between them. In turn, each sub-manifold is approximated by a low-dimensional linear sub-space while the connectivity is modeled by transition probabilities between pairs of sub-manifolds. We demonstrate that our online learning algorithm constructs an effective representation for face tracking, and its use in video-based face recognition compares favorably to the representation constructed with a batch technique.
Pedestrian Detection in Crowded Scenes
Authors: Bastian Leibe, Edgar Seemann, Bernt Schiele
Venue: CVPR
Year: 2005

In this paper, we address the problem of detecting pedestrians in crowded real-world scenes with severe overlaps. Our basic premise is that this problem is too difficult for any type of model or feature alone. Instead, we present an algorithm that integrates evidence in multiple iterations and from different sources. The core part of our method is the combination of local and global cues via probabilistic top-down segmentation. Altogether, this approach allows examining and comparing object hypotheses with high precision down to the pixel level. Qualitative and quantitative results on a large data set confirm that our method is able to reliably detect pedestrians in crowded scenes, even when they overlap and partially occlude each other. In addition, the flexible nature of our approach allows it to operate on very small training sets.
Unsupervised Learning of Object Features from Video Sequences
Authors: Marius Leordeanu, Robert T. Collins
Venue: CVPR
Year: 2005

We develop an efficient algorithm for unsupervised learning of object models as constellations of features, from low resolution video sequences. The input images typically contain single or multiple objects that change in pose, scale and degree of occlusion. Also, the objects can move significantly between consecutive frames. The content of an input sequence is unlabeled so the learner has to cluster the data based on the data's implicit coherence over time and space. Our approach takes advantage of the dependent pairwise co-occurrences of objects' features within local neighborhoods vs. the independent behavior of unrelated features. We couple or decouple pairs of features based on a probabilistic interpretation of their pairwise statistics and then extract objects as connected components of features.
Learning a Multi-Size Patch-Based Hybrid Kernel Machine Ensemble for Abnormal Region Detection in Colonoscopic Images
Authors: Peng Li, Kap Luk Chan, Shankar Muthu Krishnan
Venue: CVPR
Year: 2005

When detecting abnormalities in colonoscopic images, the location, shape and size of the abnormal regions in the image are unknown and vary across images. It is difficult to determine the appropriate patch-size for patch-based approach. So multi-size patches are used simultaneously to represent the image regions and an ensemble is constructed in which each classifier handles one patch size. The combination of classifiers trained using multiple-size patches can recognize abnormal regions more effectively than only using single-size patches. The classification of the image patches can be performed using a discriminative binary support vector machine (SVM) or a recognition-based one-class SVM. Integration of the two types of SVMs is expected to further improve abnormal region detection. Experimental results show the good performance of our proposed ensemble.