OpenRaiser/Intern-Atlas huggingface dataset:
Detection and Explanation of Anomalous Activities: Representing Activities as Bags of Event n-Grams
Authors: Raffay Hamid, Amos Y. Johnson, Samir Batta, Aaron F. Bobick, Charles L. Isbell Jr., Graham Coleman
Venue: CVPR
Year: 2005

We present a novel representation and method for detecting and explaining anomalous activities in a video stream. Drawing from natural language processing, we introduce a representation of activities as bags of event n-grams, where we analyze the global structural information of activities using their local event statistics. We demonstrate how maximal cliques in an undirected edge-weighted graph of activities, can be used in an unsupervised manner, to discover regular sub-classes of an activity class. Based on these discovered sub-classes, we formulate a definition of anomalous activities and present a way to detect them. Finally, we characterize each discovered sub-class in terms of its "most representative member" and present an information-theoretic method to explain the detected anomalies in a human-interpretable form.
Efficient Real-Time Algorithms for Eye State and Head Pose Tracking in Advanced Driver Support Systems
Authors: Riad I. Hammoud, Andrew Wilhelm, Phillip Malawey, Gerald J. Witt
Venue: CVPR
Year: 2005

This article shows cutting-edge computer vision methods employed in advanced vision sensing technologies for medical, safety and security applications, where the human eye represents the object of interest for both the imager and the computer. As the eye scans the environment, or focuses on particular objects in the scene, the processor simultaneously localizes the eye position, tracks its position and movement over time, and infers counter measures such as fatigue level, attention level, and gaze direction in real-time and automatically. The focus of this demonstration is placed on four different algorithms: auto-initialization (RHED), eye position tracking (SIRAT), eye closure recognition (HRA), driver head pose categorization.
Cloth Representation by Shape from Shading with Shading Primitives
Authors: Feng Han 0001, Song Chun Zhu
Venue: CVPR
Year: 2005

Cloth is a complex visual pattern with flexible 3D shape and illumination variations. Computing the 3D shape of cloth from a single image is of great interest to both computer graphics and vision researches. However, the acquisition of 3D cloth shape by shape from shading (SFS) is still a challenge. In this paper, we present a two-layer generative model for representing both the 2D cloth image and the 3D cloth surface. The first layer represents all the folds on cloth, which are called "shading primitives" in (Haddon and Forsyth, 1998), and thus captures the overall "skeleton structures" of cloth. We learn a number of typical 3D fold primitives using some training images obtained through photometric stereo. The 3D fold primitives yield a dictionary of 2D shading primitives/or cloth images. The second layer represents non-fold parts with very smooth (often flat) surface or shading, which interpolates the primitives in the first layer with a smoothness prior like conventional SFS. Then we present an algorithm called "cloth sketching" to find all the shading primitives on cloth image and simultaneously recover their 3D shape by fitting to the 3D fold primitives. Our sketch representation can be viewed as a 2-layer Markov random field (MRF), and it introduces some prior knowledge on the folds and has lower dimension and is more robust than the traditional shape-fmm-shading representation which assumes a MRF model on pixels. We show a number of experiments with satisfactory results in comparison to previous work.
Kernel-Based Bayesian Filtering for Object Tracking
Authors: Bohyung Han, Ying Zhu 0006, Dorin Comaniciu, Larry S. Davis
Venue: CVPR
Year: 2005

Particle filtering provides a general framework for propagating probability density functions in nonlinear and non-Gaussian systems. However, the algorithm is based on a Monte Carlo approach and sampling is a problematic issue, especially for high dimensional problems. This paper presents a new kernel-based Bayesian filtering framework, which adopts an analytic approach to better approximate and propagate density functions. In this framework, the techniques of density interpolation and density approximation are introduced to represent the likelihood and the posterior densities by Gaussian mixtures, where all parameters such as the number of mixands, their weight, mean, and covariance are automatically determined. The proposed analytic approach is shown to perform sampling more efficiently in high dimensional space. We apply our algorithm to real-time tracking problems, and demonstrate its performance on real video sequences as well as synthetic examples.
Robust Centerline Extraction Framework Using Level Sets
Authors: M. Sabry Hassouna, Aly A. Farag
Venue: CVPR
Year: 2005

In this paper, we present a novel framework for computing centerlines for both 2D and 3D shape analysis. The framework works as follows: an object centerline point is selected automatically as the point of global maximum Euclidean distance from the boundary, and is considered a point source (Ps) that transmits a wave front that evolves over time and traverses the object domain. The front propagates at each object point with a speed that is proportional to its Euclidean distance from the boundary. The motion of the front is governed by a nonlinear partial differential equation whose solution is computed efficiently using level set methods. Initially, the P/sub S/ transmits a moderate speed wave to explore the object domain and extract its topological information such as merging and extreme points. Then, it transmits a new front that is much faster at centerline points than non central ones. As a consequence, centerlines intersect the propagating fronts at those points of maximum positive curvature. Centerlines are computed by tracking them, starting from each topological point until the Ps is reached, by solving an ordinary differential equation using an efficient numerical scheme. The proposed method is computationally inexpensive, handles efficiently objects with complex topology, and computes centerlines that are centered, connected, one point thick, and less sensitive to boundary noise. In addition, the extracted paths form a tree graph without additional cost. We have extensively validated the robustness of the proposed method both quantitatively and qualitatively against several 2D and 3D shapes.
Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information
Authors: Heiko Hirschmüller
Venue: CVPR
Year: 2005

This paper considers the objectives of accurate stereo matching, especially at object boundaries, robustness against recording or illumination changes and efficiency of the calculation. These objectives lead to the proposed semi-global matching method that performs pixelwise matching based on mutual information and the approximation of a global smoothness constraint. Occlusions are detected and disparities determined with sub-pixel accuracy. Additionally, an extension for multi-baseline stereo images is presented. There are two novel contributions. Firstly, a hierarchical calculation of mutual information based matching is shown, which is almost as fast as intensity based matching. Secondly, an approximation of a global cost calculation is proposed that can be performed in a time that is linear to the number of pixels and disparities. The implementation requires just 1 second on typical images.
Level Set Active Contours on Unstructured Point Cloud
Authors: Hon Pong Ho, Yunmei Chen, Huafeng Liu 0003, Pengcheng Shi
Venue: CVPR
Year: 2005

We present a novel level set representation and front propagation scheme for active contours where the analysis/evolution domain is sampled by unstructured point cloud. These sampling points are adaptively distributed according to both local data and level set geometry, hence allow extremely convenient enhancement/reduction of local front precision by simply putting more/fewer points on the computation domain without grid refinement (as the cases in finite difference schemes) or remeshing (typical infinite element methods). The front evolution process is then conducted on the point-sampled domain, without the use of computational grid or mesh, through the precise but relatively expensive moving least squares (MLS) approximation of the continuous domain, or the faster yet coarser generalized finite difference (GFD) representation and calculations. Because of the adaptive nature of the sampling point density, our strategy performs fast marching and level set local refinement concurrently. We have evaluated the performance of the method in image segmentation and shape recovery applications using real and synthetic data.
A Semi-Supervised Active Learning Framework for Image Retrieval
Authors: Steven C. H. Hoi, Michael R. Lyu
Venue: CVPR
Year: 2005

Although recent studies have shown that unlabeled data are beneficial to boosting the image retrieval performance, very few approaches for image retrieval can learn with labeled and unlabeled data effectively. This paper proposes a novel semi-supervised active learning framework comprising a fusion of semi-supervised learning and support vector machines. We provide theoretical analysis of the active learning framework and present a simple yet effective active learning algorithm for image retrieval. Experiments are conducted on real-world color images to compare with traditional methods. The promising experimental results show that our proposed scheme significantly outperforms the previous approaches.
Tracking Multiple Objects through Occlusions
Authors: Yan Huang 0017, Irfan A. Essa
Venue: CVPR
Year: 2005

We present an approach for tracking varying number of objects through both temporally and spatially significant occlusions. Our method builds on the idea of object permanence to reason about occlusions. To this end, tracking is performed at both the region level and the object level. At the region level, a customized genetic algorithm is used to search for optimal region tracks. This limits the scope of object trajectories. At the object level, each object is located based on adaptive appearance models, spatial distributions and inter-occlusion relationships. The proposed architecture is capable of tracking objects even in the presence of long periods of full occlusions. We demonstrate the viability of this approach by experimenting on several videos of a user interacting with a variety of objects on a desktop.
Tracking Multiple Objects through Occlusions
Authors: Yan Huang 0017, Irfan A. Essa
Venue: CVPR
Year: 2005

Visual tracking of multiple objects with continuous interactions is important for many applications such as surveillance and effective user interfaces. Complex interactions between objects results in both temporally and spatially significant occlusions, making multi-object tracking a challenging problem. We present an approach for tracking varying number of objects through significant occlusions. Our approach builds on the idea of object permanence. That is, partially or fully occluded objects, even though not observable, still exist in the close proximity of their occluders. To demonstrate the viability of this approach, we have concentrated on tracking hands and objects as a person interacts with multiple objects on a desktop. We experimented with 5 indoor video sequences and present the results of two representative ones: lego sequence and shell game.
Unstructured Point Cloud Matching within Graph-Theoretic and Thermodynamic Frameworks
Authors: Anupama Jagannathan, Eric L. Miller 0001
Venue: CVPR
Year: 2005

In the context of object recognition from point cloud data, we present a thermodynamically-inspired graph theoretic algorithm to address the problem of matching the scene and the model point clouds, when the cardinalities of the two sets are orders of magnitude different. Such an approach determines a subset of points from the model that is structurally and spatially as similar as possible to the set of points in the scene. A new formulation for graph enthalpy characterizes the structural differences between point sets, which together with the existing notions of graph entropy quantifies the Gibbs' free energy. A two-scale approach is proposed, wherein, at the coarse scale, a set of points that comprise the model neighborhood around the scene is identified by minimization of entropy. At the fine scale, the desired correspondence is achieved by a refinement process, aimed at maximizing the Gibbs' free energy. The results demonstrate the robustness and efficiency of the approach.
Online Detection and Classification of Moving Objects Using Progressively Improving Detectors
Authors: Omar Javed, Saad Ali, Mubarak Shah
Venue: CVPR
Year: 2005

Boosting based detection methods have successfully been used for robust detection of faces and pedestrians. However, a very large amount of labeled examples are required for training such a classifier. Moreover, once trained, the boosted classifier cannot adjust to the particular scenario in which it is employed. In this paper, we propose a co-training based approach to continuously label incoming data and use it for online update of the boosted classifier that was initially trained from a small labeled example set. The main contribution of our approach is that it is an online procedure in which separate views (features) of the data are used for co-training, while the combined view (all features) is used to make classification decisions in a single boosted framework. The features used for classification are derived from principal component analysis of the appearance templates of the training examples. In order to speed up the classification, background modeling is used to prune away stationary regions in an image. Our experiments indicate that starting from a classifier trained on a small training set, significant performance gains can be made through online updation from the unlabeled data.
Appearance Modeling for Tracking in Multiple Non-Overlapping Cameras
Authors: Omar Javed, Khurram Shafique, Mubarak Shah
Venue: CVPR
Year: 2005

When viewed from a system of multiple cameras with non-overlapping fields of view, the appearance of an object in one camera view is usually very different from its appearance in another camera view due to the differences in illumination, pose and camera parameters. In order to handle the change in observed colors of an object as it moves from one camera to another, we show that all brightness transfer functions from a given camera to another camera lie in a low dimensional subspace and demonstrate that this subspace can be used to compute appearance similarity. In the proposed approach, the system learns the subspace of inter-camera brightness transfer functions in a training phase during which object correspondences are assumed to be known. Once the training is complete, correspondences are assigned using the maximum a posteriori (MAP) estimation framework using both location and appearance cues. We evaluate the proposed method under several real world scenarios obtaining encouraging results.
Integration of Motion Fields through Shape
Authors: Hui Ji 0002, Cornelia Fermüller
Venue: CVPR
Year: 2005

Structure from motion from single flow fields has been studied intensively, but the integration of information from multiple flow fields has not received much attention. Here we address this problem by enforcing constraints on the shape (surface normals) of the scene in view, as opposed to constraints on the structure (depth). The advantage of integrating shape is two-fold. First, we do not need to estimate feature correspondences over multiple frames, but we only need to match patches. Second, the shape vectors in the different views are related only by rotation. This constraint on shape can be combined easily with motion estimation, thus formulating motion and structure estimation from multiple views as a practical constrained minimization problem using a rank-3 constraint. Based on this constraint, we develop a 3D motion technique, which locates through color and motion segmentation, planar patches in the scene, matches patches over multiple frames, and estimates the motion between multiple frames and the shape of the selected scene patches using the image gradients. Experiments evaluate the accuracy of the 3D motion estimation and demonstrate the motion and shape estimation of the technique by super-resolving an image sequence.
Mapping Low-Level Features to High-Level Semantic Concepts in Region-Based Image Retrieval
Authors: Wei Jiang, Kap Luk Chan, Mingjing Li, HongJiang Zhang
Venue: CVPR
Year: 2005

In this paper, a novel offline supervised learning method is proposed to map low-level visual features to high-level semantic concepts for region-based image retrieval. The contributions of this paper lie in three folds. (1) For each semantic concept, a set of low-level tokens are extracted from the segmented regions of training images. Those tokens capture the representative information for describing the semantic meaning of that concept; (2) a set of posteriors are generated based on the low-level tokens through pairwise classification, which denote the probabilities of images belonging to the semantic concepts. The posteriors are treated as high-level features that connect images with high-level semantic concepts. Long-term relevance feedback learning is incorporated to provide the supervisory information needed in the above offline learning process, including the concept information and the relevant training set for each concept; (3) an integrated algorithm is implemented to combine two kinds of information for retrieval: the information from the offline feature-to-concept mapping process and the high-level semantic information from the long-term learned memory. Experimental evaluation on 10,000 images proves the effectiveness of our method.
Visual Tracking in the Presence of Motion Blur
Authors: Hailin Jin, Paolo Favaro, Roberto Cipolla
Venue: CVPR
Year: 2005

We consider the problem of visual tracking of regions of interest in a sequence of motion blurred images. Traditional methods couple tracking with deblurring in order to correctly account for the effects of motion blur. Such coupling is usually appropriate, but computationally wasteful when visual tracking is the lone objective. Instead of deblurring images, we propose to match regions by blurring them. The matching score for two image regions is governed by a cost function that only involves the region deformation parameters and two motion blur vectors. We present an efficient algorithm to minimize the proposed cost function and demonstrate it on sequences of real blurred images.
Multimodal Face Recognition: Combination of Geometry with Physiological Information
Authors: Ioannis A. Kakadiaris, Georgios Passalis, Theoharis Theoharis, George Toderici, Ioannis Konstantinidis 0001, Mohammed N. Murtuza
Venue: CVPR
Year: 2005

It is becoming increasingly important to be able to credential and identify authorized personnel at key points of entry. Such identity management systems commonly employ biometric identifiers. In this paper, we present a novel multimodal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors. Data from multiple cameras is used to construct a three-dimensional mesh representing the face and a facial thermal texture map. An annotated face model with explicit two-dimensional parameterization (UV) is then fitted to this data to construct: 1) a three-channel UV deformation image encoding geometry, and 2) a one-channel UV vasculature image encoding facial vasculature. Recognition is accomplished by comparing: 1) the parametric deformation images, 2) the parametric vasculature images, and 3) the visible spectrum texture maps. The novelty of our work lies in the use of deformation images and physiological information as means for comparison. We have performed extensive tests on the Face Recognition Grand Challenge v1.0 dataset and on our own multimodal database with very encouraging results.
8D-THERMO CAM: Combination of Geometry with Physiological Information for Face Recognition
Authors: Ioannis A. Kakadiaris, Georgios Passalis, Theoharis Theoharis, George Toderici, Ioannis Konstantinidis 0001, Mohammed N. Murtuza
Venue: CVPR
Year: 2005

Biometrics-based technologies in the area of identity management are gaining increasing importance, as a means of establishing non-falsifiable credentials for end users. However, in the three-way tug-of-war between convenient, unobtrusive data collection (required for user acceptance), accuracy in results (required for justifying deployment), and speed (required for widespread use in practice), no single biometric to date has managed to hold the middle ground that would allow for its ready adoption. The overall goal of our project is to develop the theoretical framework and computational tools that will lead to the development of a practical, unobtrusive, and accurate face recognition system for convenient and effective access control. This framework encompasses 8D characteristics of the face (3D geometry+2D visible texture+2D infrared texture, over time). In this paper, we present a novel multi-modal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors. From the fitted parametric model we extract two images corresponding to the subject's face and process these images to extract biometric signatures. Specifically, the deformation image is compressed using a wavelet transform and the vasculature graph is extracted from the parametric thermal image.
Theory for Variational Area-Based Segmentation Using Non-Quadratic Penalty Functions
Authors: Adam Karlsson, Niels Chr. Overgaard
Venue: CVPR
Year: 2005

In this paper a theory is developed for variational segmentation of images using area-based segmentation functionals with non-quadratic penalty functions in the fidelity term. Two small theorems, which we believe are new to the vision community, allow us to compute the Gateaux derivative of the considered functional, and to construct the corresponding gradient descent flow. The functional is minimized by evolving an initial curve using this gradient descent flow. If the penalty function is sub-quadratic, i.e. behaves like the p'th power of the error for p<2, the obtained segmentation model is more robust with respect to noise and outliers than the classical Chan-Vese model and the curve evolution has better convergence properties.