A Three Tiered Approach for Articulated Object Action Modeling and Recognition
Visual action recognition is an important problem in computer vision. In this paper, we propose a new method to probabilistically model and recognize actions of articulated objects, such as hand or body gestures, in image sequences. Our method consists of three levels of representation. At the low level, we first extract a feature vector invariant to scale and in-plane rotation by using the Fourier transform of a circular spatial histogram. Then, spectral partitioning [20] is utilized to obtain an initial clustering; this clustering is then refined using a temporal smoothness constraint. Gaussian mixture model (GMM) based clustering and density estimation in the subspace of linear discriminant analysis (LDA) are then applied to thousands of image feature vectors to obtain an intermediate level representation. Finally, at the high level we build a temporal multi-resolution histogram model for each action by aggregating the clustering weights of sampled images belonging to that action. We discuss how this high level representation can be extended to achieve temporal scaling in-variance and to include Bi-gram or Multi-gram transition information. Both image clustering and action recognition/segmentation results are given to show the validity of our three tiered representation.

title: A two level approach for scene recognition
abstract: Classifying pictures into one of several semantic categories is a classical image understanding problem. In this paper, we present a stratified approach to both binary (outdoor-indoor) and multiple category of scene classification. We first learn mixture models for 20 basic classes of local image content based on color and texture information. Once trained, these models are applied to a test image, and produce 20 probability density response maps (PDRM) indicating the likelihood that each image region was produced by each class. We then extract some very simple features from those PDRMs, and use them to train a bagged LDA classifier for 10 scene categories. For this process, no explicit region segmentation or spatial context model are computed. To test this classification system, we created a labeled database of 1500 photos taken under very different environment and lighting conditions, using different cameras, and from 43 persons over 5 years. The classification rate of outdoor-indoor classification is 93.8%, and the classification rate for 10 scene categories is 90.1%. As a byproduct, local image patches can be contextually labeled into the 20 basic material classes by using loopy belief propagation (Yedidia et al., 2001) as an anisotropic filter on PDRMs, producing an image-level segmentation if desired.

title: Constrained planar motion analysis by decomposition
abstract: General SFM methods give poor results for images captured by constrained motions such as planar motion. In this paper, we propose new SFM algorithms for images captured under a common but constrained planar motion: the image plane is perpendicular to the motion plane. We show that a 2D image captured under such constrained planar motion can be decoupled into two 1D images: one 1D projective and one 1D affine. We then introduce the 1D affine camera model for completing 1D camera models. Next, we describe new subspace reconstruction methods, and apply these methods to the images captured by concentric mosaics, which undergo a special case of constrained planar motion. Finally, we demonstrate both in theory and experiments the advantage of the decomposition method over the general SFM methods by incorporating the constrained motion into the earliest stage of motion analysis.

title: Efficient particle filtering using RANSAC with application to 3D face tracking
abstract: Particle filtering is a very popular technique for sequential state estimation. However, in high-dimensional cases where the state dynamics are complex or poorly modeled, thousands of particles are usually required for real applications. This paper presents a hybrid sampling solution that combines RANSAC and particle filtering. In this approach, RANSAC provides proposal particles that, with high probability, represent the observation likelihood. Both conditionally independent RANSAC sampling and boosting-like conditionally dependent RANSAC sampling are explored. We show that the use of RANSAC-guided sampling reduces the necessary number of particles to dozens for a full 3D tracking problem. This method is particularly advantageous when state dynamics are poorly modeled. We show empirically that the sampling efficiency (in terms of likelihood) is much higher with the use of RANSAC. The algorithm has been applied to the problem of 3D face pose tracking with changing expression. We demonstrate the validity of our approach with several video sequences acquired in an unstructured environment.

title: Universal Background Models for Real-time Speaker Change Detection
abstract: This paper addresses the problem of real-time speaker change detection in TV news broadcast, in which no prior knowledge on speakers is assumed. To remove the unreliable frames and background frames in the speech stream, we propose a new approach for feature categorization based on Gaussian Mixture Model - Universal Background Model (GMM-UBM). The feature vectors are categorized into three sets, which include reliable speech, doubtful speech and unreliable speech. Then a novel distance measure is presented correspondingly for real-time speaker change detection. Extensive experiments demonstrate its good performance, and intrinsic difficulties on real-time speaker change detection are discussed as well in this paper.

title: The diameter of random massive graphs
abstract: Many massive graphs (such as the WWW graph and Call graphs) share certain universal characteristics which can be described by so-called the “power law”. Here we determine the diameter of random power law graphs up to a constant factor for almost all ranges of parameters. These results show a strong evidence that the diameters of most massive graphs are about logarithm of their sizes up to a constant factor.

title: Using structure patterns of temporal and spectral feature in audio similarity measure
abstract: Although statistical characteristics of audio features are widely used for similarity measure in most of current audio analysis systems and have been proved to be effective, they only utilized the averaged feature variations over time, and thus lead to inaccuracy in some cases. In this paper, structure pattern, which describes the representative structure characteristics of both temporal and spectral features, is proposed to improve the similarity measure for audio effects. Three kind structure patterns are proposed and utilized in current work, including energy contour pattern, harmonicity pattern and pitch contour pattern. Evaluations on a content-based audio retrieval system indicate that structure patterns can improve the performance pretty much.

title: Repeating pattern discovery and structure analysis from acoustic music data
abstract: Music and songs usually have repeating patterns and prominent structure. The automatic extraction of such repeating patterns and structure is useful for further music summarization, indexing and retrieval. In this paper, an effective approach of repeating pattern discovery and structure analysis of acoustic music data is proposed. In order to represent the melody similarity more accurately, in our approach, Constant Q transform is utilized in feature extraction and a novel similarity measure between musical features is proposed. From the self-similarity matrix of the music, an adaptive method is then presented to extract all significant repeating patterns. Based on the obtained repetitions, musical structure is further analyzed using a few heuristic rules. Finally, an optimization-based approach is proposed to determine the accurate boundary of each musical section. Evaluations on various music pieces indicate our approach is promising.

title: A robust audio classification and segmentation method
abstract: In this paper, we present a robust algorithm for audio classification that is capable of segmenting and classifying an audio stream into speech, music, environment sound and silence. Audio classification is processed in two steps, which makes it suitable for different applications. The first step of the classification is speech and non-speech discrimination. In this step, a novel algorithm based on KNN and LSP VQ is presented. The second step further divides non-speech class into music, environment sounds and silence with a rule based classification scheme. Some new features such as the noise frame ratio and band periodicity are introduced and discussed in detail. Our experiments in the context of video structure parsing have shown the algorithms produce very satisfactory results.