Constrained planar motion analysis by decomposition
General SFM methods give poor results for images captured by constrained motions such as planar motion. In this paper, we propose new SFM algorithms for images captured under a common but constrained planar motion: the image plane is perpendicular to the motion plane. We show that a 2D image captured under such constrained planar motion can be decoupled into two 1D images: one 1D projective and one 1D affine. We then introduce the 1D affine camera model for completing 1D camera models. Next, we describe new subspace reconstruction methods, and apply these methods to the images captured by concentric mosaics, which undergo a special case of constrained planar motion. Finally, we demonstrate both in theory and experiments the advantage of the decomposition method over the general SFM methods by incorporating the constrained motion into the earliest stage of motion analysis.

title: A two level approach for scene recognition
abstract: Classifying pictures into one of several semantic categories is a classical image understanding problem. In this paper, we present a stratified approach to both binary (outdoor-indoor) and multiple category of scene classification. We first learn mixture models for 20 basic classes of local image content based on color and texture information. Once trained, these models are applied to a test image, and produce 20 probability density response maps (PDRM) indicating the likelihood that each image region was produced by each class. We then extract some very simple features from those PDRMs, and use them to train a bagged LDA classifier for 10 scene categories. For this process, no explicit region segmentation or spatial context model are computed. To test this classification system, we created a labeled database of 1500 photos taken under very different environment and lighting conditions, using different cameras, and from 43 persons over 5 years. The classification rate of outdoor-indoor classification is 93.8%, and the classification rate for 10 scene categories is 90.1%. As a byproduct, local image patches can be contextually labeled into the 20 basic material classes by using loopy belief propagation (Yedidia et al., 2001) as an anisotropic filter on PDRMs, producing an image-level segmentation if desired.

title: Repeating pattern discovery and structure analysis from acoustic music data
abstract: Music and songs usually have repeating patterns and prominent structure. The automatic extraction of such repeating patterns and structure is useful for further music summarization, indexing and retrieval. In this paper, an effective approach of repeating pattern discovery and structure analysis of acoustic music data is proposed. In order to represent the melody similarity more accurately, in our approach, Constant Q transform is utilized in feature extraction and a novel similarity measure between musical features is proposed. From the self-similarity matrix of the music, an adaptive method is then presented to extract all significant repeating patterns. Based on the obtained repetitions, musical structure is further analyzed using a few heuristic rules. Finally, an optimization-based approach is proposed to determine the accurate boundary of each musical section. Evaluations on various music pieces indicate our approach is promising.

title: Unsupervised speaker segmentation and tracking in real-time audio content analysis
abstract: This paper addresses the problem of real-time speaker segmentation and speaker tracking in audio content analysis in which no prior knowledge of the number of speakers and the identities of speakers is available. Speaker segmentation is to detect the speaker change boundaries in a speech stream. It is performed by a two-step algorithm, which includes potential change detection and refinement. Speaker tracking is then performed based on the results of speaker segmentation by identifying the speaker of each segment. In our approach, incremental speaker model updating and segmental clustering is proposed, which makes the unsupervised speaker segmentation and tracking feasible in real-time processing. A Bayesian fusion method is also proposed to fuse multiple audio features to obtain a more reliable result, and different noise levels are utilized to compensate for background mismatch. Experiments show that the proposed algorithm can recall 89% of speaker change boundaries with 15% false alarms, and 76% of speakers can be unsupervised identified with 20% false alarms. Compared with previous works, the algorithm also has low computation complexity and can perform in 15% of real time with a very limited delay in analysis.

title: A robust audio classification and segmentation method
abstract: In this paper, we present a robust algorithm for audio classification that is capable of segmenting and classifying an audio stream into speech, music, environment sound and silence. Audio classification is processed in two steps, which makes it suitable for different applications. The first step of the classification is speech and non-speech discrimination. In this step, a novel algorithm based on KNN and LSP VQ is presented. The second step further divides non-speech class into music, environment sounds and silence with a rule based classification scheme. Some new features such as the noise frame ratio and band periodicity are introduced and discussed in detail. Our experiments in the context of video structure parsing have shown the algorithms produce very satisfactory results.

title: Dominant Feature Vectors Based Audio Similarity Measure
abstract: This paper presents an approach to extracting dominant feature vectors from an individual audio clip and then proposes a new similarity measure based on the dominant feature vectors. Instead of using the mean and standard deviation of frame features in most conventional methods, the most salient characteristics of an audio clip are represented in the form of several dominant feature vectors. These dominant feature vectors give a better description of the fundamental properties of an audio clip, especially when frame features change a lot along the time line. Evaluations on a content-based audio retrieval system indicate an obvious improvement after using the proposed similarity measure, compared with some other conventional methods.