Video Classification and Retrieval with the Informedia Digital Video Library System
This paper is organized in three parts. The first part details some of the lower level shot classification work, the second part describes the ‘manual’ retrieval systems while the last section details the interactive retrieval system for the Carnegie Mellon University TREC Video Retrieval Track runs. The description of the data can be found elsewhere in the proceedings of the 2002 TREC conference video track overview.

title: Meta-Classification of Multimedia Classifiers
abstract: Combining multiple classifiers is of particular interest in the multimedia systems, since there is usually data of very different types/modalities that should be mined or analyzed. Our wearable ‘experience collection’ system unobtrusively records the wearer’s conversation, recognizes the face of the dialog partner and remembers his/her voice. When the system sees the same person’s face or hears the same voice it can then use a summary of the last conversation with this person to remind the wearer. To correctly identify a person from a mixture of video and audio stream, classification judgments from individual modality classifiers must be combined effectively to yield a more accurate decision. To address the problems of combination strategy in previous studies, a meta-classification strategy using Support Vector Machine is proposed. Preliminary results show that combining different face recognition and speaker identification technology by meta-classification is dramatically more effective than weighted interpolation. Meta-classification is general enough to be applied to any application that needs to combine multiple classifiers without much modification.

title: News video classification using SVM-based multimodal classifiers and combination strategies
abstract: Video classification is the first step toward multimedia content understanding. When video is classified into conceptual categories, it is usually desirable to combine evidence from multiple modalities. However, combination strategies in previous studies were usually ad hoc. We investigate a meta-classification combination strategy using Support Vector Machine, and compare it with probability-based strategies. Text features from closed-captions and visual features from images are combined to classify broadcast news video. The experimental results show that combining multimodal classifiers can significantly improve recall and precision, and our meta-classification strategy gives better precision than the approach of taking the product of the posterior probabilities.

title: Extreme video retrieval: joint maximization of human and computer performance
abstract: We present an efficient system for video search that maximizes the use of human bandwidth, while at the same time exploiting the machine's ability to learn in real-time from user selected relevant video clips. The system exploits the human capability for rapidly scanning imagery augmenting it with an active learning loop, which attempts to always present the most relevant material based on the current information. Two versions of the human interface were evaluated, one with variable page sizes and manual paging, the other with a fixed page size and automatic paging. Both require absolute attention and focus of the user for optimal performance. In either case, as users search and find relevant results, the system can invisibly re-rank its previous best guesses using a number of knowledge sources, such as image similarity, text similarity, and temporal proximity. Experimental evidence shows a significant improvement using the combined extremes of human and machine power over either approach alone.

title: A wearable digital library of personal conversations
abstract: We have developed a wearable, personalized digital library system, which unobtrusively records the wearer's part of a conversation, recognizes the face of the current dialog partner and remembers his/her voice. The next time the system sees the same person and hears the same voice, it can replay parts of the last conversation in compressed form. Results from a prototype system show the effectiveness of combining of face recognition and speaker identification for retrieving conversations.

title: Exploring the Synergy of Humans and Machines in Extreme Video Retrieval
abstract: We introduce an interface for efficient video search that exploits the human ability to quickly scan visual content, after automatic retrieval has arrange the images in expected order of relevance. While extreme video retrieval is taxing to the human, it is also extremely effective. Two variants of extreme retrieval are demonstrated, 1) RSVP which automatically pages through images with user-control of the paging speed, while the user marks relevant shots and 2) MBRP where the user manually controls paging and adjusts the number of images per page, depending on the density of relevant shots.

title: A Lattice-Based MRF Model for Dynamic Near-Regular Texture Tracking
abstract: A near-regular texture (NRT) is a geometric and photometric deformation from its regular origin - a congruent wallpaper pattern formed by 2D translations of a single tile. A dynamic NRT is an NRT under motion. Although NRTs are pervasive in man-made and natural environments, effective computational algorithms for NRTs are few. This paper addresses specific computational challenges in modeling and tracking dynamic NRTs, including ambiguous correspondences, occlusions, and drastic illumination and appearance variations. We propose a lattice-based Markov-random-field (MRF) model for dynamic NRTs in a 3D spatiotemporal space. Our model consists of a global lattice structure that characterizes the topological constraint among multiple textons and an image observation model that handles local geometry and appearance variations. Based on the proposed MRF model, we develop a tracking algorithm that utilizes belief propagation and particle filtering to effectively handle the special challenges of the dynamic NRT tracking without any assumption on the motion types or lighting conditions. We provide quantitative evaluations of the proposed method against existing tracking algorithms and demonstrate its applications in video editing

title: Several parameters of generalized Mycielskians
abstract: The generalized Mycielskians (also known as cones over graphs) are the natural generalization of the Mycielski graphs (which were first introduced by Mycielski in 1955). Given a graph G and any integer m ≥ 0, one can transform G into a new graph µm (G), the generalized Mycielskian of G. This paper investigates circular clique number, total domination number, open packing number, fractional open packing number, vertex cover number, determinant, spectrum, and biclique partition number of µm (G).

title: Circular chromatic numbers of some distance graphs
abstract: Given a set D of positive integers, the distance graph G(Z,D) has vertices all integers Z, and two vertices j and j^' in Z are adjacent if and only if |j-j^'|@?D. This paper determines the circular chromatic numbers of some distance graphs.

title: The Promise and Perils of Near-Regular Texture
abstract: Motivated by the low structural fidelity for near-regular textures in current texture synthesis algorithms, we propose and implement an alternative texture synthesis method for near-regular texture. We view such textures as statistical departures from regular patterns and argue that a thorough understanding of their structures in terms of their translation symmetries can enhance existing methods of texture synthesis. We demonstrate the perils of texture synthesis for near-regular texture and the promise of faithfully preserving the regularity as well as the randomness in a near-regular texture sample.

title: Speaker Adaptation of Fuzzy-Perceptron-Based Speech Recognition
abstract: In this paper, we propose a speech recognition algorithm which utilizes hidden Markov models (HMM) and Viterbi algorithm for segmenting the input speech sequence, such that the variable-dimensional speech signal is converted into a fixed-dimensional speech signal, called TN vector. We then use the fuzzy perceptron to generate hyperplanes which separate patterns of each class from the others. The proposed speech recognition algorithm is easy for speaker adaptation when the idea of "supporting pattern" is used. The supporting patterns are those patterns closest to the hyperplane. When a recognition error occurs, we include all the TN vectors of the input speech sequence with respect to the segmentations of all HMM models as the supporting patterns. The supporting patterns are then used by the fuzzy perceptron to tune the hyperplane that can cause correct recognition, and also tune the hyperplane that resulted in wrong recognition. Since only two hyperplane need to be tuned for a recognition error, the proposed adaptation scheme is time-economic and suitable for on-line adaptation. Although the adaptation scheme cannot ensure to correct the wrong recognition right after adaptation, the hyperplanes are tuned in the direction for correct recognition iteratively and the speed of adaptation can be adjusted by a "belief" parameter set by the user. Several examples are used to show the performance of the proposed speech recognition algorithm and the speaker adaptation scheme.