Adaptive Summarization of Digital Video Data

As multimedia applications are rapidly spreading at an ever-increasing rate, efficient and effective methodologies for organizing and manipulating these data become a necessity. One of the basic problems with such systems is finding efficient ways to summarize the huge amount of data involved. In this chapter, we start by defining the problem of key frames extraction, then review a number of proposed techniques to accomplish that task, showing their pros and cons. After that, we describe two adaptive algorithms proposed to effectively select key frames from segmented video shots where both apply a two-level adaptation mechanism. These algorithms constitute the second stage of a Video Content-based Retrieval (VCR) system that has been designed at Old Dominion University. The first adaptation level is based on the size of the input video file, while the second level is performed on a shot-by-shot basis to account for the fact that different shots have different levels of activity. Experimental results show the efficiency and robustness of the proposed algorithms in selecting the near-optimal set of key frames required to represent each shot. 701 E. Chocolate Avenue, Suite 200, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.irm-press.com IRM PRESS This chapter appears in the bo k, Video Data Management a d I formati n R trieval by Sagarmay Deb. Copyright © 2005, IRM Press, an imprint of Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. 78 Farag & Abdel-Wahab Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. INTRODUCTION As multimedia applications are rapidly spreading at an ever-increasing rate, novel techniques for organizing and abstracting the data they produce become a necessity. One of the basic problems with such systems is finding efficient ways to summarize the huge amount of data involved. To solve this problem, digital video streams need to be analyzed by first dividing each stream into a set of meaningful and manageable units — a task that was the focus of Video Shot Boundary Detection chapter. After dividing a video sequence into a number of shots, each shot still contains a large number of frames. As a result, the second stage in any video analysis system is the process of Key Frames (KFs) selection (Rui, Huang, & Mehrotra, 1998) that aims to abstract the whole shot using one frame or more. This process is known also as video summarization. Ideally, we need to select the minimal set of KFs that can faithfully represent each shot. KFs are the most important frames in a shot since they may be used to represent the shot in the browsing system as well as be used as access points. Moreover, one advantage of representing each shot by a set of frames is the reduction in the computation burden required by any content analysis system to perform similarity matching on a frame-by-frame basis. In the segmentation chapter, we have explained how an input MPEG stream is processed by first extracting the DC sequence (Yeo & Liu, 1995a) from it. Subsequently, that sequence is used as an input to a neural network module to perform the segmentation, shot boundary detection, task. The output of the segmentation stage is a group of distinct shots with clear marks to the beginning and end of each segment. This output is then fed to our KFs selection module (Farag & Abdel-Wahab, 2002a, 2002b), which is the focus of this chapter. We start by introducing a fairly simple and efficient algorithm that uses the accumulated summation of DC frames luminance differences in order to select KFs from segmented video shots. Once the accumulated summation exceeds a certain threshold, the current frame is added to the representative set in addition to the first frame (which is chosen by default), then the algorithm proceeds. To improve the basic algorithm introduced above, we propose a first-level adaptation mechanism that is based upon the dimension of the input video file. After that, a second level of adaptation based on a shot activity criterion is introduced to further improve the performance and accuracy of the selection module. At the end, another algorithm is proposed that uses a direct comparison between the summation of the luminance channel DC terms of the current frame and the corresponding summation of the last chosen key frame. The algorithm then selects the current frame into the representative set if the absolute difference is above a certain threshold. This algorithm uses a first-level adaptation policy similar to that used by the first group, but it employs a statistical criterion for the shot-by-shot adaptation level. Analyzing the results produced by both algorithms showed their efficiency and accuracy in selecting the nearoptimal set of key frames required to represent each shot in a video stream. We then conclude the chapter by comparing the performance of both algorithms. This chapter is organized as follows. In the next section, we review briefly a number of related approaches for selecting KFs. The first proposed set of algorithms is then introduced, starting with the most straightforward one and going towards the more effective ones. After that, we introduce the second proposed algorithm along with its criteria for the adaptation processes. Performance comparisons of the two proposed mechanisms are given next, followed by the Conclusion at the end of the chapter. 21 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the product's webpage: www.igi-global.com/chapter/adaptive-summarization-digitalvideo-data/30762? camid=4v1 This title is available in InfoSci-Books, InfoSci-Database Technologies, Library Science, Information Studies, and Education, InfoSci-Library and Information Science, InfoSciComputer Science and Information Technology, Science, Engineering, and Information Technology, InfoSci-Select, InfoSci-Select. Recommend this product to your librarian: www.igi-global.com/e-resources/libraryrecommendation/? id=1