nils-herrmann/clean_scirepeval huggingface dataset:
Recent advances in video coding using static background models

Sprite coding, as standardized in MPEG-4 Visual, can result in superior performance compared to common hybrid video codecs both objectively and subjectively. However, state-of-the-art video coding standard H.264/AVC clearly outperforms MPEG-4 Visual sprite coding in broad bit rate ranges. Based on the sprite coding idea, this paper proposes a video coding technique that merges the advantages of H.264/AVC and sprite coding. For that, sophisticated algorithms for global motion estimation, sprite generation and object segmentation — all needed for thorough sprite coding — are incorporated into an H.264/AVC coding environment. The proposed approach outperforms H.264/AVC especially in lower bit rate ranges. Savings up to 21% can be achieved.
Adaptively Splitted GMM With Feedback Improvement for the Task of Background Subtraction

Per pixel adaptive Gaussian mixture models (GMMs) have become a popular choice for the detection of change in the video surveillance domain because of their ability to cope with many challenges characteristic for surveillance systems in real time with low memory requirements. Since their first introduction in the surveillance domain, GMM has been enhanced in many directions. In this paper, we present a study of some relevant GMM approaches and analyze their underlying assumptions and design decisions. Based on this paper, we show how these systems can be further improved by means of a variance controlling scheme and the incorporation of region analysis-based feedback. The proposed system has been thoroughly evaluated using the extensive data set of the IEEE Workshop on Change Detection, showing an outranking performance in comparison with state-of-the-art methods.
Polyphonic musical instrument recognition based on a dynamic model of the spectral envelope

We propose a new method for detecting the musical instruments that are present in single-channel mixtures. Such a task is of interest for audio and multimedia content analysis and indexing applications. The approach is based on grouping sinusoidal trajectories according to common onsets, and comparing each group's overall amplitude evolution with a set of pre-trained probabilistic templates describing the temporal evolution of the spectral envelopes of a given set of instruments. Classification is based on either an Euclidean or a probabilistic definition of timbral similarity, both of which are compared with respect to detection accuracy.
A family of single-user autostereoscopic displays with head-tracking capabilities

We present prototypes of autostereoscopic displays which allow single users to experience stereoscopic vision without the need for special eye glasses or helmet-mounted displays. The design of the displays is based on lenticular raster plates and includes a number of novel concepts for tracking of raster plates or projection lenses to account for changes of the viewers position in front of the screen. Applications envisioned include 3-D multimedia desktop visualization for medical and biological imaging, design, and architecture, as well as computer games and 3-D virtual reality in general. Concepts and results for both high-resolution flat liquid-crystal panel monitors for PC desktop applications as well as large screen high resolution displays using rear-projection technology are discussed.
Digital video coding standards and their role in video communications

The efficient digital representation of image and video signals has been subject of considerable research over the past 20 years. With the growing availability of digital transmission links, progress in signal processing, VLSI technology and image compression research, visual communications has become more feasible than ever. Digital video coding technology has developed into a mature field and a diversity of products has been developed-targeted for a wide range of emerging applications, such as video on demand, digital TV/HDTV broadcasting, and multimedia image/video database services. With the increased commercial interest in video communications the need for international image and video coding standards arose. Standardization of video coding algorithms holds the promise of large markets for video communication equipment. Interoperability of implementations from different vendors enables the consumer to access video from a wider range of services and VLSI implementations of coding algorithms conforming to international standards can be manufactured at considerably reduced costs. The purpose of this paper is to provide an overview of today's image and video coding standards and their role in video communications. The different coding algorithms developed for each standard are reviewed and the commonalities between the standards are discussed.
A dynamic model buffer for parametric motion vector prediction in random-access coding scenarios

Motion compensated inter prediction is a powerful tool used in modern hybrid video codecs to reduce the temporal redundancy of video sequences. However, the motion information needed for motion compensation is highly redundant as well. Thus, motion vector prediction and difference coding is a common method in modern video codecs. During the standardization of HEVC, new methods for motion prediction such as temporal motion vector prediction have been analyzed. This paper presents a method for motion vector prediction from perspective motion models in random access scenarios with hierarchical group of picture structures. To enable this kind of prediction a dynamic buffer system for generating, compressing and transmitting the underlying motion models is introduced. Bit rate reductions of up to 5% underline the performance of the complete system.
Robust local optical flow: Long-range motions and varying illuminations

Sparse motion estimation with local optical flow methods is fundamental for a wide range of computer vision application. Classical approaches like the pyramidal Lucas-Kanade method (PLK) or more sophisticated approaches like the Robust Local Optical Flow (RLOF) fail when it comes to environments with illumination changes and/or long-range motions. In this work we focus on these limitations and propose a novel local optical flow framework taking into account an illumination model to deal with varying illumination and a prediction step based on a perspective global motion model to deal with long-range motions. Experimental results shows tremendous improvements, e.g. 56% smaller error for dense motion fields on the KITTI and an about 76% smaller error for sparse motion fields on the Sintel dataset.
Complementary background models for the detection of static and moving objects in crowded environments

In this paper we propose the use of complementary background models for the detection of static and moving objects in crowded video sequences. One model is devoted to accurately detect motion, while the other aims to achieve a representation of the empty scene. The differences in foreground detection of the complementary models are used to identify new static regions. A subsequent analysis of the detected regions is used to ascertain if an object was placed in or removed from the scene. Static objects are prevented from being incorporated into the empty scene model. Removed objects are rapidly dropped from both models. In this way, we build a very precise model of the empty scene and improve the foreground segmentation results of a single background model. The system was validated with several public datasets, showing many advantages over state-of-the-art static objects and foreground detectors.
Detection of static objects for the task of video surveillance

Detecting static objects in video sequences has a high relevance in many surveillance scenarios like airports and railwaystations. In this paper we propose a system for the detection of static objects in crowded scenes that, based on the detection of two background models learning at different rates, classifies pixels with the help of a finite-state machine. The background is modelled by two mixtures of Gaussians with identical parameters except for the learning rate. The state machine provides the meaning for the interpretation of the results obtained from background subtraction and can be used to incorporate additional information cues, obtaining thus a flexible system specially suitable for real-life applications. The system was built in our surveillance application and successfully validated with several public datasets.
A gradient based approach for stereoscopic error concealment

Error concealment is an important field of research in image processing. Many methods have been applied to conceal block losses in monocular images. We present a concealment strategy for block loss in stereoscopic image pairs. Unlike the error concealment techniques used for monocular images, the information of the associated image is utilized, i.e., by means of a projective transformation model, pixel values from the associated stereo image are warped to their corresponding positions in the lost block. The stereoscopic depth perception is much less affected in our approach than using monoscopic error concealment techniques.
Detecting people carrying objects based on an optical flow motion model

Detecting people carrying objects is a commonly formulated problem as a first step to monitor interactions between people and objects. Recent work relies on a precise foreground object segmentation, which is often difficult to achieve in video surveillance sequences due to a bad contrast of the foreground objects with the scene background, abrupt changing light conditions and small camera vibrations. In order to cope with these difficulties we propose an approach based on motion statistics. Therefore we use a Gaussian mixture motion model (GMMM) and, based on that model, we define a novel speed and direction independent motion descriptor in order to detect carried baggage as those regions not fitting in the motion description model of an average walking person. The system was tested with the public dataset PETS2006 and a more challenging dataset including abrupt lighting changes and bad color contrast and compared with existing systems, showing very promissing results.
Shape-adaptive DCT for generic coding of video

A low complexity shape-adaptive DCT algorithm suitable for coding pels in arbitrarily shaped image segments is introduced. In contrast to other techniques described in literature the proposed algorithm is based on predefined orthogonal sets of DCT basis functions and does not require more computations than a normal block DCT. It is shown that the shape-adaptive DCT algorithm can be easily incorporated into existing block-based JPEG, H.261, or MPEG coding schemes. Thus segment or object based coding of images and video can be provided with backward compatibility to existing coding standards. As an important feature, with the proposed technique additional content based functionalities currently discussed in the MPEG-4 standardization phase can be readily achieved.
Confocal Disparity Estimation and Recovery of Pinhole Image for Real-Aperture Stereo Camera Systems

A single dense depth estimation using stereo or defocus cannot produce a reliable result due to the ambiguity problem. In this paper, we propose a novel anisotropic disparity estimation embedding a stereo confocal constraint for real-aperture stereo camera systems. If the focal length of a real-aperture stereo camera is just changed, the depth range is localized in a focused object which can be discriminated from defocused blurring. The focal depth plane is estimated by the displacement of tensors which are derived from generalized 2D Gaussian, since the point spread functions (PSF) in defocused blurring can be approximated by a shift-invariant Gaussian function. We localize the isotropic propagation in blurring over invariance by a sparse Laplacian kernel in Poisson solution. The matching of real-aperture stereo images is performed by observing the focal consistency. However, the isotropic propagation cannot exactly hold a non-parallel surface to the lens plane i.e., unequifocal surface. An anisotropic regularization term is employed to suppress the isotropic propagation near the non-parallel surface boundary. Our method achieves an accurate dense disparity map by sampling the disparities in focal points from multiple defocus stereo images. The pels in focal points are utilized to recover the pinhole image (i.e. an ideally focused image for all different depths).
On building decentralized wide-area surveillance networks based on ONVIF

In this paper we present a decentralized surveillance network composed of IP video cameras, analysis devices and a central node which collects information and displays it in a 3D model of the complete area. The exchange of information between all components in the surveillance network takes place according to the ONVIF specification, therefore ensuring interoperability between products complying with the specification and flexibility regarding the integration of new devices and services. The collected information is displayed in a 3D model of the surveilled area, therefore providing a comfortable overview of the activity in large environments and offering the user an intuitive way to eventually interact with network devices.
Robust local optical flow estimation using bilinear equations for sparse motion estimation

This article presents a theoretical framework to decrease the computation effort of the Robust Local Optical Flow method which is based on the Lucas Kanade method. We show mathematically, how to transform the iterative scheme of the feature tracker into a system of bilinear equations and thus estimate the motion vectors directly by analyzing its zeros. Furthermore, we show that it is possible to parallelise our approach efficiently on a GPU, thus, outperforming the current OpenCV-OpenCL implementation of the pyramidal Lucas Kanade method in terms of runtime and accuracy. Finally, an evaluation is given for the Middlebury Optical Flow and the KITTI datasets.
Quadtree-based temporal trajectory filtering

In both the HEVC draft and in H.264/AVC, in-loop filters are employed to improve the subjective and the objective quality of compressed video sequences. These filters use spatial information from a single frame only. Temporal Trajectory Filtering (TTF) constitutes an alternative approach which performs filtering in the temporal domain instead. In this work, a combination of the TTF with a quadtree partitioning algorithm for applying different filter parameters to different image regions is proposed and investigated. Experiments were conducted in the environment of the HEVC test model HM 3.0. Bit rate reductions of up to 9% for the low delay high efficiency setting of HEVC are reported.
Adaptive global motion temporal prediction for video coding

Depending on the content of a video sequence and the settings used for encoding it, the amount of bits spent for the transmission of motion vector information can be enormous and in some cases even take the largest fraction of the bit rate. This is not always necessary since often wide areas, i.e. background or large foreground regions, fit the same global motion. Additionally, a global motion model using sophisticated interpolation techniques can be a better representation of movement in these regions than a motion vector that has only quarter-pel accuracy. This is true especially if scaling, rotation or perspective transformation occur. This paper presents a novel prediction technique that is based on global motion compensation and temporal filtering of previously decoded pictures. The new approach is incorporated into an H.264/AVC reference software. The new encoder outperforms the reference by up to 14%.
Audio content description with wavelets and neural nets

Precision audio content description is one of the key components of next generation Internet multimedia search machines. We examine the usability of a combination of 39 different wavelets and three different types of neural nets for precision audio content description. More specifically, we develop a novel wavelet dispersion measure that measures obtained ranks of wavelet coefficients. Our dispersion measure in conjunction with a probabilistic radial basis neural network trained by only three independent example sets obtains a success rate of approximately 78% in identifying unknown complex classical music movements.
Electrical analogs for monitoring vascular properties in artificial heart studies

The problem of choosing parametric descriptions of the systematic vascular bed suitable for monitoring beat-to-beat changes in peripheral vascular properties is considered. Three simple models with two, three, and five elements are compared, essentially exploiting the Akaike information criterion combined with reasonable requirements for estimate accuracy. Analysis of pressure data, which are either simulated by the five-element model or measured on a mock circulatory system during abrupt changes in peripheral resistance, suggests guidelines for model selection. In particular, the five-element model exhibits very close adherence to physical reality by allowing for reflection, while the classical windkessel model provides the most reproducible estimates.<<ETX>>
MPEG-4 very low bit rate video

The MPEG video group is currently developing the so-called MPEG-4 video coding standard, targeted for future interactive multimedia video communications calling for content-based functionalities, universal access in error prone environments and high coding efficiency. Besides the provisions for content-based functionalities the MPEG-4 video standard will assist the efficient storage and transmission of very low bit rate video in error prone environments. This paper outlines the techniques that are currently being investigated by MPEG-4 to provide universal accessibility of video at very low bit rates and discusses the scope of some of the promising techniques under discussion.
Robust modified L2 local optical flow estimation and feature tracking

This paper describes a robust method for the local optical flow estimation and the KLT feature tracking performed on the GPU. Therefore we present an estimator based on the L2 norm with robust characteristics. In order to increase the robustness at discontinuities we propose a strategy to adapt the used region size. The GPU implementation of our approach achieves real-time (>25 fps) performance for High Definition (HD) video sequences while tracking several thousands of points. The benefit of the suggested enhancement is illustrated on the Middlebury optical flow benchmark.
Audiovisual Anchorperson Detection for Topic-Oriented Navigation in Broadcast News

This paper presents a content-based audiovisual video analysis technique for anchorperson detection in broadcast news. For topic-oriented navigation in newscasts, a segmentation of the topic boundaries is needed. As the anchorperson gives a strong indication for such boundaries, the presented technique automatically determines that high-level information for video indexing from MPEG-2 videos and stores the results in an MPEG-7 conform format. The multimodal analysis process is carried out separately in the auditory and visual modality, and the decision fusion forms the final anchorperson segments
A universal image coding approach using sparse steered Mixture-of-Experts regression

Our challenge is the design of a “universal” bit-efficient image compression approach. The prime goal is to allow reconstruction of images with high quality. In addition, we attempt to design the coder and decoder “universal”, such that MPEG-7-like low-and mid-level descriptors are an integral part of the coded representation. To this end, we introduce a sparse Mixture-of-Experts regression approach for coding images in the pixel domain. The underlying stochastic process of the pixel amplitudes are modelled as a 3-dimensional and multi-modal Mixture-of-Gaussians with K modes. This closed form continuous analytical model is estimated using the Expectation-Maximization algorithm and describes segments of pixels by local 3-D Gaussian steering kernels with global support. As such, each component in the mixture of experts steers along the direction of highest correlation. The conditional density then serves as the regression function. Experiments show that a considerable compression gain is achievable compared to JPEG for low bitrates for a large class of images, while forming attractive low-level descriptors for the image, such as the local segmentation boundaries, direction of intensity flow and the distribution of these parameters over the image.
Multistate vs. single-state video coding over error-prone channels

Multiple description coding (MDC) is a forward error correction scheme where two or more descriptions of the source are sent to the receiver over different channels. If only one description is received the signal can be reconstructed with distortion D/sub 1/ or D/sub 2/. If both channels are received on the other hand, the combined information is used to achieve a lower distortion D/sub 0/. Multistate video coding (MSVC) is a specific MDC scheme where the video is coded into multiple independently decodable streams each with its own prediction process and state. In this paper we compare MSVC to single description coding (SDC) at different loss rates and coding options under the assumption that the motion vectors are always available. Results show that when motion vectors are received, SDC performs better than MSVC at every coding option tested. The performance difference is bigger for low motion sequences.
Visualization and navigation in image database applications based on MPEG-7 descriptors

The MPEG-7 standard is currently being developed to specify standardized interfaces that help users or agents to identify, filter, browse and efficiently retrieve audio-visual material. The purpose of the presentation is to provide an overview of the scope and potential of the MPEG-7 standard, with particular emphasis on no-text descriptors for visual content. Based on the MPEG-7 specifications, the implementation of a wealth of applications is possible. Various application scenarios are presented and discussed.
A block-adaptive skip mode for inter prediction based on parametric motion models

Motion compensated prediction (MCP) in hybrid video coding estimates a translational motion vector for a given block which is then used for residual computation. However, when complex motion like zoom, rotation, and perspective transformation occur, the translational model assumption does not always hold. This may result in higher residual energy and splitting of blocks, respectively. This paper proposes a skip mode based on higher-order parametric motion models. Often, these models provide a better prediction quality resulting in lower residual energy and larger block sizes. The proposed technique estimates a higher-order motion model between two given pictures. The encoder decides in terms of rate-distortion optimization whether to use the new skip mode for a block and therefore not to transfer any additional information like coefficient data. Experimental evaluation shows that the proposed technique can improve the coding performance of next generation video coding standards significantly.