nils-herrmann/clean_scirepeval huggingface dataset:
Extending Single-View Scalable Video Coding to Multi-View Based on H.264/AVC

An extension of single-view scalable video coding to multi-view is presented in this paper. Scalable video coding is recently developed in the Joint Video Team of ISO/IEC MPEG and ITU-T VCEG named Joint Scalable Video Model. The model includes temporal, spatial and quality scalability enhancing a H.264/AVC base layer. To remove redundancy between views a hierarchical decomposition in a similar way to the temporal direction is applied. The codec is based on this technology and supports open-loop as well as closed-loop controlled encoding. The advantage of this approach lies in its compatibility to the state of the art single-view video codec H.264/AVC and its simple decomposition structure. Encoding a base view using H.264/AVC syntax, any standard single-view decoder is able to decode the data. The hierarchical decomposition structure allows efficient access to all views and frames inside a view. This is especially important for video-based-rendering and multi-view displays, which have different requirements. The chosen decomposition structure also supports parallel processing. Gain in objective as well as subjective quality was achieved for some test sequences using a single layer. The results were compared to JSVM 5.1 (simulcast).
A decentralized privacy-sensitive video surveillance framework

With the increasing spread of accurate and robust video surveillance, applications such as crowd monitoring, people counting and abnormal behavior recognition become ubiquitous.This leads to needs of interactive systems taking into account a high degree of interoperability as well as privacy protection concerns. In this paper we propose a framework based on the ONVIF specification to support the work of video operators while implementing a privacy-by-design concept. We use an OpenGL-based 3D model of the CCTV site where we display the results of the video analytics in an avatar-based manner and give an example application on mugging detection. To place the automatically detected scene information, such as people detections and events, an automatic camera calibration is used which effectively reduces the deployment effort.
A novel inloop filter for video-compression based on temporal pixel trajectories

The objective of this work is to investigate the performance of a new inloop filter for video compression, which uses temporal rather than spatial information to improve the quality of reference frames used for prediction. The new filter has been integrated into the H.264/AVC baseline encoder and tested on a wide range of sequences. Experimental results show that the filter achieves a bit rate reduction of up to 12% and more than 4% on average without increasing the complexity of either encoder or decoder significantly.
Short-term motion-based object segmentation

Motion-based segmentation approaches employ either longterm motion information, which is computationally demanding, or suffer from lack of accuracy when employing short-term information. We present an automatic motion-based object segmentation algorithm for video sequences with moving camera, employing short-term motion information solely. For every frame, two error frames are generated using motion compensation. They are combined and a thresholding segmentation algorithm is applied. Recent advances in the field of global motion estimation enable outlier elimination in the background area, and thus a more precise definition of the foreground is achieved. We propose a simple and effective error frame generation and consider spatial error localization. Thus, we achieve improved performance compared with a previously proposed short-term motion-based method and provide subjective as well as objective evaluation.
Image Denoising Method Using Diffusion Equation and Edge Map Estimated with K-Means Clustering Algorithm

The purpose of noise filtering is to preserve features such as edge or corners, and to reduce noise. However, it is difficult to preserve them after noise filtering, because most of noise filtering algorithms have the characteristic of low pass filter, which results in the loss of signal components with high frequencies like edge or corners. Noise filtering methods using diffusion equation try to solve this problem, and improve it to some extent. Edge detector has been used in filtering methods with diffusion equation to determine the speed of diffusion term, so the performance of them depends on the edge detection algorithm to some extent. Previous noise filtering approaches of diffusion equation use a simple gradient edge detector to determine the speed of diffusion. Also, each channel of color images are filtered separately, which result in the filtered image with distortion or blurring. In this paper, we propose a new noise filtering method for color images using diffusion equation and edge map estimated with K-means clustering algorithm together.
Weighted temporal long trajectory filtering for video compression

In the context of the HEVC standardization activity, in-loop filters such as the adaptive loop filter and the deblocking filter are currently under investigation. Both filters work in the spatial domain only, despite the temporal correlation within video sequences. In this work a previously introduced filter, that uses temporal information for deblocking and denoising instead, is integrated into the HEVC test model HM 3.0. It is shown how the filter is to be adapted to work in combination with the adaptive loop filter for the HEVC low-delay profile. In addition, an optimal weighting function for the filtered luma samples based on the qunatization parameter is derived. Bit rate reductions of up to 7.6% are reported for individual sequences.
Adaptive global motion temporal filtering

The emerging standardization on high efficiency video coding (HEVC) has brought a huge improvement in terms of coding performance comparing to existing standards and approaches. One tool that provides a significant portion of the coding gain so far is loop filtering. In H.264/AVC, a deblocking filter has been used to reduce blocking artifacts. HEVC added loop filter approaches to this deblocking method that further reduce noise in the decoded video frame. All these techniques work spatially. Besides that, work has been done to further improve the quality of decoded frames by applying a temporal filtering approach. In this work, we propose adaptive global motion temporal filtering (AGMTF) that reduces noise along temporal trajectories. Experimental results show that the coding performance of the current HEVC test model HM 4.0 can be improved by up to 8.8% and 3.7% in average over a large bit rate range using this technique.
Robust local optical flow: Dense motion vector field interpolation

Optical flow methods integrating sparse point correspondences have made significant contribution in the field of optical flow estimation. Especially for the goal of estimating motion accurately and efficiently, sparse-to-dense interpolation schemes for feature point matches have shown outstanding performances. Concurrently, local optical flow methods have been significantly improved with respect to long-range motion estimation in environments with varying illumination. This motivates us to propose a sparse-to-dense approach based on the Robust Local Optical Flow method. Compared to state-of-the-art methods the proposed approach is significantly faster while retaining competitive accuracy on Middlebury, KITTI 2015 and MPI-Sintel data-set.
The MPEG-4 video standard verification model

The MPEG-4 standardization phase has the mandate to develop algorithms for audio-visual coding allowing for interactivity, high compression, and/or universal accessibility and portability of audio and video content. In addition to the conventional "frame"-based functionalities of the MPEG-1 and MPEG-2 standards, the MPEG-4 video coding algorithm will also support access and manipulation of "objects" within video scenes. The January 1996 MPEG Video Group meeting witnessed the definition of the first version of the MPEG-4 video verification model-a milestone in the development of the MPEG-4 standard. The primary intent of the video verification model is to provide a fully defined core video coding algorithm platform for the development of the standard. As such, the structure of the MPEG-4 video verification model already gives some indication about the tools and algorithms that will be provided by the final MPEG-4 standard. The paper describes the scope of the MPEG-4 video standard and outlines the structure of the MPEG-4 video verification model under development.
Optimal Wiener interpolation filters for multiresolution coding of images

A design approach is presented which allows the optimization of coefficients for symmetric and separable finite impulse response (FIR) interpolation filters for multiresolution coding schemes. The interpolation filters serve as optimal inverse filters in the Wiener sense and are designed to match the characteristics of the specific filters used for decimation as well as for the statistics of "typical" images to be reconstructed. Applied to the coding of test images in a four-level progressive pyramid scheme, the optimal interpolation filters generated substantially improved rate-distortion results compared to conventional filters.
Evaluation of pixel- and motion vector-based global motion estimation for camera motion characterization

Pixel-based and motion vector-based global motion estimation (GME) techniques are evaluated in this paper with an automatic system for camera motion characterization. First, the GME techniques are compared with a frame-by-frame PNSR measurement using five video sequences. The best motion vector-based GME method is then evaluated together with a common and a simplified pixel-based GME technique for camera motion characterization. For this, selected unedited videos from the TRECVid 2005 BBC rushes corpus are used. We evaluate how the estimation accuracy of global motion parameters affects the results for camera motion characterization in terms of retrieval measures. The results for this characterization show that the simplified pixel-based GME technique obtains results that are comparable with the common pixel-based GME method, and outperforms significantly the results of an earlier proposed motion vector-based GME approach.
A system for automatic and interactive detection of static objects

Designing static object detection systems that are able to incorporate user interaction conveys a great benefit in many surveillance applications, since some correctly detected static objects can be considered to have no interest by a human operator. Interactive systems allow the user to include these decisions into the system, making automated surveillance systems more attractive and comfortable to use. In this paper we present a system for the detection of static objects that, based on the detection of a dual background model, classifies pixels by means of a finite-state machine. The state machine provides the meaning for the interpretation of the results obtained from background subtraction and it can be optionally used to integrate user input. The system can thus be used both in an automatic and an interactive manner without requiring any expert knowledge from the user. We successfully validated the system with several public datasets.
DCT-based features for categorisation of social media in compressed domain

These days the sharing of videos is very popular in social networks. Many of these social media websites such as Flickr, Facebook and YouTube allows the user to manually label their uploaded videos with textual information. However, the manually labelling for a large set of social media is still boring and error-prone. For this reason we present a algorithm for categorisation of videos in social media platforms without decoding them. The paper shows a data-driven approach which makes use of global and local features from the compressed domain and achieves a mean average precision of 0.2498 on the Blip10k dataset. In comparison with existing retrieval approaches at the MediaEval Tagging Task 2012 we will show the effectiveness and high accuracy relative to the state-of-the art solutions.
MPEG Digital Audio-and Video-Coding Standards

his issue of IEEE Sig­ nal Processing Maga­ zine is dedicated to the audioand-video compression tech­ nology that forms the basis of the in­ ternational MPEG standards. The efficient digital representation of audio/visual signals has been the subject of considerable research over the past 20 years. Digital audioand video-coding technology has devel­ oped into a mature field and diverse products have been developed-tar­ geted for a wide range of emerging audio/visual applications, such as video on d e m a n d , digital TV/HDTV broadcasting, and mul­ timedia image/video a n d audio/speech database services. With the increased commercial in­ terest in sophisticated audio/visual communications, the need for inter­ national standards for compression technology arose. Commercially, international standardization of audio/visual communication systems and proto­ cols aims to serve two important purposes: interoperability and economy of scale. Interworking be­ tween video communication equip­ ment from different vendors is a desirable feature for users and equipment manufacturers alike. It increases the attractiveness for buy­ ing and using video communication equipment because it enables large­ scale, international audio and video data exchange via storage media or via communication networks.
MPEG-4 video and its potential for future multimedia services

The Moving Picture Experts Group (MPEG) committee, which originated the MPEG-1 and MPEG-2 video and audio compression standards, is currently developing MPEG-4 with wide industry participation. MPEG-4 is targeted for interactive multimedia applications and will become an international standard in 1998. It is expected that MPEG-4 will become the enabling technology for multimedia audio-visual communications as much as MPEG-2 has become the enabling technology for digital television. The purpose of the paper is to discuss the scope and potential of the MPEG-4 standardization activities for networked interactive multimedia applications with particular emphasis on the MPEG-4 video standard.
Functional coding of video using a shape-adaptive DCT algorithm and an object-based motion prediction toolbox

This paper presents an object-based layered video coding scheme which achieves very high compression efficiency along with the provision for advanced content-based functionalities, e.g., content-based scalability or content-based access and manipulation of video data. In a first step, a video sequence is segmented into several arbitrarily shaped "object layers." To achieve the desired content-based functionalities, a baseline shape-adaptive discrete cosine transform (DCT) coding algorithm is introduced which can be seen as an extension of conventional block-based DCT coding schemes (e.g., H.261, H.263, MPEG-1, or MPEG-2) toward coding of arbitrarily shaped image content. In order to increase compression efficiency, the baseline object-based layered approach can be extended with an object-based motion prediction toolbox. Using this toolbox, the coding scheme can potentially select specific prediction techniques for every object layer to be coded. To illustrate the concept, an extension of the baseline shape-adaptive DCT algorithm with a technique for global background motion estimation and compensation is described which significantly improves the compression efficiency of suitable video sequences compared to standard MPEG coding schemes.
Splitting Gaussians in Mixture Models

Gaussian mixture models have been extensively used and enhanced in the surveillance domain because of their ability to adaptively describe multimodal distributions in real-time with low memory requirements. Nevertheless, they still often suffer from the problem of converging to poor solutions if the main mode stretches and thus over-dominates weaker distributions. Based on the results of the Split and Merge EM algorithm, in this paper we propose a solution to this problem. Therefore, we define an appropriate splitting operation and the corresponding criterion for the selection of candidate modes, for the case of background subtraction. The proposed method achieves better background models than state-of-the-art approaches and is low demanding in terms of processing time and memory requirements, therefore making it especially appealing in the surveillance domain.
Image interpolation based on image statistics: a case study

Two novel approaches to image interpolation aimed at video service interworking, layered image coding, and image zooming are discussed. Compared to conventional interpolation filters used for image interpolation, the schemes proposed take advantage of prior knowledge of image statistics to enhance their performance. Both methods, inverse Wiener filtering (IWF) and vector interpolation (VI), use image training sequences to estimate optimal interpolation parameters. While IWF has online capabilities for parameter estimation, VI parameters need to be designed in computer intense offline optimization procedures. Both IWF and VI methods achieve superior performance compared to standard linear interpolation filters in terms of signal-to-noise ratio of the interpolated images, as well as entropy of the residual errors.<<ETX>>
A local feature based on lagrangian measures for violent video classification

Lagrangian theory provides a diverse set of tools for continuous motion analysis. Existing work shows the applicability of Lagrangian methods for video analysis in several aspects. In this paper we want to utilize the concept of Lagrangian measures to detect violent scenes. Therefore we propose a local feature based on the SIFT algorithm that incooperates appearance and Lagrangian based motion models. We will show that the temporal interval of the used motion information is a crucial aspect and study its influence on the classification performance. The proposed LaSIFT feature outperforms other state-of-the-art local features, in particular in uncontrolled realistic video data. We evaluate our algorithm with a bag-of-word approach. The experimental results show a significant improvement over the state-of-the-art on current violence detection datasets, i.e. Crowd Violence, Hockey Fight.
Adaptive Global Motion Temporal Filtering for High Efficiency Video Coding

Coding artifacts in video codecs can be reduced using several spatial in-loop filters that are part of the emerging video coding standard High Efficiency Video Coding (HEVC). In this paper, we introduce the concept of global motion temporal filtering. A theoretical framework for a concept combining the temporal overlapping of several noisy versions of the same signal is introduced. This includes a model of the motion estimation error. As an important result, it is shown that an optimum number of frames N for filtering exists. An implementation of the concept based on several versions of the HEVC test model using global motion-compensated temporal filtering shows that significant gains can be achieved.
Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms

In a broad range of computer vision tasks, convolutional neural networks (CNNs) are one of the most prominent techniques due to their outstanding performance. Yet it is not trivial to find the best performing network structure for a specific application because it is often unclear how the network structure relates to the network accuracy. We propose an evolutionary algorithm-based framework to automatically optimize the CNN structure by means of hyper-parameters. Further, we extend our framework towards a joint optimization of a committee of CNNs to leverage specialization and cooperation among the individual networks. Experimental results show a significant improvement over the state-of-the-art on the well-established MNIST dataset for hand-written digits recognition.
Human action recognition using Lagrangian descriptors

Human action recognition requires the description of complex motion patterns in image sequences. In general, these patterns span varying temporal scales. In this context, Lagrangian methods have proven to be valuable for crowd analysis tasks such as crowd segmentation. In this paper, we show that, besides their potential in describing large scale motion patterns, Lagrangian methods are also well suited to model complex individual human activities over variable time intervals. We use Finite Time Lyapunov Exponents and time-normalized arc length measures in a linear SVM classification scheme. We evaluated our method on the Weizmann and KTH datasets. The results demonstrate that our approach is promising and that human action recognition performance is improved by fusing Lagrangian measures.
Image browsing with PCA-assisted user-interaction

User interfaces for sophisticated search engines must offer users quick and easy access to the objects to be visualized. We present a browsing tool which arranges images with respect to the user search intention in a continuous and intuitive manner in real time. Since the capacity of the visual human system is higher for spatial information, we prefer a virtual 3D space for the visualization. Because our image features are described in terms of very high-dimensional MPEG-7 descriptors, we have to reduce them to only three dimensions for visual presentation. The dimension reduction is realized by an appropriate weighting of the high-dimensional descriptor components corresponding to a modification of the covariance-matrix used for principal component analysis (PCA). In addition, this modification allows us to overcome a problem arising from equally sized eigenvalues and provides varying eigenspaces nearly continuously. The technique introduced is a general approach, which can be combined with other relevance feedback methods.
A motion-enhanced hybrid Probability Hypothesis Density filter for real-time multi-human tracking in video surveillance scenarios

The Probability Hypothesis Density (PHD) filter is a multi-object Bayes filter which has been recently becoming popular in the tracking community especially for its linear complexity and its ability to filter out a high amount of clutter. However, its application to Computer Vision scenarios can be difficult as it requires high detection probabilities. Many human detectors suffer from a significant miss-match rate which causes problems for the PHD filter. This article presents an implementation of a Gaussian Mixture PHD (GM-PHD) filter which is enhanced by Optical Flow information in order to account for missed detections. We give a detailed mathematical discussion for the parameters of the proposed system and justify our results by extensive tests showing the performance in several contexts and on different datasets.
Evaluation of distance measures for MPEG-7 melody contours

In query by humming (QBH) systems, the melody contour is often used as a symbolic description of music. The MelodyContour description scheme (DS) defined by MPEG-7 is a standardized representation of melody contours. For melody comparison in a QBH system, a distance measure is required. This paper evaluates different distance measures for the MFEG-7 MelodyContour DS. The use of each measure is discussed.