nils-herrmann/clean_scirepeval huggingface dataset:
Perceptual Video Compression: A Survey

With the advances in understanding perceptual properties of the human visual system and constructing their computational models, efforts toward incorporating human perceptual mechanisms in video compression to achieve maximal perceptual quality have received great attention. This paper thoroughly reviews the recent advances of perceptual video compression mainly in terms of the three major components, namely, perceptual model definition, implementation of coding, and performance evaluation. Furthermore, open research issues and challenges are discussed in order to provide perspectives for future research trends.
Phase-Amplitude Coupling between EEG and EDA While Experiencing Multimedia Content

Emotion is a dynamic process that affects social relationships and influences the mechanisms of rational thinking and decision making. The influence of emotion on reasoning has gained a lot of attention in computer science. One approach to incorporating emotions into computers is by extracting features from various physiological signals and inferring the corresponding emotions or emotional dimensions. However, physiological processes are dynamic rather than static, and physiological signals mainly interact with each other during emotional processes rather than act individually. This study suggests that dynamical interdependencies of various signals that emanate from either the peripheral nervous system or the central nervous system might be involved in the emotional experience. Specifically, EEG (electroencephalography) signals captured from the temporal lobe are coupled with a narrow-band skin conductance response (SCR) while subjects are watching music clips, indicating that signals from the auditory cortex interact with SCR during this experience.
Correlative exploration of EEG signals for direct brain-computer communication

In this study we present a method for classifying EEG signals based on the information content of their correlative time-frequency-space representation (CTFSR). A support vector machine (SVM) kernel is proposed that can be calculated in the time domain while it computes a similarity measure in the CTFSR space. This classification method is used in a brain-computer interface (BCI) application. The use of the SVM approach allows us to propose a simple strategy for adapting the BCI to possible long term variations in the brain activity.
Video coding based on audio-visual attention

This paper proposes an efficient video coding method based on audio-visual attention, which is motivated by the fact that cross-modal interaction significantly affects humans' perception of multimedia content. First, we propose an audio-visual source localization method to locate the sound source in a video sequence. Then, its result is used for applying spatial blurring to video frames in order to reduce redundant high-frequency information and achieve coding efficiency. We demonstrate the effectiveness of the proposed method for H.264/AVC coding along with the results of a subjective evaluation.
Progressive video coding for storage applications

This paper addresses the problem of video coding for storage applications. Storage applications require several features to a coding scheme such as robustness to dubbing, trick modes and the possibility of progressive transmission with an exact rate control. A coding scheme is proposed which is robust to dubbing for an unlimited number of iterations. Furthermore, it is shown that an exact rate control with possibility of progressive transmission is achieved. Simulation results for a compression ratio of 2:1 and 4:1 are presented.
Image quality prediction for bitrate allocation

In image coding, the choice of a good image coding algorithm is very dependent on the image content. Based on this fact, dynamic coding algorithms have been designed. They try to find an optimal coding scheme for each image segment. They rely on an exhaustive search of the best coding algorithm. Evaluation of all algorithms is computationally very intensive and strongly limits the number of considered algorithms for a given application. Therefore, current standards rely on a single coding algorithm. This paper investigates a way to predict the coding quality from the image content. This prediction is based on a neural network. The coding quality is computed from image region features. Those features are easy and fast to compute, and are common to the whole set of considered coding algorithms. Therefore, the choice of the best algorithm can be based on those predicted coding qualities, and does not require the computation of all coding algorithms. The system is also fast enough to be used for dynamic bitrate allocation, and a simple algorithm to do this is proposed.
Secure JPEG scrambling enabling privacy in photo sharing

With the popularization of online social networks (OSNs) and smart mobile devices, photo sharing is becoming a part of people' daily life. An unprecedented number of photos are being uploaded and shared everyday through online social networks or photo hosting services, such as Facebook, Twitter, Instagram, and Flickr. However, such unrestrained online photo or multimedia sharing has raised serious privacy concerns, especially after reports of citizens surveillance by governmental agencies and scandalous leakage of private photos from prominent photo sharing sites or online cloud services. Popular OSNs typically offer privacy protection solutions only in response to the public demand and therefore are often rudimental, complex to use, and provide limited degree of control and protection. Most solutions allow users to control either who can access the shared photos or for how long they can be accessed. In contrast, in this paper, we take a structured privacy by design approach to the problem of online photo privacy protection. We propose a privacy-preserving photo sharing architecture based on a secure JPEG scrambling algorithm capable of protecting the privacy of multiple users involved in a photo. We demonstrate the proposed photo sharing architecture with a prototype application called ProShare that offers JPEG scrambling as the privacy protection tool for selected regions in a photo, secure access to the protected images, and secure photo sharing on Facebook.
A Boosting Approach to P300 Detection with Application to Brain-Computer Interfaces

Gradient boosting is a machine learning method, that builds one strong classifier from many weak classifiers. In this work, an algorithm based on gradient boosting is presented, that detects event-related potentials in single electroencephalogram (EEG) trials. The algorithm is used to detect the P300 in the human EEG and to build a brain-computer interface (BCI), specifically a spelling device. Important features of the method described here are its high classification accuracy and its conceptual simplicity. The algorithm was tested with datasets recorded in our lab and one benchmark dataset from the BCI Competition 2003. The number of correctly inferred symbols with the P300 speller paradigm varied between 90% and 100%. In particular, all of the inferred symbols were correct for the BCI competition dataset
Dynamic approach to visual data compression

This paper presents the Swiss Federal Institute of Technology (EPFL) proposal to MPEG-4 video coding standardization activity. The proposed technique is based on a novel approach to audio-visual data compression entitled dynamic coding. The newly born multimedia environment supports a plethora of applications which cannot be covered adequately by a single compression technique. Dynamic coding offers the opportunity to combine several compression techniques and segmentation strategies. Given a particular application, these two degrees of freedom can be constrained and assembled in order to produce a particular profile which meets the set of specifications dictated by the application. The basic principles of this approach are presented together with the data representation system. The major characteristics of dynamic coding are reviewed, along with simulation results showing the performance of such an approach in a very low bit-rate video coding environment.
Social game epitome versus automatic visual analysis

With the rapid growth of digital photography, sharing of photos with friends and family has become very popular. When people share their photos, they usually organize them in albums according to events or places. To tell the story of some important events in one's life, it is desirable to have an efficient summarization tool which can help people to get a quick overview of an album containing huge number of photos. In this paper, we analyze an approach for photo album summarization through a novel social game “Epitome” as a Facebook application. This social game can collect research data and, at the same time, it provides a collage or a cover photo of the user's photo album, while, at the same time, the user enjoys playing the game. As a benchmark comparison to this game, we performed automatic visual analysis considering several state-of-the-art features.
Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention

This paper presents a foveated coding method using audio-visual focus of attention and its evaluation through extensive subjective experiments on both standard-definition and high-definition sequences. Regarding a sound-emitting region as the location drawing the human attention, the method applies varying quality levels in an image frame according to the distance of a pixel to the identified sound source. Two experiments are presented to prove the efficiency of the method. Experiment 1 examines the validity and effectiveness of the method in comparison to the constant quality coding for high-quality conditions. In Experiment 2, the method is compared to the fixed bit rate coding for low quality conditions where coding artifacts are noticeable. The results demonstrate that the foveated coding method provides considerable coding gain without significant quality degradation, but uneven distributions of the coding artifacts (blockiness) by the method are often less preferred than the uniform distribution of the artifacts. Additional interesting findings are also discussed, such as content dependence of the performance of the method, the memory effect in multiple viewings, and the difference in the quality perception for frame size variations.
Privacy-preserving photo sharing based on a secure JPEG

Sharing photos online is a common activity on social networks and photo hosting platforms, such as Facebook, Pinterest, Instagram, or Flickr. However, after reports of citizens surveillance by governmental agencies and the scandalous leakage of celebrities private photos online, people have become concerned about their online privacy and are looking for ways to protect it. Popular social networks typically offer privacy protection solutions only in response to the public demand and therefore are often rudimental, complex to use, and provide limited degree of control and protection. Most solutions either allow users to control who can access the shared photos or for how long they can be accessed. In contrast, in this paper, we take a structured privacy by design approach to the problem of online photo privacy protection. We propose a privacy-preserving photo sharing architecture that takes into account content and context of a photo with privacy protection integrated inside the JPEG file itself in a secure way. We demonstrate the proposed architecture with a prototype mobile iOS application called ProShare that offers scrambling as the privacy protection tool for a selected region in a photo, secure access to the protected images, and secure photo sharing on Facebook.
Privacy in mini-drone based video surveillance

Mini-drones are increasingly used in video surveillance. Their areal mobility and ability to carry video cameras provide new perspectives in visual surveillance which can impact privacy in ways that have not been considered in a typical surveillance scenario. To better understand and analyze them, we have created a publicly available video dataset of typical drone-based surveillance sequences in a car parking. Using the sequences from this dataset, we have assessed five privacy protection filters via a crowdsourcing evaluation. We asked crowdsourcing workers several privacy- and surveillance-related questions to determine the tradeoff between intelligibility of the scene and privacy, and we present conclusions of this evaluation in this paper.
Graph-based approach for 3D object duplicate detection

In this paper, we consider the challenging problem of object duplicate detection and localization. Several applications require efficient object duplicate detection methods, such as automatic video and image tagging, video surveillance, and high level image or video search. In this paper, a novel graph-based approach for 3D object duplicate detection in still images is proposed. A graph model is used to represent the spatial information of the object in order to avoid making an explicit 3D object model. Therefore, better performance is achieved in terms of robustness and computational complexity.
High-performance compression of visual information-a tutorial review. I. Still pictures

Digital images have become an important source of information in the modern world of communication systems. In their raw form, digital images require a tremendous amount of memory. Many research efforts have been devoted to the problem of image compression in the last two decades. Two different compression categories must be distinguished: lossless and lossy. Lossless compression is achieved if no distortion is introduced in the coded image. Applications requiring this type of compression include medical imaging and satellite photography. For applications such as video telephony or multimedia applications, some loss of information is usually tolerated in exchange for a high compression ratio. In this two-part paper, the major building blocks of image coding schemes are overviewed. Part I covers still image coding, and Part II covers motion picture sequences. In this first part, still image coding schemes have been classified into predictive, block transform, and multiresolution approaches. Predictive methods are suited to lossless and low-compression applications. Transform-based coding schemes achieve higher compression ratios for lossy compression but suffer from blocking artifacts at high-compression ratios. Multiresolution approaches are suited for lossy as well for lossless compression. At lossy high-compression ratios, the typical artifact visible in the reconstructed images is the ringing effect. New applications in a multimedia environment drove the need for new functionalities of the image coding schemes. For that purpose, second-generation coding techniques segment the image into semantically meaningful pairs. Therefore, parts of these methods have been adapted to work for arbitrarily shaped regions. In order to add another functionality, such as progressive transmission of the information, specific quantization algorithms must he defined. A final step in the compression scheme is achieved by the codeword assignment. Finally, coding results are presented which compare state-of-the-art techniques for lossy and lossless compression. The different artifacts of each technique are highlighted and discussed. Also, the possibility of progressive transmission is illustrated.
Towards the need satisfaction in gaming: A comparison of different gaming platforms

Recent advances in Virtual Reality (VR) technologies have resulted in a wider availability of Head Mounted Displays (HMDs). However, it is still unclear if VR gaming offers a substantial added value to players. For this reason a comparison of gaming experiences on VR HMD to those on mobile and PC, two other popular gaming platforms, is performed by conducting a user study via two games available on all three platforms. We explore the QoE of gaming by investigating momentous dimensions using the Player Experience of Need Satisfaction (PENS) questionnaire. The results show higher Presence and Autonomy obtained by using HMD when compared to the two other platforms. However, these factors alone did not improve the Overall Quality. To take advantage of the new technology, satisfaction of all psychological needs, especially Competency, must be assured.
UHD video dataset for evaluation of privacy

Ultra High Definition (UHD) is one of the emerging immersive video technologies already available to public, as even some of the smartphones are capable of capturing UHD video. The increasingly widespread availability of UHD capable recording devices has important implications on privacy. This paper addresses the problem by proposing a publicly available UHD video dataset designed for evaluation of privacy issues. The dataset depicts typical surveillance scenarios of people fighting, exchanging bags, walking, and stealing, in an indoor and outdoor environment. The dataset also includes the data from the subjective assessment, which evaluated the impact of UHD on privacy compared to a currently common High Definition (HD) video and declining Standard Definition (SD) video. The results of the assessment not only demonstrate that UHD is a significantly more privacy intrusive technology when compared to HD and SD used today, but they also quantify the impact of intrusivness.
Video segmentation based on multiple features for interactive multimedia applications

We present a scheme for interactive video segmentation. A key feature of the system is the distinction between two levels of segmentation, namely, regions and object segmentation. Regions are homogeneous areas of the images, which are extracted automatically by the computer. Semantically meaningful objects are obtained through user interaction by grouping of regions according to the specific application. This splitting relieves the computer of ill-posed semantic problems, and allows a higher level of flexibility of the method. The extraction of regions is based on the multidimensional analysis of several image features by a spatially constrained fuzzy C-means algorithm. The local level of reliability of the different features is taken into account in order to adaptively weight the contribution of each feature to the segmentation process. Results on the extraction of regions as well as on the tracking of spatiotemporal objects are presented.
Orientation histogram-based matching for region tracking

A region tracking technique with particular emphasis on rotation robustness is presented. It is based on region matching divided in two consecutive steps, gradient orientation histogram matching and template matching through normalised cross correlation (NCC). Given the orientation histograms of two image patches, a novel technique is used to estimate the rotation between them together with the similarity. This estimation enhances the performance and speeds-up the process of patch recognition. Fast computation of histograms using the integral histogram approach is exploited. Experiments show a high accuracy in the estimation of location and orientation.
New trends in very low bitrate video coding

The interest in very low birate video coding has increased considerably with the creation of new services and applications requiring this class of bitrates in their function. This paper aims in presenting the current problems and requirements, as well as the efforts in very low bitrate video coding. It also provides a number of solutions in order to solve these problems and to fulfill desired requirements in very, low bitrate coding. The main emphasis was given to an approach called "second generation" which has already shown its potential to efficiently code still images at higher compression factors. In particular, three techniques will be discussed in details, because of their ability to successfully achieve very low bitrate video coding with a reasonably good quality. The simulation results obtained by these techniques under the exact same conditions are presented and the merits and drawbacks of each one are pointed out throughout the paper.
Arbitrarily-shaped wavelet packets for zerotree coding

In order to satisfy the needs of new multimedia applications, the problem of content-based video coding has to be addressed. A new approach of object interior coding is proposed. It is based on an arbitrarily-shaped subband transform followed by a generalized embedded zerotree wavelet algorithm. It is shown that the proposed technique achieves good compression results and has additional properties such as being computationally efficient, keeping the same dimensionality in the transformed domain, being perfect reconstruction, and allowing a perfect rate control. In addition a lossless mode can be defined by using an appropriate filter bank.
Hybrid spatial and temporal error concealment for distributed video coding

Distributed video coding (DVC) is based on a new paradigm in coding, which has received many interests recently. This paper proposes a hybrid spatial and temporal error concealment scheme to conceal errors in Wyner-Ziv (WZ) frames. We first use a spatial concealment based on edge directed filter. This step is exploited to improve the performance of subsequent temporal concealment. An enhanced temporal concealment based on motion compensated temporal interpolation is proposed, including motion vector refinement and smoothing, optimal compensation mode selection, and a new matching criterion for motion estimation. Simulation results show that the objective qualities as well as the perceptual qualities of the corrupted sequences are significantly improved by the hybrid error concealment, outperforming both spatial and temporal concealments alone.
Progressive polygon encoding of shape contours

This paper presents a new progressive contour coding method based on digital polygonal approximation. By applying digital geometry concepts such as digital contours, segments and distances at the approximation stage, it is possible to design an inherently progressive encoding scheme which directly exploits the resulting geometrical knowledge. Experimental results show a performance similar to that of state of the art polygon encoding methods that do not achieve progressive transmission. Applications include indexing and retrieval of arbitrarily shaped objects in image databases, as well as composition and manipulation of such image objects in the compressed domain as targeted by the future video compression standard MPEG-4.
Secure JPEG 2000-JPSEC

The Joint Photographic Experts Group (JPEG) has recently rolled out a new still image coding standard called JPEG 2000. This standard integrates an efficient image compression scheme with functionalities required by multimedia applications, such as progressiveness up to lossless coding, region of interest coding, and error resiliency. Security is a concern in many applications, and therefore also a desired functionality. This paper provides readers with insights and examples of how to combine security solutions with JPEG 2000 compression. Tools for JPEG 2000 compressed image integrity, access control, and copyright protection are presented. They can be either applied to a JPEG 2000 codestream or directly integrated into the coding/decoding operations, resulting in a fully compliant JPEG 2000 image.
Performance evaluation of two state of the art DVC codecs

The performance of existing DVC codecs is still lacking relative to that of H.264 and work is being carried out in order to close this gap. The authors of this paper have been and still are involved in the development of two DVC codecs respectively, the performance of which is compared herein. The aim is to identify strengths and weaknesses of the two codecs that can be exploited / addressed in order to improve the achieved performance relative to H.264.