nils-herrmann/clean_scirepeval huggingface dataset:
Fast H.264 Mode Decision Using Previously Coded Information

The seven prediction modes for P-slice provided in the inter-frame coding tool of H.264/AVC enables significant compression gain but at the cost of computational complexity. To address the issue, we propose in this paper a fast mode decision algorithm that is able to reduce the computational complexity of this coding tool by exploiting the coding modes of previously coded neighboring blocks and by leveraging the rate-distortion characteristics of processed modes. The target efficiency is achieved because the combination of various techniques used in the algorithm incurs little overhead. The algorithm provides an average 70% saving of encoding time for QCIF video and 73% for CIF video in comparison with the brute-force scheme, with only 0.032 dB PSNR drop or, equivalently, 0.38% bit rate increase.
Complexity-aware live streaming system

The number of client requests that a streaming server can handle is limited by both its computational resources and available bandwidth. While bandwidth capacity is critical for most streaming applications, computational resources for a server encoding live videos often become a critical factor as well. In order to serve more client requests or provide higher quality for high priority clients, it is desirable to allocate and adjust the computational resources on a per channel basis. In this paper, we proposed a complexity-aware live video streaming server system that manages the computational resources dynamically. In the proposed system, input videos are encoded with different quality levels based on their priorities and available computational resources. The computational resources for each encoder are adaptively allocated to match the time constraints. The seven quality levels defined in the XviD MPEG-4 encoder are used in our experiments. The results show that the new design is able to maximize the resource utilization by maintaining the highest priority channels quality while providing the other channels with best-effort quality.
A simulation tool for digital autofocus design

To provide smooth viewing experience, the design of continuous autofocus of video cameras needs to take the scene dynamics into consideration in the algorithm development and testing cycles. However, this is often a time-consuming and tedious process because it requires a huge amount of tests on various real scenes. We propose in this paper a simulation method to simplify the digital autofocus design process. This design-by-simulation method significantly reduces the development and testing time of the search strategy for digital autofocus. Experimental results are shown to demonstrate the effectiveness of the proposed method.
Linear regression-based adaptation of music emotion recognition models for personalization

Personalization techniques can be applied to address the subjectivity issue of music emotion recognition, which is important for music information retrieval. However, achieving satisfactory accuracy in personalized music emotion recognition for a user is difficult because it requires an impractically huge amount of annotations from the user. In this paper, we adopt a probabilistic framework for valence-arousal music emotion modeling and propose an adaptation method based on linear regression to personalize a background model in an online learning fashion. We also incorporate a component-tying strategy to enhance the model flexibility. Comprehensive experiments are conducted to test the performance of the proposed method on three datasets, including a new one created specifically in this work for personalized music emotion recognition. Our results demonstrate the effectiveness of the proposed method.
Transform-domain intra prediction for H.264

H.264/AVC is the newest video coding standard jointly developed by the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. In contrast to some previous coding standards such as H.263+ and MPEG-4 Part-2, where intra prediction is performed in the transform domain, the intra prediction of H.264 is completely defined in the pixel domain. This presents a challenge to multimedia systems in which transcoding is conducted in the transform domain for the purpose of computational efficiency. In this paper, we show how to obtain the transform domain predictions for various intra modes of H264. We begin by converting the intra prediction from the pixel domain to the transform domain through matrix manipulation. Then we show how the operations involved in the matrix manipulation can be simplified. A computational complexity analysis of each intra prediction mode of H.264 is provided.
Single Image Realism Assessment and Recoloring by Color Compatibility

In this paper, we investigate the assessment of image realism by focusing our attention on the color compatibility between an inserted object and the background in an image composite. We propose a technique based on two color compatibility properties to achieve realistic image composition. The first property is related to the color similarity, and the second one to the consistence of color tendency between image regions. We further propose algorithms based on these two properties for image realism assessment and recoloring. These algorithms only require information from the image to be tested, making them suitable for practical applications where real images are unavailable. Effectiveness of the algorithms is demonstrated through various images and verified by ground truth.
A novel automatic white balance method for digital still cameras

Automatic white balance is an important function of digital still cameras. The goal of white balance is to adjust the image such that it looks as if it is taken under canonical light. We proposed a novel technique to detect reference white points in an image. Our algorithm uses dynamic threshold for white point detection and is more flexible than other existing ad hoc algorithms. We have tested the algorithm on 50 images taken under various light sources. The results show that the algorithm is superior or comparable to other methods in both objective and subjective evaluations. The complexity of the algorithm is quite low, which makes it attractive for real-world applications.
Minimum-drift digital video downconversion

This paper presents a new technique for decoding a full-resolution video bitstream at low memory cost and displaying the signal at a lower resolution. Existing techniques solve the problem by storing the downconverted blocks into memory instead of the full-resolution blocks. While the memory is reduced, these techniques introduce drift errors because the decoder does not have the same pixels as the encoder in performing motion-compensated prediction. The approach proposed here alleviates the problem by tracking the drift at the decoder. It improves the video quality without any increase in decoder complexity. The effectiveness of the approach is evaluated using both objective and subjective tests. This minimum-drift approach is very simple to implement and can also be applied for memory reduction of a full-resolution HDTV decoder.
Pose Determination from Line-to-Plane Correspondences: Existence Condition and Closed-Form Solutions

A class of pose determination problems in which the sensory data are lines and the corresponding reference data are planes is discussed. The lines considered are different from edge lines in that they are not the intersection of boundary faces of the object. The author describes a polynomial approach that does not require a priori knowledge about the object location. Closed-form solutions for orthogonal, parallel, and coplanar feature configurations of critical importance in real applications are derived. Findings concerning the necessary and sufficient conditions under which the line-to-plane pose determination problem can be solved are described.
Motion And Depth From Binocular Orthographic Views

The paper describes a method using orthographic views that are taken from two cameras to determine the motion and depth of a rigid object. The cameras are widely separated in space and hence they may observe different object points. By tracking the object points for each camera, we show how the motion and depth of the object can be determined in two views when the relative position and orientation between the cameras are known. Both exact and noisy cases are discussed.
The AMG1608 dataset for music emotion recognition

Automated recognition of musical emotion from audio signals has received considerable attention recently. To construct an accurate model for music emotion prediction, the emotion-annotated music corpus has to be of high quality. It is desirable to have a large number of songs annotated by numerous subjects to characterize the general emotional response to a song. Due to the need for personalization of the music emotion prediction model to address the subjective nature of emotion perception, it is also important to have a large number of annotations per subject for training and evaluating a personalization method. In this paper, we discuss the deficiency of existing datasets and present a new one. The new dataset, which is publically available to the research community, is composed of 1608 30-second music clips annotated by 665 subjects. Furthermore, 46 subjects annotated more than 150 songs, making this dataset the largest of its kind to date.
Available bandwidth characteristics and estimation of in-home power line networks

Bandwidth information is critical to robust real-time multimedia transmission. However, existing bandwidth estimation (BWE) methods developed for the Internet or wireless networks are inadequate for power line networks due to the difference in media access control scheduling policies and channel noise characteristics. In this paper, we investigate the characteristics of in-home power line networks and present a real-time BWE scheme. By connecting different number of power line communication (PLC) modems, we find that the bandwidth is approximately inversely proportional to the number of PLC modems connected. Moreover, PLC modems have different characteristics of crossing different circuits under the same power meter and crossing different power meter under the same transformer. Based on the evaluation of in-home power line networks with multiple streaming, we propose an available bandwidth estimation scheme that estimates the cutoff rate from physical-layer line rate, data transmission rate, packet size, and packet loss rate. This scheme applies an iterative packet probing algorithm to estimate the available bandwidth. Experimental results show that the proposed scheme achieves good estimation accuracy.
A Regression Approach to Music Emotion Recognition

Content-based retrieval has emerged in the face of content explosion as a promising approach to information access. In this paper, we focus on the challenging issue of recognizing the emotion content of music signals, or music emotion recognition (MER). Specifically, we formulate MER as a regression problem to predict the arousal and valence values (AV values) of each music sample directly. Associated with the AV values, each music sample becomes a point in the arousal-valence plane, so the users can efficiently retrieve the music sample by specifying a desired point in the emotion plane. Because no categorical taxonomy is used, the regression approach is free of the ambiguity inherent to conventional categorical approaches. To improve the performance, we apply principal component analysis to reduce the correlation between arousal and valence, and RReliefF to select important features. An extensive performance study is conducted to evaluate the accuracy of the regression approach for predicting AV values. The best performance evaluated in terms of the R 2 statistics reaches 58.3% for arousal and 28.1% for valence by employing support vector machine as the regressor. We also apply the regression approach to detect the emotion variation within a music selection and find the prediction accuracy superior to existing works. A group-wise MER scheme is also developed to address the subjectivity issue of emotion perception.
Enhancing the perception of a hazy visual world using a see-through head-mounted device

Human perception of the visual world in the presence of fog or haze usually suffers from loss of color vibrance and details. In this paper we propose a novel method to provide a see-through head-mounted display with image dehazing capability. The proposed method projects an auxiliary image from the head-mounted device to the user's retina. The introduction of the auxiliary image enhances the contrast and brightness of the perceived visual world. This method can be applied to tourism, military, and navigation to overcome poor visual condition.
Adopting Perceptual Quality Metrics in Video Encoders: Progress and Critiques

There is a need for video encoders to generate bitstreams of quality that matches human evaluation. This is often achieved by adopting perceptual quality metrics in video encoders. A huge body of work has been devoted to the development of perceptual video quality metrics that take the properties of human visual system into account. It is our interest in this paper to study how the existing video coding systems can benefit from such perceptual quality metrics. Specifically, we examine the existing perceptual video coding systems and introduce the concept of “codec-friendliness,” meaning how the perceptual quality metrics can be nicely incorporated into the coding process of standard video coding algorithms. We conclude the study by suggesting guidelines for the development of future quality metrics from the video coding perspective.
Depth Detection of Light Field

We propose an algorithm to detect depths in a light field. Specifically, given a 4D light field, we find all planes at which objects are located. Although the exact depth of each pixel in the space is left unknown, the partial information obtained is very useful for many applications, such as synthetic aperture photography and all-focused rendering. Our algorithm measures the degree of focus of different planes by calculating the ratio of high frequencies to the low frequencies. To handle different depth distributions, we reformulate the maximum detection problem to a maximum-cover problem that can be solved efficiently by dynamic programming. Compared with auto-focusing and per-pixel depth estimation, our algorithm is much faster yet sufficiently accurate.
Tree based motion compensated video coding

We present a novel technique to encode video sequences, that performs a region-based decomposition of each frame on the basis of motion information. Using the segmentation map, any region in a frame to be encoded will be predicted from a single reference frame, using motion compensated prediction. The use of a single reference frame avoids feedback of the prediction error information in the prediction of successive frames. Coding is simply obtained by describing the segmentation map and the associated motion information. Error information will not be provided for low bit-rate applications. The segmentation map is described using a quadtree structure. Within such a tree structure, we show how motion information can be predicted either spatially or temporally, so as to minimize redundancy of information. The motion and segmentation information are estimated on the basis of a two stage process using the frame to be encoded and the reference frame: (1) a hierarchical top-down decomposition; (2) a bottom-up merging strategy. The proposed posed method is used to encode to encode QCIF video sequences with a reasonable duality at a 10 frame/s rate using roughly 20 kbit/s.<<ETX>>
Pose determination from line-to-plane correspondences: existence condition and closed-form solutions

Consideration is given to a specific pose determination problem in which the sensory features are lines and the matched reference features are planes. The lines discussed are different from edge lines of an object in that they are not the intersection of boundary faces of the object. The author describes a polynomial method that, unlike previous methods, does not require prior knowledge about the location of the object. Closed-form solutions for orthogonal, coplanar, and parallel feature configurations of critical importance in real applications are derived. Basic findings concerning the necessary and sufficient conditions under which the pose determination problem can be solved are presented.<<ETX>>
Perceptual-based coding mode decision

The framework of rate-distortion optimization (RDO) has been widely adopted for video coding to achieve a good trade-off between bit-rate and distortion. However, objective distortion metrics such as mean square error traditionally used in this framework are poorly correlated with perceptual video quality. To address this issue, we incorporate the structural similarity index as a quality metric into the framework and develop a predictive Lagrange multiplier selection technique to resolve the chicken-and-egg dilemma of perceptual-based RDO. The resulting perceptual-based RDO is then applied to H.264 intra mode decision as an illustration of the application of the proposed technique. Given a perceptual quality level, 5%–10% bit rate reduction over the JM reference software of H.264 is achieved. Subjective evaluation further confirms that, at the same bit-rate, the proposed perceptual RDO preserves image details and prevents block artifact better than the traditional RDO.
Video Streaming Over In-Home Power Line Networks

The deployment of power line communication technology for broadband video streaming remains a challenge because power lines are not originally designed for signal transmission. Scalable video is a viable approach that can cope with the bandwidth fluctuation of power line communication networks provided that the bandwidth information is available. In this paper we first investigate how the interference caused by electrical appliances or power supplies affects the power line channel bandwidth and packet transmission. Then we take the obtained characteristics of in-home power line network into account in the design of a simple but effective heuristic-based application-layer bandwidth estimation scheme, for which the cutoff rate is estimated from the packet size and the physical-layer data rates. Experimental results show that the proposed approach can effectively combat the noise interference and deliver robust video streaming over power line.
Quality enhancement of procam system by radiometric compensation

A procam system consists of a projector and a camera. In this paper, a radiometric compensation scheme is proposed for improving the projection quality of a procam system that uses a nearby non-white wall as the projection screen. The compensation scheme is capable of correcting the radiometric errors such as chroma and brightness distortions caused by the projection surface and the ambient light. It is also capable of compensating the effects of nonlinear spectral response (including vignetting) of the procam system. These important functions are achieved through a computational framework that optimizes the tradeoff between chroma and brightness distortions of the projected image and accelerates the computation of the penalty in the optimization process. Experimental results are shown to demonstrate the performance of the proposed radiometric compensation scheme.
Smooth Control of Adaptive Media Playout for Video Streaming

Client-side data buffering is a common technique to deal with media playout interruptions of streaming video caused by network jitters and packet losses of best-effort networks. However, stronger playout interruption protection inevitably amounts to larger data buffering and results in more memory requirements and longer playout delay. Adaptive media playout (AMP), also a client-side technique, can reduce the buffer requirement and avoid buffer outage but at the expense of visual quality degradation because of the fluctuation of playout speed. In this paper, we propose a novel AMP scheme to keep the video playout as smooth as possible while adapting to the channel condition. The triggering of the playout control is based on buffer variation rather than buffer fullness. Experimental results show that our AMP scheme surpasses conventional schemes in unfriendly network conditions. Unlike previous schemes that are tuned for a specific range of packet loss and network instability, the proposed AMP scheme maintains consistent performance across a wide range of network conditions.
Using motion from orthographic projections to prune 3-D point matches

The authors presented a new method using motion constraint to prune 3-D point matches. The method discards noisy z coordinates of points and uses only x and y coordinates to compute motion. This method is specifically designed to resolve the matching ambiguity due to the use of large error tolerances in rigidity tests to account for depth errors. A least-squares solution of the motion problem is derived. Results show that this method is able to detect false matches more effectively than the traditional method which uses full 3-D coordinates.<<ETX>>
Local Dimming of Liquid Crystal Display Using Visual Attention Prediction Model

Local dimming of the LED backlight is a popular technique for saving the power of a liquid crystal display. This paper presents a novel approach that improves the performance of conventional local dimming algorithms by incorporating a visual attention prediction model in the backlight dimming process. The approach saves the energy of the liquid crystal display and, in the mean time, maintains the perceptual image quality. This is achieved by preserving the backlight luminance of the image areas that attract human attention while reducing that of the other image areas. Experimental results are provided to show the effectiveness of the proposed approach.
Music recommendation based on artist novelty and similarity

Most existing systems recommend songs to the user based on the popularity of songs and singers. However, the system proposed in this paper is driven by an emerging and somewhat different need in the music industry-promoting new talents. The system recommends songs based on the novelty of singers (or artists) and their similarity to the user's favorite artists. Novel artists whose popularity is on the rise have a higher priority to be recommended. Specifically, given a user's favorite artists, the system first determines the candidate artists based on their similarity with the favorite artists and then selects those who have a higher novelty score than the favorite artists. Then, the system outputs a playlist composed of the most popular songs of the selected artists. The proposed system can be integrated into most existing systems. Its performance is evaluated using the Spotify Radio Recommender as a reference and a pool of 100 subjects recruited on campus. Experimental results show that our system achieves a high novelty score and a competitive user-preference score.