nils-herrmann/clean_scirepeval huggingface dataset:
Spherical domain rate-distortion optimization for 360-degree video coding

Emerging virtual reality (VR) applications bring much challenge to video coding for 360-degree videos. To compress this kind of video, each picture should be projected to a 2D plane (e.g. equirctangular projection map) first, adapting to the input of existing video coding systems. At the display side, an inverse projection is performed before viewport rendering. However, such a project introduces much different levels of distortions depending on the location, which makes the rate-distortion optimization process in video coding much inefficient. In this paper, we consider the distortion in spherical domain and analyse its influence to the rate-distortion optimization process. Then we derive the optimal rate-distortion relationship in spherical domain and present its optimal solution based on HEVC/H.265. Experimental results show that the proposed method can bring up to 11.5% bit-saving compared with the current HEVC/H.265 anchor for 360-degree video coding.
A fast intra prediction algorithm for 360-degree equirectangular panoramic video

360-degree video has been gaining its popularity as virtual reality (VR) technology develops in recent years. To compress these videos using standard video encoders, most of them are converted to a 2D image planar format with equirectangular projection (ERP). Compared with other encoders, High Efficiency Video Coding (HEVC) achieves significant improvements in coding efficiency. However, exhaustive computational complexity of HEVC makes it too time-consuming to compress 360-degree videos of high resolution and high frame rate for VR application. In this paper, a fast intra prediction algorithm is proposed according to the characteristics of ERP format video. Compared with original reference software HM 16.15, experimental results show that the proposed algorithm can achieve about 24.5% encoder time saving on average in All-Intra configuration with negligible quality loss.
Saliency assistant driver alerting

This paper presents a driver alerting approach based on saliency detection. Due to the unequal distribution of photoreceptors in human visual system, visual acuity decreases according to the distance to the fovea area on the retina. A driver alerting approach is proposed based on visual sensor to detect the saliency information which may not be well perceived by the driver. The image acquired by the visual sensor is analyzed by the saliency detection algorithm. The salient object Which is not perceived by the driver, i.e. is projected on the retina but is far away from the fovea area, is detected. The experiments show that the proposed approach can effectively detect salient object which is not well perceived by the driver therefore ensure better driving safety.
Unsupervised malaria parasite detection based on phase spectrum

In this paper, we propose a novel method for malaria parasite detection based on phase spectrum. The method first obtains the amplitude spectrum and phase spectrum for blood smear images through Quaternion Fourier Transform (QFT). Then it gets the reconstructed image based on Inverse Quaternion Fourier transform (IQFT) on a constant amplitude spectrum and the original phase spectrum. The malaria parasite areas can be detected easily from the reconstructed blood smear images. Extensive experiments have demonstrated the effectiveness of this novel method.
Saliency map generation based on saccade target theory

In this paper, a novel saliency map generation approach based on the saccade target theory is proposed. A probabilistic model of transsaccadic integration is built based on four cues that influence human visual attention: foveaperiphery resolution discrepancy, visual memory, oculomotor bias and inhibition of return (IOR), where visual memory is formulated as combination of the visual short-term memory (VSTM) and the visual long-term memory (VLTM). A sequence of fixations is generated based on the model to simulate the shift of attention. The proposed approach is evaluated on both perspectives of the accuracy of generated saliency maps and scanpaths. As demonstrated in the experiments, our method provides better estimated saliency maps and scanpaths on eye tracking datasets when compared to several state-of-the-art models.
Adaptive background for real-time visual tracking

Visual tracking plays a fundamental role in many applications, such as video surveillance, image compression and three-dimensional reconstruction. From the perspective of accuracy and complexity, correlation filter for target tracking has been proved to be one of the most efficient algorithms. However, it suffers from some difficulties when tracking complex objects with rotations, occlusions and other distractions. To improve its robustness, in this paper, we propose an adaptive background model (ABM) to realize real-time visual tracking with a high adaptivity and accuracy. We use the static background information to help tracking instead of only focusing on the target itself, especially when there are great appearance changes. Moreover, we use peak to side-lobe ratio to update the ABM. As shown in the experiments, our proposed method achieves effective performance in visual tracking with good tradeoff between computational complexity and accuracy.
Boundary Artifact Reduction in View Synthesis of 3D Video: From Perspective of Texture-Depth Alignment

3D Video (3DV) with depth-image-based view synthesis is a promising candidate of next generation broadcasting applications. However, the synthesized views in 3DV are often contaminated by annoying artifacts, particularly notably around object boundaries, due to imperfect depth maps (e.g., produced by state-of-the-art stereo matching algorithms or compressed lossily). In this paper, we first review some representative methods for boundary artifact reduction in view synthesis, and make an in-depth investigation into the underlying mechanisms of boundary artifact generation from a new perspective of texture-depth alignment in boundary regions. Three forms of texture-depth misalignment are identified as the causes for different boundary artifacts, which mainly present themselves as scattered noises on the background and object erosion on the foreground. Based on the insights gained from the analysis, we propose a novel solution of suppression of misalignment and alignment enforcement (denoted as SMART) between texture and depth to reduce background noises and foreground erosion, respectively, among different types of boundary artifacts. The SMART is developed as a three-step pre-processing in view synthesis. Experiments on view synthesis with original and compressed texture/depth data consistently demonstrate the superior performance of the proposed method as compared with other relevant boundary artifact reduction schemes.
Saliency based perceptual HEVC

Saliency represents the probability of human attention over the image therefore is important in understanding the importance of different areas in the image. In this paper, we consider how to utilize the saliency information in HEVC video coding. The graph-based visual saliency is incorporated with the quantization control in HEVC to reduce the bit rate of compressed video by using larger quantization parameter for the coding unit which has lower probability of attention. As shown in our experiments, the proposed method achieves up to 12% bit rate reduction without perceptual quality loss.
Performance analysis of AVS2 for remote sensing image compression

In this paper, the performance of AVS2 based Remote Sensing (RS) image compression is provided. Remote Sensing images are with distinguished features such as high spatial resolution, and high spectral resolution. Conventional image compression algorithms such as JPEG and JPEG 2000 are mostly used in current satellites. With the comparisons of different image compression techniques i.e., JPEG , JPEG 2000, and AVS2, we validate that the AVS2 outperforms the other two techniques based on our test RS images acquired from Landsat8, Terra/AQua and Ziyuan-3.
Maximizing Lifetime of Data-Gathering Trees with Different Aggregation Modes in WSNs

We study the problem of maximizing the lifetime of data-gathering tree for wireless sensor networks (WSNs). Both data routing and aggregation are considered at the same time to improve the energy efficiency for data collection in WSNs. With different data-aggregation methods, three aggregation modes are studied: full aggregation, non-aggregation, and a hybrid partialaggregation using Compressive Sensing. For each mode, an exact solution based on Mixed-integer linear programming (MIP) is proposed to find the optimal data-gathering tree. Although nonlinear relation exists between the sensor node lifetime and the number of data units that receives or transmits in each time slot, we succeed to express it by a set of linear equations. Performance results demonstrate that the lifetime of data-gathering tree can be increased tenfold with efficient data aggregation methods.
Revisiting visual attention identification based on eye tracking data analytics

Visual attention identification is crucial to human visual perception analysis and relevant applications. In this paper, we propose a comprehensive visual attention identification algorithm consisting of clustering and center identification. In the clustering process, a spatial-temporal affinity propagation method for accurate fixation clustering is proposed. For the identified clusters, the random walk based method is utilized to extract the center of each cluster, which presents the essential part of an area of interest (AOI). The proposed approach addresses the problem of fixation overlapping in eye movement analytics. Compared with the state-of-the-arts methods, the proposed method shows superior performance for different eye tracking experiments.
Thermal to Visible Facial Image Translation Using Generative Adversarial Networks

Thermal cameras can capture images invariant to illumination conditions. However, thermal facial images are difficult to be recognized by human examiners. In this letter, an end-to-end framework, which consists of a generative network and a detector network, is proposed to translate thermal facial images into visible ones. The generative network aims at generating visible images given the thermal ones. The detector can locate important facial landmarks on visible faces and help the generative network to generate more realistic images that are easier to be recognized. As demonstrated in the experiments, the faces generated by our method have good visual quality and maintain identity preserving features.
JND modeling: Approaches and applications

Just-Noticeable-Difference (JND) measures the smallest detectable difference between two signals therefore can be utilized to quantify the perceivable distortion in the noise contaminated image. In this paper, a comprehensive review of JND models is provided. Specifically, spatial domain and frequency domain JND models are introduced, together with the relevant information about the human visual system. Moreover, the progress from the local JND to global JND is also discussed which aims to exploit the perceptual redundancy in the image. The applications are also discussed in the paper.
Scanpath mining of eye movement trajectories for visual attention analysis

Eye movement reflects the shift of overt visual attention. Eye movement trajectories from a group of observers can be expressed by a representative scanpath. The representative scan-path can work as a baseline for studies on scanpath prediction as well as provide useful knowledge about group behavior in psychological studies. In this paper, we propose a new framework to summarize a representative scanpath from individual scanpaths, taking into account the spatial distribution of scan-paths rather than simply treating them as strings of characters. It consists of three steps: extract areas of interest (AOI), remove outliers and summarize scanpaths. In the last step, we develop an algorithm termed Candidate-constrained DTW Barycenter Algorithm (CDBA) by imposing 3 constraints: (1) the components of the representative scanpath must be chosen from candidates (extracted AOIs); (2) the occurrence count of each AOI in the representative scanpath cannot exceed its maximum occurrence count in individual scanpaths; (3) any two contiguous AOIs in the representative scanpath must be contiguous in at least one individual scanpath. The experiments demonstrate that the proposed method outperforms other state-of-the-art scanpath mining methods.
Object Tracking over a Multiple-Camera Network

In this paper, we describe our system for object tracking over a multiple-camera network task in BigMM Challenge in conjunction with the first IEEE International Conference on Multimedia Big Data (BigMM 2015). We focus on the detection and tracking of pedestrians and vehicles. Based on background modeling, we use HOG and SVM to detect pedestrian and morphological processing to detect vehicle in single camera then use spatio-temporal local context for robust object tracking. The features and trajectory of each object in the multiple-camera network are analyzed for matching and camera geometric projection is also employed to optimize the trajectory. We also include the trajectory visualization in our GIS based experiments.
Multi-layer Sparse Coding Based Ship Detection for Remote Sensing Images

With the development of remote sensing technology, it becomes possible for the detection and identification of targets from remote sensing images. In this paper, we propose a new method integrating the bottom-up and the top-down mechanisms for the ship detection in high resolution satellite images. We use the multi-layer sparse coding to extract the features of the RS images. Then, we get the ship candidate regions by calculating the global saliency map which may have ships in it. Deformable part model is used to extract the ship features and latent support vector machine is used for the ship identification. As demonstrated in our experiments, the proposed approach can effectively detect ship in remote sensing images.
Video Quality Assessment Based on Measuring Perceptual Noise From Spatial and Temporal Perspectives

Video quality assessment (VQA) exploits important properties of the sophisticated human visual system (HVS). In this paper, we study a series of fundamental HVS characteristics for subjective video quality assessment, and incorporate them into a systematic framework to simulate subjective evaluation on impaired videos. Based on this framework, we develop a novel full-reference metric, namely, perceptual quality index (PQI). Specifically, the proposed PQI metric comprises four major modules: 1) visual performance equation for the foveal and extra-foveal vision based on the cortical magnification theory; 2) perceptible noise detection using a spatial-temporal just noticeable difference model, and its quantification in both spatial and temporal channels, considering the varying error sensitivity due to the contrast and motion masking effects; 3) instantaneous error summation with inhibition of weak local distortions, and quality degradation accumulation over time that models the visual persistence and recency effect; and 4) fusion of the spatial and temporal noise intensities into a perceptual quality index. Compared with some state-of-the-art VQA models, the PQI metric, which exploits multiple visual properties, measures video quality more accurately and reliably on two VQA databases.
Multi-Layer Quantization Control for Quality-Constrained H.265/HEVC.

Achieving a stable video quality is an important task in video compression. In this paper, we present a multi-layer quantization control method to compress video to a certain target video quality based on the new hierarchical partition structure of H.265/HEVC. We first model the rate-quality characteristics of different video sequences using ν-support vector regression with a Gaussian radial basis function as the kernel function. Then, we propose a Kalman filter-based multi-layer quantization control method for the quality-constrained video coding. The advantage of our proposed approach in H.265/HEVC is demonstrated through experimental results. Compared with the other H.265/HEVC rate control algorithms and the fixed-QP scheme, the proposed method achieves a more stable video quality.
Maximizing Lifetime of Data-Gathering Trees With Different Aggregation Modes in WSNs

We study the problem of maximizing the network lifetime of a data-gathering tree in a wireless sensor network (WSN). Both data routing and data aggregation are considered at the same time to improve the energy efficiency for data collection in WSNs. With different data-aggregation methods, three aggregation modes are studied: full aggregation, non-aggregation, and a hybrid partial-aggregation using compressive sensing. The full aggregation means that an intermediate sensor node will aggregate all the received data from its children and its own generated data into one data unit and then send it to its parent. If no aggregation is used, an intermediate sensor node will forward all the received data units plus its own data separately. For the hybrid partial-aggregation, once the number of incoming data units is bigger than a compression threshold, they will be compressed into a constant size for transmission. For each mode, an exact solution based on mixed-integer linear programming (MIP) is proposed to find the optimal data-gathering tree with the maximum network lifetime. Although non-linear relations exist between the lifetime of a sensor node and the number of data units received or transmitted, we succeed to express it by a set of linear equations. The correctness of the MIP models is proven formally. Simulation results demonstrate that the network lifetime can be increased tenfold by changing the compression threshold from five to one, which is, however, at the expense of bigger end-to-end delay and lower data accuracy. Thus, this compression threshold should be properly chosen for real applications.
A fast mode decision algorithm for HEVC intra prediction

The latest video coding standard, High Efficiency Video Coding (HEVC) can achieve up to 50% bit rate saving while maintain the same subjective quality compared to H.264/AVC. However, this great advance is obtained at the expense of significantly increased encoder complexity. In this paper, a two-step algorithm focusing on fast mode decision is proposed for HEVC intra prediction. Firstly, depth information of CU block is utilized to skip some unlikely selected modes, with the assumption that a brute-force search for a large CU is unnecessary. Secondly, the order of Most Probable Modes (MPM) and Rough Mode Decision (RMD) is adjusted, thus the amount of mode need be evaluated is further reduced. Experimental results show that the proposed algorithm can achieve about 31% encoder time saving on average while result in negligible BD-rate loss under the All Intra configuration compared with HM 16.0.
A rate and distortion analysis for H.264/AVC video coding

This paper addresses the rate and distortion analysis problem in H.264/AVC video coding. It is well known that such an analysis is important in video coding. Exploring the relationship between the quantization and rate-distortion information of video signal can assist with the control strategies in the coding process. A novel algorithm of the rate and distortion analysis for H.264/AVC video coding is presented. The rate and distortion characteristics of video signal are approximated and estimated using statistical learning. When compared to traditional methods, the proposed method provides better approximation and estimation accuracy for the rate and distortion analysis in H.264/AVC video coding.
Remote sensing image super-resolution: Challenges and approaches

Remote sensing has a growing relevance in the modern society with the development of image processing of satellite imagery. However, due to the limitations of the current imaging sensors and the complex atmospheric conditions, we are facing great challenges in the remote sensing applications due to the limited spatial, spectral, radiometric and temporal resolutions. Therefore, super-resolution techniques have attracted much attention by which the low quality low resolution remote sensing images are enhanced. In this paper, we discuss the challenges in remote sensing image super-resolution and thereafter review the relevant approaches. More specifically, the different categories of remote sensing techniques, i.e., the learning-based, interpolation based, frequency domain based, and probability based methods, are reviewed and discussed. Furthermore, the super-resolution applications are discussed and insightful comments on future research directions are provided.
QoE-driven cache management for HTTP adaptive bit rate (ABR) streaming over wireless networks

In this paper, we investigate the problem of how to cache a set of media files with optimal streaming rates, under HTTP adaptive bit rate streaming over wireless networks. The design objective is to achieve the optimal expected QoE under a limited storage budget, which is measured by the logarithmic relation between the required bit rate and the actual streaming bit rate. We formulate the content cache management of streaming files as a constrained optimization problem. Lagrange multiplier method is employed, and we obtain the numerical solution of the optimal streaming files. Particularly, we characterize the properties of the solution, and find there is a fundamental phase change in the optimal solution as the number of cached files grows. Moreover, the simulation results indicate that with the increase of cache size, more copies of different bit rate should be cached for a better QoE. Our comprehensive investigation reveals insightful guidelines to provide HTTP ABR streaming services over wireless networks.
Subjective Panoramic Video Quality Assessment Database for Coding Applications

With the development of virtual reality, higher quality panoramic videos are in great demand to guarantee the immersive viewing experience. Therefore, quality assessment attaches much importance to correlated technologies. Considering the geometric transformation in projection and the limited resolution of head-mounted device (HMD), a modified display protocol of the high resolution sequences for the subjective rating test is proposed, in which an optimal display resolution is determined based on the geometry constraints between screen and human eyes. By sampling the videos to the optimal resolution before coding, the proposed method significantly alleviates the interference of HMD sampling while displaying, thus ensuring the reliability of subjective quality opinion in terms of video coding. Using the proposed display protocol, a subjective quality database for panoramic videos is established for video coding applications. The proposed database contains 50 distorted sequences obtained from ten raw panoramic video sequences. Distortions are introduced with the High Efficiency Video Coding compression. Each sequence is evaluated by 30 subjects on video quality, following the absolute category rating with hidden reference method. The rating scores and differential mean opinion scores (DMOSs) are recorded and included in the database. With the proposed database, several state-of-the-art objective quality assessment methods are further evaluated with correlation analysis. The database, including the video sequences, subjective rating scores and DMOS, can be used to facilitate future researches on coding applications.