nils-herrmann/clean_scirepeval huggingface dataset:
An adaptive perceptual quantization method for HDR video coding

This paper presents a new adaptive perceptual quantization method for the High Dynamic Range (HDR) content. This method considers the luminance distribution of the HDR image as well as the Minimum Detectable Contrast (MDC) thresholds to preserve the contrast information during quantization. Base on this method, we develop a mapping function for HDR video compression and apply it to a HEVC Main 10 Profile-based video coding chain. Our experiments show that the proposed mapping function can efficiently improve the quality of the reconstructed HDR video in both objective and subjective assessments.
A scale space for texture+depth images based on a discrete laplacian operator

In this paper we design a smoothing filter for texture+depth images based on anisotropic diffusion. Our proposed filter enables to generate a scale space on the texture image guided by depth information, and is linear and numerically stable. We show experimentally that using scene geometry preserves the internal structure of 3D surfaces (e.g., it avoids smoothing across object boundaries). As a consequence, the result of smoothing is more independent to changes in the camera position. To illustrate the practical utility of a scale space with such properties, we integrate our filter into the SIFT keypoint detector, getting a substantial improvement of the repeatability of detected keypoints under significant viewpoint position changes.
Forensic detection of inverse tone mapping in HDR images

High dynamic range (HDR) imaging is attracting an increasing deal of attention in the multimedia community, yet its forensic problems have been little studied so far. This paper proposes an HDR image forensic method, which aims at differentiating HDR images created from multiple low dynamic range (LDR) images from those created from a single LDR image by inverse tone mapping. For each kind of HDR image, a Gaussian mixture model is learned. Thereafter, an HDR image forensic feature is constructed based on calculating the Fisher scores. With comparison to a steganalytic feature and a texture/facial analysis feature, experimental results demonstrate the efficiency of the proposed method in HDR image forensic classification on whole images as well as small blocks, for three inverse tone mapping methods.
An evaluation of HDR image matching under extreme illumination changes

High dynamic range (HDR) imaging has potential to facilitate computer vision tasks such as image matching where lighting transformations hinder the matching performance. However, little has been done to quantify the gains with different possible HDR representations for vision algorithms like feature extraction. In this paper, we evaluate the performance of the full feature extraction pipeline, including detection and description, on ten different image representations: low dynamic range (LDR), seven different tone mapped (TM) HDR and two HDR imaging (linear and log encoded) representations. We measure the impact of using these different representations for feature matching using mean average precision (mAP) scores on four illumination change datasets. We perform feature extraction using four popular schemes in the literature: SIFT, SURF, BRISK, FREAK. With respect to previous studies, our observations confirm the advantages of HDR over conventional LDR imagery, and the fact that HDR linear values are not appropriate for vision tasks. However, HDR representations that work best for keypoint detection are not necessarily optimal when the full feature extraction is taken into account.
Local visual features extraction from texture+depth content based on depth image analysis

With the increasing availability of low-cost - yet precise - depth cameras, “texture+depth” content has become more and more popular in several computer vision and 3D rendering tasks. Indeed, depth images bring enriched geometrical information about the scene which would be hard and often impossible to estimate from conventional texture pictures. In this paper, we investigate how the geometric information provided by depth data can be employed to improve the stability of local visual features under a large spectrum of viewpoint changes. Specifically, we leverage depth information to derive local projective transformations and compute descriptor patches from the texture image. Since the proposed approach may be used with any blob detector, it can be seamlessly integrated into the processing chain of state-of-the-art visual features such as SIFT. Our experiments show that a geometry-aware feature extraction can bring advantages in terms of descriptor distinctiveness with respect to state-of-the-art scale and affine-invariant approaches.
Segmentation-based optimized tone mapping for high dynamic range image and video coding

A core part of the state-of-the art high dynamic range (HDR) image and video compression methods is the tone mapping operation to convert the visible luminance range into the finite bit depths that can be supported by the current video codecs. These conversions are until now optimized to provide backward compatibility to the existing low dynamic range (LDR) displays. However, a direct application of these methods for the emerging HDR displays can result in a loss of details in the bright and dark regions of the HDR content. In this paper, we overcome this limitation by designing a tone mapping operation which handles the bright and dark regions separately. The proposed method first finds the optimal segmentation of the HDR image into two parts, namely dark and bright regions, and then designs the optimal tone mapping for each region in terms of the mean square error between the logarithm of the luminance values of the original and reconstructed HDR content (HDR-MSE). The results indicate the superiority of the proposed method over the state-of-the art HDR coding methods.
Scream and gunshot detection and localization for audio-surveillance systems

This paper describes an audio-based video surveillance system which automatically detects anomalous audio events in a public square, such as screams or gunshots, and localizes the position of the acoustic source, in such a way that a video-camera is steered consequently. The system employs two parallel GMM classifiers for discriminating screams from noise and gunshots from noise, respectively. Each classifier is trained using different features, chosen from a set of both conventional and innovative audio features. The location of the acoustic source which has produced the sound event is estimated by computing the time difference of arrivals of the signal at a microphone array and using linear-correction least square localization algorithm. Experimental results show that our system can detect events with a precision of 93% at a false rejection rate of 5% when the SNR is 10dB, while the source direction can be estimated with a precision of one degree. A real-time implementation of the system is going to be installed in a public square of Milan.
Statistical analysis and directional coding of layer-based HDR image coding residue

Existing methods for layer-based backward compatible high dynamic range (HDR) image and video coding mostly focus on the rate-distortion optimization of base layer while neglecting the encoding of the residue signal in the enhancement layer. Although some recent studies handle residue coding by designing function based fixed global mapping curves for 8-bit conversion and exploiting standard codecs on the resulting 8-bit images, they do not take the local characteristics of residue blocks into account. Inspired by the local anisotropic characteristics of the residue signal and directional methods for motion compensated low dynamic range (LDR) video coding, in this paper we first investigate whether HDR image coding residue exhibits also local anisotropic characteristics. Specifically, we verify directional structures in residue blocks by means of auto-covariance analysis for different bitrates, spatial activities and dynamic ranges as the main variables in HDR image coding. Then, we compare the rate distortion performances of directional coding methods with the baseline residue coding methods in the literature along with different combinations of 8-bit conversion methods. The experiments indicate that content dependent 8-bit conversions and directional coding significantly outperforms the existing function based 8-bit conversions and typical coding for residue coding.
HDR Video Coding based on a temporally constrained Tone Mapping Operator

Given its potential for more realistic rendering and enhanced user experience, High Dynamic Range (HDR) imaging is raising a lot of interest both in industry and academia. In this context, efficient representation and coding techniques are needed, as HDR video entails significantly higher raw data rate. In this paper, we present a temporally constrained content-adaptive Tone Mapping Operator (TMO) in order to convert the input HDR video into a reduced bit depth video sequence which is then encoded using High Efficiency Video Coding (HEVC). As the proposed TMO simultaneously takes into account the statistical characteristics of the input frame while better preserving temporal coherence of the tone mapped video sequence, it leads to improved coding efficiency. Experimental results show that the proposed technique compares favorably with existing methods in terms of rate-distortion when using the HDR-VDP-2.2.1 quality metric.
Motion prediction of depth video for depth-image-based rendering using don't care regions

To enable synthesis of any desired intermediate view between two captured views at decoder via depth-image-based rendering (DIBR), both texture and depth maps from the captured viewpoints must be encoded and transmitted in a format known as texture-plus-depth. In this paper, we focus on the compression of depth maps across time to lower the overall bitrate in texture-plus-depth format. We observe that depth maps are not directly viewed, but are only used to provide geometric information of the captured scene for view synthesis at decoder. Thus, as long as the resulting geometric error does not lead to unacceptable synthesized view quality, each depth pixel only needs to be reconstructed at the decoder coarsely within a tolerable range. We first formalize the notion of tolerable range per depth pixel as don't care region (DCR), by studying the synthesized view distortion sensitivity to the pixel value - a sensitive depth pixel will have a narrow DCR, and vice versa. Given per-pixel DCRs, we then modify inter-prediction modes during motion prediction to search for a predictor block matching per-pixel DCRs in a target block (rather than the fixed ground truth depth signal in a target block), in order to lower the energy of the prediction residual for the block. We implemented our DCR-based motion prediction scheme inside H.264; our encoded bitstreams remain 100% standard compliant. We show experimentally that our proposed encoding scheme can reduce the bitrate of depth maps coded with baseline H.264 by over 28%.
Learning-based adaptive tone mapping for keypoint detection

The goal of tone mapping operators (TMOs) has traditionally been to display high dynamic range (HDR) pictures in a perceptually favorable way. However, when tone-mapped images are to be used for computer vision tasks such as keypoint detection, these design approaches are suboptimal. In this paper, we propose a new learning-based adaptive tone mapping framework which aims at enhancing keypoint stability under drastic illumination variations. To this end, we design a pixel-wise adaptive TMO which is modulated based on a model derived by Support Vector Regression (SVR) using local higher order characteristics. To circumvent the difficulty to train SVR in this context, we further propose a simple detection-similarity-maximization model to generate appropriate training samples using multiple images undergoing illumination transformations. We evaluate the performance of our proposed framework in terms of keypoint repeatability for state-of-the-art keypoint detectors. Experimental results show that our proposed learning-based adaptive TMO yields higher keypoint stability when compared to existing perceptually-driven state-of-the-art TMOs.
Perceived dynamic range of HDR images

Although high dynamic range (HDR) imaging has gained great popularity and acceptance in both the scientific and commercial domains, the relationship between perceptually accurate, content-independent dynamic range and objective measures has not been fully explored. In this paper, a new methodology for perceived dynamic range evaluation of complex stimuli in HDR conditions is proposed. A subjective study with 20 participants was conducted and correlations between mean opinion scores (MOS) and three image features were analyzed. Strong Spearman correlations between MOS and objective DR measure and between MOS and image key were found. An exploratory analysis reveals that additional image characteristics should be considered when modeling perceptually-based dynamic range metrics. Finally, one of the outcomes of the study is the perceptually annotated HDR image dataset with MOS values, that can be used for HDR imaging algorithms and metric validation, content selection and analysis of aesthetic image attributes.
Resource constrained efficient acoustic source localization and tracking using a distributed network of microphones

In this paper we present an efficient method to perform acoustic source localization and tracking using a distributed network of microphones. In this scenario, there is a trade-off between the localization performance and the expense of resources: in fact, a minimization of the localization error would require to use as many sensors as possible; at the same time, as the number of microphones increases, the cost of the network inevitably tends to grow, while in practical applications only a limited amount of resources is available. Therefore, at each time instant only a subset of the sensors should be enabled in order to meet the cost constraints. We propose a heuristic method for the optimal selection of this subset of microphones, using as distortion metrics the Cramer-Rao lower bound (CRLB) and as cost function the total distance between the selected sensors. The heuristic approach has been compared to an optimal algorithm, which searches the best sensor configuration among the full set of microphones, while satisfying the cost constraint. The proposed heuristic algorithm yields similar performance w.r.t. the full-search procedure, but at a much less computational cost. We show that this method can be used effectively in an acoustic source tracking application.
Effect of color space on high dynamic range video compression performance

High dynamic range (HDR) technology allows for capturing and delivering a greater range of luminance levels compared to traditional video using standard dynamic range (SDR). At the same time, it has brought multiple challenges in content distribution, one of them being video compression. While there has been a significant amount of work conducted on this topic, there are some aspects that could still benefit this area. One such aspect is the choice of color space used for coding. In this paper, we evaluate through a subjective study how the performance of HDR video compression is affected by three color spaces: the commonly used Y'CbCr, and the recently introduced ITP (ICtCp) and Ypu'v'. Five video sequences are compressed at four bit rates, selected in a preliminary study, and their quality is assessed using pairwise comparisons. The results of pairwise comparisons are further analyzed and scaled to obtain quality scores. We found no evidence of ITP improving compression performance over Y'CbCr. We also found that Ypu'v' results in a moderately lower performance for some sequences.
Keypoint detection in RGBD images based on an efficient viewpoint-covariant multiscale representation

Texture+depth (RGBD) images provide information about the geometry of a scene, which could help improve current image matching performance, e.g., in presence of large viewpoint changes. While depth has been mainly used for processing keypoint descriptors, in this paper we focus on the keypoint detection problem. In order to produce a computationally efficient viewpoint-covariant multiscale representation, we design an image smoothing procedure which locally smooths a texture image based on the corresponding depth. This yields an approximated scale space, where we can find keypoints using a multiscale detector approach. Our experiments on both synthetic and real-world images show substantial gains with respect to 2D and other RGBD feature extraction approaches.
Evaluation of Feature Detection in HDR Based Imaging Under Changes in Illumination Conditions

High dynamic range (HDR) imaging enables to capture details in both dark and very bright regions of a scene, and is therefore supposed to provide higher robustness to illumination changes than conventional low dynamic range (LDR) imaging in tasks such as visual features extraction. However, it is not clear how much this gain is, and which are the best modalities of using HDR to obtain it. In this paper we evaluate the first block of the visual feature extraction pipeline, i.e., keypoint detection, using both LDR and different HDR-based modalities, when significant illumination changes are present in the scene. To this end, we captured a dataset with two scenes and a wide range of illumination conditions. On these images, we measure how the repeatability of either corner or blob interest points is affected with different LDR/HDR approaches. Our observations confirm the potential of HDR over conventional LDR acquisition. Moreover, extracting features directly from HDR pixel values is more effective than first tonemapping and then extracting features, provided that HDR luminance information is previously encoded to perceptually linear values.
Quality of experience in UHD-1 phase 2 television: The contribution of UHD+HFR technology

A key factor to determine the quality of experience (QoE) of a video is its capability to convey the large spectrum of perceptual phenomena that our eyes can sense in real life. In order to meet this demand, the recent DVB UHD-1 Phase 2 specification employs new video features, such as higher spatial resolutions (4K/8K) and High Frame Rate (HFR). The first enables larger field of view and level of details, while the second offers sharper images of moving objects going well beyond the current frame rates. While the contribution of each of these technologies to QoE has been investigated individually, in this paper we are interested to study their interaction, and in quantifying the benefits to users from their combination. To this end, we conduct a subjective test on compressed UHD+HFR content on a recent display capable of reproducing 100 pictures per second at 2160p resolution, with the goal to assess the increase in QoE of UHD and HFR with respect to conventional video, both individually and in combination. The results indicate that for content with fast motion, at higher bitrates the combination of UHD and HFR significantly improves the QoE compared to that obtained when these features are used individually.
An image smoothing operator for fast and accurate scale space approximation

Gussian image smoothing is a fundamental operation in the extraction of scale-invariant feature points. Its computation, however, can be too expensive in some resource-constrained scenarios. Alternative solutions such as the box filter can be computed more efficiently, at the cost of a loss in feature repeatibility under some conditions. In this paper we propose a fast and accurate image smoothing operator based on integral images. It has the same order of computational complexity as the box filter, but provides much more accurate visual results and improved keypoint repeatability, which is confirmed in a feature detection scenario using SIFT features.
Using region-of-interest for quality evaluation of DIBR-based view synthesis methods

As 3D media became more and more popular over the last years, new technologies are needed in the transmission, compression and creation of 3D content. One of the most commonly used techniques for aiding with the compression and creation of 3D content is known as view synthesis. The most effective class of view synthesis algorithms are using Depth-Image-Based-Rendering techniques, which use explicit scene geometry to render new views. However, these methods may produce geometrical distortions and localized artifacts which are difficult to evaluate as they are inherently different from encoding errors and they are perceived differently by human subjects. In this paper, we propose a region-of-interest evaluation technique for view synthesis based on DIBR methods. Based on the assumption that certain areas determined by the geometrical properties of the scene are prone to distortions, we select a ROI by analyzing the multiple DIBR methods together with the ground truth. The approach is tested using a subjective evaluation view synthesis database and show that our method improves the SSIM correlation with subjective scores We also test another similar method and traditional metrics.
Privacy-Enabled Object Tracking in Video Sequences Using Compressive Sensing

In a typical video analysis framework, video sequences are decoded and reconstructed in the pixel domain before being processed for high level tasks such as classification or detection.Nevertheless, in some application scenarios, it might be of interest to complete these analysis tasks without disclosing sensitive data, e.g. the identity of people captured by surveillance cameras. In this paper we propose a new coding scheme suitable for video surveillance applications that allows tracking of video objects without the need to reconstruct the sequence,thus enabling privacy protection. By taking advantage of recent findings in the compressive sensing literature, we encode a video sequence with a limited number of pseudo-random projections of each frame. At the decoder, we exploit the sparsity that characterizes background subtracted images in order to recover the location of the foreground object. We also leverage the prior knowledge about the estimated location of the object, which is predicted by means of a particle filter, to improve the recovery of the foreground object location. The proposed framework enables privacy, in the sense it is impossible to reconstruct the original video content from the encoded random projections alone, as well as secrecy, since decoding is prevented if the seed used to generate the random projections is not available.
Coronary artery extraction and analysis for detection of soft plaques in MDCT images

In this paper we aim to develop a computationally-efficient image-segmentation procedure for detection and quantification of soft plaques in coronary arteries from multidetector CT images. The proposed method consists of three steps: extraction of the arterial lumen centerline, segmentation of the lumen and arterial wall based on a locally-adaptive mixture-model using the expectation- maximization algorithm, and detection of soft plaques based on effective cross-sectional areas of the lumen and of the wall. Preliminary results using clinical acquisitions are presented to demonstrate the effectiveness of the proposed method.