nils-herrmann/clean_scirepeval huggingface dataset:
Flicker sensitive motion tuned video quality assessment

From a series of human subjective studies, we have found that large motion can strongly suppress flicker visibility. Based on the spectral analysis of flicker videos in frequency domain, we propose a full reference video quality assessment (VQA) framework that incorporates flicker sensitive temporal visual masking. The framework predicts perceptually silenced flicker visibility using a model of the responses of primary visual cortex to video flicker, a motion energy model, and divisive normalization. By incorporating perceptual flicker visibility into motion tuned video quality measurements as in the MOVIE framework, we augment VQA performance with sensitivity to flicker. Results show that the proposed VQA framework correlates well with human results and is highly competitive with recent state-of-the-art VQA algorithms tested on the LIVE VQA database.
Asymptotic convergence of biorthogonal wavelet filters

We study the asymptotic behavior of the dual filters associated with biorthogonal spline wavelets (BSWs) and general biorthogonal Coifman wavelets (GBCWs). As the order of wavelet systems approaches infinity the BSW filters either diverge or converge to some non-ideal filters, the GBCW synthesis filters converge to an ideal halfband lowpass (HBLP) filter without exhibiting any Gibbs-like phenomenon, and a subclass of the analysis filters also converge to an ideal HBLP filter but with a one-sided Gibbs-like behavior. The two approximations of the ideal HBLP filter by Daubechies orthonormal wavelet filters and by the GBCW synthesis filters are also compared.
Multimodal Interactive Continuous Scoring of Subjective 3D Video Quality of Experience

People experience a variety of 3D visual programs, such as 3D cinema, 3D TV and 3D games, making it necessary to deploy reliable methodologies for predicting each viewer's subjective experience. We propose a new methodology that we call multimodal interactive continuous scoring of quality (MICSQ). MICSQ is composed of a device interaction process between the 3D display and a separate device (PC, tablet, etc.) used as an assessment tool, and a human interaction process between the subject(s) and the separate device. The scoring process is multimodal, using aural and tactile cues to help engage and focus the subject(s) on their tasks by enhancing neuroplasticity. Recorded human responses to 3D visualizations obtained via MICSQ correlate highly with measurements of spatial and temporal activity in the 3D video content. We have also found that 3D quality of experience (QoE) assessment results obtained using MICSQ are more reliable over a wide dynamic range of content than obtained by the conventional single stimulus continuous quality evaluation (SSCQE) protocol. Moreover, the wireless device interaction process makes it possible for multiple subjects to assess 3D QoE simultaneously in a large space such as a movie theater, at different viewing angles and distances. We conducted a series of interesting 3D experiments showing the accuracy and versatility of the new system, while yielding new findings on visual comfort in terms of disparity, motion and an interesting relation between the naturalness and depth of field (DOF) of a stereo camera.
Perceptual soft thresholding using the structural similarity index

In this paper, we present a novel algorithm for wavelet domain image denoising using the soft thresholding function. The thresholds are designed to be locally optimal with respect to the structural similarity (SSIM) index. The SSIM Index is first expressed in terms of wavelet transform coefficients of orthogonal wavelet transforms. The wavelet domain representation of the SSIM Index, along with the assumption of a Gaussian prior for the wavelet coefficients is used to formulate the soft thresholding optimization problem. A locally optimal solution is found using a quasi-Newton approach. This solution is applied to denoise images in the wavelet domain. The visual quality of the images denoised using the proposed algorithm is shown to be higher compared to the MSE-optimal soft thresholding denoising solution, as measured by the SSIM Index.
Comments on "Subband coding of images using asymmetrical filterbanks"

In the above paper, Egger and Li presented a set of two-channel filterbanks, asymmetrical filterbanks (AFB's), for image coding applications. The basic properties of these filters are linear-phase, perfect reconstruction, asymmetric lengths for dual filters, and maximum regularity. In this correspondence, we point out that the proposed AFB's are not new in the sense that the proposed construction is equivalent to the factorization of Lagrange halfband filters, which has been reported by other researchers. In addition, we correct an error in the formulation of constructing AFBs in their paper.
On the generation of random numbers from heavy-tailed distributions

A method is described for the generation of random numbers using a modification of the rejection technique, which is useful when the inverse of the underlying probability distribution function is inexpressable or is expensive to compute. However, the rejection technique can also be expensive if the underlying distribution has heavy tails. The method proposed reduces this expense by computing the random variate from a subinterval of the range space which is chosen randomly. The method is illustrated for a set of parameterized density functions. This technique has proven to be effective for investigating the robust smoothing properties of a class of nonlinear digital filters by Monte Carlo simulation.
Natural contrast statistics and the selection of visual fixations

In this paper we address the problem of visual surveillance, which we define as the problem of optimally extracting information from the visual scene with a fixating, foveated imaging system. We are explicitly concerned with eye/camera movement strategies that result in maximizing information extraction from the visual field. Here we demonstrate how a novel characterization of the contrast statistics of natural images can be used for selecting fixation points that minimize the total contrast uncertainty (entropy) of natural images. We demonstrate the performance of the algorithm and compare its performance to ground truth methods. The results show that our algorithm performs favorably in terms of both efficiency and its ability to find salient features in the image.
Fast structural similarity index algorithm

The development of real-time image quality assessment algorithms is an important direction on which little research has focused. This paper presents a design of real-time implementable full-reference image quality algorithms based on the SSIM index [2] and multi-scale SSIM (MS-SSIM) index . The proposed algorithms, which modify SSIM/MS-SSIM to achieve speed, are tested on the LIVE image quality database [13] and shown to yield performance commensurate with SSIM and MS-SSIM but with much lower computational complexity.
Snakules: A Model-Based Active Contour Algorithm for the Annotation of Spicules on Mammography

We have developed a novel, model-based active contour algorithm, termed “snakules”, for the annotation of spicules on mammography. At each suspect spiculated mass location that has been identified by either a radiologist or a computer-aided detection (CADe) algorithm, we deploy snakules that are converging open-ended active contours also known as snakes. The set of convergent snakules have the ability to deform, grow and adapt to the true spicules in the image, by an attractive process of curve evolution and motion that optimizes the local matching energy. Starting from a natural set of automatically detected candidate points, snakules are deployed in the region around a suspect spiculated mass location. Statistics of prior physical measurements of spiculated masses on mammography are used in the process of detecting the set of candidate points. Observer studies with experienced radiologists to evaluate the performance of snakules demonstrate the potential of the algorithm as an image analysis technique to improve the specificity of CADe algorithms and as a CADe prompting tool.
Color Compensation of Multicolor FISH Images

Multicolor fluorescence in situ hybridization (M-FISH) techniques provide color karyotyping that allows simultaneous analysis of numerical and structural abnormalities of whole human chromosomes. Chromosomes are stained combinatorially in M-FISH. By analyzing the intensity combinations of each pixel, all chromosome pixels in an image are classified. Due to the overlap of excitation and emission spectra and the broad sensitivity of image sensors, the obtained images contain crosstalk between the color channels. The crosstalk complicates both visual and automatic image analysis and may eventually affect the classification accuracy in M-FISH. The removal of crosstalk is possible by finding the color compensation matrix, which quantifies the color spillover between channels. However, there exists no simple method of finding the color compensation matrix from multichannel fluorescence images whose specimens are combinatorially hybridized. In this paper, we present a method of calculating the color compensation matrix for multichannel fluorescence images whose specimens are combinatorially stained.
Blind Quality Assessment of Fused WorldView-3 Images by Using the Combinations of Pansharpening and Hypersharpening Paradigms

WorldView 3 (WV-3) is the first commercially deployed super-spectral, very high-resolution (HR) satellite. However, the resolution of the short-wave infrared (SWIR) bands is much lower than that of the other bands. In this letter, we describe four different approaches, which are combinations of pansharpening and hypersharpening methods, to generate HR SWIR images. Since there are no ground truth HR SWIR images, we also propose a new picture quality predictor to assess hypersharpening performance, without the need for reference images. We describe extensive experiments using actual WV-3 images that demonstrate that some approaches can yield better performance than others, as measured by the proposed blind image quality assessment model of hypersharpened SWIR images.
Unequal error protection for foveation-based error resilience over mobile networks

In this paper, we introduce an unequal error protection technique for foveation-based error resilience over highly error-prone mobile networks. For point-to-point visual communications, visual quality can be significantly increased by using foveation-based error resilience where each frame is divided into foveated and background layers according to the gaze direction of the human eye, and two bitstreams are generated. In an effort to increase the source throughput of the foveated layer, we employ unequal delay-constrained ARQ and RCPC (rate compatible punctured convolutional) codes in H.223 Annex C. In the simulation, the visual quality is increased in the range of 0.3 dB to 1 dB over channel SNR 5 dB to 15 dB.
Automated detection of near surface Martian ice layers in orbital radar data

An algorithm is presented to automatically detect near surface ice layers in images from the Shallow Subsurface Radar (SHARAD) on NASA's Mars Reconnaissance Orbiter. Mars' ice-rich Northern Polar Layered Deposits (NPLD) represents an extensive geologic record of climate history. Identifying ice layers in cross-sectional images leads to understanding the three-dimensional structure of ice layers. Scientists have manually identified layers in large data volumes, but the automated algorithm will allow studying more images from over a thousand orbital crossings. A unique coordinate transformation, based upon the surface reflection, makes subsequent filtering and detection more effective on near surface layers. Results show promising capabilities for automatically detecting ice layers on Mars.
Optimal image transmission over Visual Sensor Networks

In this paper, we propose a methodology for optimal image transmission over a VSNs (Visual Sensor Networks) via cross-layer optimization. Toward this goal, we control the compression ratio of a captured image and network parameters such as source rate, flow rate and routing path. In particular, since this scheme is based on distributed optimization, we can avoid energy concentration in a specific node such as CH (Cluster Head) which increases the network lifetime. In the simulation, we demonstrate the network adaptation procedure over a randomly deployed VSNs and evaluate the quality of the transmitted image using the SSIM (Structural Similarity) index.
Video Quality Pooling Adaptive to Perceptual Distortion Severity

It is generally recognized that severe video distortions that are transient in space and/or time have a large effect on overall perceived video quality. In order to understand this phenomena, we study the distribution of spatio-temporally local quality scores obtained from several video quality assessment (VQA) algorithms on videos suffering from compression and lossy transmission over communication channels. We propose a content adaptive spatial and temporal pooling strategy based on the observed distribution. Our method adaptively emphasizes “worst” scores along both the spatial and temporal dimensions of a video sequence and also considers the perceptual effect of large-area cohesive motion flow such as egomotion. We demonstrate the efficacy of the method by testing it using three different VQA algorithms on the LIVE Video Quality database and the EPFL-PoliMI video quality database.
3D Facial similarity: Automatic assessment versus perceptual judgments

We develop algorithms that seek to assess the similarity of 3D faces, such that similar and dissimilar faces may be classified with high correlation relative to human perception of facial similarity. To obtain human facial similarity ratings, we conduct a subjective study, where a set of human subjects rate the similarity of pairs of faces. Such similarity scores are obtained from 12 subjects on 180 3D faces, with a total of 5490 pairs of similarity scores. We then extract Gabor features from automatically detected fiducial points on the range and texture images from the 3D face and demonstrate that these features correlate well with human judgements of similarity. Finally, we demonstrate the application of using such facial similarity ratings for scalable face recognition.
Wireless Video Quality Assessment: A Study of Subjective Scores and Objective Algorithms

Evaluating the perceptual quality of video is of tremendous importance in the design and optimization of wireless video processing and transmission systems. In an endeavor to emulate human perception of quality, various objective video quality assessment (VQA) algorithms have been developed. However, the only subjective video quality database that exists on which these algorithms can be tested is dated and does not accurately reflect distortions introduced by present generation encoders and/or wireless channels. In order to evaluate the performance of VQA algorithms for the specific task of H.264 advanced video coding compressed video transmission over wireless networks, we conducted a subjective study involving 160 distorted videos. Various leading full reference VQA algorithms were tested for their correlation with human perception. The data from the paper has been made available to the research community, so that further research on new VQA algorithms and on the general area of VQA may be carried out.
A fast Multilinear ICA algorithm

We extend our previous work on Multilinear Independent Component Analysis (MICA) by introducing a Fast-MICA algorithm that demonstrates the same improvement over classical ICA as the original MICA algorithm  while improving the computational speed by two polynomial orders of magnitude. Apart for enabling a faster determination of the multilinear structure of image patch probability density, this new approach opens up, for the first time, the possibility of computing a novel non-stationarity index based on the relative change in mutual information. We demonstrate the performance of our Fast-MICA algorithm together with an illustration of our novel non-stationarity index.
Least squares order statistic filter for signal restoration

The fundamental theory for restoring discrete waveforms immersed in independent noise using order-statistic (OS) filters, with the least-squares criterion as a fidelity measure, is developed. Nondynamical least-squares OS filter design methods are extended to the case of arbitrary discrete waveforms immersed in independent noise. A method for incorporating local structural constraints into the optimization process is introduced. A small number of these constraints can ensure that the filter designed is sensitive to local high-information signal structures. This is accomplished within a natural framework by appending the constraints into the objective function to be minimized using a Lagrangian approach. The principal drawback of OS filter design remains the complexity of computing the temporal/spatial correlations of the OS. A suboptimal approximation technique that yields good results when it is applied to the image restoration problem is developed. Some directions for future research are explored.
Video Quality Assessment on Mobile Devices: Subjective, Behavioral and Objective Studies

We introduce a new video quality database that models video distortions in heavily-trafficked wireless networks and that contains measurements of human subjective impressions of the quality of videos. The new LIVE Mobile Video Quality Assessment (VQA) database consists of 200 distorted videos created from 10 RAW HD reference videos, obtained using a RED ONE digital cinematographic camera. While the LIVE Mobile VQA database includes distortions that have been previously studied such as compression and wireless packet-loss, it also incorporates dynamically varying distortions that change as a function of time, such as frame-freezes and temporally varying compression rates. In this article, we describe the construction of the database and detail the human study that was performed on mobile phones and tablets in order to gauge the human perception of quality on mobile devices. The subjective study portion of the database includes both the differential mean opinion scores (DMOS) computed from the ratings that the subjects provided at the end of each video clip, as well as the continuous temporal scores that the subjects recorded as they viewed the video. The study involved over 50 subjects and resulted in 5,300 summary subjective scores and time-sampled subjective traces of quality. In the behavioral portion of the article we analyze human opinion using statistical techniques, and also study a variety of models of temporal pooling that may reflect strategies that the subjects used to make the final decision on video quality. Further, we compare the quality ratings obtained from the tablet and the mobile phone studies in order to study the impact of these different display modes on quality. We also evaluate several objective image and video quality assessment (IQA/VQA) algorithms with regards to their efficacy in predicting visual quality. A detailed correlation analysis and statistical hypothesis testing is carried out. Our general conclusion is that existing VQA algorithms are not well-equipped to handle distortions that vary over time. The LIVE Mobile VQA database, along with the subject DMOS and the continuous temporal scores is being made available to researchers in the field of VQA at no cost in order to further research in the area of video quality assessment.
Wideband frequency excursions in computed AM-FM image models

We examine two paradigms for computing multicomponent AM-FM image models. In channelized components analysis, estimates for one AM-FM component are extracted from each channel of a multiband filterbank. Tracked multi-component analysis represents an image using fewer components by tracking the estimated modulating functions of each component across the filter bank channels. While both approaches work well for synthetic images, they have difficulty with natural images that contain phase discontinuities. We show that phase discontinuities lead to wideband frequency excursions that make the computation of AM-FM models difficult and can also severely degrade the quality of image reconstructions obtained from the models. We use postfilters to ameliorate the effects of the frequency excursions and compute AM-FM models for two natural images.
Multiple description image coding using natural scene statistics

The statistics of natural scenes in the wavelet domain are accurately characterized by the Gaussian scale mixture (GSM) model. The model lends itself easily to analysis and many applications that use this model are emerging (e.g., denoising, watermark detection). We present an error-resilient image communications application that uses the GSM model and multiple description coding (MDC) to provide error-resilience. We derive a rate-distortion bound for GSM random variables, derive the redundancy rate-distortion function, and finally implement an MD image communication system.
Visual pattern image sequence coding

Visual pattern image sequence coding (VPISC) is a new digital image sequence (video) coding process that possesses significant advantages relative to other technologies; in particular, it is extremely efficient in terms of the computational effort required. It is designed to exploit properties of the human visual system (HVS), and thus yields high visual fidelity. Visual quality criteria are deliberately chosen over information-theoretic ones on the grounds that, in images intended for human viewing, visual criteria are the most meaningful ones. VPISC yields impressive compression comparable to other recent methods, such as motion-compensated vector quantization. VPISC divides the images into spatiotemporal cubes, which are then independently matched with one of a small, predetermined set of visually meaningful three-dimensional space-time patterns. The pattern set is chosen to conform to specific characteristics of the HVS. Also introduced are two modifications of VPISC: adaptive and foveal VPISC. These are spatiotemporally nonuniform implementations that code different portions of the image sequence at different resolutions, according to either a fidelity criterion (for AVPISC) or a foveation criterion (for FVPISC).