nils-herrmann/clean_scirepeval huggingface dataset:
Referenceless Prediction of Perceptual Fog Density and Perceptual Image Defogging

We propose a referenceless perceptual fog density prediction model based on natural scene statistics (NSS) and fog aware statistical features. The proposed model, called Fog Aware Density Evaluator (FADE), predicts the visibility of a foggy scene from a single image without reference to a corresponding fog-free image, without dependence on salient objects in a scene, without side geographical camera information, without estimating a depth-dependent transmission map, and without training on human-rated judgments. FADE only makes use of measurable deviations from statistical regularities observed in natural foggy and fog-free images. Fog aware statistical features that define the perceptual fog density index derive from a space domain NSS model and the observed characteristics of foggy images. FADE not only predicts perceptual fog density for the entire image, but also provides a local fog density index for each patch. The predicted fog density using FADE correlates well with human judgments of fog density taken in a subjective study on a large foggy image database. As applications, FADE not only accurately assesses the performance of defogging algorithms designed to enhance the visibility of foggy images, but also is well suited for image defogging. A new FADE-based referenceless perceptual image defogging, dubbed DEnsity of Fog Assessment-based DEfogger (DEFADE) achieves better results for darker, denser foggy images as well as on standard foggy images than the state of the art defogging methods. A software release of FADE and DEFADE is available online for public use: <;uri xlink:href="http://live.ece.utexas.edu/research/fog/index.html" xlink:type="simple">http://live.ece.utexas.edu/research/fog/index.html<;/uri>.
Application-Driven No-Reference Quality Assessment for Dermoscopy Images With Multiple Distortions

Dermoscopy images often suffer from blur and uneven illumination distortions that occur during acquisition, which can adversely influence consequent automatic image analysis results on potential lesion objects. The purpose of this paper is to deploy an algorithm that can automatically assess the quality of dermoscopy images. Such an algorithm could be used to direct image recapture or correction.  We describe an application-driven no-reference image quality assessment (IQA) model for dermoscopy images affected by possibly multiple distortions. For this purpose, we created a multiple distortion dataset of dermoscopy images impaired by varying degrees of blur and uneven illumination. The basis of this model is two single distortion IQA metrics that are sensitive to blur and uneven illumination, respectively. The outputs of these two metrics are combined to predict the quality of multiply distorted dermoscopy images using a fuzzy neural network. Unlike traditional IQA algorithms, which use human subjective score as ground truth, here ground truth is driven by the application, and generated according to the degree of influence of the distortions on lesion analysis.  The experimental results reveal that the proposed model delivers accurate and stable quality prediction results for dermoscopy images impaired by multiple distortions.  The proposed model is effective for quality assessment of multiple distorted dermoscopy images.  An application-driven concept for IQA is introduced, and at the same time, a solution framework for the IQA of multiple distortions is proposed.
Modeling of growth via active contour models

The authors analyze the growth of biological specimens with the help of Active Contour Models known as Snakes. Conventional shape-change models like optical flow, deformable templates, etc., primarily address rigid and non-rigid deformations which are not in the form of growth. Moreover the above models assume the shape-change between successive frames in an image sequence to be small which is not true in many real-world biomedical image sequences. The Active Contour Model presented here tracks the boundary of an object as it changes shape and can also predict how the object might change shape. The algorithm is an energy minimization technique which tracks the growth of a specimen by attracting a closed piecewise cubic curve to the boundary of the specimen in every image of a sequence. This method computes shape-change especially when it is in the form of growth-better than techniques like optical flow, deformable templates, etc. Unlike conventional shape-change algorithms, the authors' algorithm can be applied iteratively to compute drastic shape-changes also. The authors also propose a parallel framework to facilitate real-time implementation.
Theory of order statistic filters and their relationship to linear FIR filters

Necessary and/or sufficient conditions on both the filter coefficients and the signal process are derived in order that nonrecursive order statistic (OS) and linear filtering are equivalent operations. The results indicate that an understanding of OS filters hinges on a better understanding of the properties of signals containing logically monotonic components. The results extend a number of previous theories characterizing the well-known median and ranked-order filters to a broader class of filters and input signals.
Scene statistics of authentically distorted images in perceptually relevant color spaces for blind image quality assessment

Current top-performing blind image quality assessment (IQA) models rely on benchmark databases comprising of singly distorted images, thereby learning image features that are only adequate to predict human perceived visual quality on such inauthentic distortions. Furthermore, the underlying image features of these models are often extracted from the achromatic luminance channel and could sometimes fail to account for the loss of their perceived quality that might potentially be distinctly captured in a different image modality. In this work, we propose a novel IQA model that focuses on the natural scene statistics of images afflicted with complex mixtures of unknown, authentic distortions. We derive several feature maps in different perceptually relevant color spaces and extract a large number of image features from them. We demonstrate the remarkable competence of our features in improving the automatic perceptual quality prediction on images containing both synthetic and authentic distortions.
A Steerable, Multiscale Singularity Index

We propose a new steerable, multiscale ratio index for detecting impulse singularities in signals of arbitrary dimensionality. For example, it responds strongly to curvilinear masses (ridges) in images, but minimally to step discontinuities. The ratio index employs directional derivatives of gaussians, making it naturally steerable and scalable. Experiments on real images demonstrate the efficacy of the index for detecting multiscale curvilinear structures. A software version of the index can be downloaded from: http://live.ece.utexas.edu/research/SingularityIndex/SingularityIndex.zip.
Maximum-likelihood depth-from-defocus for active vision

A new method for actively recovering depth information using image defocus is demonstrated and shown to support active stereo vision depth recovery by providing monocular depth estimates to guide the positioning of cameras for stereo processing. This active depth-from-defocus approach employs a spatial frequency model for image defocus which incorporates the optical transfer function of the image acquisition system and a maximum likelihood estimator to determine the amount of defocus present in a sequence of two or more images taken from the same pose. This defocus estimate is translated into a measurement of depth and associated uncertainty that is used to control the positioning of a variable baseline stereo camera system. This cooperative arrangement significantly reduces the matching uncertainty of the stereo correspondence process and increases the depth resolution obtainable with an active stereo vision platform.
On the visibility of flicker distortions in naturalistic videos

Motion can reduce the visibility of flicker distortions. We performed human subjective tests to investigate how motion influences the visibility of flicker distortions in naturalistic videos. Forty three naïve subjects participated in two tasks (“follow the moving object” and “view freely”) and reported their percepts on 36 test videos. Flicker distortions were simulated by periodic changes of video frames at different quality levels and flicker frequencies. An eye tracker was used to monitor each subject's gaze. The results indicate that the visibility of flicker distortions is strongly reduced when the speed of coherent motion is large, and the effect is more pronounced when the video quality is poor. We conjecture that sufficiently fast and coherent motions near the gaze point mask or `silence' the perception of flicker distortions in naturalistic videos in agreement with a recently observed `motion silencing' effect on synthetic stimuli.
3D Face Recognition Founded on the Structural Diversity of Human Faces

We present a systematic procedure for selecting facial fiducial points associated with diverse structural characteristics of a human face. We identify such characteristics from the existing literature on anthropometric facial proportions. We also present three dimensional (3D) face recognition algorithms, which employ Euclidean/geodesic distances between these anthropometric fiducial points as features along with linear discriminant analysis classifiers. Furthermore, we show that in our algorithms, when anthropometric distances are replaced by distances between arbitrary regularly spaced facial points, their performances decrease substantially. This demonstrates that incorporating domain specific knowledge about the structural diversity of human faces significantly improves the performance of 3D human face recognition algorithms.
Face Detection on Distorted Images Augmented by Perceptual Quality-Aware Features

Motivated by the proliferation of low-cost digital cameras in mobile devices being deployed in automated surveillance networks, we study the interaction between perceptual image quality and a classic computer vision task of face detection. We quantify the degradation in performance of a popular and effective face detector when human-perceived image quality is degraded by distortions commonly occurring in capture, storage, and transmission of facial images, including noise, blur, and compression. It is observed that, within a certain range of perceived image quality, a modest increase in image quality can drastically improve face detection performance. These results can be used to guide resource or bandwidth allocation in acquisition or communication/delivery systems that are associated with face detection tasks. A new set of features, called qualHOG, are proposed for robust facedetection that augments face-indicative Histogram of Oriented Gradients (HOG) features with perceptual quality-aware spatial Natural Scene Statistics (NSS) features. Face detectors trained on these new features provide statistically significant improvement in tolerance to image distortions over a strong baseline. Distortiondependent and distortion-unaware variants of the face detectors are proposed and evaluated on a large database of face images representing a wide range of distortions. A biased variant of the training algorithm is also proposed that further enhances the robustness of these face detectors. To facilitate this research, we created a new distorted face database (DFD), containing face and non-face patches from images impaired by a variety of common distortion types and levels. This new data set and relevant code are available for download and further experimentation at www.live.ece.utexas.edu/research/Quality/index.htm.
Generalized coiflets with nonzero-centered vanishing moments

We show that imposing a certain number of vanishing moments on a scaling function (e.g., coiflets) leads to fairly small phase distortion on its associated filter bank in the neighborhood of DC. However, the phase distortion at the other frequencies can be much larger. We design a new class of real-valued, compactly supported, orthonormal, and nearly symmetric wavelets (we call them generalized coiflets) with a number of nonzero-centered vanishing moments equally distributed on scaling function and wavelet. Such a generalization of the original coiflets offers one more free parameter, the mean of the scaling function, in designing filter banks. Since this parameter uniquely characterizes the first several moments of the scaling function, it is related to the phase response of the lowpass filter at low frequencies. We search for the optimal parameter to minimize the maximum phase distortion of the filter bank over the lowpass half-band. Also, we are able to construct nearly odd-symmetric generalized coiflets, whose associated lowpass filters are surprisingly similar to those of some biorthogonal spline wavelets. These new wavelets can be useful in a broad range of signal and image processing applications because they provide a better tradeoff between the two desirable but conflicting properties of the compactly supported and real-valued wavelets, i.e., orthonormality versus symmetry, than the original coiflets.
A new singularity index

We propose a new ratio index for the detection of impulse-like singularities in signals of arbitrary dimensionality. We show that the new singularity index responds strongly to singularities that are like impulses or smoothed impulses in cross section. For example, it responds strongly to curvilinear masses (ridges) in images, while responding minimally to edge-like singularities. The ratio index employs directional derivatives of gaussians, which makes the index naturally scalable.
Multidimensional orthogonal FM transforms

The present a novel class of multidimensional orthogonal FM transforms. The analysis suggests a novel signal-adaptive FM transform possessing interesting energy compaction properties. We show that the proposed signal-adaptive FM transform produces point spectra for multidimensional signals with uniformly distributed samples. This suggests that the proposed transform is suitable for energy compaction and subsequent coding of broadband signals and images that locally exhibit significant level diversity. We illustrate these concepts with simulation experiments.
Passive Multimodal 2-D+3-D Face Recognition Using Gabor Features and Landmark Distances

We introduce a novel multimodal framework for face recognition based on local attributes calculated from range and portrait image pairs. Gabor coefficients are computed at automatically detected landmark locations and combined with powerful anthropometric features defined in the form of geodesic and Euclidean distances between pairs of fiducial points. We make the pragmatic assumption that the 2-D and 3-D data is acquired passively (e.g., via stereo ranging) with perfect registration between the portrait data and the range data. Statistical learning approaches are evaluated independently to reduce the dimensionality of the 2-D and 3-D Gabor coefficients and the anthropometric distances. Three parallel face recognizers that result from applying the best performing statistical learning schemes are fused at the match score-level to construct a unified multimodal (2-D+3-D) face recognition system with boosted performance. Performance of the proposed algorithm is evaluated on a large public database of range and portrait image pairs and found to perform quite well.
3D Visual Discomfort Predictor: Analysis of Disparity and Neural Activity Statistics

Being able to predict the degree of visual discomfort that is felt when viewing stereoscopic 3D (S3D) images is an important goal toward ameliorating causative factors, such as excessive horizontal disparity, misalignments or mismatches between the left and right views of stereo pairs, or conflicts between different depth cues. Ideally, such a model should account for such factors as capture and viewing geometries, the distribution of disparities, and the responses of visual neurons. When viewing modern 3D displays, visual discomfort is caused primarily by changes in binocular vergence while accommodation in held fixed at the viewing distance to a flat 3D screen. This results in unnatural mismatches between ocular fixations and ocular focus that does not occur in normal direct 3D viewing. This accommodation vergence conflict can cause adverse effects, such as headaches, fatigue, eye strain, and reduced visual ability. Binocular vision is ultimately realized by means of neural mechanisms that subserve the sensorimotor control of eye movements. Realizing that the neuronal responses are directly implicated in both the control and experience of 3D perception, we have developed a model-based neuronal and statistical framework called the 3D visual discomfort predictor (3D-VDP) that automatically predicts the level of visual discomfort that is experienced when viewing S3D images. 3D-VDP extracts two types of features: 1) coarse features derived from the statistics of binocular disparities and 2) fine features derived by estimating the neural activity associated with the processing of horizontal disparities. In particular, we deploy a model of horizontal disparity processing in the extrastriate middle temporal region of occipital lobe. We compare the performance of 3D-VDP with other recent discomfort prediction algorithms with respect to correlation against recorded subjective visual discomfort scores, and show that 3D-VDP is statistically superior to the other methods.
Comparison of regressors on 3D visual discomfort prediction

Most stereoscopic 3D (S3D) image visual discomfort predictors use the Support Vector Regressor (SVR) as the regression model. However, there are other good regression models such as the Random Forests (RF) and Gradient Boost Regression Tree (GBRT). Here we study the efficacy of these regression models for S3D image visual discomfort prediction. We deployed several regression models to predict the visual discomfort scores on S3D images using three kinds of features extracted from the images in the IEEE-SA and EPFL S3D image databases. So the prediction performance was evaluated to compare the performance of the different regression models. We also studied the issue of over-fitting which can affect model performance. The comparison results show that it is feasible and reliable to apply Random Forests or GBRT for S3D visual discomfort prediction with better performance than SVR.
Recurrent and Dynamic Models for Predicting Streaming Video Quality of Experience

Streaming video services represent a very large fraction of global bandwidth consumption. Due to the exploding demands of mobile video streaming services, coupled with limited bandwidth availability, video streams are often transmitted through unreliable, low-bandwidth networks. This unavoidably leads to two types of major streaming-related impairments: compression artifacts and/or rebuffering events. In streaming video applications, the end-user is a human observer; hence being able to predict the subjective Quality of Experience (QoE) associated with streamed videos could lead to the creation of perceptually optimized resource allocation strategies driving higher quality video streaming services. We propose a variety of recurrent dynamic neural networks that conduct continuous-time subjective QoE prediction. By formulating the problem as one of time-series forecasting, we train a variety of recurrent neural networks and non-linear autoregressive models to predict QoE using several recently developed subjective QoE databases. These models combine multiple, diverse neural network inputs, such as predicted video quality scores, rebuffering measurements, and data related to memory and its effects on human behavioral responses, using them to predict QoE on video streams impaired by both compression artifacts and rebuffering events. Instead of finding a single time-series prediction model, we propose and evaluate ways of aggregating different models into a forecasting ensemble that delivers improved results with reduced forecasting variance. We also deploy appropriate new evaluation metrics for comparing time-series predictions in streaming applications. Our experimental results demonstrate improved prediction performance that approaches human performance. An implementation of this work can be found at https://github.com/christosbampis/NARX_QoE_release.
Maximally flat bandwidth allocation for variable bit rate video

When a variable bit rate (VBR) video bitstream is transmitted into a network, the channel utilization is dependent on the maximum peak value of the traffic pattern and improved by reducing the peak rate. A lower bound on the transmission rate is derived from the given traffic cells. Based on the transmission rate constraint, the optimal bandwidth that is needed to meet the delay requirement during an allocation interval is also derived. From the delay bound and the constraints, we demonstrate that the traffic has been made as flat as possible.
Segmentation and extraction of the spinal canal in sagittal MR images

We propose a fully automated spine MRI image segmentation method based on an extension of the multiscale Singularity Index. Without any training on image patches and any user information or annotation, the proposed method relies on the properties of T2 weighted sequencing and the power of the multiscale Singularity Index. Our method detects and extracts the centerline of the human spine and segments the relevant part of the spinal body by combining the filter responses and discarding erroneous candidates for the spinal region. This segmentation step can be critical for further analysis of the spinal region either by a physician or an automated algorithm. Evaluation on a variety of spine images shows that this method achieves results comparable to human segmentations on many challenging instances.
Adaptive trimmed mean filters for image restoration

An adaptive smoothing filter is proposed for reducing noise in digital signals of any dimensionality. The adaptive procedure is based on the selection of an appropriate inner or outer trimmed mean filter according to local measurements of the tail behavior (impulsivity) of the noise process. The set of trimmed means used provides robustness against a wide range of noise possibilities ranging from very shallow tailed to very heavy tailed. A Monte Carlo analysis using a family of generalized exponential distributions supports the choice of the trimmed mean selected for measured values of an impulsivity statistic. The assumption underlying the definition of the filter is that the signal to be filtered is locally smoothly varying, and that the noise process is uncorrelated and derives from an unknown, unimodal symmetric distribution. For image-processing applications, a second statistic is used to mark the location of abrupt intensity changes, or edges; in the vicinity of an edge, the trend-preserving median filter is used. Since the impulsivity and edge statistics used in defining the adaptive filter are both functions of order statistics, the extra computation required for their calculation is minimal. Examples are provided of the filter as applied to images corrupted by a variety of noises.
Snakules: Snakes that seek spicules on mammography

We present a new method called “snakules” for the annotation of spicules on mammography. Snakules employs parametric open-ended snakes that are deployed in a region around a suspect spiculated mass location that has been identified by a radiologist or a computer-aided detection (CADe) algorithm. The set of convergent snakules deform, grow and adapt to the true spicules in the image, by an attractive process of curve evolution and motion that optimizes the local matching energy. Our results from an initial observer study involving an experienced radiologist demonstrate the strong potential of the method as an image analysis technique to improve the specificity of CADe algorithms.