nils-herrmann/clean_scirepeval huggingface dataset:
Extracting Regions of Interest from Still Images: Color Saliency and Wavelet-Based Approaches

In this paper we explore four distinct approaches to extracting regions of interest (ROI) from still images. We show the results obtained for each of the proposed approaches, and we demonstrate where each method outperforms the other. The four approaches are: 1) a block-based discrete wavelet transform (DWT) algorithm, 2) a color saliency approach, 3) a wavelet coefficients variance saliency approach, and 4) an approach based on mean-shift clustering of image pixels. The wavelet-based approaches are shown to perform well on natural scene images that usually contain regions of distinct textures. The color saliency approach performs well on images containing objects of high saturation and brightness, and the mean-shift clustering approach partitions the image into regions according to the density distribution of pixel intensities.
RRED indices: Reduced reference entropic differencing framework for image quality assessment

We study the problem of automatic “reduced reference” image quality assessment algorithms from the point of view of image information change. Algorithms that measure differences between the entropies of wavelet coefficients of reference and distorted images are designed. A family of algorithms are presented, each differing in the amount of data on which information change is predicted and ranging from almost full reference to almost no reference. A special case of this are algorithms that require just a single number from the reference for quality assessment. The algorithms are shown to correlate very well with subjective quality scores as demonstrated on the LIVE Image Quality Assessment Database.
Approximating filtered scale-variant signals

We develop theorems that place limits on the point-wise approximation of the responses of filters, both linear shift invariant (LSI) and linear shift variant (LSV), to input signals and images that are LSV in the following sense: they can be expressed as the outputs of systems with LSV impulse responses, where the shift variance is with respect to the filter scale of a single-prototype filter. The approximations take the form of LSI approximations to the responses. We develop tight bounds on the approximation errors expressed in terms of filter durations and derivative (Sobolev) norms. Finally, we find application of the developed theory to defoveation of images, deblurring of shift-variant blurs, and shift-variant edge detection.
Enhancement of Visual Comfort and Sense of Presence on Stereoscopic 3D Images

Conventional stereoscopic 3D (S3D) displays do not provide accommodation depth cues of the 3D image or video contents being viewed. The sense of content depths is thus limited to cues supplied by motion parallax (for 3D video), stereoscopic vergence cues created by presenting left and right views to the respective eyes, and other contextual and perspective depth cues. The absence of accommodation cues can induce two kinds of accommodation vergence mismatches (AVM) at the fixation and peripheral points, which can result in severe visual discomfort. With the aim of alleviating discomfort arising from AVM, we propose a new visual comfort enhancement approach for processing S3D visual signals to deliver a more comfortable 3D viewing experience at the display. This is accomplished via an optimization process whereby a predictive indicator of visual discomfort is minimized, while still aiming to maintain the viewer’s sense of 3D presence by performing a suitable parallax shift, and by directed blurring of the signal. Our processing framework is defined on 3D visual coordinates that reflect the nonuniform resolution of retinal sensors and that uses a measure of 3D saliency strength. An appropriate level of blur that corresponds to the degree of parallax shift is found, making it possible to produce synthetic accommodation cues implemented using a perceptively relevant filter. By this method, AVM, the primary contributor to the discomfort felt when viewing S3D images, is reduced. We show via a series of subjective experiments that the proposed approach improves visual comfort while preserving the sense of 3D presence.
New Directions in Image and Video Quality Assessment Plenary Talk

Perceptual Image Processing is taking an increasingly important role in the field of multimedia processing. Designing algorithms to accord with visual perception is a natural idea, but has met with limited success owing to our imperfect knowledge of the intended receiver, and indeed, of the transmitter. The receiver in this context, of course, is the marvelous human eye-cortex system, while the transmitter is the environment, which casts images of extraordinary variability onto camera and retinal sensors.
Perceptually Unequal Packet Loss Protection by Weighting Saliency and Error Propagation

We describe a method for achieving perceptually minimal video distortion over packet-erasure networks using perceptually unequal loss protection (PULP). There are two main ingredients in the algorithm. First, a perceptual weighting scheme is employed wherein the compressed video is weighted as a function of the nonuniform distribution of retinal photoreceptors. Secondly, packets are assigned temporal importance within each group of pictures (GOP), recognizing that the severity of error propagation increases with elapsed time within a GOP. Using both frame-level perceptual importance and GOP-level hierarchical importance, the PULP algorithm seeks efficient forward error correction assignment that balances efficiency and fairness by controlling the size of identified salient region(s) relative to the channel state. PULP demonstrates robust performance and significantly improved subjective and objective visual quality in the face of burst packet losses.
Foveated video image analysis and compression gain measurements

We present a framework for assessing the quality of, and determining the efficiency of foveated and compressed images and video streams. We develop unique algorithms for assessing the quality of foveated image/video data. By interpreting foveation as a coordinate transformation, we analyze the increase in compression efficiency that is afforded by our foveation approach. We demonstrate these concepts on foveated, compressed video streams using modified (foveated) versions of H.263 that are standards-compliant. In the simulations, quality versus compression is enhanced considerably by the foveation approach. We obtain compression gains ranging from 8% to 52% for I pictures and from 7% to 68% for P pictures.
FOVEA: a foveated vergent active stereo vision system for dynamic three-dimensional scene recovery

We introduce FOVEA: a foveated vergent active stereo vision system. FOVEA actively directs a pair of vergent stereo cameras to fixate on surfaces in a scene, performing multiresolution surface depth recovery at each fixation point, and accumulating and integrating a multiresolution map of surface depth over multiple successive fixations. Several features of the system are novel: an active foveated image sampling and processing strategy is shown to greatly simplify the problem of establishing correspondence; a probabilistic fixation strategy is developed that is driven by the scene structure; the system uses the fixation strategy to recover local depth maps at a high resolution at multiple fixation points, eventually mapping the entire scene; and finally, the local maps are integrated as they are acquired into a global depth map.
Segmenting cervical epithelial nuclei from confocal images Gaussian Markov random fields

Cervical cancer is always preceded by epithelial lesions which have larger and more densely spaced nuclei than normal tissue. Detecting and removing these lesions prevents the development of cervical cancer. A proposed method to detect precancerous lesion in vivo is to use the nuclear size and density information from fiber optic confocal images of the cervical epithelial tissue to classify the tissue as normal or precancerous. Automatically segmenting nuclei is challenging because they are hard to decipher from the noise in the confocal images. This paper outlines an algorithm to automatically segment cervical epithelial nuclei from fiber optic confocal videos using Gaussian Markov random fields. Gaussian Markov random fields segment images with additive Gaussian noise by modeling the underlying structure of the image. The algorithm described in this paper detects 90% of the nuclei in each frame with a 14% error rate.
AM-FM texture segmentation in electron microscopic muscle imaging

Describes the application of an amplitude modulation-frequency modulation (AM-FM) image representation in segmenting electron micrographs of skeletal muscle for the recognition of: (1) normal sarcomere ultrastructural pattern and (2) abnormal regions that occur in sarcomeres in various myopathies. A total of 26 electron micrographs from different myopathies mere used for this study. It is shown that the AM-FM image representation can identify normal repetitive structures and sarcomeres, with a good degree of accuracy. This system can also detect abnormalities in sarcomeres which alter the normal regular pattern, as seen in muscle pathology, with a recognition accuracy of 75%-84% as compared to a human expert.
Analyzing Image Structure by Multidimensional Frequency Modulation

We develop a mathematical framework for quantifying and understanding multidimensional frequency modulations in digital images. We begin with the widely accepted definition of the instantaneous frequency vector (IF) as the gradient of the phase and define the instantaneous frequency gradient tensor (IFGT) as the tensor of component derivatives of the IF vector. Frequency modulation bounds are derived and interpreted in terms of the eigendecomposition of the IFGT. Using the IFGT, we derive the ordinary differential equations (ODEs) that describe image flowlines. We study the diagonalization of the ODEs of multidimensional frequency modulation on the IFGT eigenvector coordinate system and suggest that separable transforms can be computed along these coordinates. We illustrate these new methods of image pattern analysis on textured and fingerprint images. We envision that this work will find value in applications involving the analysis of image textures that are nonstationary yet exhibit local regularity. Examples of such textures abound in
Blind/Referenceless Image Spatial Quality Evaluator

We propose a natural scene statistic based Blind/Referenceless Image Spatial QUality Evaluator (BRISQUE) which extracts the point wise statistics of local normalized luminance signals and measures image naturalness (or lack there of) based on measured deviations from a natural image model. We also model the distribution of pairwise statistics of adjacent normalized luminance signals which provides distortion orientation information. Although multi scale, the model uses easy to compute features making it computationally fast and time efficient. The frame work is shown to perform statistically better than other proposed no reference algorithms and full reference structural similarity index (SSIM).
Study of Subjective and Objective Quality Assessment of Video

We present the results of a recent large-scale subjective study of video quality on a collection of videos distorted by a variety of application-relevant processes. Methods to assess the visual quality of digital videos as perceived by human observers are becoming increasingly important, due to the large number of applications that target humans as the end users of video. Owing to the many approaches to video quality assessment (VQA) that are being developed, there is a need for a diverse independent public database of distorted videos and subjective scores that is freely available. The resulting Laboratory for Image and Video Engineering (LIVE) Video Quality Database contains 150 distorted videos (obtained from ten uncompressed reference videos of natural scenes) that were created using four different commonly encountered distortion types. Each video was assessed by 38 human subjects, and the difference mean opinion scores (DMOS) were recorded. We also evaluated the performance of several state-of-the-art, publicly available full-reference VQA algorithms on the new database. A statistical evaluation of the relative performance of these algorithms is also presented. The database has a dedicated web presence that will be maintained as long as it remains relevant and the data is available online.
Computing shape changes in SOLANUM tuberosa slices viewed through a stereo microscope

A research effort is described to compute the 3-D shape parameters of biological objects observed through a stereoscope light microscope. An algorithm for computing global shape parameters from microscopic images is presented. Algorithms for computation of global shape parameters from images obtained through a stereo light microscope are developed and applied to characterize the shape parameters of small piece of potato slices when exposed to hyperosmotic sugar solutions.<<ETX>>
Blind Prediction of Natural Video Quality

We propose a blind (no reference or NR) video quality evaluation model that is nondistortion specific. The approach relies on a spatio-temporal model of video scenes in the discrete cosine transform domain, and on a model that characterizes the type of motion occurring in the scenes, to predict video quality. We use the models to define video statistics and perceptual features that are the basis of a video quality assessment (VQA) algorithm that does not require the presence of a pristine video to compare against in order to predict a perceptual quality score. The contributions of this paper are threefold. 1) We propose a spatio-temporal natural scene statistics (NSS) model for videos. 2) We propose a motion model that quantifies motion coherency in video scenes. 3) We show that the proposed NSS and motion coherency models are appropriate for quality assessment of videos, and we utilize them to design a blind VQA algorithm that correlates highly with human judgments of quality. The proposed algorithm, called video BLIINDS, is tested on the LIVE VQA database and on the EPFL-PoliMi video database and shown to perform close to the level of top performing reduced and full reference VQA algorithms.
Multi-scale underwater descattering

Underwater images suffer from severe perceptual/visual degradation, due to the dense and non-uniform medium, causing scattering and attenuation of the propagated light that is sensed. Typical restoration methods rely on the popular Dark Channel Prior to estimate the light attenuation factor, and subtract the back-scattered light influence to invert the underwater imaging model. However, as a consequence of using approximate and global estimates of the back-scattered light, most existing single-image underwater descattering techniques perform poorly when restoring non-uniformly illuminated scenes. To mitigate this problem, we introduce a novel approach that estimates the back-scattered light locally, based on the observation of a neighborhood around the pixel of interest. To circumvent issue related to selection of the neighborhood size, we propose to fuse the images obtained over both small and large neighborhoods, each capturing distinct features from the input image. In addition, the Laplacian of the original image is provided as a third input to the fusion process, to enhance texture details in the reconstructed image. These three derived inputs are seamlessly blended via a multi-scale fusion approach, using saliency, contrast, and saturation metrics to weight each input. We perform an extensive qualitative and quantitative evaluation against several specialized techniques. In addition to its simplicity, our method outperforms the previous art on extreme underwater cases of artificial ambient illumination and high water turbidity.
Modeling and restoration of Raman microscopic images

Presents a model for Raman microscopic images. The model describes the degradation of Raman signals by non-uniform illumination, by the microscopic system, and by additive signal-dependent Gaussian noise. Using this model, synthetic images were created to validate the model. Based on these synthetic images, an anisotropic diffusion filter was applied to reduce the signal-dependent Gaussian noise and at the same time not blur the objects' boundary. A Wiener filter was used to restore the blurred Raman images by the microscopic system. And, an image ratioing method was used to correct for the non-uniform illumination. After the restoration, the mean absolute error between the restored image and the true image was minimized.
Non-uniformity correction of IR images using natural scene statistics

Infrared images are commonly afflicted by distortions such as non-uniformity. Non-uniformity is characterized by horizontal and vertical fixed pattern noise. Accurately estimating the amount of non-uniformity present in an image and removing that amount of non-uniformity noise are open problems. Several estimators of non-uniformity exist, but their ability to estimate degrades with the presence of other sources of noise. Specifically, most of these metrics lack the robustness demanded by a more complete non-uniformity model. Previous non-uniformity correction algorithms are compared and found to underperform relative to a more complete model of non-uniformity that we have developed. Using this model, we have created a new denoising algorithm, which we call the Gaussian scale mixture perceptual pattern denoiser. The new model and algorithm can fully characterize non-uniformity using covariance matrices.
Single-Scale Fusion: An Effective Approach to Merging Images

Due to its robustness and effectiveness, multi-scale fusion (MSF) based on the Laplacian pyramid decomposition has emerged as a popular technique that has shown utility in many applications. Guided by several intuitive measures (weight maps) the MSF process is versatile and straightforward to be implemented. However, the number of pyramid levels increases with the image size, which implies sophisticated data management and memory accesses, as well as additional computations. Here, we introduce a simplified formulation that reduces MSF to only a single level process. Starting from the MSF decomposition, we explain both mathematically and intuitively (visually) a way to simplify the classical MSF approach with minimal loss of information. The resulting single-scale fusion (SSF) solution is a close approximation of the MSF process that eliminates important redundant computations. It also provides insights regarding why MSF is so effective. While our simplified expression is derived in the context of high dynamic range imaging, we show its generality on several well-known fusion-based applications, such as image compositing, extended depth of field, medical imaging, and blending thermal (infrared) images with visible light. Besides visual validation, quantitative evaluations demonstrate that our SSF strategy is able to yield results that are highly competitive with traditional MSF approaches.
Multidimensional quasi-eigenfunction approximations and multicomponent AM-FM models

We develop multicomponent AM-FM models for multidimensional signals. The analysis is cast in a general n-dimensional framework where the component modulating functions are assumed to lie in certain Sobolev spaces. For both continuous and discrete linear shift invariant (LSI) systems with AM-FM inputs, powerful new approximations are introduced that provide closed form expressions for the responses in terms of the input modulations. The approximation errors are bounded by generalized energy variances quantifying the localization of the filter impulse response and by Sobolev norms quantifying the smoothness of the modulations. The approximations are then used to develop novel spatially localized demodulation algorithms that estimate the AM and FM functions for multiple signal components simultaneously from the channel responses of a multiband linear filterbank used to isolate components. Two discrete computational paradigms are presented. Dominant component analysis estimates the locally dominant modulations in a signal, which are useful in a variety of machine vision applications, while channelized components analysis delivers a true multidimensional multicomponent signal representation. We demonstrate the techniques on several images of general interest in practical applications, and obtain reconstructions that establish the validity of characterizing images of this type as sums of locally narrowband modulated components.
Blind Image Quality Assessment Without Human Training Using Latent Quality Factors

We propose a highly unsupervised, training free, no reference image quality assessment (IQA) model that is based on the hypothesis that distorted images have certain latent characteristics that differ from those of “natural” or “pristine” images. These latent characteristics are uncovered by applying a “topic model” to visual words extracted from an assortment of pristine and distorted images. For the latent characteristics to be discriminatory between pristine and distorted images, the choice of the visual words is important. We extract quality-aware visual words that are based on natural scene statistic features . We show that the similarity between the probability of occurrence of the different topics in an unseen image and the distribution of latent topics averaged over a large number of pristine natural images yields a quality measure. This measure correlates well with human difference mean opinion scores on the LIVE IQA database .
Saliency Prediction on Stereoscopic Videos

We describe a new 3D saliency prediction model that accounts for diverse low-level luminance, chrominance, motion, and depth attributes of 3D videos as well as high-level classifications of scenes by type. The model also accounts for perceptual factors, such as the nonuniform resolution of the human eye, stereoscopic limits imposed by Panum's fusional area, and the predicted degree of (dis) comfort felt, when viewing the 3D video. The high-level analysis involves classification of each 3D video scene by type with regard to estimated camera motion and the motions of objects in the videos. Decisions regarding the relative saliency of objects or regions are supported by data obtained through a series of eye-tracking experiments. The algorithm developed from the model elements operates by finding and segmenting salient 3D space-time regions in a video, then calculating the saliency strength of each segment using measured attributes of motion, disparity, texture, and the predicted degree of visual discomfort experienced. The saliency energy of both segmented objects and frames are weighted using models of human foveation and Panum's fusional area yielding a single predictor of 3D saliency.
Rate control for foveated MPEG/H.263 video

Given a set of target bits, video rate control algorithms that use Lagrange multipliers have been generally known as an optimal solution for maximizing the picture quality in the uniform spatial domain. Even if the SNR (signal-to-noise) of a picture is maximized by the rate control scheme, the visual quality can be enhanced using a suitable algorithm for the human visual system. We establish a new optimal rate control algorithm for maximizing the SNRC (signal-to-noise ratio in curvilinear coordinates) using the Lagrange multiplier. In addition, a target bit allocation technique for foveated video is introduced for simplified rate control over MEPG/H.263 video standards.