nils-herrmann/clean_scirepeval huggingface dataset:
Foveated video compression with optimal rate control

Previously, fovcated video compression algorithms have been proposed which, in certain applications, deliver high-quality video at reduced bit rates by seeking to match the nonuniform sampling of the human retina. We describe such a framework here where foveated video is created by a nonuniform filtering scheme that increases the compressibility of the video stream. We maximize a new foveal visual quality metric. the foveal signal-to-noise ratio (FSNR) to determine the best compression and rate control parameters for a given target bit rate. Specifically, we establish a new optimal rate control algorithm for maximizing the FSNR using a Lagrange multiplier method defined on a curvilinear coordinate system. For optimal rate control, we also develop a piecewise R-D (rate-distortion)/R-Q (rate-quantization) model. A fast algorithm for searching for an optimal Lagrange multiplier lambda* is subsequently presented. For the new models, we show how the reconstructed video quality is affected, where the FSNR is maximized, and demonstrate the coding performance for H.263,+,++/MPEG-4 video coding. For H.263/MPEG video coding, a suboptimal rate control algorithm is developed for fast, high-performance applications. In the simulations, we compare the reconstructed pictures obtained using optimal rate control methods for foveated and normal video. We show that foveated video coding using the suboptimal rate control algorithm delivers excellent performance under 64 kb/s.
Unequal Power Allocation for JPEG Transmission Over MIMO Systems

With the introduction of multiple transmit and re- ceiveantennas innext generationwireless systems, real-time image and video communication are expected to become quite common, since very high data rates will become available along with im- proveddatareliability.Newjointtransmissionandcodingschemes that explore advantages of multiple antenna systems matched with source statistics are expected to be developed. Based on this idea, we present an unequal power allocation scheme for transmission of JPEG compressed images over multiple-input multiple-output systems employing spatial multiplexing. The JPEG-compressed image is divided into different quality layers, and different layers are transmitted simultaneously from different transmit antennas using unequal transmit power, with a constraint on the total transmit power during any symbol period. Results show that our unequal power allocation scheme provides significant image quality improvement as compared to different equal power allo- cations schemes, with the peak-signal-to-noise-ratio gain as high as 14 dB at low signal-to-noise-ratios.
Surface Water Mapping by Deep Learning

Mapping of surface water is useful in a variety of remote sensing applications, such as estimating the availability of water, measuring its change in time, and predicting droughts and floods. Using the imagery acquired by currently active Landsat missions, a surface water map can be generated from any selected region as often as every 8 days. Traditional Landsat water indices require carefully selected threshold values that vary depending on the region being imaged and on the atmospheric conditions. They also suffer from many false positives, arising mainly from snow and ice, and from terrain and cloud shadows being mistaken for water. Systems that produce high-quality water maps usually rely on ancillary data and complex rule-based expert systems to overcome these problems. Here, we instead adopt a data-driven, deep-learning-based approach to surface water mapping. We propose a fully convolutional neural network that is trained to segment water on Landsat imagery. Our proposed model, named DeepWaterMap, learns the characteristics of water bodies from data drawn from across the globe. The trained model separates water from land, snow, ice, clouds, and shadows using only Landsat bands as input. Our code and trained models are publicly available at http://live.ece.utexas.edu/research/deepwatermap/.
Foveated video quality assessment

Most image and video compression algorithms that have been proposed to improve picture quality relative to compression efficiency have either been designed based on objective criteria such as signal-to-noise-ratio (SNR) or have been evaluated, post-design, against competing methods using an objective sample measure. However, existing quantitative design criteria and numerical measurements of image and video quality both fail to adequately capture those attributes deemed important by the human visual system, except, perhaps, at very low error rates. We present a framework for assessing the quality of and determining the efficiency of foveated and compressed images and video streams. Image foveation is a process of nonuniform sampling that accords with the acquisition of visual information at the human retina. Foveated image/video compression algorithms seek to exploit this reduction of sensed information by nonuniformly reducing the resolution of the visual data. We develop unique algorithms for assessing the quality of foveated image/video data using a model of human visual response. We demonstrate these concepts on foveated, compressed video streams using modified (foveated) versions of H.263 that are standard-compliant. We rind that quality vs. compression is enhanced considerably by the foveation approach.
Robust matrix factorization for collaborative filtering in recommender systems

Recently, matrix factorization has produced state-of-the-art results in recommender systems. However, given the typical sparsity of ratings, the often large problem scale, and the large number of free parameters that are often implied, developing robust and efficient models remains a challenge. Previous works rely on dense and/or sparse factor matrices to estimate unavailable user ratings. In this work we develop a new formulation for recommender systems that is based on projective non-negative matrix factorization, but relaxes the non-negativity constraint. Driven by a simple yet instructive intuition, the proposed formulation delivers promising and stable results that depend on a minimal number of parameters. Experiments that we conducted on two popular recommender system datasets demonstrate the efficiency and promise of our proposed method. We make available our code and datasets at https://github.com/christosbampis/PCMF_release.
Variational pattern analysis using Gabor wavelets

Measuring local frequencies is an effective approach to analyzing globally nonstationary, yet locally coherent images. The local frequencies may be manifested as, e.g., flowlike, granular, or oriented patterns. It is necessary to make spectral measurements accurate in both frequency and space, conflicting requirements constrained by a generalized uncertainty relationship. Such spectral measurements can be obtained from multiple wavelet-like channel filters that yield a locus of possible frequency solutions. Thus locus of solutions is maximally localized in both space/frequency if the channel filters are Gabor wavelets. A solution is obtained by imposing stabilizing terms developing naturally from assumptions on the signal. The frequency measurement problem is cast as a regularized extremum problem, leading to an iterative constraint propagation algorithm. As a to an iterative constraint propagation algorithm. As a byproduct local image contrast measurement is also obtained.<<ETX>>
Range segmentation using focus cues

A new scene segmentation scheme is proposed. A scene with different depth ranges (e.g., with multiple objects) is segmented into ranges. Focus cues are used to measure depth ranges and segmentation is performed using a multiresolution approach. To use focus cues in a multiresolution framework, we develop a criterion function and pyramid for focus measure. The segmentation algorithm first segments a scene at a coarse resolution and proceeds progressively to finer resolutions. This segmentation into ranges gives a description of the 3-dimensional structure of a scene. This segmentation method does not require any prior knowledge of depth ranges. The segmentation results for a scene with multiple objects with different depth ranges is presented.
Shape change analysis and shape modeling using three dimensional biomedical images

A method for computing material deformations from multidimensional images of a biological specimen undergoing shape changes is presented. The specimen data are segmented within the images and the domain of the specimen is parametrized. The parametrization is a material coordinate system for the specimen. Deformations of the material coordinate system are computed by minimizing an energy functional that is a linear combination of a brightness continuity constraint and a shape-change constraint based on differential geometric properties of the parametrization. The resulting parametrization completely describes the position and shape-change of the specimen for the entire image sequence. This approach was applied to images of a motile human white blood cell.<<ETX>>
Texas 3D Face Recognition Database

We make the Texas 3D Face Recognition Database available to researchers in three dimensional (3D) face recognition and other related areas. This database contains 1149 pairs of high resolution, pose normalized, preprocessed, and perfectly aligned color and range images of 118 adult human subjects acquired using a stereo camera. The images are accompanied with information about the subjects' gender, ethnicity, facial expression, and the locations of 25 manually located anthropometric facial fiducial points. Specific partitions of the data for developing and evaluating 3D face recognition algorithms are also included.
Determining the 3-D structure of serial-sectioned microscopic objects: analysis and limitations

The limited aperture of an optical system has the effect of zeroing a biconic portion of the 3-D spectrum of the specimen oriented along the optical axis; elsewhere, the spectrum is distorted by a low-pass function. The planar resolution of the serial sections can be improved by inverse filtering; however, the missing information cannot be recovered directly. In the absence of these effects, the sections can be processed to delineate significant changes in image intensity, or edges. The edges can be aggregated using morphology and other region correction procedures from which a 3-D representation of the specimen can be generated.<<ETX>>
Smoothing low-SNR molecular images via anisotropic median-diffusion

We propose a new anisotropic diffusion filter for denoising low-signal-to-noise molecular images. This filter, which incorporates a median filter into the diffusion steps, is called an anisotropic median-diffusion filter. This hybrid filter achieved much better noise suppression with minimum edge blurring compared with the original anisotropic diffusion filter when it was tested on an image created based on a molecular image model. The universal quality index, proposed in this paper to measure the effectiveness of denoising, suggests that the anisotropic median-diffusion filter can retain adherence to the original image intensities and contrasts better than other filters. In addition, the performance of the filter is less sensitive to the selection of the image gradient threshold during diffusion, thus making automatic image denoising easier than with the original anisotropic diffusion filter. The anisotropic median-diffusion filter also achieved good denoising results on a piecewise-smooth natural image and real Raman molecular images.
On the instantaneous frequencies of multicomponent AM-FM signals

We study the instantaneous frequencies (IFs) of multicomponent AM-FM signals by extending the work on the two-component case by Loughlin and Tacer (see ibid.,, vol. 4, no.5, p.123-25, 1997) to the more general M-component case. A novel necessary and sufficient condition for the valid interpretation of the IF as a nonnegatively weighted average of the IFs of the components is proposed, which we apply as a method of interpreting the IFs of signals having no more than three dominant components at each time. Our quantitative study shows that in general, the IF-based monocomponent AM-FM decomposition is not appropriate for the modeling and analysis of multicomponent AM-FM signals unless all the components are well separated in the time domain.
Optimizing Multiscale SSIM for Compression via MLDS

A crucial step in the assessment of an image compression method is the evaluation of the perceived quality of the compressed images. Typically, researchers ask observers to rate perceived image quality directly and use these rating measures, averaged across observers and images, to assess how image quality degrades with increasing compression. These ratings in turn are used to calibrate and compare image quality assessment algorithms intended to predict human perception of image degradation. There are several drawbacks to using such omnibus measures. First, the interpretation of the rating scale is subjective and may differ from one observer to the next. Second, it is easy to overlook compression artifacts that are only present in particular kinds of images. In this paper, we use a recently developed method for assessing perceived image quality, maximum likelihood difference scaling (MLDS), and use it to assess the performance of a widely-used image quality assessment algorithm, multiscale structural similarity (MS-SSIM). MLDS allows us to quantify supra-threshold perceptual differences between pairs of images and to examine how perceived image quality, estimated through MLDS, changes as the compression rate is increased. We apply the method to a wide range of images and also analyze results for specific images. This approach circumvents the limitations inherent in the use of rating methods, and allows us also to evaluate MS-SSIM for different classes of visual image. We show how the data collected by MLDS allow us to recalibrate MS-SSIM to improve its performance.
No-Reference Sharpness Assessment of Camera-Shaken Images by Analysis of Spectral Structure

The tremendous explosion of image-, video-, and audio-enabled mobile devices, such as tablets and smart-phones in recent years, has led to an associated dramatic increase in the volume of captured and distributed multimedia content. In particular, the number of digital photographs being captured annually is approaching 100 billion in just the U.S. These pictures are increasingly being acquired by inexperienced, casual users under highly diverse conditions leading to a plethora of distortions, including blur induced by camera shake. In order to be able to automatically detect, correct, or cull images impaired by shake-induced blur, it is necessary to develop distortion models specific to and suitable for assessing the sharpness of camera-shaken images. Toward this goal, we have developed a no-reference framework for automatically predicting the perceptual quality of camera-shaken images based on their spectral statistics. Two kinds of features are defined that capture blur induced by camera shake. One is a directional feature, which measures the variation of the image spectrum across orientations. The second feature captures the shape, area, and orientation of the spectral contours of camera shaken images. We demonstrate the performance of an algorithm derived from these features on new and existing databases of images distorted by camera shake.
Active segmentation of 3D axonal images

We present an active contour framework for segmenting neuronal axons on 3D confocal microscopy data. Our work is motivated by the need to conduct high throughput experiments involving microfluidic devices and femtosecond lasers to study the genetic mechanisms behind nerve regeneration and repair. While most of the applications for active contours have focused on segmenting closed regions in 2D medical and natural images, there haven't been many applications that have focused on segmenting open-ended curvilinear structures in 2D or higher dimensions. The active contour framework we present here ties together a well known 2D active contour model [5] along with the physics of projection imaging geometry to yield a segmented axon in 3D. Qualitative results illustrate the promise of our approach for segmenting neruonal axons on 3D confocal microscopy data.
Towards a Closed Form Second-Order Natural Scene Statistics Model

Previous work on natural scene statistics (NSS)-based image models has focused primarily on characterizing the univariate bandpass statistics of single pixels. These models have proven to be powerful tools driving a variety of computer vision and image/video processing applications, including depth estimation, image quality assessment, and image denoising, among others. Multivariate NSS models descriptive of the joint distributions of spatially separated bandpass image samples have, however, received relatively little attention. Here, we develop a closed form bivariate spatial correlation model of bandpass and normalized image samples that completes an existing 2D joint generalized Gaussian distribution model of adjacent bandpass pixels. Our model is built using a set of diverse, high-quality naturalistic photographs, and as a control, we study the model properties on white noise. We also study the way the model fits are affected when the images are modified by common distortions.
Automated facial feature detection and face recognition using Gabor features on range and portrait images

In this paper, we present a novel identity verification system based on Gabor features extracted from range (3D) representations of faces. Multiple landmarks (fiducials) on a face are automatically detected using these Gabor features. Once the landmarks are identified, the Gabor features on all fiducials of a face are concatenated to form a feature vector for that particular face. Linear discriminant analysis (LDA) is used to reduce the dimensionality of the feature vector while maximizing the discrimination power. These novel features were tested on 1196 range images. The same features were also extracted from portrait images, and the accuracies of both modalities were compared. A superior verification accuracy was obtained using the range data, and a highly competitive accuracy to that of other techniques in the literature was also obtained for the portrait data.
A two-stage framework for blind image quality assessment

Most present day no-reference/blind image quality assessment (NR IQA) algorithms are distortion specific - i.e., they assume that the distortion affecting the image is known. Here we propose a novel two stage framework for distortion-independent blind image quality assessment based on natural scene statistics (NSS). The proposed framework is modular in that it can be extended beyond the distortion-pool considered here, and each module proposed can be replaced by better-performing ones in the future. We describe a 4-distortion demonstration of the proposed framework and show that it performs competitively with the full-reference peak-signal-to-noise-ratio on the LIVE IQA database. A software release of the proposed index has been made available online: http://live.ece.utexas.edu/research/quality/BIQI_4D_release.zip.
An information fidelity criterion for image quality assessment using natural scene statistics

Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Traditionally, image QA algorithms interpret image quality as fidelity or similarity with a "reference" or "perfect" image in some perceptual space. Such "full-reference" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by arbitrary signal fidelity criteria. In this paper, we approach the problem of image QA by proposing a novel information fidelity criterion that is based on natural scene statistics. QA systems are invariably involved with judging the visual quality of images and videos that are meant for "human consumption". Researchers have developed sophisticated models to capture the statistics of natural signals, that is, pictures and videos of the visual environment. Using these statistical models in an information-theoretic setting, we derive a novel QA algorithm that provides clear advantages over the traditional approaches. In particular, it is parameterless and outperforms current methods in our testing. We validate the performance of our algorithm with an extensive subjective study involving 779 images. We also show that, although our approach distinctly departs from traditional HVS-based methods, it is functionally similar to them under certain conditions, yet it outperforms them due to improved modeling. The code and the data from the subjective study are available at .
No-Reference Image Quality Assessment in the Spatial Domain

We propose a natural scene statistic-based distortion-generic blind/no-reference (NR) image quality assessment (IQA) model that operates in the spatial domain. The new model, dubbed blind/referenceless image spatial quality evaluator (BRISQUE) does not compute distortion-specific features, such as ringing, blur, or blocking, but instead uses scene statistics of locally normalized luminance coefficients to quantify possible losses of “naturalness” in the image due to the presence of distortions, thereby leading to a holistic measure of quality. The underlying features used derive from the empirical distribution of locally normalized luminances and products of locally normalized luminances under a spatial natural scene statistic model. No transformation to another coordinate frame (DCT, wavelet, etc.) is required, distinguishing it from prior NR IQA approaches. Despite its simplicity, we are able to show that BRISQUE is statistically better than the full-reference peak signal-to-noise ratio and the structural similarity index, and is highly competitive with respect to all present-day distortion-generic NR IQA algorithms. BRISQUE has very low computational complexity, making it well suited for real time applications. BRISQUE features may be used for distortion-identification as well. To illustrate a new practical application of BRISQUE, we describe how a nonblind image denoising algorithm can be augmented with BRISQUE in order to perform blind image denoising. Results show that BRISQUE augmentation leads to performance improvements over state-of-the-art methods. A software release of BRISQUE is available online: http://live.ece.utexas.edu/research/quality/BRISQUE_release.zip for public use and evaluation.
Melanoma Classification on Dermoscopy Images Using a Neural Network Ensemble Model

We develop a novel method for classifying melanocytic tumors as benign or malignant by the analysis of digital dermoscopy images. The algorithm follows three steps: first, lesions are extracted using a self-generating neural network (SGNN); second, features descriptive of tumor color, texture and border are extracted; and third, lesion objects are classified using a classifier based on a neural network ensemble model. In clinical situations, lesions occur that are too large to be entirely contained within the dermoscopy image. To deal with this difficult presentation, new border features are proposed, which are able to effectively characterize border irregularities on both complete lesions and incomplete lesions. In our model, a network ensemble classifier is designed that combines back propagation (BP) neural networks with fuzzy neural networks to achieve improved performance. Experiments are carried out on two diverse dermoscopy databases that include images of both the xanthous and caucasian races. The results show that classification accuracy is greatly enhanced by the use of the new border features and the proposed classifier model.