nils-herrmann/clean_scirepeval huggingface dataset:
Local Image Analysis By Gabor Wavelets

Considerable recent attention has focused on multi-scale signal processing paradigms that entail decompositions into elementary signals, or wavelets, that embody local spatial and/or temporal structures. While much research has been dedicated towards understanding the mathematical properties of these representations and methods for computing/inverting them, considerable effort is also being applied to exploiting them in a wide variety of problems, e.g., in the analysis and coding of two-dimensional images. spectrally localized basis. Measuring locally-occurring signal frequencies is focal in many applications, particularly those involving nonstationary coherent signals, where a significant fraction of the local signal spectrum is concentrated over a narrow frequency range varying smoothly over space. The locally emergent frequencies characterizing such signals are manifested as, e.g., flow-like, granular, or oriented patterns in optical images, pure tonal components in music or whale song, and quasi-regular structures in three-dimensional micrographs of optical density. Measuring emergent image frequencies requires spectral measurements accurate in both frequency and space, conflicting requirements balanced by a generalized uncertainty relationship. Such spectral measurements can be obtained from the responses of multiple wavelet-like channel filters that sample the image spectrum, and that yield a locus of possible solutions for each locally emergent frequency. It is shown analytically that this locus of solutions is maximally localized in both space and frequency, in the sense of a generalized p-uncertainty, when the channel filters are Gabor wavelets. A constrained solution is obtained by imposing a stabilizing term which develops naturally from the image model. The measurement of frequencies is then cast as an ill-posed extremum problem regularized by the stabilizing term, leading to an iterative constraint propagation algorithm. The efficacy of the technique is demonstrated by application to a variety of image processing tasks, including image texture segmentation, flow computation, local measurement of fractal image dimension, and the measurement of three-dimensional object shape from projected surface texture. We consider the restricted problem of measuring image frequencies on a spatio-
Local bandwidth constrained fast inverse motion compensation for DCT-domain video transcoding

Discrete cosine transform (DCT) based digital video coding standards, such as MPEG and H.26x, are becoming more widely adopted for multimedia applications. Since the standards differ in their format and syntax, video transcoding, where a compressed video bit-stream is converted from one format to another, is of interest for purposes such as channel bandwidth adaptation and video composition. DCT-domain video transcoding is generally more efficient than spatial-domain transcoding. However, since the data is organized block by block in the DCT domain, inverse motion compensation becomes the bottleneck for DCT-domain methods. We propose a novel local bandwidth constrained fast inverse motion compensation algorithm operating in the DCT domain. Relative to Chang's algorithm, we achieve computational improvement of 25%-55% without visual degradation. A by-product of our algorithm is a reduction of blocking artifacts in very low bit-rate compressed video sequences. The proposed algorithm can be combined with other reported fast methods for more computational savings. We also present a look-up-table (LUT) based implementation method by modeling the statistical distribution of the DCT coefficients in natural images and video sequences. By this method, we obtain a further 31%-48% improvement in computation. The memory requirement of the LUT is about 800 kB, which is reasonable. Moreover, the LUT can be shared by multiple DCT-domain video processing applications running on the same computer or video server.
Natural scene statistics of color and range

Color and depth play important roles in natural scenes and in vision, and their perception is related. Extensive work has been conducted on studying the luminance statistics of natural scenes; however, there is very little work done on analyzing the statistics between luminance and range in natural scenes, not to mention color and range. In this paper, we present the LIVE Color+3D Database, which contains 12 sets of color images with corresponding ground truth range maps in a high-definition resolution of 1280×720. We examined the statistical distribution of range gradients conditioned on the Gabor responses of the color images, as well as the variations of statistical measures of range gradients with changes in the Gabor responses. The analysis results show that the distributions of range gradients conditioned on the Gabor responses have very similar exponential shapes for both luminance and chrominance channels. Moreover, we also found that the depth difference between neighboring pixels increases as the corresponding magnitudes of the Gabor responses rise.
Low delay foveated visual communications over wireless channels

The great potential of "foveated imaging" lies in the entropy reduction relative to the original image while minimizing the loss of visual information. Utilizing human foveation combined with video compression, as well as communication and human-machine interface techniques, more efficient multimedia services are expected to be provided in the near future. In this paper, we introduce a prototype for foveated visual communications as one of future human interactive multimedia applications, and demonstrate the benefit of the foveation over fading statistics in the downtown area of Austin, Texas. In order to compare the performance with regular video, we use spatial/temporal resolution and source transmission delay as the evaluation criteria.
COPERM: transform-domain energy compaction by optimal permutation

Compaction by optimal permutation (COPERM) is a tool for transform domain energy compaction of broadband signals, whose foundation is a simple but powerful idea: any signal can be transformed to resemble a more desirable (e.g., from a transform-domain compaction viewpoint) signal from a class of "target" signals (e.g., DCT basis functions) by means of a suitable permutation of its samples. One application of transform-domain energy compaction is in lossy compression. We pursue one possible thread in detail and demonstrate some interesting broadband image compression results.
Learning a Continuous-Time Streaming Video QoE Model

Over-the-top adaptive video streaming services are frequently impacted by fluctuating network conditions that can lead to rebuffering events (stalling events) and sudden bitrate changes. These events visually impact video consumers’ quality of experience (QoE) and can lead to consumer churn. The development of models that can accurately predict viewers’ instantaneous subjective QoE under such volatile network conditions could potentially enable the more efficient design of quality-control protocols for media-driven services, such as YouTube, Amazon, Netflix, and so on. However, most existing models only predict a single overall QoE score on a given video and are based on simple global video features, without accounting for relevant aspects of human perception and behavior. We have created a QoE evaluator, called the time-varying QoE Indexer, that accounts for interactions between stalling events, analyzes the spatial and temporal content of a video, predicts the perceptual video quality, models the state of the client-side data buffer, and consequently predicts continuous-time quality scores that agree quite well with human opinion scores. The new QoE predictor also embeds the impact of relevant human cognitive factors, such as memory and recency, and their complex interactions with the video content being viewed. We evaluated the proposed model on three different video databases and attained standout QoE prediction performance.
Statistical model of color and disparity with application to Bayesian stereopsis

Extensive research has been conducted relating the natural scene statistics of luminance and depth; however, very little work has been done on analyzing the statistical relationships between depth and chromatic information. In this paper, we examine and derive statistical models between disparity and both luminance and chrominance information by transforming natural images into the more perceptually relevant CIELAB color space. To demonstrate the effectiveness of these models, we further exploit them with application to Bayesian stereo algorithms. The simulation results show that incorporating the derived statistical models augments the performance of Bayesian stereo algorithms. In addition, these results also support psychophysical evidence that chromatic information can improve binocular visual processing.
Properties of multichannel texture analysis filters

Texture analysis algorithms that decompose images into oriented spatial frequency channels are studied. Optimality properties for texture segmentation filters are considered using idealized (narrowband) image texture models. The functional uncertainty of the channel filters is shown to define a tradeoff between spectral selectivity and accuracy in boundary localization that is optimized by the 2-D Gabor functions. The idealized texture model is then relaxed to analyze the effects of textural perturbations interpreted as localized amplitude and phase variations on the segmentation. The effects of these perturbations are found to be effectively ameliorated with postdetection smoothing.<<ETX>>
SpEED-QA: Spatial Efficient Entropic Differencing for Image and Video Quality

Many image and video quality assessment (I/VQA) models rely on data transformations of image/video frames, which increases their programming and computational complexity. By comparison, some of the most popular I/VQA models deploy simple spatial bandpass operations at a couple of scales, making them attractive for efficient implementation. Here we design reduced-reference image and video quality models of this type that are derived from the high-performance reduced reference entropic differencing (RRED) I/VQA models. A new family of I/VQA models, which we call the spatial efficient entropic differencing for quality assessment (SpEED-QA) model, relies on local spatial operations on image frames and frame differences to compute perceptually relevant image/video quality features in an efficient way. Software for SpEED-QA is available at: http://live.ece.utexas.edu/research/Quality/SpEED_Demo.zip.
Generalizing a closed-form correlation model of oriented bandpass natural images

Building natural scene statistic models is a potentially transformative development for a wide variety of visual applications, ranging from the design of faithful image and video quality models to the development of perceptually optimized image enhancing techniques. Most predominant statistical models of natural images only characterize the univariate distributions of divisively normalized bandpass image responses. Previous efforts towards modeling bandpass natural responses have not focused on finding closed-form quantative models of bivariate natural statistics. Towards filling this gap, Su et al.  recently modeled spatially adjacent bandpass image responses over multiple scales; however, they did not consider the effects of spatial distance between the bandpass samples. Here we build on Su et al.'s model and extend their closed-form correlation model to non-adjacent distant bandpass image responses over multiple spatial orientations and scales.
Blind quality assessment of JPEG2000 compressed images using natural scene statistics

Measurement of image quality is crucial for many image-processing algorithms, such as acquisition, compression, restoration, enhancement and reproduction. Traditionally, researchers in image quality assessment have focused on equating image quality with similarity to a 'reference' or 'perfect' image. The field of blind, or no-reference, quality assessment, in which image quality is predicted without the reference image, has been largely unexplored. In this paper, we present a blind quality assessment algorithm for images compressed by JPEG2000 using natural scene statistics (NSS) modelling. We show how reasonably comprehensive NSS models can help us in making blind, but accurate, predictions of quality. Our algorithm performs close to the limit imposed on useful prediction by the variability between human subjects.
Sampled efficient full-reference image quality assessment models

Existing Ml-reference image quality assessment models first compute a full image quality-predictive feature map followed by a spatial pooling scheme, thereby producing a single quality score. Here we study spatial sampling strategies that can be used to more efficiently compute reliable picture quality scores. We develop a random sampling scheme on single scale full-reference image quality assessment models. Based on a thorough analysis of how this random sampling strategy affects the correlations of the resulting pooled scores against human subjective quality judgements, a highly efficient grid sampling scheme is proposed which replaces the ubiquitous convolution operations with local block-based multiplications. Experiments on four different databases show that this block-based sampling strategy can yield results similar to methods that use a complete image feature map, even when the number of feature samples is reduced by 90%.
Semi-automatic morphological measurements of 2-D and 3-D microvascular images

The technique of serial optical sectioning by laser scanning confocal microscopy (LSCM), in conjunction with off-line digital image analysis, was used to quantize the morphological changes occurring during angiogenesis and revascularization of pancreatic islets transplanted at the renal subcapsular site in rats. The process consisted of in-vivo imaging of the microvasculature which was optically enhanced by the administration of a fluorescent probe into the circulating blood. Serial two-dimensional (2-D) optical sections were obtained through the vascular bed at varying z-intervals in order to perform a computer reconstruction of the complete three-dimensional (3-D) morphology. Image processing algorithms such as gray level thresholding, median filtering, skeletonization, region labeling and template matching were applied to compute the vessel density, lengths and diameters of the neovasculature, and the tortuosity index.<<ETX>>
Perceptually Scalable Extension of H.264

We propose a novel visual scalable video coding (VSVC) framework, named VSVC H.264/AVC. In this approach, the non-uniform sampling characteristic of the human eye is used to modify scalable video coding (SVC) H.264/AVC. We exploit the visibility of video content and the scalability of the video codec to achieve optimal subjective visual quality given limited system resources. To achieve the largest coding gain with controlled perceptual quality degradation, a perceptual weighting scheme is deployed wherein the compressed video is weighted as a function of visual saliency and of the non-uniform distribution of retinal photoreceptors. We develop a resource allocation algorithm emphasizing both efficiency and fairness by controlling the size of the salient region in each quality layer. Efficiency is emphasized on the low quality layer of the SVC. The bits saved by eliminating perceptual redundancy in regions of low interest are allocated to lower block-level distortions in salient regions. Fairness is enforced on the higher quality layers by enlarging the size of the salient regions. The simulation results show that the proposed VSVC framework significantly improves the subjective visual quality of compressed videos.
Analysis of multichannel narrow-band filters for image texture segmentation

A model for texture analysis and segmentation using multiple oriented channel filters is analyzed in the general framework. Several different arguments are applied leading to the conclusion that the two-dimensional Gabor filters possess strong optimality properties for this task. Properties of the multiple-channel segmentation approach are analyzed. In particular, perturbations of textures from an ideal model are found to have important effects on the segmentation that can usually be ameliorated by simply preceding the segmentation process by a logarithmic operation and using a low-pass postfilter prior to making region assignments. The difficult problems of space-variant textures and multiple component textures are also considered. Local spatial frequency estimation approaches are suggested that use the responses as constraints in estimating the locally emergent texture frequencies. Complex texture aggregates containing multiple shared frequency components can be analyzed if the textures are distinct and few in number.
Predicting the Quality of Fused Long Wave Infrared and Visible Light Images

The capability to automatically evaluate the quality of long wave infrared (LWIR) and visible light images has the potential to play an important role in determining and controlling the quality of a resulting fused LWIR-visible light image. Extensive work has been conducted on studying the statistics of natural LWIR and visible images. Nonetheless, there has been little work done on analyzing the statistics of fused LWIR and visible images and associated distortions. In this paper, we analyze five multi-resolution-based image fusion methods in regards to several common distortions, including blur, white noise, JPEG compression, and non-uniformity. We study the natural scene statistics of fused images and how they are affected by these kinds of distortions. Furthermore, we conducted a human study on the subjective quality of pristine and degraded fused LWIR-visible images. We used this new database to create an automatic opinion-distortion-unaware fused image quality model and analyzer algorithm. In the human study, 27 subjects evaluated 750 images over five sessions each. We also propose an opinion-aware fused image quality analyzer, whose relative predictions with respect to other state-of-the-art models correlate better with human perceptual evaluations than competing methods. An implementation of the proposed fused image quality measures can be found at https://github.com/ujemd/NSS-of-LWIR-and-Vissible-Images. Also, the new database can be found at
Natural DCT statistics approach to no-reference image quality assessment

General-purpose no-reference image quality assessment approaches still lag the advances in full-reference methods. Most no-reference methods are either distortion specific (i.e. they quantify one or more distortions such as blur, blockiness, or ringing), or they train a learning machine based on a large number of features. In this approach, we propose a discrete cosine transform (DCT) statistics-based support vector machine (SVM) approach based on only 3 features in the DCT domain. The approach extracts a very small number of features and is entirely in the DCT domain, making it computationally convenient. The results are shown to correlate highly with human visual perception of quality.
Streaking in median filtered images

This paper presents a probabilistic analysis of the streaking or blotching effect commonly observed in median filtered signals in both one and two dimensions. The effcts are identified as runs of equal or nearly equal values which create visual impressions that have no visual correlate. For one-dimensional discrete iid random signals with continuous input probability densities, the probability of a streak of length L occurring is computed and shown to be independent of the input probability distribution. Expressions for the first and second moments of the streak length are also derived, and certain asymptotic results are given. As the analysis and definition of the analogous effect in two dimensions is less tractable, the probability that medians taken from distinct overlapping windows will take the same value is derived for various filter geometries. The analytic results are supported by examples using both one- and two-dimensional signals.
Fixation selection by maximization of texure and contrast information

We present information-theoretic underpinnings of a computation theory of low-level visual fixations in natural images. In continuation of our prior work on optimal contrast-based fixations , we develop an optimum texture- based fixation selection algorithm based on a recent theory of non-stationarity measurement in natural images . Thereafter we propose a simple coupling of the optimal texture-based and contrast-based fixation features to produce a new algorithm called CONTEXT, which exhibits robust performance for fixation selection in natural images. The performance of the fixation algorithms are evaluated for natural images by comparison to randomized fixation strategies via actual human fixations performed on the images. The fixation patterns obtained outperform randomized, GAFFE-based [3], and Itti  fixation strategies in terms of matching human fixation patterns. These results also demonstrate the important role that contrast and textural information play in low-level visual processes in the Human Visual System (HVS).
RRED Indices: Reduced Reference Entropic Differencing for Image Quality Assessment

We study the problem of automatic “reduced-reference” image quality assessment (QA) algorithms from the point of view of image information change. Such changes are measured between the reference- and natural-image approximations of the distorted image. Algorithms that measure differences between the entropies of wavelet coefficients of reference and distorted images, as perceived by humans, are designed. The algorithms differ in the data on which the entropy difference is calculated and on the amount of information from the reference that is required for quality computation, ranging from almost full information to almost no information from the reference. A special case of these is algorithms that require just a single number from the reference for QA. The algorithms are shown to correlate very well with subjective quality scores, as demonstrated on the Laboratory for Image and Video Engineering Image Quality Assessment Database and the Tampere Image Database. Performance degradation, as the amount of information is reduced, is also studied.
Multi-user real-time wireless video with perceptual constraints

The current military and civil ground based surveillance and reconnaissance is characterized by a large number of sensor assets, feeding resource constrained analysis nodes, and constrained by extremely limited transport bandwidth. The complexity of such systems is captured by the Networks of Systems (NoS) concept which use tools from wireless engineering and control theory. In this paper, the role of perceptual quality constraints on video-based NoS is investigated. An algorithm to determine the feasible set of nodes and optimal packet schedule via a Markov decision process, given their perceptual constraints and channel qualities is proposed. The H.264/SVC codec is used for compression, while the visual information fidelity pixel-domain (VIFP) full-reference image quality assessment model is used to measure perceptual quality. The algorithm is implemented in custom C;; software with over-the-air transmissions using 802.11 WLAN hardware. Real-time encoding is done for over-the-air transmission of the video. The results indicate that gains of 10-30% can be realized for a three node system.