nils-herrmann/clean_scirepeval huggingface dataset:
Hierarchical visual pattern image coding

Visual pattern image coding (VPIC) is a flexible image coding device offering excellent visual fidelity at high compression ratios with unprecedented, extremely low coding complexity. Hierarchical visual pattern image coding (HVPIC) schemes which achieve significantly increased compression (approaching 35:1) with improved fidelity and the very low complexity associated with VPIC are described. HVPIC employs standard VPIC embedded in a multiresolution (pyramid) structure. Low-resolution images coded at the upper levels augment coding of the higher-resolution images. Interactions between resolution levels is simple and efficient, yielding little increase in complexity relative to VPIC.<<ETX>>
A Completely Blind Video Integrity Oracle

Considerable progress has been made toward developing still picture perceptual quality analyzers that do not require any reference picture and that are not trained on human opinion scores of distorted images. However, there do not yet exist any such completely blind video quality assessment (VQA) models. Here, we attempt to bridge this gap by developing a new VQA model called the video intrinsic integrity and distortion evaluation oracle (VIIDEO). The new model does not require the use of any additional information other than the video being quality evaluated. VIIDEO embodies models of intrinsic statistical regularities that are observed in natural vidoes, which are used to quantify disturbances introduced due to distortions. An algorithm derived from the VIIDEO model is thereby able to predict the quality of distorted videos without any external knowledge about the pristine source, anticipated distortions, or human judgments of video quality. Even with such a paucity of information, we are able to show that the VIIDEO algorithm performs much better than the legacy full reference quality measure MSE on the LIVE VQA database and delivers performance comparable with a leading human judgment trained blind VQA model. We believe that the VIIDEO algorithm is a significant step toward making real-time monitoring of completely blind video quality possible.
Multiscale stereopsis via Gabor filter phase response

The most difficult task in stereopsis is to resolve the correspondence problem from a pair of stereo 2-D images. By observing that the perspective projections of 3-D objects can be described as a phase shift issue, we solve the correspondence problem by employing the phase response via multichannel Gabor filters as matching primitive. Our stereo algorithm is biologically motivated and based on a multiscale, coarse-to-fine strategy. This approach produces a dense disparity map without complex feature extraction and interpolation. Results of the algorithm for both synthetic and natural stereo images are presented.<<ETX>>
Fast computation of dense stereo correspondence by stochastic sampling of match quality

We present a method for computing dense stereo correspondences in calibrated monocular video by iteratively and stochastically sampling match quality values in the disparity search space. Most existing methods exhaustively compute local correspondence quality before searching for a globally optimal solution. Instead, we iteratively refine a correspondence estimate by perturbing it with random noise and formulating an influence at each sample based on the perturbation and its effect on correspondence match quality. Local influence is aggregated to recover consistent trends in match quality caused by the piecewise-continuous structure of the scene. Correspondence estimates for a given frame pair are seeded with the estimates from the previous frame pair, allowing convergence to occur across multiple frame pairs.
Range image quality assessment by Structural Similarity

We propose a new quality metric for range images that is based on the multi-scale Structural Similarity (MS-SSIM) Index. The new metric operates in a manner to SSIM but allows for special handling of missing data. We demonstrate its utility by reevaluating the set of stereo algorithms evaluated in the Middlebury Stereo Vision Page http://vision.middlebury.edu/stereo/. The new algorithm which we term Range SSIM (R-SSIM) Index possesses features that make it an attractive choice for assessing the quality of range images.
Objective quality assessment of multiply distorted images

Subjective studies have been conducted in the past to obtain human judgments of visual quality on distorted images in order, among other things, to benchmark objective image quality assessment (IQA) algorithms. Existing subjective studies primarily have records of human ratings on images that were corrupted by only one of many possible distortions. However, the majority of images that are available for consumption are corrupted by multiple distortions. Towards broadening the corpora of records of human responses to visual distortions, we recently conducted a study on two types of multiply distorted images to obtain human judgments of the visual quality of such images. Further, we compared the performance of several existing objective image quality measures on the new database and analyze the effects of multiple distortions on commonly used quality-determinant features and on human ratings.
Optimizing 3D image display using the stereoacuity function

We develop an algorithm that predicts the best presentation of a stereo 3D image in the sense of viewers' preference. The algorithm operates in three steps. First, the 3D image is classified as either a “foreground dominant” or “background dominant” image. Next, for “foreground dominant” images, a model of the stereoacuity function is used to optimize the perceptual 3D resolution; for “background dominant” images, the nearest surface is placed in the 3D plane of the display screen. A human study was conducted to assess the algorithm and showed that the proposed model produced 3D images which had the best 3D quality scores among several candidate algorithms.
Maximizing image quality over Visual Sensor Networks via DCT bit allocation

Recently, VSNs (Visual Sensor Networks) are becoming of increased interest in a variety of applications. Due to power constraints at each visual sensor node, the processing of live video is constrained. In this paper, we propose a DCT (Discrete Cosine Transform)-based bit allocation scheme to maximize image quality for a given bit rate constraint. Two major featured algorithms are proposed : “SLO” (Searching from the Lowest Order) and “SPP” (Searching the Present optimal bit allocation from the Previous optimal bit allocation). The proposed algorithms reduce the search range based on a simple IQA (Image Quality Assessment) model and DCT statistics. Thus, they demonstrate low-complexity and fast computation. In the simulations, we show the superiority of the proposed algorithms over conventional approaches.
Computer lipreading for improved accuracy in automatic speech recognition

Among the various methods that have been proposed to improve the robustness and accuracy of automatic speech recognition (ASR) systems, lipreading has received little attention until very recently. However, results from the psychological literature indicate that lipreading, in conjunction with auditory perception, can provide a strong improvement in speech recognition and understanding in humans. We have developed a novel speaker-dependent lipreading system that uses hidden Markov models. An audiovisual system known as Lipreading to Enhance Automatic Perception of Speech (LEAPS) is described, in which the lipreading system is used in conjunction with an audio ASR system in order to improve the accuracy of the latter, especially under degraded acoustical conditions. Experimental results are presented for two small phoneme discrimination tasks, as well as a medium vocabulary isolated word recognition task. In all cases, performance of the combined system is superior to that of the audio system, with a reduction in errors ranging from 20 to 65%.
On the statistical optimality of locally monotonic regression

Locally monotonic regression is a recently proposed technique for the deterministic smoothing of finite-length discrete signals under the smoothing criterion of local monotonicity. Locally monotonic regression falls within a general framework for the processing of signals that may be characterized in three ways: regressions are given by projections that are determined by semimetrics, the processed signals meet shape constraints that are defined at the local level, and the projections are optimal statistical estimates in the maximum likelihood sense. the authors explore the relationship between the geometric and deterministic concept of projection onto (generally nonconvex) sets and the statistical concept of likelihood, with the object of characterizing projections under the family of the p-semi-metrics as maximum likelihood estimates of signals contaminated with noise from a well-known family of exponential densities.
Skewed 2D Hilbert transforms and computed AM-FM models

Computed AM-FM models represent images in terms of instantaneous amplitude and frequency modulations. However, the instantaneous amplitude and frequency of a real valued image are ambiguous. We apply the directional 2D Hilbert transform to compute a complex extension for a real image. This extension, called the analytic image, admits most of the attractive properties of the 1D analytic signal. However, the analytic image is not unique: for a given real image, taking the Hilbert transform in the horizontal and vertical directions yields different complex extensions and differing computed AM-FM models. We show that these two differing models are essentially equivalent and develop explicit formulations relating them.
Adaptive variable baseline stereo for vergence control

Active control of camera positioning is an important tool in the recovery of scene information. In this paper an important innovation is introduced which uses the local spatial frequency content of the acquired images to adaptively control stereo baseline, improving both the reliability and speed of the depth recovery process. This technique was designed to support the operation of an active vergent stereo imaging system by providing a coarse map of depth for camera fixation, providing an initial map of surface depth for accurate vergence.
Optimal order statistic filters with coefficient censoring

Order statistic (OS) filters are noted for their robust properties for smoothing or estimating signals immersed in noise. The effect of coefficient censoring on the smoothing performance of OS filters is considered. Coefficient censoring can be effected either by adding zero coefficients to an existing filter, or by computing the optimal (MSE) censored coefficients for a set filter span. The second possibility is considered. In both cases, censoring can yield increased robustness and improved edge retention. OS filters in common use which can be interpreted as censored OS filters include the median filter, rank-order filters, and the trimmed mean filters. While optimal censored filters are generally suboptimal relative to optimal uncensored filters, they often yield superior performance for signals containing edge discontinuities.<<ETX>>
Foveated Analysis and Selection of Visual Fixations in Natural Scenes

The ability to automatically detect visually interesting regions in images has practical applications in the design of active machine vision systems. Analysis of the statistics of image features at observers gaze can provide insights into the mechanisms of fixation selection in humans. Using a novel foveated analysis framework, in which features were analyzed at the spatial resolution at which they were perceived, we studied the statistics of four low-level local image features: luminance, contrast, center-surround outputs of luminance and contrast, and discovered that the image patches around human fixations had, on average, higher values of each of these features than the image patches selected at random. Center-surround contrast showed the greatest difference between human and random fixations, followed by contrast, center-surround luminance, and luminance. Using these measurements, we present a new algorithm that selects image regions as likely candidates for fixation. These regions are shown to correlate well with fixations recorded from observers.
Epipolar Spaces for Active Binocular Vision Systems

Depth recovery for active binocular vision systems is simplified if the camera geometry is known and corresponding points can be restricted to epipolar lines. Unfortunately, computation of epipolar lines requires calibration which can be complex and inaccurate. While it is possible to register images without geometric information, such unconstrained algorithms are usually time consuming and prone to error. In this paper we propose a compromise. Even without the instantaneous knowledge of the system geometry, we can restrict the region of correspondence by imposing limits on the possible range of configurations, and as a result, confine our search for matching points to epipolar spaces. For each point in one image, we define the corresponding epipolar space in the other image as the union of all associated epipolar lines over all possible system geometries. Epipolar spaces eliminate the need for calibration at the cost of an increased search region.
Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index

It is an important task to faithfully evaluate the perceptual quality of output images in many applications, such as image compression, image restoration, and multimedia streaming. A good image quality assessment (IQA) model should not only deliver high quality prediction accuracy, but also be computationally efficient. The efficiency of IQA metrics is becoming particularly important due to the increasing proliferation of high-volume visual data in high-speed networks. We present a new effective and efficient IQA model, called gradient magnitude similarity deviation (GMSD). The image gradients are sensitive to image distortions, while different local structures in a distorted image suffer different degrees of degradations. This motivates us to explore the use of global variation of gradient based local quality map for overall image quality prediction. We find that the pixel-wise gradient magnitude similarity (GMS) between the reference and distorted images combined with a novel pooling strategy-the standard deviation of the GMS map-can predict accurately perceptual image quality. The resulting GMSD algorithm is much faster than most state-of-the-art IQA methods, and delivers highly competitive prediction accuracy. MATLAB source code of GMSD can be downloaded at http://www4.comp.polyu.edu.hk/~cslzhang/IQA/GMSD/GMSD.htm.
Local bandwidth constrained fast inverse motion compensation for DCT-domain video transcoding

DCT-based digital video coding standards such as MPEG and H.26x are becoming more widely adopted for multimedia applications. Since the standards differ in their format and syntax, video transcoding, where a pre-coded video bit-stream is converted from one format to another format, is of interest for purposes such as channel bandwidth adaptation and video composition. DCT-domain video transcoding is generally more efficient than spatial domain transcoding. However, since the data is organized block by block in the DCT-domain, inverse motion compensation becomes the bottleneck for DCT-domain methods. We propose a novel local bandwidth constrained fast inverse motion compensation algorithm operating in the DCT-domain. Relative to Chang's (1995)algorithm, the proposed algorithm achieves computational improvement of 25% to 55% without visual degradation. A by-product of the proposed algorithm is a reduction of blocking artifacts in very low bit-rate compressed video sequences.
A joint source-channel distortion model for JPEG compressed images

The need for efficient joint source-channel coding is growing as new multimedia services are introduced in commercial wireless communication systems. An important component of practical joint source-channel coding schemes is a distortion model to measure the quality of compressed digital multimedia such as images and videos. Unfortunately, models for estimating the distortion due to quantization and channel bit errors in a combined fashion do not appear to be available for practical image or video coding standards. This paper presents a statistical model for estimating the distortion introduced in progressive JPEG compressed images due to both quantization and channel bit errors. Important compression techniques such as Huffman coding, DPCM coding, and run-length coding are included in the model. Examples show that the distortion in terms of peak signal to noise ratio can be predicted within a 2 dB maximum error.
COPERM: transform-domain energy compaction by optimal permutation

COPERM is a novel paradigm for energy compaction and signal compression, whose foundation is a simple but powerful idea: any signal can be transformed to resemble a more desirable signal from a class of "target" signals, by means of a suitable permutation of its samples. The approach is well-suited for transform domain energy compaction prior to transform-domain compression of persistent broadband signals. The associated optimal permutation precoders are surprisingly simple, and the permutation precoding overhead can be made modest-resulting in improved overall rate-distortion performance.
Relating spatial and spectral models of oriented bandpass natural images

Univariate models of the Natural Scene Statistics (NSS) of perceived digital pictures have been deployed in a wide variety of image and video processing applications. However, much less effort has been made towards understanding, modeling, and using bivariate image NSS. Towards filling this gap, Su et al. developed a closed form correlation model of oriented bandpass natural images applicable to adjacent pixels. We later extended this model to account for pixels separated by larger spatial distance. Here, we expand our previous work further to model the bivariate responses of bandpass spatial filters covering a wider range of bandwidths. Furthermore, we study the relationship between the parameters of the closed form correlation model and image spectral models.
Statistical modeling of multi-camera images

This paper aims at developing mutual statistical descriptions of images in a visual sensor network. A two-dimensional Gaussian scale mixture (GSM) model is proposed to describe the joint distribution of wavelet coefficients in the same subband of two different images. This model is tested on two image databases, and it is shown that the proposed model always achieves a smaller divergence from the empirical distribution than a (jointly) Gaussian model. The GSM model is then extended to blocks of wavelet coefficients in the same subband and scale of non-overlapping images. A new denoising algorithm is devised that uses this model and achieves better performance than denoising based on marginal models.
Look-up-table based DCT domain inverse motion compensation

DCT-based digital video coding standards such as MPEG and H.26x have been widely adopted for multimedia applications. Thus video processing in the DCT domain usually proves to be more efficient than in the spatial domain. To directly convert an inter-coded frame into an intra-coded frame in the DCT domain, the problem of DCT domain inverse motion compensation was studied by Chang and Messerschmitt(1995). Since the data is organized block by block in the DCT domain, the DCT domain inverse motion compensation is computationally intensive. In this paper, a look-up-table (LUT) based method for DCT domain inverse motion compensation is proposed by modeling the statistical distribution of the DCT coefficients in typical images and video sequences. Compared to the method of Chang et al., the LUT based method can save more than 50% of the computing time based on experimental results. The memory requirement of the LUT is about 800 KB which is reasonable. Moreover, the LUT can be shared by multiple DCT domain video processing applications running on the same computer.
A Pervasive Network Control Algorithm for Multicamera Networks

Owing to the increasingly large volume and complexity of captured videos, renewable energy systems based on solar energy are of particular interest in the design of energy harvesting (EH) wireless visual sensor networks (WVSNs). Since additional energy consumption following image capture occurs owing to image processing, mote operation, data transmission, and reception, the capture rate significantly affects the lifetime of a node. To this end, we explore a novel energy-efficient framework for EH-WVSN design by developing an optimal algorithm named capture rate and pervasive network control for multicamera networks where the quality of service is maximized by obtaining optimal values for the capture rate, allocated energy, and transmit power, based on field of view-based networking in the presence of event and power acquisition patterns. Through simulations, we demonstrate the feasibility of EH-WVSNs in terms of energy consumption, energy allocation, and capture rate in a realistic scenario (parking surveillance).
Transfer Function Model of Physiological Mechanisms Underlying Temporal Visual Discomfort Experienced When Viewing Stereoscopic 3D Images

When viewing 3D images, a sense of visual comfort (or lack of) is developed in the brain over time as a function of binocular disparity and other 3D factors. We have developed a unique temporal visual discomfort model (TVDM) that we use to automatically predict the degree of discomfort felt when viewing stereoscopic 3D (S3D) images. This model is based on physiological mechanisms. In particular, TVDM is defined as a second-order system capturing relevant neuronal elements of the visual pathway from the eyes and through the brain. The experimental results demonstrate that the TVDM transfer function model produces predictions that correlate highly with the subjective visual discomfort scores contained in the large public databases. The transfer function analysis also yields insights into the perceptual processes that yield a stable S3D image.
On asymptotic convergence of the dual filters associated with two families of biorthogonal wavelets

The asymptotic behavior of the dual filters associated with biorthogonal spline wavelet (BSW) systems and general biorthogonal Coifman wavelet (GBCW) systems are studied. As the order of the wavelet systems approaches infinity, the magnitude responses of the dual filters in the BSW systems either diverge or converge to some nonideal frequency responses. However, the synthesis filters in the GBCW systems converge to an ideal halfband lowpass filter without exhibiting any Gibbs-like phenomenon, and a subclass of the analysis filters also converge to an ideal halfband lowpass filter but with a one-sided Gibbs-like behavior. The two approximations of the ideal lowpass filter by the filter associated with a Daubechies orthonormal wavelet and by the synthesis filter in a GBCW system of the same order are compared. Such a study of the asymptotic behaviors of wavelet systems provides insightful characterization of these systems and systematic assessment and global comparison of different wavelet systems.