nils-herrmann/clean_scirepeval huggingface dataset:
Towards Understanding and Modeling Audiovisual Saliency Based on Talking Faces

Usual attention is an important mechanism of the human visual system. It allows reducing the amount of information to be processed and accelerates the overall process of vision. Several models for images and videos have been proposed in the literature with encouraging results. However, most existing saliency models do not take into account the multimodal aspect of the video (audio and image). In this paper, we propose to investigate the influence of audio on visual attention. From one side, we carried out eye-tracking experiments for recording subjects' eye movement when watching videos. From another side, eye positions (fixation duration) were used for computing and comparing video attention maps (through eye-movements) with and without audio using state-of-the-art measures. Results therefore showed that audio significantly affect the viewer's attention and consequently it must be taken into consideration for the development of any multimedia saliency model. The findings of these experiments have been used for the development of audiovisual saliency model based on talking face. Result obtained with our model were assessed using usual measurements and showed good performance with regards to ground truth.
An efficient demosaicing technique using geometrical information

Color image sensors use color filter arrays (CFA) to capture information at each sensor pixel position and require color demosaicing to reconstruct full color images. The quality of the demosaicked image is hindered by the sensor characteristics during the acquisition process. In this work, we propose a bandelet-based demosaicing method for color images. To this end, we have used a spatial multiplexing model of color in order to obtain the luminance and the chrominance components of the acquired image. Then, a luminance filter is used to reconstruct the luminance component. Thereafter, based on the concept of maximal gradient of multivalued images, we propose an extension of the bandelet representation for the case of multivalued images. Finally, demosaicing is performed by merging the luminance and each of the chrominance component in the multivalued bandelet transform domain. The experimental evaluation of the proposed scheme shows beneficial performance over existing demosaicing approaches.
Visual attention modeling for 3D video using neural networks

Visual attention is one of the most important mechanisms in the human visual perception. Recently, its modeling becomes a principal requirement for the optimization of the image processing systems. Numerous algorithms have already been designed for 2D saliency prediction. However, only few works can be found for 3D content. In this study, we propose a saliency model for stereoscopic 3D video. This algorithm extracts information from three dimensions of content, i.e. spatial, temporal and depth. This model benefits from the properties of interest points to be close to human fixations in order to build spatial salient features. Besides, as the perception of depth relies strongly on monocular cues, our model extracts the depth salient features using the pictorial depth sources. Since weights for fusion strategy are often selected in ad-hoc manner, in this work, we suggest to use a machine learning approach. The used artificial Neural Network allows to define adaptive weights based on the eye-tracking data. The results of the proposed algorithm are tested versus ground-truth information using the state-of-the-art techniques.
A no-reference color video quality metric based on a 3D multispectral wavelet transform

In this work, a no reference objective color video quality assessment metric is presented. First, a multi-valued 3D subband wavelet decomposition is defined. This wavelet representation is used to decompose the video sequence in order to describe the different channels in the human visual system (HVS). Thereafter, based on the properties of the HVS, a perceptual mask that integrates spatio-temporal contrast sensitivity function (CSF) and luminance sensitivity is applied to each wavelet band. Then, we define a flow tensor between successive frames. This flow tensor is weighted by the perceptual mask and is used to define a NR color video quality metric. Particularly, based on the eigenvalues and eigenvectors of the flow tensor, we study the inter-frame coherence and the sharpness of edges in the successive frames. Experiments performed on video sequences indicate that the objective scores obtained by the proposed metric agree well with the subjective assessment scores.
Stereoscopic 3D image quality assessment based on cyclopean view and depth map

This paper presents a full reference quality assessment metric for stereoscopic images based on perceptual binocular characteristics. To ensure that the predicted 3D quality of experience is as reliable and close as possible to 3D human perception, the proposed stereoscopic image quality assessment (SIQA) method is relying on the cyclopean image. Our approach is motivated by the fact that in case of asymmetric quality, 3D perception mechanisms place more emphasis on the view providing the most important and contrasted information. We integrated this psychophysical findings in the proposed 3D-IQA framework thanks to a weighting factor based on local information content. Add to that, to take into account the disparity/depth masking effect, we modulate the obtained quality score of each pixel of the cyclopean image according to its location in the scene. Experimental results show that the proposed metric correlates better with human judgement than the state-of-the-art metrics.
Stereo image coding based on the binocular compensation/suppression

In this work, a wavelet-based coding scheme for stereoscopic images is presented. The proposed scheme effectively integrates the coding of the disparity map in addition to the reference image. Hence, it first calculates the wavelet transform of the left and right images. A process for detecting an optimal scale for the Discrete Wavelet Transform (DWT) is then applied. The sub-bands of the last retained level of the DWT are then segmented into quadtree. The disparity map is obtained by studying the similarity between the dyadic squares belonging to the left image and those of the right image. This is achieved by minimizing an energy function defined on the spatial properties of the dyadic squares. Thus, the binocular compensation/suppression process allows at this stage to take advantage of the gap existing generally between the two views. Finally, the coefficients of the reference image, the residual image and the disparity map are encoded using a quantization step followed by an entropic coding. Of course, the target image is predicted using the aforementioned content. The experimental results of the proposed approach shows a good performance compared to the literature.
Using Spatial and Compressed Features for Multi-Spectral Image Retrieval

This work lies within the scope of content-based image retrieval and browsing. Our interest is related on the one hand, to the extraction of features in the compressed domain with the JPEG2000 standard and on the other hand, to the multispectral images acquired within the framework of project PIMHAI (European project INTERREG III B ATLANTIC AREA) dedicated to the monitoring of the environment of the Atlantic littoral. Multispectral images are characterized by a high number of bands and an important resolution. In order to extract the relevant content from the images and then to build the signatures which will represent them, we jointly used features calculated in the spatial domain and others in the compressed domain. These latter features are applied to all the sub-bands of the wavelet transform (DWT) except for low frequency sub-band (named LL). This one is represented using features from the spatial domain. The similarity measurement between images is calculated using the fusion of the features carried out by the means of weighting factors resulting from a preliminary training. The results are posted on a browsing interface, which is installed in order to simplify browsing and retrieval.
Low-complexity enhanced lapped transform for image coding in JPEG XR / HD photo

JPEG-XR is a new image compression standard that aims at achieving state-of-the-art image compression, while simultaneously keeping the encoder and decoder complexities as low as possible. JPEG-XR  is based on Microsoft technology known as HDPHOTO and makes use of a block-transform. This transform, known as Lapped Biorthogonal Transform (LBT), requires only a small memory footprint while providing the compression benefits of a larger block transform. In this work, we propose to replace the LBT by a representation in Legendre orthogonal polynomial basis. The motivation behind using the Legendre polynomials is that, in general, moment functions of orthogonal polynomials provide better feature representations over other type of moments [7]  and have some properties related to the human visual system (HVS) . However, Legendre polynomials have a unit weight function and recurrence relation involving real coefficients, which make them suitable for defining image representation. We show that the expansion in Legendre polynomial basis can be implemented via lifting operations and has the same computation complexity as the LBT. The experimental evaluation of our modified JPEG-XR scheme shows beneficial improvements in terms of visual quality over the standard JPEG-XR.
An efficient region of interest acquisition method for dynamic magnetic resonance imaging

Motivated by work in the area of dynamic magnetic resonance imaging (MRI), we develop a new approach to the problem of reduced-order MRI acquisition. Efforts in this field have concentrated on the use of Fourier and singular value decomposition (SVD) methods to obtain low-order representations of an entire image plane. We augment this work to the case of imaging an arbitrarily-shaped region of interest (ROI) embedded within the full image. After developing a natural error metric for this problem, we show that determining the minimal order required to meet a prescribed error level is in general intractable, but can be solved under certain assumptions. We then develop an optimization approach to the related problem of minimizing the error for a given order. Finally, we demonstrate the utility of this approach and its advantages over existing Fourier and SVD methods on a number of MRI images.
Multiscale methods for the segmentation and reconstruction of signals and images

This paper addresses the problem of both segmenting and reconstructing a noisy signal or image. The work is motivated by large problems arising in certain scientific applications, such as medical imaging. Two objectives for a segmentation and denoising algorithm are laid out: it should be computationally efficient and capable of generating statistics for the errors in the reconstruction and estimates of the boundary locations. The starting point for the development of a suitable algorithm is a variational approach to segmentation (Shah 1992). This paper then develops a precise statistical interpretation of a one dimensional (1-D) version of this variational approach to segmentation. The 1-D algorithm that arises as a result of this analysis is computationally efficient and capable of generating error statistics. A straightforward extension of this algorithm to two dimensions would incorporate recursive procedures for computing estimates of inhomogeneous Gaussian Markov random fields. Such procedures require an unacceptably large number of operations. To meet the objective of developing a computationally efficient algorithm, the use of previously developed multiscale statistical methods is investigated. This results in the development of an algorithm for segmenting and denoising which is not only computationally efficient but also capable of generating error statistics, as desired.
Accurate model-based high resolution cardiac image reconstruction in dual source CT

Cardiac imaging represents one of the most challenging imaging problems, requiring high spatial and temporal resolutions along with good tissue contrast. One of the newest clinical cardiac CT scanners incorporates two source-detector pairs in order to improve the temporal resolution by two-fold. To achieve the highest spatial resolution, reconstructions using iterative techniques may be desired. Yet the complexity of the dual-source geometry makes accurate system modeling a challenge. In this paper, we present a model-based iterative reconstruction algorithm for the dual-source CT. We demonstrate, using a total variation formulation, the results of our reconstructions. To accelerate the processing and enhance the quality of the result, we also incorporate a simplified detector response function in the forward projector. A segment of heavily-calcified coronary artery is used to demonstrate the spatial and temporal resolution of this approach with the dual-source system.
Estimation of dynamically evolving ellipsoids with applications to medical imaging

The estimation of dynamically evolving ellipsoids from noisy lower-dimensional projections is examined. In particular, this work describes a model-based approach using geometric reconstruction and recursive estimation techniques to obtain a dynamic estimate of left-ventricular ejection fraction from a gated set of planar myocardial perfusion images. The proposed approach differs from current ejection fraction estimation techniques both in the imaging modality used and in the subsequent processing which yields a dynamic ejection fraction estimate. For this work, the left ventricle is modeled as a dynamically evolving three-dimensional (3-D) ellipsoid. The left-ventricular outline observed in the myocardial perfusion images is then modeled as a dynamic, two-dimensional (2-D) ellipsoid, obtained as the projection of the former 3-D ellipsoid. This data is processed in two ways: first, as a 3-D dynamic ellipsoid reconstruction problem; second, each view is considered as a 2-D dynamic ellipse estimation problem and then the 3-D ejection fraction is obtained by combining the effective 2-D ejection fractions of each view. The approximating ellipsoids are reconstructed using a Rauch-Tung-Striebel smoothing filter, which produces an ejection fraction estimate that is more robust to noise since it is based on the entire data set; in contrast, traditional ejection fraction estimates are based only on true frames of data. Further, numerical studies of the sensitivity of this approach to unknown dynamics and projection geometry are presented, providing a rational basis for specifying system parameters. This investigation includes estimation of ejection fraction from both simulated and real data.
A novel hybrid algorithm for accelerating CT reconstructions and improving low-dose image quality

In this paper we introduce a new algorithm for reconstruction of low-dose CT images. The approach, called multi-resolution feature fusion (MRFF), combines the textural qualities of conventional filtered-back projection images, with the noise suppression ability of non-quadratic regularized iterative reconstructions, to form a fast image reconstruction with good noise texture properties. Low-dose abdominal CT data is used to illustrate the properties of MRFF.
A statistical tomographic approach to synthetic aperture radar image reconstruction

We formulate the spotlight-mode synthetic aperture radar (SAR) imaging problem as a maximum a posteriori (MAP) estimation problem. Our statistical perspective offers an alternative to standard SAR processing techniques by incorporating prior knowledge about either the data quality or the underlying reflectivity function. In our formulation we consider exploiting the similarity between computer assisted tomography (CAT) and spotlight-mode SAR, which was first exposed by Munson et al. (1983). We present the effectiveness of the proposed approach by imagery formed from simulated spotlight-mode SAR data.
Calcium De-blooming in Coronary CT Image

Recent development of Multi-detector Computerized Tomography (MDCT) holds the promise of a non-invasive diagnostic method to evaluate coronary artery disease as an alternative to standard angiography. A major impediment preventing its utilization in routine clinical practice is the presence of image "blooming" artifacts due to vascular calcium. For high-risk patients with serious coronary stenosis who often have significant calcified plaque, this problem becomes very acute. In this paper, we examine an approach to ameliorate the degradations due to the presence of calcium and thus increase the applicability of MDCT. We assume the blooming effect is well modeled as a linear convolutional blur. We then use this model as the basis to perform image restoration, thus deblurring the original image. The aim is to produce CT images with reduced calcium artifacts. We evaluate our method on simulated, ex vivo and clinical data and show that this method has the potential to improve coronary CT image and thus improve stenosis diagnosis.
LADAR Range Image Segmentation using Curve Evolution and Maximum Likelihood Estimation

In this paper, we develop a new maximum likelihood-based, curve evolution approach for laser radar range image segmentation. This approach combines a hybrid scene model for representing the range distribution of the field and a statistical mixture model for the range data measurement noise. The image segmentation problem is formulated as an energy minimization problem which jointly estimates the target boundary together with the target region intensity and background texture directly from the noisy range data. Curve evolution techniques and an expectation-maximization algorithm are jointly employed as an efficient solver for minimizing the objective energy
Using shape distributions as priors in a curve evolution framework

We propose a novel framework for constructing and using a shape prior in a curve evolution framework. The prior shape information is captured through shape distributions, which are histograms of features derived from the shape boundary. The resulting prior captures perceptual shape similarity, is robust to small sample size, and is flexible. We further derive a curve evolution force that corresponds to this prior. This enables us to use this prior to perform tasks such as mean shape calculation and image segmentation within a curve evolution framework.
Graph-Cut Based Discrete-Valued Image Reconstruction

Efficient graph-cut methods have been used with great success for labeling and denoising problems occurring in computer vision. Unfortunately, the presence of linear image mappings has prevented the use of these techniques in most discrete-amplitude image reconstruction problems. In this paper, we develop a graph-cut based framework for the direct solution of discrete amplitude linear image reconstruction problems cast as regularized energy function minimizations. We first analyze the structure of discrete linear inverse problem cost functions to show that the obstacle to the application of graph-cut methods to their solution is the variable mixing caused by the presence of the linear sensing operator. We then propose to use a surrogate energy functional that overcomes the challenges imposed by the sensing operator yet can be utilized efficiently in existing graph-cut frameworks. We use this surrogate energy functional to devise a monotonic iterative algorithm for the solution of discrete valued inverse problems. We first provide experiments using local convolutional operators and show the robustness of the proposed technique to noise and stability to changes in regularization parameter. Then we focus on nonlocal, tomographic examples where we consider limited-angle data problems. We compare our technique with state-of-the-art discrete and continuous image reconstruction techniques. Experiments show that the proposed method outperforms state-of-the-art techniques in challenging scenarios involving discrete valued unknowns.
Imaging of Moving Targets With Multi-Static SAR Using an Overcomplete Dictionary

This paper presents a method for imaging of moving targets using multi-static radar by treating the problem as one of joint spatial reflectivity signal inversion with respect to an overcomplete dictionary of target velocities. Existing approaches to dealing with moving targets in SAR solve the nonlinear problem of target scattering and motion estimation typically through decoupled matched filtering. In contrast, by using an overcomplete dictionary approach we effectively linearize the forward model and solve the moving target problem as a larger, unified regularized inversion problem subject to sparsity constraints. This unified framework allows estimation of scatter motion and reflectivity to be done in an optimal and global way. We show examples of the potential of the new method for sensing configurations with transmitters and receivers randomly dispersed in a multi-static geometry within a narrow forward cone around the scene of interest.
A semidefinite programming approach to source localization

A new algorithm based on semidefinite programming is presented for estimation of distributed source fields measured through a sensor array. The problem can be viewed as a subclass of inverse problems, which have been extensively investigated in the literature. Our approach is based on the so called information based complexity (IBC) paradigm, which formalizes the notion of seeking the set of all solutions that are consistent with the observed data. We formulate our problem as a question of estimating the source up to a prespecified resolution (average source field in a neighborhood) from the observed data with optimal accuracy. We show that this problem is convex and can be reformulated as a semidefinite program.
Sparsity-Driven Sparse-Aperture Ultrasound Imaging

We propose an image formation algorithm for ultrasound imaging based on sparsity-driven regularization functionals. We consider data collected by synthetic transducer arrays, with the primary motivating application being nondestructive evaluation. Our framework involves the use of a physical optics-based forward model of the observation process; the formulation of an optimization problem for image formation; and the solution of that problem through efficient numerical algorithms. Our sparsity-driven, model-based approach achieves the preservation of physical features while suppressing spurious artifacts. It also provides robust reconstructions in the case of sparse observation apertures. We demonstrate the effectiveness of our imaging strategy on real ultrasound data
Robust estimation of arterial strain from non-invasive carotid ultrasound images

We have developed a novel estimator of arterial wall strain from non-invasive ultrasound images of the pulsating carotid artery. This strain estimator exhibits improved robustness to noise when compared with conventional methods and addresses the need for non-invasive monitoring of regional arterial wall stiffness and composition remodeling caused by vascular disease. We demonstrate its effectiveness in simulated ultrasound sequences and provide an example of strain estimation from carotid ultrasound imaging of a normal volunteer.
Wavelet packet based transient signal classification

Nonstationary signals are not well suited for detection and classification by traditional Fourier methods. An alternate means of analysis needs to be used so that valuable time-frequency information is not lost. The wavelet packet transform is one such time-frequency analysis tool. The authors summarize efforts which examine the feasibility of applying the wavelet packet transform to automatic transient signal classification through the development of a classification algorithm for biologically generated underwater acoustic signals in ocean noise. The formulation of a wavelet-packet based feature set specific to the classification of snapping shrimp and whale clicks is given.<<ETX>>