nils-herrmann/clean_scirepeval huggingface dataset:
Image saliency by isocentric curvedness and color

In this paper we propose a novel computational method to infer visual saliency in images. The method is based on the idea that salient objects should have local characteristics that are different than the rest of the scene, being edges, color or shape. By using a novel operator, these characteristics are combined to infer global information. The obtained information is used as a weighting for the output of a segmentation algorithm so that the salient object in the scene can easily be distinguished from the background. The proposed approach is fast and it does not require any learning. The experimentation shows that the system can enhance interesting objects in images and it is able to correctly locate the same object annotated by humans with an F-measure of 85.61% when the object size is known, and 79.19% when the object size is unknown, improving the state of the art performance on a public dataset.
Combining Head Pose and Eye Location Information for Gaze Estimation

Head pose and eye location for gaze estimation have been separately studied in numerous works in the literature. Previous research shows that satisfactory accuracy in head pose and eye location estimation can be achieved in constrained settings. However, in the presence of nonfrontal faces, eye locators are not adequate to accurately locate the center of the eyes. On the other hand, head pose estimation techniques are able to deal with these conditions; hence, they may be suited to enhance the accuracy of eye localization. Therefore, in this paper, a hybrid scheme is proposed to combine head pose and eye location information to obtain enhanced gaze estimation. To this end, the transformation matrix obtained from the head pose is used to normalize the eye regions, and in turn, the transformation matrix generated by the found eye location is used to correct the pose estimation procedure. The scheme is designed to enhance the accuracy of eye location estimations, particularly in low-resolution videos, to extend the operative range of the eye locators, and to improve the accuracy of the head pose tracker. These enhanced estimations are then combined to obtain a novel visual gaze estimation system, which uses both eye location and head information to refine the gaze estimates. From the experimental results, it can be derived that the proposed unified scheme improves the accuracy of eye estimations by 16% to 23%. Furthermore, it considerably extends its operating range by more than 15° by overcoming the problems introduced by extreme head poses. Moreover, the accuracy of the head pose tracker is improved by 12% to 24%. Finally, the experimentation on the proposed combined gaze estimation system shows that it is accurate (with a mean error between 2° and 5°) and that it can be used in cases where classic approaches would fail without imposing restraints on the position of the head.
A Linear Kalman Filter for MARG Orientation Estimation Using the Algebraic Quaternion Algorithm

Real-time orientation estimation using low-cost inertial sensors is essential for all the applications where size and power consumption are critical constraints. Such applications include robotics, human motion analysis, and mobile devices. This paper presents a linear Kalman filter for magnetic angular rate and gravity sensors that processes angular rate, acceleration, and magnetic field data to obtain an estimation of the orientation in quaternion representation. Acceleration and magnetic field observations are preprocessed through a novel external algorithm, which computes the quaternion orientation as the composition of two algebraic quaternions. The decoupled nature of the two quaternions makes the roll and pitch components of the orientation immune to magnetic disturbances. The external algorithm reduces the complexity of the filter, making the measurement equations linear. Real-time implementation and the test results of the Kalman filter are presented and compared against a typical quaternion-based extended Kalman filter and a constant gain filter based on the gradient-descent algorithm.
Visual Gaze Estimation by Joint Head and Eye Information

In this paper, we present an unconstrained visual gaze estimation system. The proposed method extracts the visual field of view of a person looking at a target scene in order to estimate the approximate location of interest (visual gaze). The novelty of the system is the joint use of head pose and eye location information to fine tune the visual gaze estimated by the head pose only, so that the system can be used in multiple scenarios. The improvements obtained by the proposed approach are validated using the Boston University head pose dataset, on which the standard deviation of the joint visual gaze estimation improved by 61:06% horizontally and 52:23% vertically with respect to the gaze estimation obtained by the head pose only. A user study shows the potential of the proposed system.
GUMS: A generalized unified model for stereo omnidirectional vision (demonstrated via a folded catadioptric system)

This paper introduces GUMS, a complete projection model for omnidirectional stereo vision systems. GUMS is based on the existing generalized unified model (GUM), which we extend in order to satisfy a tight relationship among a pair of omnidirectional views for fixed baseline sensors. We exemplify the proposed model's calibration via a single-camera coaxial omnistereo system in a joint bundle-adjusted fashion. We compare our coupled method against the naive approach where the calibration of intrinsic parameters is first performed individually for each omnidirectional view using existing monocular implementations, to then solve for the extrinsic parameters as an additional step that has no effect on the intrinsic model solutions initially computed. We validate GUMS and its calibration effectiveness using both real and synthetic systems against ground-truth data. Our calibration method proves successful for correcting the unavoidable misalignment present in vertically-configured catadioptric rigs. We also generate 3D point clouds employing the calibrated GUMS systems in order to demonstrate the qualitative outcome of our contribution.
Facial Expression Recognition: A Fully Integrated Approach

The most expressive way humans display emotions is through facial expressions. Humans detect and interpret faces and facial expressions in a scene with little or no effort. Still, development of an automated system that accomplishes this task is rather difficult. There are several related problems: detection of an image segment as a face, facial features extraction and tracking, extraction of the facial expression information, and classification of the expression (e.g., in emotion categories). In this paper, we present our fully integrated system which performs these operations accurately and in real time and represents a major step forward in our aim of achieving a humanlike interaction between the man and machine.
Joint Attention by Gaze Interpolation and Saliency

Joint attention, which is the ability of coordination of a common point of reference with the communicating party, emerges as a key factor in various interaction scenarios. This paper presents an image-based method for establishing joint attention between an experimenter and a robot. The precise analysis of the experimenter's eye region requires stability and high-resolution image acquisition, which is not always available. We investigate regression-based interpolation of the gaze direction from the head pose of the experimenter, which is easier to track. Gaussian process regression and neural networks are contrasted to interpolate the gaze direction. Then, we combine gaze interpolation with image-based saliency to improve the target point estimates and test three different saliency schemes. We demonstrate the proposed method on a human-robot interaction scenario. Cross-subject evaluations, as well as experiments under adverse conditions (such as dimmed or artificial illumination or motion blur), show that our method generalizes well and achieves rapid gaze estimation for establishing joint attention.
Calibration-Free Gaze Estimation Using Human Gaze Patterns

We present a novel method to auto-calibrate gaze estimators based on gaze patterns obtained from other viewers. Our method is based on the observation that the gaze patterns of humans are indicative of where a new viewer will look at. When a new viewer is looking at a stimulus, we first estimate a topology of gaze points (initial gaze points). Next, these points are transformed so that they match the gaze patterns of other humans to find the correct gaze points. In a flexible uncalibrated setup with a web camera and no chin rest, the proposed method was tested on ten subjects and ten images. The method estimates the gaze points after looking at a stimulus for a few seconds with an average accuracy of 4:3°. Although the reported performance is lower than what could be achieved with dedicated hardware or calibrated setup, the proposed method still provides a sufficient accuracy to trace the viewer attention. This is promising considering the fact that auto-calibration is done in a flexible setup, without the use of a chin rest, and based only on a few seconds of gaze initialization data. To the best of our knowledge, this is the first work to use human gaze patterns in order to auto-calibrate gaze estimators.
Fast Hyperspectral Unmixing in Presence of Nonlinearity or Mismodeling Effects

This paper presents two novel hyperspectral mixture models and associated unmixing algorithms. The two models assume a linear mixing model corrupted by an additive term whose expression can be adapted to account for multiple scattering nonlinearities (NL), or mismodeling effects (ME). The NL model generalizes bilinear models by taking into account higher order interaction terms. The ME model accounts for different effects, such as endmember variability or the presence of outliers. The abundance and residual parameters of these models are estimated by considering a convex formulation suitable for fast estimation algorithms. This formulation accounts for constraints, such as the sum-to-one and nonnegativity of the abundances, the nonnegativity of the nonlinearity coefficients, the spectral smoothness of the ME terms and the spatial sparseness of the residuals. The resulting convex problem is solved using the alternating direction method of multipliers whose convergence is ensured theoretically. The proposed mixture models and their unmixing algorithms are validated on both synthetic and real images showing competitive results regarding the quality of the inference and the computational complexity when compared to the state-of-the-art algorithms.
Hyperspectral Remote Sensing Data Analysis and Future Challenges

Hyperspectral remote sensing technology has advanced significantly in the past two decades. Current sensors onboard airborne and spaceborne platforms cover large areas of the Earth surface with unprecedented spectral, spatial, and temporal resolutions. These characteristics enable a myriad of applications requiring fine identification of materials or estimation of physical parameters. Very often, these applications rely on sophisticated and complex data analysis methods. The sources of difficulties are, namely, the high dimensionality and size of the hyperspectral data, the spectral mixing (linear and nonlinear), and the degradation mechanisms associated to the measurement process such as noise and atmospheric effects. This paper presents a tutorial/overview cross section of some relevant hyperspectral data analysis methods and algorithms, organized in six main topics: data fusion, unmixing, classification, target detection, physical parameter retrieval, and fast computing. In all topics, we describe the state-of-the-art, provide illustrative examples, and point to future challenges and research directions.
Hyperspectral coded aperture (HYCA): A new technique for hyperspectral compressive sensing

In this paper, we develop a new lossy compression framework for hyperspectral images, termed hyperspectral coded aperture (HYCA), which combines the ideas of spectral unmixing and compressive sensing. It takes advantage of two main properties of hyperspectral data, namely the high spatial correlation that can be observed in the data and the generally low number of endmembers needed in order to explain the data. In other words, our proposed approach intends to exploit the fact that the high dimensional hyperspectral data lives in a subspace of much lower dimension due to the mixing phenomenon. Our experimental results, conducted with synthetic hyperspectral data, indicate that the proposed approach represents a promising new strategy.
Phase Unwrapping via Graph Cuts

Phase unwrapping is the inference of absolute phase from modulo-2pi phase. This paper introduces a new energy minimization framework for phase unwrapping. The considered objective functions are first-order Markov random fields. We provide an exact energy minimization algorithm, whenever the corresponding clique potentials are convex, namely for the phase unwrapping classical Lp norm, with pges1. Its complexity is KT(n,3n), where K is the length of the absolute phase domain measured in 2pi units and T(n,m) is the complexity of a max-flow computation in a graph with n nodes and m edges. For nonconvex clique potentials, often used owing to their discontinuity preserving ability, we face an NP-hard problem for which we devise an approximate solution. Both algorithms solve integer optimization problems by computing a sequence of binary optimizations, each one solved by graph cut techniques. Accordingly, we name the two algorithms PUMA, for phase unwrapping max-flow/min-cut. A set of experimental results illustrates the effectiveness of the proposed approach and its competitiveness in comparison with state-of-the-art phase unwrapping algorithms
A Dynamic Unmixing Framework for Plant Production System Monitoring

Hyperspectral remote sensing or imaging spectroscopy is an emerging technology in plant production monitoring and management. The continuous reflectance spectra allow for the intensive monitoring of biophysical and biochemical tree characteristics during growth, through for instance the use of vegetation indices. Yet, since most of the pixels in hyperspectral images are mixed, the evaluation of the actual vegetation state on the ground directly from the measured spectra is degraded by the presence of other endmembers, such as soil. Spectral unmixing, then, becomes a necessary processing step to improve the interpretation of vegetation indices. In this sense, an active research direction is based on the use of large collections of pure spectra, called spectral libraries or dictionaries, which model a wide variety of possible states of the endmembers of interest on the ground, i.e., vegetation and soil. Under the linear mixing model (LMM), the observed spectra are assumed to be linear combinations of spectra from the available dictionary. Combinatorial techniques (e.g., MESMA) and sparse regression algorithms (e.g., SUnSAL) are widely used to tackle the unmixing problem in this case. However, both combinatorial and sparse techniques benefit from appropriate library reduction strategies. In this paper, we develop a new efficient method for library reduction (or dictionary pruning), which exploits the fact that hyperspectral data generally lives in a lower-dimensional subspace. Specifically, we present a slight modification of the MUSIC-CSR algorithm, a two-step method which aims first at pruning the dictionary and second at infering high-quality reconstruction of the vegetation spectra on the ground (this application being called signal unmixing in remote sensing), using the pruned dictionary as input to available unmixing methods. Our goal is two-fold: 1) to obtain high-accuracy unmixing output using sparse unmixing, with low-execution time; and 2) to improve MESMA performances in terms of accuracy. Our experiments, which have been conducted in a multi-temporal case study, show that the method achieves these two goals and proposes sparse unmixing as a reliable and robust alternative to the combinatorial methods in plant production monitoring applications. We further demonstrate that the proposed methodology of combining a library pruning approach with spectral unmixing provides a solid framework for the year-round monitoring of plant production systems.
Hyperspectral Pansharpening: A Review

Pansharpening aims at fusing a panchromatic image with a multispectral one, to generate an image with the high spatial resolution of the former and the high spectral resolution of the latter. In the last decade, many algorithms have been presented in the literatures for pansharpening using multispectral data. With the increasing availability of hyperspectral systems, these methods are now being adapted to hyperspectral images. In this work, we compare new pansharpening techniques designed for hyperspectral data with some of the state-of-the-art methods for multispectral pansharpening, which have been adapted for hyperspectral data. Eleven methods from different classes (component substitution, multiresolution analysis, hybrid, Bayesian and matrix factorization) are analyzed. These methods are applied to three datasets and their effectiveness and robustness are evaluated with widely used performance indicators. In addition, all the pansharpening techniques considered in this paper have been implemented in a MATLAB toolbox that is made available to the community.
Class-specific poisson denoising by patch-based importance sampling

In this paper, we address the problem of recovering images degraded by Poisson noise, where the image is known to belong to a specific class. In the proposed method, a dataset of clean patches from images of the class of interest is clustered using multivariate Gaussian distributions. In order to recover the noisy image, each noisy patch is assigned to one of these distributions, and the corresponding minimum mean squared error (MMSE) estimate is obtained. We propose to use a self-normalized importance sampling approach, which is a method of the Monte-Carlo family, for the both determining the most likely distribution and approximating the MMSE estimate of the clean patch. Experimental results shows that our proposed method outperforms other methods for Poisson denoising at a low SNR regime.
Interferometric Phase Image Estimation via Sparse Coding in the Complex Domain

This paper addresses interferometric phase image estimation, i.e., the estimation of phase modulo-2π images from sinusoidal 2π-periodic and noisy observations. These degradation mechanisms make interferometric phase image estimation a quite challenging problem. We tackle this challenge by reformulating the true estimation problem as a sparse regression, often termed sparse coding, in the complex domain. Following the standard procedure in patch-based image restoration, the image is partitioned into small overlapping square patches, and the vector corresponding to each patch is modeled as a sparse linear combination of vectors, termed the atoms, taken from a set called dictionary. Aiming at optimal sparse representations, and thus at optimal noise removing capabilities, the dictionary is learned from the data that it represents via matrix factorization with sparsity constraints on the code (i.e., the regression coefficients) enforced by the ℓ1 norm. The effectiveness of the new sparse-coding-based approach to interferometric phase estimation, termed the SpInPHASE, is illustrated in a series of experiments with simulated and real data where it outperforms the state-of-the-art.
Wall position and thickness estimation from sequences of echocardiographic images

Presents a new method for endocardial (inner) and epicardial (outer) contour estimation from sequences of echocardiographic images. The framework herein introduced is fine-tuned for parasternal short axis views at the papillary muscle level. The underlying model is probabilistic; it captures the relevant features of the image generation physical mechanisms and of the heart morphology. Contour sequences are assumed to be two-dimensional noncausal first-order Markov random processes; each variable has a spatial index and a temporal index. The image pixels are modeled as Rayleigh distributed random variables with means depending on their positions (inside endocardium, between endocardium and pericardium, or outside pericardium). The complete probabilistic model is built under the Bayesian framework. As estimation criterion the maximum a posteriori (MAP) is adopted. To solve the optimization problem, one is led to (joint estimation of contours and distributions' parameters), the authors introduce an algorithm herein named iterative multigrid dynamic programming (IMDP). It is a fully data-driven scheme with no ad-hoc parameters. The method is implemented on an ordinary workstation, leading to computation times compatible with operational use. Experiments with simulated and real images are presented.
Hyperspectral Image Segmentation Using a New Bayesian Approach With Active Learning

This paper introduces a new supervised Bayesian approach to hyperspectral image segmentation with active learning, which consists of two main steps. First, we use a multinomial logistic regression (MLR) model to learn the class posterior probability distributions. This is done by using a recently introduced logistic regression via splitting and augmented Lagrangian algorithm. Second, we use the information acquired in the previous step to segment the hyperspectral image using a multilevel logistic prior that encodes the spatial information. In order to reduce the cost of acquiring large training sets, active learning is performed based on the MLR posterior probabilities. Another contribution of this paper is the introduction of a new active sampling approach, called modified breaking ties, which is able to provide an unbiased sampling. Furthermore, we have implemented our proposed method in an efficient way. For instance, in order to obtain the time-consuming maximum a posteriori segmentation, we use the α-expansion min-cut-based integer optimization algorithm. The state-of-the-art performance of the proposed approach is illustrated using both simulated and real hyperspectral data sets in a number of experimental comparisons with recently introduced hyperspectral image analysis methods.
Phase imaging via sparse coding in the complex domain based on high-order svd and nonlocal BM3D techniques

The paper addresses interferometric phase image estimation, that is, the estimation of phase modulo-2π images from sinusoidal 2π-periodic and noisy observations. These degradation mechanisms make interferometric phase image estimation a challenging problem. We tackle this challenge by reformulating the true estimation problem as a sparse regression in the complex domain. Following the standard procedure in patch-based image restoration, the image is partitioned into small overlapping square patches. BM3D algorithm equipped with high order SVD (HOSVD) is used to form complex domain frames suitable to sparse representations of the complex-valued data. HOSVD applied to the groups of BM3D data enables the design of spatially variant and data adaptive orthonormal complex domain transforms. The effectiveness of the new sparse coding based approach to interferometric phase estimation, termed Interferometric PHASE via Block matching and High order SVD (InPHASE-BHS) is illustrated in a series of simulation experiments where it outperforms the state-of-the-art.