nils-herrmann/clean_scirepeval huggingface dataset:
Energy Demodulation of Two-component AM-FM Signal Mixtures

is the Teager-Kaiser energy operator ,  and dots denote time derivatives. The ESA is efficient and has very low computational complexity and excellent time resolution. However, if x ( t ) is a multicomponent AM-FM signal, then bandpass filtering is needed to isolate each component before applying the ESA. The bandpass filtering causes problems when the components have overlapping spectra. In this paper, we present the energy demodulation of mixtures (EDM) algorithm for the demodulation of two-component AM-FM signals of the form
Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition

While the accuracy of feature measurements heavily depends on changing environmental conditions, studying the consequences of this fact in pattern recognition tasks has received relatively little attention to date. In this paper, we explicitly take feature measurement uncertainty into account and show how multimodal classification and learning rules should be adjusted to compensate for its effects. Our approach is particularly fruitful in multimodal fusion scenarios, such as audiovisual speech recognition, where multiple streams of complementary time-evolving features are integrated. For such applications, provided that the measurement noise uncertainty for each feature stream can be estimated, the proposed framework leads to highly adaptive multimodal fusion rules which are easy and efficient to implement. Our technique is widely applicable and can be transparently integrated with either synchronous or asynchronous multimodal sequence integration architectures. We further show that multimodal fusion methods relying on stream weights can naturally emerge from our scheme under certain assumptions; this connection provides valuable insights into the adaptivity properties of our multimodal uncertainty compensation approach. We show how these ideas can be practically applied for audiovisual speech recognition. In this context, we propose improved techniques for person-independent visual feature extraction and uncertainty estimation with active appearance models, and also discuss how enhanced audio features along with their uncertainty estimates can be effectively computed. We demonstrate the efficacy of our approach in audiovisual speech recognition experiments on the CUAVE database using either synchronous or asynchronous multimodal integration models.
A Comparison of the Squared Energy and Teager-Kaiser Operators for Short-Term Energy Estimation in Additive Noise

Time-frequency distributions that evaluate the signal's energy content both in the time and frequency domains are indispensable signal processing tools, especially, for nonstationary signals. Various short-time energy computation schemes are used in practice, including the mean squared amplitude and Teager-Kaiser energy approaches. Herein, we focus primarily on the short- and medium-term properties of these two energy estimation schemes, as well as, on their performance in the presence of additive noise. To facilitate this analysis and generalize the approach, we use a harmonic noise model to approximate the noise component. The error analysis is conducted both in the continuous- and discrete-time domains, deriving similar conclusions. The estimation errors are measured in terms of normalized deviations from the expected signal energy and are shown to greatly depend on both the signals' spectral content and the analysis window length. When medium- and long-term analysis windows are employed, the Teager-Kaiser energy operator is proven superior to the common squared energy operator, provided that the spectral content of the noise is more lowpass than the corresponding signal content, and vice versa. However, for shorter window lengths, the Teager-Kaiser operator always outperforms the squared energy operator. The theoretical results are experimentally verified for synthetic signals. Finally, the performance of the proposed energy operators is evaluated for short-term analysis of noisy speech signals and the implications for speech processing applications are outlined.
Phase-modulated resonances modeled as self-similar processes with application to turbulent sounds

In this paper, we propose a nonlinear stochastic model for time-varying resonances where the instantaneous phase (and frequency) of a sinusoidal oscillation is allowed to vary proportionally to an /spl alpha/-stable self-similar stochastic processes. The main motivation of our work stems from previous experimental and theoretical evidence that speech resonances in fricative sounds can be modeled phenomenologically as AM-FM signals with randomly varying instantaneous frequencies and that several signal classes related to turbulent phenomena are self-similar 1/f processes. Our general approach is to model the instantaneous phase of an AM-FM resonance as a self-similar /spl alpha/-stable process. As a special case, this random phase model includes the class of random fractal signals known as fractional Brownian motion. We theoretically explore this random modulation model and analytically derive its autocorrelation and power spectrum. We also propose an algorithm to fit this model to arbitrary resonances with random phase modulation. Further, we apply the above ideas to real speech data and demonstrate that this model is suitable for resonances of fricative sounds.
The MOBOT human-robot communication model

This paper reports on work related to the modelling of Human-Robot Communication on the basis of multimodal and multisensory human behaviour analysis. A primary focus in this framework of analysis is the definition of semantics of human actions, i.e. verbal and non-verbal signals, in a specific context with distinct Human-Robot interaction states. These states are captured and represented in terms of communicative behavioural patterns that influence, and in turn are adapted to the interaction flow with the goal to feed a multimodal human-robot communication system. This multimodal HRI model is defined upon, and ensures the usability of a multimodal sensory corpus acquired as a primary source of data retrieval, analysis and testing of mobility assistive robot prototypes.
AM-FM energy detection and separation in noise using multiband energy operators

This paper develops a multiband or wavelet approach for capturing the AM-FM components of modulated signals immersed in noise. The technique utilizes the recently-popularized nonlinear energy operator Psi (s)=(s)/sup 2/-ss to isolate the AM-FM energy, and an energy separation algorithm (ESA) to extract the instantaneous amplitudes and frequencies. It is demonstrated that the performance of the energy operator/ESA approach is vastly improved if the signal is first filtered through a bank of bandpass filters, and at each instant analyzed (via Psi and the ESA) using the dominant local channel response. Moreover, it is found that uniform (worst-case) performance across the frequency spectrum is attained by using a constant-Q, or multiscale wavelet-like filter bank. The elementary stochastic properties of Psi and of the ESA are developed first. The performance of Psi and the ESA when applied to bandpass filtered versions of an AM-FM signal-plus-noise combination is then analyzed. The predicted performance is greatly improved by filtering, if the local signal frequencies occur in-band. These observations motivate the multiband energy operator and ESA approach, ensuring the in-band analysis of local AM-PM energy. In particular, the multi-bands must have the constant-Q or wavelet scaling property to ensure uniform performance across bands. The theoretical predictions and the simulation results indicate that improved practical strategies are feasible for tracking and identifying AM-FM components in signals possessing pattern coherencies manifested as local concentrations of frequencies.
Tongue tracking in Ultrasound images with Active Appearance Models

Tongue Ultrasound imaging is widely used for human speech production analysis and modeling. In this paper, we propose a novel method to automatically detect and track the tongue contour in Ultrasound (US) videos. Our method is built on a variant of Active Appearance Modeling. It incorporates shape prior information and can estimate the entire tongue contour robustly and accurately in a sequence of US frames. Experimental evaluation demonstrates the effectiveness of our approach and its improved performance compared to previously proposed tongue tracking techniques.
Poisson-Haar Transform: A nonlinear multiscale representation for photon-limited image denoising

We present a novel multiscale image representation belonging to the class of multiscale multiplicative decompositions, which we term Poisson-Haar transform. The proposed representation is well-suited for analyzing images degraded by signal-dependent Poisson noise, allowing efficient estimation of their underlying intensity by means of multiscale Bayesian schemes. The Poisson-Haar decomposition has a direct link to the standard 2-D Haar wavelet transform, thus retaining many of the properties that have made wavelets successful in signal processing and analysis. The practical relevance and effectiveness of the proposed approach is verified through denoising experiments on simulated and real-world photon-limited images.
Unifying the random walker algorithm and the SIR model for graph clustering and image segmentation

In this paper, we explore the image segmentation task using a graph clustering approach. We formulate this clustering as a diffusion scheme whose steady state is determined by the Random Walker (RW) method. Then, we discover the equivalence of this diffusion with the Susceptible - Infected - Recovered (SIR) model, a well-studied epidemic propagation model. We further argue that using a Region Adjacency Graph (RAG) exploits the clustering properties and leads to a dimensionality reduction. Finally, we propose a novel method called Normalized Random Walker (NRW) algorithm which extends the RW method. Qualitative and quantitative experiments validate the efficiency and robustness of our method, with respect to parameter tuning, seed quality and location.
Augmented Human State Estimation Using Interacting Multiple Model Particle Filters With Probabilistic Data Association

The accurate human gait tracking is an important factor for various robotic applications, such as robotic walkers aiming to provide assistance to patients with different mobility impairment, social robot companions, etc. A context-aware robot control architecture needs constant knowledge of the user's kinematic state to assess the patient's gait status and adjust its movement properly to provide optimal assistance. In this letter, we present a novel human gait tracking approach that uses two particle filters (PFs) and probabilistic data association (PDA) with an interacting multiple model (IMM) scheme for a real-time selection of the appropriate motion model according to the human gait analysis and the use of the Viterbi algorithm for an augmented human gait state estimation. The gait state estimates also interact with the IMM as a prior information that drives the Markov sampling process, while the PDA ensures that the legs of the same person are coupled. The observation data in this work are provided by a laser range finder mounted on a robotic assistant walker. A detailed experimental validation is presented using ground truth data from a motion capture system, which was used in real experiments with elder subjects who presented various mobility impairments. The validation analysis regards the algorithm's accuracy, robustness to occlusions and clutter, and the gait state classification success, subject to the effect of different number of samples used in the PFs. The results for the elder subjects show the dynamics of the proposed algorithm to be used in a real-time application due to its efficacy to provide accurate and robust augmented human gait estimates with a small number of particles.
Product-HMMs for automatic sign language recognition

We address multistream sign language recognition and focus on efficient multistream integration schemes. Alternative approaches are investigated and the application of Product-HMMs (PHMM) is proposed. The PHMM is a variant of the general multistream HMM that also allows for partial asynchrony between the streams. Experiments in classification and isolated sign recognition for the Greek Sign Language using different fusion methods, show that the PHMMs perform the best. Fusing movement and shape information with the PHMMs has increased sign classification performance by 1,2% in comparison to the Parallel HMM fusion model. Isolated sign recognition rate increased by 8,3% over movement only models and by 1,5% over movement-shape models using multistream HMMs.
Comparative experimental validation of human gait tracking algorithms for an intelligent robotic rollator

Tracking human gait accurately and robustly constitutes a key factor for a smart robotic walker, aiming to provide assistance to patients with different mobility impairment. A context-aware assistive robot needs constant knowledge of the user's kinematic state to assess the gait status and adjust its movement properly to provide optimal assistance. In this work, we experimentally validate the performance of two gait tracking algorithms using data from elderly patients; the first algorithm employs a Kalman Filter (KF), while the second one tracks the user legs separately using two probabilistically associated Particle Filters (PFs). The algorithms are compared according to their accuracy and robustness, using data captured from real experiments, where elderly subjects performed specific walking scenarios with physical assistance from a prototype Robotic Rollator. Sensorial data were provided by a laser rangefinder mounted on the robotic platform recording the movement of the user's legs. The accuracy of the proposed algorithms is analysed and validated with respect to ground truth data provided by a Motion Capture system tracking a set of visual markers worn by the patients. The robustness of the two tracking algorithms is also analysed comparatively in a complex maneuvering scenario. Current experimental findings demonstrate the superior performance of the PFs in difficult cases of occlusions and clutter, where KF tracking often fails.
Multigrid Geometric Active Contour Models

Geometric active contour models are very popular partial differential equation-based tools in image analysis and computer vision. We present a new multigrid algorithm for the fast evolution of level-set-based geometric active contours and compare it with other established numerical schemes. We overcome the main bottleneck associated with most numerical implementations of geometric active contours, namely the need for very small time steps to avoid instability, by employing a very stable fully 2-D implicit-explicit time integration numerical scheme. The proposed scheme is more accurate and has improved rotational invariance properties compared with alternative split schemes, particularly when big time steps are utilized. We then apply properly designed multigrid methods to efficiently solve the occurring sparse linear system. The combined algorithm allows for the rapid evolution of the contour and convergence to its final configuration after very few iterations. Image segmentation experiments demonstrate the efficiency and accuracy of the method
On separating amplitude from frequency modulations using energy operators

To estimate the amplitude envelope and instantaneous frequency of an AM-FM signal the authors developed a novel approach that uses nonlinear combinations of instantaneous signal outputs from an energy-tracking operator to separate its output energy product into its amplitude modulation and frequency modulation components. This energy separation algorithm is then applied to search for modulations in speech resonances, which the authors model using AM-FM signals. The theoretical and experimental results demonstrate that the energy separation algorithm, due to its low computational complexity and instantaneously adapting nature, is very useful in detecting modulation patterns in speech and other time-varying signals.<<ETX>>
Speech nonlinearities, modulations, and energy operators

An AM-FM model for representing modulations in speech resonances is investigated. Specifically, an FM model is proposed for the time-varying formants whose amplitude varies as the envelope of an AM signal. To detect the modulations the energy operator Psi ( chi ) and its discrete counterpart are applied. It is found that Psi can approximately track the envelope of AM signals, the instantaneous frequency of FM signals, and the product of these two functions in the general case of AM-FM signals. Several experiments on the application of this AM-FM modeling to speech signals, band pass filtered via Gabor filters are reported.<<ETX>>
Bottom-Up &amp; Top-down Object Detection using Primal Sketch Features and Graphical Models

A combination of techniques that is becoming increasingly popular is the construction of part-based object representations using the outputs of interest-point detectors. Our contributions in this paper are twofold: first, we propose a primal-sketch-based set of image tokens that are used for object representation and detection. Second, top-down information is introduced based on an efficient method for the evaluation of the likelihood of hypothesized part locations. This allows us to use graphical model techniques to complement bottom-up detection, by proposing and finding the parts of the object that were missed by the front-end feature detection stage. Detection results for four object categories validate the merits of this joint top-down and bottom-up approach.
Modulation-feature based textured image segmentation using curve evolution

In this paper we incorporate recent results from AM-FM models for texture analysis into the variational model of image segmentation and examine the potential benefits of using the combination of these two approaches for texture segmentation. Using the dominant components analysis (DCA) technique we obtain a low-dimensional, yet rich texture feature vector that proves to be useful for texture segmentation. We use an unsupervised scheme for texture segmentation, where only the number of regions is known a-priori. Experimental results on both synthetic and challenging real-world images demonstrate the potential of the proposed combination.
On amplitude and frequency demodulation using energy operators

It is shown that the nonlinear energy-tracking signal operator Psi (x)=(dx/dt)/sup 2/-xd/sup 2/x/dt/sup 2/ and its discrete-time counterpart can estimate the AM and FM modulating signals. Specifically, Psi can approximately estimate the amplitude envelope of AM signals and the instantaneous frequency of FM signals. Bounds are derived for the approximation errors, which are negligible under general realistic conditions. These results, coupled with the simplicity of Psi , establish the usefulness of the energy operator for AM and FM signal demodulation. These ideas are then extended to a more general class of signals that are sine waves with a time-varying amplitude and frequency and thus contain both an AM and an FM component; for such signals it is shown that Psi can approximately track the product of their amplitude envelope and their instantaneous frequency. The theoretical analysis is done for both continuous- and discrete-time signals.
Towards a behaviorally-validated computational audiovisual saliency model

Computational saliency models aim at predicting, in a bottom-up fashion, where human attention is drawn in the presented (visual, auditory or audiovisual) scene and have been proven useful in applications like robotic navigation, image compression and movie summarization. Despite the fact that well-established auditory and visual saliency models have been validated in behavioral experiments, e.g., by means of eye-tracking, there is no established computational audiovisual saliency model validated in the same way. In this work, building on biologically-inspired models of visual and auditory saliency, we present a joint audiovisual saliency model and introduce the validation approach we follow to show that it is compatible with recent findings of psychology and neuroscience regarding multimodal integration and attention. In this direction, we initially focus on the "pip and pop" effect which has been observed in behavioral experiments and indicates that visual search in sequences of cluttered images can be significantly aided by properly timed non-spatial auditory signals presented alongside the target visual stimuli.
Energy separation in signal modulations with application to speech analysis

An efficient solution to the fundamental problem of estimating the time-varying amplitude envelope and instantaneous frequency of a real-valued signal that has both an AM and FM structure is provided. Nonlinear combinations of instantaneous signal outputs from the energy operator are used to separate its output energy product into its AM and FM components. The theoretical analysis is done first for continuous-time signals. Then several efficient algorithms are developed and compared for estimating the amplitude envelope and instantaneous frequency of discrete-time AM-FM signals. These energy separation algorithms are used to search for modulations in speech resonances, which are modeled using AM-FM signals to account for time-varying amplitude envelopes and instantaneous frequencies. The experimental results provide evidence that bandpass-filtered speech signals around speech formants contain amplitude and frequency modulations within a pitch period.
The MOBOT rollator human-robot interaction model and user evaluation process

In this paper we discuss the integration of a communication model in the MOBOT assistive robotic platform and its evaluation by target users. The MOBOT platform envisions the development of cognitive robotic assistant prototypes that act proactively, adaptively and interactively with respect to elderly humans with slight walking and cognitive impairments. The respective multimodal action recognition system has been developed to monitor, analyze and predict user actions with a high level of accuracy and detail. The robotic platform incorporates a human-robot communication model that has been defined with semantics of human actions in interaction, their capture and their representation in terms of behavioral patterns, to achieve an effective, natural interaction, aiming to support elderly users of slight walking and cognitive inability. The platform has been evaluated in a series of validation experiments with end users, the procedure and results of which are also presented in this paper.
Image inpainting with a wavelet domain Hidden Markov tree model

We present a novel technique for image inpainting, the problem of filling-in missing image parts. Image inpainting is ill-posed and we adopt a probabilistic model-based approach to regularize it. The main elements of our image model are, first, an over-complete complex-wavelet image representation, which ensures good shift invariance and directional selectivity and, second, a discrete-state/continuous-observation hidden Markov tree model for the wavelet coefficients, which captures key statistical properties of natural image wavelet responses, such as heavy-tailed histograms and persistence of large wavelet coefficients across scales. We show how these ideas can be integrated into a multi-scale generative process for natural images and present alternative deterministic and Markov chain Monte Carlo algorithms for image inpainting under this model. We demonstrate the effectiveness of the method in digitally restoring images of ancient wall-paintings.