nils-herrmann/clean_scirepeval huggingface dataset:
Multi-room speech activity detection using a distributed microphone network in domestic environments

Domestic environments are particularly challenging for distant speech recognition: reverberation, background noise and interfering sources, as well as the propagation of acoustic events across adjacent rooms, critically degrade the performance of standard speech processing algorithms. In this application scenario, a crucial task is the detection and localization of speech events generated by users within the various rooms. A specific challenge of multi-room environments is the inter-room interference that negatively affects speech activity detectors. In this paper, we present and compare different solutions for the multi-room speech activity detection task. The combination of a model-based room-independent speech activity detection module with a room-dependent inside/outside classification stage, based on specific features, provides satisfactory performance. The proposed methods are evaluated on a multi-room, multi-channel corpus, where spoken commands and other typical acoustic events occur in different rooms.
Affine models for image matching and motion detection

A model is developed for detecting the displacement field in spatiotemporal image sequences. It allows for affine shape deformations of corresponding spatial regions and for affine transformations of the image intensity range. The model includes the block matching method as a special case. A least-squares algorithm is used to find the model parameters. It is experimentally demonstrated that the affine matching model performs better than other standard approaches. The resulting 2-D motion estimates are then used by a 3-D affine model and a least-squares algorithm that recover 3-D rigid body motion and depth from two perspective views.<<ETX>>
A unification of linear, median, order-statistics and morphological filters under mathematical morphology

This paper presents a summary of a unified theory, based upon mathematical morphology, of all translation-invariant and increasing systems. Examples of such systems are morphological transformations of signals, order-statistics filters and some linear shift-invariant filters. Our theoretical research showed that every such system can be uniquely represented by the minimal elements of its kernel and realized as a minimal combination of morphological erosions.
Segmentation of soil section images using connected operators

Segmentation of soil section images is an important task for automating the measurement of the grains' properties as well as for detecting and recognizing objects in the soil, important for its bioecological quality. We apply several types of morphological systems to watershed-based segmentation of soil section images. We use efficient connected operators such as reconstruction open-closing and area open-closing as well as some relatively new operators, the levelings, for image denoising, simplification and feature/marker extraction. Further, we introduce an improvement of the reconstruction operators used in segmentation, based on a generalized multiscale connectivity analysis.
Coupled geometric and texture PDE-based segmentation

In this paper, along with recent trends in segmentation using multiple image cues, we examine the integration of modulation texture features, image contrast and region size for decomposition of an image in homogenous regions. First, we propose the use of a morphological PDE-based segmentation scheme of the watershed type, based on seeded region-growing and level curve evolution with speed depending on contrast and size. Second, we analyze object surface texture by modelling image variations as local spatial modulation components estimated via multi-frequency filtering and instantaneous energy-tracking operators. By separately exploiting contrast and texture information, through multiscale image decomposition, we propose a PDE-based coupled segmentation method. Experimental results on various classes of images such as soilsections, aerial and natural scenes indicate that the combined effect of image decomposition and multi-cue segmentation improves the overall segmentation process.
Higher order differential energy operators

Instantaneous signal operators /spl Upsi//sub k/(x)=x/spl dot/x/sup (k-1)/-xx/sup (k)/ of integer orders k are proposed to measure the cross energy between a signal x and its derivatives. These higher order differential energy operators contain as a special case, for k=2, the Teager-Kaiser (1990) operator. When applied to (possibly modulated) sinusoids, they yield several new energy measurements useful for parameter estimation or AM-FM demodulation. Applying them to sampled signals involves replacing derivatives with differences that lead to several useful discrete energy operators defined on an extremely short window of samples.<<ETX>>
Experimental comparison of human gait tracking algorithms: Towards a context-aware mobility assistance robotic walker

Towards a mobility assistance robot for the elderly, it is essential to develop a robust and accurate gait tracking system. Various pathologies cause mobility inabilities to the aged population, leading to different gait patterns and walking speed. In this work, we present the experimental comparison of two user leg tracking systems of a robotic assistance walker, using data collected by a laser range sensor. The first one is a Kalman Filter tracking system, while the second one proposes the use of Particle Filters. The tracking systems provide the positions and velocities of the user's legs, which are used as observations into an HMM-based gait phases recognition system. The spatiotemporal results of the HMM framework are employed for computing parameters that characterize the human motion, which subsequently can be used to assess and distinguish between possible motion disabilities. For the experimental comparison, we are using real data collected from an ensemble of different elderly persons with a number of pathologies, and ground truth data from a GaitRite System. The results presented in this work, demonstrate the applicability of the tracking systems in real test cases.
Recognitionwith raw canonical phonetic movement and handshape subunits on videos of continuous Sign Language

The visual processing of Sign Language (SL) videos offers multiple interdisciplinary challenges for image processing and recognition. Based on tracking and visual feature extraction, we investigate SL visual phonetic modeling by exploiting statistical subunit (SU) models of movement-position and handshape. We further propose a new framework to construct a data-driven lexicon that retains phonetics' movement information and to perform automatic recognition of continuous SL videos. We construct phonetically meaningful transition SU, named as raw canonical phonetic subunits (SU-CanRaw). Then, we integrate via a Hidden Markov Model multistream scheme the SU-CanRaw extended for both hands, with handshape SU, based on our previous work on Affine-invariant Shape-Appearance Models. By applying the all-inclusive framework on continuous SL videos, we automatically generate a data-driven lexicon that can be further exploited, for automatic analysis of SL corpora, and continuous SL recognition. The recognition experiments, conducted on a newly acquired continuous SL corpus, lead to promising results.
Partial differential equations in image analysis: continuous modeling, discrete processing

Most traditional methodologies in digital image processing start with a discrete model which is directly related to the subsequent discrete processing of the image data. We review some examples and focus on some details of an emerging new methodology that starts from some continuous models provided by PDEs and proceeds with discrete processing of the image data via the numerical implementation of these PDEs on some discrete grid. We examine the PDEs modeling multiscale morphological image analysis and the related eikonal PDE of optics. The PDE approach is very promising because it provides many new mathematical models and has connections with the physics of imaging.
Face Active Appearance Modeling and Speech Acoustic Information to Recover Articulation

We are interested in recovering aspects of vocal tract's geometry and dynamics from speech, a problem referred to as speech inversion. Traditional audio-only speech inversion techniques are inherently ill-posed since the same speech acoustics can be produced by multiple articulatory configurations. To alleviate the ill-posedness of the audio-only inversion process, we propose an inversion scheme which also exploits visual information from the speaker's face. The complex audiovisual-to-articulatory mapping is approximated by an adaptive piecewise linear model. Model switching is governed by a Markovian discrete process which captures articulatory dynamic information. Each constituent linear mapping is effectively estimated via canonical correlation analysis. In the described multimodal context, we investigate alternative fusion schemes which allow interaction between the audio and visual modalities at various synchronization levels. For facial analysis, we employ active appearance models (AAMs) and demonstrate fully automatic face tracking and visual feature extraction. Using the AAM features in conjunction with audio features such as Mel frequency cepstral coefficients (MFCCs) or line spectral frequencies (LSFs) leads to effective estimation of the trajectories followed by certain points of interest in the speech production system. We report experiments on the QSMT and MOCHA databases which contain audio, video, and electromagnetic articulography data recorded in parallel. The results show that exploiting both audio and visual modalities in a multistream hidden Markov model based scheme clearly improves performance relative to either audio or visual-only estimation.
GridNews: A distributed automatic Greek broadcast transcription system

In this paper, a distributed system storing and retrieving Broadcast News data recorded from the Greek television is presented. These multimodal data are processed in a grid computational environment interconnecting distributed data storage and processing subsystems. The innovative element of this system is the implementation of the signal processing algorithms in this grid environment, offering additional flexibility and computational power. Among the developed signal processing modules are: the Segmentor, cutting up the original videos into shorter ones, the Classifier, recognizing whether these short videos contain speech or not, the Greek large-vocabulary speech Recognizer, transcribing speech into written text, and finally the text Search engine and the video Retriever. All the processed data are stored and retrieved in geographically distributed storage elements. A user-friendly, web-based interface is developed, facilitating the transparent import and storage of new multimodal data, their off-line processing and finally, their search and retrieval.
Hidden markov modeling of human pathological gait using laser range finder for an assisted living intelligent robotic walker

The precise analysis of a patient's or an elderly person's walking pattern is very important for an effective intelligent active mobility assistance robot. This walking pattern can be described by a cyclic motion, which can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing and recognizing a pathological human walking gait pattern. Our framework utilizes a laser range finder sensor to detect and track the human legs, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait patterns. We demonstrate the applicability of this setup using real data, collected from an ensemble of different elderly persons with a number of pathologies. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the classification of specific walking pathologies, which is needed for the development of a context-aware robot mobility assistant.
On the Effects of Filterbank Design and Energy Computation on Robust Speech Recognition

In this paper, we examine how energy computation and filterbank design contribute to the overall front-end robustness, especially when the investigated features are applied to noisy speech signals, in mismatched training-testing conditions. In prior work (“Auditory Teager energy cepstrum coefficients for robust speech recognition,” D. Dimitriadis, P. Maragos, and A. Potamianos,  2005), a novel feature set called “Teager energy cepstrum coefficients” (TECCs) has been proposed, employing a dense, smooth filterbank and alternative energy computation schemes. TECCs were shown to be more robust to noise and exhibit improved performance compared to the widely used Mel frequency cepstral coefficients (MFCCs). In this paper, we attempt to interpret these results using a combined theoretical and experimental analysis framework. Specifically, we investigate in detail the connection between the filterbank design, i.e., the filter shape and bandwidth, the energy estimation scheme and the automatic speech recognition (ASR) performance under a variety of additive and/or convolutional noise conditions. For this purpose: 1) the performance of filterbanks using triangular, Gabor, and Gammatone filters with various bandwidths and filter positions are examined under different noisy speech recognition tasks, and 2) the squared amplitude and Teager-Kaiser energy operators are compared as two alternative approaches of computing the signal energy. Our end-goal is to understand how to select the most efficient filterbank and energy computation scheme that are maximally robust under both clean and noisy recording conditions. Theoretical and experimental results show that: 1) the filter bandwidth is one of the most important factors affecting speech recognition performance in noise, while the shape of the filter is of secondary importance, and 2) the Teager-Kaiser operator outperforms (on the average and for most noise types) the squared amplitude energy computation scheme for speech recognition in noisy conditions, especially, for large filter bandwidths. Experimental results show that selecting the appropriate filterbank and energy computation scheme can lead to significant error rate reduction over both MFCC and perceptual linear predicion (PLP) features for a variety of speech recognition tasks. A relative error rate reduction of up to ~ 30% for MFCCs and ~ 39% for PLPs is shown for the Aurora-3 Spanish Task.
Nonlinear speech analysis using models for chaotic systems

In this paper, we use concepts and methods from chaotic systems to model and analyze nonlinear dynamics in speech signals. The modeling is done not on the scalar speech signal, but on its reconstructed multidimensional attractor by embedding the scalar signal into a phase space. We have analyzed and compared a variety of nonlinear models for approximating the dynamics of complex systems using a small record of their observed output. These models include approximations based on global or local polynomials as well as approximations inspired from machine learning such as radial basis function networks, fuzzy-logic systems and support vector machines. Our focus has been on facilitating the application of the methods of chaotic signal analysis even when only a short time series is available, like phonemes in speech utterances. This introduced an increased degree of difficulty that was dealt with by resorting to sophisticated function approximation models that are appropriate for short data sets. Using these models enabled us to compute for short time series of speech sounds useful features like Lyapunov exponents that are used to assist in the characterization of chaotic systems. Several experimental insights are reported on the possible applications of such nonlinear models and features.
Symbolic signal representation using nonlinear filtering

A symbolic modeling method is developed for images and imagelike signals based on their shape-size structure. A signal is represented as a minimal nonlinear superposition of simpler parts, namely, the symbols, which are translated and scaled shape patterns drawn from a finite collection. To obtain the model parameters, multiscale morphological openings and the related pattern spectrum are used to help decide which shape patterns exist in the signal and at which scales. Then the pattern locations are found via a local search at points of generalized morphological skeletons. This model is useful for signal abstractions and object recognition.<<ETX>>
Graph-Driven Diffusion and Random Walk Schemes for Image Segmentation

We propose graph-driven approaches to image segmentation by developing diffusion processes defined on arbitrary graphs. We formulate a solution to the image segmentation problem modeled as the result of infectious wavefronts propagating on an image-driven graph, where pixels correspond to nodes of an arbitrary graph. By relating the popular susceptible-infected-recovered epidemic propagation model to the Random Walker algorithm, we develop the normalized random walker and a lazy random walker variant. The underlying iterative solutions of these methods are derived as the result of infections transmitted on this arbitrary graph. The main idea is to incorporate a degree-aware term into the original Random Walker algorithm in order to account for the node centrality of every neighboring node and to weigh the contribution of every neighbor to the underlying diffusion process. Our lazy random walk variant models the tendency of patients or nodes to resist changes in their infection status. We also show how previous work can be naturally extended to take advantage of this degree-aware term, which enables the design of other novel methods. Through an extensive experimental analysis, we demonstrate the reliability of our approach, its small computational burden and the dimensionality reduction capabilities of graph-driven approaches. Without applying any regular grid constraint, the proposed graph clustering scheme allows us to consider pixel-level, node-level approaches, and multidimensional input data by naturally integrating the importance of each node to the final clustering or segmentation solution. A software release containing implementations of this paper and supplementary material can be found at: http://cvsp.cs.ntua.gr/research/GraphClustering/.
Analysis of Kirlian images: feature extraction and segmentation

Images of high geometrical complexity are found in various applications in the fields of image processing and computer vision. In this paper we utilize general processing techniques, mainly based on image morphology. We focus on Kirlian images, which due to their high complexity, comprise of features appearing in many biomedical images. In this paper, a first approach is given on the extraction of specific features dealing with the size and geometrical structure of Kirlian images. The extraction is implemented with the use of tools provided by the broader field of computer vision, thus providing a multi-faceted description of the images. Furthermore, this paper provides and promotes the use of automatically extracted information. Finally, efficient algorithms for obtaining the information on the size and structure of Kirlian images are presented and a number of conclusions are drawn and discussed that provide an insight on the underlying information within a highly complex image such as Kirlian images.
Multimodal Saliency and Fusion for Movie Summarization Based on Aural, Visual, and Textual Attention

Multimodal streams of sensory information are naturally parsed and integrated by humans using signal-level feature extraction and higher level cognitive processes. Detection of attention-invoking audiovisual segments is formulated in this work on the basis of saliency models for the audio, visual, and textual information conveyed in a video stream. Aural or auditory saliency is assessed by cues that quantify multifrequency waveform modulations, extracted through nonlinear operators and energy tracking. Visual saliency is measured through a spatiotemporal attention model driven by intensity, color, and orientation. Textual or linguistic saliency is extracted from part-of-speech tagging on the subtitles information available with most movie distributions. The individual saliency streams, obtained from modality-depended cues, are integrated in a multimodal saliency curve, modeling the time-varying perceptual importance of the composite video stream and signifying prevailing sensory events. The multimodal saliency representation forms the basis of a generic, bottom-up video summarization algorithm. Different fusion schemes are evaluated on a movie database of multimodal saliency annotations with comparative results provided across modalities. The produced summaries, based on low-level features and content-independent fusion and selection, are of subjectively high aesthetic and informative quality.
Modulation features for speech recognition

Automatic speech recognition (ASR) systems can benefit from including into their acoustic processing part new features that account for various nonlinear and time-varying phenomena during speech production. In this paper, we develop robust methods to extract novel acoustic features from speech signals of the modulation type based on time-varying models for speech analysis. Further, we integrate the new speech features with the standard linear ones (mel-frequency cesptrum) to develop a augmented set of acoustic features and demonstrate its efficacy by showing significant improvements in HMM-based word recognition over the TIMIT database.
Spatial bayesian surprise for image saliency and quality assessment

We propose an alternative interpretation of Bayesian surprise in the spatial domain, to account for saliency arising from contrast in image context. Our saliency formulation is integrated in three different application scenaria, with considerable improvements in performance: 1) visual attention prediction, validated using eye- and mouse-tracking data, 2) region of interest detection, to improve scale selection and localization, 3) image quality assessment to achieve better agreement with subjective human evaluations.
Speech analysis and feature extraction using chaotic models

Nonlinear systems based on chaos theory can model various aspects of the nonlinear dynamic phenomena occuring during speech production. In this paper, we explore modem methods and algorithms from chaotic systems theory for modeling speech signals in a multidimensional phase space and for extracting nonlinear acoustic features. Further, we integrate these chaotic-type features with the standard linear ones (based on cepstrum) to develop a generalized hybrid set of short-time acoustic features for speech signals and demonstrate its efficacy by showing significant improvements in HMM-based word recognition.