Using spatial cues for meeting speech segmentation

This work investigates the validity and accuracy of using spatial cues with time-delay estimation (TDE) as a method of segmenting multichannel recorded speech by speaker location. In environments such as meetings where speakers do not significantly alter position, segmentation by speaker location essentially leads to segmentation by speaker 'turn'. The proposed system calculates location information using TDEs and spatial cues extracted from multichannel meeting audio recordings. This location information is then input into a simple segmentation algorithm. Experiments have been performed on both theoretical and real meeting recordings with non-overlapping speakers, and theoretical recordings with overlapping speakers. Segmentation results reveal the most robust cue to be a combination of spatial information and TDEs. This cue combination leads to greater segmentation accuracy for classifying individual speakers and detecting overlapping sections than using spatial cues or time-delay information alone.

Related papers:

title: Location based speaker segmentation
abstract: The paper proposes a technique that segments audio according to speakers and based on their location. In many multi-party conversations, such as meetings, the location of participants is restricted to a small number of regions, such as seats around a table, or at a whiteboard. In such cases, segmentation according to these discrete regions would be a reliable means of determining speaker turns. We propose a system that uses microphone pair time delays as features to represent speaker locations. These features are integrated in a GMM/HMM framework to determine an optimal segmentation of the audio according to location. The HMM framework also allows extensions to recognise more complex structures, such as the presence of two simultaneous speakers. Experiments testing the system on real recordings from a meeting room show that the proposed location features can provide greater discrimination than standard cepstral features, and also demonstrate the success of an extension to handle dual-speaker overlap.

title: Segmenting multiple concurrent speakers using microphone arrays
abstract: Speaker turn detection is an important task for many speech processing applications. However, accurate segmentation can be hard to achieve if there are multiple concurrent speakers (overlap), as is typically the case in multi-party conversations. In such cases, the location of the speaker, as measured using a microphone array, may provide greater discrimination than traditional spectral features. This was verified in previous work which obtained a global segmentation in terms of single speaker classes, as well as possible overlap combinations. However, such a global strategy suffers from an explosion of the number of overlap classes, as each possible combination of concurrent speakers must be modeled explicitly. In this paper, we propose two alternative schemes that produce an individual segmentation decision for each speaker, implicitly handling all overlapping speaker combinations. The proposed approaches also allow straightforward online implementations. Experiments are presented comparing the segmentation with that obtained using the previous system.

title: Clustering and segmenting speakers and their locations in meetings
abstract: The paper presents a new approach toward automatic annotation of meetings in terms of speaker identities and their locations. This is achieved by segmenting the audio recordings using two independent sources of information: magnitude spectrum analysis and sound source localization. We combine the two in an appropriate HMM framework. There are three main advantages of this approach. First, it is completely unsupervised, i.e. speaker identities and number of speakers and locations are automatically inferred. Second, it is threshold-free, i.e. the decisions are made without the need of a threshold value which generally requires an additional development dataset. The third advantage is that the joint segmentation improves over the speaker segmentation derived using only acoustic features. Experiments on a series of meetings recorded in the IDIAP smart meeting room demonstrate the effectiveness of this approach.

title: Spatial Hearing: The Psychophysics of Human Sound Localization
abstract: Part 1 Introduction: auditory events and auditory space systems analysis of the auditory experiment remarks concerning experimental procedures (psychometric methods, signals and sound fields, probe microphones). Part 2 Spatial hearing with one sound source: localization and localization blur the sound field at the two ears (propagation in the ear canal, the pinna and the effect of the head, transfer functions of the external ear) evaluating identical ear input signals (directional hearing in the median plane, distance hearing and inside-the-head locatedness) evaluating nonidentical ear inputs signals (interaural time differences, interaural level differences, the interaction of interaural time and level differences) additional parameters (motional theories, bone-condition, visual, vestibular and tactile theories). Part 3 Spatial hearing with multiple sound sources and in enclosed spaces: two sound sources radiating coherent signals (summing localization, the law of the first wavefront, inhibition of the primary sound) two sound sources radiating partially coherent or incoherent signals (the influence of the degree of coherence, binaural signal detection) more than two sound sources and diffuse sound fields. Part 4 Progress and trends since 1972: preliminary remarks the physics of the external ear (transfer functions of the external ear, area function and termination of the ear canal, analysis of transfer characteristics) evaluation of monaural attributes of the ear input signals evaluation of interaural attributes of the ear input signals (lateralization and multiple auditory events, summing localization and the law of the first wavefront, binaural localization, signal detection, and speech recognition in the presence of interfering noise, models of binaural signal processing) examples of applications (the auditory spatial impression, dummy-head stereophony). Part 5 Progress and trends since 1982: preliminary remarks binaural room simulation and auditory virtual reality binaural signal processing and speech enhancement the precedence effect - a case of cognition.

title: Binaural cue coding-Part II: Schemes and applications
abstract: Binaural Cue Coding (BCC) is a method for multichannel spatial rendering based on one down-mixed audio channel and side information. The companion paper (Part I) covers the psychoacoustic fundamentals of this method and outlines principles for the design of BCC schemes. The BCC analysis and synthesis methods of Part I are motivated and presented in the framework of stereophonic audio coding. This paper, Part II, generalizes the basic BCC schemes presented in Part I. It includes BCC for multichannel signals and employs an enhanced set of perceptual spatial cues for BCC synthesis. A scheme for multichannel audio coding is presented. Moreover, a modified scheme is derived that allows flexible rendering of the spatial image at the receiver supporting dynamic control. All aspects of complete BCC encoder and decoder implementations are discussed, such as down-mixing of the input signals, low complexity estimation of the spatial cues, and quantization and coding of the side information. Application examples are given and the performance of the coder implementations are evaluated and discussed based on subjective listening test results.

title: Binaural Cue Coding Applied to Stereo and Multi-Channel Audio Compression
abstract: Reference LCAV-CONF-2005-047 URL: www.aes.org Record created on 2005-10-07, modified on 2017-05-12

title: High-quality Parametric Spatial Audio Coding at Low Bitrates
abstract: This convention paper has been reproduced from the author’s advance manuscript, without editing, corrections, or consideration by the Review Board. The AES takes no responsibility for the contents. Additional papers may be obtained by sending request and remittance to Audio Engineering Society, 60 East 42nd Street, New York, New York 10165-2520, USA; also see www.aes.org. All rights reserved. Reproduction of this paper, or any portion thereof, is not permitted without direct permission from the Journal of the Audio Engineering Society.

title: Robust Localization in Reverberant Rooms
abstract: Talker localization with microphone arrays has received significant attention lately as a means for the automated tracking of individuals in an enclosure and as a necessary component of any general purpose speech capture system. Several algorithmic approaches are available for speech source localization with multi-channel data. This chapter summarizes the current field and comments on the general merits and shortcomings of each genre. A new localization method is then presented in detail. By utilizing key features of existing methods, this new algorithm is shown to be significantly more robust to acoustical conditions, particularly reverberation effects, than the traditional localization techniques in use today.

title: Data Mining Practical Machine Learning Tools And Techniques With Java Implementations
abstract: Thank you for reading data mining practical machine learning tools and techniques with java implementations. As you may know, people have look hundreds times for their favorite novels like this data mining practical machine learning tools and techniques with java implementations, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they juggled with some malicious bugs inside their laptop.