Impact of Microphone position Measurement Error on Multi Channel Distant Speech Recognition & Intelligibility (2021)

Abstract: It was shown in (Raikar et al., 2020) that the measurement error in the microphone position affected the room impulse response (RIR) which in turn affected the single channel speech recognition. In this paper, we ex-tend this to study the more complex and realistic scenario of multi channel distant speech recognition. Specifically we simulate m speakers in a given room with n microphones speaking without overlap. Then channel audio is beamformed and passed through a speech to text (s2t) engine. We compare the s2t accuracy when the microphone locations are known exactly (ground truth) with the s2t accuracy when there is a measurement error in the location of the microphone. We report the performance of an end-to-end s2t on beamformed input in terms of character error rate (CER) and and also speech intelligibility and quality in terms of STOI and PESQ respectively.

## introduction the multi-path reflections (attributed by rt60) in an open enclosure is caused during hands free speech communication. such multi-path reflections along side noise impinging on multiple microphones result in noisy reverberated speech which has a deteriorating impact on the distant speech recognition performance @xcite . further, the quality and intelligibility of the speech in hands free communication would also deteriorate @xcite @xcite . there is an urgent need for noise reduction, dereverberation and conjunction of both during communications. in this context, multi-microphone-based approaches exploiting spatial acoustic cues such as spatial diversity, inter-intensity differences and inter-time differences, receives particular interest accurate estimation of rt60 plays an important role in several applications like (a) sound re-production of geometry aware room @xcite @xcite , (b) reconstruction of the room geometry @xcite @xcite , (c) robust automatic speech recognition (asr) @xcite @xcite and (d) speech enhancement @xcite @xcite . in a single channel (microphone) scenario, several techniques exist to estimate the room impulse response (rir) @xcite when the microphone position is not erroneous. though, given the room geometry, rir computing is non-trivial; rir estimates require the exact location of both the source and the microphone. a comparative study for blind reverberation time estimation in single microphone scenario is explored in @xcite . a slight displacement (due to human interventions or due to routine maintenance etc. @xcite @xcite ) in the microphone position could severely hamper the rir estimates @xcite . in particular, the impact of measurement error in microphone position on speech intelligibility and quality is explored in @xcite . in practical applications, a microphone array in comparison to single microphone, grants more benefits @xcite especially due to the spatial information and associated applications like directional or arrival (doa), location of sound source and room information @xcite . however, as in single microphone case, calibration error because of displaced microphone array is not trivial to model in a real time scenarios @xcite . this is, primarily because it is computationally expensive and prone to error. in (muthuku-marasamy and donohue, 2009), a delay and sum beamforming (dsb) technique is used to model location errors analytically, they show that the measurement error in microphone position affects the intelligibility and quality due to change in the overall rir. dsb approach has two drawbacks, namely, it requires (a) a large number of sensors to improve the snr and (b) it cannot adapt to varying noisy conditions. to overcome the limitations of dsb, adaptive beamformers like capon @xcite and minimum-variance distortionless response (mvdr) beamformers have been introduced to perform joint noise and reverberation cancellation. in @xcite , a joint noise cancellation and dereverberation is illustrated in generalized side lobe canceller (gsc) framework, while in @xcite , a nested structure in the gsc framework is proposed. as opposed to beamforming, there are adaptive filtering based approaches that do not require spatial information of the speech source. in @xcite , a multi-channel linear prediction (mclp) in kalman filtering domain is proposed for blind dereverberation. however, they fail to perform in the presence of noise as they focus only on reverberation. multi-channel beamformers are prone to measurement error due to change in microphone array position, which affects the rir. this brings to focus, the question, does microphone measurement error affect beamforming performance? in particular, the impact of such displacement error, for single microphone channel, on speech intelligibility and quality has been explored in @xcite . as an extension, it is of interest to explore and investigate the performance of dsb and adaptive beamformer (mvdr) for multi channel distant automatic speech recognition (asr), intelligibility and quality. towards this study, we simulate m speakers in a given room with n microphones speaking without overlap. the output of n channel audio is beamformed and passed through a speech to text (s2t) engine. we compare the s2t accuracy when the microphone locations are known exactly (i.e. ground truth) with the s2t accuracy, when there is a measurement error in the location of the microphone location. the experimental results illustrate that the measurement error in microphone position has a significant effect on s2t performance. consequently, the main contribution of this paper is the formulation of the problem to enable analysis of the effect of microphone position measurement error on distant speech recognition as well as speech intelligibility and quality. note that in this paper, we make no attempt to introduce a new technique or algorithm to improve the distant speech recognition and intelligibility scores; rather the experimental studies reported in this paper should allow for development of new techniques in the future. ## problem formulation let us assume a room r(l, w, h) of dimension l × w × h. let there be n , note that h kl is the rir and is a function of c the speed of sound, f s the sampling frequency of utterance, l × w × h volume of the room, β the reverberation time, (x s , y s , z s ) the location of the speaker, and (x r , y r , z r ) the rectangular coordinates of the microphone. let an @xmath0, y , z ] be made in measuring the position of the l th microphone r l , so the measured location of the r l is r subsequently there is an error introduced in the rir, namely, h * @xmath1, f s , r l , s * , l, β, n). clearly an error in measurement of the microphone r l results in an error in the output speech, namely, the actual output of the l th microphone, if there were no measurement errors, is (1). also in all our experiments we assume to be gaussian n (0, σ 2 ) with 0 mean and σ 2 = 0.1, 0.5, 1 as was done in @xcite . multi channel distant speech recognition involves beamforming (b) the multi channel speech, namely, (o 1 , o 2 , • • • , o m ) from m microphones, to form an equivalent of a close microphone (single channel) speech followed by asr (s2t). for convenience let us represent this process as as can be seen an error ( x , y , z ) in measuring the position of a microphone results in an error at the output of the microphone, namely, o l (3), this results in an error in speech recognition output t , namely in this paper, we analyze the error in the recognition (5) of speech because of an error in measurement of the location of the microphone. ## conclusions in this paper, we addressed the impact of error in measuring the position of microphone position on (a) the performance of a multi-channel distant speech recognition in terms of cer and (b) quality and intelligibility of beamformed speech with two well know beamformers, namely, dsb and mvdr. experimental analysis showed, that with increased measurement error in the location of microphone the quality of mult-channel distant speech recognition deteriorates (higher cer) so does the speech intelligibility and quality, as expected. we further showed that the effect of microphone position measurement error on distant speech recognition in terms of cer is equivalent to a close microphone speech being effected by an additive environmental noise in the range of 5 to 10 db.