2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC)

GENERALIZED AMPLITUDE INTERPOLATION BY β-DIVERGENCE FOR VIRTUALMICROPHONE ARRAY
Hiroki Katahira1 , Nobutaka Ono2,3 , Shigeki Miyabe1 , Takeshi Yamada1 , Shoji Makino1
1

2

University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8577 Japan
National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda, Tokyo, 101-8430 Japan
3The Graduate University for Advanced Studies (Sokendai)
katahira@mmlab.cs.tsukuba.ac.jp, onono@nii.ac.jp,{miyabe, maki}@tara.tsukuba.ac.jp, takeshi@cs.tsukuba.ac.jp

ABSTRACTIn this paper, we present a generalization of the virtual microphone array we previously proposed to increase the microphone elements by nonlinear interpolation. In the previous work, we generated a virtual observation from two actual
microphones by an interpolation in the logarithmic domain.
This corresponds to a linear interpolation of the phase and thegeometric mean of the amplitude. In this paper, we generalize this interpolation using a linear interpolation of the phaseand a nonlinear interpolation of the amplitude with adjustablenonlinearity based on β-divergence. Improvement of the array signal processing performance is obtained by appropriate
tuning of the parameter β. We evaluate the improvement inspeech enhancement using a maximum SNR beamformer.

Actual microphone signal
Outputs

Virtualmicrophone signals

Signal Processing͙͙Interpolation

Fig. 1: Block diagram of signal processing with virtual microphone array techniqueĐƚƵĂůDŝĐƌŽƉŚŽŶĞ

ĐƚƵĂůDŝĐƌŽƉŚŽŶĞ

sŝƌƚƵĂůDŝĐƌŽƉŚŽŶĞ

Index Terms— virtual microphone, array signal processing, speech enhancement, maximum SNR beamformer

Fig. 2: Arrangement of actual and virtual microphoneswas an improvement in the speech enhancement performance
when we used a maximum SNR beamformer. In a linear interpolation in the complex logarithmic domain, the phase islinearly interpolated because the phase is defined as the imaginary part of the complex logarithm. It is appropriate because
the phase changes linearly between two microphones if a single plane wave arrives. However, there is no reason for theamplitude to be interpolated in the logarithmic scale.
In this paper, we consider a generalization of this virtual
microphone array technique. By introducing β divergence,
we propose a new nonlinear amplitude interpolation with adjustable nonlinearity. The logarithmic interpolation used in
the previous work is given by β = 1, and the linear interpolation in the amplitude domain is given by β = 2. The
speech enhancement performance with the maximum SNR
beamformer is evaluated experimentally with different β values and different numbers of virtual channels.

1. INTRODUCTION
A microphone array is a signal processing framework basedon multichannel observation and it is important for blindsource separation (BSS) [1], direction of arrival (DOA) estimation [2] and speech enhancement. The array signal
processing performance depends on the number of microphones. Although several methods such as time-frequency(T-F) masking [3] and multichannel Wiener filter [4] can work
well with a small number of microphones, better performancecan be expected if more microphones are available.
Therefore, we have investigated an approach for improving the performance of array signal processing by virtually
increasing the number of channels [5]. The concept of a “virtual microphone array”, which is an attempt to estimate orcreate an acoustic observation at a place where there are noactual microphones, can be found in other contexts such as
introducing higher order statistics [6] or spatial sound acquisition [7, 8]. To increasing the number of linearly independent
observations, we employed a linear interpolation in the complex logarithmic domain [5]. We also confirmed that there

2. VIRTUAL MICROPHONE BY INTERPOLATIONIn our virtual microphone array technique, we create arbitrary
channels of virtual microphone signals as a synthesis of 2150

2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC)

channels of actual microphone signals, and then we performthe array signal processing with an observed signal consisting of both actual and virtual microphone signals as shownin Fig. 1. Virtual microphone signals are generated as estimates of signals observed with a virtual microphone placed
at a point where there is no actual microphone. A virtual microphone signal v = v (ω, t, α) is generated as an estimatedobservation obtained with a virtual microphone placed at theα : (1 − α) internal division point of the positions of two
actual microphones (Fig. 2). The simplest approach is linear
interpolation as,v = (1 − α) x1 + αx2 ,between nonnegative values such as amplitude. For instance,
β-divergence is used as the cost function of nonnegative matrix factorization (NMF) [9, 10]. The β-divergence between
a virtual microphone signal amplitude Av and the i-th actual
microphone signal amplitude Ai is defined as
Dβ (Av , Ai )
Av (log Av − log Ai ) + (Ai − Av ) (β = 1)
Av Av(7)− log−1(β = 0).
= AiAiAβAv Aβ−1Aβvi+ i −(otherwise)β (β − 1)
ββ−1(1)

Dβ is continuous at β = 0 and β = 1. For β-divergencebased interpolation, we consider the amplitude Av to minimize the sum σβ of the β-divergence between the amplitude values of an actual microphones signal and virtual microphone signal weighted by a virtual microphone position
α,where xi = xi (ω, t) denotes the observed signal of the ith actual microphone at frequency ω, and time t. However,virtual microphone signals derived from linear interpolation
are linearly dependent and the linear interpolation does notprovide any new statistical information for array signal processing. Thus, we generate a virtual microphone signal as aninterpolation in the nonlinear function domain and we havealready proposed a virtual microphone derived from complex
logarithmic interpolation [5] written as
v=exp ((1 − α) log x1 + α log x2 ) .

(3)The interpolation consists of the linear interpolation of thephase angle and logarithmic interpolation of the amplitudeand Eq. (2) is rewritten as
exp ((1 − α) log A1 + α log A2 ) ,Av=ϕvv= (1 − α) ϕ1 + αϕ2 ,= Av exp (jϕv ) ,= (1 − α) Dβ (Av , A1 ) + αDβ (Av , A2 ) , (8)Avβ= argminAv σDβ .

(9)By differentiating σDβ with Av and setting it at 0, the amplitude interpolation extended with β-divergence is obtainedasexp ((1 − α) log A1 + α log A2 ) (β = 1)) 1
Avβ = (.
 (1 − α) Aβ−1 + αAβ−1 β−1(otherwise)
21(10)
Similarly to β-divergence Dβ , Avβ is continuous at β = 1 as() 1β−1 β−1lim (1 − α) Aβ−1+αA
12β→1(11)= exp ((1 − α) log A1 + α log A2 ) .

(2)The complex logarithmic function derives the logarithmic
amplitude and phase of signal as real and imaginary parts
respectively as follows,
log xi = log |xi | + j∠xi .

σDβ(4)

When β is set at 1, the interpolation is equal to the logarithmic
interpolation noted in Eq. (4). The amplitude interpolation inEq. (10) is assumed to be the β −1 norm of a vector composedof amplitude weighted by α. Therefore, it also approaches themaximum selection and minimum selection taking the limitof β → +∞ and β → −∞ respectively as

(5)
(6)

where Ai = |xi (ω, t)| and ϕi = ∠xi (ω, t) represent for theamplitude and phase of the i-th actual microphone signal, respectively. The linear interpolation of phase angle in Eq. (5)is assumed to be appropriate with the assumption of a single
plane wave propagation model because the phase difference
between microphones is in linear relation to the microphonepositions. In contrast, the logarithmic amplitude interpolationin Eq. (4) is not based on a specific model or there are no reasons for amplitude to be interpolated in the logarithmic scale.
Therefore, in the next section we consider the extension ofamplitude interpolation method by introducing β-divergence.

Avβ
Avβ

= max (A1 , A2 ) (β → +∞) ,= min (A1 , A2 ) (β → −∞) .

(12)
(13)

The phase of a signal is linearly interpolated in a similar way
to complex logarithmic interpolation as in Eq. (5), and a virtual microphone signal is obtained similarly to Eq. (6),v= Avβ exp (jϕv ) .

(14)

3. INTRODUCTION OF β-DIVERGENCE FORAMPLITUDE INTERPOLATION

4. SPEECH ENHANCEMENT WITH MAXIMUM
SNR BEAMFORMERIn this section, we introduce β-divergence for amplitude interpolation. β-divergence is widely used criteria for margin

We apply the virtual microphone array technique to a maximum SNR beamformer [11] to evaluate the performance. A151

2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC)

maximum SNR beamformer requires the covariance matricesof the target-only period and the interference-only period asprior information of speech enhancement. However, it requires no information about sound direction such as steeringvectors.

Interference:-90㼻䡚 -30㼻30㼻 䡚 90㼻
8 sources

1.5 m

4.1. Construction of maximum SNR beamformer
Speech enhancement with a beamformer is realized by constructing a multichannel filter given by[]Tw (ω) = w1 (ω) , · · · , wM (ω) ,(15)4 cmFig. 3: Source and microphone layout in experiment

to reduce the contamination of interference sources, wherewi∗ (ω) is a filter for the i-th channel and {·}∗ denotes a complex conjugation. The enhanced signals y (ω, t) are given as
the inner product of the filter and the observed signal vector,y (ω, t) = wH (ω) x (ω, t) .

5. EXPERIMENTS
We conducted speech enhancement experiments with themaximum SNR beamformer to evaluate the performancewith different β values.

(16)

In a maximum SNR beamformer, the filter w (ω) is designed
to maximize the ratio λ (ω) of the power between the targetonly period ΘT , and the interference-only period ΘI :5.1. Experimental conditions
The layout of the sources and actual microphones is shownin Fig. 3, and other experimental conditions are shown in Table 1. We used the 3 samples of Japanese and English speech
for the target signals, and we performed 5 directions of arrival(DOA) experiments for each sample target signal, giving atotal of 15 combinations of target DOA and speech samples.
We used a mixture of 8 speech signals for the interferencesignal. The speech signals arrive from 8 different directions
simultaneously. The observed signals were formed as the convolutive mixture of measured impulse responses and speech
signals. We placed virtual microphones between two actual
microphones at regular intervals, thus the parameter α of thei-th virtual microphone was given as

Hλ(ω) =w (ω)RT (ω)w(ω),wH (ω)RI (ω)w(ω)(17)

where RT (ω) and RI (ω) represent the covariance matricesof the target-only period and interference-only periods, respectively. The covariance matrices are calculated as
1 ∑RT (ω) =xT (ω, t)xH
(18)
T (ω, t),|ΘT |t∈ΘT
1 ∑xI (ω, t)xH
(19)
RI (ω) =I (ω, t),|ΘI |t∈ΘI

where xT is the observed signal vector in the target-only period and xI is the observed signal vector in the interferenceonly period. The filter w (ω) that maximizes the ratio λ (ω) isgiven as an eigenvector corresponding to the maximum eigenvalue of the following generalized eigenvalue problem;
RT (ω)w(ω) = λ(ω)RI (ω)w(ω).

α=(20)

Since the maximum SNR beamformer w (ω) has a scaling
ambiguity, the beamformer is compensated in [12] as:(21)

where bk (ω) is the k-th component of b (ω) given byb(ω) =Rx (ω)w(ω),wH (ω)Rx (ω)w(ω)(22)

Rx (ω) =1∑x(ω, t)xH (ω, t).
T t=1(23)i,N +1(24)

where N is the number of inserted virtual microphones.
Speech enhancement was conducted with microphone arrays consisting of 2 actual microphones and N virtual microphones, thus giving (N + 2) channels in total. In thisexperiment, the first microphone was chosen as the reference(k = 1) for scale compensation described in section 4.2. Unlike our previous work [5], we here performed the experiment
without any regularization to the covariance matrices.
To evaluate the performance of the beamformer, we used
an objective criterion, the signal-to-distortion ratio (SDR) andthe signal-to-interference ratio (SIR) [13]. We show the mean
SDR and SIR values for 15 combinations of target DOA andspeech samples.

4.2. Scaling compensation of beamformerw(ω) ← bk (ω)w (ω) ,Target:-20, -10, 0, 10 or 20㼻

5.2. Results and discussion

TFigure 4 shows the relation between the speech enhancement
performance and β with different virtual microphone layouts,

1522014 14th International Workshop on Acoustic Signal Enhancement (IWAENC)

ϭϰEсϵEсϳ
Eсϱ
Eсϯ
Eсϭ

^Z΀Ě΁

ϲ
ϱ
ϰ

EсϬ

ϭϮ
Eсϭ
Eсϯ
Eсϱ
Eсϳ

Ϯ

EсϵEсϬ

Eсϳ

Eсϭ

Eсϱ

Eсϯ

Eсϯ

EсϱϭϭϭϬϵϴ

ϯ

Eсϵϭϯ

EсϬ^/Z΀Ě΁ϳ

Eсϭ
EсϳϳͲϮϬͲϭϬϬɴϭϬEсϬϲϮϬEсϵͲϮϬͲϭϬ(a) SDRϬɴϭϬϮϬ(b) SIRϳϲ͘ϱϲϱ͘ϱϱϰ͘ϱϰϯ͘ϱϯɴсϭ;ůŽŐͿɴсϬɴсͲϮϬ^/Z΀Ě΁^Z΀Ě΁

Fig. 4: The relationship between β and separation performanceɴсϮϬDĂǆ;ɴсьͿDŝŶ;ɴсͲьͿϬϭϮ ϯ ϰ ϱ ϲηŽĨsŝƌƚƵĂůDŝĐƐϳϴϵϭϰϭϯϭϮϭϭ
ϭϬϵϴϳϲɴсϭ;ůŽŐͿɴсϬɴсͲϮϬɴсϮϬ
DĂǆ;ɴсьͿ
DŝŶ;ɴсͲьͿϬ(a) SDRϭϮ ϯ ϰ ϱ ϲηŽĨsŝƌƚƵĂůDŝĐƐϳϴ

ϵ(b) SIRFig. 5: Performance shift by number of virtual microphonesTable 1: Experimental conditions# of actual microphones# of virtual virtual microphones
Real microphone spacing
Reverberation time
Sampling rate
FFT frame length
FFT frame shift
Test signal length
Target only period length |θT |Interference only period length |θI |the distortion of output sound is reduced with the parameteris set at β = −20. The decay of SDR with a parameter βof around 0 is possibly caused by noise attributed to the rank
deficiency of the covariance matrices. The parameter β seems
to have an effect to adjust the degree of rank deficiency. Thus,
we need to examine the relationship between parameter β andrank deficiency.

20–94 cm
640 ms
8 kHz
1024 samples
256 samples
20 sec10 sec10 sec

6. CONCLUSION
We proposed the generalization of the virtual microphonearray technique that introduced β-divergence for the interpolation of amplitude. We compared the performance of a
maximum SNR beamformer with different β values. With aconventional complex logarithmic interpolation (β = 1), theSDR has a peak about 1 dB higher than the actual microphonearray. In contrast, the SDR is improved about 2.8 dB compared with a real microphone array with β set at −20 or +20.
Therefore, we confirm the effectiveness of the introduction of
β-divergence into the virtual microphone array technique.

and Fig. 5 shows the relationship between performance andnumber of virtual microphones with several β values. Please
note that when the number of virtual microphones is zero itmeans that the beamformer is processed solely with actual microphone signals and without virtual microphones. The SDR
is improved when a few virtual microphones are inserted withany parameter β. The SIR is improved as the number of virtual microphones is increased. In contrast, the SDR is improved particularly when the parameter β is far from 0, andthe SDR decreases with a parameter β of around 0 including alogarithmic interpolation (β = 1). The output sound of beamformer becomes distorted when a large number of virtual microphones is introduced with the parameter β = 1. However

7. REFERENCES[1] S. Makino, T.-W. Lee, and H. Sawada, Blind Speech
Separation, Springer, 2007.

153

2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC)[2] R. O. Schmidt, “Multiple emitter location and signal parameter estimation,” IEEE Trans., vol. 34, no. 3, 1986.
[3] O. Yilmaz and S. Rickard, “Blind separation of speech
mixtures via time-frequency masking,” IEEE Trans on
Signal Processing, vol. 52, no. 7, pp. 1830–1847, 2004.
[4] N. Q. K. Duong, E. Vincent, and R. Gribonval, “Underdetermined reverberant audio source separation using afull-rank spatial covariance model,” IEEE Trans.on Audio Speech Language Process, vol. 18, no. 7, pp. 1830–1840, 2010.
[5] H. Katahira, N. Ono, S. Miyabe, T. Yamada, andS. Makino, “Virtually increasing microphone array elements by interpolation in complex-logarithmic domain,”
Proc. EUSIPCO, vol. TH-L 5.3, pp. 1–5, 2013.
[6] P. Chevalier, L. Albera, A. Ferréol, and P. Comon, “Onthe virtual array concept for higher order array processing,” IEEE Trans. Signal Processing, vol. 53, no. 4, pp.
1254–1271, 2005.
[7] G. Del Galdo, O. Thiergart, T. Weller, and E. A. P. Habets, “Generating virtual microphone signals using geometrical information gathered by distributed arrays,”
Proc. HSCMA, pp. 185–190, 2011.
[8] K. Kowalczyk, A. Craciun, and E. A. P. Habets, “Generating virtual microphone signals in noisy environments,”
Proc. EUSIPCO, 2013.
[9] M. Nakano, H. Kameoka, J. Le Roux, Y. Kitano,
N. Ono, and S. Sagayama, “Convergence-guaranteedmultiplicative algorithms for nonnegative matrix factorization with β-divergence,” Proc. IEEE MLSP, pp. 283–288, 2010.
[10] C. Févotte and J. Idier, “Algorithms for nonnegative matrix factorization with the β-divergence,” Neural Computation, 2011.
[11] H. L. Van Trees, Optinum Array Processing, John Wiley& Sons, 2002.
[12] S. Araki, H. Sawada, and S. Makino, “Blind speech
separation in a meeting situation with maximum SNR
beamformers,” Proc. ICASSP, vol. I, pp. 41–45, 2007.
[13] E. Vincent, R. Gribonval, and C. Févotte, “Performancemeasurement in blind audio source separation,” IEEE
Trans. on Audio, Speech & Language Processing, vol.
14, no. 4, pp. 1462–1469, 2006.

154